#clojure-spec

generated UTC: 2023-11-22 19:43
latest data: https://clojurians-log.clojureverse.org/clojure-spec/2023-11-16
messages: 41621
pro tips:
* Double click on text to filter by it. (doubleclick + cmd-f for extra points).
* Click on date to keep day visible regardless of filter.
* Click on time to keep hour visible regardless of filter.
#2016-05-2512:06sveriIs there something in spec that is comparable to string? but for a boolean value?#2016-05-2512:06sveriexcept #(instance? Boolean %)#2016-05-2512:10richhickey@sveri: not yet. Obviously there are several more useful predicates that could go into core. Considering which ones now#2016-05-2512:10sveri@richhickey: Yea, it would be nice to be consistent, much less work for the brain then šŸ™‚#2016-05-2512:19moxaj
(spec/def ::foo (spec/cat :kw keyword? :int integer?))
(spec/def ::bar (spec/coll-of ::foo []))
(spec/explain ::bar [[:a 10] [:b 20] [:c "30"]])
;; => val: [[:a 10] [:b 20] [:c "30"]] fails predicate: (coll-checker :spec/foo)
Why doesn't the error reporting go into detail (like: val "30" fails predicate: integer?) Is this intentional?
#2016-05-2512:44sveriHm, is it ok to use ::spec/name for my own specs? Also I have not found the source for ::spec/string, neither by searching github (which is a bit harder, because it ignores "::", nor by looking at the api.#2016-05-2512:44sveriAlso, is it correct that ::spec/name has string? as predicate?#2016-05-2512:59kingoftheknoll@alexmiller: About a week ago you hinted on Reddit at improved error messages for Clojure. Donā€™t think anyoneā€™s said that here but it seems like we just got a peek into how that might be accomplished if all of Clojure.core implements specs.#2016-05-2514:08mjhamrickWhat's the best way to have fdefs be checked during lein test? Instrument all works at the repl, but seems to not during test. #2016-05-2514:10anmonteiro@mjhamrick: what Iā€™ve done is (clojure.spec.test/check-var #ā€™fn-speced-with-fdef)#2016-05-2514:14mjhamrick@anmonteiro: are you just putting that in each test? #2016-05-2514:15anmonteiroso far, yes#2016-05-2514:15anmonteirothe project where I tried it was pretty small#2016-05-2514:24sveri@mjhamrick: I did put (s/instrument-all) at the top of each test namespace#2016-05-2514:35Alex Miller (Clojure team)@kingoftheknoll: yes, I have done additional work in this area#2016-05-2514:44Alex Miller (Clojure team)@sveri ::spec/name and ::spec/string don't exist - what is that from?#2016-05-2514:46sveri@alexmiller: Thats why I have not found it in the code. Not sure where exactly it came from, its just that cursive offered it via code completion. Maybe I typed it in a different namespace or whatever. Anyway, good to know.#2016-05-2514:52michaeldrogalis@richhickey @alexmiller: I have another proposed enhancement to explain-data. Several predicates, such as keyword?, or integer? are programmatically easily recognizable via :pred for non-conforming values. It's a straightforward equality check (e.g. (= 'keyword? (:pred error-map))). Other predicates are, to my understanding, more difficult to recognize. Take s/keys and s/tuple for example. In the former, keys yields a predicate when a required key is missing. While the predicate is somewhat understandable when you eye ball it, picking it out programmatically has been error-prone. The same goes for s/tuple when the number of arguments is incorrect.#2016-05-2514:53Alex Miller (Clojure team)an example would help make this specific#2016-05-2514:53michaeldrogalisThose functions have privileged access to which predicates are failing. It might be helpful to provide named predicates in those cases so that consumers of explain-data could be more intelligent. See - https://gist.github.com/MichaelDrogalis/016ac62cf3f1e899fb89fd79f2de2277#2016-05-2514:54Alex Miller (Clojure team)thx for the example! :)#2016-05-2514:54Alex Miller (Clojure team)oh the contains stuff that's inside keys#2016-05-2514:55Alex Miller (Clojure team)I'll have to defer that to Rich, I think he's not around atm#2016-05-2514:55moxajIt seems to me that inner conformed values are lost when using coll-of or map-of. Example:
(spec/conform (spec/coll-of (spec/and integer?
                                      (spec/conformer (constantly nil)))
                            [])
              [1 2 3])
returns [1 2 3] instead of [nil nil nil]. Meanwhile, cat and keys preserves them. How come?
#2016-05-2514:56michaeldrogalis@alexmiller: Sure šŸ™‚ Ill be back later.#2016-05-2514:57Alex Miller (Clojure team)@moxaj Rich said above "currently conform doesn't flow into coll-of, so the value is never conformed, only checked" which I think is likely related to this#2016-05-2514:57moxaj@alexmiller alright, thanks!#2016-05-2515:41sveriI am converting a lib from schema to spec and in schema I was able to inline define the return values / args. Is there a way to do that in spec too? The alternative in this case is a lot more code.#2016-05-2515:44Alex Miller (Clojure team)for map keys, no - that's the whole point of the map / attributes split#2016-05-2515:44Alex Miller (Clojure team)the benefit is that you are creating named reusable semantics for ::entityname, ::ns, etc etc#2016-05-2515:44Alex Miller (Clojure team)so you can use those elsewhere too#2016-05-2515:45sveri@alexmiller: Yea, I understand the benefits, just wanted to make sure I did not miss anything#2016-05-2515:48mpenetIs it possible to hook ones own error message if a predicate fails? Ex we have schemas that takes a string that should be a valid parse of our internal query language, and if fails should return a nicely structured error with line/col num and a human readable message. With prismatic Schema it is possible to do with our own schema type extending a protocol for explain/spec.#2016-05-2515:51mpenetCorrect me if I am wrong, but I think so far I can only get a generic "predicate foo? failed" type of message#2016-05-2515:55Alex Miller (Clojure team)there is not currently a way to provide a custom error for a failing predicate#2016-05-2515:56Alex Miller (Clojure team)you can use explain-data to detect problems and produce a custom error message#2016-05-2515:56Alex Miller (Clojure team)but @richhickey can comment on whether that might be something we could do#2016-05-2515:57mpenetthat's what I suspected. Something like ex-info for specs could be useful#2016-05-2516:05Alex Miller (Clojure team)why not explain-data for that?#2016-05-2516:05Alex Miller (Clojure team)or maybe I misunderstand the suggestion#2016-05-2516:05mpenetI actually missed it, that could work yes#2016-05-2516:06Alex Miller (Clojure team)that's what instrument does on a failing function spec#2016-05-2516:15mpenethmm I am not sure that's really equivalent. I would have to wrap all schemas that contains these values with a special validation fn I think. I would like to have my own datastructure returned by explain-data (or something else) when the predicate fails. I am not sure that's doable with the current approach where predicates return just a potentially truthy value.#2016-05-2516:16mpenetbut I could be wrong. I played for 20min with specs so far.#2016-05-2516:24mpenetthis might sound like a small concern, but in a larger context it's very useful: we have a whole api around the manipulation of data containing such queries, being able to use the same schema type and framework and not wrap the whole thing because of a single type in a potentially nested Schema is a nice feature, in our case it also integrates for free with frameworks that just follow this Schema format (rest server, swagger etc).#2016-05-2517:38eraserhdFirst piece of feedback is that clojure.spec makes me really want ā€œboolean?"#2016-05-2517:49sveri@eraserhd: @richhickey said they consider adding this and some more core types (I felt the need for it too :-))#2016-05-2517:52eraserhd@sveri yay šŸ™‚#2016-05-2517:52eraserhdIā€™m still playing with it, but the second thing Iā€™m working through is the repetition, especially for function arguments.#2016-05-2518:16Alex Miller (Clojure team)@eraserhd: we have boolean? in almost every example namespace we wrote. so, agreed. :) definitely near the top of the wanted-predicates list.#2016-05-2518:16eraserhd@alexmiller: cool šŸ™‚#2016-05-2518:18eraserhdAnother interesting problem I encountered: I used a set for boolean: #{true false}. But sets arenā€™t treated differently in this context, although I might expect that they are. Namely #(#{true false} false) ;=> false. So having a set in a spec which contains nil or false is awkward.#2016-05-2518:19Alex Miller (Clojure team)yeah, don't do that :)#2016-05-2518:19Alex Miller (Clojure team)it is kind of an interesting wrinkle though that the obvious way to write boolean? is with #(instance? Boolean %)#2016-05-2518:20eraserhdIt might at least deserve a call-out in docs.#2016-05-2518:21eraserhdAlso, a lot of things seem to leak their internal implementations wrt error messages.#2016-05-2518:23Alex Miller (Clojure team)but Clojure uses (only) canonical boolean values#2016-05-2518:23Alex Miller (Clojure team)@eraserhd: prob better for a mention in the ref doc (which isn't there yet)#2016-05-2518:24Alex Miller (Clojure team)@eraserhd: example re errors?#2016-05-2518:24eraserhd@alexmiller: I encountered a couple. Just a minute and Iā€™ll paste.#2016-05-2518:55sveriAnyone else seeing problems with reloading test namespaces in the REPL (cursive)? I just commented out some tests, reloaded the test ns in the REPL, hit run-tests, but still, all tests were run. I noticed something similar with test-refresh not recognizing commented fdefs. I cannot point at anything right now, but, there seems to be something broken#2016-05-2519:01kovasb@cfleming: are there gonna be slick Cursive toolips for providing associated specs when editing functions arguments?#2016-05-2519:02kovasbI imagine I'm editing the arg to some function that takes a big map, and as I'm filling in the components of the map its telling me what each piece should conform to#2016-05-2519:09arohnerWhat is the spec for ā€˜map? i.e. in core.typed, Iā€™d write something like (t/ann map [ [ X -> Y] [X] -> [Y] ). i.e. it takes a collection of X, and a function of X->Y, and returns a collection of Y. Is there a way to do that in spec?#2016-05-2519:25bbrinckIā€™m missing something related to recursive specs. I want to define a vaguely hiccup-like structure where the first element is the name, followed by some attributes, followed by a vector of children. What am I doing wrong here? https://gist.github.com/bhb/6cfcb3b38757442aec4ba5db46148699#2016-05-2519:38hiredmanif you add another (s/spec ....) around the ::tag I get a success, but I am not sure why#2016-05-2519:42hiredmanI guess you are in the regex context that matches the outer vector, so you need another s/spec to create a new regex context for the list of child vectors, then you need another s/spec to get out of the regex context of the vector of children and specify an individual child#2016-05-2519:48anmonteiroIā€™d also be interested to see the spec for clojure.core/map#2016-05-2519:48bbrinck@hiredman: Ah, thanks! I had to read that a few times, but now it makes sense šŸ™‚#2016-05-2519:56sveriOk, I dont understand how to use instrument. My guess was, that if I put (s/instrument-all) into a namespace all functions are instrumented whenever I load the file into the REPL. But thats not the case, instead I have to execute that function everytime after I loaded a namespace.#2016-05-2519:56sveriAlso, it works when I put it to the end of a ns#2016-05-2520:02dryewoHi all, first I want to thank Rich&Co for this awesome innovation :+1: And I have a question:
(s/def ::even-big-or-small (s/and (s/or :very-big #(> % 1000)
                                        :very-small #(< % 1))
                                  even?))
(s/valid? ::even-big-or-small 10000)
;; throws CompilerException java.lang.IllegalArgumentException: Argument must be an integer: [:very-big 10000]
what am I doing wrong?
#2016-05-2520:07dryewojust noticed, works fine the other way around:
(s/def ::even-big-or-small (s/and even?
                                  (s/or :very-big #(> % 1000)
                                        :very-small #(< % 1))))
#2016-05-2520:12anmonteiro@dryewo: in the first case, itā€™ll conform to :very-big and output [:very-big 1000]#2016-05-2520:13anmonteiro
(s/def ::even-big-or-small (s/and (s/or :very-big #(> % 1000)
                                               :very-small #(< % 1))
                                        #(even? (second %))))

#2016-05-2520:13anmonteirothis works#2016-05-2520:13anmonteiroeven? gets [:very-big 1000] so we are interested in the second element#2016-05-2520:13anmonteirothatā€™s the explanation, but Iā€™m also unsure if itā€™s expected#2016-05-2520:14dryewoyes, I also got this idea of whatā€™s happening, but is it the correct behavior?#2016-05-2520:15dryewoshouldnā€™t s/and be order agnostic?#2016-05-2520:16anmonteiroIā€™ve read somewhere that itā€™s not, by design#2016-05-2520:16anmonteirobut I canā€™t recall the reason#2016-05-2520:18dryewoIā€™m reading the guide: http://clojure.org/guides/spec maybe I should first read it to the end before asking more questions šŸ™‚#2016-05-2520:25kenny@dryewo: Look into https://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec/conformer#2016-05-2520:26dryewothanks, there is even a section about it in the end of the guide, Iā€™ll look into it#2016-05-2520:35kennyIs there a pattern for whether you spec functions or use spec for validation (pre/post-conditions)? There doesn't seem to be a reason for pre/post-conditions if you spec a function.#2016-05-2520:39sveriI dont understand that, I have the code in the following snippet, and it fails with:
(remove-autoinc-columns [{:foo "some_t"}])
ExceptionInfo Call to #'de.sveri.clospcrud.s-play/remove-autoinc-columns did not conform to spec:
At: [:args] val: ([{:foo "some_t"}]) fails predicate: (cat :cols :de.sveri.clospcrud.s-play/columns),  Extra input
:clojure.spec/args  ([{:foo "some_t"}])
  clojure.core/ex-info (core.clj:4617)
#2016-05-2521:10anmonteiro@sveri: you need to wrap ::column in s/spec#2016-05-2521:10anmonteirolike this: (s/def ::columns (s/cat :col (s/* (s/spec ::column))))#2016-05-2522:34Alex Miller (Clojure team)@arohner: @anmonteiro here's a spec for map (assumes a seqable? predicate like the one in core.incubator)#2016-05-2522:34Alex Miller (Clojure team)
(s/fdef clojure.core/map
  :args (s/cat :f ifn?
               :colls (s/* seqable?))
  :ret (s/or :seq seqable? :transducer ifn?))
#2016-05-2522:35anmonteirosimpler than I imagined#2016-05-2522:35Alex Miller (Clojure team)I've left off :fn because I'm lazy but you could also verify some additional things#2016-05-2522:36Alex Miller (Clojure team)like :f / :transducer or :colls / :seq are the valid combos and even things like whether the :ret cardinality is minimum of the input colls cardinality#2016-05-2522:37Alex Miller (Clojure team)but that's enough to detect things like (map inc 100)#2016-05-2522:37Alex Miller (Clojure team)which is the kind of thing that normally leads to IllegalArgumentException Don't know how to create ISeq from: java.lang.Long#2016-05-2522:38Alex Miller (Clojure team)which, if it's occurring in a nested context can be pretty confusing#2016-05-2522:40tylerAnyone given the test namespace a try yet? Iā€™m trying to figure out how check-var works. It appears to be hanging half the time I try it and passing tests the other half.#2016-05-2522:45arohner@alexmiller: is it possible to check that fā€™s input matches the coll?#2016-05-2522:45arohnerright now I donā€™t see a way to do that#2016-05-2522:46arohnerbecause there doesntā€™ seem to be a way to ā€˜compareā€™ specs#2016-05-2522:46Alex Miller (Clojure team)you would have to use a custom predicate to do so#2016-05-2522:47Alex Miller (Clojure team)the :args spec could be (s/and <current> #(custom predicate on the args))#2016-05-2522:48Alex Miller (Clojure team)I guess you might need to describe :f with an fspec to get the right parts to work with#2016-05-2522:48Alex Miller (Clojure team)I haven't done it!#2016-05-2522:57tylerIā€™ve created a gist with my attempt to try and use check-var it is returning nil, is that expected behavior? https://gist.github.com/tvanhens/28b7f744d799910750d8ae3942680997#2016-05-2523:11hiredmando you have the test.check library on the classpath?#2016-05-2523:34tylerYup gen/generate works#2016-05-2523:46stuarthalloway@tyler: there is a bug there, I hit it too#2016-05-2523:47stuarthallowayI think it is fixed on master#2016-05-2523:52tylerAh awesome thanks#2016-05-2523:53richhickeyyah#2016-05-2523:57eraserhd@alexmiller An example wrt errors: (spec/explain (spec/tuple number?) []) gives an error about #(clojure.core/= (clojure.core/count %) 2).#2016-05-2523:57eraserhds/2/1/#2016-05-2523:59richhickey@eraserhd: "val: [] fails spec: _ predicate: (clojure.core/= (clojure.core/count %) 1)ā€ says the collection should have count == 1#2016-05-2600:00richhickeytuple is not a parameterized coll (see coll-of for that), (tuple number?) says vector of exactly one number#2016-05-2600:01richhickeytuple is for short tuples of heterogeneous types each of which will be listed#2016-05-2600:01eraserhd@richhickey Oh hi! My comment is about the message, not that it should passā€¦#2016-05-2600:01richhickeye.g. (tuple string? number?)#2016-05-2600:01eraserhdI was saying that it reveals implementation detail of spec/tuple.#2016-05-2600:02richhickeyvs what?#2016-05-2600:04eraserhdI was reading the rationale document as saying that we want to capture the predicates the user supplied, so I was hoping for something like: val: [] fails spec: (tuple number?).#2016-05-2600:04richhickeythat doesnā€™t tell you much#2016-05-2600:04eraserhdHrmm, this is true.#2016-05-2600:05richhickeyand the interior preds can be arbitrarily complex, will walk down#2016-05-2600:06richhickeymuch better than ā€œyou failed the top"#2016-05-2600:07eraserhdHrmm. So my (clearly not thought through) expectation would be that count would fail at tuple, but sub-types would fail more specifically. Thatā€™s not an idea I can make coherent, though.#2016-05-2600:10eraserhd(Anyhoo, super excited about clojure.specā€™s new ideas, cf prismatic)#2016-05-2600:13richhickeyhave fun!#2016-05-2600:38shaunxcodeis it possible create a spec for a map like {[300 200 54] :status/open} e.g. map keyed by vector of integers#2016-05-2600:42tyler@shaunxcode yes I believe so using map-of#2016-05-2600:44tyler
(s/map-of (s/coll-of integer? []) #{:status/open})
#2016-05-2601:47cfleming@kovasb: Sure, sounds doable#2016-05-2606:41borkdudeShould I include test.check as an explicit dependency when using spec? I got: FileNotFoundException Could not locate clojure/test/check/generators__init.class or clojure/test/check/generators.clj on classpath. clojure.lang.RT.load (RT.java:456)#2016-05-2606:42borkdudewhen instrumenting the adder example from the guide https://clojure.org/guides/spec (cc @alexmiller )#2016-05-2607:15sveri@borkdude: Yes, you need it for most cases if you want to use spec in your tests. There was only one function that works without test.check (I think it was spec/gen, but not sure). I have read it somewhere in the official docs in the last days.#2016-05-2607:18sveriAnyone else got an idea why this snippet fails?#2016-05-2607:19sveri@anmonteiro: wrapping it in (s/spec... doesnt fix it unfortunately#2016-05-2608:50sveriAlso, what works is:
(s/conform ::columns [{:foo "some_t"}])
=> {:col [{:foo "some_t"}]}
which is strange, cause all I do next is to put it into the argument definition of my function
#2016-05-2609:07sveriCan someone reproduce this?#2016-05-2609:52dominicmIs there space in spec, for a specification to take parameters via lexical scope? This may be out of scope for spec. For example, on http request, I take a snapshot of my database, and I want to check if a value is in that database. But I want to have my predicates registered via s/def so that I can take advantage of :via and such. Use case is checking if an email is taken during user registration.#2016-05-2612:10anmonteiroLibrary authors might want to add specs to their existing projects but also keep supporting earlier versions of Clojure#2016-05-2612:10anmonteiroIs this a use case that has been thought about, or isnā€™t there any interest in supporting such use case?#2016-05-2612:12anmonteiroor are we just supposed to solve it through versioning?#2016-05-2612:32Alex Miller (Clojure team)Specs could be provided in an adjunct library too#2016-05-2612:34Alex Miller (Clojure team)Ultimately, things must move forward to get the benefits of new features as we did with records, protocols, transducers, reader conditionals, etc etc#2016-05-2612:34anmonteiroAgreed wrt. 2nd point#2016-05-2612:35anmonteiroand yea, the optional library add-on could also be a possibility for initial compatibility, hadnā€™t thought about that#2016-05-2612:35anmonteirothanks#2016-05-2612:35bronsa@anmonteiro: having spec in both clojure.core + an external library could cause namespace conflicts#2016-05-2612:36Alex Miller (Clojure team)They can also be in a separate namespace in your library jar that's just not loaded (unless you're on 1.9)#2016-05-2612:36anmonteiro@bronsa: I understand, but I didnā€™t mean that.#2016-05-2614:10Alex Miller (Clojure team)1.9.0-alpha3 is now out with several bug fixes and improvements for spec https://groups.google.com/forum/#!topic/clojure/WxT9kPIwlYI#2016-05-2614:11Alex Miller (Clojure team)The guide is now out of date but I will have it updated shortly#2016-05-2614:43dominicm@alexmiller: Is there much of a story for what I mentioned previously? https://clojurians.slack.com/archives/clojure_spec/p1464256334000558 Trying to decide how to approach the problem of generating/validating forms.#2016-05-2615:07Alex Miller (Clojure team)@dominicm: you could register (at runtime) an s/def that partialed/closed-over something known at runtime like a database snapshot. This implies that you are using spec to do runtime data validation. Whether or not this is a good idea, I'm not sure. :)#2016-05-2615:14dominicm@alexmiller: Yeah, I did wonder if dynamic vars might be a solution, and a bad idea. I'm definitely thinking in the domain of variable user-input, as opposed to contracts during development. I like the idea of spec as a ubiquitous format that the clojure community uses to describe data. Some data just can't be controlled (user input). Maybe this is way out of scope for spec though, and you guys think that should be a different library altogether. There's definitely a lot of overlap in that space (email still needs to be a string, over 5 chars, match a regex, before it is db checked). Nothing says a validation library can't register things for you of course. But I try to keep core at the top, not other libraries.#2016-05-2616:01gtrakThe registry itself is pretty simple, I think a 3rd-party lib could sub out the behavior you want but use registered specs along the way.#2016-05-2616:03dominicm@gtrak: Yeah, that's what I was thinking about. But if it is in scope for spec, that would be pretty cool.#2016-05-2616:06gtrakmy understanding so far is that spec itself is mostly concerned with the lib-sharing/transmission/self-describing-edn use-cases, but there are a lot of developer-convenience type cases we'd also want to use it for (overlap with Schema), but it can't integrate all of them.#2016-05-2616:15gtrakFor instance, the first thing I wrote was a multimethod type->spec coercer, which is a special case extension of the built 'conform', I wouldn't expect that to get into clojure.spec, but I might expect it to be available as a lib, since it's so easy to write.#2016-05-2616:16gtrakconformer-helpers..#2016-05-2616:19gtrakhttps://gist.github.com/gtrak/9b6d16d0423b284292dbbdd095bf91e9#2016-05-2616:21gtrakusage: (s/def ::my-key-that-only-matters-to-my-app (coercer integer?))#2016-05-2616:25gtrakthe one thing I'd request from clojure.spec itself was a way to provide a better error message than what s/conformer creates, which in this case is too generic, a failure in the pred: (fn [from] (coerce from to)). It's possible I can do so by dropping down a level in abstraction but haven't gotten that far.#2016-05-2616:50dominicm@gtrak: oh I agree! I had wondered if that would be possible! Shame you beat me to it. But I feel like being unable to lexically scope a spec in some way hinders me a little. Maybe I'm looking at it wrong, or maybe I can come up with a clever roundabout way to do it and lib it up.#2016-05-2616:55gtrakI think I overheard that specs are serializable somewhere, so lexical scope could really mess that up.#2016-05-2617:00dominicm I probably shouldn't refer to scope. What I want is a solution to the problem of runtime values effecting the running of my predicates, without relying on state. Am I making sense?#2016-05-2618:19gtrakI don't see how it's possible to do that without state, maybe a dynamic/threadlocal registry, but then the state's in the registry.#2016-05-2618:19gtrakor you create a one-off spec every time#2016-05-2618:20gtrakor every consumer of the registry has to let you tunnel things into specs#2016-05-2620:52sveri@alexmiller: I just found a possible bug in the following snippet:#2016-05-2620:54sveriRunning lein test for this code will run into a never ending loop when it tries to test leiningen.s-play-test#2016-05-2620:54sveriI would expect an error message here#2016-05-2620:55sveriThe cause is this line (s/fdef foo :args (s/cat)) with the incomplete (s/cat) expression.#2016-05-2621:19Alex Miller (Clojure team)Feel free to file a jira for it#2016-05-2706:52dominicmhttps://clojurians.slack.com/archives/clojure_spec/p1464286849000602 I wouldn't say that's a terrible idea#2016-05-2708:22dominicmhttps://clojurians.slack.com/archives/clojure_spec/p1464278761000588 I was under this impression also. But I just found this message on google groups: https://groups.google.com/d/msg/clojure/5sZdCPQgZz4/7rpuiaj1AgAJ > OTOH, you may encounter user- or externally-supplied data at runtime and want to use the facilities of spec to validate/process it. Then you can use valid? or conform explicitly to do so.#2016-05-2719:56Alex Miller (Clojure team)I have added a new section to the spec guide about generators http://clojure.org/guides/spec#_generators#2016-05-2719:57Alex Miller (Clojure team)I still expect there to be a bit more to come on that page about generators and testing#2016-05-2719:57Alex Miller (Clojure team)but seemed better to get something out there sooner#2016-05-2719:58michaeldrogalis@alexmiller Was there any discussion with respect to predicate failures being recognizable from keys, tuple, etc? I can dig up the Gist link if you need for reference#2016-05-2719:59Alex Miller (Clojure team)there have been so many things coming in I don't honestly remember if I talked about it with Rich - I have it in my "things to talk about" file#2016-05-2720:02sveri@alexmiller: done so: http://dev.clojure.org/jira/browse/CLJ-1934#2016-05-2720:04sashtonis this a good place to mention spec guide (potential) typos?#2016-05-2720:08michaeldrogalis@alexmiller: No prob, thanks!#2016-05-2720:10Alex Miller (Clojure team)@sashton: sure!#2016-05-2720:10Alex Miller (Clojure team)@sveri thx!#2016-05-2720:12sashton@alexmiller: Under Collections, it has the following: * Collection - (s/coll-of float?) I think coll-of needs a []#2016-05-2720:12Alex Miller (Clojure team)yep#2016-05-2720:12Alex Miller (Clojure team)thx#2016-05-2720:13Alex Miller (Clojure team)I'll fix this one but for future reference, you can send issues/PRs for the site too http://clojure.org/community/contributing_site#2016-05-2720:41Alex Miller (Clojure team)@sveri I have a patch to fix your bug, should go in the next alpha#2016-05-2721:12sveri@alexmiller: Great, looking forward to testing it#2016-05-2721:13Alex Miller (Clojure team)patch is on the ticket if you want to build your own version of clojure#2016-05-2721:13Alex Miller (Clojure team)I added a test to the tests for it too#2016-05-2721:14sveriYea, I just saw it, I have not setup the clojure build chain, so I just wait for it, no urgency on my side. Thanks for the quick fix#2016-05-2801:27cfleming@alexmiller: You mentioned you have a spec for spec. Any interest in showing that? It would help with adding support in Cursive.#2016-05-2801:30Alex Miller (Clojure team)It's almost certainly pretty buggy so it's not something I really want to publish yet#2016-05-2801:30cflemingOk.#2016-05-2801:31Alex Miller (Clojure team)I have turned on instrumentation with it and had it actually find bugs while I was working though :)#2016-05-2801:32cflemingYeah, Iā€™m planning to merge in the validation code I demoed at the conj, having real-time validation in Cursive for specs would be nice.#2016-05-2801:32Alex Miller (Clojure team)I can send it to you privately if you want#2016-05-2801:33cflemingAnd using specs to help with completion is probably nice too, although I need to work with it more to see whatā€™s possible.#2016-05-2801:33cflemingOk, thanks. Iā€™m not in a big rush, I wonā€™t get to it for a week or two probably.#2016-05-2801:33cflemingBut Iā€™d be interested to see it.#2016-05-2801:35sekaoanyone have an example of using fdef with variadic args?#2016-05-2802:04sekaoahh i guess itā€™s something like (s/cat :args (s/* string?))#2016-05-2811:42Alex Miller (Clojure team)@sekao: yep although you don't even need the s/cat there just '(s/* string?)' would be sufficient#2016-05-2811:53francescohow could I write a spec for, say, clojure.core/conj that enforces that in (conj coll x) x satisfies the same spec as the elements of coll? Is this kind of reasoning within the intended scope of clojure.spec?#2016-05-2811:59Alex Miller (Clojure team)That's not a true constraint of conj :)#2016-05-2811:59francescototally agree on that.#2016-05-2812:00Alex Miller (Clojure team)But you could spec something like that with the :fn spec in fdef which relates the args and ret#2016-05-2812:01Alex Miller (Clojure team)It's passed a map with the conformed value of both#2016-05-2812:49sveriA bit more feedback. I constantly run into errors like this:
In: [0] val: ([{:name "age", :type :int, :null false}]) fails at: [:args] predicate: (cat :cols :de.sveri.clospcrud.schema/columns),  Extra input
:clojure.spec/args  ([{:name "age", :type :int, :null false}])
  clojure.core/ex-info (core.clj:4617)
Most of the times the problem was that I was missing a (s/spec ::columns) expression around the spec definition. Still, everytime I see this error it is so far away from what the problem actually is, that I need time to remember this. Worse is, that at first I look and search if I did something wrong while transforming my schema to spec. Maybe a somewhat better error message is possible?
#2016-05-2812:49sveri@alexmiller:#2016-05-2814:35Alex Miller (Clojure team)Not sure what a generically better error would be#2016-05-2816:50Alex Miller (Clojure team)Interesting - file a jira if you want so we don't lose it over the weekend#2016-05-2817:26sveri@alexmiller: For me it would be something like: "Maybe you forgot a spec around a regex? Try (s/spec ...)" But, I dont know how generic that error is and if it would fit everytime?#2016-05-2817:54moxaj@alexmiller done, see http://dev.clojure.org/jira/browse/CLJ-1935#2016-05-2818:16sekaois there a way to exclude certain vars from being tested by run-all-tests? i have some side-effecting functions that will cause some havoc if run in generative tests šŸ˜ƒ#2016-05-2818:39arohnerIā€™m not entirely clear on when generative testing happens. In some cases (side-effects), Iā€™d like the ability to check conform but not do generative testing#2016-05-2818:54sekaofor now i copied the code into my project and added an if statement that checks for :no-check in the varā€™s metadata#2016-05-2819:33bbrinck@arohner: My understanding is that generative testing only happens if you invoke run-all-tests. If you just wanted to test specific functions in your test suite, you could do something like:
(deftest test-my-function
  (is (= true (:result (t/check-var #ā€™my-function)))))
#2016-05-2819:34arohner@bbrinck: Iā€™m pretty sure there are cases aside from that that can trigger it#2016-05-2819:34arohnerIIRC, richā€™s example from the other day will#2016-05-2819:34arohnerone sec#2016-05-2819:35arohnerindeed:#2016-05-2819:35arohner
repl> (defn foo [fnn] (fnn 42))
#'repl/foo
repl> (s/fdef foo :args (s/cat :f (s/fspec :args (s/cat :i integer?)
                                     :ret integer?)))
repl/foo
repl> (s/instrument 'foo)
#'repl/foo
repl> (foo #(when (even? %) 42))  

ExceptionInfo Call to #'repl/foo did not conform to spec:
In: [0] val: nil fails at: [:args :f :ret] predicate: integer?
:clojure.spec/args  (#object[repl$eval66335$fn__66336 0x55f2e735 "
#2016-05-2819:37arohner@bbrinck: ^^#2016-05-2819:38arohnerthat ran generative testing on a normal call#2016-05-2819:38arohnerwhich is something Iā€™d like control over#2016-05-2819:38bbrinckI may be misunderstanding, but I don't think it did. I think it just checked that the spec matched the invocation: (foo #(when (even? %) 42)) #2016-05-2819:39arohner42 should pass there. The generative example passed zero#2016-05-2819:40arohnerhrm nvm#2016-05-2819:40bbrinckI think this is just a confusing error message. The value passed is not 0#2016-05-2819:40bbrinckit's #(when (even? %) 42), which returns nil, which violates the spec#2016-05-2819:41arohner
(foo #(do (println %) (when (even? %) 42)))
-1
0
-1
0
-1
0
-1
#2016-05-2819:42arohner
(s/unstrument 'foo)
#'repl/foo
repl> (foo #(do (println %) (when (even? %) 42)))
42
42
#2016-05-2819:45bbrinckHm, yeah, you're right, I was reading that wrong. I still don't think (from playing with other examples) that the generative testing occurs unless you do one of the testing calls. I think this example might just have a bug. I could be misunderstanding though.#2016-05-2819:46bbrinck(I'm trying to try it myself, but on my version of clojure, the fdef is not working for some reason)#2016-05-2819:47arohnerwhich testing call did I run there?#2016-05-2819:53bbrinckHm, you're absolutely right. I did not expect that.#2016-05-2819:55bbrinck@arohner: Yep, you're correct - there was no testing call, but it's doing some generation for the args for the inner function.#2016-05-2819:59bbrinck@arohner: interestingly, that does not seem to occur in all cases:
(s/fdef double
        :args (s/cat :x integer?)
        :ret integer?)
(defn double [x]
  (if (zero? x)
    "zero"
    (* 2 x)))

(s/instrument 'double)

(double 1)
#2016-05-2819:59arohneryeah, I think fspec is the culprit#2016-05-2820:01bbrinckah, yes, the docs do mention that now that I read them a little more carefully šŸ˜„ . Well, TIL#2016-05-2820:05bbrinckAlso, FWIW, that example won't quite work on 1.9.0-alpha3. fspec seems to require a ret now#2016-05-2820:08bbrincknvm, i realized it was a copy/paste fail on my part (didn't grab the full fspec above)#2016-05-2820:18sveriHi, did anyone try to generate a boolean spec? Like this: (s/def ::required #(instance? Boolean %)). Calling (gen/sample (s/gen ::required)) fails for me with: Exception Unable to construct gen.... This becomes problematic if ::required is part of a nested spec, how would I provide a generator for that?#2016-05-2820:28seancorfieldThat specific case is covered in the (updated) documentation @sveri #2016-05-2820:30sveri@seancorfield: Ah, thats nice, I just solved it by defining boolean as true / false: (s/def ::boolean #{true false}) for which the generator works too#2016-05-2820:31seancorfield"There are some common cases that currently donā€™t have standard predicates, but good generators exist. Itā€™s likely there will be changes in this area in the future but for now you might find these useful: (defn boolean? [x] (instance? Boolean x)) (s/def ::boolean (s/with-gen boolean? #(gen/boolean))) (defn uuid? [x] (instance? java.util.UUID x)) (s/def ::uuid (s/with-gen uuid? #(gen/uuid)))"#2016-05-2820:31seancorfield(That didn't paste well -- sorry, on my phone)#2016-05-2820:35sveriYea, no problem, I get it, thank you šŸ™‚#2016-05-2821:00borkdudeJust finished reading the guide. Good stuff.#2016-05-2821:12borkdudeWhy is (doc subs) printing Spec on the last line. Bug?
user=> (doc subs)
-------------------------
clojure.core/subs
([s start] [s start end])
  Returns the substring of s beginning at start inclusive, and ending
  at end (defaults to length of string), exclusive.
Spec
nil
#2016-05-2821:33borkdudeWhat about multi-arity functions like subs, how to write spec for both cases?#2016-05-2821:37borkdudeLike this? (s/fdef clojure.core/subs :args (s/or :two-args (s/cat :s string? :start integer?) :three-args (s/cat :s string? :start integer? :end integer?))#2016-05-2821:41borkdudeI'm not sure if I should wrap with s/spec, both seem to work: (s/fdef clojure.core/subs :args (s/or :two-args (s/spec (s/cat :s string? :start integer?)) :three-args (s/spec (s/cat :s string? :start integer? :end integer?))))#2016-05-2901:11Alex Miller (Clojure team)@borkdude: there is new code in doc to find and print specs, but it shouldn't print that if there are none so that's a bug#2016-05-2901:11Alex Miller (Clojure team)@borkdude: for multi-arity, you can cover all options in a single spec via regex#2016-05-2901:14Alex Miller (Clojure team)you can just do (s/fdef clojure.core/subs :args (s/cat :s string? :start integer? :end (s/? integer?)) :ret string?)#2016-05-2901:18Alex Miller (Clojure team)you can even add a :fn #(clojure.string/includes? (-> % :args :s) (:ret %)) to verify it returns a string included in the original#2016-05-2901:19Alex Miller (Clojure team)I didn't test any of that but it should be pretty close#2016-05-2901:31sekaocould we get a way to exclude vars from run-all-tests, like ^:no-check in core.typed? i jerry-rigged a solution earlier today but it required modifying the function directly. i have lots of side-effecting functions that shouldnt be running generative in tests.#2016-05-2901:35Alex Miller (Clojure team)it's best to file a jira for anything so we don't lose it, esp over the long weekend#2016-05-2901:36sekaook will do#2016-05-2901:36Alex Miller (Clojure team)thx!#2016-05-2901:36Alex Miller (Clojure team)that way we can track that stuff and you can see what's happening with it too#2016-05-2901:46sekaoah i shouldā€™ve looked there first, iā€™ll just comment on CLJ-1936 as it sounds similar#2016-05-2907:33borkdude@alexmiller: thanks šŸ™‚#2016-05-2907:44borkdude@alexmiller: promoted this example to a gist: https://gist.github.com/borkdude/0665078edb40fb0e1551c1d29655c2d6#2016-05-2921:00sveriHey, I have again a problem I dont understand:#2016-05-2921:02sveriWhen I copy the part from args: ({:name "QpJ50qrS1m24V", :type :varchar, :unique true, :required false, :autoinc true, :null true, :pk true, :max-length 14, :fk-name "Tp9tG8hwUXK0"})and run the spec validation on it it succeeds. What bothers me, is that the last line in the message wraps the columns in an extra list: :clojure.spec/args (({:name "QpJ50qrS1m24V" ... I am not sure what that means.#2016-05-3008:29cflemingWhen s/spec is used to create a new sequential context, I canā€™t find a way to capture the sequential thing itself.#2016-05-3008:29cflemingFor example:
(s/def ::ingredient (s/cat :quantity number? :unit keyword?))
(s/def ::recipe (s/cat :amount (s/spec ::ingredient) :description string?))
(s/conform ::recipe [[2.0 :teaspoon] "Cinnamon"])
=> {:amount {:quantity 2.0, :unit :teaspoon}, :description "Cinnamon"}
#2016-05-3008:30cflemingHere, there seems to be no way to capture the [2.0 :teaspoon] vector object if I also want to match its contents.#2016-05-3011:17Alex Miller (Clojure team)@cfleming: you are matching its contents via the cat. If you want to receive the vector as the conformed value you could use coll-of to do a different kind of match or use a conformer to transform the matched result into any arbitrary structure#2016-05-3011:20Alex Miller (Clojure team)Probably the latter is what you want#2016-05-3017:33arohneris there a way to update a map spec? Iā€™d like to say ā€œthis fn takes a foo map, and returns a foo map with an extra key added onā€. Itā€™d be nice to say (s/def bar (conj foo ::extra-key)), without specifying the keys of the second map explicitly#2016-05-3018:01arohnerarg. repl development overwrites instrumentation#2016-05-3020:00moxajQuestion: why is the second argument (`retag`) necessary in a multi-spec expression? In the guide, it is the same as the dispatch function, and I don't see any use case where providing a different function would be beneficial. And if it's the same, naming it again is redundant, as it can be retrieved via the public field dispatchFn.#2016-05-3020:46seancorfieldAs an experiment, Iā€™ve added an optional spec namespace to clojure.java.jdbc: https://github.com/clojure/java.jdbc/blob/master/src/main/clojure/clojure/java/jdbc/spec.clj ā€” feedback welcome (the tests attempt to require that ns and instrument clojure.java.jdbc when running tests, so under 1.9.0 the specs are actually checked for all calls in the tests)#2016-05-3021:12potetmIs there any mechanism that will allow you to include possible exceptions as part of a spec?#2016-05-3022:26kovasbHow can I spec a function that takes an atom(x), where x conforms to a spec?#2016-05-3022:28kovasbthinking something like atom-of instead of coll-of#2016-05-3100:00kgzmHow would one use spec with records, when records use unqualified keywords and spec mandates the use of namespaced keywords?#2016-05-3100:36arohner@kovasb: AFAIK, that doesnā€™t exist currently, but it should be possible to write from cribbing coll-of#2016-05-3100:46arohner@potetm: ATM, no#2016-05-3101:39cfleming@alexmiller: Ok thanks, I donā€™t fully understand the second option but Iā€™ll investigate it.#2016-05-3101:44potetm@cfleming: https://clojure.org/guides/spec#_conformers#2016-05-3101:44potetmI guess the obvious followup (maybe for #C06E3HYPR?) is: Are there plans to support exceptions in specs?#2016-05-3101:56seancorfieldGiven that the spec describe data structures, and function inputs/output ā€” what would it mean to "support exceptions" in specs?#2016-05-3101:57potetmYeah that's fair.#2016-05-3101:58potetmMy first pass idea was just to "list the possible exceptions". But I'm totally open to the idea that that's just a bad idea.#2016-05-3101:59potetmI haven't thought it through or anything. Just noticed that I was hitting problems during generative testing because I couldn't declare that, say, arithmetic overflow, was an okay exception.#2016-05-3102:00cflemingIt seems like something you might want to specify.#2016-05-3102:00cflemingItā€™s a potential output from a function, more or less.#2016-05-3102:00seancorfieldI can see pros and cons. Certainly in (unit) tests it can be valuable to say "given these inputs, I expect the following exception"...
#2016-05-3102:01potetmcfleming: Yeah that was my thought as well.#2016-05-3102:01seancorfieldBut is that a failure to conform to its spec? It depends on whether the spec is considered to be "valid for all inputs that are valid" (and therefore "undefined" for invalid inputs)...#2016-05-3102:02seancorfieldI just went through this exercise with java.jdbc because there are lots of ways to generate exceptions from those functionsā€¦ but Iā€™m not sure how to specify that...#2016-05-3102:03seancorfieldIā€™m not even sure if it makes sense to try to specify that? What if the DB is down, or the table you ask for doesnā€™t exist or thereā€™s a syntax error in your SQLā€¦? How would you codify that in clojure.spec?#2016-05-3102:03seancorfieldHow could you even list all possible exceptions?#2016-05-3102:04potetmYeah it's possible that that would be overly restrictive.#2016-05-3102:04seancorfield(each JDBC driver throws its own types ā€” do you just say "could throw Exception"ā€¦?)#2016-05-3102:05seancorfieldItā€™s a hard problem, either way. clojure.spec definitely has gaps when you have stateful functions (generative testing on java.jdbc functions isnā€™t possible in general ā€” most garbage input will produce an exception, even if it is in the right "form").#2016-05-3102:05potetmI dunno, perhaps not since it's not exactly stopping you from doing anything at the end of the day.#2016-05-3102:11cfleming@seancorfield: Yeah, spec is mostly not that useful for me either, for similar reasons.#2016-05-3102:11mfikesSo Sean and Nolen both seem to have created a my.lib.main-ns.spec namespace to hold specs. Wonder if that will become a de facto place to put them for libraries.#2016-05-3102:12seancorfieldI took that approach so that you didnā€™t have to load the spec ns ā€” so you could use the code with Clojure < 1.9.0#2016-05-3102:13seancorfieldAlthough the build system shot me in the foot since it compiles all namespaces(!).#2016-05-3102:13seancorfieldConsequently, the build plain olā€™ fails on Clojure < 1.9.0. Not sure how to approach that (make the entire namespace conditional? Ugh!).#2016-05-3102:15mfikesHmmā€¦ need a good solution to that problem. Every lib will face it.#2016-05-3102:15potetmSpec actually does this for test.check.#2016-05-3102:15potetmIt's non-trivial.#2016-05-3102:15cflemingIā€™m just cutting and pasting clojure.spec.* and using Clojure 1.7.#2016-05-3102:16mfikesYeah, the dynaload. Hmm#2016-05-3102:16seancorfield@potetm: But you canā€™t have an optional namespace that uses spec and get it through the build system.#2016-05-3102:16seancorfieldI already deal with conditional loading in the java.jdbc tests.#2016-05-3102:17seancorfieldLocally, I can run lein test-all and pass tests on every Clojure version from 1.4 to 1.9 ā€” and only on 1.9 does it use the spec.#2016-05-3102:18seancorfieldBut the contrib build system tries to compile the namespaces in the projectā€¦ So maybe thereā€™s some Maven incantation I can use to exclude the namespace from compilation (or whatever Maven is trying to do with it).#2016-05-3102:19potetmRight so this is all in the build tool. Yeah I got nuthin....#2016-05-3102:19seancorfieldhttp://build.clojure.org/job/java.jdbc-test-matrix/410/ and http://build.clojure.org/job/java.jdbc-test-matrix/410/CLOJURE_VERSION=1.4.0,jdk=OpenJDK%201.6/console#2016-05-3102:21potetmWell except for this problem, that tool seems pretty slick šŸ™‚ I've never seen anything like it before.#2016-05-3102:21mfikesMaybe someone can make a hacked-up stub do-nothing compatibility lib in the clojure.spec and cljs.spec namespaces that consumers on 1.8 can use, just so things can be loaded. Ugh.#2016-05-3102:21seancorfield@potetm: Have you looked at core.typed or Prismatic Schema at all?#2016-05-3102:22potetmNo I mean the version matrix. All the JDKs all the clojure versions. A really nice idea for open tooling. (I've always had control over JDK and clj versions.)#2016-05-3102:22seancorfieldTomorrow Iā€™ll talk to @alexmiller about the build system and see if we can figure out something ...#2016-05-3102:51seancorfieldSean: 1, Maven: 0 šŸ˜ˆ So it turns out a contrib project can override the parent pom.xml and suppress the Maven-initiated compile phase: https://github.com/clojure/java.jdbc/commit/b224a3e86df8c00a33a16122e6fcc531c5f71e2e#2016-05-3102:51seancorfieldNow I feel dirty for having to learn that much about Maven šŸ˜ž#2016-05-3102:59seancorfieldApparently if I want to get really "clever" I could probably create a conditional profile based on the Clojure version and have it still run compile if weā€™re using 1.9.0ā€¦ Ugh! <profiles> ...#2016-05-3112:22andrewhr@seancorfield: lein's {:aot :all} doesn't kick in for included dependencies, did it? Maybe that's the reason for mavencompile's strategy#2016-05-3114:40akiel@anmonteiro You had an example of clojure.test integration were you used spec-is. I canā€™t find spec-is in the sources. Iā€™m also interested in a proper clojure.test integration of clojure.spec.test/check-var. clojure.spec.test/run-tests only works in the REPL for me. I need it in leiningen.#2016-05-3114:41anmonteiro@akiel: spec-is was something I wrote:
(defmacro spec-is [res]
  `(clojure.test/is (true? (:result ~res))))
#2016-05-3114:44akiel@anmonteiro: ok thanks this works.#2016-05-3115:58akiel@anmonteiro: Iā€™ve extended the is macro in the following gist. This looks even better to my eyes. https://gist.github.com/alexanderkiel/931387c7a86c1879b2267ad064067af7#2016-05-3117:33seancorfield@andrewhr: The pom.xml file indicates the compile is just a "sanity check" so Iā€™m taking that as "optional"...#2016-05-3119:05ghadiIs there anything in between s/keys and s/map-of for heterogenous maps --- maps with keywords and structured values as keys?#2016-05-3120:34seancorfieldAs I specā€™d java.jdbc I found myself wanting a way to constrain the values of the optional keys in the optional opts argument. I suspect the only way is (s/and (s/keys ā€¦) #(some custom predicate)) ?#2016-05-3120:35seancorfield(Iā€™d have to take a look at what s/keys conforms values to)#2016-05-3120:40kennyAre you guys tending to write your fdefs in a separate ns or in the same ns above/below where the function is written?#2016-05-3120:41sekaoiā€™ve been putting them above my functions so far#2016-05-3120:45kennyYeah that is great for readability but it seems like it kinda pollutes your ns.#2016-05-3121:53zaneHow would one spec a map with non-keyword keys and heterogenous values (the specs of which are dependent on the keys)?#2016-05-3121:55zaneThat is, how would you spec ::person from the guide if ::first-name ::last-name etc were strings?#2016-05-3122:13zaneOr, to put it another way, is there a version of clojure.spec/keys that works with non-keyword keys?#2016-05-3122:18seancorfield@zane: I suspect youā€™d need to say (s/map-of string? ::s/any) and then have some custom predicate on the conformed value.#2016-05-3122:19zaneRight, okay.#2016-05-3122:19zaneAs I'd feared.#2016-05-3122:20seancorfield@kenny: I think overall Iā€™d prefer data structure specs in a separate ns and fdef alongside the functions themselves but I wonā€™t have a solid feel for that until weā€™ve used it a bit more heavily.#2016-05-3122:20seancorfieldIn clojure.java.jdbc, I put all the specs in a separate ns so users on Clojure < 1.9.0 could still use the library.#2016-05-3122:33kennyMakes sense.#2016-05-3122:58zane@seancorfield: If I wanted to try to recover the the features of clojure.spec/keys but for maps with non-keyword keys would you recommend I implement clojure.spec/Spec myself?#2016-05-3123:05seancorfieldI suspect that will be a lot of work (but I havenā€™t looked at it). Since the push is for namespaced keywords ā€” but unnamespaced keywords are also supported ā€” I guess I would have to question your desire to define an API based on maps with non-keyword keys?#2016-05-3123:05seancorfieldBy which I mean, what specifically is it that youā€™re trying to spec out here that isnā€™t a "regular" Clojure map?#2016-05-3123:06seancorfield(and, perhaps therefore, spec "at large" is not designed for your use case?)#2016-05-3123:10seancorfieldMy sense with clojure.spec is that itā€™s opinionated deliberately to encourage a particular style of API specificationā€¦ "idiomatically Clojurey"ā€¦ The comments in particular about namespaced keywords being "tragically underutilized" and that namespace-qualified keywords is "a practice weā€™d like to see grow"...#2016-06-0100:03kennyHow are you guys spec'ing recursive functions where intermediate return values have a different spec than the final return value? Are you just adding an or to the :ret or are you writing your function in such a way that that isn't possible (e.g. bundling the recursion in the function by creating an anonymous fn inside your function)?#2016-06-0100:19seancorfield@kenny: Can you give an example of such a scenario?#2016-06-0100:20seancorfield(I generally try to avoid functions that can return multiple types although Iā€™ll often have functions that return something-or-nil)#2016-06-0100:20kenny
(defn new-matrix-nd
  "Returns a new general matrix of the given shape. Shape can be any sequence of
  integer dimension sizes (including 0 dimensions)."
  [dims]
  (if-let [dims (seq dims)]
    (vec (repeat (first dims) (new-matrix-nd (next dims))))
    0.0))
#2016-06-0100:20kenny
(s/fdef new-matrix-nd
        :args (s/cat :dims (s/nilable coll?))
        ;; ret is not a matrix because new-matrix-nd is a recursive fn
        :ret (s/or :m vector? :n number?))
#2016-06-0100:20kennyIdeally the spec should be
(s/fdef new-matrix-nd
        :args (s/cat :dims (s/nilable coll?))
        :ret matrix?)
#2016-06-0100:20seancorfieldso (new-matrix-nd []) produces 0.0?#2016-06-0100:21kennyYes#2016-06-0100:21kennyBut the final return value is always a matrix#2016-06-0100:21seancorfieldUnless you accidentally call it with nil or an empty sequence of dimensions...#2016-06-0100:22seancorfieldSo a zero-dimensional "matrix" is a scalarā€¦ hmmā€¦ thatā€™s an interesting one...#2016-06-0100:23kennyRight#2016-06-0100:24kennyI can wrap up the recursion inside the function#2016-06-0100:24kennyThat would probably be cleaner to the outside world#2016-06-0100:27seancorfieldIā€™m trying to think how specs are going to be much use to you here thoā€™ā€¦ since any call site is going to get "vector or scalar" as a result...#2016-06-0100:28kennyHmm.. Yeah it might not make sense.#2016-06-0100:29seancorfieldeven your :args spec doesnā€™t buy you much here: nil or an arbitrary collection ā€” you probably want (s/coll-of integer? []) to give you more checking ā€” and Iā€™d be tempted to disallow nil and require an empty sequence of dimensionsā€¦ unless you have a really good reason for allowing nil there?#2016-06-0100:31seancorfieldMaybe the Clojure/core folks need to give a bit more guidance in the Rationale as to how they expect spec to be used in the real world? /cc @alexmiller#2016-06-0100:32seancorfieldIā€™d imagined it as a set of specifications for some pretty high-level parts of your code, around your domain objects, or possibly as the definition of a library APIā€¦ but Iā€™m not sure about the latter yet...#2016-06-0100:32kennynil is just the terminating value. It seems wrapping the recursion inside the fn may solve this.#2016-06-0100:33seancorfieldAt work weā€™re looking at spec to write a specification of our domain model and some of the high-level business logic that operates on that. I donā€™t know how far "down" the call tree weā€™ll go...#2016-06-0100:36seancorfieldThe Rationale says "the supply chain isnā€™t burdened with correctness proof. Instead we check at the edges and run tests" ā€¦ "Invariably, people will try to use a specification system to detail implementation decisions, but they do so to their detriment."#2016-06-0100:38kennyThis cleans up the spec...
(defn new-matrix-nd
  "Returns a new general matrix of the given shape. Shape can be any sequence of
  integer dimension sizes (including 0 dimensions)."
  [dims]
  (letfn [(new-matrix-nd' [dims]
            (if-let [dims (seq dims)]
              (vec (repeat (first dims) (new-matrix-nd' (next dims))))
              0.0))]
    (new-matrix-nd' dims)))
#2016-06-0100:39kenny
(s/fdef new-matrix-nd
        :args (s/cat :dims (s/coll-of integer? []))
        :ret (s/or :m matrix? :s scalar?))
#2016-06-0100:40seancorfieldBut matrix? still allows a scalar, right?#2016-06-0100:40seancorfieldOtherwise (new-matrix-nd []) will pass the :args check but fail the :ret spec.#2016-06-0100:41kennyEdited to fix...#2016-06-0100:42seancorfieldYour refactoring hasnā€™t changed the spec thoā€™...#2016-06-0100:43kennyNo nil allowed in the fn#2016-06-0100:43seancorfieldWell, the spec disallows it, but that was true of the earlier function as well.#2016-06-0100:44kennyYou could pass nil to the previous fn. That should not be allowed.#2016-06-0100:45kennyThough that spec actually isn't working. (s/valid? (s/coll-of integer? []) nil) => true#2016-06-0100:45seancorfield(then I was wrong, s/coll-of allows nil?)#2016-06-0100:46kennyI guess. That doesn't seem right though#2016-06-0100:47kenny(s/valid? (s/* integer?) nil) => true#2016-06-0100:48seancorfield
boot.user=> (s/conform (s/* integer?) nil)
[]
boot.user=> (s/conform (s/coll-of integer? []) nil)
nil
#2016-06-0100:49seancorfield
boot.user=> (s/conform (s/* integer?) (list 1 2 3))
[1 2 3]
boot.user=> (s/conform (s/coll-of integer? []) (list 1 2 3))
(1 2 3)
#2016-06-0100:50seancorfieldSo coll-of is the spec for a (possibly nil) collection ā€” sequence of values ā€” and * is a regex to match zero or more items and returns a vector.#2016-06-0100:51kennyWas coll-of designed such that a possibly nil collection of values is valid?#2016-06-0100:52kennyNot sure how that can be an artifact as nil is inherently false so it would seem to be by design?#2016-06-0100:52seancorfield
boot.user=> (source s/coll-checker)
(defn coll-checker
  "returns a predicate function that checks *coll-check-limit* items in a collection with pred"
  [pred]
  (let [check? #(valid? pred %)]
    (fn [coll]
      (c/or (nil? coll)
            (c/and
             (coll? coll)
             (every? check? (take *coll-check-limit* coll)))))))
nil
^ this is what coll-of calls and it has a very specific check for nil
#2016-06-0100:53kennyHmm. Wonder why#2016-06-0100:53seancorfieldBecause nil-punning is idiomatic and encouraged?#2016-06-0100:54kennyI thought the idea was to be explicit about nil with nilable#2016-06-0100:54seancorfield(thatā€™s a question for Clojure/core really butā€¦)#2016-06-0100:55kennyMaybe we can get some input tomorrow on this.#2016-06-0100:57kennyI suppose it does make sense coming from the Java static typing world because you could pass null in place of any argument. We don't need to adhere to that though.#2016-06-0101:05seancorfieldBut any function that accepts a (general) collection is almost certainly going to accept nil Iā€™d sayā€¦?#2016-06-0101:06kennyWhy make any assumptions?#2016-06-0101:06seancorfieldAs noted above/elsewhere, clojure.spec is deliberately opinionated šŸ™‚#2016-06-0101:07seancorfield(not saying I agree, just observing)#2016-06-0101:11kennySeems strange, considering: https://www.infoq.com/presentations/Null-References-The-Billion-Dollar-Mistake-Tony-Hoare#2016-06-0101:11seancorfieldVery different idioms. If null isnā€™t treated as falsey, then you have a very different style of programming.#2016-06-0101:13seancorfieldFor example, in some languages youā€™d have a Maybe monad and you would chain operations using monadic functions. Scala has Option[T] and flatMap, as I recall. Haskell has Maybe t and fmap (yes?). But in Clojure youā€™d chain such operations using some-> or some->> and theyā€™d just be regular functions.#2016-06-0101:14seancorfieldOr you use if-let / when-let#2016-06-0101:15seancorfieldI was chatting with someone the other day and saying that I almost never hit a NPE in Clojure these days. About the only time I still do is in Java interop code (usually calling into the String class for something!).#2016-06-0101:15kennyTrue (though we can follow that style of programming is Clojure e.g. cats). Anyways, I suppose that is a tangent and doesn't really pertain to this situation.#2016-06-0101:16kennyYeah you're totally right. I can't recall the last NPE I got.#2016-06-0101:17seancorfield(the irony of this conversation today is that someone reported an error against java.jdbc that a particular erroneous call to one of the functions produces a NPE ā€” for invalid arguments)#2016-06-0101:18seancorfieldThe latest version of java.jdbc accepts those calls without error thoā€™...#2016-06-0101:20kennyAnyways, the primary reason I don't want nil to exist in this particular library is there is no such idea in the math world.#2016-06-0102:52andrewhrMy feelings about clojure.spec are pretty similar to yours @seancorfield... most for library public APIs (think pedestal, with namespaces keywords and data-based DSLs) or for general domain definition. Following the guidelines presented on Clojure Applied, I imagine using specs at domain edges and possibly Records only as a implementation detail of that domain (as suggested in the book).#2016-06-0102:55andrewhrI still need more though (or some good rule of thumbs) on the usage of namespaced keywords per se as a design tool. When to use a fully qualified - and by definition more coupled to implementation - vs just simple namespaces aka :foo/bar. And when it will be fine to use unqualified ones as envisioned by the Clojure/core style#2016-06-0102:55seancorfieldRecords require non-namespaced keys thoā€™ so I wonder how that sits with specs? (and I donā€™t think clojure.spec actually supports records?)#2016-06-0102:57andrewhrYeah, as far as I got it does not. Considering this style of "Records as an implementation detail", that makes sense no? I just following through the documentation snippet you shared#2016-06-0103:00cflemingIā€™m interested to know how spec handles records, too.#2016-06-0103:01cflemingI guess you treat the fields as non-namespaced keys?#2016-06-0103:12mikethompsonI'm considering spec from an Information Model point of view. Let's say I wanted to create a spec for car. It might have keys like colour and year. That part is easy. But what about manufacturer where I don't want containment, and instead I want a reference - an id? I could say the spec for manufacturer was integer?, but how should I be more specific and say it is a reference to another spec. My mind is wandering forward and imagining a tool which can read the central registry and produce a nice SVG Information Model. But it can't do that unless it knows that the manufacturer in car is more than an integer? ... it is actually a reference to another entity.#2016-06-0103:26seancorfield@mikethompson: Sure you can have ::manufacturer-id that specifies actual valid manufacturer IDs (or whatever you needed) and for testing you could specify a generator that produced (a small subset of) manufacturer IDs.#2016-06-0103:57zane> I guess I would have to question your desire to define an API based on maps with non-keyword keys? @seancorfield: Oh, believe me: If we were in control of this data we would use keyword keys. We're trying to enforce / check our assumptions about data we don't control.#2016-06-0104:08mikethompson@seancorfield: Yep, understand that. I'm just wondering about how that SVG Information Model Diagram tool could work. By looking in the central registry, how does it know that ::manufacturer-id is a reference to ::manufacturer. I can't see how that can be captured.#2016-06-0104:09mikethompsonAlmost seems like metadata on ::manufacturer-id#2016-06-0104:09seancorfield@mikethompson: maybe :manufacturer/id and :manufacturer/spec?#2016-06-0104:10seancorfieldConventions are needed somewhere, somehow.#2016-06-0104:10mikethompsonAhh. Metadata via naming convention šŸ™‚#2016-06-0104:11seancorfield@zane: If the keys are strings when you get them from "the outside" then maybe keywordize them for use "inside"?#2016-06-0104:11mikethompsonAnyway, all good. I was just wondering if I was missing something. Thanks#2016-06-0104:13seancorfield@mikethompson: No idea what Clojure/core might recommend here. I was trying to follow the Datomic lead there but I haven't looked at how referential specifications might work yet... Definitely something we'll run into at work as we move forward with this...#2016-06-0104:16seancorfield@zane: If you keywordize them "as-is" (maybe putting a standard namespace qualification on them too?), then you can validate with the full force of spec? At work, we're considering using a naming strategy with java.jdbc to create namespaced keys from the DB rather than just column names, so we can move straight to spec. Not sure how that will work out yet but we're planning a spike along those lines next week.#2016-06-0105:16josh.freckletonI'm still a bit new and wanting to bring in some typing. I'm not familiar with the respective libs, is spec effectively replacing schema and typed.clojure?#2016-06-0105:21seancorfieldHi @josh.freckleton -- have you read the Rationale for spec on the http://clojure.org site?#2016-06-0105:21seancorfield(it tries to clarify the spec vs type system position)#2016-06-0105:24josh.freckleton@seancorfield: I have, I'm just trying to clear things up a bit, and I guess i'm wondering where I should invest my time šŸ™‚#2016-06-0105:26seancorfieldspec is different to both core.typed and Schema...#2016-06-0105:29josh.freckleton@seancorfield: oh, can I ask another question while we're talking about this? in schema or spec, could I have a case (switch) according to types, or defmethods on different types? Would there be a prefered method?#2016-06-0105:29seancorfieldcore.typed is intended to provide a type checking system based on static analysis of the code with annotations. Schema is a runtime validation system.#2016-06-0105:30seancorfieldNot sure what you're asking... I think I'd have to see an example of what you're trying to do...?#2016-06-0105:32josh.freckleton(ahh, your typed vs. schema example makes sense. i still need to digest this) So say I have a few different "types", and I want to map a fn over them which is specific to their type, I've just started to learn about defmethods, and of course there are switch statements (`case`) switching on the type of the object under consideration. If I, say, had a list of things that could be all different types, what would be the best way in to switch on their types?#2016-06-0105:33seancorfieldSorry, I still don't understand...#2016-06-0105:33josh.freckletonmy fault, I'll be more specific...#2016-06-0105:39josh.freckletonI'm working with the free monad + interpreter pattern trying to build some code that takes a data model, and can interpret it in various ways. The important part is that the data model, essentially the Free Monad instance, is a recursive "type" where each "node" of this data model can be different types. Currently, I signify their "type" as a key :type in a mapping, and I (case (:type obj... to decide what to do with each "node". For example, one node can be a "User", and another can be "Owned Books", and another can be "Email", "Password", etc. As I interpret this data model, I will want to treat Users differently from Emails, and Passwords, etc. The 3 options I know of are to 1. defmethod according to different, real types, 2. to case over their type if spec or schema allows me to switch on types, or 3. what I have now with a map, and a "`:type`" key. Maybe I'm completely misguided in how I'm trying to solve this though, and I'm completely open to you annihilating my idea if it's far out-field! Does that make more sense?#2016-06-0105:40seancorfieldSo you want polymorphic dispatch on the :type key in the hash map?#2016-06-0105:41seancorfielddefmulti / defmethod would work for that. Or use a protocol with multiple defrecord implementations.#2016-06-0105:42seancorfield(the latter would be actual types and would no longer need the :type key)#2016-06-0105:43seancorfielddefmulti would use a dynamic dispatch based on aspects of the data structure, in your case the :type key.#2016-06-0105:44seancorfieldSo, basically, defmulti is a sort of case statement dispatch šŸ™‚#2016-06-0105:47josh.freckletonOk, I'm glad that way would work, thx. And would their be an idiomatic way to do it without declaring polymorphic methods, like: (case (spec/type obj) :a "a" :b "b"), or case (schema/type obj) :a "a" :b "b") I haven't seen anything in the docs that fits that ability to check the type of an obj, but I maybe I've missed it?#2016-06-0105:47seancorfieldYeah, I think you're misunderstanding what spec and Schema are about.#2016-06-0105:48seancorfieldWhy wouldn't you just do (case (:type obj) :a "a" :b "b") at that point? The :type key gives you what you need.#2016-06-0105:50seancorfieldOr (defmulti foo :type) (defmethod foo :a [obj] "a") (defmethod foo :b [obj] "b")#2016-06-0105:50josh.freckletonThat's probably why I couldn't ask it clearly šŸ˜’ I was just curious if that was an option, it seems similarish to me of Haskell's matching on different types. I think that's the "feel" I was going for, but again I could be wrong since I'm new the functional world.#2016-06-0105:52josh.freckletonfor example:
f Nothing = foo
f (Just x) = bar
You've been a big help sean, helping me prune my search tree of what I need to study, haha. Thank you so much, and thanks for staying up late to help us noobs out!
#2016-06-0105:55seancorfieldIn Clojure, Maybe would more likely just be nil or not-`nil` and you wouldn't pattern match, just (defn f [x] (if x 'bar 'foo))#2016-06-0105:56josh.freckletonoh sure, and there's also a maybe monad, I'm thinking for matching on many (10-100) custom types.#2016-06-0105:57seancorfieldRight, so if you want polymorphic dispatch on a single argument (type), you probably want a protocol and defrecord.#2016-06-0105:58seancorfieldIf you want more ad hoc polymorphism, defmulti is probably your tool#2016-06-0105:58seancorfieldClojure is very different from things like Haskell, since there's no extant type system, even tho' Clojure is strongly typed (at runtime).#2016-06-0105:58seancorfieldSo the idioms are very different.#2016-06-0105:59seancorfieldClojure's polymorphism is very different from OOP languages as well.#2016-06-0106:01josh.freckletonK, I had been considering these different options, and with your suggestion I'll zoom in on defrecord/`defmulti`, that helps me cut out tomorrows work šŸ™‚#2016-06-0106:01josh.freckletonThank you!!!#2016-06-0106:05seancorfieldHave fun!#2016-06-0106:24ikitommi@mikethompson: @seancorfield was thinking also for doing registry visualization to learn things. Did a schema->graphviz tools some time ago, which helps a lot. Are you already working on this?#2016-06-0106:25mikethompson@ikitommi: thought experiment for me. So, no, not working on it.#2016-06-0106:55seancorfieldI'm not a very visual person so it's a "no" on my end too. #2016-06-0114:23nwjsmithI'm having difficulty spec-ing a map. Would like to re-use spec for the keys and values. In particular, I'm trying to spec this bit of Datomic Pull: map-spec = { ((attr-name | limit-expr) (pattern | recursion-limit))+ }. I've already got specs for ::attr-name, ::limit-expr, ::pattern and ::recursion-limit, but I can't figure out how to re-use them in the spec for ::map-spec#2016-06-0114:23nwjsmithI thought s/map-of would be the ticket, but it works with predicates not specs.#2016-06-0114:40manutter51@nwjsmith: See the Entity Maps section of the guide http://clojure.org/guides/spec#2016-06-0114:41manutter51you want s/keys basically#2016-06-0114:44nwjsmithAFAICT s/keys is too strict, e.g I can't see how to specify that all keys conform to (or :attr-name ::attr-name :limit-expr ::limit-expr). It will only let me specify particular keys.#2016-06-0114:45nwjsmithI think what I'm looking for is something half-way between s/keys and s/map-of.#2016-06-0115:14zane@seancorfield: Those suggestions are great. Thanks! Full support for maps with keys that aren't keywords would still make sense to me, though. There are plenty of domains where what you're modeling with data really does require keys of other types.#2016-06-0115:51seancorfield@zane: I suspect maps with "other types" of keys are more likely to be homogeneous since you wouldnā€™t be able to enumerate all the keys (i.e., to list which are required and which are optional).#2016-06-0115:51seancorfieldWe have some maps from strings to hash maps, and some from longs to strings etc. But those donā€™t have specific required / optional keys ā€” theyā€™re "lookup tables".#2016-06-0115:52seancorfield(itā€™s an interesting area of discussion thoā€™)#2016-06-0115:52zaneThat makes sense, and matches my experience.#2016-06-0115:56nwjsmith@zane reading through your conversation w/ @seancorfield it looks like you and I have the same issue. Have you looked into implementing clojure.spec/Spec?#2016-06-0115:57zane@nwjsmith: Only superficially.#2016-06-0116:00zane@seancorfield: I'm realizing that one problem with the keywordize-before-running-specs approach is that any resulting error messages will refer to the keyword names rather than the string ones. That limits their utility significantly (makes them less useful to the calling client, for example).#2016-06-0116:02zaneMaybe validating JSON data via spec just isn't an intended use case?#2016-06-0116:04seancorfieldHow are you getting the JSON data? When we get JSON from a web service etc, we have clj-http keywordize the result, and we do the same with Cheshire as well. Basically, if we ingest JSON in any manner, we always convert to keywords at the boundary and validate afterward.#2016-06-0116:05zane(Aside: I'm learning a lot from your responses! Thanks!)#2016-06-0116:06zane@seancorfield: Right, right. But our intent was to then return any error messages explain-data produces from speccing the request to the calling client.#2016-06-0116:07zaneexplain-data's output is going to refer to the keywordized request, which could be confusing to the client if they're, say, unfamiliar with Clojure.#2016-06-0116:07seancorfieldHmm, Iā€™m not convinced explain-data is a good format for clients since it refers to the internals of your specs ā€¦ and what about localization of messages, and error codes for lookup in a reference manual for your API?#2016-06-0116:15zaneHmm. I suppose I didn't notice anything about specs that made me think they were fundamentally internally-facing. I'm curious what your thinking is there.#2016-06-0116:17zaneI had imagined that error codes could be generated via a transformation of the explain-data output. (Localization isn't necessary in this particular case since this is API is only used internally.)#2016-06-0116:40seancorfieldIf youā€™re going to transform the explain-data output, wouldnā€™t you then want to convert the keywords (including the spec names) to strings anyway?#2016-06-0116:41seancorfieldTaking this example from the Guide:
(s/explain-data ::name-or-id :foo)
;;=> {:clojure.spec/problems
;;    {[:name] {:pred string?, :val :foo, :via []},
;;     [:id] {:pred integer?, :val :foo, :via []}}}
#2016-06-0116:46seancorfieldYouā€™d want to convert :name and :id and whatever was in :via since those are keywords labeling parts of specs and paths through things ā€” so why not convert the actual keys as well?#2016-06-0116:47seancorfield(having said that, we havenā€™t gotten as far as trying this sort of thing to generate responses to clients ā€” we currently have custom validationā€¦ and that validation checks the types of values passed in as well as the structure so Iā€™m not sure how much we could delegate to specā€¦ but itā€™s an interesting option)#2016-06-0116:56zaneYes, you could totally convert the keywords back to strings, but you wouldn't have to if spec had better support for non-keyword keys in the first place. That's my whole point.#2016-06-0116:56zaneBut this is totally something that could be added via an extension library, so I guess I should roll up my sleeves. :relaxed:#2016-06-0117:01seancorfield(BTW, Alex Miller just replied on clojure-dev that heā€™s on vacation this week which is why heā€™s not participating here right now ā€” expect more feedback next week!)#2016-06-0117:39iwankaramazowIs it possible to dynamically generate specs from some sort of schema? I have a rest api which exposes a schema like this /contacts/schema (json)
{ "type": "Struct",
  "keys": {
    "first_name": {"type": "String"},
    "last_name": {"type": "String"},
    "is_organization": {"type": "Boolean"},
    "updated_at": {"type": "DateTime", "options": {"format": "%FT%TZ"}},
    "emails": {
      "type": "List",
      "schema": {
        "type": "Struct",
        "keys": {
          "label": {"type": "String"},
          "email": {"type": "String"}}}}}}
From reading the docs, it wasn't clear if I could generate dynamic specs from the above schema.
#2016-06-0118:20seancorfield@iwankaramazow: Given that clojure.specs API is pretty much all macros, my initial reaction would be "No". I would expect you could generate a Clojure source file from your JSON and use that to define the specs, however.#2016-06-0118:21seancorfieldIā€™m not sure how much of the underlying spec representation is publicly exposed to support dynamic generation of specs...#2016-06-0118:22iwankaramazow@seancorfield: Thanks for the response! Feared this, I'll experiment a bit#2016-06-0118:25arohnerIā€™ll take the contrary stance#2016-06-0118:26arohnerit is possible to build a spec from that, but it requires building some new ā€˜primitives'#2016-06-0118:27arohnerit is possible to build a fn that validates a map containing strings, and it is possible to build a test.check generator for for that schema#2016-06-0118:27arohneritā€™ll take a bit of work, but itā€™s doable#2016-06-0118:29arohner@seancorfield: specā€™s two primitives are predicates and test.check generators. If you can build a predicate for it, you can use it in spec. If you can build a test.check generator, you can use it to generate#2016-06-0118:30arohners/keys is big because it generates a bunch of predicates, one for map? and then one for each key/value. thereā€™s nothing special about it#2016-06-0118:32iwankaramazow@arohner: sounds logical, I'll brew me some algorithm. Thanks for the input#2016-06-0118:43seancorfield@arohner: What about dynamically (programmatically) registering those freshly built specs? Is enough of the machinery exposed for that?#2016-06-0118:44arohneryes#2016-06-0118:44seancorfield(and, yes, I know itā€™s all feasible since even private functions are accessible and spec is all built in Clojure ā€” but my "No" was meant to indicate whether it is a realistic goal to attempt and I still maintain the answer is negative there)#2016-06-0118:45arohnerSure, we have the tools to rewrite spec. I didnā€™t mean to go that far#2016-06-0118:46arohnerMy point is just that 1) any predicate is usable in spec 2) s/keys uses no special machinery 3) you can build a predicate to validate maps of strings 4) you can build a test.check generator to generate maps of strings#2016-06-0118:46arohners/keys is the way it is because Rich is being opinionated, not that other map predicates shouldnā€™t exist#2016-06-0118:57seancorfieldSome of the machinery youā€™d need is private in clojure.spec (e.g., res) and some of the macros that youā€™d need to replicate as functions are a lot of code. So, yeah, "itā€™ll take a bit of work" is certainly true.#2016-06-0119:13arohnerok, youā€™re right that youā€™d lose the per-field error messages#2016-06-0119:16arohnerbecause currently, a predicate would look like (defn struct? [m] (and (map? m) (contains? m ā€œtypeā€) ā€¦)#2016-06-0119:18arohneractually#2016-06-0119:18arohnerone sec#2016-06-0119:20arohner
(s/conform (s/and map? (fn [m] (contains? m "type"))) {"type" "structā€})
#2016-06-0119:20arohners/keys just builds up a set of #(contains? % <key>) predicates. So you just (s/and map? <chain-of-key-checking-preds>)#2016-06-0119:21arohnerhrm, so that gets you key checking, but doesnā€™t conform values yet#2016-06-0119:24arohneryou could use more ands to check values, but ideally we could reuse the pass-specs-to-check-values thing#2016-06-0119:31seancorfieldā˜ļø:skin-tone-2: the sign of a Clojure developer with too much time on his handsā€¦ šŸ˜† Itā€™s easy to get drawn into the "Hmm, thatā€™s an interesting problem!" rabbit-hole!#2016-06-0121:00kennyWhich is preferred?
(s/conform (s/cat :v (s/spec (s/* number?))) [[1 2 3]])
=> {:v [1 2 3]}
(s/conform (s/cat :v (s/coll-of number? [])) [[1 2 3]])
=> {:v [1 2 3]}
I assume the latter as it feels cleaner.
#2016-06-0121:03mario@kenny: As far as I understand, the former will allow any seq, while the latter will only allow vectors. So depends on what you want.#2016-06-0121:04kenny@mario: Nope:
(s/valid? (s/cat :v (s/coll-of number? [])) '((1 2 3))) => true
#2016-06-0121:05kennyThe vec at the end is just the value s/conform conforms to.#2016-06-0121:06marioI see. Then I donā€™t know šŸ™‚#2016-06-0121:17jebberjeb@kenny: why are you using the s/cat there? Do you really want to spec that it's a collection with only one other nested collection (of numbers)?#2016-06-0121:24kenny@jebberjeb: Yes.#2016-06-0121:33stathissiderisis there any way to say that I want 2 optional elements at the end of a cat but they have to be both present or not at all?
(s/conform (s/cat :e1 string? :e2 (s/? keyword?) :e3 (s/? keyword?)) ["foo"])
#2016-06-0121:34stathissiderisa bit like that, but is e2 is there, e3 should be there as well#2016-06-0121:34jebberjeb@kenny @mario That second argument to s/coll-of is only used when working with the generator for that spec, I believe. This succeeds but probably isn't a good idea: (s/conform (s/? (s/coll-of number? '())) [[1 2 3]])#2016-06-0121:38jebberjeb@stathissideris: think you want (s/cat :s string? :opt (s/? (s/cat :k1 keyword? :k2 keyword?)))#2016-06-0121:38stathissideris@jebberjeb: that looks like it could work, let me try!#2016-06-0121:39stathissiderisworks, many thanks šŸ™‚#2016-06-0121:41jebberjeb@stathissideris: yw#2016-06-0122:31arohnerstathissideris: use s/& around the cat to pass in a pred#2016-06-0122:32stathissideris@arohner: thanks! that's an option too, but I think I'm happy with @jebberjeb 's solution#2016-06-0122:33arohnerah yes, I missed the ā€˜at the endā€™ part, so you can use an optional cat#2016-06-0122:33arohnerin general, & is used to add extra constraints to a seq#2016-06-0123:03kennyWhat is the best way to spec multiple arity functions where the args in one overload aren't necessarily related to the args in other overloads. Example:
(defn triangular-matrix
  "Returns a triangular matrix created from `coll`. `coll` is a 1D vector where each element will be used to create the
  triangular matrix. `upper?` is set to true to create an upper triangular matrix, false for a lower triangular matrix.
  `diagonal` is a 1D vector of elements on the diagonal. `off-diagonal` is a 1D vector of upper or lower matrix elements."
  ([coll upper?]
    ;;impl
    )
  ([diagonal off-diagonal upper?]
    ;;impl
    ))
#2016-06-0123:07arohnerI think thatā€™s: (s/alt (s/cat :coll (s/coll-of ā€¦) :upper? boolean?) (s/cat :diagonal (s/coll-of ā€¦) :off-diagonal ā€¦.))#2016-06-0123:08arohner@kenny: ^^#2016-06-0123:09kennyYeah that makes sense to me#2016-06-0123:11kennyWhy is the common pattern to use cat instead of tuple for spec'ing args?#2016-06-0123:33stathissiderisspec looks amazing so far, the only thing missing is coercion#2016-06-0123:33stathissideristhis is the only point where I think schema is a bit more useful#2016-06-0123:33stathissiderisbut still, spec is much more expressive#2016-06-0123:50kenny@arohner: The only thing slightly annoying about that method is needed to specify keys for alt. The keys in this case are pretty arbitrary.#2016-06-0123:52stathissiderisI have a map that looks like this, but if the key is say :x I have a special spec for that value only: (s/map-of (s/or :k keyword? :k string?) ::s/any)#2016-06-0123:52stathissiderishow would I express that?#2016-06-0123:53stathissideriscustom predicate?#2016-06-0123:59stathissiderisvalidation works with a custom predicate:
(s/def ::simple-attr-map
  (s/map-of (s/or :k keyword? :k string?) ::s/any))

(s/def ::attr-map
  (fn [{:keys [transform] :as m}]
    (and (s/valid? ::simple-attr-map (dissoc m :transform))
         (or (not transform)
             (s/valid? ::transform transform)))))
#2016-06-0200:00stathissiderisbut as I expected, you lose conform destructuring and explain can't explain the problem anymore#2016-06-0200:00stathissideris...so different solutions would be welcome#2016-06-0200:01george.w.singerClojurescript is failing to compile my file if my fdefs are located above their corresponding defns. Is this the way things are intended?#2016-06-0200:01george.w.singerI.E., you can't spec your function before you define it?#2016-06-0200:02kenny@george.w.singer: Yes. The var is not defined yet. You could declare the var if you must write the spec above the function. I have just been writing my specs below my function definitions.#2016-06-0200:03george.w.singerOk, thanks.#2016-06-0200:05george.w.singerMacro sugar needs to be added to spec IMO#2016-06-0200:05george.w.singerSomething like#2016-06-0200:07george.w.singer
(fn-spec
 fn-name ::arg1, ::arg2 -> ::ret-spec 
                             :such-that    ::ret-value-greater-than-max-of-its-inputs)
This would at once declare our fn-name and annotate it with corresponding specs using intuitive Haskell-like syntax.
#2016-06-0200:13stathissideris@george.w.singer: you could write that macro šŸ™‚#2016-06-0200:14george.w.singeryea šŸ˜„#2016-06-0200:23arohner@george.w.singer: I think thatā€™s a CLJS-impl bug, Iā€™d report it#2016-06-0200:24arohnerin CLJ, you can (s/fdef new-fn) ahead of the var#2016-06-0200:26kenny@george.w.singer: Does it work if you do
(s/fdef 'your-sym
        :args ...
        :ret ...)
#2016-06-0200:35george.w.singerIt does sometimes but then the whole thing becomes buggy. Sometimes it doesn't compile. All tests pass#2016-06-0200:35george.w.singerReally strange behavior is exhibited#2016-06-0200:36george.w.singerThe strange behavior goes away if put them before#2016-06-0200:54kennyI seem to be getting a strange error when creating a spec for a function with a type hinted argument. Error:
java.lang.ClassCastException: clojure.spec$spec_checking_fn$fn__11414 cannot be cast to clojure.lang.IFn$DO
Example:
(defn bar
  [f]
  (f 0.0))

(s/fdef bar
        :args (s/cat :f fn?)
        :ret number?)

(defn foo
  [^double value]
  (bar (fn [_] value)))

(s/fdef foo
        :args (s/cat :v number?)
        :ret number?)

(defn foo2
  [value]
  (bar (fn [_] value)))

(s/fdef foo2
        :args (s/cat :v number?)
        :ret number?)

(foo 1.0) => error
(foo2 1.0) => 1.0

(s/instrument-all)
#2016-06-0200:57kennyProblem exists in 1.9.0-alpha4.#2016-06-0202:05seancorfieldLooks like primitive hinting encodes the argument and return type to be encoded in the underlying type (if you add ^double as a return type hint, the cast is to IFn$DD, for example).#2016-06-0202:51seancorfieldVery surprised to see that calling an instrumented function can actually load and call clojure.test.check to do generative testing as part of the conformance test, without explicitly calling clojure.spec.test/run-all-tests etc.#2016-06-0202:51seancorfieldhttp://dev.clojure.org/jira/browse/CLJ-1936#2016-06-0207:59stathissiderisis this a bug?
spec> (gen/sample (s/gen #{1 0}) 10)
(0 0 1 0 0 0 1 0 1 0)
spec> (gen/sample (s/gen #{false true}) 10)
(true true true true true true true true true true)
#2016-06-0208:02hiredmans/gen takes a spec, which is a predicate#2016-06-0208:02hiredman#{false true} will only return true, as a predicate, for true#2016-06-0208:14stathissideris@hiredman: thanks for your explanation, but I still don't get how I would generate true/false randomly#2016-06-0208:17slipsetIā€™ve defined these specs and vars#2016-06-0208:17slipset
(s/def ::point (s/cat :x number? :y number?))
(s/def ::body (s/+ ::point))
(s/conform ::point [2 3])
(s/conform ::body [[2 3]])
(s/explain ::body [[2 3]])
#2016-06-0208:18slipset::point conforms nicely, but ::body does not conform, and the explanation is:#2016-06-0208:18slipsetIn: [0] val: [2 3] fails spec: :snake-game.utils/point at: [:x] predicate: number?#2016-06-0208:19slipsetI would have thought that (s/+ ::point) would mean one or more points?#2016-06-0208:31stathissideris@slipset: (s/def ::point (s/spec (s/cat :x number? :y number?)))#2016-06-0208:32stathissiderisThe combination of s/+ and s/cat means that body is defined as a series of :xs and :ys without nesting#2016-06-0208:32slipsetthanks šŸ™‚#2016-06-0208:37stathissiderisfor the record, I ended up with this for booleans:
(defn- boolean? [x] (instance? Boolean x))
(s/def ::boolean (s/with-gen boolean? #(gen/boolean)))
#2016-06-0209:25stathissiderisis there any way to make s/cat specs generate a vector?#2016-06-0209:57slipsetWhat @stathissideris said :)#2016-06-0220:38surreal.analysisThis is a pretty minor note, but this appears to be one of the only channels with _ separation instead of -. Any chance an admin could change that?#2016-06-0220:39arohnerand as we all know, clojure names should prefer - to _
#2016-06-0220:40seancorfieldI see no stinkinā€™ underscores here...#2016-06-0311:16jcfI'm having some trouble spec'ing a fn that walks maps allowing you to hyphenate keys etc. I wonder if someone can point out my mistake as I'm having a hard time decrypting the error messageā€¦
(defn walk-map
  "Recursively apply a function to all map entries. When map is nil returns an
  empty map."
  [f m]
  (if m
    (walk/postwalk (fn [x] (if (map? x) (into {} (map f x)) x)) m)
    {}))

(s/fdef walk-map
  :args (s/cat :f (s/fspec :args (s/tuple ::s/any ::s/any) :ret ::kv)
               :m ::maybe-any-map)
  :ret ::any-map)

(defn hyphenate-keys
  "Recursively transforms all map keys from underscored strings to hyphenated
  keywords."
  [m]
  (walk-map (fn [[k v]] [(hyphenated-keyword k) v]) m))

(s/fdef hyphenate-keys
  :args (s/cat :m ::maybe-any-map)
  :ret ::any-map)
#2016-06-0311:16jcfThe error:
ERROR in (t-hyphenate-keys) (core.clj:4631)
expected: (= (sut/hyphenate-keys {:a 1, "a" 2}) {:a 2})
  actual: clojure.lang.ExceptionInfo: Call to #'example.common/walk-map did not conform to spec:
In: [0] val: ({}) fails at: [:args :f] predicate: (apply fn),  nth not supported on this type: PersistentArrayMap
:clojure.spec/args  (#object[example.common$hyphenate_keys$fn__20568 0x754a38a0 "
#2016-06-0311:17jcfI've tried a s/cat to capture ::s/any but I'm sure I'm missing something.#2016-06-0311:18jcfWith the s/cat that I'd expect I need:
ERROR in (t-walk-map) (core.clj:4631)
expected: (= (sut/walk-map (fn [[k v]] [(name k) (if (number? v) (inc v) v)]) {:a 1, :b {:c 2, :d {:e 3}}}) {"a" 2, "b" {"c" 3, "d" {"e" 4}}})
  actual: clojure.lang.ExceptionInfo: Call to #'example.common/walk-map did not conform to spec:
In: [0] val: ([[] []]) fails at: [:args :f] predicate: (apply fn),  clojure.lang.PersistentVector cannot be cast to clojure.lang.Named
:clojure.spec/args  (#object[example.common_test$fn__21912$fn__21926 0x7787a602 "
#2016-06-0311:20jcf
(s/fdef walk-map
  :args (s/cat :f (s/fspec :args (s/cat :kv ::s/any) :ret ::kv)
               :m ::maybe-any-map)
  :ret ::any-map)
Hmm.
#2016-06-0311:22jcfShouldn't an fspec with :args (s/cat :kv ::s/any) match any and all args?#2016-06-0312:27jcfThis is what's confusing me. This spec looks right, but doesn't work in the fspecs :args.
(s/explain-data (s/cat :kv (s/tuple ::s/any ::s/any)) [[:a {:b 2}]])
#2016-06-0312:31manutter51what is your hyphenated-keyword fn doing? Iā€™m not sure Iā€™m following whatā€™s happening there, but it looks like itā€™s blowing up in that function, and thatā€™s causing the spec failure#2016-06-0312:33jcf
(defn- hyphenated-keyword
  [x]
  (if (or (string? x) (keyword? x))
    (-> x keyword->string infl/hyphenate keyword)
    x))
@manutter51: I've used this code in a number of projects for a few years now. The fns are pretty reliable.
#2016-06-0312:36jcfThe relevant specs I'm using:
(s/def ::any-map
  (s/map-of (s/nilable ::s/any) (s/nilable ::s/any)))

(s/def ::maybe-any-map
  (s/nilable ::any-map))
#2016-06-0312:36jcfNot sure if ::s/any is already nilableā€¦ šŸ™‚#2016-06-0312:41jcfIn the walk-map test I get a cast exception:
ERROR in (t-walk-map) (core.clj:4631)
expected: (= (sut/walk-map (fn [[k v]] [(name k) (if (number? v) (inc v) v)]) {:a 1, :b {:c 2, :d {:e 3}}}) {"a" 2, "b" {"c" 3, "d" {"e" 4}}})
  actual: clojure.lang.ExceptionInfo: Call to #'example.common/walk-map did not conform to spec:
In: [0] val: ([[] []]) fails at: [:args :f] predicate: (apply fn),  clojure.lang.PersistentVector cannot be cast to clojure.lang.Named
:clojure.spec/args  (#object[example.common_test$fn__21912$fn__21926 0x5ea46e79 "
In the tests that rely on walk-map:
ERROR in (t-underscore-keys) (core.clj:4631)
expected: (= (sut/underscore-keys {"a-b" 1}) {"a_b" 1})
  actual: clojure.lang.ExceptionInfo: Call to #'example.common/walk-map did not conform to spec:
In: [0] val: ([[] []]) fails at: [:args :f] predicate: (apply fn)
:clojure.spec/args  (#object[example.common$underscore_keys$fn__20550 0x1f36bed9 "
#2016-06-0312:42jcfThese error messages aren't intuitive. I've got a feeling this will be another thing newcomers to Clojure really struggle with.#2016-06-0312:45jcfThis looks like it might help with the vague error message: https://github.com/clojure/clojure/commit/68fe71f0153bb6062754442c0a61c075b58fd9bc#2016-06-0312:45jcfIt recurses into a ::pcat to expand out the error explanation, and I think I'm looking at a ::pcat above.#2016-06-0312:48jcfOkay, so only the commented out test fails:
(deftest t-walk-map
  (are [f m x] (= (sut/walk-map f m) x)
    identity nil    {}
    identity {}     {}
    identity {:a 1} {:a 1}

    ;; (fn [[k ^long v]] [(name k) (if (number? v) (inc v) v)])
    ;; {:a 1 :b {:c 2 :d {:e 3}}}
    ;; {"a" 2 "b" {"c" 3 "d" {"e" 4}}}
    ))
#2016-06-0312:53jcfOkay. Looks like spec was exercising my walk-map with keys like [] and (). I was then calling name on vectors etc.#2016-06-0312:54jcfFixed that by making my test fn valid. šŸ™‚
(deftest t-walk-map
  (are [f m x] (= (sut/walk-map f m) x)
    identity nil    {}
    identity {}     {}
    identity {:a 1} {:a 1}

    (fn [[k v]]
      [(if (named? k) (name k) k)
       (if (number? v) (inc ^long v) v)])
    {:a 1 :b {:c 2 :d {:e 3}}}
    {"a" 2 "b" {"c" 3 "d" {"e" 4}}}))
#2016-06-0312:54jcfOne down. One to go!
ERROR in (t-underscore-keys) (core.clj:4631)
expected: (= (sut/underscore-keys nil) {})
  actual: clojure.lang.ExceptionInfo: Call to #'example.common/walk-map did not conform to spec:
In: [0] val: ([[] []]) fails at: [:args :f] predicate: (apply fn)
:clojure.spec/args  (#object[example.common$underscore_keys$fn__20550 0x6dd8e64b "
#2016-06-0312:56jcfI wonder if it's the same problem. I need to make sure I have named keys.#2016-06-0314:01jcfIt was. Because I had enabled instrumentation my functions were being called with vectors, sets, etc. and those weren't supported.#2016-06-0314:50benzapso i'm converting from schema, and i'm having some issues with respect to preventing clutter within my namespace. maybe i'm not structuring my data correctly#2016-06-0314:51benzapex. `(def TextBlock {:type (schema/eq "Text") :foreground-color Color :background-color Color :style {(schema/optional-key :bold) schema/Bool (schema/optional-key :underline) schema/Bool (schema/optional-key :italic) schema/Bool} :text Letter}) (s/def ::foreground-color ::color) (s/def ::background-color ::color) (s/def ::style (s/keys :opt-un [::bold ::underline ::italic])) (s/def ::textblock (s/keys :req-un [::type ::foreground-color ::background-color ::style]))`#2016-06-0314:52benzaphow would I apply s/def to ::bold, ::underline and ::italic, without s/def? I don't want those in the namespace#2016-06-0314:52benzapif I did have them in the namespace, i'd prefer to have them called something like ::font-style-underline, but the underlying spec would need to accept :underline as the style key#2016-06-0314:54benzapIs there any way to do this? I'm confused on whether I should be inlining the spec stuff. I'm sticking to what I did with plumatic.schema by placing all of my schemas in one file called schemas.cljs#2016-06-0315:20jcf@benzap: I don't think you can. You need to def the keywords in order to enable reuse.#2016-06-0315:21jcfAnd I'd think you only want really common schema in a global namespace as spec is embracing namespaced keywords.#2016-06-0315:22jcfSo, :font/bold and :font/underline instead of :schemas/font-style-underline etc.#2016-06-0315:24jcfI've used specs like :common/any-map because I want nilable maps in a lot of places, but there might be a better way I haven't yet found.#2016-06-0315:25jcf(And I've done away completely with my shared schema namespaces for now.)#2016-06-0315:29benzap@jcf: I see, so I guess i'll have to start embracing namespaced keywords as well šŸ™‚#2016-06-0315:30benzapI already have a font.cljs, would it be common to place spec in-line with the code?#2016-06-0315:33jcfThat's what I'm doing, and what I've seen in all the examples.#2016-06-0315:34jcfIt makes sense keeping everything starting with font in font.cljs; that's where I'd look at least!#2016-06-0319:08nwjsmithHrm, is there a way to specify a transform to a spec's conformed data?#2016-06-0319:09nwjsmithI'm speccing datomic query, which accepts either a list or a map, and I'd like them to conform the same way#2016-06-0319:32nwjsmithFound it! Looks like clojure.spec/conformer is what I was after.#2016-06-0321:03hiredmanI haven't seen this discussed elsewhere, but as a warning, protocol functions called in a non-higher order way (where clojure knows the protocol function is being called an creates an optimized call site for it) don't properly check args and returns against specs if instrumented#2016-06-0415:48nwjsmithI'd like to ship a library without requiring test.check, and have the generators dynamically loaded as they are with clojure.spec, but I can't figure out how to do that. Do I wrap all calls to clojure.spec.gen functions in clojure.spec.gen/delay?#2016-06-0415:50nwjsmith(the library is essentially a bunch of specs, some of which require custom generators)#2016-06-0418:40hiredmanthat is kind of interesting to think about, if you are defining custom specs with generators in your source, then test.check needs to be a regular dependency not a test scoped dependency#2016-06-0418:41hiredmana delay delays runtime code from running, not compile time stuff#2016-06-0418:41hiredmanlike var resolution#2016-06-0418:42hiredmanyou likely need some combination of something like dynaload from clojure.spec.gen and the delay there and maybe clojure.core/delay#2016-06-0418:43bbloomor just put your generators in another namespace?#2016-06-0418:45hiredmanthat would still have issues with any kind of tooling that loads all namespaces (in the absence of test.check)#2016-06-0418:45bbloomwhat kind of tooling does that? seems kinda like a bad idea šŸ˜›#2016-06-0418:47hiredmanI was thinking of stuff like lein check, but that only loads stuff from the current project#2016-06-0418:49bbloomon another trackā€¦. 1st order predicate expressions and regular languages are closed under intersection ā€” would be cool if somebody wrote an ā€˜intersectā€™ function for spec, could be useful for static analysis#2016-06-0418:52arohner@bbloom: elaborate?#2016-06-0418:55bbloom@arohner: well you could just write an abstract interpreter that walks the clojure AST and instead of an environment of symbols -> values, have an environment of symbols -> specs#2016-06-0418:55arohneroh, yeah#2016-06-0418:55bbloomeach time you use something, you call intersect on it#2016-06-0418:55bbloomif the result is void, you have a "type error"#2016-06-0418:55arohneryeah, just wasnā€™t sure what you meant by intersect#2016-06-0418:55arohnerI have about half of that written#2016-06-0418:56arohnercurrently reimplementing the regex part of spec to handle it#2016-06-0418:56bbloomif each spec defines a set, intersect = set intersection#2016-06-0418:57bbloomintersecting two predicates is just and-ing them, but you could do better for some predciates, cat, repeat, alt, etc#2016-06-0418:58arohneryeah, itā€™s a lot more work than that šŸ™‚#2016-06-0418:59bbloomcertainly not trivial, but should be at least straightforward, no?#2016-06-0418:59arohneryeah#2016-06-0418:59arohnerparsing specs and the regex stuff is straightforward, but a ton of code#2016-06-0419:00arohnersame w/ flowing predicates through a fn body#2016-06-0419:01bbloomyeah, i have a clojure interpreter that iā€™ve copy/pasted and modified 4 or 5 times for code walking purposes - someday iā€™ll make it a library#2016-06-0419:01bbloomin the mean time: https://github.com/brandonbloom/metaclj/tree/master/src/metaclj/impl#2016-06-0419:02bbloomthe transform namespace is an example of a code walk on that ast & you could replace that with a type check walk#2016-06-0419:03bbloomhereā€™s an interpreter: https://github.com/brandonbloom/eclj/blob/master/src/eclj/interpret/meta.eclj#2016-06-0419:03arohnerIā€™m already walking the tools.analyzer output, so itā€™d be a good amount of work to swap#2016-06-0419:04bbloomnot suggesting that you switch to my thing šŸ˜› just showing that it can be much less code than tools analyzer, which does all the analysis all at once#2016-06-0419:26bbloomanyway @arohner: excited to see what you come up with šŸ™‚#2016-06-0419:27arohneryeah, me too. Just need to finish the regex matching crap#2016-06-0421:04gfredericksthe first big question about clojure.spec that I'm going to try to dive into is how I can get functionality similar to https://github.com/gfredericks/schema-bijections#2016-06-0421:05gfredericksso I suspect (ha!) I'll be reading a bunch of the clojure.spec source soon#2016-06-0421:20gfredericksis it weird that s/tuple takes preds instead of specs?#2016-06-0422:11benzapI thought preds were specs#2016-06-0422:12benzapI have a tuple '(s/def ::color (s/tuple ::byterange ::byterange ::byterange ::byterange))' that seems to work @gfredericks#2016-06-0422:19benzapas of now, I have successfully converted my project from plumatic.schema to cljs.spec!#2016-06-0422:19benzapstill have a lot to learn about it, but it sure is nice#2016-06-0423:19gfredericksbenzap: yeah it does seem to work, but that means the docs are misleading I think: preds are specs but specs aren't preds#2016-06-0500:08Alex Miller (Clojure team)the terminology drifted a bit during dev#2016-06-0502:55adambrosIs there anything planned in terms of having predicates/specs with custom error messages/data? Iā€™m speaking from a validation perspective, because the alternative is to wrap explain-data#2016-06-0502:56adambrosit also seems that for related reasons, I would be writing a conform-or-explain! function that throws with explain-data when the conform fails#2016-06-0503:02adambrosWhile Iā€™m asking, it would be a useful feature to be able to somehow get the path at which a predicate is being run. This one is coming up as Iā€™m trying to secure a datomic transaction wrapper by looking up if a user has access to a certain :db/id thatā€™s potentially nested within a map.#2016-06-0503:25seancorfield@adambros: which alpha are you using? I thought they'd added a path result in explain in one of the later alpha builds.#2016-06-0503:27adambrosiā€™m referring to accessing the path from within the predicate#2016-06-0503:28adambrosmight be bad idea (TM) but its something that would ameliorate my problem#2016-06-0503:30seancorfieldBut each predicate only knows what it is called on -- it can't know the whole path, since a predicate is a general function.#2016-06-0503:32seancorfieldIf your predicate is string? for example...#2016-06-0503:32adambrosit could if there was a dynamic var *path* or something. I think i could solve this another way so its fine#2016-06-0503:33seancorfieldDynamic vars are evil šŸ˜†#2016-06-0503:33adambros*evil*!#2016-06-0503:33seancorfieldšŸ™‚#2016-06-0503:33adambrosim more interested in my other questions, so i probably shouldnā€™t have asked about it#2016-06-0503:37seancorfieldOk, what are the other questions? I think I missed those...#2016-06-0503:38adambroscustom error message for predicates/specs something like (s/def :title (s/either string? {:message ā€œinvalid title"}))#2016-06-0503:39adambroswhere eitherā€™s predicate either succeeds or puts the 2nd argument in explain-dataā€™s result#2016-06-0503:40adambrosand the other was about a potential builtin for validation, conform-or-explain! that throws if it cant conform#2016-06-0503:41seancorfieldI get the impression from the discussions here that explain-data is indeed the preferred basis for any custom error messages. I think there is a JIRA ticket for something like conform-or-explain tho'... let me go look...#2016-06-0503:46seancorfieldHmm, no, maybe it was just a discussion on the clojure-dev mailing list.#2016-06-0503:47adambrosseems like itā€™d be a simple wrapper, with an extra arity for a function to call with the data when it fails#2016-06-0503:47adambros(defn conform-or-explain [spec x & [on-fail]] ...)#2016-06-0503:48adambrosby default itā€™d throw the error in an ex-info#2016-06-0503:51seancorfieldAh, I remember now, it was a valid or explain wrapper that was discussed -- for use in :pre assertions.#2016-06-0503:55seancorfieldBut, yeah, I think the wrapper is so easy to write -- and some folks would want to throw explain-data and some would throw explain as a string so I don't think there's a one size fits all here.#2016-06-0503:56adambrosfair enough, sounds like I just need to have some sort of spec namespace that declares these wrappers#2016-06-0503:58seancorfieldOne of the great things about Clojure: it's easy to write an adapter layer to make the supplied libraries match each specific application's needs šŸ™‚#2016-06-0503:58seancorfieldWe have a bunch of wrappers at World Singles. We have several of our own string functions in a namespace that supplements clojure.string.#2016-06-0504:07seancorfieldWe also have some collection functions, a bunch of date stuff, wrappers around system level stuff. Wrappers around JDBC stuff (and, heck, I maintain clojure.java.jdbc!).#2016-06-0513:30benzapanyone know of a good way to define a spec for an atom holding a piece of structured data?#2016-06-0513:31benzapseems like I would need to deref in order to perform the spec, so a lot of wrapping even for simple checks#2016-06-0514:01nwjsmith@benzap: I think someone asked this a few days ago and the answer was "There isn't because atoms aren't 'data'". IIRC there was also a suggestion to take a look at coll-of, which might have some bits you could crib to implement an atom-of. I might look into this today.#2016-06-0515:09Alex Miller (Clojure team)personally, I think you should work with state by a) defining your problem in terms of data b) writing nearly every function as a pure function from data to data c) isolating the use of stateful entities to as small number of functions as possible d) focusing on writing great specs for b in terms of a#2016-06-0515:10Alex Miller (Clojure team)not every function has to have a spec#2016-06-0518:50gfrederickshas anybody started a clojure.spec utility library yet because I keep thinking about doing that#2016-06-0518:51gfredericksshould probably call it spook#2016-06-0519:00Alex Miller (Clojure team)you may want to check your urban dictionary before you venture down that path#2016-06-0519:00gfredericksurban dictionary has all the best words#2016-06-0519:01gfredericksspock#2016-06-0519:01Alex Miller (Clojure team)totally on a tangent, the urban dictionary was hiring for a full stack clojure position recently#2016-06-0519:02Alex Miller (Clojure team)spock is much better :)#2016-06-0519:02Alex Miller (Clojure team)although I think there is a scala (or groovy?) testing library called that already#2016-06-0519:02Alex Miller (Clojure team)https://github.com/spockframework/spock#2016-06-0519:03Alex Miller (Clojure team)spackle?#2016-06-0519:06gfredericksshpec#2016-06-0519:07gfrederickssplec, shplec#2016-06-0519:08bronsashpec sounds like something sean connery would say#2016-06-0519:08gfredericksokay so we already have a mascot#2016-06-0519:09gfredericks(require [com.gfredericks.shpec :as s'])#2016-06-0519:10bronsajust realized 'foo' is valid clojure#2016-06-0519:10gfredericksI only wish the syntax highlighters were more often aware of that#2016-06-0519:10gfredericks...because I use that all the time#2016-06-0519:34mfikesReplete (standalone ClojureScript iOS app) has an update in the App Store with support for cljs.spec.#2016-06-0520:16gfredericksśpƩć#2016-06-0520:55stathissiderisis there any way to make s/cat specs generate a vector?#2016-06-0520:59gfredericksstathissideris: both s/coll-of and s/tuple can generate vectors -- are either of those usable for you?#2016-06-0521:00stathissideris@gfredericks: tuple would work for my validation, but ideally I would like to keep the destructuring that s/cat provides#2016-06-0521:01gfrederickswhat you want is a way to gen/fmap the default generator#2016-06-0521:01Alex Miller (Clojure team)use with-gen to override the generator with fmap#2016-06-0521:01gfredericksalexmiller: but how do you get the default generator to pass to fmap?#2016-06-0521:02Alex Miller (Clojure team)well you could call s/gen on the original spec but I guess you may end up repeating yourself#2016-06-0521:02gfredericksthat was the hack I was about to suggest#2016-06-0521:03gfrederickswe can put a helper for this into shpec#2016-06-0521:04stathissideristhank you both! I'm not very well-versed with test.check, but I'm saving this part of the log so that I can go do some reading. It will hopefully make more sense after that šŸ™‚#2016-06-0521:05gfredericksstathissideris: we're suggesting (s/def ::foo <definition>) (s/def ::foo (s/with-gen <definition> (let [g (s/gen ::foo)] #(gen/fmap vec g))))#2016-06-0521:05gfredericksand <definition> being there twice is the repetition alex mentioned#2016-06-0521:07stathissiderisgreat thanks, I'll give it a go. In my case <definition> is not too long, so it could be ok#2016-06-0521:08gfrederickscould even try factoring it out to a let and see if that works#2016-06-0521:08stathissideris...or even a macro is that doesn't work#2016-06-0521:11gfredericksalexmiller: the default generator for s/and is necessarily pretty hacky; should there be some documentation somewhere suggesting an override?#2016-06-0521:12gfrederickssomehow overriding test.check's normal error message for such-that would be a fancy approach#2016-06-0521:17Alex Miller (Clojure team)I think there is some desire to have a better (programmatic) way to have a conversation about filtering with such-that than just boolean yes/no#2016-06-0521:18Alex Miller (Clojure team)but I'm sure Rich would have more to say about that#2016-06-0521:20gfredericksthat sounds sooper fancy#2016-06-0521:21gfredericksalexmiller: do you know if recursive specs is the only reason for the no-arg-function-returning-a-generator pattern?#2016-06-0521:22Alex Miller (Clojure team)Stu worked on that but I assume the dynamic loading is another aspect#2016-06-0521:22gfredericksoh that sounds plausible#2016-06-0612:08mandersonCurious if there has been any thought given to adding a doc string parameter to clojure.spec/def like there is in clojure.core/def? I understand the spec should be self-explanatory, but it's nice to be able to capture additional information/description about the spec. I've found myself adding comments that would be nice to have as doc strings that could then also be parsed programatically.#2016-06-0612:09Alex Miller (Clojure team)yes, there has been thought about it#2016-06-0612:11Alex Miller (Clojure team)not sure if anyone noticed, but you can now call doc with a registered spec keyword to get it's spec definition. that's a place that docstring could show up.#2016-06-0612:11Alex Miller (Clojure team)but it's something Rich is still thinking about afaik#2016-06-0612:14mandersoncool, thanks Alex. I think the built in doc string could be really useful.#2016-06-0612:14mandersonAlso, didn't notice the doc addition. That's great.#2016-06-0613:53mike_ananevHello all! I learn clojurescript. Today I discovered that clojurescript doesn't throw Exceptions in code above like Clojure. Can clojure.spec help in such situations? I use reagent and want prevent state updating from using inappropriate function.#2016-06-0614:02nwjsmith@mike1452: Hey Mike, welcome to ClojureScript! clojure.spec can definitely help you here:#2016-06-0614:03mike_ananev@nwjsmith: Thank you!!#2016-06-0615:26gfredericksnwjsmith: the spec needs to be (s/cat :arg integer?)' instead of just integer?`, right?#2016-06-0615:26gfrederickss/integer/number/#2016-06-0615:27nwjsmithThat's what I thought at first, but maybe this is a situation where a pred is implicitly converted to a spec?#2016-06-0615:29gfredericksI guess I'll just go try it :)#2016-06-0615:29nwjsmithwhen passed to valid? or conform, predicates will be implicitly converted to a spec, which I think is what happens here.#2016-06-0615:31gfredericksit throws an error when you call it with a number as well#2016-06-0615:31gfredericksit's not about spec vs predicate it's about spec-for-a-single-arg vs spec-for-an-arglist#2016-06-0615:33nwjsmithooooh#2016-06-0615:36nwjsmith@mike1452 looks like the example I gave above is wrong. The call to s/fdef should be:
(s/fdef inc :args (s/cat :x number?))
thanks @gfredericks
#2016-06-0615:41gfredericksnp#2016-06-0616:47mike_ananev@nwjsmith: got it! Thanks!#2016-06-0618:03angusiguessIs there a nice way to spec the following?#2016-06-0618:04angusiguess"I want a map of arbitrary keywords whose values must conform to some predicate"#2016-06-0618:05nwjsmithmap-of#2016-06-0618:06angusiguess@nwjsmith: Beauty, thank you šŸ™‚#2016-06-0618:06nwjsmithnp#2016-06-0622:42wilkerluciowith clojure.spec I'm finding myself wanting to use some kind long namespaces for my keywords, they are just getting to be a borden to write, for example (s/keys :req [:my-app.some-ns.clock/at, :my-app.some-ns.clock/namespace]), the namespace my-app.some-ns.clock doesn't actually exists, it's just helpful to avoid name clashing, I'm writing those in the my-app.some-ns#2016-06-0622:42wilkerlucioI was thinking that would be useful to be able to alias those namespaces so they can be used in a simpler way, for example:#2016-06-0622:44wilkerlucioit's possible to alias like this with the currently clojure features?#2016-06-0622:45noonianI believe if you require a namespace like (ns my-ns (:require [my-app.some-ns.clock :as clock])) you can use ::clock/foo and it will resolve to :my-app.some-ns.clock.#2016-06-0622:46wilkerlucio@noonian: yes, but that would require me to create a file with that namespace, that's what I would like to avoid#2016-06-0622:47noonianAh got it, sorry I didnā€™t read your message carefully enough.#2016-06-0622:47jr@wilkerlucio: see this proposed feature http://dev.clojure.org/jira/browse/CLJ-1910#2016-06-0622:48wilkerluciothanks @jr, I had saw that one, and I agree that will help, but only on map cases... when you are setting it as keywords for lists (on the s/keys for example) you can't use that, so we would still have to repeat the typing, I believe aliasing is a more general solution here that could be very useful with long namespaces#2016-06-0622:48hiredmanyou don't actually need to create a file now#2016-06-0622:49hiredmancreate-ns#2016-06-0622:51hiredman
user=> (create-ns 'my-app.some-ns.clock)
#object[clojure.lang.Namespace 0x3b7d3a38 "my-app.some-ns.clock"]
user=> (alias 'clock 'my-app.some-ns.clock)
nil
user=> ::clock/foo
:my-app.some-ns.clock/foo
user=> 
#2016-06-0623:19wilkerlucio@hiredman: thanks, I'm going to try that soon, do you think that can work on cljs as well?#2016-06-0623:21hiredmancljs is a different kettle of fish, I am not sure#2016-06-0623:31wilkerlucioyeah, I have to fix some compilation issues here and then I'll try it out#2016-06-0702:33Alex Miller (Clojure team)@wilkerlucio: in addition to the new namespaced support in CLJ-1910 and CLJ-1919 we are considering changes related to aliasing of non-existent namespaces too#2016-06-0702:34wilkerlucio@alexmiller: cool, that would be nice, specially for cljs where I can't do the trick mentioned by @hiredman#2016-06-0702:37Alex Miller (Clojure team)I did some recon on it a few weeks ago and we talked about it briefly again today#2016-06-0702:39wilkerlucioI'm happy to hear that this is being considered, I think with clojure.spec the usage of fully namespaced keywords is going to have a great increase in usage#2016-06-0708:08jcfFound some clojure.spec benchmarks over on Reddit: https://muhuk.github.io/validation-benchmark/ N.B. I haven't independently verified any of the work.#2016-06-0708:29slipsetplaying with specs and tests in Clojurescript I get this error:#2016-06-0708:29slipset actual: #error {:message "Call to [object Object] did not conform to spec#2016-06-0708:29slipsetThe error is probably correct, but the message leaves something to be desired.#2016-06-0708:58slipsetHaving said that, specs are awesome!#2016-06-0711:41mishadoffHi, could you help me understand clojure.spec Here is the little snippet:
(s/def ::name string?)
(s/def ::specific-name #{"John" "Jack"})
(s/def ::age (s/and integer? #(<= 0 % 100)))
(s/def ::person (s/keys :req-un [::name ::age]))
(s/explain ::person {:name "John" :age 10})
Is there a way to validate that map contains specific key like :name, but value should be matched against another spec, ::specific-name?
#2016-06-0711:46slipset(s/def ::person (s/and (s/keys :req-un [::name ::age]) #(#{ā€œJohnā€ ā€œJackā€} (:name %))))#2016-06-0711:47slipsetnot answering your question, but should work.#2016-06-0711:49mishadoffyes, but would be good to use already defined spec ::specific-name instead of copying its implementation#2016-06-0711:55slipsetDon't know your code, but you could (def specific-names #{...}) and use that var in the specs#2016-06-0711:57mishadoffI can, but that not a spec anymore#2016-06-0712:12Alex Miller (Clojure team)you can do (s/def ::name ::specific-name) to just alias an existing one#2016-06-0712:12Alex Miller (Clojure team)spec will "chase" registered names like that#2016-06-0712:14Alex Miller (Clojure team)or you could define a predicate that is reused for multiple specs#2016-06-0714:12bronsalooks like spec might need some performance tuning https://muhuk.github.io/validation-benchmark/#2016-06-0714:27Alex Miller (Clojure team)I don't think that test is very good, but that may also be true. will be looking at it today#2016-06-0714:31seancorfieldThe use case for clojure.spec is different to Schema tho', right?#2016-06-0714:33seancorfieldYou might have conform in several places for destructuring and some calls to valid? but you're not going to have everything instrumented in production code. #2016-06-0714:33seancorfieldWhereas folks do have Schema enabled in production don't they? Hence the focus on performance there. #2016-06-0714:34gfredericksschema aims to be used similarly though#2016-06-0714:34seancorfieldAh, so not in production code then?#2016-06-0714:34gfrederickswhen you decorate functions with schemas they don't run by default#2016-06-0714:34gfredericksand similar to spec you can turn them all on#2016-06-0714:34gfredericksfor testing#2016-06-0714:35gfredericksand you can call them explicitly to validate in/out in production#2016-06-0714:35gfredericksso maybe they focus on performance primarily for that last case?#2016-06-0714:44Alex Miller (Clojure team)we still want spec to have good performance, even when not used in production :)#2016-06-0715:09bronsa@seancorfield: we disable schema in prod, performance hit is way too much#2016-06-0715:19wilkerlucio@mishadoff: like alexmiller said, you can create a new one, if you still need that key on the current namespace, make up a new namespace, eg: (s/def :other-ns.anything/name ::specific-name) and then (s/keys :req-un [:other-ns.anything/name])#2016-06-0715:21mishadoff@wilkerlucio: thanks, for now Iā€™ve decided to use same names for keys and specs#2016-06-0715:22wilkerlucio@mishadoff: cool, that is the preferred way I believe šŸ™‚#2016-06-0715:23mishadoff@wilkerlucio: yes, but sometimes you have maps with unqualified keys from third-party libs#2016-06-0715:24wilkerluciothat's true, maybe with the clojure.spec people will start using more fully qualified keywords from now one, and possibly with specs already coming from the library itself, will be awesome šŸ™‚#2016-06-0715:53gfredericksis :opt in s/keys just for documentation purposes?#2016-06-0715:53gfredericksI can't figure out how else it affects anything#2016-06-0716:09angusiguess@gfredericks: does it work with generators?#2016-06-0716:09gfredericksoh that's probably true#2016-06-0716:10gfredericksyes indeed#2016-06-0716:11gfrederickscool.#2016-06-0716:42Alex Miller (Clojure team)yes, those reasons :)#2016-06-0716:44Alex Miller (Clojure team)bunch of new predicates (most with gen support) just landed in master https://github.com/clojure/clojure/commit/58227c5de080110cb2ce5bc9f987d995a911b13e#2016-06-0716:45Alex Miller (Clojure team)also, the long-awaited seqable? :)#2016-06-0717:01andrewhrAlso, a nice bonus to see an abstraction for Inst šŸ™‚#2016-06-0717:05seancorfieldAnd Alpha 5 is comingā€¦ when? šŸ˜ø#2016-06-0717:29gfredericksalexmiller: oh boy this inst generator#2016-06-0717:30Alex Miller (Clojure team)@seancorfield: winding its way through the tubes#2016-06-0717:31gfredericksTIL that java.util.Date does weird things with negative years#2016-06-0717:31Alex Miller (Clojure team)ha#2016-06-0717:31Alex Miller (Clojure team)probably better to use something like inst-in with that :)#2016-06-0717:33gfrederickscoming up with reasonable generators in a lot of cases is difficult :/#2016-06-0717:33gfredericksreasonable defaults I mean#2016-06-0717:34gfredericksstrings are my best example of that#2016-06-0717:34Alex Miller (Clojure team)yeah#2016-06-0717:34Alex Miller (Clojure team)here's a bunch of stuff that in no way resembles reality #2016-06-0717:34gfredericksa uniform distribution over unicode characters would give you 99% unprintable things#2016-06-0717:40Alex Miller (Clojure team)1.9.0-alpha5 https://groups.google.com/d/msg/clojure/D_s9Drua6D4/CTWk12cXDQAJ#2016-06-0717:41Alex Miller (Clojure team)notably for spec is the new unform which lets you conform ... backwards#2016-06-0717:42Alex Miller (Clojure team)also in addition to the vast quantity of new predicates in core, there are now specs for long, double, and instant ranges in spec#2016-06-0717:42Alex Miller (Clojure team)and everything gens yay#2016-06-0717:45gfredericksyay#2016-06-0717:46seancorfieldThatā€™s an awesome new release! Thank you (to everyone involved)!#2016-06-0717:52angusiguessyessssss#2016-06-0718:10brabsterhey folks, I have a question on map-of in spec that I'd appreciate some help with#2016-06-0718:10brabsterhopefully this is a good forum, even if it turns out to just be me being dumb!#2016-06-0718:11brabsterif I use s/or in the value spec in a map-of, I lose the qualifier I get when I use it on its own#2016-06-0718:12brabstereg. as per docs (s/def ::thing (s/or :name string? :id integer?)) conforms eg. (s/conform ::thing "bob") to [:name "bob"]#2016-06-0718:12brabsterbut... (s/def ::map-of-kw-to-thing (s/map-of keyword? ::thing)) doesn't behave the same was under conform#2016-06-0718:13brabster(s/conform ::map-of-kw-to-thing {:foo "bob"}) gives {:foo "bob"}, not {:foo [:name "bob"]} as I was expecting#2016-06-0718:14brabsterAm I missing something? I expected ::thing to behave the same way regardless of where it was used#2016-06-0718:16brabsteroh sorry I see a message in the history that I missed from @alexmiller: Rich said above "currently conform doesn't flow into coll-of, so the value is never conformed, only checked" which I think is likely related to this - sorry!#2016-06-0718:20brabsteris there somewhere I can go to see whether this is planned or to ask for it?#2016-06-0718:22gfredericksqueue the warnings from all your favorite libraries that clojure.core/boolean? is being replaced by their local copy#2016-06-0718:25pheuteryep.#2016-06-0718:26Alex Miller (Clojure team)hopefully all benign if the work we did in 1.7 was successful :)#2016-06-0718:29gfrederickswhat sort of work?#2016-06-0718:30hiredmanisn't http://dev.clojure.org/jira/browse/CLJ-1591 still open? I didn't accidentally re-open it did I?#2016-06-0718:32Alex Miller (Clojure team)it's open but it's very narrow in scope iirc - only when defining a new fn of the same name in terms of the old function of the same name, right?#2016-06-0718:32Alex Miller (Clojure team)the general case of just overlapping the name was resolved afaik#2016-06-0718:33gfredericksah#2016-06-0718:33Alex Miller (Clojure team)prior issues came up when we added "update"#2016-06-0718:33bronsayeah, I don't actually think that's a bug#2016-06-0718:33Alex Miller (Clojure team)we have since added other things that were in general use without issue (although I can't remember an example now)#2016-06-0718:34bronsathe fact that def declares the var at analysis time is the only way to write recursive functions with defn#2016-06-0718:34hiredmanmmm, I guess I am note sure what version of clojure the guy was running when I w as helping him debug this#2016-06-0718:34Alex Miller (Clojure team)@hiredman: I'm not sure if your final example in the comments is actually the same thing or not, not sure?#2016-06-0718:34bronsaFWIW I couldn't reproduce that example#2016-06-0718:35gfredericksshould it be possible to write a function that verifies that none of the keyword-references used in any specs have typos?#2016-06-0718:35gfredericksusing keywords instead of vars makes the whole thing feel a lot more typo-sensitive#2016-06-0718:36hiredmanyeah, I didn't look at the history to see the current state of the bug, just spent a while trying to figure out why this guys defmulti was resulting in an unbound var, and after way too long I remember that bug, and he said changing the name of his function fixed it, but that is kind of loose, who knows what is happening on the other end of irc#2016-06-0718:36gfredericksby "typos" I mean a keyword that is supposed to refer to a spec but no spec has been registered#2016-06-0718:36Alex Miller (Clojure team)that's not a typo#2016-06-0718:36Alex Miller (Clojure team)necessarily#2016-06-0718:37Alex Miller (Clojure team)as long as it's registered by the time you use it, it's fine#2016-06-0718:37gfredericksright, so I mean "never registered"#2016-06-0718:37Alex Miller (Clojure team)well never is a long time#2016-06-0718:37gfredericks:)#2016-06-0718:37gfredericksI could have a test in a codebase that looks at this for example#2016-06-0718:38gfredericksso it would have the responsibility to ensure that all pertinent code is loaded already
#2016-06-0718:38pheuterSo Iā€™m trying to play around with the new clojure.spec generators, and I keep getting this error: Var clojure.test.check.generators/large-integer is not on the classpath. I definitely have test.check as a dep and i can access various vars under the clojure.test.check.generators namespace, but large-integer isnā€™t one of them.#2016-06-0718:39gfrederickspheuter: did you add test.check magically later?#2016-06-0718:39gfredericksthat's when I get that error#2016-06-0718:39pheuterno, we had it as a dep for a while#2016-06-0718:39gfrederickspheuter: is it an older version? large-integer is new#2016-06-0718:39Alex Miller (Clojure team)are you using test.check 0.9.0 ?#2016-06-0718:39pheuteryeah, 0.9.0#2016-06-0718:39Alex Miller (Clojure team)do you have an older version also on the classpath?#2016-06-0718:39pheuterlet me clear deps and try to re fetch#2016-06-0718:39Alex Miller (Clojure team)lein deps :tree#2016-06-0718:40gfredericksyeah do tree ā†‘ and look for conflicts#2016-06-0718:40gfredericks(at the top of the output)#2016-06-0718:42pheutergood call, will take a look and make some exclusions, thanks!#2016-06-0718:50Alex Miller (Clojure team)@brabster: I asked Rich, he said the point of that was that conform samples (does not check every value) for perf reasons and that is unlikely to change. we might make the docs better around it though#2016-06-0718:53brabster@alexmiller: thanks for the info, would suggest mentioning it with an example in the docs as that's not the behaviour I'd expect from map-of - working around it at the moment but would have been a nice multi-method dispatch off the conformed inputs, looks ugly in comparison now. Will share when pushed up to github, maybe I'm just misusing the functionality!#2016-06-0718:55Alex Miller (Clojure team)I making some updates to the guide for alpha5 right now - I'll try to add a note about that#2016-06-0718:56brabsterbtw spec is great imho - really neat and well thought through after a few hours of using it#2016-06-0719:06gfredericksalexmiller: the thing that makes me uneasy is when I have to type a fully qualified keyword referencing a spec#2016-06-0719:06gfredericksbut where it's actually defined elsewhere using ::#2016-06-0719:07gfredericksor regardless actually#2016-06-0719:07gfrederickswith var namespaces the compiler will catch renamings that were only partially done#2016-06-0719:07gfrederickswith specs it just looks like something isn't defined yet#2016-06-0719:58pheuterany ideas on how to use generators, possibly custom, to generate Datomic facts? More specifically, Iā€™d like to generate legitimate temp ids, but itā€™s not clear to me how to express that as a predicate.#2016-06-0719:59gfrederickstemp ids are a custom datomic type aren't they?#2016-06-0719:59pheutera thought is to use a set predicate thatā€™s populated with a bunch of tempids#2016-06-0719:59gfredericksso the predicate would be #(instance? ThatType %)#2016-06-0720:00gfrederickswhat's the API to create a tmp id? it's a 0-arg function and it returns a unique object each time?#2016-06-0720:00pheutereven simpler, they can be verified as longs! the problem is being able to actually transact them against a test db#2016-06-0720:00pheuter(tempid part)#2016-06-0720:00gfrederickslong?#2016-06-0720:00gfrederickslongs?#2016-06-0720:00gfredericksthat can't be right...#2016-06-0720:01gfredericks(tempid part) returns a long?#2016-06-0720:01pheuterwell not exactly, it returns a :db/id reader value, but you can pull out an :idx long from it#2016-06-0720:01pheuterthe problem is getting the generate to generate values in sequential order#2016-06-0720:01gfrederickssequential?#2016-06-0720:02gfrederickswhy does that matter?#2016-06-0720:02pheuterperhaps that part doesnā€™t matter, thatā€™s just my interpretation of the error I get: db.error/not-a-partition Entity id -1000441 is not in a valid partition#2016-06-0720:02pheuterwhere -1000441 is a sample temp id#2016-06-0720:02gfredericksare you passing a valid partition to tempid when you call it?#2016-06-0720:03gfredericksoh um#2016-06-0720:03pheuterso how would i call it?#2016-06-0720:03gfredericksyou just generate a random long and construct the thing that way rather than calling tempid?#2016-06-0720:04pheuterright, since -1 < n < -100000 is reserved for tempids in the user partition#2016-06-0720:04gfredericksbut -1000441 is out of range, does that mean you're accidentally generating out of range numbers?#2016-06-0720:04pheutersorry, -1000000 is the actual bound#2016-06-0720:04gfredericksstill out of range#2016-06-0720:04pheuteryouā€™re right, let me double check#2016-06-0720:05pheuteriā€™ll feel very silly if thatā€™s the case šŸ˜•#2016-06-0720:05gfredericksfeeling silly is what programming is all about#2016-06-0720:06mishahttp://blog.cognitect.com/cognicast/103#2016-06-0720:08pheuter :db.error/not-a-partition Entity id -3427 is not in a valid partition ĀÆ\(惄)/ĀÆ#2016-06-0720:10pheuterah, it may be how iā€™m transacting that value, since it expects it in a #db/id reader value#2016-06-0720:10pheuterat least when using the map form#2016-06-0720:20zaneFor anyone following along, the approach we're gonna try is using clojure.test.check.generators/fmap in conjunction with datomic.api/tempid and clojure.test.check.generators/choose.#2016-06-0720:24gfrederickszane: I assume you are also pheuter; now that I think about it I suspect you'll have to be careful about uniqueness?#2016-06-0720:25pheuteryes#2016-06-0720:25gfredericksyou can use gen/vector-distinct to get a collection of distinct things#2016-06-0720:25pheuterthat was my follow-up :relaxed:#2016-06-0720:25pheuteroh awesome#2016-06-0720:26zaneGood observation!#2016-06-0720:27zaneYes, @pheuter and I are colleagues.#2016-06-0720:35pheuterHm, looks like gen/vector-distinct is not one of the lazy combinators in clojure.spec.gen#2016-06-0720:36gfredericksyou can use it directly from clojure.test.check.generators/vector-distinct#2016-06-0720:36gfrederickswrap it in a function if you have to#2016-06-0720:36pheuteryeah, makes sense#2016-06-0720:36pheuterthanks#2016-06-0804:13shaunxcodein the cognicast espisode rich mentions a new syntax for default namespaces for keywords in a map - do we know what that looks like yet?#2016-06-0804:17noonian@shaunxcode: this jira item has some info http://dev.clojure.org/jira/browse/CLJ-1910?page=com.atlassian.streams.streams-jira-plugin:activity-stream-issue-tab#2016-06-0812:13moxaj@alexmiller It's a shame that conform does not (and probably won't) flow into map-of and coll-of šŸ˜ž. I have a use case where performance is mostly irrelevant and inner values should be conformed. It seems the functionality is there, but instead i'll have to resort to some error-prone, hairy tree transformation. Would you perhaps consider a dynamic var or an optional argument to map-of and coll-of to enable such transformations (at the expense of performance)?#2016-06-0812:41Alex Miller (Clojure team)@moxaj there actually is 'coll-check-limit' now #2016-06-0812:42Alex Miller (Clojure team)Defaults to 100 but you could set it higher#2016-06-0812:42Alex Miller (Clojure team)I'm not sure whether that changes what flows though#2016-06-0812:49moxaj@alexmiller It does not, currently. Just to be clear, only the size restriction is settled on, the flowing stuff might be subject to change?#2016-06-0813:06Alex Miller (Clojure team)Let's just say I'm not the one making the decision :)#2016-06-0813:59richhickeya conforming coll spec would not have the same name - either you are exhaustive/conform or neither, not going to be a dynamic magic thing#2016-06-0814:26dominicmIs there a way to specify in spec, that a key is the focus of my validation, (or where the path of the failure would be), but take other values from the map? Useful for having two fields that match for example.#2016-06-0814:32Alex Miller (Clojure team)you can write a custom predicate that validates anything you like#2016-06-0814:32Alex Miller (Clojure team)like #(= (:foo %) (:bar %))#2016-06-0814:32Alex Miller (Clojure team)not sure if I'm answering your question though#2016-06-0814:55wilkerlucio@dominicm: I guess you want something like (s/and (s/keys :req [::password ::password-confirmation]) #(= (::password %) (::password-confirmation %)))#2016-06-0814:57dominicm@alexmiller @wilkerlucio Do these methods set the paths? So that path would/could be set to [::password-confirmation]#2016-06-0814:58wilkerlucio@dominicm: sorry, I don't understand what you mean by set the paths, can you please clarify?#2016-06-0814:58dominicm@wilkerlucio: In the failure, you get a path when you do a key.#2016-06-0814:59wilkerluciowell, you always have to be composing, so it may depend where the failure is#2016-06-0814:59wilkerlucioif it's a key with a problem, it will point directly to it#2016-06-0815:00wilkerlucioif it's on the last checker (the match one) it will point to the map and say that the fail comes from the map and point the predicate out#2016-06-0815:00wilkerluciodoes that answer your question?#2016-06-0815:02wilkerlucio@dominicm: I think is a good idea to try the (explain) on the REPL so you see if the explanations of the errors met your needs#2016-06-0815:03dominicm@wilkerlucio: Yeah. It answers my question. I'm trying to figure out how I can that the predicate after failure, actually comes from ::password-confirmation when doing explain.#2016-06-0815:08dominicm
(s/def ::a boolean?)
(s/def ::b boolean?)

(s/def ::thing (s/keys :req [::a ::b]))

(s/explain
  ::thing
  {::a true
   ::b "foo!"})
;; => In: [:foo.core/b] val: "foo!" fails spec...
#2016-06-0815:08dominicmI want to be able to set that In for say password validation. So that it's set to ::password-confirmation#2016-06-0815:18dominicmTo clarify, I want this:
(s/def ::a boolean?)
(s/def ::b boolean?)

(s/def ::thing (s/and (s/keys :req [::a ::b])
                      #(= (::a %) (::b %))))

(s/explain
  ::thing
  {::a true
   ::b false})
To result in: ;; => => In: [:foo.core/b] val: "foo!" fails spec...
#2016-06-0815:59wilkerlucio@dominicm: not sure if there is an easy way to do that, you might get some insight by checking the sources for (s/keys)#2016-06-0816:00dominicm@wilkerlucio: Looks like it works on reification. Fun fun fun.#2016-06-0816:15brabsteris there any way to tell what spec was used to conform a value?#2016-06-0816:15Alex Miller (Clojure team)no?#2016-06-0816:17brabsterI was looking for a way to define a function to transform conformed output#2016-06-0816:18brabsterso if I had a way of extracting the keyword that defined the spec I could use a multimethod to do that - probably misthinking it#2016-06-0816:20brabstersure I can find another way, thought it was worth asking anyway. thanks @alexmiller?#2016-06-0816:22nwjsmith@brabster: if you would like to customize a spec's conform function, you can do it by implementing the Spec protocol: https://github.com/nwjsmith/datomic-spec/blob/master/src/datomic_spec.clj#L286-L293#2016-06-0816:24nwjsmithAlthough I'm not sure that's the best way to go about it, I just wanted my Datomic queries to conform the same whether in map or list form#2016-06-0816:25Alex Miller (Clojure team)I think you're treading on impl details there that can change without warning, fyi#2016-06-0816:29nwjsmithšŸ˜“#2016-06-0816:37nwjsmithI'm going to try the multi-spec route to see if I can get similar results. Maybe I don't want conformed map and list queries to be the same.#2016-06-0816:42brabster@nwjsmith: thanks for offering up the suggestion anyway šŸ™‚#2016-06-0818:17borkdudeWe're having a meetup about spec and are wondering about the output in this snippet:
(s/explain-data (s/cat :cat1 keyword?
              :cat2 (s/& (s/cat :cat1 integer?
                                :cat2 integer?)
                         #(> (:cat1 %) (:cat2 %)))
              :cat3 keyword?)
       [:foo 42 43 :baz])
#2016-06-0818:28fentonhaving issues just getting specs going... lein repl complains about spec not being defined#2016-06-0818:34nwjsmith@fenton have you tried lein clean?#2016-06-0818:34nwjsmithWhat is the output of lein deps :tree?#2016-06-0818:36fentonthank you...i had it over-ridden in ~/.lein/profiles.clj ! doh!#2016-06-0819:02richhickey@borkdude: what are you wondering?#2016-06-0819:03richhickeythe fact that it reports the error at :baz ?#2016-06-0819:05richhickeythatā€™s because :cat2 has eaten the ints, then failed its secondary pred. Currently & canā€™t know that its preds wonā€™t be made happy by some future input, so itā€™s still working on the & and then canā€™t eat :baz#2016-06-0819:09richhickeycurrently regex ops can communicate that they can succeed w/o more input, but not that they are ā€˜fullā€™, which is what would be required for & to bail out#2016-06-0819:22richhickeythat particular case could probably be handled though#2016-06-0819:36dryewowhat does (s/cat :a :b) mean? It compiles, but matches only empty lists#2016-06-0819:36dryewoto me it does not seem a valid spec, it should complain#2016-06-0819:37gfredericksyeah the :b seems like the potentially invalid part#2016-06-0819:37gfrederickswell though#2016-06-0819:37dryewoIā€™m asking because I had smth like (s/cat :url ::url) and I forgot to define ::url above#2016-06-0819:37gfrederickskeywords are predicates#2016-06-0819:37gfredericksso it's probably being treated that way#2016-06-0819:38gfredericksbut arguably it would be worth not treating them that way since you can't used namespaced keywords as predicates anyhow#2016-06-0819:38gfredericksthough if it matches empty lists then I'm probably mistaken about the treating-it-like-a-predicate part#2016-06-0819:38dryewo(s/valid? :a {:a 1}) throws an exception#2016-06-0819:46ikitommicould there be spec-enabled versions of fn and defn?#2016-06-0819:46borkdude@richhickey: thanks!#2016-06-0819:47ikitommiwriting a function (defn age-plus1 [{:keys [::age]}] (inc age)) already defines something about itā€™s inputs, the specs could be extracted from the source?#2016-06-0820:00richhickey@borkdude: next alpha will do this ^#2016-06-0820:00richhickeythanks for the report#2016-06-0820:11angusiguess@ikitommi: If I understand correctly, the reason for fdef and explicit instrumentation is to prevent default instrumentation of functions.#2016-06-0820:39stathissideris@richhickey: In the cognicast you mentioned a bit about the possibility of generating specs because they're data. It's not obvious to me how I would be able to do that, could you point me the right direction?#2016-06-0820:42richhickeythey are data as code is data, youā€™d generate them as you would any other code#2016-06-0820:42stathissiderisoh that's what you meant! thanks šŸ™‚#2016-06-0820:43richhickeycall s/form on a spec for an example#2016-06-0820:44stathissideristhat's very useful... I'm playing with a (simple) heuristic that would produce specs from example data#2016-06-0820:44stathissiderishard to get right, but maybe I can come up with something that's good enough#2016-06-0820:46richhickeyspecs for spec will help, e.g. the spec for s/cat:#2016-06-0820:46stathissiderisare those somewhere in the code already, or still unreleased?#2016-06-0820:47richhickeynot yet released#2016-06-0820:47stathissiderisbut that's the 2-stage generate you mentioned in the cognicast#2016-06-0820:47richhickeyI have to go through them with @alexmiller still#2016-06-0820:48richhickeyright, thatā€™s for testing mostly. Working from example data is interesting, but not possible in the general case#2016-06-0820:48richhickeytesting and parsing I should say#2016-06-0820:48richhickeyalthough now with unform, the toolchains get interesting#2016-06-0820:49stathissiderisI'm inspired by F#'s type providers... if they can do it to a useful extent, I think the idea has some mileage#2016-06-0820:50richhickeyif not regex-requiring sequences, not so bad. keysets mostly and leaf scalar types#2016-06-0820:51richhickeystill you wonā€™t be able to tell required from optional, etc#2016-06-0820:51stathissiderisyeah figuring out regexes would take some crazy sequence alignment algorithms (as in biology)#2016-06-0820:52stathissideriswell, if a key appears in only half the maps, we can assume it's optional... I'm thinking this could be a cool way to inspect data#2016-06-0820:55richhickeysounds like fun#2016-06-0820:56stathissideriswe'll see how far I can take it! Imagine applying this to functions while exercising an existing codebase to figure out signatures.#2016-06-0821:01nwjsmithAs I understand from the docs multi-spec is meant for tagged maps, but would be appropriate for specs that apply to either maps or sequences?#2016-06-0821:02nwjsmithAn example:
(defmulti query-form (fn [query] (if (map? query) :map :list)))

(defmethod query-form :map [_]
  (s/keys :req-un [::find] :opt-un [::with ::in ::where]))

(defmethod query-form :list [_]
  (s/cat :find (s/cat :find-kw #{:find} :spec ::find-spec)
         :with (s/? (s/cat :with-kw #{:with} :variables (s/+ ::variable)))
         :in (s/? (s/cat :in-kw #{:in} :inputs (s/+ ::input)))
         :where (s/? (s/cat :where-kw #{:where} :clauses (s/+ ::clause)))))

(s/def ::query
  (s/multi-spec query-form (fn [g _] g)))
#2016-06-0821:02nwjsmithWhat do I lose by dropping the re-tagging in multi-spec?#2016-06-0821:04richhickeyyou can do any discrimination you can with a multimethod - tag keys in maps is just an example. Also note that retag could be polymorphic#2016-06-0821:05sparkofreasonSaw an earlier comment that instrumenting protocol implementations doesn't always result in the spec being validated. Will this be supported at some point? Or does it even make sense, i.e. would it be "better" to spec a map with function specs instead of using protocols?#2016-06-0821:05richhickeymulti-spec is for data-dependent specs, generally#2016-06-0821:06richhickeyspec + protocols tbd. Certainly protocol behind API fn is good practice and supported (spec the API), but many people expose their protocols directly#2016-06-0821:26dryewois there a way to disallow non-mentioned keys? (s/valid? (s/keys :opt-un [::foo ::bar]) {:foo 1 :baz 2})#2016-06-0821:44nwjsmithAn and with a map-of spec and your keys spec will do it I think#2016-06-0821:45dryewomap-of is for homogenous maps#2016-06-0821:46nwjsmithYes, but you can use map-of's kpred to ensure the keys are the ones you accept#2016-06-0821:47dryewoah, you mean put a set there?#2016-06-0821:47nwjsmithyes#2016-06-0821:48dryewoindeed, it works#2016-06-0821:48dryewothanks#2016-06-0821:49richhickeyyou can use s/and with s/keys as well#2016-06-0821:50richhickey(s/and (s/keys ā€¦) some-further-constraints)#2016-06-0821:51richhickeymap-of not nearly as powerful, wonā€™t do per-key validation etc#2016-06-0821:52richhickeymap-of is for map-as-collection, keys for map-as-information-object#2016-06-0821:53dryewoI see#2016-06-0821:53dryewonow Iā€™m trying to support :opt in this case#2016-06-0821:53richhickeyyou should always question why you are limiting keys#2016-06-0821:54richhickeyjust creating brittleness#2016-06-0821:55dryewowell, Iā€™m just exploring the possibilities#2016-06-0821:55richhickeybut people keep asking for this, maybe weā€™ll have a :closed option to keys#2016-06-0821:55dryewothis works:
(s/valid? (s/and (s/keys :req-un [::foo] :opt-un [::bar])
                   (s/map-of #{:foo :bar} nil))
            {:foo 1})
#2016-06-0821:56richhickeymap-of samples, will not test every key, so a set not a good predicate#2016-06-0821:56dryewobut keys will#2016-06-0821:57seancorfieldA use case I can think of is where you might want to limit keys on a map you expose as a data structure outside your code boundary: for example something you convert to JSON to return from a web service or when you insert! into a SQL database (using clojure.java.jdbc for example, where additional columns would lead to an exception). But Iā€™d question whether those are places where you should rely on something like clojure.spec to catch the error (instead of a more explicit approach).#2016-06-0821:58seancorfieldWe have select-keys for that sort of scenario, right?#2016-06-0821:59richhickeyyeah#2016-06-0822:00richhickey@dryewo: keys is open, so will let other keys through, and map-of wonā€™t necessarily catch them, it just samples for type consistency, not particular values#2016-06-0822:01richhickeyyou could use #(every? #{:foo :bar} (keys %)) as the second pred#2016-06-0822:05dryewoI tried this: (s/valid? (s/map-of #{:foo :bar} nil) {:foo 1 :baz 2}) and figured that map-of limits possible keys to those that satisfy the set. But you say itā€™s something that can change in the future?#2016-06-0822:06dryewothe solution with every? looks quite ultimate#2016-06-0822:06dryewoI found *coll-check-limit* in the sources, that explains#2016-06-0822:07richhickeyneither coll-of nor map-of do exhaustive checking, nor conforming. We will make the docs clearer#2016-06-0822:07richhickeythey are ok for very large collections#2016-06-0822:07dryewothanks, that makes sense#2016-06-0822:07ghadii don't want closed sets of keys without a compelling example#2016-06-0822:08ghadihaven't seen one yet#2016-06-0822:08richhickeyenough people seem to want exhaustive conforming version so weā€™ll likely have both#2016-06-0822:08richhickey@ghadi: well, thatā€™s why open is how it is, I agree šŸ™‚#2016-06-0822:08gfredericksthe only thing that comes to mind for me is the principle of being conservative about the data you return from an API#2016-06-0822:09richhickeyyou just engender brittle code on the other end#2016-06-0822:09gfredericksoh if the closedness is publicized then yeah#2016-06-0822:09gfredericksI was just imagining using it for checking that you aren't accidentally leaking extra stuff#2016-06-0822:10richhickeywell as @seancorfield said, you can call select-keys#2016-06-0822:10gfredericksbut I suppose specs are intended to be publicized#2016-06-0822:10richhickeyright#2016-06-0822:10gfrederickscool, I'm on board#2016-06-0822:11richhickeythereā€™s a bit in the podcast interview about change, while maybe not made clear yet, this kind of thing is an important part of the design#2016-06-0822:12gfredericksbecause it means adding new keys is backwards compatible?#2016-06-0822:13richhickeythere are separate semantics for accept/return, but they align with co-contravariance, and the keyset and regex system will let us say when things accept/return compatible things#2016-06-0822:13gfrederickshow will predicate equality work in that case? check that the forms are the same?#2016-06-0822:14richhickeyi.e. :req less on what you accept is compatible, :req (essentially guaranteeing) more on return compatible, etc#2016-06-0822:15richhickeyregexes can be tested to see that they new accepts everything old does (accept) and old accepts everything new does (return)#2016-06-0822:15richhickeybase predicates presumed immutable, yes forms#2016-06-0822:16richhickeywhat I hope people find is that at the fine granularity of spec, things rarely change, and if the do they must compatibly#2016-06-0822:18richhickeyand it could get fancier (generative testing to do the non-spec preds)#2016-06-0822:18gfredericksrichhickey: I asked on the ML about tactics for moving libraries forward; in the podcast you mentioned something about adding numbers at the end of function names, do you imagine that being a sort of last resort?#2016-06-0822:18gfredericksstu also seemed to mention creating entirely new namespaces, so I'm curious how common you imagine each of those tactics being#2016-06-0822:20richhickeyhow often do you change the sigs of public fns? I think for algorithmic things rarely and for informational things only in spec-compatible ways. Itā€™s the breaking change that;s the last resort, the numbering of fn names is just being honest with your users#2016-06-0822:22richhickeyThe problem with libraries is that a lib has 50 fns and when you add 2 new ones it gets a different ā€˜versionā€™ and no one knows whatā€™s different. What they actually depend upon are the individual fns that they call, not the lib. Stable specs will show the true granularity of change (usually additive)#2016-06-0822:24richhickeyIā€™d much rather deal with foo-2 (which btw can co-exist with old foo) than ā€˜version 2ā€™ of foo.#2016-06-0822:24richhickeyI can move to foo-2 when I please#2016-06-0822:24gfredericksyeah, I definitely appreciate that#2016-06-0822:25gfredericksthere are a handful of minor warts in the test.check API that feel hard to improve without breaking#2016-06-0822:25gfredericksfor some value of "breaking"#2016-06-0822:26gfredericksat least if "the API is surprising and hard to sort through" is part of the problem, as then adding a foo-2 might maintain those downsides#2016-06-0822:26gfredericksso I suppose I'm having to accept not being able to "fix" things in that way#2016-06-0822:27richhickeyfor big changes you can also do test.check2#2016-06-0822:27gfredericksas the lib name, or namespace name?#2016-06-0822:28richhickeylib/ns same difference#2016-06-0822:28richhickeythen people can move when they please, specs will be in different nses, so they havenā€™t changed#2016-06-0822:29ghaditest.check2.11#2016-06-0822:30gfredericksyep#2016-06-0822:31gfredericksrichhickey: thanks#2016-06-0822:31richhickey@gfredericks: I would like to be able to convey explanation data through such-that somehow#2016-06-0822:31gfredericksrichhickey: alex mentioned something about having a fancier such-that to make the generator for s/and more robust, do you know if that's a different issue from what you just mentioned?#2016-06-0822:32gfredericks(just so I can keep track)#2016-06-0822:32richhickeymaybe a (compatible!) extra arg which is a fn to call on failing val to get exception text)#2016-06-0822:33richhickeynot sure if thatā€™s what @alexmiller was talking about#2016-06-0822:33gfrederickswhat he said exactly was: "I think there is some desire to have a better (programmatic) way to have a conversation about filtering with such-that than just boolean yes/no"#2016-06-0822:34richhickeyyes, so right now the pred fails and test.check canā€™t say anything useful because the pred is boolean#2016-06-0822:34ghadire: open/closed keysets -- I haven't had the need to ensure a closed set, but when working with an API not under my control, it is useful to detect added information#2016-06-0822:34richhickeybut spec knows what the conditions were#2016-06-0822:35gfredericksrichhickey: okay so he was talking about the same thing? I interpreted his statement to imply some super-fancy way of giving such-that hints about other things to try#2016-06-0822:35richhickey@ghadi: yes, thatā€™s different, and has been requested - (keys-not-in spec map)#2016-06-0822:36richhickey@gfredericks: nope, just better failure reporting, because the failure to gen could be in a nested spec weā€™d like to at least tell the user which one#2016-06-0822:38gfredericksrichhickey: is this a fair summary? http://dev.clojure.org/jira/browse/TCHECK-107#2016-06-0822:44ghadicould something along the lines of keys-not-in work at any path? use case is detecting a fully compatible change made on an external API#2016-06-0822:46gfredericksrichhickey: after thinking about it a bit, I'm imagining a function that's expected to return a throwable, that way you can customize the error text and the ex-data#2016-06-0822:47gfredericksI guess that means the stack trace starts in a weird place though, which I was trying to avoid :/#2016-06-0822:54kendall.buchananBig thanks to everyone who contributed to clojure.spec. Spent two days building an internal library on it, and itā€™s, so, so, so, so, so, soā€¦ enjoyable.#2016-06-0823:46bbloom@ghadi: another use case i just ran into in a (non-clojure) project: warning on configuration errors#2016-06-0823:47bbloomhad a config.toml file and somebody had a rogue [Section] and a Key = true line was getting put in to the wrong section, so the config setting wasnā€™t taking hold#2016-06-0823:47bbloomi used a feature of our toml parser to print all the unrecognized keys to the logs#2016-06-0902:53arohner@bbloom: @ghadi yes, figwheel uses that idea to great effect in config file validation#2016-06-0902:53arohnerā€œunrecognized key :foo under :bar, you probably want it in :baz#2016-06-0907:02Oliver GeorgeHello. My CLJS apps use a lot of string keys. Primarily for performance reasons, transforming JSON has a cost. I went looking for the equivalent of the :strs feature of map destructuring, perhaps something like :req-strs and :opt-strs could be added to s/keys.#2016-06-0907:02Oliver GeorgeFor now the workaround seems to be using a conformer...#2016-06-0907:02Oliver George
(def strs->keys 
  "Turn map with string keys into map of keyword keys.  Without this we can't use s/keys."
  (s/conformer #(reduce-kv (fn [m k v] (assoc m (keyword k) v)) {} %)))
(s/conform
  (s/and strs->keys
         (s/keys :req-un [::id ::options]))
  {"id" :id
   "options" [{:id 1} {}]})
#2016-06-0907:03Oliver GeorgeAm I missing anything?#2016-06-0907:35hiredman
user=> (s/valid? (s/spec #(contains? % "foo")) {"foo" 1})
true
user=> 
#2016-06-0907:35hiredmanit depends on what you are looking to get out of it, I guess#2016-06-0912:56dominicm@hiredman: You lose thikngs like path lookups doing that. I think that's the key feature that goes missing. Spec has a bit of a balancing act to perform making sure that via/paths are added. s/keys is a bit complicated in implementation, but I would be thinking that a similar-but-different implementation should surface for string keywords.#2016-06-0913:34gfredericksI've thought about string keys some too w.r.t. https://github.com/gfredericks/schema-bijections#2016-06-0913:35gfredericksI figured it could be handled with a bijection system that doesn't produce a spec on the other side, but that doesn't help if you're avoiding transformations for performance#2016-06-0914:42rickmoynihanHey... I've been very pleased with clojure.spec, which looks absolutely fantastic, and a nice step up from plumatic.schema - and I am fully intending to play with it properly (I've only really had time to read the announcement, listen to Rich on the cognicast, and looked at a few examples)... Anyway, one thing I'd really like to do is understand how this fits with JSON schema... Now, clojure.spec is clearly far superior to JSON schema, but it's not used by the javascript community etc... So I was wondering if anyone has thought about how one might be able to build a JSON schema from a clojure.spec. As spec is way more expressive, I'm guessing that to do this you'd have to stick to a subset of spec (and I guess you could probably use clojure.spec to specify that subset)... The basic idea would be to write a super-spec in spec; and output JSON schemas to interoperate with other tooling/libraries in other languages etc... e.g. swagger clients etc. If this is possible from within clojure you'd then be able reuse that subset of spec (which generated JSON schemas), and layer ontop the additional expressivity and specs you'd want clojure side.#2016-06-0915:38gfredericksrickmoynihan: the library I linked to just above has a lot to do with that problem, but it works with plumatic/schema. I've been pondering how best to translate it to clojure.spec#2016-06-0915:38gfredericksthe idea is it uses a set of transformations to turn an internal schema (the nice one) into a different schema (the coarser json-style schema in this case), with functions generated to transform data structures between the two#2016-06-0915:44rickmoynihangfredericks: interesting#2016-06-0915:45gfredericksit's still a bit half-baked, but I used it a bunch on a previous project#2016-06-0915:46gfredericksin particular it doesn't do a good job of distinguishing between bijections and non-bijections (coercions, projections, whatever you might call them); I'm thinking ou could do that interestingly using test.check generators#2016-06-0915:49gfrederickse.g., if my JSON API can accept {"unitCount":42} as well as {"unitCount":42.0}, you might specify that in the schema, but say that the first one is preferred, so that the transformation function always uses that, but could also have a generator that would take {"unitCount" 42} and generate sometimes {"unitCount" 42} and other times {"unitCount" 42.0M}#2016-06-0915:49gfredericksso it would give you a way to test flexible APIs in a robust way#2016-06-0915:51gfredericksunrelated: if I'm reading the clojure.spec source correctly, it ought to be safe to add specs to test.check (and to clojure.core) without the danger of an infinite loop during instrumentation -- does that sound right?#2016-06-0915:51rickmoynihandoes spec support coercion? I though rich said it wasn't part of its job#2016-06-0915:51gfredericksspecs can refer to themselves via the keyword#2016-06-0915:52gfredericks(s/def ::tree (s/or :leaf #{:leaf} :children (s/tuple ::tree ::tree)))#2016-06-0915:53gfredericksexample value: [[:leaf [:leaf :leaf]] [[:leaf [:leaf :leaf]] [:leaf :leaf]]]#2016-06-0915:53rickmoynihanyeah but I didn't think spec could coerce a value from "foo" into :foo - or can it?#2016-06-0915:53gfredericksoh geez sorry#2016-06-0915:53gfredericksI misread "coercion" as "recursion"#2016-06-0915:53rickmoynihanlol šŸ™‚#2016-06-0915:53gfredericks(and thus thought your question was unrelated to what I was previously talking about)#2016-06-0915:54gfredericksokay right afaik it doesn't support coercion#2016-06-0915:54gfredericksI'm not sure if the "external spec" idea would work or not#2016-06-0915:54gfrederickse.g., with json you'd want to have string keys, which isn't natural#2016-06-0915:54rickmoynihantbh I'm not too worried about the coercion case... I'd just like to write clojure.specs (even in a limited subset of spec) and output JSON schemas from them#2016-06-0915:55gfredericksI think that by itself would probably be easier, if you only support a subset of schemas#2016-06-0915:55rickmoynihanwell I'm sure that would be possible too#2016-06-0915:56rickmoynihanit doesn't need to be perfect - I'd just like to define a service in clojure, and emit a JSON schema that's good enough to catch errors between service boundaries - where client services might be implemented in javascript/ruby etc... And ideally also target swagger, to give client developers access to some swagger tooling / swagger-docs on the other side. (Though that's just an extra nice to have)#2016-06-0916:06gfredericksI'm trying to write (for fun) a variant of defn where each arg can have a spec and you can have multiple bodies with the same arity as long as the specs are different#2016-06-0916:06gfredericksi.e., you can overload on spec, not just arity#2016-06-0916:10rickmoynihaninteresting idea#2016-06-0916:10rickmoynihanI really like multispec#2016-06-0916:11rickmoynihanI'm guessing you just generate those when the arities are the same?#2016-06-0916:11gfredericksI was going to start with just one body that takes varargs and parses them#2016-06-0916:15gfredericksincidentally I have to parse arglists for this and spec is really good at that :D#2016-06-0916:15ikitommi@rickmoynihan: there is a open issue in ring-swagger to add support for spec, haven't had to to investigate yet. (Currently supports the Plumatic Schema). Could split the lib into separate submodules for the different sources models.#2016-06-0916:17rickmoynihanikitommi: does ring-swagger include use a separate library for JSON schema; or bake its own one? I'm no swagger expert - but I'd assumed that Swagger included all of JSON schema...#2016-06-0916:18rickmoynihanI just think it'd be most useful to target JSON schema directly rather than drag in swagger/ring etc too#2016-06-0916:19rickmoynihanhttps://github.com/metosin/ring-swagger/issues/95#2016-06-0916:23rickmoynihanWe're in the fortunate situation of not having a lot of legacy plumatic/schemas - mainly because we only started using schemas a month or two before spec's announcement... so I'm not too concerned about plumatic support for legacy reasons... But I'm curious whether anyone think's there's a future for plumatic/schema beyond just supporting legacy... i.e. does it have uses that clojure.spec can't support so well?#2016-06-0916:25gfredericksthere's nothing I used it for that I would keep using it or
#2016-06-0916:25gfrederickss/or/for/#2016-06-0916:26gfrederickseven if there's something minor it could do better I feel like there's more leverage to be had using The One True Tool#2016-06-0916:26gfredericksI'll make a clojure.spec utility library if I have to though, for specialized stuff that clojure.spec decides not to support directly#2016-06-0916:27gfrederickse.g., maybe for the super succinct map syntax plumatic/schema has for defining map schemas#2016-06-0916:27rickmoynihanthe only things I think plumatic/schema has over clojure.spec right now are: 1. it's perhaps a little closer to json schema - and possibly less work to bridge into json schema 2. no automated coercions#2016-06-0916:28wilkerlucioI'm wondering here about sort of generic specs, I'm thinking on the channel case, I can spec the return of a function to return a channel, maybe would be nice to be able to annotate also the expected value that will come from the channel, do you people have any thoughts on how to deal with stuff like this?#2016-06-0916:45richhickey@gfredericks: having the fn return data which you throw is better - (s/keys :opt-un [:ex-message :ex-data])?#2016-06-0917:34ikitommi@rickmoynihan: the plumatic->json schema is done within the lib (the json-schema ns, based on protocol & supporting multimethod). Swagger (OpenAPI nowadays) only supports only a "pragmaticā€ subset of JSON Schema as itā€™s original target was the OO-languages like Java - has client code generators for those. Things like oneOf or anyOf are not supported - the requests to add those are over 2y old now. I think having a separate pure spec<->json schema would be awesome! Having it work with swagger requires some extra work, happy to do/help with that.#2016-06-0918:30gfredericksrichhickey: yeah that's essentially what I was thinking; can you think of any args that should be passed? if we make it a one-arg function that's passed an empty map that would be better for backwards-compatibly adding more things later :) I'm wondering because if there's no need for any args then it's not clear why it even needs to be a function and not just passing the data directly#2016-06-0918:52gfredericksthis fancy defn is fun; I suppose it's a lot like core.match#2016-06-0918:56gfredericksuh oh I just made it stack overflow#2016-06-0919:03gfredericksokay here it is -- defn+spec, where each arg can be decorated with a spec and you can overload a function by spec: https://gist.github.com/gfredericks/e4a7eafe5dcf1f4feb21ebbc04b6f302#file-defn-spec-clj-L73#2016-06-0919:05gfredericksthere's a note in there about a stack overflow that I haven't tried to debug, that happens when I add a spec to the defn+spec macro itself#2016-06-0919:08fxposterhi everyone I have 2 questions about clojure.spec, which are not that obvious from the beginning: 1. is it okay to use predicates that connect to external resources for validation? ie: I have a tree of "paths" and want to validate that it matches the filesystem or posts on some website? I don't see any techncal problems with that, but maybe there are other solutions for that? 2. for example, I have 2 data structures: new {"application": {"branches": {"release": "1.0", "snapshot": "1.1"}}} and old {"application": {"branches": {"release": "0.9", "snapshot": "1.0ā€}}} and I want to actually check that the new value is a "valid update" of the old one, based on some constraints (for example: "new value contains at least all branches from the previous one" and "all versions in new value are greater than the same ones in the old one"). is there a way to at least partially express that in clojure.spec and get validation + error reporting? Thanks#2016-06-0919:12gfredericksI think the predicates are supposed to be pure functions#2016-06-0919:12gfredericksI couldn't say where the first place you would run into trouble would be though#2016-06-0920:28akielCan someone explain what I found in my snippet?#2016-06-0920:36benzapdoes replacing sp/alt with sp/or make it work?#2016-06-0920:37akielYes!#2016-06-0920:37benzaphaha, i've had the same issue#2016-06-0920:38benzapsomeone worked out the differences. It has something to do with how sp/alt is used in regex, and sp/or is used otherwise.#2016-06-0920:38akielSo sp/alt on something which is not a regexp doesnā€™t work? Ok I see...#2016-06-0920:38benzapSimply put yes. Someone else on here had a really good explanation, but I didn't completely follow#2016-06-0920:39benzapHe was wondering why sp/alt and sp/or were so similar, and produced the same results in some contexts#2016-06-0920:39benzaphad to do with the regex distinction#2016-06-0920:40akielso maybe we need spec for spec - I mean itā€™s all macros - everything can happen#2016-06-0920:41benzaphaha, i'm curious to see how much spec is used šŸ™‚#2016-06-0920:41benzapI wonder if they'll use clojure.spec on all of the clojure.core to try and get away from the java stracktraces#2016-06-0920:42akielcat and alt says: ā€œreturns a regexā€, and says: "returns a specā€ - so maybe a spec is not a regex šŸ™‚#2016-06-0920:42benzapah ok#2016-06-0920:42akieltype systems would help rant#2016-06-0920:43benzapI was talking to someone about clojure.spec on freenode#programming, and he said clojure.spec is like contracts in racket#2016-06-0920:44benzapthe idea being that they're supposed to be more powerful, since you can apply predicates#2016-06-0920:45akielyes spec is more powerful as a type system, because you can inspect actual values at runtime#2016-06-0920:45benzapya, the whole instrumentation is pretty neat#2016-06-0920:45benzapi've used it, but it's rather slow. I need to start using it per namespace#2016-06-0920:46akieljust use it only in dev and test#2016-06-0920:46benzapI have been, but it's even slow then. Maybe 20 times slower#2016-06-0920:46benzapmakes testing rather slow#2016-06-0920:46benzapIt isn't an issue if I run it once in a while, so i'll throw down instrumentation every once in a while#2016-06-0920:47benzapBut I wonder if there's any room for improvement in performance, it would make it very appealing to just leave instrumentation enabled even in production#2016-06-0920:48benzapThere's a reddit post comparing different schema/data validation libraries. No one has commented on it yet, and I wish someone more knowledgable would#2016-06-0920:48benzaphttps://www.reddit.com/r/Clojure/comments/4mxqcy/spec_performance_comparison/?ref=share&amp;ref_source=link#2016-06-0920:48akielThan I think, you apply spec to inner loop things. I would apply it only to my public API.#2016-06-0920:48benzapya, that makes sense#2016-06-0920:49benzaplike, only validate data that comes in from an external source for production?#2016-06-0920:50akielIn production I would check data coming over wire directly without instrumentation, just in normal code.#2016-06-0920:51akielI do this with Schema already.#2016-06-0920:51benzapI've been using the clojurescript version of clojure.spec, cljs.spec#2016-06-0920:51benzapoh nice#2016-06-0920:51benzapyeah, I converted my project i've been working on from schema to spec#2016-06-0920:51benzapdidn't take that long, but it did require quite a bit of refactoring#2016-06-0920:52benzapI'm still not sure where I should put the specs. I kind of placed them at the beginning of the files, and would s/fdef after each function#2016-06-0920:53benzapI suppose that's good enough, I don't think i've tested s/fdef when the function hasn't been declared yet, so I don't think I could place them in another file?#2016-06-0920:53akielIā€™m not completely sure either. But I have seen putting s/fdef before each funtion often.#2016-06-0920:53benzapoh, before?#2016-06-0920:53benzaphmm#2016-06-0920:54akielyes it is possible#2016-06-0920:54benzapthat's good to know#2016-06-0920:54akielits like contract first and than the implementation#2016-06-0920:54benzapthat makes sense#2016-06-0920:54benzapi'll have to consider moving my stuff into a separate folder#2016-06-0920:55benzapit's hard to get used to, really. I considered it an eye-sore at first#2016-06-0920:55benzapdefinitely useful though, I caught a lot of bugs early#2016-06-0920:56akielopposite to schema, the annotations are not inline to the function - so the possibilities are broader - like in core.typed I think - never used it#2016-06-0920:57benzapYeah, that would explain my confusion. At first, I had a hard time figuring out how to convert from schema#2016-06-0920:58benzapmade a lot more sense once I got started#2016-06-0920:58benzapwhen I used schema, I had originally defined the structures in a separate file, sortof like how you would define a jsonschema#2016-06-0920:58arohner@benzap: itā€™s probably slower because itā€™s doing generative testing at runtime. Hopefully that becomes a config setting#2016-06-0921:00benzap@arohner: that's interesting, I thought it was trying to conform each function with respect to it's defined spec. If it's also throwing in generative testing, I can see that causing some performance issues#2016-06-0921:01arohnerwell, fspec does. checking fdef now#2016-06-0921:02benzapa somewhat related topic, there's a library in clojure called specter, which compiles itself to increase performance#2016-06-0921:03benzapI wonder if the same concept could be applied to clojure.spec, where you could pre-compile the validator for your functions#2016-06-0921:19fentonwhat would a spec look like that does the following: if a map has one key, it should have another key#2016-06-0921:19bbrinckAnd in plumatic schema, you can create validators ahead of time for performance.#2016-06-0921:24fentonoops wrong description of problem. if a key in a map has a certain value, then another key should be present. writing specs can be a challeng...like a whole nother language! šŸ™‚#2016-06-0921:31angusiguess@fenton That is likely a multispec#2016-06-0921:31angusiguessWhere you write a multimethod looking for the first key, and that dispatches another spec to check for the second.#2016-06-0921:32fenton@angusiguess: ok, will look into that. thx.#2016-06-0921:32angusiguessšŸ˜…#2016-06-0921:58arohnercould also be a simple s/or#2016-06-0921:58arohner(s/or map? (s/and (s/keys [::foo ::bar]))#2016-06-0921:58arohnerdepends on how complex it needs to be#2016-06-0922:01danielcompton@rickmoynihan: we use coercions which are extremely handy#2016-06-0922:03danielcomptonIf spec (or surrounding tooling) doesnā€™t support automated coercions, then weā€™ll need to keep schemas for our boundary interfaces between webapp and RethinkDB#2016-06-0923:09rickmoynihandanielcompton: Yeah - I've used coercions before too - and I don't dispute their utility at boundaries... I guess you could easily build a specialised coercion library on top of spec though... don't know enough yet how you might do that... Regardless I much prefer what I've seen of spec to schema; and don't think coercions are enough of a feature on their own to either not use spec, or use schema as well as spec... I'd definitely much prefer something that worked with spec#2016-06-1002:23sparkofreasonIs there a straightforward to define a function spec and reuse with multiple fdef's?#2016-06-1007:43mpenet@rickmoynihan @danielcompton : same here, also free type hinting, custom explain messages, performance and the list goes on. So far we have no reason to migrate really, we're waiting to see where clj.spec is going, it's probably a bit too early#2016-06-1013:51ikitommiBesides coercions, my top feature request to spec: helper-fn to create a spec from a vanilla clojure (function) var. It would understand the Clojure destructuring syntax. Something like this:
(require '[clojure.spec :as s])

(s/def ::age integer?)
(s/def ::fullname string?)
(s/def ::role keyword?)

(defn doit [{:keys [::fullname ::age ::role] :or {:boss ::role}}] [fullname age role])

(-> #'doit s/extract-spec s/describe)
; => (keys :req [:user/age :user/fullname] :opt [:user/role])
* the responsibility to extract (and use) the specs would be on the user - I would use these on the web-tier to auto-extract docs & do coercion in the web-api tier with our libs * would not add new meta-data to vars (arguments are already in :arglists) * no need to describe the shape of the data twice (both for the function arguments & for it's spec) * (optionally the :ret and :fn could be read from the Var metadata too) Thoughts?
#2016-06-1014:01angusiguessI think this is rad, my only real question is why might we rely on spec to do it?#2016-06-1014:01angusiguessThere are pretty good hooks to pull validation information out of a defined spec and an alternate defn like this should be a macro.#2016-06-1014:02angusiguessI don't work on spec so this is grain of salt stuff, but it seems like the opinion of spec is that people could conceivably bring their own sugar but under the hood there's a common language for validation.#2016-06-1014:09Alex Miller (Clojure team)@ikitommi: you could build that from what exists now. since it wouldn't generically apply, I don't think we would do that as part of core or anything. One thing that would help is a spec for destructuring, which I have and which will be released at some point in some form (details TBD still)#2016-06-1014:16ikitommi@alexmiller: spec for destucturing sounds cool. Did a dummy version of the extractor, will play more with it. Are there any caveats in playing with :arglists? #2016-06-1014:18Alex Miller (Clojure team)there are a few cases where people have abused it a bit in what was put in it (data.generators is one that comes to mind) but generally should be fine#2016-06-1014:19Alex Miller (Clojure team)I think I would also consider allowing overrides via lookup in the registry - s/fdef registers stuff there under the fn symbol and those can be obtained via s/fn-specs#2016-06-1014:19Alex Miller (Clojure team)so you have an existing registry for overrides of things you couldn't build automatically#2016-06-1014:21Alex Miller (Clojure team)also note that CLJ-1919 will add a new syntax for namespaced key destructuring.#2016-06-1014:21Alex Miller (Clojure team)your example there is not syntactically correct btw - the keys of :or should always be unqualified symbols (matching the bindings that are created). there are some bugs in this area in current Clojure that will be fixed in CLJ-1919.#2016-06-1014:22Alex Miller (Clojure team)so that is, what you have there probably works now, but by accident not intent, and will change#2016-06-1014:24ikitommiuh, copy-paste error in the code. But thanks! will check out your pointers.#2016-06-1015:10wilkerluciowhat's the correct way to express to an fspec that a function takes no arguments?#2016-06-1016:13gfredericks(s/cat) I'd guess#2016-06-1016:13gfredericksmaybe #{()} would work too#2016-06-1016:16gfredericksalexmiller: there's no reason not to add specs to test.check is there?#2016-06-1016:17gfredericksassuming it accounts for older clojures#2016-06-1016:18gfredericksI guess this question is a superset of seancorfield's question on the ML, but also about test.check in particular since it's used in clojure.spec#2016-06-1016:24seancorfieldI haven't moved forward with that since the first cut. Want to see more discussion on the ML first. #2016-06-1016:24seancorfield(sorry if I don't follow up for a few hours -- doors closing en route for a cat show!)#2016-06-1016:28Alex Miller (Clojure team)None other than that it then requires Clojure 1.9#2016-06-1016:28Alex Miller (Clojure team)Which requires test.check#2016-06-1016:46gfredericksokay, cool; I'll probably do a separate .specs namespace like sean did#2016-06-1017:11jcfAnyone tried writing specs for stateful objects like database connections, or a Datomic database?#2016-06-1017:14jcfI can't think of a nice way to specify a function takes a db, and some other args. To generate a db I need a database connection that isn't available when I define my specs.#2016-06-1017:16jcfImagine a trivial example like this:
(s/fdef load-entity
  :args (s/cat :db ::d/db :tx ::entity-tx)
  :ret ::entity)

(defn load-entity
  [db tx]
  (d/entity db [:entity/id (:entity/id tx)]))
#2016-06-1017:22jcfI can't generate Datomic entities either. I need a DB, which needs a connection, which needs a URI.#2016-06-1017:24jcfIs there a way to say a spec can't be generated automatically so I can test other specs in this namespace maybe?#2016-06-1017:27arohnerisnā€™t your DB predicate just #(instance? datomic.whatever.Db %)?#2016-06-1017:27pheuter@jcf: i just went through that exercise of creating a Datomic Db#2016-06-1017:27pheuter
(s/def ::db
  (s/with-gen #(instance? datomic.db.Db %)
    (fn []
      (gen/fmap (fn [facts] (-> (helpers/empty-db)
                               (helpers/transact facts)))
                entities-generator))))
#2016-06-1017:27jcfI need a generator to go with that spec @arohner.#2016-06-1017:28pheuter
(s/def ::db
  (s/with-gen #(instance? datomic.db.Db %)
    (fn []
      (gen/fmap (fn [facts] (-> (helpers/empty-db)
                               (helpers/transact facts)))
                entities-generator))))
#2016-06-1017:28jcfI've got basic predicate fns like these:
(defn db?
  [x]
  (instance? datomic.Database x))

(defn entity?
  [x]
  (instance? datomic.Entity x))
#2016-06-1017:28jcf@pheuter: what's facts, and what does (helpers/empty-db) look like?#2016-06-1017:29pheuterwhere entities is a vector-distinct-by of :db/id values#2016-06-1017:30pheuterempty-db just creates an empty datomic db that has been primed with a schema#2016-06-1017:30jcfSo you've got some hardcoded Datomic URI or something?#2016-06-1017:30jcfYou must have global state floating around, right?#2016-06-1017:31pheuterNo, we just generate random URIs, we use in-memory databases for development / testing#2016-06-1017:32jcfThat is global state. My Datomic connection is managed with components, and they're stopped/started around tests.#2016-06-1017:33jcfI could have two Datomic databases with separate connections. That wouldn't be possible with (helpers/empty-db).#2016-06-1017:33pheuteryes, we use mount to start and stop our connections as well#2016-06-1017:34pheuterwe dont use mount for testing though, and thatā€™s when we generate db values#2016-06-1017:34pheuteractually, we do use mount for certain state, and rely on dynamic values using test/use-fixtures#2016-06-1017:35pheuteryour predicates look fine, but you wonā€™t be able to use them for generating out-of-the-box, will need to use s/with-gen#2016-06-1017:56dominicm https://clojurians.slack.com/archives/clojure-spec/p1465578891000846 I've been trying to figure this out also. My solutions have involved macros and sideband data. Not elegant at all. I also ignored generators. #2016-06-1018:05gfredericksseancorfield: were you thinking of having a conditional require in the .jdbc namespace? otherwise you'd have the problem of up-to-date users having to opt-in to the specs#2016-06-1018:05gfrederickswhen there are use cases for the specs besides explicit testing, e.g. clojure.repl/doc#2016-06-1019:34jcfIs there a way to merge two s/keys specs?#2016-06-1019:34jcfI must be missing something. Back to the manual!#2016-06-1019:35donaldballIā€™ve been idly thinking about the problem of db args as well, though in the sql context. The problem seems the same for datomic and jdbc though: a spec saying the db arg is e.g. a jdbc connection isnā€™t sufficient. You really want a spec that says this value is a jdbc connection to a database with at least a certain schema and maybe even a certain set of entities.#2016-06-1019:35jcfOh wait, just me being stupid apparently.#2016-06-1019:36jcf@donaldball: I'm using this at the mo:
(defn- with-datomic
  [f]
  (let [running (-> (config/read-config :test)
                    (assoc :uri (str "datomic:mem://" (UUID/randomUUID)))
                    map->Datomic
                    component/start)]
    (try
      (f running)
      (finally
        (component/stop running)))))

(defn entity?
  [x]
  (instance? datomic.Entity x))

(s/def ::d/entity
  (s/with-gen
    entity?
    (fn [] (with-datomic (fn [{:keys [conn]}]
                           (gen/fmap
                            #(d/entity (d/db conn) %)
                            gen/int))))))
#2016-06-1019:37jcfDon't love it if I'm honest. Creating a new connection for every test is pretty inefficient, but it works.#2016-06-1019:38jcfIf you've not used Component or Datomic that's probably meaningless.#2016-06-1021:02gfredericksoh woah#2016-06-1021:04gfredericksI hadn't thought about this generator setup encouraging people to use stateful resources in their generators#2016-06-1021:12dominicm@gfredericks: I'd say that the encouragement and bias towards namespace level hoisting (with s/def) takes away our ability to lexically scope and generally pass around explicit arguments. Diving too deeply into that statement takes you to mount vs component.#2016-06-1021:15gfredericksdominicm: the way I've used generators in the past is purely data-driven, so there's not even component-like stuff until the test starts running; but if you have a spec that's explicitly for a stateful thing, then you can't write a generator for it that way#2016-06-1021:16gfredericksmy gut would be to try to keep doing the data-driven thing, and so not use generators for specs that describe stateful things#2016-06-1021:16gfredericksnot sure how easy that is with datomic#2016-06-1021:16gfredericksa datomic entity is a map of attributes, is that right?#2016-06-1021:17gfredericksif that's the case you could make most of your specs just expect maps#2016-06-1021:17gfredericksand could do low-level testing with maps instead of entities#2016-06-1021:24dominicm@gfredericks: It's definitely difficult to manage with generators. The discussion (in my opinion) transcends just testing. Doing explicit s/explain-data on a runtime database is also a use-case.#2016-06-1021:24gfredericksthat should work with map specs though I would think?#2016-06-1021:29dominicm(s/explain-data {:email " The unique email checker needs a database to make it's check. If I make it part of the map data (assoc m ::db (d/db conn)), then when I check, the returned data gives me an incorrect path to the error position.#2016-06-1021:30dominicmI have just got a macro working, which would take the error generated, and readjust the path for you. But it's still somewhat unnatural.#2016-06-1021:32gfredericksso your spec is making database queries?#2016-06-1021:33ghadithat is a bad idea#2016-06-1021:36gfredericksI think specs should be pure functions#2016-06-1021:43ghadiabsolutely#2016-06-1021:44gfrederickssomething that's not a pure function can be a plain ole test :)#2016-06-1022:14eggsyntaxI need to write a script to generate some Datomic seed data, and I'm experimenting with using spec to do so. Two questions: 1) One part of the seed data I need to write is the key/val that'll generate a temporary db/id. If I were hand writing the seed data, it would look like {:db/id #db/id[:db.part/user -1015948] ... }. Can anyone help me understand how I would go about creating a spec for that? 2) ideally, it'd be nice to just spec that part of the schema, and then somehow use that to generate the seed data, but I'm not quite sure how go about that. Any hints?#2016-06-1022:15eggsyntaxIn other words, how can I spec a tagged literal like that?#2016-06-1022:30eggsyntaxGot it.
(defn db-id? [v] (instance? datomic.db.DbId v))
(s/def ::db-id db-id?)
#2016-06-1022:35eggsyntax(or at least that's the spec for the tagged literal itself. I'm not quite sure how to get from that to a spec that'll produce seed data as above. I guess I'll need to write a generator for it, but it's gonna take some experimentation for sure.#2016-06-1022:37hiredmanusing spec just to generate data seems super weird, why wouldn't you use the generators from test.check or https://github.com/clojure/data.generators directly?#2016-06-1022:39eggsyntaxWell, if I can figure out the relationship between a spec for the schema and a spec for the seed data, I can use the spec to do validation as well as generating seed data, and maybe find other uses for it as well.#2016-06-1022:40eggsyntaxAnd it's also partly a clojure.spec learning exercise for myself šŸ™‚#2016-06-1111:05borkdudeDoes spec offer anything in relation to coercing like Schema does?#2016-06-1111:36Alex Miller (Clojure team)Not really. It does have conformers but their intent is a bit different.#2016-06-1111:38Alex Miller (Clojure team)@rymndhng: this is the intended behavior - map-of samples its values for performance reasons so doesn't conform everything. This been a common question though and something will probably be added for it.#2016-06-1208:02patrkrisHi Everyone. How do you name aggregate stuff with namespaced keywords? Say I have a map that would look like this without namespaced keywords
{:first-name "John"
 :last-name "Doe"
 :address
  {:street "Example Street"
   :street-number "413"
   :city "Example city"}}
The keys at the first level might be namespaced like :customer/first-name and :customer/last-name, but how would I name the keys inside the address map?
#2016-06-1208:31hiredman
{:customer/first-name "John"
 :customer/last-name "Doe"
 :customer/street "Example Street"
 :customer/street-number "413"
 :customer/city "Example city"
 :customer/address-fields #{:customer/street
                            :customer/street-number
                            :customer/city}}
#2016-06-1208:40patrkris@hiredman: thanks. when would you say it's appropriate to create a "child namespace", e.g. :customer.address/*?#2016-06-1214:27sanderWhat is more idiomatic,
(ns work.invoice
  (:require [clojure.spec :as s]))
(s/def ::invoice (s/keys :req [::number ::date ::amount]))
or
(ns work.core
  (:require [clojure.spec :as s]
            [work.invoice :as i))
(s/def ::invoice (s/keys :req [::i/number ::i/date ::i/amount]))
? The first is less typing work, but I don't really like :work.invoice/invoice as a spec name. Maybe this?
(ns work.invoice
  (:require [clojure.spec :as s]))
(s/def :work/invoice (s/keys :req [::number ::date ::amount]))
#2016-06-1217:06danstoneDoes anyone know of a way to compose key sets? e.g something like (s/merge ::foo ::bar) edit: and works for validation, but does not yield a working generator...#2016-06-1217:10t3chnoboyHi! Iā€™m trying to validate a javascript class and not sure how to make it work:
(s/def ::transport (s/or :websocket js/WebSocket
                         :long-poll js/Phoenix.LongPoll))

(s/conform ::transport js/WebSocket)
For objects I use #(instance? Class %) and it works as expected.
#2016-06-1217:34t3chnoboyok, Iā€™ve just made it work:
(s/def ::transport (s/or :websocket #(= js/WebSocket %)
                         :long-poll #(= js/Phoenix.LongPoll %)))

(s/conform ::transport js/WebSocket)
#2016-06-1300:00t3chnoboyIs there a way to validate core.async chan type?#2016-06-1300:02t3chnoboyI want to define a spec for a function which returns the following map:
{:in-chan (chan)
 :out-chan (chan)}
#2016-06-1300:05jfntn@t3chnoboy: you should be able to make a predicate for a ManyToManyChannel instance, or perhaps more generically test that (satisfies? ReadPort in-chan) and (satisfies? WritePort out-chan) also both defined somewhere in core.async...#2016-06-1300:11t3chnoboy@jfntn something like:
#(instance? cljs.core.async.impl.channels.ManyToManyChannel %)
?
#2016-06-1300:12t3chnoboyDoesnā€™t look pretty...#2016-06-1300:17jfntnindeed, think you could import ManyToManyChannel and reference it directly though#2016-06-1300:21jfntnThe protocol check is probably a better way to go however#2016-06-1308:48dominicm@gfredericks: Yep, my specs make db queries. Spec has s/valid? to accomodate runtime validations (not just function checking in development time). I honestly think having two libraries, one for "types" and one for "db connecting checkers" would be wasted effort.#2016-06-1311:10jimmyhi guys, I want to discuss a bit about spec. Spec is amazing, I can see the workflow that we can try out things in dynamic clojure, then add spec to it later to ready for production. But would spec have an optimization behind to check for example: we define a function with the args which is an integer? then spec will also generate a function with type hint integer beside doing the validation ? Then we can have something that is very flexible at output error and very fast code as well. Just my thought.#2016-06-1312:10shemis there a way to write a spec that says "this map should contain these keywords, and only once each"?#2016-06-1312:17stathissideris@shem: but keys can only appear once each in a map... by definition!#2016-06-1312:21shemergh, right. was eyeing generator output that spat several instances of the map. need more coffee#2016-06-1312:42jimmyhi guys, how do we define a spec that can validate both namespaced key and non namespace key, for example : user/first-name and first-name one of those would be valid.#2016-06-1314:51manderson@nxqd:
(s/def ::first-name string?)
=> :user/first-name

(s/def ::map-spec (s/keys :opt [::first-name]
                          :opt-un [::first-name]))
=> :user/map-spec

(s/valid? ::map-spec {::first-name "joe"})
=> true

(s/valid? ::map-spec {:first-name "joe"})
=> true
#2016-06-1315:27settingheadhas anyone tried using core.async with clojure.spec? i ended up having a lot of functions that returns a channel in my code (`(defn [] (go ā€¦))`). iā€™d like to spec these functions beyond checking its type is ManyToManyChannel (i.e. checking what comes out the channel is valid based on a spec). Iā€™m thinking of adding transducers to the returning channels. but is there a better overall approach? should i write my functions this way to begin with?#2016-06-1315:41jimmy@manderson: I meant
(s/valid? ::map-spec {:user/first-name "joe"})
and (s/valid ::map-spec {:first-name "joe"})
#2016-06-1315:43manderson::first-name is the same as :user/first-name so either should work with above spec. Is that what you're asking?#2016-06-1315:43manderson^ is the same if you are in the same namespace. :: simply appends the current namespace qualification to the keyword#2016-06-1316:18jimmy@manderson: sorry, I was out. I will check on this, I think I miss understand something here.#2016-06-1316:27jimmyyeah I did mis understand, it works fine. with user/first-name. thanks#2016-06-1316:32jimmy@manderson: ah I have another question, if I refer the spec in another ns, it shouldn't work with :user/first-name case as long as I understand ?#2016-06-1316:38mandersonusing the definition of :user/map-spec as defined above, it will validate fine for :user/first-name or :first-name. The namespaced keywords reflect the namespace they were defined in if set with ::.#2016-06-1316:41jimmyok, I see. in s/keys is there a way that we can validate something like :first-name or :user/first-name, one of those is required.#2016-06-1317:02mandersonhm, good question. this seems to work:
(s/def ::map-spec2 
  (s/or :qual (s/keys :req [::first-name] 
                        :req-un []) 
         :simple (s/keys :req [] 
                         :req-un [::first-name])))
#2016-06-1317:03mandersonthe keywords in or are just tags...#2016-06-1319:24danstoneQuestion, when should one specify a property of a function in the :fn of an fdef rather than in a proper test.check property?#2016-06-1320:48danielcomptonIā€™m pretty sure the answer will be ā€œnoā€, but is there any way to use spec to define newtypeā€™s, i.e. I have a username and a full-name which are both strings. Can I spec functions to stop passing a username where a full-name is required? For this discussion, assume that both strings have no distinguishing features you could use.#2016-06-1320:50danstoneYou might be able to use conform with a custom conformer for 'full-name'. edit: never mind, it won't work - as you say such a conforming fn would never know the difference between the two values#2016-06-1403:25mfikesItā€™s too bad (s/describe (s/spec #(< % 3))) has to return something that doesnā€™t quite look like an anonymous function literal#2016-06-1403:34mfikesIā€™m not sure what to make of the fact that
(s/def ::a (s/and #(> % 2) #(< % 5)))
(s/def ::b #(and (> % 2) (< % 5)))
look the same. Perhaps it is OK because they behave the same way.
#2016-06-1405:02jimmyhi guys, how do we use variable inside of (s/def (s/keys)) like this
(def a [::first-name])
(s/def ::user (s/keys :req a))
It's a value in a macro in a macro. And I'm no macro master ...
#2016-06-1405:44jimmyI have an error while trying to generate from clojure.spec, I think this would be the try out such-that limitation in clojure.spec implementation.
(defn str-limit [lower upper] (s/and string? #(<= lower (count %) upper)))
(s/def :project/description (us/str-limit 116 1000)) ;; the gen doesn't work if the lower value is around above 100
(s/def ::project-gen (s/keys :req [:project/description]))
(gen/generate (s/gen ::project-gen))

;; -- error
ExceptionInfo Couldn't satisfy such-that predicate after 100 tries.  clojure.core/ex-info (core.clj:4703)
#2016-06-1407:09stathissideris@nxqd: This is because string? generates totally random strings and then checks the second predicate to see if they conform, and gives up after 100 tries. Consider wrapping the spec with s/with-gen and providing your own generator.#2016-06-1409:07jimmy@stathissideris: thanks !#2016-06-1411:26skapoorhi guys, am playing with clojure spec and running into an un-expected behavior: (s/valid? #{nil "a" "b"} nil) ;; returns false when it should be true#2016-06-1411:28minimalnil is not truthy so the set as a predicate is returning a non-truthy value#2016-06-1411:29minimal
(s/valid? #(contains? #{nil "a"} %) nil)
true
#2016-06-1411:30skapoor@minimal yeah, I tried that and it works.. but it should return true when a set is used in specs too right?#2016-06-1411:33minimalIf you are relying on the value returned from the set then it needs to be truthy to be valid#2016-06-1411:36skapoorokay, (map #{"a"} ["a" nil]) and (map #{"a" nil} ["a" nil]) both return ("a" nil)#2016-06-1411:37minimalThe not-found value of a set is also nil so it is troublesome if you donā€™t expicitly check using contains?.#2016-06-1411:38skapoori'm using sets in specs to define an enum of values with the possibility of nil.... so it looked like a convenient way. guess, I'll have to either use s/or nil? ... or #(contains?)#2016-06-1411:39minimalor you can use a defualt value that isnā€™t nil. (#{} 1) => nil (get #{} 1 :not-found) => :not-found#2016-06-1411:40minimalYeah itā€™s tricky#2016-06-1411:41skapoorok thanks. i'm wondering if I should file an issue for this on jira..#2016-06-1411:42minimalI donā€™t think itā€™s an issue#2016-06-1411:42skapooronly because the contains? behavior is right but when a set is used as a function call it's not.#2016-06-1411:43minimalUsing set as a predicate is a convenience but you are relying on an implicit conversion to boolean based on the value in the set#2016-06-1411:45gfrederickseither choice is surprising#2016-06-1411:46gfredericksto somebody#2016-06-1411:46gfredericksdepending on whether you expect the set to be treated specially or to be treated like a predicate#2016-06-1411:46skapoorthe implicit conversion to boolean is being done by the internal clojure implementation..#2016-06-1411:49skapoor@minimal: but I get it, it won't be considered a defect. so I'll just use a different way. thanks!#2016-06-1415:20Alex Miller (Clojure team)@mfikes from your question way back on describe - s/form is useful for distinguishing these#2016-06-1415:23Alex Miller (Clojure team)@skapoor: you could also wrap s/nilable around the set (s/nilable #{"a"})#2016-06-1417:39arohner@alexmiller: whatā€™s the rationale behind instrument only checking :args?#2016-06-1417:53Alex Miller (Clojure team)shift in perspective - instrument is about checking invocations are correct, not verifying the function itself is correct. (this is similar to how only the :args are checked in macro fdefs). The check and test functions are for verifying functionality of the code.#2016-06-1418:05bfabryis there still a way to set up your app so every spec is checked on every invocation?#2016-06-1418:41arohner@alexmiller: this seems to severely hamper specā€™ing non-pure functions (i.e. things that are difficult/impossible to write generators for)#2016-06-1418:42arohnerand IMO, validation and generative testing are still too tightly coupled#2016-06-1418:49Alex Miller (Clojure team)@bfabry: no, but that was not really the intention of instrument, which is about verifying that callers of a function are calling it correctly. If you want to test the the behavior of your functions are in accordance with your specs, you should use the spec.test functions to test your functions.#2016-06-1418:51bfabry@alexmiller: sure, and no doubt were we to use spec we would use the spec.test functions. but like @arohner mentioned not all functions are pure, and if I'm going to write all those specs then I might as well get some extra value from them for free by having them always be checked in lower environments where I do not care about performance#2016-06-1418:51Alex Miller (Clojure team)@arohner: there are still more things coming that will help with verifying aspects of non-pure functions#2016-06-1418:53arohner@alexmiller: one nice feature of schema is that you can choose to use validation at e.g. user-input boundaries, in production. Instrument seems more designed for dev-time atm#2016-06-1418:53Alex Miller (Clojure team)@arohner spec does not remove the need to write tests for your functions. those tests can use invoke specs to validate as appropriate#2016-06-1418:54Alex Miller (Clojure team)@arohner: instrument is only designed for dev time#2016-06-1418:54Alex Miller (Clojure team)you should not instrument in production#2016-06-1418:55Alex Miller (Clojure team)you can choose to explicitly conform with spec at boundaries if you like#2016-06-1418:56Alex Miller (Clojure team)there will be a conform-ex that validates or throws if not, not quite in yet#2016-06-1419:03bfabryif I explicitly conform, then it will happen in production, when I only want it in staging. it also adds a whole bunch of boilerplate to every single function. I don't really understand the reasoning here, being able to reuse the :ret and :fn specs for extra checking when performance isn't a consideration just seems like an obvious win. and I mean, I definitely can still do that, writing my own macro that wraps all functions or whatever, but it sounds like I won't be the only one#2016-06-1419:10Alex Miller (Clojure team)there is an assertion facility still coming as well#2016-06-1419:11Alex Miller (Clojure team)the point is that checking ret/fn every time should be redundant to what you have (presumably) already confirmed in testing - that your function works.#2016-06-1419:16bfabryI'm maybe a bit skeptical that adding generative testing (while definitely awesome) is going to straight away mean I stop writing functions that produce unexpected outputs when they encounter production data. and I'm a big fan of fail fast with a good error message when that does happen#2016-06-1419:19Alex Miller (Clojure team)you (will be) able to assert that return (if instrumented at dev time) or choose to explicitly validate it at production time if you want#2016-06-1419:20Alex Miller (Clojure team)it's unclear to me if you're talking about dev or prod#2016-06-1419:20bfabryactually talking about master/staging#2016-06-1419:21Alex Miller (Clojure team)fair enough - so you can turn on instrumentation in staging#2016-06-1419:21Alex Miller (Clojure team)that will check args on functions#2016-06-1419:21bfabryon my laptop/travis I run unit and generative tests, in master/staging the application runs "production-like" but with assertions turned on for :args :ret :fn, production the app runs with no assertions <-- this is my ideal scenario#2016-06-1419:22Alex Miller (Clojure team)there will be an assertion facility that you can use to (explictly) check ret/fn for instrumented functions#2016-06-1419:22danstoneAs the channel is a little bit louder this evening, I thought I'd pose a question I asked yesterday again: What is the intended usage of :fn in fdef - In the docs it says something like 'relationships between args and the return'. Is the idea here to restrict it to type-y properties (e.g arity 1 of map returning a transducer rather than a seq)? The reason I ask is it's possible to define many more general properties of functions as part of the spec. As spec gets richer I imagine it may be possible to auto-generate the code for many properties (idempotency is easy if you have spec'd the args)#2016-06-1419:23Alex Miller (Clojure team)I think what I'm talking about is close to that, but varies in that ret/fn are not automatically checked but require an explicit assert (which is not active in production)#2016-06-1419:29bfabryright. and so I'll probably end up writing a macro that wraps every function to add that explicit assert, and I've got a feeling that a whole lot of people will do that. we use plumatic/schema atm which checks arg/return values when validation is turned on, and I'd say the same number of bugs are caught by the return value checking as the arg value checking. aaaanyway, writing the macro is nbd, and maybe it'll turn out I don't actually need it or I'm the only person who does šŸ™‚#2016-06-1419:41Alex Miller (Clojure team)one question I have is whether you're getting the same level of generative testing from schema that you can get from spec (that is, whether more of those bugs could/should have been caught earlier)#2016-06-1419:41Alex Miller (Clojure team)also keep in mind that the return value often is an input to another function, which can check its args#2016-06-1419:44bfabryno, we're definitely not. but like I said I'm just a bit skeptical that generative testing will suddenly mean these issues disappear, and it costs me like an extra $5 per month to run extra validation in the staging environment so why not? the return value will likely be the input value to another function, but maybe that function doesn't have a spec yet, because we didn't feel it was worth the time to write yet, or maybe it's too broadly defined etc. If I believed I could perfectly specify every function up front so that bugs were impossible I'd be writing haskell šŸ˜†#2016-06-1419:48Alex Miller (Clojure team)maybe you should just write your code without the bugs?#2016-06-1419:48Alex Miller (Clojure team)šŸ˜€#2016-06-1419:53bfabryhaha#2016-06-1420:49wilkerluciohey people, does anyone here found/created a generator for cljs to create strings from regexps? I was hoping to use the string-from-regexp from test.chuck but I just realised it's implementation is for CLJ only#2016-06-1420:56Alex Miller (Clojure team)I would ask @gfredericks#2016-06-1421:09wilkerluciothanks Alex, I opened an issue on test.chuck, he will see it I think šŸ™‚#2016-06-1421:09wilkerlucioone more thing, given I have my fdef all spec set, I remember seeing a simple command to test that function but I'm not finding it, what's the simplest way to run generative specs on a function?#2016-06-1421:15Alex Miller (Clojure team)clojure.spec.test/check-var#2016-06-1421:21wilkerlucio@alexmiller: thanks, would you consider adding that to the clojure spec guide?#2016-06-1421:21Alex Miller (Clojure team)yeah, that's on the todo list - I was actually working on some alpha6 related updates right now#2016-06-1421:21wilkerlucionice šŸ™‚#2016-06-1421:51ikitommiIf I have understood correctly, the registry doesnā€™t have any tools for handling duplicate definitions?#2016-06-1421:53ikitommiso, if there are multiple definitions fos person/id, the last one stands?#2016-06-1421:56bfabry@ikitommi: yeah just replaces https://github.com/clojure/clojure/blob/master/src/clj/clojure/spec.clj#L261#2016-06-1422:10ikitommiIs this good? should the specs be immutable by default? If there are name clashes, depending on the import order of namespaces, the specs might mean different thing.#2016-06-1422:11ikitommior some hooks for the registry to resolve those clashes.#2016-06-1422:14wilkerlucio@ikitommi: this is the reason they are encouraging namespaced keywords I think, do you have a situation where the same namespace is loaded on multiple files?#2016-06-1422:15bfabryfunctionally the same as multiple (def's isn't it?#2016-06-1422:20bsimadoes anyone have examples of using fdef and check-var? I'm having trouble getting it to work in my test suite#2016-06-1422:23ikitommi@wilkerlucio: true that - the specs must have a namespace, but it doesnā€™t have to be a clojure namespace. One might have multiple :order/ids in a large system. Should not, but could.#2016-06-1422:23wilkerlucio@bsima: check this snippet, may help you:#2016-06-1422:25wilkerlucio@ikitommi: that's some sort of thinking shift, moving from those to fully qualified, there are going to be some nice helpers to deal with longer namespaces in 1.9#2016-06-1422:26wilkerlucioyes, the problem exists like you said, but it's the same for def as mentioned by @bfabry, I guess it's just about people starting moving towards fully qualified namespaces to avoid name clashes, I believe when it's the norm will be very positive for everyone#2016-06-1504:44jimmyhi guys, how do we use variable inside s/keys like this
(def a [::first-name])
(s/def ::user (s/keys :req a))
#2016-06-1509:54danstone@nxqd: I don't think we can because s/keys is a macro. Though I'm not sure if this is worth it for keys, as it doesn't capture predicates, I would expect it to be a function. As far as I can tell the only thing macro'ey it does is look at forms like (or ::foo ::bar).#2016-06-1517:50Alex Miller (Clojure team)At the moment you can't do this except via eval or something#2016-06-1517:50Alex Miller (Clojure team)But there has been talk of adding a fn entry point that would allow it#2016-06-1518:57seancorfieldWith check-fn and check-var, is there a way to get a "nice" explanation of the failures, like explain produces?#2016-06-1518:57seancorfieldI'm trying to see how the changes in Alpha 6 affect the testing workflow. #2016-06-1519:47eggsyntax@nxqd: (somewhat belatedly) you can replace the var with the function that populates it (it can cache its results, if it's an expensive call), if it's a var that it makes sense to populate at macro eval time.#2016-06-1519:48eggsyntaxUnrelated: test.check has a fn (`fmap`) that lets you call an arbitrary fn on the results of a generator. But is there any way to create a generator that just calls an arbitrary fn? I haven't found one.#2016-06-1519:59eggsyntaxThe only solution I've found so far is to do (fmap (fn [_] do-what-I-want) some-arbitrary-gen), but it's pretty ugly to put in a generator and then ignore what it generates.#2016-06-1520:52jannisFolks, is it a good idea to define fdef function/macro specs next to the actual function/macro definition or would you rather define them in separate my-project.specs kind of namespace?#2016-06-1520:53jannisI'd like to associate most of my functions with specs for automatted random function testing but at the same time I don't want to clutter my code base with specs too much.
#2016-06-1520:53jannisI guess there are no best practices established yet?#2016-06-1520:54eggsyntaxI've been debating that myself, haven't come to any particular conclusion.#2016-06-1520:55eggsyntaxCurrently defining them on their own because it's still fairly experimental for us...but as we fully integrate them, I'll definitely consider moving them to live next to what they're speccing.#2016-06-1521:11seancorfieldI think weā€™ll have data specs in separate namespaces. Not sure yet about function specs. Part of me would like them above function definitions ā€” that seemed natural for when we used Schema and core.typed.#2016-06-1521:12seancorfieldWe went back and forth between core.typed and Schema several times before we abandoned them. With spec being in core, I think weā€™re more likely to stick with it.#2016-06-1521:12jannisYeah. It also means you don't have to require the function namespaces and the function spec namespaces when you want to test functions.#2016-06-1521:13seancorfieldAlthough, https://github.com/clojure/java.jdbc/blob/master/src/test/clojure/clojure/java/test_jdbc.clj#L28-L32#2016-06-1521:13seancorfieldhttps://github.com/clojure/java.jdbc/blob/master/src/main/clojure/clojure/java/jdbc/spec.clj#2016-06-1521:14seancorfieldThere the function specs are in a separate clojure.java.jdbc.spec namespace, below the data specs.#2016-06-1521:14seancorfieldBut that was mostly to ensure pre-1.9 code can still use the library#2016-06-1521:15jannisHere's my first attempt at writing function specs alongside the actual functions: https://github.com/workfloapp/macros/blob/jannis/clojure-spec/src/main/workflo/macros/props.cljc#L9#2016-06-1521:18jannisIt would feel a little more natural to define the function spec like`:pre` and :post, e.g.`(defn my-fun [arg1 arg2] {:spec {:args ... :ret ... :fn ...}} <body>)` or something like it, although there are good reasons to keep them separate.#2016-06-1523:22bbrinck@jannis: What do you think about putting the fdefs before the function? IMHO, that's a little clearer. I like having the function specs near the functions since that provides documentation when reading the function. Not sure about data specs, though.#2016-06-1523:29bbrinckor rather, put each fdef before its associated fn#2016-06-1523:45bfabrywhat have I messed up here...?
Clojure 1.9.0-alpha7
Java HotSpot(TM) 64-Bit Server VM 1.8.0_65-b17
        Exit: Control+D or (exit) or (quit)
    Commands: (user/help)
        Docs: (doc function-name-here)
              (find-doc "part-of-name-here")
Find by Name: (find-name "part-of-name-here")
      Source: (source function-name-here)
     Javadoc: (javadoc java-object-or-class-here)
    Examples from : [clojuredocs or cdoc]
              (user/clojuredocs name-here)
              (user/clojuredocs "ns-here" "name-here")
boot.user=> (require '[clojure.spec :as s])
nil
boot.user=> (require '[clojure.spec.test :as t])
nil
boot.user=> (defn foo [x] (inc x))
#'boot.user/foo
boot.user=> (s/fdef foo :args (s/cat :x integer?) :ret integer?)
boot.user/foo
boot.user=> (t/check-var foo)

java.lang.IllegalArgumentException: No :args spec for 
#2016-06-1523:51bfabry
boot.user=> (s/fn-spec foo)
nil
boot.user=> (doc foo)
-------------------------
boot.user/foo
([x])
Spec
  args: (cat :x integer?)
  ret: integer?
nil
boot.user=> (s/fn-spec 'foo)
#2016-06-1600:35seancorfield(t/check-var #ā€™foo) ā€” it accepts a Var.#2016-06-1600:36seancorfieldSimilarly (s/fn-spec #ā€™foo) ā€” although the result of that is less useful to print since it is an object.#2016-06-1600:37seancorfieldYou could also do (t/check-fn foo (s/fn-spec #ā€™foo))#2016-06-1600:38seancorfieldcheck-fn is so you can provide an (s/fspec :args ā€¦ :ret ā€¦ :fn ā€¦) for an arbitrary function without fdef.#2016-06-1601:23leifpHi, all. I hacked up specs for most fns in clojure.core and 1/3rd of the macros (WIP). I don't know much about clojure.spec (of course), though, so it wasn't the good kind of hacking,. Hold your nose if necessary: https://gist.github.com/leifp/abe50082f6baa0063f8b7840e80657af#2016-06-1607:08bfabryahhhhh thanks @seancorfield#2016-06-1609:58jannisShouldn't ((instrument #'my-fun) :foo) throw an exception with an explanation if my-fun has a spec defined for it with fdef and if the returned value does not conform to the spec of my-fun?#2016-06-1609:59jannisInstead it just returns :clojure.spec/invalid.#2016-06-1610:04jannisOh, that's because that's what my-fun returns in this case. So it doesn't even fail, it just returns its value, even if that doesn't conform to the function spec.#2016-06-1610:41patrkrisI see clojure.spec uses "named arguments" in several places (e.g. (clojure.spec/fdef :args ... :ret ....)). Is there any convention as to when to use named arguments vs. a map? Maybe named arguments are primarily used for macros?#2016-06-1611:10pithylessHi, all. Has someone come across the need to have an s/keys that is more strict (does not allow extra keys)?#2016-06-1611:14jrychter@pithyless: I have. I am guessing Rich thought it isn't a good idea, as in the long term it could hurt expandability and composability. But I still think we should be able to do it in some cases.#2016-06-1611:22pithylessI'm sure it's been brought up internally; I wonder how likely it would be to get an :only option added to s/keys.#2016-06-1611:56patrkris@pithyless: some semi-related discussion here: https://groups.google.com/forum/#!topic/clojure/UVgXXcIxhJQ#2016-06-1612:00jrychterAs I suspected ā€” "spec embraces the idea of open maps". But there are cases where you don't want open maps. A good example is the current world of lein-cljsbuild, where I regularly pull my hair out because it isn't clear if a particular setting is in the right place. Extra keys do no harm, so people tend to put config options all over the place, and you end up with a terrible mess. Figwheel recently started doing some great work towards fixing this ā€” and this kind of config checking should really be "these keys, and these keys only".#2016-06-1612:02pithylessI've got some half-baked solution with clojure.set/difference and keys, but I wish it were simpler.#2016-06-1612:05jrychterI am hoping someone will write a validations library similar to bouncer, based on clojure.spec.#2016-06-1612:10jannisThings are getting a little meta over here... I'm spec'ing out a data format and a parse function that first uses clojure.spec/conform to validate and normalize the input and then passes it on to a transformation function. For that transformation function the input data format is the output of conform, so I now have a spec for what conform returns as well. šŸ™‚#2016-06-1612:46ghadi@pithyless: https://clojurians.slack.com/archives/clojure-spec/p1465423760000580#2016-06-1612:53pithyless@ghadi: Thanks for linking to the archives. I understand why the default is what it is, my use case falls clearly in the "don't want to accidentally be leaking stuff" camp. But it's a minor thing... clojure.spec is nice šŸ™‚#2016-06-1613:16dominicm@jrychter: As someone working on this: spec isn't suited to the task.#2016-06-1613:17jrychter@dominicm: oh šŸ˜ž I was hoping that one could build a validation system based on what explain-data returnsā€¦#2016-06-1613:17dominicmThat's the easy bit.#2016-06-1613:19jrychterI currently use bouncer, but I have to work around its problems, and as I migrated from schema to spec, I was hoping I could get rid of it entirely.#2016-06-1613:19dominicm@jrychter: When you do user registration validation, you need to check email uniqueness. So that requires 2 things: 1. Data to validate, the email 2. Sideband data, the database (connection). Spec has a funnel for 1. Not one for 2.#2016-06-1613:20dominicm@jrychter: https://github.com/leonardoborges/bouncer/issues/43 šŸ˜› I know bouncer. It would be nice.#2016-06-1613:20ghadispec has nothing to do with integrity checks (#2)#2016-06-1613:21dominicmYour only options for validating in that way, and neither of these are particularly spec-y, are dynamic vars, or the weird macro I started working on, and gave up on because it was a big ball of mud.#2016-06-1613:21jrychterAs for (2) above, I currently use global state encapsulated using mount.#2016-06-1613:22dominicm@jrychter: You may not have as much trouble.#2016-06-1613:22dominicmHowever, I land on the weavejester side of the camp. šŸ˜›#2016-06-1613:22jrychterMeaning, some of my bouncer validations actually use database or even (gasp) network connections.#2016-06-1613:22dominicmYep, it's a necessity.#2016-06-1613:22dominicm@ghadi: What kind of check does spec do?#2016-06-1613:22jrychterEU VAT numbers are an example.#2016-06-1613:28dominicm> weavejester side of the camp I should clarify this means that I use component.#2016-06-1613:48dominicm@ghadi: I'm mostly interested in the terminology and reading on the subject.#2016-06-1615:32seancorfieldAs I recall, the discussion about closed maps ended with consensus that using select-keys was appropriate for "not leaking additional keys", but there wasn't much consensus on validation of closed maps. #2016-06-1615:33seancorfieldI got the impression @richhickey is "considering" adding support for it due to repeated requests from the community but he doesn't consider it a good idea. #2016-06-1615:37seancorfieldI think designing systems to simply ignore extra keys is more robust, but I also accept there are cases where that is difficult (the example I gave is clojure.java.jdbc/insert! where a map is converted to a SQL insert statement without reflection and therefore extra keys become extra columns and will be rejected by the database; the alternative is to introspect the database and ... well, do what? Silently ignore the extra columns? Throw a different exception?)..#2016-06-1615:39seancorfieldAnd there in lies the issue: the behavior for extra keys is not knowable at that level - it would have to be an application-level decision (and the application has the means to do that reflection and call select-keys if it wants).#2016-06-1615:41seancorfieldWhich is why I side with Rich that it's not clojure.spec's job to directly support restricting keys, since it shouldn't be the norm, and you can do it via custom predicates already if you really have one of those odd cases where you really do need to check. #2016-06-1615:42donaldballGoes back to my point that if you want that level of validation, just specifying that the db arg to insert! is a Connection or a Connectable or whatever is not sufficient; it would need to specify that the database in question has such and such a schema#2016-06-1615:44seancorfieldRight, and that's a whole different thing. java.jdbc provides metadata/reflection APIs but doesn't use them internally (for performance mostly),#2016-06-1616:46bhaumanA strict key set constraint can make a lot sense depending on the situation. This depends on the api of course, but I can't imagine a higher level api, that third parties are going to rail against, not having tighter key set constraints.#2016-06-1616:48bhaumanThat being said, you can compose a strict map key set and description with spec fairly handily.#2016-06-1616:51bhaumanSpec is beautiful.#2016-06-1617:48wilkerlucio@pithyless: if you really want to get just the strict keys from a map, I would suggest using select-keys as people mentioned before, as a plus, I wrote a simple utility that can extract the keys from a keys spec, it's still a naive implementation (it doens't consider req-un or opt-un) but can be starting point if you want to go further on the idea:#2016-06-1617:52wilkerlucioso I believe if extra keys are harmful on your case, you can use this kind of trick to remove unwanted ones, so we need to force the restriction of the keys, but just eliminate the ones you can't deal with (and only if you can't silently pass it on)#2016-06-1617:59seancorfield@wilkerlucio: The s/def for ::name and ::email donā€™t do anything in that exampleā€¦?#2016-06-1618:01seancorfieldAre you doing something else to check the values of the ::my-aggregate map conform to those specs, by convention (based on the key names)?#2016-06-1618:01wilkerlucio@seancorfield: the idea here was just to show an example of how to do a select-keys while reusing the specs, you are supposed to call s/valid? yourself before doing the strict-keys on this case#2016-06-1618:02seancorfieldOK, so youā€™re relying on convention that the (qualified) key names are the same as the specs that apply to their respective values?#2016-06-1618:03wilkerlucioyes#2016-06-1618:03wilkerluciobut like I said, this is a naive implementation, if you really wanna rely on it, will need more work there#2016-06-1618:04seancorfieldFYI, instead of (->> (s/form spec) (next) (partition 2) (map vec) (into {})) you could just do (->> (s/form spec) next (apply hash-map))#2016-06-1618:05wilkerlucio@seancorfield: thanks for that, I'll update the snippet šŸ™‚#2016-06-1618:11bfabryhas anyone come up with a good way of speccing non-keyword maps with known keyval pairs?#2016-06-1618:30zaneDoes spec just not play well with tools.namespace?#2016-06-1618:30zaneOr with the REPL?#2016-06-1618:32zane
dev=> (defn f [x] (inc x))
#'dev/f
dev=> (s/fdef f :ret pos?)
dev/f
dev=> (s/instrument #'f)
#'dev/f
dev=> (f 1)
2
dev=> (f -3)
-2
#2016-06-1618:33bfabryinstrument doesn't check :ret or :fn in the latest alpha#2016-06-1618:33bfabryonly :args#2016-06-1618:33zaneOuch.#2016-06-1618:33zaneIs that going to change?#2016-06-1618:33zaneDid it check :ret and :fn in previous alphas?#2016-06-1618:33bfabryyes#2016-06-1618:34zaneā€¦ Huh.#2016-06-1618:34bfabrythe reasoning is that :ret and :fn are for checking whether the function is correct, which should happen when you're using the functions in clojure.spec.test. :args are for checking the function was invoked correctly#2016-06-1618:38zaneSo, we have a function that reads environment variables. That function has a :ret spec on it that validates that required environment variables are set and have valid values.#2016-06-1618:38zaneMy understanding now is that instrument will not help me here and I should use s/conform?#2016-06-1618:40bfabryyeah, s/conform or s/valid? or whatever explicitly. there's also an s/conform-ex coming iirc#2016-06-1618:41zaneWhere should I look for info on s/conform-ex?#2016-06-1618:41bfabryin a subsequent release šŸ˜†#2016-06-1618:43bfabryI'm sure it's coming soon, they've been evolving very rapidly#2016-06-1618:44zaneUnderstood.#2016-06-1618:44zaneDo you know what s/conform-ex going to do?#2016-06-1618:45bfabryconform or throw an exception on failure I assume#2016-06-1618:45zaneAh, I see.#2016-06-1618:46settingheadlooks like there was a commit 3 hours ago: https://github.com/clojure/clojure/commit/aa9b5677789821de219006ece80836bd5c6c8b9b#2016-06-1618:54leifpAs mentioned before, I've (roughly) spec'ed a good chunk of clojure.core. I made that an actual repo in case someone wants to test and/or beautify them: https://github.com/leifp/spec-play#2016-06-1618:57zane@bfabry: When is the :ret argument to fspec ever used, then?#2016-06-1618:58bfabrywhen clojure.test.* functions run generative tests#2016-06-1618:59zaneGot it. So only for generative testing.#2016-06-1618:59zaneOof.#2016-06-1619:05zaneThat seems like a very weird design choice to me.#2016-06-1619:08bfabryI'm not real sold on it either#2016-06-1619:08bfabrybut I haven't used spec on anything big enough to be confident#2016-06-1619:18robert-stuttafordso, i'm new to test.check in general. anyone know how i might generate a set of keywords, from a known set of possible keywords?#2016-06-1619:19robert-stuttaforde.g. i have #{:a :b :c :d :e} and i want generated subsets of same#2016-06-1619:20bfabry(s/exercise #{:foo :bar}) => ([:bar :bar] [:bar :bar] [:bar :bar] [:bar :bar] [:foo :foo] [:foo :foo] [:bar :bar] [:foo :foo] [:bar :bar] [:foo :foo])#2016-06-1619:21bfabrythis is probably better#2016-06-1619:21robert-stuttafordinteresting#2016-06-1619:21bfabry(s/exercise (s/coll-of #{:foo :bar} #{})) => ([#{} #{}] [#{} #{}] [#{} #{}] [#{:bar :foo} #{:bar :foo}] [#{} #{}] [#{:bar :foo} #{:bar :foo}] [#{:bar :foo} #{:bar :foo}] [#{:bar} #{:bar}] [#{:bar :foo} #{:bar :foo}] [#{:bar :foo} #{:bar :foo}])#2016-06-1619:22robert-stuttafordi'm modelling Magic the Gathering cards as an exercise#2016-06-1619:22bfabryhaha, nice#2016-06-1619:22robert-stuttaford
(s/def ::type #{:land :creature :artifact :enchantment :sorcery :instant :planeswalker})
(s/def ::types (s/with-gen
                 (s/and set? (s/+ ::type))
                 #( ? )))
#2016-06-1619:23bfabryanyway yeah s/exercise generates data that suits a spec, a spec of "sets of these keys" is (s/coll-of #{:keys} #{})#2016-06-1619:23robert-stuttaford::types works, but it can't be generated because of how s/and generators work: generate for the first and discard anything that doesn't satisfy the rest of the ands#2016-06-1619:23bfabryI think you just want (s/coll-of ::type #{})#2016-06-1619:24robert-stuttafordindeed, thank you#2016-06-1619:25bfabry
(s/def ::type #{:land :creature :artifact :enchantment :sorcery :instant :planeswalker})
=> :kafka-google-connector.runner/type
(map first (s/exercise (s/coll-of ::type #{})))
=>
(#{}
 #{:planeswalker}
 #{:artifact}
 #{:land}
 #{:creature :planeswalker}
 #{:sorcery}
 #{:land :planeswalker :sorcery}
 #{:instant :enchantment :land :planeswalker}
 #{:instant :enchantment :creature :land :planeswalker :sorcery}
 #{:land :sorcery})
#2016-06-1619:25bfabryman.. that's pretty neat#2016-06-1619:26robert-stuttafordok. my next question (which is the thing i really want to solve, now that i've softened you up šŸ™‚ ) is how might i write a generator when i'm using s/and on two s/keys specs?#2016-06-1619:26robert-stuttaford
(s/def ::base-card (s/keys :req-un [::name ::types ::metadata]
                           :opt-un [::sub-type ::legendary? ::world?]))

(s/def ::cost string?) ;; todo

(s/def ::spell (s/and ::base-card (s/keys :req-un [::cost])))
#2016-06-1619:27bfabryI've not actually looked into generators sorry#2016-06-1619:27robert-stuttafordah šŸ™‚ worth a try!#2016-06-1619:27robert-stuttafordit's a super-interesting problem to solve#2016-06-1619:27robert-stuttafordto me, anyway#2016-06-1619:47wilkerlucio@robert-stuttaford: just a suggestion, since you are modeling something new, maybe would be better to use the namespaced keys instead of clear ones, with namespaced keys you can for example validate a map keys even if you don't know the aggregate name for it#2016-06-1620:18bhaumanSpit-balling on strict keys just for the fun of it. I would love some feedback.#2016-06-1620:27leifpbhauman: What are the advantages of this vs. just (s/& (s/keys ...) #(only-these-keys % [:k ...])) ? Or a macro that expands into that.#2016-06-1620:29bhauman@leifp: the only interesting thing here is the explain data#2016-06-1620:29bhaumanwhere it points exactly to the key that failed#2016-06-1620:30bhauman
In: [:there] val: :there fails spec: :howdy/fine at: [:there] predicate: #{:builds :server-port :server-ip :http-server-root}
#2016-06-1620:34bhaumanit will create explain data for all the keys that failed#2016-06-1620:51bfabry@robert-stuttaford: I wrote this which works. I don't know how sane it is. my guess is "not very"
(defmacro extend-keys [spec-name & {:keys [req-un opt-un]}]
  (let [spec-m (apply hash-map (rest (s/form spec-name)))]
    `(s/keys :req-un ~(into (:req-un spec-m) req-un)
             :opt-un ~(into (:opt-un spec-m) opt-un))))
=> #'kafka-google-connector.runner/extend-keys
(s/def ::spell (extend-keys ::base-card :req-un [::cost]))
=> :kafka-google-connector.runner/spell
#2016-06-1621:11leifpbhauman: Hmm... I guess there is no explain equivalent of conformer or with-gen, so I can't really think of another way to do it than reifying Spec. Your impl. looks fine, but it doesn't seem to explain the extra keys if one of the required keys fails its spec.#2016-06-1621:13bhauman@leifp: I haven't looked at that, must be because of the s/and#2016-06-1621:14bhaumanbtw I've iterated on it a bit#2016-06-1621:16bhauman@leifp: what do you mean by extra keys? you mean in the explain data?#2016-06-1621:23leifp@bhauman:
user=> (s/explain (strict-keys :req [::r]) {::r "bad" ::extra 2})
In: [:user/r] val: "bad" fails spec: :user/r at: [:user/r] predicate: number?
In: [:user/extra] val: :user/extra fails at: [:user/extra] predicate: #{:user/r}  ;; <<< expected, not present
#2016-06-1621:23leifpThat output line was expected and not present, I mean.#2016-06-1621:24bhaumanoh yeah it short cuts#2016-06-1621:27bhaumans/and short cuts#2016-06-1621:27bhaumanso that makes sense#2016-06-1621:33bhaumanI would need to compose over the keys-spec to get that behavior#2016-06-1707:04Oliver GeorgeHi Specy Specers. What's the most human friendly way of viewing the output from check-var? I see that reporter-fn can be provided, is there a commonly used one? (I'm using CLJS so perhaps that makes a difference).#2016-06-1709:27jrychterI find myself writing lots of (s/and string? seq) to specify non-empty strings. Also, I'm missing a predicate for strings of length from n to m (analogous to int-in-range?).#2016-06-1709:34jannisIs it possible that s/with-gen alters the spec it defines a generator for? I have a simple (s/cat :base ... :children ...) spec that works fine but as soon as I wrap it in (s/with-gen <spec> #(gen/tuple (s/gen ...) (s/gen ...))), data that would previously conform to the spec now becomes invalid.#2016-06-1709:35jannisNote: I', not generating the data using the generator yet. I'm using hand-written data.#2016-06-1709:42jannisHere's a minimal example: https://gist.github.com/Jannis/5dcc91473f20861d154dc8be2fff2bfd#2016-06-1715:53seancorfield@olivergeorge: there's a discussion about that on the main Clojure mailing list. I'm very interested in the answers to that question. #2016-06-1716:37leifp@jannis: It looks like it's introducing a new regex context like spec does:
user=> (s/explain (s/cat :x ::pair) '[a [b c]])
Success!
user=> (s/explain (s/cat :x ::pair-with-gen) '[[a [b c]]])
Success!
#2016-06-1717:34sparkofreasonI'm working on some code that does simulations over state machines. The transition functions tend to have a lot of detailed conditional logic, and as a result test.check doesn't seem to be a great fit for validating the functions (random inputs tend to be ignored or lead to errors, and writing generators to provide valid input is essentially the same as writing the state machine model). When instrument-all checked the return value it was very useful, since I caught a lot of errors of omission, misspelled keywords, etc. I'd like to suggest we have an option to check :ret and :fn for cases like this.#2016-06-1717:50seancorfieldYeah, whilst I agree in principle with the justification @alexmiller offered as to why :ret and :fn checking was removed in Alpha 6, I also agree that there is potentially a lot of value in having the option to be able to instrument functions in a way that does conform the result at least.#2016-06-1717:50seancorfieldThis is the commit that changed the behavior: https://github.com/clojure/clojure/commit/30dd3d8554ff96f1acda7cbe31470d92df2f565a?diff=split#2016-06-1717:55sparkofreasonThanks, I may use that to hammer out my own version of instrument for now.#2016-06-1717:56bfabryI think there's gotta be something coming for spec'ing impure functions that will cover this#2016-06-1717:56bfabryotherwise you could never spec them, really, or the spec would be pointless#2016-06-1718:05Alex Miller (Clojure team)@seancorfield: Rich has some ongoing work, I'm not sure what the endpoint will be on this. you might have noticed that explain-out was made public today in master#2016-06-1718:06Alex Miller (Clojure team)@bfabry: not every function is a great candidate for generative testing via spec (but the spec may still be useful for docs or other purposes)#2016-06-1718:07eggsyntax@alexmiller: where was the explanation @seancorfield mentioned of why :ret and :fn checking was removed? Iā€™m curious to read it.#2016-06-1718:08Alex Miller (Clojure team)mailing list#2016-06-1718:08Alex Miller (Clojure team)https://groups.google.com/d/msg/clojure/RLQBFJ0vGG4/UGkYS7U_CQAJ#2016-06-1718:08eggsyntaxThanks šŸ™‚#2016-06-1718:09Alex Miller (Clojure team)@seancorfield: I think Stu is looking at some testing-related mods too btw#2016-06-1718:27seancorfieldre: explain-out ā€” I already added a comment on that commit thanking him for that šŸ™‚#2016-06-1718:29seancorfield@alexmiller: I really do appreciate the steady stream of alpha builds so we can all try this stuff out and provide feedback.#2016-06-1718:30Alex Miller (Clojure team)I'm sure there will be more :)#2016-06-1718:34seancorfieldHaving an option on instrument to use the old version of spec-checking-fn with :ret and :fn conforming would be very nice. I think checking just :args is the right choice for most cases of instrumentation, but I think being able to "fully instrument" certain functions would be very valuable ā€” especially for functions that cannot easily be tested the generative way.#2016-06-1718:36seancorfieldFor example, working on java.jdbcā€™s specs, they canā€™t reasonably be tested generatively because many of them are side-effecting (updating the database) and writing generators that conformed to the database schema would be ā€¦ a huge amount of work, if itā€™s even feasible (e.g., unique key constraints etc?).#2016-06-1718:36seancorfieldSo losing the ability to conform the :ret and :fn specs there is kind of a big deal, IMO.#2016-06-1718:39seancorfieldIā€™ll be interested to see how this all ends up since any given code base is going to have a mix of functions that can reasonably be tested generatively and functions that canā€™t, so (clojure.spec.test/run-all-tests) needs a way to distinguish those, right?#2016-06-1718:51seancorfield(mind you, right now I canā€™t run generative testing on java.jdbc because the system doesnā€™t know how to generate a java.sql.Connectionā€¦ which might be an interesting exercise šŸ™‚ )#2016-06-1718:52seancorfield(and Iā€™d also need a generator for a java.sql.PreparedStatement)#2016-06-1718:53seancorfieldWhat is the recommendation for stuff like that? How would you even write a generator for some of these Java objects?#2016-06-1719:34tomcWould anyone be willing to offer some guidance on conventions for attributes shared by entities of different types? For instance, I have "question" and "survey" entities, each of which can have names. Right now I'm using (s/def :entity/name string?) and entities of either type can have an :entity/name attribute along with their type-specific attributes. The alternative of course is to define both :question/name and :survey/name. I haven't seen the :entity/attr pattern elsewhere, and I'm wondering whether there's a reason for that.#2016-06-1720:03wilkerlucio@tomc: I believe you can share the attribute as long as the semantic is the same, for example, a car name may have a different semantic from a person name, but that will depend on the requirements on your system#2016-06-1720:09tomc@wilkerlucio: thanks a lot, that's helpful.#2016-06-1720:12gphilippIā€™m trying to generate dates within a range with spec, and Iā€™m stucked with this piece of code which generates #inst whose year is above 9999, making them unrecognized when i try to def them manually afterwards : (gen/sample (s/gen inst?) 60) #2016-06-1720:13gphilippex of data generated: #inst"26138-06-03T15:09:43.670-00:00 #2016-06-1720:14gphilipp
(def d1 #inst"26138-06-03T15:09:43.670-00:00")
CompilerException java.lang.RuntimeException: Unrecognized date/time syntax: 26138-06-03T15:09:43.670-00:00, compiling:(/Users/gilles/dev/try-spec/src/try_spec/core.clj:23:46) 
#2016-06-1720:18wilkerlucio@gphilipp: you can try something like this:#2016-06-1720:18wilkerluciothe first number, 100000 is a cap to limit the increment, and the 1465391285642 is the ms for a start date, adjust those to met your needs#2016-06-1720:19wilkerlucioah, and this example is for CLJS, please change the Date initialisation if you are using on Clojure#2016-06-1720:25gphilipp@wilkerlucio: thx, I will try this#2016-06-1720:28leifp@gphilipp: There is also an clojure.spec/inst-in macro.#2016-06-1720:28seancorfieldThe default 100 tests for test.check can take a really long time on some fairly simple looking specs...#2016-06-1720:29seancorfieldā€¦running 50 tests wasnā€™t too bad but it seems to be taking more than linearly longer to do 5, 10, 25, 50, 100...#2016-06-1720:31leifp@seancorfield: Yeah, as well as being very dependent on ordering. The caveats about "generate-and-test" style vs. constraint satisfaction are in full effect here.#2016-06-1720:32seancorfieldOn the plus side, I figured out how to write generators for stuff like java.sql.Connection etc šŸ™‚#2016-06-1720:32seancorfieldAnd I also finally figured out how s/keys and the test.check integration hang togetherā€¦ which answered a question Iā€™d asked someone else here before I understood what was going on...#2016-06-1720:35gphilippThanks @leifp, (gen/sample (s/gen (s/inst-in #inst "2016-01-01" #inst "2016-12-31")) 100) did the trick#2016-06-1720:39leifp@gphilipp: The end date is exclusive, remember.#2016-06-1720:39gphilippah, correct#2016-06-1720:40gphilippI still wonder why (gen/sample (s/gen inst?) 60) generates invalid instants.#2016-06-1720:44leifpSee? Generative testing is going to help us avoid the Year 26k Problem.#2016-06-1721:08seancorfieldFYI, this is the spec that takes a crazy long time to gen test: https://github.com/clojure/java.jdbc/blob/spec-gen/src/main/clojure/clojure/java/jdbc/spec.clj#L205#2016-06-1721:10seancorfield(and it took me a while to even get so far as to make run-all-tests actually start runningā€¦ with-gen taking a 0-arity function that returns a generator is very counter-intuitive and I kept getting that wrong šŸ˜ž )#2016-06-1721:57leifp@seancorfield: The (s/* (s/or ...)) combo seems to be the culprit. It would probably be very fast if you could limit it to a max size. I don't know how to do that, though.#2016-06-1721:59seancorfieldI guess I could always add a custom generator around it just to avoid that?#2016-06-1721:59seancorfield(and, thanks for the pointer on that!)#2016-06-1722:18leifp@seancorfield: This works: (s/def ::column-spec (s/with-gen (s/cat :spec (s/* (s/alt :kw keyword? :str string?))) #(g/vector (g/one-of [g/string g/keyword]) 0 6))) Still not what I'd call lightning-fast, but you can bound the generation time by decreasing the max. And note that g/ refers to clojure.test.check.generators, the equivalent using clojure.spec.gen didn't work.#2016-06-1722:20leifpKind of clunky, though, and you'd need to do that everywhere you have (s/* (s/alt ...)). Maybe if enough people run into performance problems, rep will be made public, or the regex generators will be optimized.#2016-06-1722:28seancorfieldThanks @leifp#2016-06-1723:58leongrapenthinAssume I parse with conform. Then I have functions that operate on the value returned by conform. I can't get a spec for the value returned by conform (so that I can spec said functions) - It seems like this could be automated though. Imagine (s/conform-spec ::my-spec) would return the spec of the return value of calling (s/confom ::my-spec 42)#2016-06-1800:08wilkerlucioI noticed (s/exercice number?) can generate some NaN entries on cljs, is that by design?#2016-06-1800:10leongrapenthinSo conform-spec as described above can apparently be implemented quite easily via (defn conform-spec [s] (s/and (s/conformer #(s/unform s %)) s))#2016-06-1800:17leongrapenthinBut it would probably be more valuable if a spec could give a spec of what its conform* returns#2016-06-1803:55wildermuthnIs there a changelog for Spec? Iā€™m looking at some of the recent commits to cljs.spec, and see that the docs have been updated. But would be great to know if there was another place to reference to see the updates.#2016-06-1804:05wildermuthnThe more I learn about Spec, the more I think it is going to be regarded one of the essential Clojure features, on par with immutable data. For instance, I found out about conformer, and not ten minutes later had the beginning of a solution that Iā€™ve never had for parsing JSON enums in a sane way. I shared this on #C03S1L9DN, but interested in getting feedback if Iā€™m going at this the wrong way:#2016-06-1808:37vikeriA question about spec and generators, is it able to generate data from a spec directly or does it generate data and test it against the spec? As an example I suppose that a generator for string will only generate strings, but what if you have some slightly more complicated predicates like strings with the length 3? Will it understand to only generate strings with the length three? I guess this would need some way of inverting an if statement, which sounds difficult. The main question in hence: How ā€smartā€ can the generator be?#2016-06-1810:21Alex Miller (Clojure team)Well it's not magic :)#2016-06-1810:23Alex Miller (Clojure team)If you have (s/and string? #(= (count %) 3)) the and generator generates based on the first pred, then filters based on the subsequent ones#2016-06-1810:24Alex Miller (Clojure team)So it will generate random strings and keep those that are length 3#2016-06-1810:25Alex Miller (Clojure team)That's probably a restrictive enough filter that you'll need to supply a custom generator#2016-06-1810:28Alex Miller (Clojure team)@wildermuthn: I list incremental changes for each release in the announcement notes in the Clojure mailing list#2016-06-1810:29Alex Miller (Clojure team)@wildermuthn: not clear to me what you're asking about with conformers above #2016-06-1812:20ghadi@alexmiller: s/every has a tiny typo in it causing this:
user=> (s/valid? (s/every #{:foo}) [:foo :foo])
true
user=> (s/valid? (s/every #{:foo}) 42)
true  ;; incorrect
#2016-06-1812:21ghadihttps://github.com/clojure/clojure/blob/master/src/clj/clojure/spec.clj#L1095#2016-06-1812:21ghadis/:invalid/::invalid#2016-06-1812:24Alex Miller (Clojure team)Thx#2016-06-1814:31nhaWhat is wrong with the following ?
(defn add []
  "a")

(s/fdef add
        :ret int?)

(s/instrument #'add)

(add) ;;=> "a"

#2016-06-1814:32nhaI was expecting an error on the return value.#2016-06-1814:33minimalInstrument was changed to only check :args#2016-06-1814:35nhaOoh so how do I check the return value ? (ie. what would be a simple test here ?)#2016-06-1814:37nhaAlso is there a constraint on the order of things ? Can I s/fdef before defn-ing a function ? Can I s/instrument-all when I want ?#2016-06-1814:41minimalFor now you have to use the functions from the clojure.spec.test namespace like check-var#2016-06-1814:43nhaThanks I will dig into it. Those can do "normal" testing as well right ? ie. not generative testing#2016-06-1814:43minimalYou can s/fdef before. The docs say the instrument fns are idempotent#2016-06-1814:43nhaThanks šŸ˜„#2016-06-1814:44nhaAah I read that but assumed it meant it can be called many times#2016-06-1814:44minimalYeah#2016-06-1814:46nhaAny reason the :ret has been removed ? Will it come back at some point ?#2016-06-1814:46eggsyntax@nha: see https://groups.google.com/forum/#!msg/clojure/RLQBFJ0vGG4/UGkYS7U_CQAJ#2016-06-1815:03nhaSo it looks like clojure.spec.test is only for generative testing, I still have to use whatever I was using if I want to explicitly give input arguments to the tested function (ex. for things too hard to generate). Correct ?#2016-06-1815:17gfredericksnha: sounds right#2016-06-1820:56arohnerFYI, Iā€™m reliably reproducing the ā€™no such var encore/bytes?ā€™ thing that @seancorfield reported the other day#2016-06-1820:56arohnerI donā€™t understand the cause, but Iā€™m experiencing the same behavior#2016-06-1821:08seancorfieldThere's an updated Encore that fixes that. Some interaction with cljx / do / defn in that case - and conflicting with a (new) core function. #2016-06-1821:09seancorfieldBut I also eliminated every single dependency conflict and that solved the other, similar problems I had after that. #2016-06-1821:09arohneryeah šŸ˜ž#2016-06-1821:09seancorfieldSo I'm not sure what the root cause is at this point. #2016-06-1821:09arohnerIā€™m hoping the clojure bug gets fixed#2016-06-1821:10arohneris there a way to re-use fdef or fspec on other functions?#2016-06-1821:10arohnerIā€™d like to say ā€œthese two defns have the same fnspec"#2016-06-1821:14arohnerah, looks like I can just (s/def ::name (s/fspec)) (s/def ::foo ::name)#2016-06-1821:15bbloom@arohner: if two functions have the same fspec, wouldnā€™t they just be the same function? (or your fspec contains no interesting properties?)#2016-06-1821:16arohner@bbloom no, there are lots of things that have the same spec that are interesting#2016-06-1821:16arohneri.e. simple math#2016-06-1821:16arohner+, *, - all take two numbers, return a number#2016-06-1821:16arohnerin this case, parsing#2016-06-1821:16arohnertake some input, return a parsed value and the rest of the stream#2016-06-1821:17bbloomseems like youā€™d want to reuse a spec for args and result, but a different spec for the property relating args to results#2016-06-1821:17bbloomthere might be a missing abstraction here#2016-06-1821:18bbloomweā€™ve got specs for data and specs for functions which contain arg spec, return spec, and relation spec, but we donā€™t have a spec for args + return (signature?) without relation#2016-06-1821:18arohneryes, Iā€™d like to get more specific in the future#2016-06-1821:18arohnerand I still have an open challenge out to ā€œcorrectlyā€ spec clojure.core/map#2016-06-1821:19arohnerI assert it canā€™t reasonably be done right now#2016-06-1821:19bbloomcorrectly = ?#2016-06-1821:19arohnerā€˜map takes a fn of type X -> Y, and a seq of X, and returns a seq of Y'#2016-06-1821:20bbloomah yeah, spec lacks logic variables#2016-06-1821:20bbloomcanā€™t do context sensitive parsing#2016-06-1821:21eggsyntax@arohner: how could you verify that itā€™s a fn of type X -> Y in a language that doesnā€™t do type declaration?#2016-06-1821:22eggsyntaxIs it sufficient to hand it an X, designate whatever comes out as Y, and then go from there?#2016-06-1821:22arohnerwell, I can spec that f is X->Y now, and spec that coll is coll-of X#2016-06-1821:22eggsyntaxAh, gotcha.#2016-06-1821:23eggsyntaxMakes total sense; somehow I was assuming f was unspecced.#2016-06-1821:23arohnerbut that doesnā€™t narrow down mapā€™s return spec#2016-06-1821:23arohnerbecause mapā€™s spec is just seq or whatever#2016-06-1823:42zpinterhello everyone! wondering if anybody knows of a clean way to define the spec for a map and its keys without having to repeat the list of keys again for the map spec#2016-06-1823:43zpintermy first attempt was something like this:
(def style-keys
   [
    (s/def ::padding ::dimen-spec)
    (s/def ::padding-left ::dimen-spec)
    (s/def ::padding-right ::dimen-spec)
    (s/def ::padding-top ::dimen-spec)
    (s/def ::padding-bottom ::dimen-spec)])

(s/def ::style (s/keys :opt style-keys))
#2016-06-1823:43zpinterhowever, s/keys doesn't seem to like having :opt passed as a symbol (presumably since it's a macro)#2016-06-1823:44zpinterthis version compiles and works fine
(s/def ::padding ::dimen-spec)
(s/def ::padding-left ::dimen-spec)
(s/def ::padding-right ::dimen-spec)
(s/def ::padding-top ::dimen-spec)
(s/def ::padding-bottom ::dimen-spec)

(s/def ::style (s/keys :opt [::padding ::padding-left ::padding-right ::padding-top ::padding-bottom]))
#2016-06-1823:44zpinterhowever, every time I add a new key, I have to update two different places (which I'd ideally like to avoid in this case, since there'd be a lot of keys)#2016-06-1823:45bbloomzpinter: iā€™m far from an expert (i havenā€™t tried spec properly yet) but i think you can just use s/and, right?#2016-06-1823:45zpinterthe recommendation from #C03S1L9DN was to make my own macro, which seems like a good approach, just wondering there was anything built-in#2016-06-1823:47bbloomoh, nvmd, i think i misunderstood what you were asking for#2016-06-1823:49bbloombut yeah, iā€™d guess making your own macro is the way to go - but depending on what youā€™re doing, the redundancy might make sense in order to allow decomposition later#2016-06-1823:49bbloomfor example, you may want to limit the styles on some thing to only color related styles or only spacing styles or what not#2016-06-1823:50bbloomso youā€™d do (s/def ::paddingā€¦, (s/def ::padding-left ... and then group those with s/and to form ::padded#2016-06-1823:50bbloomthen you can have (s/def style (s/and ::padded ::colored ::etc-etc-etc#2016-06-1823:51zpinterhmm... does s/and work with s/keys?#2016-06-1823:51zpinterI guess you could use s/and to combine multiple (s/keys :opt values....#2016-06-1823:51bbloomiā€™d be surprised if it doesnt, but like i said - i havenā€™t installed the alpha yet to try it šŸ˜›#2016-06-1823:55zpinterwill keep investigating, thanks @bbloom#2016-06-1823:56eggsyntax@zpinter: I'd be interested in seeing what you come up with šŸ™‚#2016-06-1902:10Alex Miller (Clojure team)You can compose any kind of spec with and#2016-06-1902:11Alex Miller (Clojure team)Note also that s/keys validates all keys in the map regardless of what's in req or opt#2016-06-1902:12Alex Miller (Clojure team)So you don't technically have to list them as opt keys#2016-06-1910:25akielHas someone thought about specs for ex-data?#2016-06-1913:30gfredericks(s/fspec :args ... :ret ... :throws ...)#2016-06-1913:57cigitiaIn clojure.spec, is there a way to associate custom generators with your own functions, the way that clojure.spec.gen/gen-for-pred and gen-builtins associate generators with various clojure.core functions?#2016-06-1915:00gfrederickscigitia: you can make a spec that has a particular generator associated with it using spec/with-gen#2016-06-1915:01cigitia@gfredericks: Yes, though that requires either registering the spec returned by with-gen under a keyword, or using with-gen inline every time you use the function.#2016-06-1915:02cigitiaIā€™m wondering whether itā€™s possible to have generators when using functions directlyā€”#2016-06-1915:02cigitiaLike how int? has a generator even when you use int? directly as a predicate, due to the way gen-for-pred works.#2016-06-1915:16gfredericksI think you can say (def int? (spec/with-gen #(instance? Integer %) gen/large-integer))#2016-06-1915:16gfredericksI might be lying#2016-06-1915:59Alex Miller (Clojure team)There is more stuff coming in this area soon#2016-06-1916:16akiel@gfredericks: is :throws discussed somewhere? I canā€™t find something about it.#2016-06-1916:57gfredericksakiel: no I just made it up#2016-06-1916:57gfredericksI was imagining an API that related to what you were asking about
#2016-06-1917:26akiel@gfredericks: Yes thatā€™s exactly what Iā€™m after. @alexmiller What do you think about exception data specs? As far as I see it, exception data keys are not even documented normally.#2016-06-1917:33Alex Miller (Clojure team)We are not going to spec exceptions#2016-06-1917:34Alex Miller (Clojure team)You could of course create specs for the ex-info map keys and I can see that possibly useful in functions that receive ex-info data after an exception has happened (to generate user errors or error pages etc)#2016-06-1917:37akielDo you think that the shape of the ex-data should be public API of a function or do you are more in line with Joshua Bloch: Effective Java Item 57: Use exceptions only for exceptional conditions were he suggests that exception should not be used for decision making.#2016-06-1918:20Alex Miller (Clojure team)Yes#2016-06-1918:21Alex Miller (Clojure team)Unless you're abusing it for performance :)#2016-06-1919:24akiel@alexmiller: sorry for asking further - yes public or yes not-public?#2016-06-1919:30hiredmannot specing execeptions makes a lot of sense#2016-06-1919:34hiredman1. spec is not a type system 2. spec is primarily about specifying data and exceptions are not data, they are weird control flow operation in a language 3. specs for functions specify what valid argument data looks like, and given that valid argument data, what a valid result looks like, and that is how the generative stuff works, and in that model there is no place for exceptions#2016-06-1919:35hiredmangiven a function F, if F throws E, to ensure correctness you want to look at all callers of F, and ensure that they handle E correctly, which is exactly the opposite of how the generative testing works (if I understand correctly)#2016-06-1919:36akiel@hiredman: in case APIā€™s donā€™t use exceptions for control flow Iā€™m with you. But than we donā€™t need ex-info and ex-data at all and should stop build APIā€™s which put data into exceptions#2016-06-1919:38akiels/conform is a good example for an API not using exceptions - it returns ::s/invalid#2016-06-1919:40bhaumanIs there a way to validate that you have defined all namespaced keywords in the your spec definitions? And get a warning if some are missing?#2016-06-1919:41hiredmanwrite a predicate that checks the spec registry#2016-06-1919:46bhaumanhmmm ... so say compose a macro over spec/keys, capture the key args and and then warn if if the keys are not present in the registry? or were you thinking something else?#2016-06-1919:47bhaumanif you wanted to verify everything with a single call though it seems like you would have to parse the describe of each of the registry members#2016-06-1919:47hiredman
user=> (s/valid? (s/map-of #(if (and (keyword? %) (namespace %)) (contains? (s/registry) %) true) ::s/any) {:a 1})
true
user=> (s/valid? (s/map-of #(if (and (keyword? %) (namespace %)) (contains? (s/registry) %) true) ::s/any) {::a 1})
false
user=> (s/def ::a ::s/any)
:user/a
user=> (s/valid? (s/map-of #(if (and (keyword? %) (namespace %)) (contains? (s/registry) %) true) ::s/any) {::a 1})
true
user=> 
#2016-06-1919:49bhaumancool yep, but was thinking of a different use case, where you are inspecting the self referential integrity of the registry itself#2016-06-1919:50hiredmannot sure I follow? that s/map-of spec will fail if given a map with namespaced keys that are not speced#2016-06-1919:51hiredmanoh#2016-06-1919:51hiredmanI think I get it now#2016-06-1919:52hiredmanyeah, spec seems to be slightly later binding than clojure is#2016-06-1919:53bhaumanyeah, it's doable, but I think it would be nice to have some help beyond describe#2016-06-1919:54hiredmanyou'd like ::foo to throw an error earlier then when you run the spec, basically, like clojure does with vars#2016-06-1919:55bhaumanI'd like to do ( check-missing (clojure.spec/registry))#2016-06-1919:56bhaumanto help me in the repl so that I can see if I forgot something#2016-06-1919:58hiredmanthat might not be currently possible, I don't think specs have a way to get all the specs they depend on#2016-06-1919:58hiredmanwhich might be an interesting enhancement#2016-06-1919:59bhaumanexactly, it would be nice to have a spec walker#2016-06-1920:03bhaumanor at least a corollary to describe that returns the actual internal data#2016-06-2002:05Alex Miller (Clojure team)there will likely eventually be a deep-describe, which will require something like that#2016-06-2013:54alqvistI feel that the usefulness of s/inst-in is hampered by cljs incompability. Any words of wisdom?#2016-06-2014:43alqvistOr I could just use the latest clojurescript - Never mind#2016-06-2015:04angeliniis there anyway to reproduce the behaviour of alpha-5ā€™s s/instrument without using spec.test? The rational of only checking :args makes sense, but it was really useful when testing code at the REPL to verify that :ret and :fn weā€™re correct.#2016-06-2015:32adamfreywhen you have a test.check generator that is failing a such-that predicate 100 times, is it possible to print out the failing values, just to make the generator less opaque? Nothing stood out to me in the source code for doing that.#2016-06-2017:39akielAre there plans to support some kind of validation context? I think about a spec for JSON Web Tokens which are signed. Such a validation would need the public key as context. Or do I stretch the intended usage patterns here?#2016-06-2019:41Alex Miller (Clojure team)@adamfrey nothing explicit but you can gen/sample the s/gen of some simpler part to get an idea. It's usually the first pred in an s/and that is creating things that won't pass the later preds in the s/and#2016-06-2019:41Alex Miller (Clojure team)@akiel no plans for that#2016-06-2019:42Alex Miller (Clojure team)Prob a sign you are taking things too far :)#2016-06-2020:10akiel@alexmiller thanks#2016-06-2020:38Alex Miller (Clojure team)@cigitia: this channel is for spec, please take to #off-topic #2016-06-2020:40cigitiaMy apologies; accidentally sent to wrong Slack team; meant to sent to personal friends#2016-06-2108:07borkdudemakes me wonder about the spec for get-in: (get-in {:a "lol"} nil) ;;=> {:a "lol"}#2016-06-2108:08borkdudeI guess it could be correct if nil is regarded as the empty sequence#2016-06-2111:55akiel@borkdude: nil is the empty sequence šŸ™‚ I think it should be possible to write a spec for get-in.#2016-06-2114:33jannisIs there any chance of opts being added to clojure.spec.test/run-(all-)tests? I have a couple of functions that exceed the GC limit if I run 100 tests against them. 25 or 50 is fine, so I'd like to set :num-tests globally instead of having to use check-var for every single function I want to test.#2016-06-2114:35jannisAn alternative idea would be per-`fdef` opts perhaps?#2016-06-2114:58jannisOr a way to exclude specific functions from run-(all-)tests?#2016-06-2115:06akiel@jannis why are your functions are so memory hungry? do they have side effects or are the inputs to big? for the latter, I would rather optimize the generators.#2016-06-2115:07jannisGood point, it's probably the generators generating big inputs. I'll double-check.#2016-06-2115:09jannisAlthough... the generator is already generating relatively small data.#2016-06-2115:10jannisThat's the function: https://github.com/workfloapp/macros/blob/jannis/clojure-spec/src/main/workflo/macros/props.cljc#L324 and this is the generator: https://github.com/workfloapp/macros/blob/jannis/clojure-spec/src/main/workflo/macros/props.cljc#L90#2016-06-2115:10angusiguesssuch-thats are a good thing to look out for.#2016-06-2115:12jannisI have overriden almost all generators because my data specs seem to be too complex to use the default generators.#2016-06-2115:12angusiguessOne thing we ran into was the following:#2016-06-2115:12angusiguessIf there's a significant distance between your generator and your validation predicate that generator tends to perform poorly.#2016-06-2115:13jannisOf course, that makes sense. That's why I overrode most of the generators so that they don't have to be tried a lot to have conforming data generated.#2016-06-2115:15angusiguessI haven't tried it, but does increasing the heap size help?#2016-06-2115:16jannisWithout knowing much about it, GC limit exceeded to me sounds like it's generating too much data too quickly for GC to catch up. Let me re-run the tests with a heap size of 2GB.#2016-06-2115:17angusiguessGC Limit exceeded usually means that the ratio of GC pauses to 'useful' computation is quite high.#2016-06-2115:18angusiguessSo more heap space can reduce pauses.#2016-06-2115:18jannisOk#2016-06-2115:23jannisIt's also a test performance issue by the way. With 100 tests, that single function test runs for ~5-10 minutes until it errors out with the GC limit. I'm aiming for a fast test suite, so being able to reduce the number of tests globally - or for this function - would help.#2016-06-2115:56jannisIncreasing the heap size avoids it failing after 5-10 minutes. Instead it has now been running for more than half an hour. šŸ˜‰#2016-06-2115:58akiel@jannis I have you source code open. But I never used boot before and Iā€™m using cursive - so its not so easy for me to get it running.#2016-06-2115:58jannis@akiel: Once you have boot installed, all you should need is boot test-once#2016-06-2116:20akiel@jannis: the generator of ::property-spec is already very slow. Try (s/exercise ::properties-spec 15) and than with 20. The thing is that generated samples become bigger and bigger if you generate more. test.check has some internal size thing. You have to improve the generator.#2016-06-2116:21jannis@akiel: Cool, thanks for investigating. I'll see what I can do. šŸ™‚#2016-06-2121:15cigitiaIs there any particular reason that the sequence regex ops do not recognize strings as seqables of chars? For instance, ā€abā€ does not conform to (s/cat :a #(= % \a), :b #(= % \b)), while [\a \b] does.#2016-06-2121:19cigitiaIn addition, are there plans for negative-lookahead regex ops, like those from &? For instance, in PEGs / parsing expression grammars, & has a negative counterpart called !Ā . In particular, an end-of-sequence regex op would be very usefulā€”it could be called ::end, and it could be equivalent to (! ::any). As far as I can tell, this isnā€™t currently possible.#2016-06-2122:25bfabrylol, @cigitia your slack-fu is struggling#2016-06-2122:25Alex Miller (Clojure team)@cigitia: you're posting news links again#2016-06-2122:25cigitiaArgh, sorry#2016-06-2122:26Alex Miller (Clojure team)@cigitia There are good tools for string regex. Spec is never going to be great for that and there are no plans to use it for that#2016-06-2122:27Alex Miller (Clojure team)No plans for negative look ahead ops #2016-06-2122:27cigitiaOkay, thanks#2016-06-2122:27Alex Miller (Clojure team)They are generally not needed for the kinds of things we expect you to do with spec regex#2016-06-2122:32cigitiaI had actually been working on a PEG library that could create grammars for generic EDN data, including but not limited to strings, before Spec was announced#2016-06-2122:34cigitiaOnce Spec was announced, it seemed like there wasn't much room for such a library anymore, and so I thought about instead making a library that would utilize specs and convert them into parser transducers#2016-06-2122:35cigitiaMaybe there's still room for the first idea, though, then, hm#2016-06-2202:59Oliver GeorgeI was expecting s/conform to return fully "conformed" data but it doesn't seem to work like that.#2016-06-2203:00Oliver George
=> (s/conform (s/coll-of (s/conformer (fn [x] (println "I am conforming" x) (str x))) []) ["1" 2 :3])
I am conforming 1
I am conforming 2
I am conforming :3
["1" 2 :3]
#2016-06-2203:01Oliver Georgeit uses the conformer in order to check the data but returns the un-conformed data. Seems to be "non-recursive"#2016-06-2203:02Oliver GeorgeIs that intended behaviour? Perhaps I'm attempting to use it incorrectly - as a data verification & transformation tool.#2016-06-2203:40Alex Miller (Clojure team)The problem here is that coll-of samples its contents and doesn't flow the entire contents#2016-06-2203:40Alex Miller (Clojure team)There are some things in this area changing in next alpha#2016-06-2203:57Oliver GeorgeThanks Alex, I think I see similar behaviour with map-of.#2016-06-2204:02Alex Miller (Clojure team)Yes#2016-06-2204:02Alex Miller (Clojure team)every and every-kv are new things already in master but not yet released#2016-06-2204:03Alex Miller (Clojure team)One of those sets will conform all args, although I'm not sure which#2016-06-2204:08Oliver GeorgePerfect. Thanks.#2016-06-2211:38akielIs there a way to spec values which will be conveyed over a channel or returned by a promise?#2016-06-2211:53Alex Miller (Clojure team)nothing built in yet#2016-06-2211:54Alex Miller (Clojure team)ultimately I expect there will be some things provided for core.async#2016-06-2211:55Alex Miller (Clojure team)you could supply a map transducer to an async channel that called s/conform or something like that though#2016-06-2212:09akielsome kind of spec wrapper which takes a spec and validates channel values on the go would be nice#2016-06-2212:11akielI only like to be sure that nothing speaks at this conceptionally before spec gets final#2016-06-2214:02eggsyntaxtest.check has a fn (`fmap`) that lets you call an arbitrary fn on the results of a generator. But is there any way to create a generator that just calls an arbitrary fn? I haven't found one. The only solution I've found so far is to do (fmap (fn [_] do-what-I-want) some-arbitrary-gen), but it's pretty ugly to put in a generator and then ignore what it generates.#2016-06-2214:04eggsyntax(I understand that shouldn't be a typical use, but it'd be good to have an escape hatch for cases where you have an existing source of randomness that you don't want to have to duplicate)#2016-06-2214:43akiel@eggsyntax: As you said already, itā€™s not the typical use to implement the root generator yourself. You have to understand that test.check needs to control the generated values in a way to get shrinking work. So I have no answer to your question only the suggestion: Donā€™t do this! šŸ™‚#2016-06-2214:44eggsyntaxHeh, fair enough.#2016-06-2214:46eggsyntaxAlthough in this case I'm still gonna do it...again, existing (complex) source of domain-specific randomness.#2016-06-2214:46eggsyntaxMaybe I'll have to figure out how to make a Rose tree from it...#2016-06-2214:47eggsyntaxI realize there's a likely cost in terms of shrinking, though.#2016-06-2218:07Alex Miller (Clojure team)@eggsyntax: have you looked at return ?#2016-06-2219:22eggsyntax@alexmiller: I have, but it looks like that'll only generate a constant value, right? I need something that'll call a fn every time it needs a new one.#2016-06-2219:39angusiguesstest.check does have a no-shrink qualifier too I believe.#2016-06-2219:39angusiguessIf the value doesn't benefit from shrinking (like uuids for ex)#2016-06-2223:06bostonaholichas anyone come up with a reasonable clojure.spec/def for a database argument?#2016-06-2223:06bostonaholicall Iā€™m using at the moment is ::s/any#2016-06-2223:06bostonaholic
(defn get-user [db id] ...)

(s/def ::database ::s/any)

(s/def ::user (s/keys :req [::id ::email]))

(s/fdef get-user
        :args (s/cat :db ::database :id ::id)
        :ret ::user)
#2016-06-2223:07bostonaholicbut obviously that wonā€™t work with generators#2016-06-2223:08bostonaholicIā€™m not sure thereā€™s much value here, but interested in hearing what a possible solution might be#2016-06-2223:40eggsyntax@bostonaholic: what should the DB argument look like?#2016-06-2223:41bostonaholicwell, itā€™s a datomic db#2016-06-2223:42eggsyntaxAh, ok, you hadnā€™t mentioned it was datomic.#2016-06-2223:42bostonaholicyeah, sorry#2016-06-2223:42bostonaholicthatā€™s why Iā€™m thinking this isnā€™t a good idea#2016-06-2223:44Oliver George@alexmiller picking up on my question about fully conforming yesterday. I tried every-kv this morning (cljs release) and I see the same behaviour:#2016-06-2223:45Oliver George
=> (do
  (s/def ::tree (s/or
                  :branch (s/every-kv keyword? ::tree)
                  :value ::s/any))
  (s/conform ::tree {:A {:B 2}}))

[:branch {:A {:B 2}}]
Was hoping for [:branch {:A [:branch {:B [:value 2]}]}] or similar.
#2016-06-2223:45eggsyntaxYou can certainly check the type using a predicate built from instance?#2016-06-2223:46eggsyntaxThat might be enough for your needs.#2016-06-2223:47bostonaholicyeah, thatā€™s the only other thing I could think of#2016-06-2223:49eggsyntaxThat's enough to show it's a legit database...you could also do something like
(s/and #(instance? datomic.db.Db %)
                       (s/keys :req-un [::key1 ::key2]))
with further specs for those keys. (don't have a datomic-using project right in front of me, I'm improvising on the type, and assuming that it's a record)
#2016-06-2223:50eggsyntax(which could be dead wrong šŸ™‚ )#2016-06-2223:51bostonaholicyouā€™re right#2016-06-2223:56Alex Miller (Clojure team)@olivergeorge its not done yet#2016-06-2223:56eggsyntax@bostonaholic: ooh, quite a few keys there, huh? (:id :memidx :indexing :mid-index :index :history :memlog :basisT :nextT :indexBasisT :elements :keys :ids :index-root-id :index-rev :asOfT :sinceT :raw :filt)#2016-06-2223:57bostonaholicheh, yeaahhhhhhhhh#2016-06-2223:58Alex Miller (Clojure team)@olivergeorge: I believe when it is done map-of will be what you want #2016-06-2300:01Oliver GeorgeThanks again Alex. I suspected that might be the case.#2016-06-2301:08eggsyntax@alexmiller: is there a roadmap online anywhere for what changes are definitely coming? Or even something that discusses some of them?#2016-06-2301:34Alex Miller (Clojure team)no#2016-06-2301:34eggsyntaxOh well šŸ™‚#2016-06-2301:35Alex Miller (Clojure team)there are definitely changes around instrument and the c.s.test namespace coming soon#2016-06-2301:35Alex Miller (Clojure team)and map-of and coll-of work is under way#2016-06-2301:35Alex Miller (Clojure team)there are a bunch of other things on the list but I donā€™t how that will all pan out#2016-06-2301:36eggsyntaxExcellent! I donā€™t think Iā€™ve actually said this to yā€™all before about spec, but thanks šŸ™‚. Iā€™ve been working with it for about a week, and itā€™s just terrific work.#2016-06-2301:36eggsyntaxJust how powerful it is didnā€™t even fully sink in until Iā€™d dived into it for a while.#2016-06-2301:36eggsyntax^ @alexmiller#2016-06-2301:39seancorfieldIf I have (s/keys :req-un [::foo ::bar]), am I correct that something in clojure.spec will expect ::foo and ::bar to exist as specs, for some operations? I was trying to add stuff to clojure.java.jdbc.spec to support generative testing (by writing custom generators for db-spec, java.sql.Connection, and java.sql.PreparedStatement etc) and I started getting errors that the "unqualified" keys in some of my specs didnā€™t resolve...#2016-06-2301:40seancorfieldThat wasnā€™t obvious to me from the documentation about s/keys and the -un keys in maps ā€” but it makes sense from the p.o.v. of actually trying to generate maps to pass into functions whose arguments are specā€™d as s/keys...#2016-06-2301:53seancorfield(FWIW, folks can look at the work-in-progress experiment on a gen-testable java.jdbc spec here https://github.com/clojure/java.jdbc/blob/spec-gen/src/main/clojure/clojure/java/jdbc/spec.clj )#2016-06-2302:31Alex Miller (Clojure team)if youā€™re trying to gen keys in a map, then yes you would need them to have a definition#2016-06-2302:31Alex Miller (Clojure team)otherwise how would you gen them?#2016-06-2302:33Alex Miller (Clojure team)are you getting this while genā€™ing or during something else?#2016-06-2303:01seancorfieldDuring genā€™ing. And, yes, it makes sense. It just wasnā€™t obvious from the docs...#2016-06-2303:02seancorfieldBecause I missed this (very important) phrase "These variants specify namespaced keys used to find their specification"#2016-06-2303:04seancorfieldI just didnā€™t associate the ::first-name etc with the specs given two examples above since they were "unqualified keywords" in my mind.#2016-06-2303:06seancorfieldNot sure what to suggest to make that clearerā€¦? Perhaps repeat the specs in the :unq/person example?
(s/def ::first-name string?)
(s/def ::last-name string?)
(s/def ::email ::email-type)

(s/def :unq/person
  (s/keys :req-un [::first-name ::last-name ::email]
          :opt-un [::phone]))
#2016-06-2303:06seancorfield(and now I notice that ::phone is not provided as a spec anywhere on that page)#2016-06-2303:07seancorfieldObvious in hindsight, of course.#2016-06-2308:34Oliver GeorgeThis is a thought in passing. Often we want confidence that some data is fully realized. It seems like a challenging case. We have realized? as a test and could write a recursive ::realized spec. It is challenging since displaying the data in reporting might "realize" it. So the test modifies the data.#2016-06-2308:35Oliver GeorgePerhaps not a clojure.spec domain issue... data is valid, it just doesn't exist yet!#2016-06-2308:35Oliver GeorgeWe've seen cases where lazy and dynamic aspects cause surprises. Right now we're considering a class of problems where delayed evaluation makes debugging harder.#2016-06-2308:36Oliver GeorgeJust raising in case it's something of interest as spec matures.#2016-06-2308:48thomasdeutschhow can i spec this structure?
{::items-by-id {"id-1" {::id "id-1"
                  ::title "first-item"} 
          "id-2" {::id "id-2" 
                  ::title "second-item"}}}
i would like to spec that every item ::id is the same id as its key on the parent map.
#2016-06-2310:50wagjoAre optional docstrings for s/def (CLJ-1965) something you'd like to have or were they left out intentionally?#2016-06-2312:01Alex Miller (Clojure team)@wagjo: still on the table#2016-06-2312:02Alex Miller (Clojure team)@olivergeorge: I donā€™t expect spec is going to add anything re lazy/realized data#2016-06-2312:02Alex Miller (Clojure team)@seancorfield in my next guide pass I will see if I can do anything to bring that out#2016-06-2317:10seancorfieldMuch appreciated https://clojurians.slack.com/archives/clojure-spec/p1466683358000870#2016-06-2320:48uwoIs there a suggested way to maintain metadata about which keys a spec is concerned:
(s/def :bill/delivery inst?)
(s/def :bill/pickup inst?)
(s/def :bill/pickup-gt-delivery (fn [{:keys [bill/delivery bill/pickup]}] (> pickup delivery)))
(s/def ::bill (s/and (s/keys :req [:bill/delivery :bill/pickup])
                     :bill/pickup-gt-delivery))

;some lookup
{:bill/pickup-gt-delivery [:bill/delivery :bill/pickup]}
#2016-06-2321:40Alex Miller (Clojure team)no, I donā€™t think so#2016-06-2321:40Alex Miller (Clojure team)
user=> (s/explain-data ::bill {})
{:clojure.spec/problems {[] {:pred [(contains? % :bill/delivery) (contains? % :bill/pickup)], :val {}, :via [:user/bill], :in []}}}
#2016-06-2321:41Alex Miller (Clojure team)The explain-data will tell you what itā€™s missing in the form of predicates but thatā€™s not what youā€™re asking#2016-06-2321:41Alex Miller (Clojure team)(btw, as of the next alpha, youā€™ll be able to rewrite pickup-gt-delivery as (fn [{:bill/keys [delivery pickup]}] (> pickup delivery)) )#2016-06-2322:07uwo@alexmiller: cool. thanks for the response!#2016-06-2402:22bsimaHow do you spec protocol functions? Is it the same as spec-ing a regular function? (I haven't actually tried it yet...)#2016-06-2402:23bsimaI tried to spec a multimethod using fdef a few days ago and it didn't work but I didn't have time to figure out why#2016-06-2402:37seancorfield@bsima: Read this issue for some information about spec and protocols http://dev.clojure.org/jira/browse/CLJ-1941 (regarding instrumentation)#2016-06-2402:42bsimamm thanks seancorfield#2016-06-2415:11jannisHow do I best write a generator for :args? It seems like even if I use (gen/cat <sequence-generating generators>) it generates a sequence that is then passed to :args as the first argument, instead of "splicing" it into the macro/function call.#2016-06-2415:13akiel@jannis Is it not sufficient for you to write individual generators? Do you have many dependencies between your args?#2016-06-2415:16jannis@akiel: This is my example: https://gist.github.com/Jannis/51ebdd54e10074fd9c574a54dd8421c9#2016-06-2415:17jannisThe args are fairly flexible and somehow what gets generated for name and & forms is not quite what it should be. Perhaps my :args spec is wrong (thinking var args perhaps need special treatment)?#2016-06-2415:21jannisEven if I just use (s/cat :name ::command-name) and drop the & forms argument in the macro, check-var complains with "Wrong number of args (1) passed to: command/defcommand". Something there is odd.#2016-06-2415:22akielYes (s/cat :name ::command-name) generates only one argument#2016-06-2415:23akielvaragrs is no special - you have already a s/* in your args#2016-06-2415:23akielwhat is ::command-name?#2016-06-2415:23jannisBut one argument should be fine for a macro that takes exactly one argument.#2016-06-2415:24jannisIt's a symbol? spec#2016-06-2415:24akielah yes you are right - one arg should be possible#2016-06-2415:25jannisI can paste the entire file once my laptop has rebooted.#2016-06-2415:30akielI get the same wrong number of args message with check-var#2016-06-2415:30akielbut if I convert the macro into a function, it works#2016-06-2415:30akielther may be some issues using char-var with macros#2016-06-2415:31akielI never specā€™ed macros before#2016-06-2415:31jannisYep, I never ran into this with specs for functions.#2016-06-2415:32jannisActually, this is a Clojure + ClojureScript macro, which will just pass on the arguments to a function that does the actual work. I could move the spec to that function instead.#2016-06-2415:32jannisThat could work around the problem.#2016-06-2415:34akielbut speccing macros is an interesting feature on its own - but check-var might simply not work with it#2016-06-2415:35jannisCould be?#2016-06-2415:36akielI simply donā€™t know. Iā€™ll hit this if a spec my first macro. But I have currently none.#2016-06-2415:36jannisOk šŸ™‚#2016-06-2415:37jannisMoving the spec to the function and turning the var-args into a (s/cat :name ... :forms (s/spec (s/cat :description ...))) subspec structure does the job šŸ™‚#2016-06-2415:37jannisAll the generated inputs are now useful.#2016-06-2415:40akielnice šŸ™‚#2016-06-2416:00kendall.buchananQuestion about the direction of namespaced keywords:#2016-06-2416:01kendall.buchananWith Clojurescript (Om appears to be pushing namespaced keywords) on the front end, specs in the middle, and Datomic on the end, namespaced keywords feel natural.#2016-06-2416:01kendall.buchananBut if youā€™re working with a Javascript front-end, that doesnā€™t support them. And a SQL database on the other. Whatā€™s one to do in the middle if you want to leverage specs?#2016-06-2416:01kendall.buchananJust :req-un everything?#2016-06-2416:02kendall.buchananOr create translation layers across the board?#2016-06-2416:02akiel:req-un works fine - the only thing you loose is automatic validation of all keys even if not present in req or opt#2016-06-2416:04kendall.buchananMakes sense. Iā€™ve hesitated on spec for fear that they canā€™t live in ā€œthe real worldā€ (a non-full-Clojure stack environment).#2016-06-2416:05akieleven in clojure itself - nobody can change existing apis - so un-namespaced keywords will be still there#2016-06-2416:06kendall.buchananK, thanks @akiel.#2016-06-2417:49seancorfield@kendall.buchanan: Iā€™m working on making namespaced keyword support in java.jdbc a lot easier. In the upcoming 0.6.2 release youā€™ll be able to specify a :qualifier "foo" option on calls to get :foo/col-name back from all queries. I need to figure out the final story for what happens if you push :foo/col-name into java.jdbc.#2016-06-2417:50kendall.buchanan@seancorfield: Thatā€™s very cool. I suspect work will need to be done on all ends of the stack to make namespaced keys more tenable for the average project.#2016-06-2417:56seancorfieldNow I look at it, I probably wonā€™t need to do anything for pushing namespaced columns into java.jdbc:
user=> (def db-spec {:dbname "mydb" :dbtype "mysql" :user "tester" :password "secret"})
#ā€™user/db-spec
user=> (j/query db-spec ["select * from status"])
({:id 1, :name "approved"} {:id 2, :name "new"} {:id 3, :name "rejected"})
user=> (j/query db-spec ["select * from status"] {:qualifier "status"})
({:status/id 1, :status/name "approved"} {:status/id 2, :status/name "new"} {:status/id 3, :status/name "rejected"})
user=> (j/insert! db-spec :status {:status/name "test"})
({:generated_key 13})
user=> (j/query db-spec ["select * from status"] {:qualifier "status"})
({:status/id 1, :status/name "approved"} {:status/id 2, :status/name "new"} {:status/id 3, :status/name "rejected"} {:status/id 13, :status/name "test"})
#2016-06-2417:58seancorfieldAnd due to a very recent change, even this works as expected:
user=> (j/insert! db-spec :status {:status/name "qualifier"} {:qualifier "result"})
({:result/generated_key 14})
#2016-06-2418:12Alex Miller (Clojure team)@jannis: check-var (very soon to be just ā€œtestā€ in the next alpha) is not going to work with macros#2016-06-2418:14Alex Miller (Clojure team)@kendall.buchanan: we are considering adding something in s/keys that lets you alias unaliased keys to arbitrary specs, but no guarantees#2016-06-2418:19jannis@alexmiller: Will there be an equivalent for fdef'ed macros?#2016-06-2418:19Alex Miller (Clojure team)you can fdef macros now#2016-06-2418:20Alex Miller (Clojure team)itā€™s just that check-var is not going to work to test them, afaik#2016-06-2418:23akielIs there a way to compose multiple s/key specs into one big one so that a map has to satisfy them all?#2016-06-2418:24bfabry@akiel: I wrote a macro that did that because I couldn't find anything. it was kinda painful#2016-06-2418:26akiel@bfabry: I need it to work with multi-spec because I have a big environment map were I have multiple subsets of varing keysets.#2016-06-2418:28bfabryI don't know enough about multi-spec to know if it would help, but here it is (very ugly) https://clojurians.slack.com/archives/clojure-spec/p1466110303000387#2016-06-2418:30akielAh s/form is nice. Never used it.#2016-06-2418:34akiel@bfabry: Thanks but your macro will only work with static s/keys specs.#2016-06-2418:34bfabryyup, I couldn't find a way around that because s/keys is a macro#2016-06-2418:39Alex Miller (Clojure team)@akiel you can just do s/and of multiple s/keys#2016-06-2418:39bfabry@alexmiller: that won't create good generators though#2016-06-2418:39Alex Miller (Clojure team)true#2016-06-2418:41akiel@alexmiller: Ok I only have to wrap all subsequent s/keys into a one-arg function and ignore the conforming value?#2016-06-2418:48akiel@alexmiller: ok it works#2016-06-2418:48Alex Miller (Clojure team)s/and will create a spec, not sure why you need to make it into a function#2016-06-2418:48akielNo my reading of the doc was just to loose.#2016-06-2418:48Alex Miller (Clojure team)the resulting conformed value should be the map#2016-06-2418:49akielI thought there would be always a predicate which gets the previous value. Bit this counts only for preds and not for specs.#2016-06-2418:51Alex Miller (Clojure team)not sure I understood that#2016-06-2418:54akielThe example is (s/and even? #(< % 42)) and the doc says: ā€œSuccessive conformed values propagate through rest of predicates.ā€ So I thought you can have one initial spec and than only predicates which are called with the previous conformed value.#2016-06-2418:54bfabry@alexmiller: out of interest is there any move to make some of these macros functions? a dynamic s/keys seems like it could be useful#2016-06-2418:55Alex Miller (Clojure team)@akiel: yes, although I think maybe youā€™re reading too much into the choice of ā€œspecā€ and ā€œpredicateā€ there#2016-06-2418:56Alex Miller (Clojure team)and combines specs which can be either predicates or other things#2016-06-2418:56Alex Miller (Clojure team)the conformed value flows through them#2016-06-2418:57akiel@alexmiller Ah ok. I see. The specs get also the previous conformed value and just use it to be able to return the whole thing at the end.#2016-06-2418:58Alex Miller (Clojure team)@bfabry: yes, thatā€™s possible. things are macros so that we capture forms for reporting purposes. but there may still be function forms as well (which would fill in under the macro forms)#2016-06-2418:59bfabrycool cool#2016-06-2419:19fentonI have a multi-spec. My in data can look like: {:method :search :search-term "blah" :auth "abc123" :gps-lat 3.4 :gps-long 5.3} or {:method :get-products :product-id 12345 :auth "abc123" :gps-lat 3.4 :gps-long 5.3}. My multi-spec already keys off the :method key. My data has some shared keys: :auth, :gps-lat, :gps-long and some keys that are not shared: :search-term and :product-id. Ideally I'd like to create a spec that merges two s/keys specs into a single s/keys spec, however s/cat creates a spec that expects two hashes in a list as opposed to a single hash with merged keys.#2016-06-2419:21Alex Miller (Clojure team)I talked through some options for something like this with @luke - maybe he could elaborate on where he ended up#2016-06-2419:24Alex Miller (Clojure team)iirc, he ended up with a spec for the common keys and separate specs for each variant#2016-06-2419:24Alex Miller (Clojure team)and then specs for each variant that anded common and separate#2016-06-2419:25Alex Miller (Clojure team)and then instead of multi-spec, itā€™s an s/or of the concrete variants#2016-06-2419:26Alex Miller (Clojure team)with the downside that itā€™s a closed system#2016-06-2419:26Alex Miller (Clojure team)with multi-spec, youā€™d have to repeat the keys in each variant#2016-06-2419:28fenton@alexmiller: okay, i'll look into what it looks like with s/or option and compare that to repeating in a multi-spec scenario. Thanks.#2016-06-2419:29fentonthe closed system you mention doesn't use s/keys?#2016-06-2419:41Alex Miller (Clojure team)no it would#2016-06-2419:43Alex Miller (Clojure team)common = (s/keys ā€¦) A = (s/and common (s/keys Aā€¦)) B = (s/and common (s/keys Bā€¦)) all = (s/or :a A :b B)#2016-06-2419:44Alex Miller (Clojure team)that doesnā€™t help you with the conforming question though#2016-06-2419:45fenton@alexmiller: k i'll give that a try.#2016-06-2423:52leongrapenthin@fenton @alexmiller I ran into this too, it seems a common scenario with Datomic "polymorphism" aka {:payment/amount 42, :payment/type :payment.type/stripe, :payment.stripe/payment-id 1234} - it would be great to have a merge-keys operation that takes two s/keys and merges their implementations - what do you think?#2016-06-2423:55Alex Miller (Clojure team)Talked about it with Rich today#2016-06-2423:55Alex Miller (Clojure team)We'll see#2016-06-2501:28seancorfieldAlpha 8 is looking like a great release, based on the commits over the last few days!#2016-06-2502:14Alex Miller (Clojure team)Gonna be a big one :)#2016-06-2502:32seancorfieldAny likely ETA @alexmiller ?#2016-06-2502:46Alex Miller (Clojure team)Soon#2016-06-2502:47Alex Miller (Clojure team)Didn't quite finish everything today as expected so prob early next week#2016-06-2505:29bbloomrather than complain about having to upgrade fipp for new reader/printer forms, iā€™ll instead boast: open source contribution opportunities! clojure 1.9 functionality PRs welcome šŸ˜‰#2016-06-2518:26fenton@alexmiller: small typo in spec guide. the description around function specs uses an int? predicate instead of integer?#2016-06-2518:30Alex Miller (Clojure team)That's correct as is#2016-06-2518:30fentonoh, hmm, couldn't find an int? predicate#2016-06-2518:31Alex Miller (Clojure team)int? is a new predicate in 1.9 that matches any fixed precision integer#2016-06-2518:31gfredericksso in particular not bigints?#2016-06-2518:31fentonah, good to know.#2016-06-2518:31Alex Miller (Clojure team)It was briefly called long?#2016-06-2518:31gfredericksbigints are the only difference between int? and integer??#2016-06-2518:32bronsaAFAICT yes#2016-06-2518:33gfredericksinteresting#2016-06-2518:33gfredericksI wonder why that's considered more useful than the other way#2016-06-2518:34bronsawell there are a number of constructs in clojure that don't work with idxs bigger than Long/MAX_VALUE so I'm guessing int? was provided to cover those use-cases in specs#2016-06-2518:35bronsaguarding against i.e. (range 0 (inc Long/MAX_VALUE))#2016-06-2518:35gfredericksinteresting#2016-06-2518:36bronsa(although there are also other constructs that don't work with idx bigger than Integer/MAX_VALUE like dotimes IIRC?)#2016-06-2518:36bronsanope dotimes accepts longs#2016-06-2518:37bronsaah, I'm thinking of for and doseq, they're constrained by Collection.count returning int#2016-06-2519:08Alex Miller (Clojure team)Yes widening from just long to other fixed precision was the reason#2016-06-2519:08Alex Miller (Clojure team)In particular to capture int#2016-06-2519:11Alex Miller (Clojure team)Conceptually we would prefer if users primarily focus not on the Java types but on integer fixed vs integer arbitrary precision as the main "types"#2016-06-2519:40mathias_dwprobably a dumb question, but Iā€™m having to call s/instrument-all every time I make a change to the code or the specs. Is there a way of automating that, or is my workflow simply wrong?#2016-06-2519:42mathias_dw(or is it maybe some nrepl/cider-related thing?)#2016-06-2519:47Alex Miller (Clojure team)You shouldn't have to do that #2016-06-2519:47mathias_dwok, thanks. Iā€™ll try without the cider/nrepl stuff#2016-06-2519:48Alex Miller (Clojure team)That was true of the very first release but was fixed early on#2016-06-2519:48mathias_dwIā€™m using alpha5#2016-06-2519:49Alex Miller (Clojure team)Yeah shouldn't be a problem #2016-06-2519:49Alex Miller (Clojure team)Must be something in the workflow#2016-06-2519:49mathias_dwok, thanks. Will try to pinpoint it and raise it in the relevant place#2016-06-2520:45gfredericksI'm having trouble with a recursive spec on master; not a problem on alpha7: https://www.refheap.com/120803#2016-06-2520:45gfredericksI scanned the commit messages and didn't see anything suspicious#2016-06-2520:46gfredericksif I had to guess I'd say it's something to do with coll-of suggesting that there's ambiguity#2016-06-2520:48Alex Miller (Clojure team)That code is brand new and on my plate to test more on Monday morning so I will def look at it#2016-06-2520:49gfrederickscool, thanks#2016-06-2607:03seantempestais there a way to test if a spec exists for a namespaced key?#2016-06-2611:05Alex Miller (Clojure team)There is a 'registry' function #2016-06-2611:05Alex Miller (Clojure team)In master for the next alpha there will be 'get-spec'
#2016-06-2612:41jannisNeat#2016-06-2618:35fentonI have defined an api spec for front end to communicate with backend. The messages include an auth token that I strip off soon and forward the rest of the message to internal handling functions. It would be cool to be able to create a spec on my backend that takes a spec defined for the API and removes aspects of it, like for example the auth-token key of the map. I guess the generic manipulating of a spec like a regular hash or something would be interesting.#2016-06-2618:36fentonotherwise i have to duplicate the definition of the message....trying to keep it DRY.#2016-06-2618:55seancorfield@fenton: why not define the spec once for the back end, then define the spec for the front end as "back end" and "auth token"?#2016-06-2618:57fenton@seancorfield: I've pulled my api spec into a third project that is *.cljc with reader conditionals so it can be included as a library for both front and backend. I don't think I want to pull in my backend as a library to my front end....#2016-06-2618:58fentonnot sure if thats the best way to do it tho?#2016-06-2618:58seancorfieldJust the spec for the shared portion is what I'm suggesting. #2016-06-2618:59fentonahh that makes sense#2016-06-2618:59seancorfieldSo if your API and your back end traffic in a common subset of data, that spec needs to be external to both as a shared spec. #2016-06-2619:00fentonyeah, i like that...thanks for the idea!#2016-06-2619:02fentonhmmm...might run into same problem about how to make a spec like that... can't merge a spec map.#2016-06-2619:02fentonmap spec.#2016-06-2619:03seancorfieldYou use s/and to specify the API must conform to common spec and also to the auth token spec. You don't merge the maps. #2016-06-2619:04seancorfieldAlex talked about that technique here a day or two ago. #2016-06-2619:05fentonthat one?#2016-06-2619:06seancorfieldI'm on my phone so I can't dig into code. #2016-06-2619:07fentoni didn't try that approach, but wondering if that doesn't make data that looks like: [{:auth-token "abc123"} {:method :search :search-term "blah"}]#2016-06-2619:08fentonwhich is okay...i guess...#2016-06-2619:10seancorfieldSince maps are open, saying some data conforms to multiple map specs should still leave you with one map?#2016-06-2619:11seancorfield(Replete doesn't seem to have clojure.spec or I could try that out)#2016-06-2619:12fentonhmmm, not sure, making the (s/keys :req ... might be a problem the :req part...meaning each needs all?#2016-06-2619:12mfikes@seancorfield: It does, but it is under cljs.spec and slightly older at this point.#2016-06-2619:13seancorfieldAh, thanks @mfikes !#2016-06-2619:14seancorfield@fenton: if A needs key X and it needs key Y then that is additive and it needs keys X and Y - which is a merge and therefore what you want. #2016-06-2619:15gfredericksis (s/and (s/keys ...) (s/keys ...)) going to be special-cased to aid things like generation?#2016-06-2619:17seancorfield(s/conform (s/and (s/keys :req [::a]) (s/keys :req [::b])) {::a 1, ::b 2}) => {:cljs.user/a 1, :cljs.user/b 2}#2016-06-2619:18seancorfield(In cljs.spec in Replete)#2016-06-2619:19seancorfieldDang, I love that I can test that on my phone while lazing in bed on a Sunday!!!#2016-06-2619:19gfredericksI can't tell if that is supposed to answer my question or not :)#2016-06-2619:20gfredericksI can try it myself though, let's see...#2016-06-2619:20mfikes@seancorfield: And if we figure it out, clojure.spec will act as an alias for cljs.spec in bootstrapped ClojureScript šŸ™‚#2016-06-2619:20seancorfieldIt was for @fenton - I'm typing on my phone, excuse the slow responses. #2016-06-2619:20fenton@seancorfield: yeah, that looks good...#2016-06-2619:20seancorfieldReplete is truly awesome. #2016-06-2619:21fentonwhat is replete?#2016-06-2619:21gfredericksnope, the generator doesn't work#2016-06-2619:21fenton@seancorfield: ahh googled it.#2016-06-2619:21mfikesIt is for lazy people who donā€™t want to crawl out of bed to their computer to start a REPL.#2016-06-2619:21mfikes(Like me. šŸ™‚ )#2016-06-2619:22fentonlol!#2016-06-2619:25fentonhappy lazing folks, thanks for the help! šŸ™‚#2016-06-2619:26seancorfield@gfredericks: the generator doesn't work for and with two keys?#2016-06-2619:27gfredericksseancorfield: right, which makes sense given what I know about s/and#2016-06-2619:27gfredericksit'd have to be special-cased to have any chance of succeeding#2016-06-2619:27gfredericksbecause the s/and generator just generates the first thing and filters based on matching the rest of them#2016-06-2619:28seancorfieldwith-gen to the rescue! šŸ˜† #2016-06-2619:29seancorfieldI have nothing. #2016-06-2619:29gfredericksthere'll be a helper for this in my inevitable utility library#2016-06-2717:09angusiguess::args should be an s/cat#2016-06-2717:09angusiguess(s/fdef takes-a-map :args (s/cat :a ::A)))#2016-06-2717:09angusiguessArgs is always assumed to be a list of 0 or more arguments#2016-06-2717:10angusiguessOr rather, a spec for a list of 0 or more arguments#2016-06-2722:43bhaumanSo I'd like to do some spec work in an environment that needs to run on an earlier version of Clojure. I'd like to conditionally include the clojure.spec library if the Clojure version isn't 1.9 and if clojure.spec isn't already loaded. Is there an established smart pattern for doing this? Say requiring a library that does a conditional load?#2016-06-2723:08seancorfieldI donā€™t know how "established" it is, but I did this for java.jdbcā€™s test namespace...#2016-06-2723:08seancorfieldhttps://github.com/clojure/java.jdbc/blob/master/src/test/clojure/clojure/java/test_jdbc.clj#L28-L32#2016-06-2723:09seancorfieldAnd the specs themselves are in a separate namespace from the code https://github.com/clojure/java.jdbc/blob/master/src/main/clojure/clojure/java/jdbc/spec.clj#2016-06-2723:10seancorfieldAn alternative approach is taken in clojure.core regarding Java versions: https://github.com/clojure/clojure/blob/d274b2b96588b100c70be065f949e1fdc9e7e14d/src/clj/clojure/core.clj#L6654-L6658#2016-06-2723:10seancorfieldā€¦andā€¦ https://github.com/clojure/clojure/blob/d274b2b96588b100c70be065f949e1fdc9e7e14d/src/clj/clojure/core_instant18.clj#2016-06-2723:41bhauman@seancorfield: thanks! I appreciate it. I'll take a gander :)#2016-06-2803:32Oliver GeorgeNew favourite spec trick. Wrap your spec to generate warnings instead of erroring out.
(defn warn
  "Throw warning if spec is invalid but always pass."
  [spec]
  (fn [x] (when-not (s/valid? spec x)
            (js/console.warn (s/explain-str spec x) x))
    x))
#2016-06-2806:52Oliver GeorgeAnd now not. Turns out the stack trace from normal errors is more informative (cljs).#2016-06-2808:56rauhI have within one ns two different specs for :db/id (one for data coming from datascript and one for datomic ids). How can I specify this in spec?#2016-06-2809:46rauhIs this the way to do this: https://gist.github.com/rauhs/3bf96b33f37f051859d5071cdde70148 ?#2016-06-2810:06puzzlerHow would you specify a "vector of anything"?#2016-06-2810:06puzzler(coll-of ??? [])#2016-06-2810:07puzzlerWhat goes in the ??? position?#2016-06-2810:07puzzlerIs there some sort of top-level type in spec?#2016-06-2810:08puzzlerI guess I could just use vector?, but I'd like it ideally to generate lots of random Clojure things to go in the vector.#2016-06-2810:09puzzlerSimilarly, what about a map where the keys can be anything but the vals are more restricted?#2016-06-2810:10puzzler(map-of ??? integer?)#2016-06-2810:15rauh@puzzler: (spec/coll-of ::spec/any [])#2016-06-2810:18puzzlerhanks#2016-06-2810:18puzzlerthanks#2016-06-2810:18puzzlerHmmm, looks like that conforms non-vectors too.#2016-06-2810:19puzzlerWhere is ::spec/any documented?#2016-06-2812:01jannis@puzzler: (s/spec vector?)?#2016-06-2812:03Alex Miller (Clojure team)There is new support for this in next alpha#2016-06-2812:03jannisCool#2016-06-2812:04jannisI have a lot of vector?-based specs that currently all require custom generators.#2016-06-2812:04Alex Miller (Clojure team)coll-of has many new options#2016-06-2812:17Alex Miller (Clojure team)Can now indicate the expected incoming type separately from the conformed and generated type#2016-06-2812:51Alex Miller (Clojure team)Also, Rich added s/merge yesterday to create spec that merges multiple map-validating specs and will gen as you expect (union of keys)#2016-06-2812:53Alex Miller (Clojure team)@jannis in next alpha you will be able to do (s/coll-of int? :kind []) to indicate a vector of ints#2016-06-2812:54jannis@alexmiller: Awesome šŸ™‚#2016-06-2812:58puzzlerWhen a spec is complicated and composed of many component specs, I'm having trouble figuring out, stylistically, which of those component specs to register to namespaced keys, versus just def'ing them to regular vars. Any tips on this?#2016-06-2812:59puzzlerIt seems to me that when spec'ing a bunch of optional keywords for a function, to get good error messages I really want to be able to say "These are the only keys allowed in this map of optional keywords and values". Is there any way to do that?#2016-06-2813:11Alex Miller (Clojure team)generally, I think you should not def anything to a regular var#2016-06-2813:11Alex Miller (Clojure team)and you should register specs for all map attributes and for any major ā€œtypes"#2016-06-2813:11Alex Miller (Clojure team)anything recursive will require registration at the recursion points#2016-06-2813:12Alex Miller (Clojure team)and you might want to register things at a finer granularity if you find it gives you more meaningful explain errors#2016-06-2813:13Alex Miller (Clojure team)re restricted key sets - no, this is intentionally not provided, but you can s/and a constraint like #(every? #{:allowed :keys :in :map} (keys %))#2016-06-2813:25puzzlerThanks. Early on, I predicted that a lot of people would re-invent slightly different definitions of validating or conforming combined with assertions in the event of a failure, and I've been seeing that happen. Is something along the lines of assert-conform and assert-valid? likely to make it into a future alpha?#2016-06-2814:10Alex Miller (Clojure team)yes, there are likely to be a conform-ex and some kind of assert, but I donā€™t think those will be in next alpha#2016-06-2815:48bhaumanI've been wanting to add an extended :reason to certain specs, especially if they have strange interdependencies#2016-06-2816:05bhaumanIf you look sideways at this you could see the possibility of specs with composable prose explanations#2016-06-2820:41kendall.buchananSuper quick question: If Iā€™m referencing a spec in another namespace, do I need to :require that namespace too?#2016-06-2820:42kendall.buchananIā€™m finding the app runs fine in the REPL, but at compilation, it canā€™t resolve the specs.#2016-06-2820:42bhaumanyep#2016-06-2820:43bhaumanIf I'm understanding you correctly, all specs need to be required before use#2016-06-2820:43bhaumanin order for the registry to populate#2016-06-2820:44kendall.buchananOkay, I was afraid of that šŸ˜‰#2016-06-2820:44kendall.buchananThank you!#2016-06-2820:46martinklepschSo, form-validation & clojure.spec, I assume people have tried it already, are there any writeups/snippets?#2016-06-2820:48martinklepsch@bhauman: @kendall.buchanan I find it interesting that this can create an implicit dependency between namespaces that is not declared via the ns form, maybe for "good style" you should require the ns containing the spec even if you don't need to (technically)?#2016-06-2820:50kendall.buchananIā€™ll admit ā€“ I still find the coupling between specs and namespaces odd. Namespaced keywords seem to make sense when youā€™re trying to define a domain path, but sometimes those domain paths donā€™t match your namespace hierarchy. But Clojure basically says it has to.#2016-06-2820:50kendall.buchananStill struggling to understand it. But, seems fine.#2016-06-2820:50bhauman@kendall.buchanan: you are aware that you don't have to use an existing namespace correct?#2016-06-2820:51bhaumanjust a namespaced keyword#2016-06-2820:51kendall.buchananYeah, totally, but I guess Iā€™m struggling to see the established path forward.#2016-06-2820:51martinklepsch> but sometimes those domain paths donā€™t match your namespace hierarchy#2016-06-2820:51martinklepsch:+1: (same struggle here)#2016-06-2820:53bhaumanIn terms of dependency management, I think if people look at the way they did plumatic/schema or any add hoc type validation.#2016-06-2820:54bhaumanit's like any dependency#2016-06-2820:55angusiguessWe've started defining a lot of our specs in a separate heirarchy. It's a bit early to tell how well that works but thus far it's been pretty useful in terms of reducing circular dependency issues, making specs equally usable in tests and code, etc.#2016-06-2820:55kendall.buchanan@bhauman, would you recommend (or consider as a viable path) to always use domain-based keywords school/:type and eschewing ::type (contained in my.app.model.school? Just ditching the latter pattern?#2016-06-2820:56angusiguessSo that's one possibility, is to use ::type but to place those definitions in a namespace that is a little descriptive of the domain model.#2016-06-2820:56bhaumanI wouldn't be the one to ask, but I would say no#2016-06-2820:57bhaumanA possiblity: As long as your keys are distinct, you can keep it all in one file.#2016-06-2820:57kendall.buchanan@angusiguess: Kind of like app.spec.school/type? Thatā€™s more or less what Iā€™ve done for objects that cut across many domains.#2016-06-2820:58angusiguess@kendall.buchanan: That's precisely what I do, including a namespace for more generic types.#2016-06-2820:58angusiguessLike
app.spec/uuid-str
or similar.
#2016-06-2820:58kendall.buchananRight.#2016-06-2820:58kendall.buchananYeah, exactly what weā€™ve started doing.#2016-06-2820:59angusiguessThen if we have aggregates that our functions take we'll define them in the function's namespace.#2016-06-2820:59kendall.buchananAnd I think it works well. I guess I like the ::thing syntax, for brevity, and realize it ties things up into the namespace. I guess weā€™re learning as weā€™re going, though.#2016-06-2821:00kendall.buchanan(By the way, I have no proposed alternative, so not complaining. Just been a source of friction for me.)#2016-06-2821:00angusiguessDefining everything in-line bit us pretty quick.#2016-06-2821:03kendall.buchananRight, @angusiguessā€¦ which, I have to say, clojure.spec is what it claims: built for composability. Iā€™ve been a heavy Schema user, and have to say, Iā€™m getting more reuse out of spec by far.#2016-06-2822:17Alex Miller (Clojure team)Clojure 1.9.0-alpha8 is now out with lots and lots of spec updates https://groups.google.com/d/msg/clojure/vF3RuDWuX8I/pvn4IUuUAwAJ#2016-06-2822:41richhickeyLots of new spec stuff in alpha 8 - conforming coll support, plus count/kind control, instrument can generate stubs, gen/spec overrides in instrument/test, merge keys specs#2016-06-2822:44bfabryawesome#2016-06-2823:05Timwow sounds good#2016-06-2902:47Alex Miller (Clojure team)Iā€™ve updated the guide for 1.9.0-alpha8 although there will be some additional updates http://clojure.org/guides/spec#2016-06-2909:20martinklepschI have a question about spec'ing maps: lets say I have two keys that should follow the same spec, is the intended way to define proxy specs which I then use as map keys?#2016-06-2911:28Alex Miller (Clojure team)You can register the map keys to point to either the same spec definition or to another registered spec. I think doing this will be common. One layer of registered base "types" and another layer of domain attributes.#2016-06-2912:41martinklepschThanks Alex#2016-06-2912:42martinklepsch
- [changed] explain-data - output is now a vector of problems with a :path element, not a map keyed by path
Whats the reasoning behind this change? Could there be multiple problems referencing the same path?
#2016-06-2912:58richhickey@martinklepsch: yes, multiple problems at same path happens with collections (was already happening with ā€˜keys' which jammed them all into same pred, now doesnā€™t)#2016-06-2914:05robert-stuttaford@martinklepsch: on your question from 11:20 today, do you mean like what i've done with ::pos-int here? https://github.com/robert-stuttaford/mtg/blob/master/src/mtg.clj#2016-06-2914:07martinklepsch@robert-stuttaford: yeah kind of I guess#2016-06-2914:08martinklepschif you want to use a generic spec in keys without using it's key you need a proxy like this#2016-06-2914:12robert-stuttafordyeah#2016-06-2914:21Alex Miller (Clojure team)@robert-stuttaford: there is a pos-int? predicate now btw in core#2016-06-2914:22Alex Miller (Clojure team)that ::types spec you have can now be accomplished with coll-of in alpha8 too: (s/coll-of ::type :kind set? :min-count 1)#2016-06-2914:23Alex Miller (Clojure team)that version has the benefit of automatically genā€™ing too#2016-06-2914:25Alex Miller (Clojure team)And I think your ::spell, ::creature, ::planeswalker, etc can also be handled by s/merge now#2016-06-2914:25Alex Miller (Clojure team)new toys!#2016-06-2914:30robert-stuttafordoh that is rad#2016-06-2914:31robert-stuttafordthanks alex!#2016-06-2914:34robert-stuttafordhow far, as a %, would you say the core.spec slated for 1.9 is?#2016-06-2914:35robert-stuttafordit seems like you've still got a lot of meat on the bone, given all the stuff that gets added with every alpha#2016-06-2914:37richhickey@robert-stuttaford: not a lot of big things - assertions, perf tweaks, reconciling regexes and vectors, some refinement of keys*, then details#2016-06-2914:38richhickeythen moving to infrastructure around test-running etc, the whole model implied by instrument+test which is different from current practice#2016-06-2914:38ghadiOn the Complexity and Performance of Parsing with Derivatives -- PLDI 2016 https://www.youtube.com/watch?v=WPde6DDLtXg#2016-06-2914:38ghadiiteration.next of matt might & co's work#2016-06-2914:39ghadihttps://arxiv.org/pdf/1604.04695v1#2016-06-2914:39robert-stuttafordfantastic, rich! very exciting set of tools you're building, here. fundamental, like the data-oriented approach and immutability are. going to take some time to soak in#2016-06-2914:40robert-stuttafordi realise this may well be answered with 'maybe :-)', but i'm super curious how you plan to leverage spec in Datomic šŸ™‚#2016-06-2914:42richhickey@ghadi: thanks for the links#2016-06-2914:42robert-stuttafordDatalog is an obvious one, as is transaction data. wondering how else it may improve the experience of programming with it#2016-06-2914:43richhickey@robert-stuttaford: still TBD (but lots of obvious opportunities), first things first, and thatā€™s a stable clojure.spec#2016-06-2914:43robert-stuttafordof course šŸ™‚#2016-06-2914:43richhickeyI can say we are getting a lot out of using it in the development and testing of Datomic already#2016-06-2914:44robert-stuttafordi can't think of a better way to dogfood it#2016-06-2914:45richhickeythe new stubbing support is something to check out - you can now take a version of your wire interface, spec it, and get a no-wire stub for free from spec#2016-06-2914:46ghadirad#2016-06-2914:46robert-stuttafordah - if i understand you correctly, you mean inferring a spec from an existing data set?#2016-06-2914:47robert-stuttafordif so, thanks .. i was planning to do this as a learning exercise!#2016-06-2914:48richhickeyno, letā€™s say you have a data-driven wire protocol that talks to a service - you spec the protocol, instrument its namespace saying :stub in the overrides, and then run tests of code that consumes the protocol. Now those tests donā€™t talk to a service at all but get checking of their payloads and generated returns#2016-06-2914:49ghadiaside: I really like the syntax of (s/alt :foo x :bar y). That is something I didn't get quite right in pex, my vm-based PEG parser; I made capturing things too verbose. Definitely going to adopt that approach#2016-06-2914:49richhickeyyou can selectively override gens to generate ā€˜statefulā€™ things like connections#2016-06-2914:50robert-stuttafordoh, right!#2016-06-2914:50richhickey@ghadi: yeah, the path system is something I think people resist initially, but it keeps paying dividends#2016-06-2914:51robert-stuttafordman. i feel like i'm going to need a spec sabbatical.#2016-06-2914:53robert-stuttafordso many new toys, but at such a fundamental place in the language.. it's going to touch everything!#2016-06-2914:53ghadicurious, why do people resist the path system?#2016-06-2914:54richhickeysometimes you donā€™t care about the labels or want to see them in the destructuring#2016-06-2914:54robert-stuttafordis https://clojure.github.io/clojure/branch-master/clojure.spec-api.html current as of alpha8?#2016-06-2914:54ghadiroger#2016-06-2914:55richhickeybut they let us talk about the structure of a spec all the way down, which is huge#2016-06-2914:55robert-stuttafordnot seeing anything about stub in there, yet#2016-06-2914:55ghadimy plan for better dev-time experience in the PEG library was to provide a tracing mode on all the patterns attempted#2016-06-2914:56richhickey@robert-stuttaford: (doc test/instrument)#2016-06-2914:56ghadisince backtracking is likely in a failure, and most of the ways to handle it were through heuristics like longest match#2016-06-2914:56robert-stuttafordthanks#2016-06-2914:56Alex Miller (Clojure team)@robert-stuttaford: I think those are lagging an alpha or two, I will get them updated#2016-06-2914:57richhickeyoops, didnā€™t realize API docs were behind#2016-06-2915:02bhaumanalright, I'm running into an issue in alpha7
(s/def ::string  (s/or :string string?))
  
  (s/def ::myid ::string)
  
  (s/explain (s/and
              (s/keys :opt-un [::myid])
              (s/keys :req-un [::myid]))
             {:myid "asdf"})
#2016-06-2915:03bhaumanResults in
In: [:myid] val: [:string "asdf"] fails spec: :strictly-specking.test-schema/myid at: [:myid :string] predicate: string?
#2016-06-2915:03bhaumanit appears to be validating against the conformed value#2016-06-2915:06bhaumanI'll check against alpha8, in a bit, i have to port some stuff#2016-06-2915:07bhaumanand it's strange because this works
(s/def ::myid string?)
  
(s/explain (s/and
              (s/keys :opt-un [::myid])
              (s/keys :req-un [::myid]))
             {:myid "asdf"})
#2016-06-2915:11bhaumanand that makes sense because the conformed value is just a string#2016-06-2915:16richhickeyā€˜andā€™ does flow conformed values - "Successive conformed values propagate through rest of predicates."#2016-06-2915:18richhickeythat may change for ā€˜mergeā€™, which is what you should switch to for this application#2016-06-2915:19richhickeythere may also be and and and-> variants since sometimes you need this (e.g. when preceding pred is a regex) and sometimes not#2016-06-2915:24bhaumangreat, looking at merge now#2016-06-2915:29bhaumanthe above works with merge right now and inspection reveals that the conformed values aren't flowing right now.#2016-06-2915:32richhickey@bhauman: merge does flow through conform, not explain (thatā€™s broken)#2016-06-2915:32bhaumanoh gotcha#2016-06-2915:33richhickeybut no reason for it to flow through conform, that will change. The big benefit of merge is that it will gen where ā€˜andā€™ cannot for this#2016-06-2915:34bhaumangood deal#2016-06-2915:36bhaumanmy use case is taking an existing keys spec and constraining it by moving some keys into :req#2016-06-2915:37richhickeyI understand, and that will be common#2016-06-2915:38richhickeyalso various intersections#2016-06-2915:38robert-stuttafordjust so i'm sure i understand what you're both talking about, are you talking about what i'm trying to do here? https://gist.github.com/robert-stuttaford/e1bc7bfbcdf62020277dda1a277394ca#2016-06-2915:38richhickeyso no flow is the right thing#2016-06-2915:38robert-stuttafordwhere i want to gen (s/and (s/keys ...) (s/keys ...)) automatically?#2016-06-2915:39robert-stuttafordseems like s/merge may be what i need to use instead#2016-06-2915:39richhickeyyes, thatā€™s precisely the point of merge#2016-06-2915:39robert-stuttaford\o/ rad#2016-06-2915:39richhickeyit gens map unions#2016-06-2915:40richhickeysubject to the (temporary) flow problem @bhauman identifies above#2016-06-2915:41robert-stuttafordgotcha#2016-06-2915:42robert-stuttafordi've just seen http://dev.clojure.org/jira/browse/CLJ-1910 and http://dev.clojure.org/jira/browse/CLJ-1919. it's going to be terribly tempting to rewrite everything!#2016-06-2915:49robert-stuttafordi guess there'll be ways to add or remove namespaces from these maps?#2016-06-2915:51richhickeythis only affects reading, not the map#2016-06-2915:52richhickeyi.e. itā€™s not a lingering property of the map#2016-06-2915:54robert-stuttafordah, so if i wanted to add namespaces to a non-nsed map, i'm going to do so to the keys in the map myself#2016-06-2915:54richhickeythis is only reading/printing#2016-06-2915:54robert-stuttafordgotcha, thanks#2016-06-2916:16seancorfieldhttps://clojurians.slack.com/archives/clojure-spec/p1467211082001261 I hope that whatever comes out of this doesnā€™t make it harder to use other testing libraries (like Expectations) harder.#2016-06-2916:17seancorfield(but, yeah, great changes in Alpha 8ā€¦ I updated java.jdbc to accommodate enough changes to get it running again but I havenā€™t dug deep into redoing the specs to leverage the new features)#2016-06-2916:51jannisAn optional piece of data inside a tuple is not supported, I assume? (s/def ::my-tuple (s/tuple (s/? keyword?) symbol?)) claims ['foo] is invalid: [foo] fails spec: :workflo.macros.command/my-tuple predicate: (= (count %) 2)#2016-06-2916:51jannisMy current "workaround" is to use (s/and vector? (s/cat :optional (s/? ...) :mandatory ...))#2016-06-2916:52jannisWhich is ok, it does mean that you have to write a custom generator for it though.#2016-06-2916:53jannisAlso, is there something like s/tuple for fixed-size non-vector sequences with a specific order of subspecs?#2016-06-2916:57Alex Miller (Clojure team)You can use s/cat, s/? etc for things like this #2016-06-2916:58Alex Miller (Clojure team)If you have optionality regex is definitely the right choice over tuple#2016-06-2916:58jannisOk#2016-06-2916:58Alex Miller (Clojure team)We are still looking at options to also enforce vector on regex in a good way#2016-06-2917:00Alex Miller (Clojure team)s/and is sufficient as you saw but needs custom gen as you said#2016-06-2917:04jannis@alexmiller: Ah, s/cat didn't work because I forgot s/spec around it to make it a subspec and require the content of s/cat to be in a child vector or sequence.#2016-06-2918:14uwomy colleagues would like to create a single source of truth to generate both datomic schema and specs. Is this advisable? I was under the impression that perhaps spec would be written mostly by hand and not codegen#2016-06-2918:48Alex Miller (Clojure team)seems low on the crazy scale to me#2016-06-2918:50uwohaha. thanks#2016-06-2918:54Alex Miller (Clojure team)code is data and all that :)#2016-06-2920:07tylerIs clojure.spec.test/test meant to be used with clojure.test?#2016-06-2920:13tylerI guess more specifically, what is the idiomatic way to set up generative tests with clojure.spec is there an equivalent of defspec from test.check for clojure.spec?#2016-06-2920:24Alex Miller (Clojure team)fully utilizing the instrumentation and test facilities in clojure.spec.test is a different kind of process from the existing clojure.test and I donā€™t think we will provide a linkage like defspec.#2016-06-2920:24Alex Miller (Clojure team)we have more to say about this and I expect we will have more guidance to provide before 1.9 is released#2016-06-2920:26tylerGotcha, is it expected to be a developer concern? Or will there be additional functionality added to integrate it with clojure.test? The return seems more data-centric than clojure.test.#2016-06-2920:35Alex Miller (Clojure team)I donā€™t currently expect anything that integrates it with clojure.test. The returns are more similar to the results from test.check as thatā€™s what weā€™re actually invoking.#2016-06-2920:37Alex Miller (Clojure team)there are levels of question here - some around whatā€™s provided in core, another level around runners, and another level around build tooling#2016-06-2920:41tylerI suppose the core of my question is around whether or not clojure.spec.test/test will integrate into the existing test tooling or is it expected that new test tooling will need to be built? instrument seems to fit into the current ecosystem but it seems like clojure.spec.test/test will require new tooling.#2016-06-2920:44puzzlerIs there a difference between (s/def ::type vector?) and (s/def ::type (coll-of ::s/any :kind vector?)) ?#2016-06-2920:47martinklepschWhen a required key is missing from a map the error looks like this:
{:pred [(contains? % :to) (contains? % :subject)],
   :val {},
   :via [:forms.app/email-form],
   :in [],
   :path []}
Is there a way to get an easier to parse description that will allow me to (programatically) figure out which keys are missing from a map?
#2016-06-2920:50angusiguessYou can get that data structure with s/explain-data#2016-06-2921:03wilkerlucio@tyler: if you want to run generative tests and have report of it in clojure.test you can use the clojure.spec.test/test like this: (is (true? (-> (clojure.spec.test/test 'my.ns/some-fn) :result)))#2016-06-2921:06tylerThanks @wilkerlucio. I had been playing around with that a little bit but the output isnā€™t very informative for failures for that. I think Iā€™m going to go the route of experimenting with developing my own runner. Just wanted to make sure I wasnā€™t duplicating work/effort.#2016-06-2921:11martinklepsch@angusiguess: right, what I pasted is part of what s/explain-data returns, was just wondering if there's a better way to get a list of missing keys#2016-06-2921:12wilkerlucio@tyler: try using something like this:#2016-06-2921:13wilkerlucionot super pretty, but might be a good start šŸ™‚#2016-06-2921:13tylerThanks Iā€™ll give that a go#2016-06-2921:16puzzlerIs there an explanation somewhere of how the sampling of every works? I'd like to get a sense for the probability that an error is detected if I switch from coll-of to every.#2016-06-2921:18mcorbinHello, i play with clojure-spec and i don't understand this :#2016-06-2921:19mcorbinWhy explain returns Success ? The second condition in the "s/and" should return false with a string parameter#2016-06-2921:24leongrapenthin@mcorbin I believe the lambda pred is always invoked with a vector, i. e. [:string "Hello Clojurians"]#2016-06-2921:27bhauman@mcorbin: theres a discussion of this from earlier today ...#2016-06-2921:27bhaumansee my posts above#2016-06-2921:28mcorbin@leongrapenthin: you win, thanks @bhauman i will look, thx#2016-06-2921:41Alex Miller (Clojure team)@puzzler: if you mean ā€œhow it samplesā€ - no there intentionally is not, as it may change in undefined ways in the future#2016-06-2921:43Alex Miller (Clojure team)all you should count on is that it will sample no more than *coll-check-limit* elements#2016-06-2922:03seancorfield@tyler: FYI, we use Expectations to drive test.check stuff and ended up with a pattern like this /cc @wilkerlucio
(expect (more-of {:keys [result num-tests]}
                 true result
                 100 num-tests)
        (tc/quick-check 100
                        (prop/for-all [s gen/string]
                                      (= 0 (ip->long (str/replace s #"[0-9]" "."))))))
#2016-06-2923:21puzzlerSuggestion: Right now it looks like every and coll-of check for distinctness with (apply distinct? ...). Would recommend (or (set? ...) (apply distinct? ...)) to short-circuit when a set is passed in to a function that expects a collection of distinct items.#2016-06-3000:16jjcomerIā€™m trying to use the new stub feature, but I canā€™t seem to get the stub to take hold. The println still happens for me:
(defn ding
  [x]
  (println "Hello World"))

(s/fdef ding
        :args (s/cat :x pos-int?)
        :ret nil?)

(defn adding
  [x y]
  (ding x)
  (+ x y))

(s/fdef adding
        :args (s/cat :x pos-int? :y pos-int?)
        :ret pos-int?
        :fn (fn [{:keys [args ret]}]
              (= ret (+ (:x args) (:y args)))))

(defn test-adding
  []
  (try
    (st/instrument `adding
                   {:stub #{`ding}})
    (adding 2 3)
    (finally (st/unstrument `adding))))
#2016-06-3000:29puzzlerWhat happened to instrument-all and instrument-ns in the new alpha?#2016-06-3000:33jjcomer@puzzler: All is instrument with no params and ns is passing an ns to instrument#2016-06-3000:49puzzlerthanks#2016-06-3000:50puzzlerIs there any way to keep the instrumenting on during development? It seems that when you recompile a namespace it blows away the instrumentation.#2016-06-3001:26seancorfieldSince instrument updates the Vars in place, Iā€™d expect that (so, "no", I think is the answer).#2016-06-3001:26seancorfieldIf you recompile the namespace, youā€™re replacing all the Vars, so youā€™d need to re-run instrument.#2016-06-3001:31gfrederickspuzzler: one option is to put (s/instrument ...) at the bottom of relevant namespaces#2016-06-3001:31gfredericksoptions that don't modify the source code would depend on how you're loading your code#2016-06-3001:31puzzlerIn theory, defn could be rebound to automatically instrument functions that have specs.#2016-06-3001:33seancorfieldThat would only help if the spec was defined ahead of the function...#2016-06-3001:37puzzlerI'm not saying this should be the general definition of defn, I just mean that when you choose to instrument all, a new version of defn (and fdef to handle the case of function before spec) would ensure the instrumentation never gets out of sync at the REPL, which I'm finding to be an annoyance. Right now, you change the spec and the instrumented function doesn't see it, or you change the function and the spec-checking goes away. Too easy to forget to re-run instrumentation.#2016-06-3001:38seancorfieldI have my specs in a different namespace...#2016-06-3001:39puzzler@seancorfield: Are you saying that having specs in a different namespace alleviates this issue? Seems to me it would just make it even more complicated.#2016-06-3001:39seancorfieldIā€™m saying your suggestions wouldnā€™t address that problem.#2016-06-3001:39seancorfieldi.e., itā€™s not a general solution#2016-06-3001:40seancorfield(and I donā€™t know what the general solution is)#2016-06-3001:41seancorfieldFor the specific case of java.jdbc, I have the test namespace ā€” which loads the specs ā€” also run instrument.#2016-06-3001:42seancorfieldSo the code is separate from the specs, but the tests depend on them.#2016-06-3001:42puzzlerThe first time you call instrument-all, it's calling your spec name space which registers the specs in a global registry. So a defn that was aware of the specs in the registry would help that direction. Going the other direction, if you edit the spec in the separate file, that spec refers to the var in the implementation namespace, so that function could be re-instrumented. So I think it would probably work. What am I missing?#2016-06-3001:43seancorfieldI think itā€™s very dependent on workflow. Iā€™m waiting to see what Clojure/core come up with (based on the discussion earlier about whatā€™s coming for specs in the test tooling area).#2016-06-3001:44seancorfieldWhat you raise is certainly an issue that can make working with spec in the REPL pretty annoying.#2016-06-3001:44puzzlerI'm only imagining this would be helpful in a mode where all instrumentation is turned on for everything. I don't have a good handle on the workflow for spec yet, so I agree with your point that we'll see how this evolves.#2016-06-3001:46seancorfieldRight now Iā€™m only instrumenting on a per-namespace basis as I work with / test a particular namespace. I ran into some early weirdness with instrument-all (and no longer remember the details) and backed off thatā€¦ may have been back when it caused generative testing to occur on higher-order function specs?#2016-06-3001:46seancorfieldSo it may be that separating the generative part of function specs from instrumentation has addressed the issues I ran into...#2016-06-3001:47puzzlerThe only issue I have with putting specs in a separate namespace is that it diminishes their value as a form of documentation. (Actually, this is also an issue with putting tests in a separate namespace, which I've gotten used to, so maybe I'll get used to putting specs elsewhere as well).#2016-06-3001:48puzzlerIn your split-namespace approach, do you see specs in the doc strings of your functions?#2016-06-3001:49seancorfieldIf youā€™re working in the test namespace (where the spec has been loaded), yes.#2016-06-3001:50seancorfieldI havenā€™t settled on it as a normal way to work yet. Weā€™re still exploring several options#2016-06-3001:50puzzlerBut users of your library generally wouldn't see that if they call doc on a function at the REPL, right?#2016-06-3001:50seancorfieldIf they load the specs, yes.#2016-06-3001:50seancorfieldThis allows the code to be used by pre-1.9 clients.#2016-06-3001:51seancorfieldFor java.jdbc, it supports all the way back to 1.4#2016-06-3001:51seancorfieldand even in 1.9, the specs are optional but available.#2016-06-3001:51puzzlerAh, yes, I can see how that would be the most critical thing to keep in mind for java.jdbc
#2016-06-3001:51seancorfieldwhen I run the java.jdbc tests on 1.9, it loads the specs and instruments the ns (not all, just java.jdbc)#2016-06-3001:52seancorfieldSo, for library code that might have pre-1.9 users, separate specs is pretty much the only possibility.#2016-06-3001:52puzzlerCould the main sourcecode namespace do that test on version number, and automatically load the specs for docstring purposes?#2016-06-3001:53seancorfieldFor application code thatā€™s is bound to 1.9, I can imagine having specs intermingled ā€” maybe.#2016-06-3001:53seancorfieldYeah, I guess java.jdbcā€™s source namespace could auto-load the specs on 1.9. Iā€™m just not sure that would be what all its users would expect.#2016-06-3001:54seancorfieldAfter all, if you (instrument) your code would you expect all the external libraries to slow down due to specs?#2016-06-3001:55seancorfieldSome specs can have a really high performance overhead so I donā€™t think the library should dictate that.#2016-06-3001:55puzzlerI'm thinking that my number one use for specs will be to get good error messages when working with other libraries, so yes, I think that's what I personally would want. The biggest source of errors in my code is having a false expectation about how someone else's code should work.#2016-06-3002:06seancorfieldGood to know.#2016-06-3002:07seancorfieldWeā€™ve jumped on the Alpha builds at work (in production) in preparation for using clojure.spec fairly heavily but weā€™re still in the early exploration phase. Weā€™re already leveraging the new predicates in production code thoā€™.#2016-06-3002:09seancorfieldWe havenā€™t fully decided what weā€™ll spec out yet vs what we wonā€™t bother with. Iā€™ve been exploring using java.jdbc with namespaced keys as part of this work.#2016-06-3002:10seancorfieldWe have over 30Kloc production code and over 11Kloc tests so whatever we do will need to be a gradual process...#2016-06-3002:11seancorfieldā€¦expanding what we can do with generative testing alone might bring us a good "bang for our buck".#2016-06-3002:18Tim@seancorfield: at world singles?#2016-06-3002:19seancorfieldYup. Just pushed a build to QA based on Clojure 1.9.0 Alpha 8 so thatā€™ll go to production in our next build.#2016-06-3002:20seancorfield(we already have Alpha 7 in production)#2016-06-3003:58cap10morganI'm getting Don't know how to create ISeq from: spec_demo.core$full_name when running (stest/instrument full-name) in an ns where I have a fn full-name and have (s/fdef full-name ...). It works if I leave the argument off (so it instruments everything), but I thought it was supposed to take symbol args now too in alpha8?#2016-06-3004:22seancorfield(stest/instrument 'full-name) <-- needs to be a symbol#2016-06-3004:26seancorfieldTo explain the error message, (stest/instrument full-name) will evaluate full-name, since it's a function it evaluates to its class name which is a class full_name inside the package spec_demo.core because the Clojure is spec-demo.core/full-name.#2016-06-3004:26seancorfieldHope that helps @cap10morgan ?#2016-06-3004:50puzzlerI'm having trouble ascertaining the purpose of the retag in multi-spec. Can someone clarify when you'd actually see the retag?#2016-06-3011:35Alex Miller (Clojure team)In gen#2016-06-3011:35cap10morgan@seancorfield: ah, yes, thanks. I was trying #'full-name and getting a var but totally forgot about 'symbol.#2016-06-3011:37cap10morganI'm giving a presentation tonight on clojure.spec at Den of Clojure (Denver meetup). So putting together as many live demoes as I can.#2016-06-3011:44cap10morganHmm, but it still doesn't work. (stest/instrument 'full-name) doesn't seem to turn on instrumentation (I get an exception when an invalid arg blows up in the fn body rather than spec telling me it's invalid) and (stest/test 'full-name) just returns (). These all worked with just (stest/instrument) and (stest/test).#2016-06-3011:46Alex Miller (Clojure team)If you use back tick it will resolve the symbol to fully qualified#2016-06-3011:47Alex Miller (Clojure team)The symbol has to be fully qualified#2016-06-3011:47Alex Miller (Clojure team)Symbol, not var#2016-06-3011:49cap10morganOK, that fixed it. Thanks, @alexmiller. I think that's the first time I've used back tick outside of a macro. Makes sense that it would need to be fully-qualified since it's going into a central registry.#2016-06-3011:59cap10morganDid clojure.spec.test.check go away or change in alpha8? Trying to change how many tests get run in clojure.spec.test/test and test's docstring says to pass a map with a :clojure.spec.test.check/opts keyword.#2016-06-3012:11Alex Miller (Clojure team)That's just a keyword namespace, not an actual namespace#2016-06-3012:22cap10morganhuh, OK. not sure how to alias it, then. but I guess I don't strictly need to.#2016-06-3012:32jjcomerDoes anyone have an example of stub or replace? Neither seem to be working for me, the original fns still execute.#2016-06-3012:52puzzler@alexmiller I thought multi-specs couldn't auto-generate data since the possibilities can't be enumerated. And if you have to write your own generator, I don't see how the retag comes into play.#2016-06-3013:25bhaumanI'm sure that this has probably been discussed before, a bunch, but I am late to the game and curious about and flowing conformed values to successive predicates. Initially, it seems very surprising that a successive predicate is operating on a different__ value. This seems semantically more like and->. Is this in the Guide? It's also curious that the underlying implementation of merge is named and-preds. I know there is a reason for this and admit that I haven't looked closely enough at it.#2016-06-3014:57fenton@puzzler: you can auto-generate data for multi-specs.#2016-06-3015:47Alex Miller (Clojure team)@puzzler did you read the doc-string for multi-spec? I think it answers all these questions.#2016-06-3015:47Alex Miller (Clojure team)@bhauman: and vs and-> is under consideration. for now, and flows conformed values.#2016-06-3015:48bhaumancool, i didn't know and-> was under consideration#2016-06-3015:49Alex Miller (Clojure team)Rich mentioned it in the back chat#2016-06-3015:51bhaumanoh and a big big + šŸ’Æ on every#2016-06-3015:53flipmokidHi, I'm looking to write a spec for a map in clojurescript which is used to check user input. One of the keys requires that the input be an int and be in a set of allowed values which comes from a server (so it's an async call). What is the best way of achieving this?#2016-06-3015:59bhaumanif its deep in the spec, make the call to the server first, and then make a binding, and have your predicate read the binding#2016-06-3016:02bhaumanbut otherwise I would separate this concern from the spec#2016-06-3016:02bhaumanand do a second pass#2016-06-3016:03Alex Miller (Clojure team)you donā€™t have to make it a fn spec to use spec - you can explicitly call s/valid? to make that check#2016-06-3016:08flipmokid@bhauman: Thanks, that make sense (having that separate from the spec). BTW thanks for figwheel! @alexmiller I wanted to originally do something like (s/and integer? (call-service)) where call-service returns a channel which eventually returns a set of ints. I wasn't sure of the best way of doing this as that function isn't a predicate I'm fairly new to clojure btw so please bear with me!#2016-06-3016:09Alex Miller (Clojure team)have you considered just writing some Clojure code? :)#2016-06-3016:10Alex Miller (Clojure team)Iā€™m not sure spec is buying you anything for what youā€™re describing#2016-06-3016:12flipmokidI will give it a go. Time to crack open Clojure Applied again šŸ™‚#2016-06-3016:14Alex Miller (Clojure team)well nothing about spec in there :)#2016-06-3016:15Alex Miller (Clojure team)those idiots were too dumb to write about it#2016-06-3016:17flipmokidLol. The book is great, thank you! I meant using the book to write some clojure code. I thought I should do the server check separately from the spec but just wanted to make sure.#2016-06-3016:17flipmokidThanks guys#2016-06-3016:17flipmokidMuch appreciated#2016-06-3016:18Alex Miller (Clojure team)you shouldnā€™t register a spec thatā€™s calling a server, but you could use s/validate? to verify a value conforms to a spec (which has a predicate, which is based on a set obtained from elsewhere)#2016-06-3016:19Alex Miller (Clojure team)however, that ā€œcheckā€ is so simple, that you might as well just check if the value is in the set (without using spec)#2016-06-3016:23flipmokidWith the first option, what would be the best way of waiting for the set to come back out of the channel in clojurescript inside the predicate? The best my mind could do was poll the channel until there was something on it. The current map spec I have does have a lot of keys which are much more involved than this one, but this is the only one that needs to check against a remote resource.#2016-06-3017:10fentonwhere do we report issues found with spec?#2016-06-3017:13fentonbut change :matching-p to :matching and it works okay... seems hyphens don't work for matching values.#2016-06-3017:16fentonhmm... hold on that... maybe dirty repl#2016-06-3018:49blanceI just read about clojure.spec and it seems like a great tool for testing with test.check. To utilize it for unit testing, should I specify a spec for every functions using fdef in my application so that I just call clojure.spec.test/test to do unit testing? I've never used schema before so I'm not sure which function/args/data I should specify those and which not to#2016-06-3018:58seancorfield@blance: Have you read the Rationale for clojure.spec? It talks about when to use spec and what sort of stuff itā€™s designed for.#2016-06-3018:59seancorfieldhttps://clojure.org/about/spec#2016-06-3018:59bhaumanSo here is another thing, the :in path on key value errors in (s/every-kv ..) end withs a 0 1 presumably to disambiguate between the key and the value, but this renders the :in path pretty tough to use to look up a value in a nested structure. I'm thinking that intention was to have the :in path terminate with the map key and the 0 or 1 should be at the end of :path only#2016-06-3018:59eggsyntax@blance: & lots more detail in the guide: http://clojure.org/guides/spec#2016-06-3019:00eggsyntaxNotably http://clojure.org/guides/spec#_testing#2016-06-3019:01blanceYea I've read both.. It must be that I read through them too fast. I'll read them again.#2016-06-3019:02blanceI mean I know how to use them, but I'm not sure on where to use it. Does it make sense to say have a fdef for each single function in my app?#2016-06-3019:03eggsyntaxA reasonable shorthand might be: any function that you would otherwise have hand-written a test for.#2016-06-3019:03eggsyntax(and of course for fns where spec is valuable in other ways, eg runtime validation of inputs)#2016-06-3019:04eggsyntaxThat's just my take on it, though, and I'm a spec beginner too. As I guess are we all šŸ˜‰#2016-06-3019:05blanceyou would want your unit test to cover as much functions as possible right? so write spec for as much functions as possible as well?#2016-06-3019:06blanceI feel like some fns would take as long time to write a spec to validate its input&output as to write the actual function itself#2016-06-3019:06eggsyntaxMatter of taste. Personally I don't shoot for complete test coverage, just coverage of fns that a) are tricky, b) are at risk for regression bugs, c) could use a test as usage documentation.#2016-06-3019:07eggsyntaxAnd yeah, it's definitely arguable that hand-written tests are a better tool to reach for in some cases than spec is. It'll be interesting to watch as different ideas about best practices for spec emerge.#2016-06-3019:08eggsyntax(although the new stub/mock functionality makes it a tool I'm likely to reach for a lot more often)#2016-06-3019:10blancethat make sense. and I would assume you only need to use clojure.spec.test/test for unit testing? Do you still hand write test to cover corner cases?#2016-06-3019:10blancewhat's the stub/mock functionality? is that also in spec?#2016-06-3019:11blancesorry I have lots of questions as I'm new to testing as well:sweat_smile:#2016-06-3019:11eggsyntax:stub replaces a fn with a stub that checks :args, then uses the :ret spec to generate a return value. https://github.com/clojure/clojure/blob/master/src/clj/clojure/spec/test.clj#2016-06-3019:12eggsyntaxTo me it seems like generative testing is, in general, more likely to catch corner cases than hand-written tests, unless there's a specific corner case you want to test for that isn't likely to be generated.#2016-06-3019:13eggsyntax(oh, :stub is an optional arg to instrument btw)#2016-06-3019:13blancethat's cool! didn't see that in the doc#2016-06-3019:13eggsyntaxIt's new as of alpha8. I think the guide isn't quite caught up yet.#2016-06-3019:14blanceIll take a look at it. Thanks!#2016-06-3019:16bfabry@fenton: there's no way and could combine the generators. there's a new function s/merge specifically for that scenario though#2016-06-3019:16fenton@bfabry: okay, will check that.#2016-06-3019:29seancorfield@blance: If you specify "every" function then your system will be brittle: changing anything will likely break specs and youā€™ll have to constantly rewrite them ā€” itā€™ll be hard to do refactoring. I think specs make sense on "module" boundaries and on "library APIs".#2016-06-3019:31seancorfieldAs for generative testing, you can test a function against a spec (in the test suite) without needing to fdef it in the main code. That seems like a reasonable approach to me.#2016-06-3019:31fentonhow to spec for a string that should be an alpha-numeric?#2016-06-3019:31seancorfieldUse a predicate with a string regex?#2016-06-3019:33fenton@seancorfield: sorry not understand, example?#2016-06-3019:34seancorfield#(re-find #"^[a-zA-Z0-9]+$" %)#2016-06-3019:34seancorfield(thereā€™s sure to be a shortcut for that pattern#2016-06-3019:35seancorfieldSo youā€™d have (def ::alphanumeric (s/and string? #(re-find #"ā€¦" %)))#2016-06-3019:37fenton@seancorfield: okay that makes sense...thx.#2016-06-3019:37seancorfield
boot.user=> (s/def ::alphanumeric (s/and string? (partial re-find #"^\w+$")))
:boot.user/alphanumeric
boot.user=> (s/exercise ::alphanumeric)
(["Q" "Q"] ["7" "7"] ["4" "4"] ["6z6" "6z6"] ["RjN" "RjN"] ["r05" "r05"] ["9" "9"] ["o8m" "o8m"] ["5j7YFcm" "5j7YFcm"] ["RFNe5Xj3" "RFNe5Xj3"])
#2016-06-3019:39fenton@seancorfield: even nicer!#2016-06-3019:39eggsyntaxIt'd be nice to do something like (s/def ::alphanumeric (s/and string? #(Integer/parseInt %))), but that'll throw an exception rather than failing...#2016-06-3019:40blance@seancorfield: I'm actually working on clojurescipt. most of my fns are api calls, transition db to new states, and UI stuffs. No need to write spec for any of them I guess?#2016-06-3019:41seancorfield@eggsyntax: you would need both a predicate to match numeric strings and a conformer to convert to Long I think?#2016-06-3019:41bhaumanI looked into the :in path problems for maps that I mentioned above, it seems like ::kfn isn't enough to eliminate the following tuple key.#2016-06-3019:41eggsyntax@seancorfield: makes sense.#2016-06-3019:41eggsyntax(& at that point the regex solution is nicer)#2016-06-3019:41seancorfield@blance: Hard to say in absolute terms. Try a few approaches and see what works best for you?#2016-06-3019:42blancesounds good. Thanks!#2016-06-3019:42seancorfieldIā€™d probably spec the API itself (between client and server) and then maybe some of the bigger components within each side. But I suspect you have a lot of "trivial" functions which wouldnā€™t be worth specā€™ing?#2016-06-3019:43eggsyntaxNew motto: "Don't spec the small stuff"#2016-06-3019:43eggsyntaxšŸ˜‰#2016-06-3019:43blanceapi calls are async, or rather returning a core.async channel#2016-06-3019:43blanceso not sure how to spec that#2016-06-3019:47bhaumanits expressive enough for me to fix it in use though#2016-06-3019:48bfabryI figure fdefs probably live at a similar level to unit tests. if you unit test every function in your application it'll be annoying to change, but unit testing still holds value at some "right" level. same with fdefs#2016-06-3019:48bfabryfdefs have the added advantage that they also give you runtime contract checking and easier to understand documentation than unit tests#2016-06-3019:49puzzler@alexmiller I read the doc-string about multi-spec but (sg/generate (s/gen my-multi-spec)) throws ExceptionInfo Unable to construct gen at: [] for: so I must be misunderstanding the point about generation.#2016-06-3020:17fenton@puzzler: did u see the example i made for u?#2016-06-3020:29puzzler@fenton no, didn't see that. Where?#2016-06-3020:29puzzler@fenton, ah, I see it now.#2016-06-3020:32puzzler@fenton, hmm, not sure why it isn't working for me. Maybe one of the individual multimethods can't generate but the error message is misleading, making it seem like the overall multispec is failing.#2016-06-3020:32fenton@puzzler: did my example work for you?#2016-06-3020:38puzzler@fenton: no, I copied and pasted your code and tried it at the REPL but I'm getting NullPointerException clojure.test.check.generators/call-gen (generators.cljc:41) This is on the latest alpha. Not sure what's going on.#2016-06-3020:41puzzler@fenton, oh I called gen and generate in the wrong order on your code. So yours is now working for me. But I had it right on my example which wasn't working. So still trying to figure that out...#2016-06-3020:43puzzler@fenton, ok figured it out. I wasn't calling gen on the keyword associated with the multispec, I was calling it on the actual multispec.#2016-06-3020:43puzzler@fenton confusing error messages#2016-06-3020:54puzzlerIn some cases the docstring info provided automatically by spec isn't terribly useful, especially in the case of multi-specs. Is there any way to manually add info to the spec docstring?#2016-06-3021:19fenton@puzzler glad u got it work....dunno about ur last comment tho. šŸ™‚#2016-06-3023:20blanceWould you put all spec defination at one place? say my.appns.spec? If so, any trick to refer to a namespaced keyword?#2016-06-3023:21blanceI assume nobody want to type {:my.appname.is.very.long/key1 "foo" :my.appname.is.very.long/key2 "bar"} `#2016-06-3023:25blanceAlso i'm wondering if clojure.spec can spec arbitrary nested vector? for example, geoJSON coordinates can be [1 1] or
[
      [ [100.0, 0.0], [101.0, 0.0], [101.0, 1.0], [100.0, 1.0], [100.0, 0.0] ],
      [ [100.2, 0.2], [100.8, 0.2], [100.8, 0.8], [100.2, 0.8], [100.2, 0.2] ]
      ]
or [ [100.0, 0.0], [101.0, 1.0] ]
#2016-06-3023:27Alex Miller (Clojure team)You have optionality and recursion, so without knowing more than those examples, yes#2016-06-3023:28Alex Miller (Clojure team)@blance if you are making a spec namespace, I would propose the plural (specs) rather than spec. We will be doing clojure.core.specs for example.#2016-06-3023:29Alex Miller (Clojure team)For namespaces, you can alias with :as in require then autoresolve with :: keywords#2016-06-3023:31blancelike my.appname.specs :as specs then ::specs/key1 right?#2016-06-3023:31Alex Miller (Clojure team)Yeah#2016-06-3023:32blancecool thanks!#2016-06-3023:41blanceOn a side note, I feel like clojure team does not want us to put all specs at one place, otherwise we would have the option to just describe un-namespaced key which is more convenient.#2016-07-0103:30seancorfieldIā€™m working with instants and ended up with this code:
(defn years-ago
  "Given a number of years, return an Instant that long ago."
  [y]
  (-> (LocalDate/now) (.minusYears y) .atStartOfDay (.atOffset ZoneOffset/UTC) .toInstant))

(defn age-18-120?
  "Given an Instant, return true if it represents a date of birth such
  that someone would be between 18 and 120 years old."
  [i]
  (s/inst-in-range? (years-ago 120) (years-ago 18) i))

(s/def ::date-of-birth (s/with-gen age-18-120?
                         (fn [] (s/gen (s/inst-in (years-ago 120) (years-ago 18))))))
Could this be done more cleanly? (without the duplication of the range of years)
#2016-07-0104:24bfabry@seancorfield:
(s/def ::age-18-120? (s/inst-in (years-ago 120) (years-ago 18)))
=> :kafka-google-connector.runner/age-18-120?
(defn age-18-120? [x] (s/valid? ::age-18-120? x))
=> #'kafka-google-connector.runner/age-18-120?
(s/def ::date-of-birth ::age-18-120?)
=> :kafka-google-connector.runner/date-of-birth
(s/exercise ::date-of-birth 1)
=> ([#inst"1970-01-01T00:00:00.000-00:00" #inst"1970-01-01T00:00:00.000-00:00"])
#2016-07-0104:26bfabrythough the age-18-120? fn seems totally redundant at that point, but if you still wanted it#2016-07-0104:28seancorfieldNo, because the years-ago calls must happen when the predicate is applied, not just once at compile time.#2016-07-0104:28bfabryriiiiight, makes sense#2016-07-0104:34seancorfieldAnd I'm assuming the with-gen generator-returning fn is called each whenever the spec is exercised / test-gen'd but that's not really as important since tests are short-lived, whereas the spec has to be long-lived and check the correct range of dob each time it's conformed / used for validation.#2016-07-0109:58Oliver George@seancorfield: could "now" be part of the data structure you spec/validate? that makes it a simple input rather than implied context.#2016-07-0111:13Oliver GeorgeWill it make sense to update defn to allow a spec to be provided as metadata? seems like it could overlap with pre/post...
(defn option-match
  "Default search for local datasource: case-insensitive substring match"
  [simple? option query]
  {:args (s/cat :simple? boolean? :option ::option :query string?)}
#2016-07-0111:58Alex Miller (Clojure team)No, we won't be doing that#2016-07-0113:09seantempestaSo, specs can also be used to convert invalid data into valid data? Iā€™m reading thisā€¦
conformer
macro
Usage: (conformer f)
       (conformer f unf)
takes a predicate function with the semantics of conform i.e. it should return either a
(possibly converted) value or :clojure.spec/invalid, and returns a
spec that uses it as a predicate/conformer. Optionally takes a
second fn that does unform of result of first
So how do you use the feature to get the response (possibly converted)?
#2016-07-0113:11bhauman@seantempesta: you have to provide a function that returns :invalid or a result that is the converted value#2016-07-0113:12bhauman(conformer (fn [a] (if (= a "one") 1 :clojure.spec/invalid)))#2016-07-0113:20seantempestaholy crap, this is great!#2016-07-0113:20seantempestathanks @bhauman!#2016-07-0114:10ghadiwould love to see this answered @alexmiller https://clojurians.slack.com/files/glv/F1MQPJBQT/Specifying_an____annotated_value_map_.md#2016-07-0115:33wilkerlucio@ghadi: one simple option that I see is to move the cached data to the map meta-data, this way the map validation can stay simple#2016-07-0115:36glvBut Iā€™d like to validate the cached data as well. For purposes of the code, itā€™s not valid if those keys arenā€™t there.#2016-07-0115:36glv(That was clumsily worded, but you know what I mean ā€¦ the code expects those keys.)#2016-07-0115:44wilkerlucio@glv: I had a different idea, what you think on this:#2016-07-0115:44wilkerluciohere, I used a conformer to remove the namespaced keys before doing the map validation, but checking the keys before doing it#2016-07-0115:44wilkerluciothis way all namespaced keys will be validated, then removed just to check the rest of the map#2016-07-0115:53glvHmm ā€¦ thatā€™s better than my current horrific solution. šŸ™‚#2016-07-0116:30seancorfield@olivergeorge: Interesting ideaā€¦ The data structure comes from the database and we already decorate it with some computed fields that are transient (short-lived) so adding a current timestamp to the data wouldnā€™t be a hardship.#2016-07-0116:56Alex Miller (Clojure team)@ghadi: sorry, moving today!#2016-07-0116:56jjcomerIā€™m trying to use spec-test/test in my clojure.test tests (that was a lot of test :)). When I try to realize the result of test I get the following exception
java.lang.ClassCastException: clojure.lang.AFunction$1 cannot be cast to clojure.lang.MultiFn, compiling:(clojure/test/check/clojure_test.cljc:95:1)
#2016-07-0116:56ghadi@alexmiller: good luck!#2016-07-0116:56jjcomerIf I run the test outside of a deftest it works no problem#2016-07-0116:57jjcomerDoes anyone have a strategy they are using to use spec-test/test within a deftest?#2016-07-0117:18glv@alexmiller no worries ā€¦ whenever things calm down is fine.#2016-07-0119:14akielWhat is the idiomatic way to spec an empty :args seq? I currently use (s/cat).#2016-07-0212:03leongrapenthinSeems like the "annotated value map" is also asked for here: https://stackoverflow.com/questions/38151446/how-can-i-spec-a-hybrid-map#2016-07-0314:26Alex Miller (Clojure team)This is explained in the generators with s/and section in http://clojure.org/guides/spec #2016-07-0314:27Alex Miller (Clojure team)You should probably also look at s/int-in for int ranges with gen support#2016-07-0321:50bhaumanI'm curious if it's an eventual goal to make explain's :in path work with core functions like get-in? Right now the spec'ing of maps as a sequential collection via (every (tuple k? v?)) is causing some ambiguity in the :in path.#2016-07-0408:16dhruv@alexmiller: thank you very much! That explanation clears it up for me, plus s/int-in works a treat for my purposes :+1::skin-tone-3:#2016-07-0413:51eggsyntaxIt'd be useful to have an undef fn or something like that, to clear a spec from the registry. Or some way to clear the registry entirely. Mostly just for use during development. Or, for that matter, just make the registry-ref atom publicly accessible.#2016-07-0414:03rauh@eggsyntax: Hack: (reset! @#'spec/registry-ref {})#2016-07-0414:04eggsyntax@rauh: Oooh, slick. Thanks!#2016-07-0414:07eggsyntaxI didn't realize that'd get around the private declaration. Good to know in general.#2016-07-0414:32Alex Miller (Clojure team)@eggsyntax weā€™ve talked about this and I expect there will be something public like this#2016-07-0414:32Alex Miller (Clojure team)eventually :)#2016-07-0414:32eggsyntaxSounds good. & Andre's hack will do me fine for now šŸ™‚#2016-07-0501:50wildermuthnReading ā€œOut of the Tarpitā€ (http://shaffner.us/cs/papers/tarpit.pdf), and maybe Iā€™ve just got Spec on my mind, but this looks pretty familiar:#2016-07-0501:50wildermuthnhttps://www.dropbox.com/s/fjh9fi3r0e4ldt2/Screenshot%202016-07-04%2021.50.02.png?dl=0#2016-07-0502:04wildermuthnThe articleā€™s description of "Functional Relational Programmingā€ includes a section on ā€œEssential Logicā€ that seems to correspond very well with Spec!#2016-07-0509:58Alex Miller (Clojure team)I think it's no accident that Clojure+Datomic overlap pretty well with Out of the Tarpit#2016-07-0513:06Alex Miller (Clojure team)s/and flows conformed values through the predicates and the s/+ conforms to a vector#2016-07-0513:07Alex Miller (Clojure team)Rich has mentioned a split into s/and and s/and-> to have both choices#2016-07-0514:22vikeri@alexmiller: Thanks! Yes it could probably be useful sometimes to not have it flow through the predicates but evaluate them separately.#2016-07-0515:01Alex Miller (Clojure team)separately, the need to validate this specific case of a regex and vector? is common and one Iā€™ve run into many times in specā€™ing macros and there might be some addition specifically for this#2016-07-0520:27aengelbergHas there been any talk about allowing local registries to use with clojure spec functions rather than only using the global registry? Similar to (make-hierarchy) and derive.#2016-07-0520:35Alex Miller (Clojure team)I haven't heard such talk :)#2016-07-0520:51pdlugIā€™m trying to use multi-spec but it doesnā€™t seem to work with defmethodā€™s :default dispatch, is this a bug or intentional?#2016-07-0520:55kidpolloI donā€™t think multi-spec behaves exactly like that. I believe there is no default to be had.#2016-07-0521:00pdlugThatā€™s unfortunate. Is there a better way to handle the validation where a field in the map determines the type and you want to explicitly constrain some types but provide a default base?#2016-07-0521:01pdlugEx: modeling analytics events, the :type provides the type of event, for :search events :terms are required, for :page-view events :url is required but for all other events there are no other constraints on the map#2016-07-0521:02kidpolloIt seems it only dispatches on key defined in s/multi-spec not on the this#2016-07-0521:05kidpollohmmm but I just tried (s/def :foo/bad-entity (s/multi-spec entity :does-not-exist)) (gen/sample (s/gen :foo/bad-entity) 1) and the program goes into an infinite loop#2016-07-0521:06kidpollowait! not infinite loop I just get ExceptionInfo Couldn't satisfy such-that predicate after 100 tries.#2016-07-0521:07kidpollomakes sense it either matches a spec or it does not. I am back to my original argument šŸ˜›#2016-07-0521:08pdlugI think itā€™s unexpected behavior if multi-spec works w/ multimethods but then doesnā€™t support the default behavior of them#2016-07-0521:08kidpollo@pdlug: yah I had the same issue I want to dispatch on type but it has to be based on a key on your map šŸ˜ž#2016-07-0521:08pdlugI still donā€™t see anything else that supports my example use case#2016-07-0521:11kidpolloyeah, my problem is that I want to dispatch on something else than a namespaced key on the map. I want to dispatch on meta from the map. Not always you want to expose the ā€œtypeā€ on the actual entity#2016-07-0521:13pdlugmakes sense#2016-07-0521:14pdlugah, I found this open issue: http://dev.clojure.org/jira/browse/CLJ-1935#2016-07-0521:16kidpolloright!#2016-07-0521:47Alex Miller (Clojure team)@pdlug you might also see if s/merge helps you in combining base+specific s/keys#2016-07-0522:03kidpolloYeah but you would be able to have a generic event spec for all the other types.#2016-07-0612:23ikitommiIs there a easy way to select a conformer based on external input? E.g. for the web - conform strings to numbers only for query parameters, not for json params.#2016-07-0614:51pdlug@alexmiller: Iā€™m not familiar with the clojure bug process, that issue CLJ-1935 says the status is ā€œTriagedā€ what does that mean exactly? Is there a way to tell if it will be addressed or slated for a release?#2016-07-0614:57artemyarulinhi, what is the right way to enable spec checks globally for my project? I would like that by default everything got checked. I guess I can put something like that in project file:
:repl-options {:init (do (clojure.spec/instrument))}
and then everything should be checked when Iā€™m in a REPL? Oh whatā€™s the right way?
#2016-07-0614:58petterik@pdlug: You can find the entire JIRA workflow here "How a Ticket Becomes a Commit": http://dev.clojure.org/display/community/JIRA+workflow I found it very useful#2016-07-0614:59pdlug@petterik: ah thatā€™s very helpful. Thanks!#2016-07-0615:53Alex Miller (Clojure team)@pdlug in this case, it means ā€œI think Rich should look at itā€ but he has not yet#2016-07-0616:23pdlug@alexmiller: ok, thanks#2016-07-0701:50seancorfieldI thought alias had been updated so you didnā€™t have to have a real namespace for the second argument?#2016-07-0701:52seancorfieldIt seems not. I was hoping to use alias to shorten a lot of qualified keywords (without having to actually load a namespace).#2016-07-0702:13Alex Miller (Clojure team)no, weā€™re just considering that#2016-07-0702:49seancorfieldNow weā€™re beginning to use namespace-qualified keywords a lot more, I can say that would be a very convenient addition! šŸ™‚#2016-07-0703:18Alex Miller (Clojure team)as a short-term workaround you can do something like this: https://github.com/clojure/clojure/blob/master/src/clj/clojure/spec/test.clj#L17-L19#2016-07-0703:37seancorfieldThatā€™s an interesting hack! šŸ™‚#2016-07-0706:07mandragoranI was wondering how I would spec an async function (a function returning a core.async channel). Would I need to write an async version of fdef or is there another way?#2016-07-0708:19wagjoI'm a bit struggling to choose between :foo.vocabulary/term-uri and :foo.vocabulary.term/uri or just :term/uri. I know it's probably a bit soon, but any best practices for choosing the name(space)s?#2016-07-0709:14mandragoran@wagjo: I've been taking advantage of the new ::alias/some-keyword feature. So I'd probably call it ::term/uri and map the term alias in all namespaces using it to foo.vocabulary.term.#2016-07-0710:13vikeriHow can I reference the first argument of a function in fdef :arg when the argument does not have a name since Iā€™m destructuring it immediately with {:keys [a b]}?#2016-07-0710:48vikerinvm, realized the keywords in :arg (s/cat are not tied to the variable names#2016-07-0711:07Alex Miller (Clojure team)@wagjo if you're in a lib or other public project, you should add enough context to be distinguishable from anyone else (something like your maven group/artifact id). If in a private app, I would probably do less just for typing sake. I think it will be useful to put specs for the domain in a small number of namespaces (maybe 1). It might be useful to separate generic specs from domain attributes (which alias them) too.#2016-07-0711:10Alex Miller (Clojure team)@mandragoran: we will probably have some async related specs eventually. Until we do I think it's better to do type checks on ReadPort and WritePort than specifically on ManyToManyChannel as those are more generic#2016-07-0712:08gfredericksad-hoc higher-order specs?#2016-07-0712:45lvhHow do I write a spec for a map of unqualified keywords where the keyword itself is something dumb thatā€™s going to collide a lot? In particular, swagger (the api format) has a toplevel key, also called ā€œswaggerā€.#2016-07-0713:08Alex Miller (Clojure team)can you give an example? not getting the question.#2016-07-0714:06lvhSure! Letā€™s say Iā€™m parsing swagger#2016-07-0714:06lvh
(def sample-swagger
  {:swagger "2.0"
   :info {:title "my sample api for puredanger"
          :version "elventy flive"}})
#2016-07-0714:07lvhLetā€™s say Iā€™m in a ns defining swagger in general (doesnā€™t matter much, could also be :swagger/whatever). Iā€™m doing something like:
(spec/def ::swagger
  (spec/keys :req-un [:swagger/swagger
                      ::info]))

(spec/def :swagger/swagger
  ;; The top-level key is called swagger. This feels dumb.
  #{"2.0"})

(spec/def ::info
  (spec/keys :req-un [::api-title
                      ::api-version]))

(spec/def ::api-title string?)
(spec/def ::api-version string?)
#2016-07-0714:08lvhIt seems like Iā€™m naming :swagger/swagger what it is because :req-un expects to find the names of map keys that way, but having a ::swagger and a :swagger/swagger looks confusing#2016-07-0714:09lvhSimilarly, ::api-title actually needs to be : or something, but the tons of nsā€™es make it annoying to type#2016-07-0714:09lvhI guess Iā€™m asking if thereā€™s a way to distinguish between the key under which something appears in a map, and its spec name#2016-07-0714:10lvhIIUC there are some new reader macros in the current alpha, I donā€™t know if they help with this#2016-07-0714:10lvhthey might limit the typing to just the spec defs so at least the destructuring looks good, I suppose?#2016-07-0714:12lvhif I had a choice, in this code, I would refer to that specced value (esp. after conform) as ::api-title ā€” the only reason I wouldnā€™t do that is because the JSON Iā€™m getting from swagger (a standard, so I canā€™t change it) uses keys that donā€™t collaborate well with that#2016-07-0714:13lvhIf I could provide, say, an alias, or say that under key :title Iā€™m expecting to see spec ::api-title or something, thatā€™d help too#2016-07-0714:16lvhbailing out and using a fn to just assert that some keys are some particular value doesnā€™t feel very idiomatic either#2016-07-0714:33lvhI guess you can define specs in terms of other specs? That doesnā€™t seem to help a lot but Iā€™ll play with it#2016-07-0715:40donaldballHas anyone run across or is writing a lib of commonly useful strong specs, e.g. for urls, hostnames, unix ports, valid sql identifiers, etc.?#2016-07-0715:52Alex Miller (Clojure team)I haven't seen anyone doing so yet but it seems like a reasonable and perhaps inevitable thing to do #2016-07-0716:40Alex Miller (Clojure team)@lvh aliasing is a good and useful thing, so creating api-title and api-version are just fine as base specs (you may want to use some other namespace though like :swagger/api-title or :swagger.api/title), then s/def code-localized versions to match the unqualified attributes that alias those base specs, like (s/def ::title :swagger/api-title)#2016-07-0716:41lvhOK, so I define the ā€œrealā€ spec under :swagger.ap/title, and then a local alias. Makes sense. Thanks!#2016-07-0716:42Alex Miller (Clojure team)if you create the keyword namespace as an actual namespace, you can alias it, then use the alias in the autoresolved keyword like ::sw/api-title (after something like (alias ā€˜sw ā€˜swagger.api))#2016-07-0716:44Alex Miller (Clojure team)the key there is that alias currently has to refer to a real (loaded) namespace. That might change still but a workaround is what you see here https://github.com/clojure/clojure/blob/master/src/clj/clojure/spec/test.clj#L17-L19#2016-07-0716:44Alex Miller (Clojure team)switch to (and thus create) the namespace, then switch back, and alias#2016-07-0717:32leongrapenthinToday we wanted to create a spec that validates a percentage in five percent steps. It occured to us that int-in doesn't support a step argument like range does. Has this been discussed or is a ticket worth a shot?#2016-07-0717:39sparkofreasonBefore I get too far in reinventing the wheel, is anyone else out there working on converting spec to Datomic/Datascript schema?#2016-07-0717:48Alex Miller (Clojure team)@leongrapenthin: no, it's not going to do that#2016-07-0717:49leongrapenthin@dave.dixon: I have thought about it and realized that generating a spec from Schema is likely more desirable#2016-07-0717:50donaldballIā€™m not sure what Iā€™m doing wrong with this simple spec:#2016-07-0717:50donaldball
(s/def ::foo (s/spec #(= :foo %) :gen (gen/return :foo)))
:user/foo
user> (s/gen (s/get-spec ::foo))
ClassCastException clojure.test.check.generators.Generator cannot be cast to clojure.lang.IFn  clojure.spec/spec-impl/reify--13361 (spec.clj:800)
#2016-07-0717:50leongrapenthin@alexmiller: Ok, thanks#2016-07-0717:50Alex Miller (Clojure team)@leongrapenthin: just do (set (range 0 100 5))#2016-07-0717:51donaldballOops, I misread the contract for s/spec, carry on#2016-07-0717:51leongrapenthin@donaldball: Wrap (gen/return ...) in a (fn [] )#2016-07-0717:51Alex Miller (Clojure team)yep, or put # at the beginning#2016-07-0717:52Alex Miller (Clojure team)
(s/def ::foo (s/spec #(= :foo %) :gen #(gen/return :foo)))
(gen/sample (s/gen ::foo))
#2016-07-0717:55leongrapenthin@alexmiller: Yeah, but that's gonna be quite the error message šŸ™‚ - We came up with (s/and (s/int-in 0 101) #(zero? (mod % 5))) if I remember correctly. Generator worked. Projecting this scenario on per permille or larger spaces was what made us wonder why a step is not supported.#2016-07-0717:56Alex Miller (Clojure team)because itā€™s a min/max check, and youā€™re specifying enumerated values. thatā€™s just not what it is intended to do.#2016-07-0717:56Alex Miller (Clojure team)I think your spec is better anyways#2016-07-0717:57Alex Miller (Clojure team)you could combine int-in with it too#2016-07-0717:57Alex Miller (Clojure team)nvm, youā€™re doing that#2016-07-0717:58leongrapenthinYeah, the reasoning makes sense#2016-07-0717:59Alex Miller (Clojure team)the hard part of int-in is generating uniformly across a well defined set and youā€™re leveraging that by building on top of it#2016-07-0718:00leongrapenthinLooking at that from a different perspective it is only enumerated because of the min-max check#2016-07-0718:00leongrapenthinYou could argue that everything in a min-max interval is enumerated then#2016-07-0718:01Alex Miller (Clojure team)well enumerations have obvious usage issues when there are a lot of values :)#2016-07-0718:04leongrapenthinMy case is a very specific enumeration and can't be discarded as just any enumeration. Every step range of allowed values is a subset of an integer range with the same boundaries.#2016-07-0718:05leongrapenthinSo from that perspective I think it still makes sense as a feature of a min/max check.#2016-07-0718:05leongrapenthinThe min/max check is also an enumeration with a step of 1, then.#2016-07-0718:14leongrapenthinBetter said, for our usecase it would be preferable if it was an in-range check than a min/max check with the word range meant in the same way as in clojure.core/range#2016-07-0718:17Alex Miller (Clojure team)well, range does a lot of stuff (reverse ranges, infinite ranges, half-open ranges, support for all number types, etc)#2016-07-0718:18Alex Miller (Clojure team)in-range implies to me a check that would need to enumerate the range to determine conformance and gen would be a lot harder.#2016-07-0718:18Alex Miller (Clojure team)int-in is intentionally much narrower#2016-07-0718:25leongrapenthinAgreed. I think what I want to say is that int-in as a name doesn't necessarily limit the spec to a min/max check#2016-07-0718:27leongrapenthinA fully featured in-range is likely an overkill for the rare amount of usecases#2016-07-0718:44leongrapenthinThanks for discussing. Having thought about it I won't be able to resist to submit a small patch over the weekend for reconsideration, though šŸ™‚#2016-07-0718:46Alex Miller (Clojure team)I think Rich is unlikely to ok that#2016-07-0718:48kidpolloI am running stest/check on an fdef that never finishes. It just keeps going and I don't know If its stuck or what is the deal. I have independently tested the input and output specs for possible problems and am getting nowhere. Any tips on how to debug this?#2016-07-0718:58kidpolloexercise-fn seems to work fine#2016-07-0719:02leongrapenthin@kidpollo: Isolating the problem by starting with dumbed down specs and adding until the problem occurs?#2016-07-0719:02kidpolloI also tried (s/exercise-fn my-fn 100) and it takes like 1 min but it works.#2016-07-0719:03leongrapenthin@kidpollo: Probably recursion in s/map-of or every-kv?#2016-07-0719:04leongrapenthinMight be that it just takes so long to print the result because it prints a huge generated value#2016-07-0719:05kidpolloCould be.#2016-07-0719:05kidpolloI am running on the repl#2016-07-0719:05kidpolloon intellij#2016-07-0719:06leongrapenthinYou could eliminate this quickly with setting print-length#2016-07-0719:07Alex Miller (Clojure team)dump stack and see what itā€™s doing#2016-07-0719:07Alex Miller (Clojure team)I think intellij has a button for it#2016-07-0719:08kidpolloohh let me see#2016-07-0719:10Alex Miller (Clojure team)I guess maybe thatā€™s only in the debugger mode#2016-07-0719:14Alex Miller (Clojure team)
(map println (.dumpAllThreads (java.lang.management.ManagementFactory/getThreadMXBean) false false))
#2016-07-0719:14Alex Miller (Clojure team)will serve in a pinch#2016-07-0719:16kidpolloyah I was able to dump them in debug mode#2016-07-0719:16kidpollojust see a bunch of locked threads in generators#2016-07-0719:18kidpollodamn 700% cpu usage
#2016-07-0719:21kidpolloI guess i need to find what spec is going crazy. Start with simpler specs maybe? The input and output to this fn are multispecs. The maps are not so complicated maps.#2016-07-0719:47kidpollo@alexmiller @leongrapenthin the problem seems to be happening in alpha-9. I went back to alpha 8 and the test finishes. It takes a while but it finishes#2016-07-0719:57Alex Miller (Clojure team)I think check changed from 100 to 1000 iterations in alpha9#2016-07-0720:20kidpollooh my that would explain it#2016-07-0720:20kidpollowhen it produced errors it did end earlier#2016-07-0720:21kidpolloI guess Ill leave it over night to see how long it actually takes#2016-07-0720:40Alex Miller (Clojure team)That count is an option you can pass to check though iirc#2016-07-0721:12kidpollowhat is iirc?#2016-07-0721:13jr(if I recall correctly)#2016-07-0721:17kidpollonice! snme in spanish šŸ˜› (si no me equivoco)#2016-07-0721:18kidpollolol ^ totally does not exist in spanish šŸ˜›#2016-07-0721:26kidpolloIt seems it would look like:#2016-07-0721:28kidpolloIs it encouraged that all clojure projects start using fully qualified keywords?#2016-07-0721:47Alex Miller (Clojure team)Nothing categorical. Qualified keywords have utility - if they make sense for you, use them#2016-07-0721:48Alex Miller (Clojure team)I think spec and the other syntax changes have increased their utility and decreased their verbosity, maybe changing that equation#2016-07-0721:52danielcomptonAnd their visibility šŸ™‚#2016-07-0721:52Alex Miller (Clojure team)Well I don't think that changes their utility :)#2016-07-0721:53seancorfieldI can see us (World Singles) using qualified keywords very heavily as it will make some of our code much more explicit. Hence the recent changes to java.jdbc to make it easier to get qualified keywords back in result sets (the new :qualifier option to everything that produces result sets).#2016-07-0722:43ajssIn spec/fdef, is there any way to make one element of :args depend on another? I've got a function that takes two maps, but wants the keys of each map to be the same.#2016-07-0722:55kidpolloyou can define the relationship between inputs and outputs in :fn#2016-07-0722:55glvRight, but not constrain inputs based on relationships.#2016-07-0722:56glv(For example, I have functions that take a grid and a coordinate within that grid, and obviously the coordinate must be within the bounds of the grid to be valid. But I donā€™t think thereā€™s a way of expressing that.)#2016-07-0722:58kidpolloSounds similar to the ranger-rand example in http://clojure.org/guides/spec#_spec_ing_functions#2016-07-0722:59kidpolloinstead of using :ret just use named args in :args#2016-07-0723:01ajssso (s/and (s/cat :a map? šŸ˜› map?) #(= (keys (:a %)) (keys (:b %))))?#2016-07-0723:02ajssah - ": b" = šŸ˜›#2016-07-0723:03ajsswill run out of generator iterations won't it?#2016-07-0723:23kidpollowell you would need to replace map? with a more "narrowed down" spec#2016-07-0723:28ajssah, yes that would work. Thanks šŸ™‚#2016-07-0723:31Alex Miller (Clojure team)@glv you can s/and an arbitrary predicate in the :args spec#2016-07-0723:33Alex Miller (Clojure team)and flows conformed values so it should already be nicely destructured if you add it last#2016-07-0723:35glvAh, gotcha. Iā€™ll try to incorporate that into my specs when I get a chance.#2016-07-0723:38glv(Also, a reminder about https://clojurians.slack.com/files/glv/F1MQPJBQT/Specifying_an____annotated_value_map_.md (and @wilkerlucioā€™s solution https://clojurians.slack.com/archives/clojure-spec/p1467387857001732)#2016-07-0800:49donaldballI started a library with some ostensibly generally useful specs, e.g.: https://github.com/SparkFund/useful-specs/blob/master/src/specs/internet.clj#2016-07-0800:49donaldballWould very much appreciate feedback on design, etc. Itā€™s my second day using spec in anger, as it were.#2016-07-0801:25Oliver Georgeit'd be nice if there was a way to modify how instrument reports. the default spec-checking-fn can be super verbose when turning values to str for the ex-info message.#2016-07-0801:26Oliver GeorgeHere's my quick workaround. Not perfect but gives me something nicer to investigate: https://gist.github.com/olivergeorge/c4a26886b7bbd05f2e9018614a2213ff#2016-07-0801:26Oliver GeorgeI rely on cljs-devtools to make the console.debug output to be expadable#2016-07-0802:07Alex Miller (Clojure team)@glv I have much to report re the ā€œhybrid mapā€ stuff, but not quite ready yet to do so#2016-07-0802:08Alex Miller (Clojure team)have been working on it much of yesterday and today#2016-07-0802:11Alex Miller (Clojure team)@olivergeorge: you have explain-data ex-data coming back in the exception - why not write a function that customizes output from that, rather than hacking spec-checking-fn?#2016-07-0802:13glvThat tells me that it's a legit problem that y'all are taking seriously and working on, so I'm cool with that. :-)#2016-07-0802:16bbloom@glv: There some considerable advantages to just using another map. Iā€™m curious what @alexmiller is working on, but still: you should just add a :grid key or similar#2016-07-0802:17bbloomwhat if you want to be able to represent a board without a starting coordinate?#2016-07-0802:17bbloomyou can easily take a grid and merge in some extra options, but how easy is it to dissoc them?#2016-07-0802:18bbloomwhat if you want to change the representation? now youā€™ve coupled your param spec with your grid spec#2016-07-0802:18bbloomwhat advantage is there to abusing the fact that keywords and grid coordinates are disjoint? slightly shorter syntax? seems not worth the complexity#2016-07-0802:18glvThis isn't the representation of the grid; it's the result of a particular kind of analysis of the grid, which always, inherently has a starting coord.#2016-07-0802:19bbloomcomposite structures come in two flavors: homogenous and heterogenous#2016-07-0802:19bbloomof course if you have disjoint key sets, you can combine them#2016-07-0802:19bbloombut itā€™s just clearer to keep them separate#2016-07-0802:20bbloomalso when combined, you canā€™t enumerate keys without filtering extra keys#2016-07-0802:20glvI'm certainly willing to entertain arguments that I should push the coord->distance mapping down a level.#2016-07-0802:20bbloomi guess iā€™m making such arguments šŸ˜›#2016-07-0802:21bbloombut more importantly: what advantage is there to flattening it?#2016-07-0802:21glvSo far, I don't find them convincing. :-)#2016-07-0802:21glvI see three advantages.#2016-07-0802:21Alex Miller (Clojure team)@glv totally legit - I have a couple different solutions with existing tools and something new for just this problem that may help if it comes together#2016-07-0802:22bbloomiā€™m not saying hybrid maps are never useful#2016-07-0802:23bbloomiā€™m saying that itā€™s pretty rare and barring additional info, i donā€™t think youā€™ve found a justified use case šŸ™‚#2016-07-0802:25glv1: the coord->distance mapping is the primary result of the algorithm. The ::start-coord, ::max-coord, and ::max-distance entries are annotations that could be reconstructed (with some cost) if they weren't present. But the only way to make a non-hybrid map would be to leave the secondary annotations at the top level and push the primary info down into a nested map. That doesn't seem right to me.#2016-07-0802:26bbloomyou can return those keys + a ::grid key and then create a helper function which simply extracts the grid key ā€” then the caller can decide what is ā€œprimaryā€ or not#2016-07-0802:26glv2: the code that's involvedā€”both the code that builds the map and that which uses itā€”would get more complex and clumsy (and slower) that way#2016-07-0802:27bbloomreally? how?#2016-07-0802:29glv3: That's the way it is now, and it has never caused a hint of a problem; the only reason to change would be to accommodate clojure.spec. #2016-07-0802:29Alex Miller (Clojure team)I donā€™t think itā€™s that rare/weird#2016-07-0802:29Alex Miller (Clojure team)others (including me) have run into same problem#2016-07-0802:30glvHow? Isn't that obvious? The vast majority of the updates and accesses will be to the coord->distance mapping, so pushing that a level deeper means extra lookups in the vast majority of cases. #2016-07-0802:31bbloomalexmiller: any examples you can share? i donā€™t think iā€™ve ever encountered such a hybrid map that didnā€™t (surprisingly quickly) need a (->> hybrid keys (filter #(ā€¦ that could just be replaced by (-> hybrid :foo keys)#2016-07-0802:31bbloom@glv: just build the grid and then assoc the extra keys in at the end#2016-07-0802:31Alex Miller (Clojure team)sure, I ran into it with map destructuring (so, a syntax example) - the base map is symbol->any but also special options like :keys, :syms, :or, :as etc#2016-07-0802:32bbloomalexmiller: heh, yeeaaaah i was wondering if you were going to mention macros. destructuring is like the ultimate core macro šŸ˜‰#2016-07-0802:32Alex Miller (Clojure team)itā€™s fun#2016-07-0802:32bbloom@alexmiller itā€™s funny b/c syntax is the one place where ā€œjust a little bit shorter syntaxā€ is totally worth it for a common operation#2016-07-0802:32glv@bbloom: even if that were sensible (it's not) it would only help on the creation side of the process, not the use side.#2016-07-0802:34bbloom(:grid (let [start 123, m ā€¦build up m hereā€¦.] {:start start :m m}))#2016-07-0802:34glvIt's not sensible because keeping track of those values through the process (so that you could assoc them in at the end) would be clumsy compared to just tracking them in the map as I go.#2016-07-0802:34bbloomseems pretty sensible to me šŸ™‚#2016-07-0802:35bbloomer i meant: :grid m#2016-07-0802:35glv::start-coord is predefined; the other two emerge out of the process itself.#2016-07-0802:35bbloomgotcha#2016-07-0802:35bbloomiā€™d use mutation šŸ˜‰#2016-07-0802:37bbloom@alexmiller: one thing iā€™ve discovered tinkering on language design: it turns out that language designers have to do the things that they tell other people not to do, heh#2016-07-0802:37Alex Miller (Clojure team)that is a thing#2016-07-0802:38glvcf. Stu's recent tweet about why clojure.core can't be idiomatic. #2016-07-0802:40glvhttps://twitter.com/stuarthalloway/status/741860271925432324#2016-07-0802:41bbloomanyway, @glv definitely wait to see what alex et al come up with. the emitting things during the middle is IMO almost a reason to abuse the key space šŸ™‚#2016-07-0802:41glv"abuse" :-)#2016-07-0802:42bbloomiā€™m not above loaded language šŸ˜›#2016-07-0802:43bbloomthanks for the discussion! gotta run#2016-07-0802:43glvSure!#2016-07-0806:41Oliver George@alexmiller thanks for the suggestion. I would love to do that but don't know how. I presume you are suggesting a big try/catch around my code... my code is typically om/re-frame UI stuff which complicates that.#2016-07-0806:41Oliver GeorgePerhaps I'm missing something obvious#2016-07-0814:01donaldballJust to confirm, we donā€™t yet have a way to unite the specs for e.g. homogenous (`map-of`) and heterogenous (`keys`) maps?#2016-07-0814:06glvOnly by doing something like this: https://clojurians.slack.com/archives/clojure-spec/p1467387857001732#2016-07-0814:07glvWhich isn't bad at all, but could be made more expressive.#2016-07-0814:14Alex Miller (Clojure team)no, thatā€™s bad :)#2016-07-0814:14Alex Miller (Clojure team)b/c it turns it from a declaration of truth into a process#2016-07-0814:15Alex Miller (Clojure team)there are a couple of ways to do it with existing tools#2016-07-0814:18Alex Miller (Clojure team)one other way is to treat the map as a coll-of s/or of s/tuples (where each tuple describes different kinds of map entries). For cases where the ā€œhybridā€-ness does not include registered attributes, this is probably a good choice. So something like a map that was either string->string or number->number.#2016-07-0814:19Alex Miller (Clojure team)you can handle registered attributes in the same way, but then you lose the goodness of s/keys and the attribute semantics#2016-07-0814:20Alex Miller (Clojure team)so in that case, you want to s/merge an s/keys (for the registered attributes) with something that more accurately states the truth of the map definition#2016-07-0814:20Alex Miller (Clojure team)sorry for not being detailed here - I hope to write a blog with the detail#2016-07-0814:21Alex Miller (Clojure team)and then Iā€™m working on something new that will make this a little more palatable, but itā€™s not quite done yet#2016-07-0814:23glvWell, compared to my attempt at a solution, itā€™s not bad. šŸ™‚#2016-07-0817:29eggsyntaxIn 1.9, is there a way to disable namespaced-map output (eg for data that's going to be parsed by a process that's unaware of the idiom)? ie to get output like {:foo/bar 1} rather than #:foo{:bar 1}? [Edit:simplify example]#2016-07-0817:53eggsyntaxAnswer, if anyone's curious: CLJ-1967 will address this. For now ya gotta hack it if ya need it. http://dev.clojure.org/jira/browse/CLJ-1967#2016-07-0818:12donaldballAre double-in and friends macros so that clojure core doesnā€™t require test check or as a performance optimization?#2016-07-0819:56donaldballexercise-fn is pretty rad, but correct me if Iā€™m wrong, it looks like if the fn in question has an options map specified with s/keys using :opt or :opt-un, the chances of getting a map with those keys isā€¦ small#2016-07-0819:57donaldballShould the generator for s/keys bias towards optional keysā€™ presence?#2016-07-0820:08jrI think they are macros because they build on other macros (like spec)#2016-07-0820:11stuarthalloway@donaldball: double-in is a macro to capture the form for error reporting, IIRC#2016-07-0820:18Alex Miller (Clojure team)in general, all of the spec creators are macros to capture forms#2016-07-0820:20donaldballIā€™m afraid the implication is not clear to me šŸ˜•#2016-07-0820:20donaldballI just wrote a decimal-in spec creator fn: https://github.com/SparkFund/useful-specs/blob/master/src/specs/number.clj#L6#2016-07-0820:21donaldballWhat am I missing by (lazily) writing it as a fn instead of a macro?#2016-07-0820:26Alex Miller (Clojure team)
(s/explain (s/double-in :min 0.0 :max 5.0) 20.0)
val: 20.0 fails predicate: (<= % 5.0)
=> nil
(s/explain (decimal-in :min 0.0 :max 5.0) 20.0)
nil nil 0.0 5.0
val: 20.0 fails predicate: pred
#2016-07-0820:26Alex Miller (Clojure team)that ^^#2016-07-0820:26Alex Miller (Clojure team)by using let from test.check, youā€™ve also made test.check a production time dependency for you#2016-07-0820:27Alex Miller (Clojure team)(which is a bummer given how nice let is - I keep wanting it too)#2016-07-0820:29Alex Miller (Clojure team)but you can easily rewrite that with gen/fmap#2016-07-0820:29donaldballYeah, I plan to replace it with fmap ā€¦ jinx#2016-07-0820:30Alex Miller (Clojure team)double-in as a macro (and spec under it) allows the form of the predicate to be captured as a form as well as evaluated as code#2016-07-0820:31donaldballThanks, that makes good sense#2016-07-0820:31donaldballBtw any opinion on my optional keys generator observation?#2016-07-0820:34Alex Miller (Clojure team)did you maybe not generate enough to tell?#2016-07-0820:35Alex Miller (Clojure team)
(gen/sample (s/gen (s/keys :req [::a ::b] :opt [::c ::d])))
=>
(#:spec.examples.guide
 {:a 0, :b 0}
 #:spec.examples.guide
 {:a 0, :b 0}
 #:spec.examples.guide
 {:a 0, :b -2, :d -1, :c -1}
 #:spec.examples.guide
 {:a 0, :b -2, :c -2}
 #:spec.examples.guide
 {:a 3, :b -1}
 #:spec.examples.guide
 {:a 0, :b 3, :c 0}
 #:spec.examples.guide
 {:a 13, :b -29}
 #:spec.examples.guide
 {:a -17, :b -1}
 #:spec.examples.guide
 {:a -12, :b 4, :c 0}
 #:spec.examples.guide
 {:a 5, :b 5, :c -8, :d -9})
#2016-07-0820:35Alex Miller (Clojure team)I see optionals showing up pretty regularly there#2016-07-0820:37donaldballHmm, maybe Iā€™m doing something else wrong. Thanks.#2016-07-0820:40Alex Miller (Clojure team)Iā€™m not discounting the possibility of something wrong either :)#2016-07-0820:40Alex Miller (Clojure team)just that itā€™s not obvious to me#2016-07-0820:40donaldballHmm, looks like keys* and keys have very different behavior#2016-07-0820:41donaldball
(gen/sample (s/gen (s/keys* :opt-un [::foo])))
(() () (:_.c/*! [[{{} (Kk.B/?Q)}]]) (:AC!e._*0_/V ((({:*g #uuid "99a8b1d6-d3c3-40f7-a7c9-0fd7beaf14e8"})))) (:q?a.*kS.K*q?/?*k ()) (:x8x6?A.!/qU_m? {#uuid "5f81c97f-0e94-4a95-bdae-ff2adaba5378" 5} :T9y17p.!7p/!5a8 () :?x.d95?.*.OVnzN_/?3?J* [] :_-+U.z.F2JXV6.U070/+9a2? [()] :xQb ["<[ƊĀ©" .EM_N 1]) (:FtROj-1 [(19 :q0P:x5F:*T*41_G:H5_G:N-xN)] :?6IS.a+.AE4/vq2* [] :G.G+.d7Z!.w_+.a.C23/? () :?P*M/tj_Zg_ ()) (:XX [()] :F_Rv+.T+*n.Vp+2VR.w/+1tJQTg () :+R+9.r.per24*!9._NNt4*+.+.d5*T!.!9c/I {} :b.mw*bbbXT.*T+*-?g._h!!3_tg.Jvx.I.*++g14/+-S?Znvq () :K.H220.A9?g?h.s71-_+42.!9.-.z9iq4D*/jr83+S {} :x0w5+v.W!l6!s+/*5a? {0 -7}) (:Y!!D9w/J- [] :+!HS_l.XG+._XK*Bw_.UNW?.FPV20-3+_.*MYrk.Z1.pk-_!_/X*v {g._Y?!.QN+9g!/q.zC00Wj4 \space, Bj_t9- J?E**, true s!E-fSL4.R3Oj-A4._!vJ1!!.Vs5iz*n9a.-dM?5!9c.U.?7.*y/?-p+a!Lc, 1.0 :kP+b2Y:D*AA:74:U:-6yKx:phX!9??:I*Is, #uuid "f34904dd-a05a-4247-ad6d-1972baa0f58f" true, false -4.0, -5 false, \Ć¾ 5} :y*!G [:*_p?d?:-!:9f9:l0-3-Cw_5:2uT7f:46-_8P!h:C49*!*?gb:QJw:x8D \Åø 0.5 1/6] :mH.FH_PUY/Ik!3+z? ([{([]) [[*QN.]]}]) :ez0_.qeTiE_?.?-.d11.DrQDGZXu7.j9+7ou/N3M*T {} :!vdNK.-!!cb/Gv {ZS.K_d.fWR2/f+6.e?S. y4, v.w "|u\b"}) (:j65f.Z3r*.e8V7+6Y.L5_d.l+R8.dTp.L.l0n*9Z.AmW*_W8/c___ ({{} [()]})))
user> (gen/sample (s/gen (s/keys :opt-un [::foo])))
({} {} {} {:foo 1} {:foo 0.75} {:foo 0} {} {:foo -2} {:foo -8} {})
#2016-07-0820:42Alex Miller (Clojure team)oh, there was a bug in keys* gen that Rich fixed in master today#2016-07-0820:42Alex Miller (Clojure team)you said keys :)#2016-07-0820:43donaldballSorry, I just assumed keys* was a pure wrapper macro#2016-07-0820:43Alex Miller (Clojure team)https://github.com/clojure/clojure/commit/1e236448104fc8a0fc51e26eae7cdb7e650b7ae9#2016-07-0820:44Alex Miller (Clojure team)itā€™s a composition of &, a conformer, and keys#2016-07-0823:15ajssnot sure if others are interested, but I've figured out the nice way to do dependent types for spec/fdef arguments : use with-gen on the :args parameter
(defn sum [a b] (+ a b)) 
(spec/fdef sum {:args (spec/with-gen (spec/tuple int? int?) (constantly (gen/fmap (fn [x] [(* x 2) (* x 3)]) gen/int)))})
#2016-07-0901:23donaldballIā€™m kinda wishing s/def allowed a docstring. The generated docstring is okay, but I think Iā€™d like to be able to communicate the intent also.#2016-07-0902:01donaldballWeirdly, I had a good test running in an earlier repl with stest/check but now itā€™s raising#2016-07-0902:01donaldball
Caused by: java.lang.ClassCastException: clojure.lang.AFunction$1 cannot be cast to clojure.lang.MultiFn
#2016-07-0914:35robert-stuttafordso we have s/merge to make a single s/keys out of two other s/keys. i'm wondering, how do i spec a map that has some base set of req+opt keys, but also has an optional set of keys that must appear together as if speced with s/keys :req ... ?#2016-07-0919:38arohnerhas anyone seen java.lang.IllegalArgumentException: No implementation of method: :conform* of protocol: #'clojure.spec/Spec found for class: clojure.spec$spec_impl$reify__13361#2016-07-0919:38arohnerIā€™m doing something wrong, but I havenā€™t found it yet#2016-07-0919:40arohnerhrm, looks like I needed to clean after upgrading clojure#2016-07-0920:50kendall.buchanan@donaldball: Iā€™ve been experiencing the same with check-var#2016-07-1012:17jjcomer@donaldball: I get that error when I execute in a deftest.#2016-07-1015:12donaldballI get it both when executing in a deftest and in cider, though it works fine from a repl#2016-07-1015:16donaldballI was just experimenting with ignoring Richā€™s advice and expressing specs for strings as spec regexes. I find I actually quite like a couple of things that fall out: the form is easier to understand that the string regex, and you get a generator for free.#2016-07-1015:17donaldballhttps://gist.github.com/dball/e096fc7fb600fbc53fd94e4b367cf68f#2016-07-1015:19donaldballBut unless Iā€™m missing something, you pretty much have to write one spec for the seq of chars and another spec for the string that unwraps it as a seq#2016-07-1015:20donaldballSetting aside the merits of the idea, is there a way I could do with s/and and a conformed value?#2016-07-1018:25craigyhey all, I'm trying to figure out if I can use spec for parsing strings (sentences) into structured data. I'm struggling with whether or not I will need to split around spaces, and how to handle things like phrases (sequences of words) if I use s/cat. is this kind of use case intended at all, or am I better off with something else?#2016-07-1019:02mishanot-sure-fry#2016-07-1019:07xcthulhuHow do I write my own generator for cljs.spec?#2016-07-1019:07xcthulhuI have data structures like {0 0.0, 1 0.1, 2 0.2, 3 0.3, 4 0.4, 5 1.0}#2016-07-1019:07xcthulhu(maps of integers from 0 to 5, increasing in value as the index increases, where one of the values must be 1.0 and all of the values must be within [0,1])#2016-07-1019:08xcthulhuI can generate these things with this function:
(defn generate-random-map
  "Return a random structured map"
  []
  (let [values (sort (repeatedly 6 rand))
        pivot-value (last values)]
    (into {} (map-indexed vector (map #(/ % pivot-value) values)))))
#2016-07-1020:19gfrederickshuh.#2016-07-1020:23gfredericksxcthulhu: https://www.refheap.com/121363#2016-07-1020:27xcthulhuI'd rather not rope in test.check. Too bad that clojure.spec doesn't let you register generators separately from specs#2016-07-1020:28xcthulhuBecause I wouldn't mind associating a generator with my specs in my unit tests where I do use test.check#2016-07-1020:34xcthulhuI think you can do this with just just spec since it gives you 'cat' and 'fmap'#2016-07-1020:35gfredericksxcthulhu: I think clojure.spec is designed with the intention that you rope in test.check for non-trivial generator needs#2016-07-1020:35gfredericksis it just that you don't want test.check required in a non-test namespace?#2016-07-1020:37xcthulhuYup#2016-07-1020:38xcthulhuBut like I said, they do give you enough so you don't need test.check, it's just clumsy#2016-07-1020:38gfrederickswhere's fmap?#2016-07-1020:39gfredericksclojure.spec.gen/fmap?#2016-07-1020:39gfredericksyou could probably rewrite my code using clojure.spec.gen if that's what you really want#2016-07-1020:40gfrederickslooks like gen/shuffle isn't there#2016-07-1020:41henrytill@xcthulhu: arenā€™t you already pulling in test.check as a transitive dep if yr using something the gen namespace?#2016-07-1020:41gfredericksnot necessarily#2016-07-1020:42gfrederickssince it's lazy loaded you can get away with not having test.check available if you never run the generators/tests#2016-07-1020:42gfrederickswhich I think is the whole point of the lazy loading#2016-07-1020:42henrytillok, makes sense#2016-07-1020:43xcthulhuYeah#2016-07-1020:43xcthulhuOr cljs.spec.impl.gen for the brave#2016-07-1020:44gfredericksI wonder how they do lazy loading in cljs#2016-07-1020:46xcthulhuDynaload#2016-07-1020:46xcthulhuhttps://github.com/clojure/clojurescript/blob/master/src/main/cljs/cljs/spec/impl/gen.cljc#2016-07-1020:47gfredericksoh man#2016-07-1020:47gfredericksa custom var deftype#2016-07-1020:59xcthulhuYeah, it's pretty hacks. Anyway, if I read this right they lazily load vector from test.check. Sadly, clojureScript doesn't give you double*, only double#2016-07-1021:00xcthulhuAnyway, thank you I see how to do this crazy thing now#2016-07-1022:00xcthulhuHere's the generator I ended up going with BTW:
(gen/fmap
   (fn [values]
     (let [max-value (apply max values)
           min-value (apply min values)]
       (->> values
           sort
           (map #(-> % (- min-value) (/ (- max-value min-value))))
           (drop 1)
           (zipmap (range)))))
   (gen/vector
    (gen/such-that (complement #{infinity, (- infinity)}) (gen/double))
    7))
#2016-07-1022:01xcthulhuThis hacks around the fact that clojurescript doesn't have double*#2016-07-1022:02xcthulhuinfinity is defined to be Double/INFINITY and js/Number.INFINITY in the jvm and js respectively#2016-07-1022:02gfredericksthat should still give you nans though#2016-07-1022:02xcthulhureally?#2016-07-1022:02gfredericksgen/double generates nans#2016-07-1022:03xcthulhuNot with high probability, but I'll go add a restriction on to the such-that#2016-07-1022:03gfredericksshould be high enough to matter#2016-07-1022:03gfredericksunless I screwed something up#2016-07-1022:04gfredericksroughly 1% of the time#2016-07-1022:12xcthulhuHere we go:
(defn nan?
  "Tests if a value is a NaN or not"
  [x]
  #?(:clj (and (double? x) (.isNaN x))
     :cljs (js/isNaN x)))

;;;;;;;;;;;;;;;;

(gen/fmap
   (fn [values]
     (let [max-value (apply max values)
           min-value (apply min values)]
       (->> values
           sort
           (map #(-> % (- min-value) (/ (- max-value min-value))))
           (drop 1)
           (zipmap (range)))))
   (gen/vector
    (gen/such-that (complement (some-fn #{infinity, (- infinity)} nan?))
                   (gen/double))
    7))
#2016-07-1022:12xcthulhuHopefully this yak has been shaved by now#2016-07-1022:17gfredericks:)#2016-07-1102:22Alex Miller (Clojure team)There is s/double-in with infinity and nan support#2016-07-1106:32robert-stuttafordforgive the repost, folks, but i'm a little stumped šŸ™‚#2016-07-1106:34robert-stuttafordhow do i specify that an optional set of keys must either appear together or not at all?#2016-07-1108:34mandragoran@robert-stuttaford: you can write your own predicate (-> (s/keys :opt [...]) (s/and #(every (partial contains? %) [...]))#2016-07-1109:04robert-stuttafordtrue, thank you. i wonder if it's a use-case that has been considered by Rich et al?#2016-07-1112:51Alex Miller (Clojure team)Have you looked at the and/or support in s/keys :req?#2016-07-1112:59Alex Miller (Clojure team)was thinking something like (s/def ::combined (s/keys :req [(or (and ::a ::b) (and ::a ::b ::c ::d))]))#2016-07-1113:00Alex Miller (Clojure team)however b/c of open maps, I donā€™t think the latter two will fail as expected - for that you probably need and an additional pred#2016-07-1114:48robert-stuttafordthanks Alex#2016-07-1117:17semperosI wonder if thereā€™s a place for an s/xor within the clojure.spec space where one, and only one, of the provided specs may conform, or if this can be concisely expressed with a combination of the existing features#2016-07-1119:31semperosAlex or others, would appreciate your thoughts if/when you have time ^^#2016-07-1120:55arohnerHereā€™s a fun one. If I start up a repl and clojure.test/run-tests, everything is fine. If I lein test, I get java.lang.Exception: :clojure.spec/unknown is not a fn, expected predicate fn#2016-07-1120:56arohnerAFAIK, Iā€™m not doing anything weird. Everything is required in the proper order, not using load, etc#2016-07-1121:02arohnerAlso AFAIK, Iā€™m not referring to any spec that hasnā€™t been required#2016-07-1121:25arohnermore weirdness: I run my tests from a repl, everything is fine. I run c.spec.tests/instrument, run the same tests, and I get No implementation of method: :conform* of protocol: #'clojure.spec/Spec found for class: clojure.spec$spec_impl$reify__13372. Reload all my source namespaces, everything works again#2016-07-1122:44Alex Miller (Clojure team)I have seen some stuff like this with mixture of aot and non aot on classpath#2016-07-1215:04akielCan someone please comment on http://dev.clojure.org/jira/browse/CLJ-1966? I would like to know if I possibly understand something wrong.#2016-07-1216:37peejaI'm afraid I'm still failing to understand the difference between keys and keys*. I understand that only keys* works in a cat, but I don't understand why. Is this explained somewhere?#2016-07-1216:39Alex Miller (Clojure team)@akiel rich hasn't looked at it yet#2016-07-1216:41Alex Miller (Clojure team)@peeja keys is a spec for a map. keys* is a spec for a sequential list of alternating keys and vals#2016-07-1216:42peejaOhhhhh!#2016-07-1216:42peejaSo they both work in the same contexts, but match different things#2016-07-1216:43Alex Miller (Clojure team)I'd say they work in different contexts to match the same thing :)#2016-07-1216:43peejaBy "context" I mean where they go in a spec#2016-07-1216:44akiel@alexmiller itā€™s ok, as long as itā€™s on your list - thanks#2016-07-1216:44peejaYeah, I was misreading the guide. Thanks!#2016-07-1218:51donaldballRevisiting the ā€œspec strings as char seqsā€ idea from the weekend, I just threw this together:#2016-07-1218:51donaldballhttps://gist.github.com/dball/d9f9bb7b6ea796e0de2ae49f6b21066e#2016-07-1218:51donaldballBad idea or simply misguided? šŸ˜›#2016-07-1219:16Alex Miller (Clojure team)@donaldball: from a spec perspective I think it's much better to use an actual regex for matching#2016-07-1219:17donaldballThe things Iā€™m finding attractive about expressing specs on char seqs are that the spec regex forms are often more readable, but more to the point: you get working generators for free#2016-07-1219:18Alex Miller (Clojure team)There is a project for assembling readable regexes, can't remember the name#2016-07-1219:22arohnerthereā€™s https://github.com/cgrand/regex#2016-07-1219:23arohnerthat doesnā€™t use java.util.regex.Pattern though#2016-07-1220:24Alex Miller (Clojure team)yeah, that one#2016-07-1220:24Alex Miller (Clojure team)and also test.chuck has support for generating strings based on a regex#2016-07-1221:23sparkofreasonWhen writing regex specs, is there a way to get at the bindings "so far"? For instance, say I want to validate a datom [e a v], where a is keyword that represents a spec. I'd like to use that spec to validate v, and get "nice" output from explain that the failing spec is the one represented by a.#2016-07-1221:23danburtonoohh dependent specs, interesting#2016-07-1221:47Alex Miller (Clojure team)@dave.dixon custom predicate? (s/and <datom-spec> #(s/valid? (:a %) (:v %)))#2016-07-1221:47Alex Miller (Clojure team)doesnā€™t integrate into explain of course#2016-07-1221:50bhaumanMulti spec using second as the dispatch?#2016-07-1221:51bhaumanThen use a confirmed#2016-07-1221:51Alex Miller (Clojure team)nice#2016-07-1221:51bhaumanConformer#2016-07-1222:04danielcomptonhttps://clojurians.slack.com/archives/clojure/p1468341869003436#2016-07-1222:04danielcompton@alexmiller: what is the thinking behind it being under consideration? What are the downsides of allowing docstrings on specs?#2016-07-1222:13sparkofreason@alexmiller: would be nice if there were a way to pass explain results out of custom predicates. Though perhaps not required in most cases. #2016-07-1222:24Alex Miller (Clojure team)@danielcompton asking the wrong person :)#2016-07-1222:25Alex Miller (Clojure team)@dave.dixon: while it's tempting, I don't think Rich is inclined to support that for uniformity.#2016-07-1222:28danielcomptonhttp://dev.clojure.org/jira/browse/CLJ-1965#2016-07-1300:00sparkofreason@alexmiller: Anyway, I'd rather see support for this specific case, which seems common in Clojure, where we have a sequence of keywords followed by associated values. Just supporting the case of having the value validated against the spec named by the keyword would cover a lot of ground, as opposed to having a general dependent spec scheme. There's a nice symmetry with the map behavior then.#2016-07-1300:02Alex Miller (Clojure team)Actually you could do this with keys* couldn't you?#2016-07-1300:07Alex Miller (Clojure team)(s/cat :e any? :av (s/keys* :opt-un [::a]))#2016-07-1300:11seancorfieldOne thing Iā€™m finding with clojure.spec as applied to user input is that I often seem to need a two-step process: the first is a very basic predicate and a conformer, the second is the real "spec" (and perhaps and conformer).#2016-07-1300:11seancorfieldExample, user inputs a country code and Iā€™d like it to be case-insensitive.#2016-07-1300:12seancorfieldSo the first spec is that itā€™s a two-character string and the conformer is clojure.string/upper-case; then the second spec actually looks up the (now upper-case) two-character string and verifies itā€™s a valid ISO country code (and could conform it to country information).#2016-07-1300:13seancorfieldIs that somehow a poor use of clojure.spec? Or is there perhaps a better way to deal with input that needs a little "cleanup" before being validated/conformed?#2016-07-1300:17luke@seancorfield hm. If a non-uppercase string is to be considered ā€œvalidā€ input, then Iā€™d say itā€™s on your country-code lookup function to handle case insensitive input. That seems like it would be easier than having two specs.#2016-07-1300:18lukeUnless you want to have a spec for your intermediate data format for some reason.#2016-07-1300:19seancorfieldWell, several of the inputs ā€” and country is actually possible here ā€” are sets of strings. For example, ::gender is #{"male" "female"} so it validates and generates nicely, but if you want it to be case-insensitive then you need to do "something" that isnā€™t necessarily easy with a single spec.#2016-07-1300:20seancorfield(`::country` could be (set (Locale/getISOCountries)) for example)#2016-07-1300:23seancorfieldI started leaning toward two specs because internally I wanted ::gender #{:male :female} for the domain model and "appropriate strings" for the input and the conformer there would be (comp keyword str/lower-case) ā€¦ assuming you can validate the input strings with a different spec.#2016-07-1300:24seancorfieldPerhaps Iā€™m making life more complicated for myself by trying to avoid with-gen here...#2016-07-1300:24lukeYeahā€¦ two specs isnā€™t bad if you do want separate models (one for the domain, one for the input) and are ok specā€™ing them separately.#2016-07-1300:25lukeIt is also possible in theory to write a case-insensitive helper, which given a set of strings returns a (generating) spec that accepts any capitalization.#2016-07-1300:30seancorfieldHmm, Iā€™ll have to give it more thought. So far have specs for input fields that conform to the domain model values, and specs for the domain model itself is feeling like the right way to go, as it allows me to generate data for both layers: API (input) and "system" (domain model).#2016-07-1300:32seancorfield(weā€™re pushing very hard to tease apart what is some rather complected logic in our current codebase ā€” we currently do a sort of transform-validate-update operation on each field and weā€™ve figured out a nice clean way to separate validate from update ā€” with a view to pushing the updates out to a queue and applying them asynchronously ā€” but that pre-validation transform on a few fields is proving problematic šŸ˜ø )#2016-07-1300:33seancorfieldThanks for the input so far @luke#2016-07-1300:34lukeyep if you want separate layers, and youā€™ve fully reified both layers, then specs for each layer seems like the way to go!#2016-07-1300:34lukeno hard answers, just what comes off the top of my head when I hear the question šŸ™‚#2016-07-1301:05sparkofreason@alexmiller: Thanks, that works great. Knew there had to be a way to do that, but had my head stuck in regex-land...#2016-07-1301:15sparkofreasonSlightly shorter version: (s/cat :e any? :av (s/keys*))#2016-07-1302:22Alex Miller (Clojure team)Yup#2016-07-1303:17lvhIs there a way to specify ā€œstrings that match a particular regexā€ in a way that produces the correct generator?#2016-07-1303:18lvh(test.chuck I think has that)#2016-07-1303:20lvhit seems like test.check would have a real bad time generating strings until they accidentally match that regex#2016-07-1303:20lvh(that also seems like a general problem, which is why Iā€™m assuming thereā€™s a general soltuion)#2016-07-1303:23gfredericksthe test.chuck generator could be moved to test.check proper if somebody wants to rewrite the instaparse regex parser in pure clojure#2016-07-1303:24lvhWhat is it now? Bunch of Java?#2016-07-1303:24lvhAlso hi gfredericks burning the midnight Chicago oil too I see šŸ™‚#2016-07-1303:24lvhI havenā€™t really read the spec source code any maybe I should just go do that#2016-07-1303:25lvhIā€™m presuming thereā€™s a way to map predicates to generators#2016-07-1303:25gfrederickslvh: I'm saying the test.chuck code uses instaparse#2016-07-1303:25lvhand Iā€™m guessing thatā€™s why spec/and is not just every-pred#2016-07-1303:25gfrederickslvh: hi#2016-07-1303:25lvhoh, right, and test.check doesnā€™t get to depend on instaparse#2016-07-1303:25lvhgot it#2016-07-1303:26gfredericksexactly#2016-07-1303:26gfredericksso you'd have to rewrite this in clojure somehow: https://github.com/gfredericks/test.chuck/blob/master/resources/com/gfredericks/test/chuck/regex.bnf#2016-07-1303:26lvhthat doesnā€™t sound very fun#2016-07-1303:27lvhhow would you feel about clojure.spec-specific things being added to test.chuck?#2016-07-1303:27lvhsay, a spec that understands how to map to generators in test.chuck, like, say, hypothetically, if someone wanted to match a regex šŸ˜‰#2016-07-1303:28gfredericksI keep thinking I'm going to want a similar clojure.spec utility library#2016-07-1303:29gfredericksclojure.schpec#2016-07-1303:29lvhšŸ’Æ#2016-07-1303:30lvhwould depend upon.#2016-07-1303:31lvhI guess Iā€™m screwed anyway because this is a keyword, not a string, so unless thereā€™s something that understands keywords, Iā€™m going to manually assign a generator#2016-07-1303:32gfredericksI get the impression that manually assigning generators is not meant to be too rare#2016-07-1303:32lvhsame here#2016-07-1303:33lvhIā€™ll defer to you since you obviously have a lot more experience here, but just generating samples based on only predicates is going to break really badly really quickly#2016-07-1303:33lvh(basically as soon as the probability of the predicate passing is not very big)
#2016-07-1303:34gfredericksyeah I don't think filtering on predicates is expected to be a very useful approach#2016-07-1303:34gfredericksclojure.spec has that behavior in some places because there's not any other reasonable default, except maybe just refusing to try#2016-07-1303:35lvhyeah, fortunatelyi tā€™s mostly just for dev and test that you do that#2016-07-1303:36gfredericksmostly?#2016-07-1303:36lvhso itā€™s fine if the generator just shrugs and gives up after n consecutive failures#2016-07-1303:36lvhI have used test.check in other directions šŸ™‚#2016-07-1303:36lvhtechnically still a test I guesS?#2016-07-1303:36lvhspecifically, I generated sequences of events/actions on openstack (rackspace public cloud)#2016-07-1303:37lvhto then assert that autoscale (https://github.com/rackerlabs/otter) would still figure a way out#2016-07-1303:37lvhso it found things like ā€œit gets unhappy if there are two expected networks these servers should be attached to and one of them disappears"#2016-07-1303:38gfredericksfun#2016-07-1314:08Alex Miller (Clojure team)http://blog.cognitect.com/blog/2016/7/13/screencast-spec-leverage#2016-07-1314:08Alex Miller (Clojure team)First in a series of spec screencasts ^^#2016-07-1315:14peejaIs it possible to match data against a spec to find the paths at which a particular spec is used?#2016-07-1315:15peejaThat is, I want something like conform, but rather than look things up by tag, I want to look things up by spec keyword#2016-07-1315:15peeja(and find the tag-based paths to those parts)#2016-07-1315:16peeja"Find me every Foo in this data structure"#2016-07-1315:38Alex Miller (Clojure team)Iā€™d say no, nothing like that. might help back up to the problem youā€™re trying to solve though#2016-07-1315:49peejaYeah, I think I'm actually down the wrong path here. (But it's hard to tell, since I haven't entirely decided on what I'm trying to do.) šŸ™‚#2016-07-1315:57Alex Miller (Clojure team)thatā€™s usually a good first step :)#2016-07-1317:51joshmillerA question I posed originally in #C03S1KBA2: Iā€™m having some trouble understanding how keywords work with respect to clojure.specā€¦ Why is it that (s/def :spec-test-ns/name string?) works, but (s/def (keyword ā€œspec-test-nsā€ ā€œnameā€) string?) fails with CompilerException java.lang.AssertionError: Assert failed: k must be namespaced keyword or resolvable symbol? The type of both is clojure.lang.Keyword.#2016-07-1318:06semperosthis is the assert that is being tripped @joshmiller : (c/assert (c/and (named? k) (namespace k)) "k must be namespaced keyword or resolvable symbolā€)#2016-07-1318:10semperos(I know not fully helpful, sorry)#2016-07-1318:10joshmiller@semperos: Yeah ā€” but (instance? clojure.lang.Named (keyword "spec-test-ns" "idā€)) is true and (namespace (keyword "spec-test-ns" "idā€)) is "spec-test-nsā€, so I donā€™t understand how that assert is failing.#2016-07-1318:12joshmillerIā€™m digging into what happens with macroexpand on s/def to see if maybe thatā€™s the issue.#2016-07-1318:13semperosyeah, itā€™s essentially itā€™s trying (#'s/named? '(keyword "spec-test-ns" "nameā€))#2016-07-1318:14semperosand a List is not a named thing, note the ā€™ in the macro definition of s/def#2016-07-1318:15joshmillerYeah, I see (clojure.spec/def-impl (quote (keyword "spec-test-ns" "name")) (quote clojure.core/string?) string?) vs (clojure.spec/def-impl (quote :spec-test-ns/name) (quote clojure.core/string?) string?) when doing it the ā€œrightā€ way#2016-07-1318:43Alex Miller (Clojure team)I wanted to let everyone know that I have been continuing to update and extend the spec guide (at http://clojure.org/guides/spec) since the early days and it might have some things you havenā€™t seen yet in it. I made a list of things that have been added here: https://groups.google.com/d/msg/clojure/fYwDe3ygcm8/JZmmdjsVCQAJ#2016-07-1318:46wildermuthnThanks!#2016-07-1318:48Alex Miller (Clojure team)I donā€™t know that hardly anyone has noticed or used the instrument :stub/:replace functionality, which lets you do some pretty interesting things both for example tests or in tandem with check.#2016-07-1318:58joshmillerI ended up being able to dynamically generate the keywords for specs in other namespaces by using a macro where I could unquote the keyword generation, but if anyone knows of another way to go about it, Iā€™d love to hear it.#2016-07-1320:04Oliver GeorgeIs there a standard way to add a generator to complement an instance? spec based on a constructor fn. I imagine you would want a spec/gen for each constructor arg. #2016-07-1320:06Oliver Georges/instance google.map.LatLng ::lat ::long #2016-07-1320:17Oliver GeorgeI think the answer is a combination of s/with-gen, s/gen and gen/fmap #2016-07-1321:08puzzlerWatching the screencast about how spec accomplishes multiple things at once got me thinking about how it is difficult to get all the listed benefits simultaneously. For example, if you put your specs in the same namespace as the code, you can't easily make custom generators because it depends on test.check which is generally only a dev dependency. If you put your specs in a separate namespace, you can't rely on predicates defined in your main namespace. I have not yet found a practical way to divide up code involving specs without giving something up. A lot of that has to do with Clojure's limitation/restriction regarding circular dependencies. I hope more info about how to use spec in the context of a real-world project will eventually be provided.#2016-07-1321:13donaldballYou can still make custom generators if you restrict yourself to the vars in clojure.spec.gen#2016-07-1321:21mike_ananevIs there any recommendation to put spec and fn's in one file or separate them?#2016-07-1321:23puzzler@donaldball: the guide says the vars in clojure.spec.gen just dynamically link to clojure.test-check, so those vars won't do anything unless you have clojure.test-check listed in your (dev) dependencies. Assuming I understand the guide correctly...#2016-07-1321:24puzzler@donaldball: Or is your point that the code will at least pass compilation, so it doesn't matter if those vars don't actually do anything? (I haven't tried that myself, so I don't know what happens if you don't have test.check active).#2016-07-1321:25donaldballThe latter#2016-07-1321:25puzzler@donaldball: Ah, that's helpful to know.#2016-07-1321:27puzzler@mike1452: Generally, people have been reporting that they have been separating the specs from the implementation, but there are limitations that come with that approach (as far as I can tell).#2016-07-1321:33mike_ananev@puzzler thanx! may be new screencasts from Cognitect will show us a code arrangement with spec.#2016-07-1321:38seancorfield@puzzler: Weā€™re mostly keeping specs separate from code. So our spec ns requires our code ns, and any code that wants the specs applied requires the spec ns (and the code ns if needed). Thatā€™s how weā€™ve solved the circular reference issue.#2016-07-1322:42donaldballIs that mostly a concession to the fact that youā€™re working in a codebase that needs to support older versions of clojure or an explicit design choice?#2016-07-1322:43donaldballWhere Iā€™m playing with specs, and I spec my fns, I tend to like having them adjacent to the fn definitions#2016-07-1323:03seancorfield@donaldball: The bulk of the specs are for layers of our domain model ā€” data ā€” so thatā€™s independent of functions, so it made sense to have a separate namespace for them. And then to put the handful of function specs in there too.#2016-07-1323:03seancorfieldAnd as long as you load the spec namespace into the system before you run the functions that attempt to conform data to those specs, youā€™re good.#2016-07-1323:26wilkerlucioI'm choosing now to use they adjacent of the code, like donald is doing, I'm not sure if it's the best way but I'm enjoying it so far, will be nice to see the benefits/problems on the long run as people mature specs on their codebases#2016-07-1323:28wilkerluciofor those using on a separeted namespace, what name conventions are you using for the specs namespaces? @seancorfield @puzzler#2016-07-1323:31seancorfield@wilkerlucio: It depends on what weā€™re specā€™ing. We have a set of specs for domain model data (in ns like ws.domain.something where something is the particular domain model "object" being described) and then we have a set of specs for input fields for APIs (in ns like ws.something.field).#2016-07-1323:31seancorfieldBut itā€™s early days and weā€™re moving code around. A lot.#2016-07-1323:32seancorfieldWe also have structures that tie things together (so we have hash maps that describe the relationship between domain model things, database tables and columns, validation and update routines ā€” using qualified keywords that are resolved to symbols later).#2016-07-1323:33wilkerlucionice, sounds cool šŸ™‚#2016-07-1323:36wilkerlucioand for specs about functions, are you using something like: for namespace x.y.z have a x.y.z-specs, something like that?#2016-07-1323:54seancorfieldNo, those specs just go in the namespace for the "something" that those functions would (primarily) operate on. Theyā€™re fairly coarse-grained domain concepts (for our systems ā€” "member" covers a lot of ground).#2016-07-1404:40mjmeintjesHi. Is there a way to use spec to generate maps with "dependent" properties. For example, suppose my map spec requires than both :a and šŸ˜› contain strings that have to be the same length?#2016-07-1404:42mjmeintjesThat should be - Is there a way to use spec to generate maps with "dependent" properties. For example, suppose my map spec requires than both :a and :b contain strings that have to be the same length?#2016-07-1404:56seancorfield@mjmeintjes: Well, you can use s/and to restrict what a spec generates but that might not be the easiest way to make the generator work...#2016-07-1405:05seancorfieldHmm, perhaps it's OK after all:
boot.user> (require '[clojure.spec :as s])
nil
boot.user> (s/def ::a string?)
:boot.user/a
boot.user> (s/def ::b string?)
:boot.user/b
boot.user> (s/def ::foo (s/and (s/keys :req [::a ::b]) #(= (count (::a %)) (count (::b %)))))
:boot.user/foo
boot.user> (s/exercise ::foo)
([#:boot.user{:a "", :b ""} #:boot.user{:a "", :b ""}] [#:boot.user{:a "z", :b "S"} #:boot.user{:a "z", :b "S"}] [#:boot.user{:a "GF", :b "y8"} #:boot.user{:a "GF", :b "y8"}] [#:boot.user{:a "c5kf9D7nt88PhP2zRiIj", :b "Ax9Ua1vd561LvuWZ0o5r"} #:boot.user{:a "c5kf9D7nt88PhP2zRiIj", :b "Ax9Ua1vd561LvuWZ0o5r"}] [#:boot.user{:a "", :b ""} #:boot.user{:a "", :b ""}] [#:boot.user{:a "yaLME3MI4uSh0", :b "FIxs99gp43jSe"} #:boot.user{:a "yaLME3MI4uSh0", :b "FIxs99gp43jSe"}] [#:boot.user{:a "9", :b "3"} #:boot.user{:a "9", :b "3"}] [#:boot.user{:a "f7nOvuwZ30U000f", :b "3enZpUsZXP9ZVX1"} #:boot.user{:a "f7nOvuwZ30U000f", :b "3enZpUsZXP9ZVX1"}] [#:boot.user{:a "8aDmY08S8Q", :b "rHFcc5rAN9"} #:boot.user{:a "8aDmY08S8Q", :b "rHFcc5rAN9"}] [#:boot.user{:a "Ds8zm6", :b "yM0N6p"} #:boot.user{:a "Ds8zm6", :b "yM0N6p"}])
#2016-07-1406:21mjmeintjes@seancorfield: Great, thanks! Looks promising, I'll investigate further.#2016-07-1414:43glvAfter converting some property-based tests to use specā€™s generators, I noticed that some of the tests became much slower. Investigation reveals that the growth characteristics of specā€™s generators can be quite different from those of test.check. Hereā€™s an example:#2016-07-1414:46glvIn most cases I doubt the difference would be very noticeable, but in my case two such generated integer values were being used as the dimensions of a two-dimensional matrix structure, so the size really blew up quickly.#2016-07-1414:49glvI can see a rationale for making the growth a little steeper than the test.check version, but this seems a bit severe. šŸ™‚#2016-07-1417:04glvEspecially since the default for :num-tests is 1000. šŸ˜®#2016-07-1417:24seancorfieldI would say, in this case where the amount of data you test is N-squared based on random generated values, you probably want a custom generator that heavily restricts those dimension values?#2016-07-1417:24seancorfieldHow big, realistically, is a matrix you would be handling?#2016-07-1417:26glvOh, sure ā€¦ Iā€™ve done exactly that, now. Just wondering whether this is an appropriate default.#2016-07-1417:27glvGiven the actual behavior over 1000 iterations, youā€™d run into problems even if the amount of data was linear wrt the integer value.#2016-07-1417:29glvAnd given that the visible symptom is just ā€œmy test run seems to just stallā€ itā€™s a bit confusing.#2016-07-1417:55glv(Oh, and ā€¦ my first attempt at constraining the generator was just to slap a #(< % 25) predicate on there, and then I got sporadic "Couldn't satisfy such-that predicate after 100 triesā€ failures. Which was easy enough to fix, but still leads me to think that the generator growth is just too fast.)#2016-07-1420:19Timwhat is the justification of having spec in separate namespaces than the actual functions?#2016-07-1420:44seancorfield@tmtwd: per my earlier comment https://clojurians.slack.com/archives/clojure-spec/p1468450981002667#2016-07-1421:52kendall.buchananWhat happened to instrument-ns in alpha10? Not possible anymore?#2016-07-1421:56glvfolded into instrument. (instrument ā€˜my.namespace)#2016-07-1421:57kendall.buchananAh, awesome.#2016-07-1421:58glvAh, Iā€™ve figured out whatā€™s going on with the growth rate of (s/gen pos-int?) mentioned above: some of the generators for built-in predicates use test.check functions that ignore the size guidance. This means (I think) that they wonā€™t shrink properly when an error is found: https://github.com/clojure/clojure/blob/d920ada9fab7e9b8342d28d8295a600a814c1d8a/src/clj/clojure/spec/gen.clj#L128-L185#2016-07-1422:03glv@alexmiller: Iā€™d love to hear your thinking on this when you get a chance. Discussion starts at https://clojurians.slack.com/archives/clojure-spec/p1468507432002694#2016-07-1422:04ghadiI think he'll be back tomorrow Glenn#2016-07-1422:04ghadiout#2016-07-1422:04glvOK, thanks!#2016-07-1512:13vikeriCan I spec an infinite sequence?#2016-07-1512:14gfredericksLike (s/* ___)?#2016-07-1512:14gfredericksI suppose the validation is the sticky part#2016-07-1512:14vikeriExactly#2016-07-1512:14gfredericksI think that's what s/every is for, if you need incomplete validation#2016-07-1512:15gfredericksI guess technically s/* isn't the same thing as infinite#2016-07-1512:15gfrederickss/āˆž#2016-07-1512:16vikeriWith s/+ I just got an infinite loop. (It tries to check every element in the infinite sequence I guess) s/every looks promising.#2016-07-1512:25vikeris/every did the trick! Thanks!#2016-07-1512:39vikeriOr, it works to validate it like this (s/valid? (s/every :my/pred) (repeat my-item)) but not inside :args in s/fdef. Any ideas?#2016-07-1512:55gfredericksdo you just mean that it's not validating at all because you haven't instrumented the function? or is it breaking in some way?#2016-07-1513:08vikeri@gfredericks It goes into an infinite loop when I put it in my fdef and then execute the function.'#2016-07-1513:09vikeriIf I test it in the repl with s/valid? it works fine#2016-07-1514:33rickmoynihanjust starting to play with spec... How would you say "A map must have either the keys :foo or :bar or both (but not neither)"#2016-07-1514:35rickmoynihanguessing I just need to supply all three combos#2016-07-1514:37minimalYeah use s/or the guide has an example http://clojure.org/guides/spec#_instrumentation#2016-07-1514:38minimal
(s/or :ok (s/keys :req [::result])
          :err (s/keys :req [::error])
#2016-07-1514:41rickmoynihanmakes perfect sense - thanks#2016-07-1515:38richhickey@rickmoynihan: there is direct support for or in keys:#2016-07-1515:39richhickeythe use of s/or in the example from the guide (mentioned above) is to distinguish 2 different kinds of maps#2016-07-1515:40rickmoynihanthanks richhickey šŸ™‚#2016-07-1515:42minimalooh nice#2016-07-1515:42rickmoynihanAm I right in thinking that the 'or symbol there is actually clojure.core/or - and spec interprets the symbol itself as meaning or?#2016-07-1515:43rickmoynihanahh yes keys is a macro#2016-07-1515:43richhickeyā€˜orā€™ and ā€˜andā€™ are syntax inside keys#2016-07-1515:43rickmoynihanwhen I first saw spec, I wondered why you hadn't done that! Super nice!#2016-07-1515:45rickmoynihansymbols are data too#2016-07-1515:53stuarthalloway@glv have any more info on test.check fns ā€œignoring size guidanceā€? Starting to read the source now, and it looks like we might need to explicitly opt in to sized in some places#2016-07-1515:55glvA lot of the generators for built-in predicates use large-integer* from test.check, which doesnā€™t use sized. Perhaps instead use the more specialized things like pos-int, s-pos-int, etc.?#2016-07-1515:58stuarthalloway@glv I am confused because large-integer* calls large-integer** which does use sized#2016-07-1515:58glvHmmm ā€¦ I thought I traced through all that yesterday.#2016-07-1516:00stuarthalloway@glv your proposal still may be the right thing, I just want to understand why we are getting the current behavior#2016-07-1516:00glvRight, exactly. Investigating.#2016-07-1516:01stuarthalloway@glv it looks to me like s-pos-int use size on integers, and large-integer** uses size on bit count#2016-07-1516:01glvAh, right.#2016-07-1516:02stuarthallowayand if we switch to s-pos-int we would never get larger values#2016-07-1516:02glvThat might explain the weird, long-cycle behavior of (s/gen int?) in that example I pasted yesterday.#2016-07-1516:02stuarthalloway
(apply max (gen/sample gen/int 1000000))
=> 99
#2016-07-1516:03stuarthallowaygrr, maybe neither growth pattern is ideal#2016-07-1516:04glvRight. I think (Iā€™m no expert) thatā€™s why test.check has both. pos-int (and family) for integers that are used as sizes for things, and large-integer for integers used in mathematical functions.#2016-07-1516:04stuarthallowaywhereas spec has specific (int-in et al) and non-specific (pos-int et al)#2016-07-1516:05glvYes.#2016-07-1516:05glvWhich is what I switched to.#2016-07-1516:07glvAnd the only reason that kind of annoys me is that those are generators only, not specs. It doesnā€™t seem good to have to specify a custom generator for such a common case. (My hunch, which may be wrong, is that in most systems, functions that build and manipulate data structures are more common than functions that do complex math on integers.)#2016-07-1516:09stuarthalloway@glv in cases where you donā€™t specify, I would like to see it grow more slowly, but still grow#2016-07-1516:09glvAgreed.#2016-07-1516:09stuarthallowayhitting the hammock on this one#2016-07-1516:09stuarthallowayto be more honest, grabbing some lunch šŸ™‚#2016-07-1516:10stuarthallowaybut we do have a hammock in the office#2016-07-1516:10glvOF COURSE YOU DO#2016-07-1516:19glv@stuarthalloway: Oh wait ā€¦ of course int-in is a spec, not just a generator. The real issue is that I want a large upper-bound in production, but smaller in tests (because a 100x100 matrix will be much slower with a negligible chance of exposing bugs that wouldnā€™t be seen in a 20x20 matrix over a bunch of tests. But thereā€™s no reason to restrict it so tightly outside of the tests. (Maybe my case is just weird and special. But I do think the existing growth pattern is surprising, and I strongly suspect that it negatively affects shrinking.)#2016-07-1517:22hlshipSo Iā€™m seeing something odd in the interaction of s/every-kv and s/conform.
(s/def ::path-spec (s/cat :method #{:get :post} :path string?))
=> :io.aviso.config.spec/path-spec
(s/def ::paths (s/every-kv ::unqualified-keyword ::path-spec))
=> :io.aviso.config.spec/paths
(s/conform ::paths {:one [:get "foo"]})
=> {:one [:get "foo"]}
Iā€™d expect that to be {:one {:method :get :path ā€œfooā€}}.
#2016-07-1517:22hlshipBTW:
(s/def ::unqualified-keyword
  (s/with-gen
    (s/and keyword?
           #(-> % namespace nil?))
    gen/keyword))
#2016-07-1517:28seancorfield@hlship: Why not use simple-keyword?#2016-07-1517:29hlshipI canā€™t find that ā€¦ do you have a link to docs?#2016-07-1517:32hlshipAnyway, any insight into the s/conform part?#2016-07-1517:32seancorfieldUse s/map-of instead.#2016-07-1517:32seancorfieldI donā€™t think every-kv conforms the values?#2016-07-1517:33hlshipThanks for the tip ā€¦ must have missed that.#2016-07-1517:35hlshipThanks, just what I needed.#2016-07-1517:37seancorfieldYeah, just confirmed via the docstrings ā€” map-of "Unlike every-kv, map-of will exhaustively conform every value." ā€” every-kv (based on every) "Note that every does not do exhaustive checking, rather it samples *coll-check-limit* elements. Nor (as a result) does it do any conforming of elements."#2016-07-1517:38seancorfield(I did not know that before you asked so, thank you, "learn something new every day" šŸ™‚ )#2016-07-1517:46glv@hlship: Docs for simple-keyword? are here. It is new in 1.9. https://clojure.github.io/clojure/branch-master/clojure.core-api.html#clojure.core/simple-keyword?#2016-07-1517:47glvSo you can do without ::unqualified-keyword and just use simple-keyword? directly in map-of.#2016-07-1518:14stuarthalloway@glv so test.check does two flavors of things:#2016-07-1518:14stuarthalloway1. built-ins that bottom at choose use the size argument twice, to control the rate of growth and to limit the values reached#2016-07-1518:15stuarthalloway
(->> (gen/sample-seq gen/s-pos-int 1000)
        (take 200))
#2016-07-1518:16stuarthalloway2. built-ins that bottom at large-integer use the size argument only once, to control (approximately) the number of bits allowed#2016-07-1518:16stuarthalloway
(->> (gen/sample-seq (gen/large-integer* {:min 1 :max 1000}) 13)
        (take 200))
#2016-07-1518:17stuarthallowaynote the 13 (bits) vs 1000 (value) passed as size to gen/sample-seq#2016-07-1518:18stuarthallowayspec does not (yet) even provide a path to setting the size argument#2016-07-1519:55stuarthalloway@glv ignore that crap explanation ^^. Bottom line is we want a generator that will grow nicely with a default max-size, working on it.#2016-07-1520:06glv@stuarthalloway: Right, sounds great. Thanks!#2016-07-1520:29arohnerWith multispec, is there a way to refer to a specific instance by name?#2016-07-1520:29arohnerIā€™d like to set up my multi-spec hierachy, but then in some cases say ā€˜this fn takes this specific type from the hierarchy'#2016-07-1520:30arohnerI know I can (s/and ::foo #(= :type (:foo %)), but itā€™d be nice to have some sugar on that#2016-07-1600:01gfredericksstuarthalloway: resizing with some multiple of square root would give another size growth pattern#2016-07-1601:14Alex Miller (Clojure team)I think Stu left this on my plate so I will take a look next week, thx#2016-07-1604:13sparkofreasonIf we conform a map to a spec using the unnamespaced keys, is there any way to get the map returned using the fully-qualified specs keywords as keys? I threw together a hacky and incomplete version of this, hoping there's an easier way.#2016-07-1612:37Alex Miller (Clojure team)@dave.dixon spec doesn't provide an explicit tool for that#2016-07-1612:38Alex Miller (Clojure team)It is possible if you did an s/merge of s/keys and s/map-of with :conform-keys true and a conformer key pred#2016-07-1614:57lvhHey; is there any particular library for common specs like URI/URLs, dates, &c thatā€™s forming/informally?#2016-07-1614:58lvhIt might be useful to do so generally because a lot of this stuff appears to be string matching a regex#2016-07-1614:58lvhmeaning that if itā€™s third-party, test.chuck can automatically give you generators#2016-07-1615:06gfredericksso you're saying it's time for me to create clojure.schpec#2016-07-1615:07lvhIā€™m writing a library that I think I will call ehm, specimen or something#2016-07-1615:07lvhitā€™s json-schema => spec#2016-07-1615:08lvhbecause I was translating swagger => spec and that was really boring repetitive work#2016-07-1615:09lvhbut I figured I could get a computer to do it for me#2016-07-1615:09lvhthe json-schema spec is also itself available as a json-schema#2016-07-1615:09lvhIā€™m not sure what that means yet#2016-07-1615:18donaldballIā€™ve done some initial work towards such a lib, but itā€™s more aspirational and experimental than not atm: https://github.com/SparkFund/useful-specs/tree/master/src/specs#2016-07-1615:25lvhawesome#2016-07-1615:25lvhone of the problems I saw with Schema was that there were a bunch of subtly overlapping ones#2016-07-1615:25lvhnot necessarily well-maintained#2016-07-1615:29lvhgfredericks: Is json-schema => spec something youā€™d consider for clojure.schpec?#2016-07-1615:29lvhAlso, if itā€™s clojure.schpec thatā€™d be like chuck, outside the stdlib, so nobody has to rewrite instaparse?#2016-07-1615:30donaldballYeah, I hope that between clojure core and 1 or 2 community libs, weā€™ll have a solid shared design vocabulary#2016-07-1615:30lvh(heck maybe instaparse should just go into clojure, maybe not though)#2016-07-1615:30lvhdonaldball: side-effecty namespace loading womp womp šŸ˜ž#2016-07-1615:30donaldballLoading the tld resource?#2016-07-1615:31lvhno, no#2016-07-1615:31lvhdonaldball: Not specific to your thing#2016-07-1615:31lvhJust that I have to require the ns somewhere before I can go use :whatever/whatever, because of the side-effecty def, right?#2016-07-1615:31lvhkinda the same problem as having multimethod impls#2016-07-1615:32lvhdonaldball: Itā€™s not a criticism of your project; it is a generic problem#2016-07-1615:32donaldballOh, yeah, the keyword focus has some interesting consequences#2016-07-1615:32gfrederickslvh: yes to both of those#2016-07-1615:32lvhcool#2016-07-1615:32gfredericksthe idea would be "a big pile of half-baked utilities related to clojure.spec"#2016-07-1615:33lvhI want to pronounce schpec as speck, so now I am hungry#2016-07-1615:36gfrederickscould also name it spock I suppose :/#2016-07-1615:36gfredericksI'm not big on star trek themes in general but I don't know what sort of mascot is suggested by the name "schpec"#2016-07-1615:38gfredericksclojure.spook#2016-07-1615:44donaldballspectacles#2016-07-1615:44donaldballThey help you see things more clearly#2016-07-1615:45gfredericksthat's way too positive#2016-07-1615:45gfredericksI think ghost-themed is promising#2016-07-1615:46gfredericksmaybe spelled spooc#2016-07-1615:46lvhspook has unfortunate connotations though#2016-07-1615:46gfredericksI did like "spectacles" when I was accidentally reading it with emphasis on the middle syllable though#2016-07-1615:47lvhyou could totally pull off Speck for shpec#2016-07-1615:47lvhhttps://images.duckduckgo.com/iu/?u=https%3A%2F%2Ftse3.mm.bing.net%2Fth%3Fid%3DOIP.Me86b5b0ff0b002aee957411c8d9fb0eaH0%26pid%3D15.1&amp;f=1#2016-07-1615:47gfredericksdammit that happened with spuck too#2016-07-1615:47lvhunfortunate connotations?#2016-07-1615:47gfredericksyeah#2016-07-1615:48lvhI donā€™t recongize that one, but yeah that sucks#2016-07-1615:48lvhitā€™s weird because I work with a ton of former intelligence people#2016-07-1615:49lvhso spook is a generally accepted word for them internally#2016-07-1615:49gfredericksaccording to alexmiller at least; I'm not sure if he was thinking of the urban dictionary entry, which has the usual difficult-to-assess relevance level#2016-07-1615:51lvhIā€™ve heard first, mostly outside of the US (Jewish parts of Antwerp, so may be reasonably common Yiddish though), the rest look made up,#2016-07-1615:52lvhspook is very definitely relevant according to some of my non-former-intelligence coworkers from the deep south#2016-07-1615:55donaldballspooky or spookily donā€™t carry the spy connotation#2016-07-1615:59lvhdonaldball: Not worried about the spy connotation, worried about the racist epithet for people of color šŸ™‚#2016-07-1615:59lvhdonaldball: it was weird to the deep south coworkers because they werenā€™t used to the former but definitely heard the latter growing up#2016-07-1615:59lvhI think spooky doesnā€™t have that one either though#2016-07-1616:05donaldballWow. I grew up in Tidewater and thought Iā€™d heard ā€˜em all, but thatā€™s a new one to me.#2016-07-1616:05glvWow. I heard it used that way as a kid (visiting family in Mississippi, probably) but had completely forgotten that usage. That's ā€¦ progress, I guess?#2016-07-1616:06donaldballCache invalidation and naming things, right?#2016-07-1616:08gfrederickssperc#2016-07-1616:14lvhgfredericks: sperc sounds derpy, for sure#2016-07-1616:18gfredericksthere's always spork#2016-07-1616:29glvā€¦ which has been used as the name of a testing library before, so no chance for confusion! šŸ™„#2016-07-1616:30gfredericksthere are no names left#2016-07-1616:31gfredericksI've half a mind to call it lots-of-code-related-to-clojure.spec#2016-07-1616:36lvhshpec is great though!#2016-07-1616:36lvhsounding like meat products doesnā€™t sound bad#2016-07-1616:37gfredericksdon't like including the letter 'c'?#2016-07-1616:37lvhsorry, my misspelling; no opinion šŸ™‚#2016-07-1616:37lvhschpec?#2016-07-1616:39gfredericksyeah#2016-07-1618:02gfredericksalexmiller: I just made this commit (on a branch): https://github.com/clojure/test.check/commit/f7d9230a3afae303fdb466230a780aa71901a853#2016-07-1618:02gfredericksThat's the only potential improvement to test.check that I've heard discussed that would have to be released before 1.9 is released#2016-07-1618:18lvhgfredericks: So, JSON Schemaā€¦ I was thinking of just having something that generates specs for me; except of course itā€™s hard to check that something was (keys :req-un [...]) or whatever because they donā€™t compare identically; so I guess now Iā€™m building a schema, using test.check to generate samples, and then using a JSON schema validator that consumes to validate that the things that were generated validate the original schema#2016-07-1618:18gfredericksyour problem is you can't compare clojure.spec specs for equality?#2016-07-1618:20lvhyeah#2016-07-1618:20lvhI have a json schema and I want a fn that returns the equivalent specs#2016-07-1618:20lvhas a map, probably, because having a fn eval a bunch of spec/def exprs seems bad#2016-07-1618:20gfrederickscan you use the data representation of the schema to do that in a good enough way?#2016-07-1618:21lvhI donā€™t know, thatā€™s the hypothesis, yes#2016-07-1618:21lvhare you asking if the very idea of going from json schema -> spec is possible?#2016-07-1618:21lvhI also want it as data because I presume that you will want to modify/extend it#2016-07-1618:25gfredericksI mean clojure.spec lets you get the form of a spec#2016-07-1618:25gfrederickssomehow#2016-07-1618:25gfredericksI might be misunderstanding#2016-07-1618:26lvhIā€™ll take a look at the api docs then#2016-07-1618:26lvhI was going to say that the main problem I have then is that the registry is global state, and Iā€™m generating them for tests#2016-07-1618:26lvhIā€™m hoping I can with-redefs it or something#2016-07-1618:27lvh(spec/form spec) apparently!#2016-07-1618:29gfredericksyeah that's what I was thinking of#2016-07-1618:29gfredericksgenerate uuid namespaces to register your specs under :)#2016-07-1618:29lvh(clojure.spec/keys :req [:xyzzy.spec.json-schema/a])#2016-07-1618:29lvhis what I got out#2016-07-1618:30lvhawesome, thanks šŸ™‚#2016-07-1618:30lvhIā€™m still writing the generative tests though#2016-07-1618:47lvh(s/form string?) ;;=> :clojure.spec/unknown šŸ˜ž#2016-07-1618:47lvhsame with inst?#2016-07-1618:48gfredericksprobably with any direct pred#2016-07-1618:49gfredericks(s/def ::string string?) and (s/form ::string) would work I bet#2016-07-1619:36sparkofreason@alexmiller: Thanks, slick solution much better than my hand-mangling of the spec. Almost worked, s/merge caused the unnamespaced keys to also make it through. Using s/and did the trick.#2016-07-1620:21glv@lvh if it's a problem that the registry is global state, do you have to use it? Can you just generate the specs and handle them as you need in local state, rather than using s/def?#2016-07-1621:24lvhglv: Iā€™m speccing maps, so wouldnā€™t I need to use keys or every-kv?#2016-07-1621:24lvhkeys takes a spec name#2016-07-1622:15glvHmm ā€¦ you're right about keys, but every-kv just takes two predicates. (And you'd probably want to use map-of instead of every-kv, unless you don't want full validation.)#2016-07-1622:18glvThat is an interesting problem, though ā€¦ I can see how keys would be useful for spec'ing data structures in contexts where you wouldn't want to use the global registry.#2016-07-1623:30sparkofreasonAre there going to be specs for spec?#2016-07-1700:14gfredericksI think so#2016-07-1701:30Alex Miller (Clojure team)Yes to some degree. I went through and wrote most of them at one point but things have drifted. However, we explicitly don't check macro fdefs in spec as that will cause an infinite loop. #2016-07-1701:31Alex Miller (Clojure team)So they may exist mostly as documentation#2016-07-1713:14gfredericksthe uncheckable macro?#2016-07-1721:01lvhWhy is keys a macro?#2016-07-1721:15gfredericksI expect the same reason most of the spec definers is a macro#2016-07-1721:15gfrederickss/is/are/#2016-07-1721:35lvhI donā€™t know what that reason is (I just know that itā€™s annoying when Iā€™m dynamically defining specs and itā€™s complaining that (some expr) isnā€™t a bunch of namespaced keywords)#2016-07-1721:45gfredericks"Yes, this means that more of the surface area of clojure.spec will be macros, but specs are overwhelmingly written by people and, when composed, manually so."#2016-07-1721:45gfrederickswhy're you generating your specs#2016-07-1721:48lvhI have some json schema, Iā€™m trying to produce specs#2016-07-1721:50gfrederickswhy not go the other direction?#2016-07-1721:50lvhso, theyā€™re macros because they want to see the exact expr being used?#2016-07-1721:50lvhhow do you mean?#2016-07-1721:50lvhoh#2016-07-1721:50lvhI see. Iā€™m not going the other direction because Iā€™m not writing the json schema#2016-07-1721:50lvhitā€™s other peopleā€™s json schema, for other services; also json-schema for swagger, which I in turn use to generate specs for even more third party stuff#2016-07-1721:51lvhunfortunately itā€™s unlikely that I get all of these vendors to port their swagger specs to clojure.spec šŸ˜„#2016-07-1721:51gfredericksyes they want to see the exact expression#2016-07-1721:52gfredericksso you probably have to think of it more as a pre-compile step#2016-07-1721:55lvhas in, have the macro-hood leak through?#2016-07-1721:55lvhthatā€™s a good idea, right now I solved it with eval, but bleh#2016-07-1721:56gfrederickseval's okay#2016-07-1721:56lvhIā€™d like to have the specs available easily as a map, because since itā€™s autogenerated thereā€™s a pretty good chance that youā€™ll want to amend the result#2016-07-1721:56lvhe.g. specify a generator, or make a constraint more precise (or both)#2016-07-1721:56gfrederickssimilar is (defmacro load-specs [] (code to make spec forms)) (load-specs)#2016-07-1721:57lvhyeah thatā€™s basically what Iā€™m doing after#2016-07-1721:57lvhI should probably learn the difference between generating intern and def in a macro#2016-07-1721:57lvhbut thatā€™s definitely not ontopic for this channel šŸ™‚#2016-07-1721:58gfredericksif you want to really wash your hands of dynamic runtime monkeydoodling you could generate the code and spit it to a source file that goes into version control#2016-07-1721:58gfredericksso it turns into a dev-time task#2016-07-1721:59lvhthat sounds like something a go programmer might do šŸ˜‰
#2016-07-1721:59lvhIā€™ve already dodged that bullet with caesium (generating an interface)#2016-07-1816:12kendall.buchananAny hints on limiting :num-tests in stest/check? (stest/check b/match-klasses {:clojure.spec.test/opts {:num-tests 10}}) isnā€™t working. Itā€™s gotta be something simple Iā€™m missing.#2016-07-1816:13kendall.buchananWhoops, found it!:#2016-07-1816:13kendall.buchanan(stest/check b/match-klasses {:clojure.spec.test.check/opts {:num-tests 10}})#2016-07-1819:28codonnellI'm struggling to create a spec for dates with a custom conformer, unformer, and generator. I'd love input on how to make this work or if I'm just trying to fit a square peg into a round hole. See the snippet for my attempts.#2016-07-1820:08wilkerlucio@kendall.buchanan quick tip: you can probably replace :clojure.spec.test.check/opts with ::stest/opts#2016-07-1820:09kendall.buchananAh, will ::stest/opts create an alias?#2016-07-1820:10wilkerluciowhen you import a namespace into a short name (like stest for clojure.spec.test.check) you can use double colon notation :: and the namespace will be expanded#2016-07-1820:11codonnell@kendall.buchanan @wilkerlucio I think in this case stest resolves to clojure.spec.test, not clojure.spec.test.check, so you can't use it here.#2016-07-1820:12kendall.buchananTrue.#2016-07-1820:12wilkerluciocorrect#2016-07-1820:12wilkerluciothat was the part I wasn't sure, to where the alias was pointing, eheh#2016-07-1820:12kendall.buchananIn any case, I made myself a helper function and referenced that instead.#2016-07-1820:13wilkerlucioI'm looking forward for the feature that will enable us to alias namespaces without having to require it (specially good for referencing namespaces that not exists, for namespacing only), alex said it's on the table#2016-07-1820:13codonnellthat would be handy#2016-07-1820:34seancorfield@wilkerlucio: Did you see the workaround Alex posted? Using in-ns to create the namespace in memory without needing a file for it.#2016-07-1820:34wilkerlucio@seancorfield: I didn't, seems cool, do you know if that works with cljs?#2016-07-1820:36seancorfieldSorry, no idea.#2016-07-1820:59wilkerlucio@seancorfield: do you have the snippet for that? I tried to search here, but no lucky#2016-07-1821:42seancorfieldOh, you just use (in-ns 'qualified.keyword.namespace) (in-ns 'original.namespace) to create qualified.keyword.namespace in memory and then switch back to whatever ns you were in.#2016-07-1821:43seancorfieldThen you can do (alias 'q 'qualified.keyword.namespace) ā€” because that ns exists now ā€” and then ::q/k expands to :qualified.keyword.namespace/k as "expected".#2016-07-1823:22codonnellIf anyone is curious, I solved my problem above with a dirty hack that relies heavily on implementation. (See snippet.)#2016-07-1823:26codonnellIf with-gen* passed unc along to spec-impl, I could do it with just spec.#2016-07-1913:42borkdude@seancorfield: any reason you're not using create-ns there?#2016-07-1916:01Alex Miller (Clojure team)yeah, create-ns is equally valid#2016-07-1916:53seancorfield@borkdude: I just lifted that code from the example @alexmiller pointed us all to šŸ™‚#2016-07-1922:29seancorfieldRemind me: is there an easy way to create a generator based on a regular expression string?#2016-07-1922:31seancorfield@gfredericks: Looks like I could use test.chuck for this? Although it says it doesnā€™t support anchors (yet)?#2016-07-1922:49gfredericksseancorfield: yes and yes; are you using nontrivial anchors or just the ^...$ sort of thing?#2016-07-1922:51gfredericksI can't remember if supporting general anchors was just a bit messy or if it was Really Hard the way backreferences &c. are#2016-07-1922:52gfredericksor maybe backreferences are easier but lookahead/behind are hard#2016-07-1922:52gfredericksman it's been a while#2016-07-1922:53seancorfieldIā€™m only using #"^ā€¦$"#2016-07-1922:53seancorfieldI can lift out the string between as a separate regex for generation if need be.#2016-07-1922:58gfredericksthat's exactly what I've done#2016-07-1922:58gfrederickswhen in a similar position#2016-07-1922:59gfredericksseancorfield: do you need the anchors there? e.g., re-matches already only matches the whole string#2016-07-1922:59seancorfieldI can change how itā€™s used (from re-find to re-matches).#2016-07-1922:59gfredericksI think I was using anchors because prismatic/schema required them#2016-07-1923:06seancorfieldLooks like it doesnā€™t handle a pattern like [^\s] correctly maybe? I get spaces in the generated strings...#2016-07-1923:06gfredericksthat sure oughta work#2016-07-1923:06gfredericksthe thing is Surprisingly Robustā„¢#2016-07-1923:07gfredericksif it can't handle something it should throw an exception when you try to create the generator#2016-07-1923:07gfredericksif you can share the regex I can try to reproduce#2016-07-1923:07seancorfieldSureā€¦ let me test something smaller...#2016-07-1923:09seancorfieldAh, no, itā€™s doing the right thingā€¦ that generated data is not what it seems šŸ˜ Let me try something else...#2016-07-1923:10seancorfieldWhat appeared to be a space is actually the character for decimal code 55335 (!) so it looks like weā€™re good. Excellent!#2016-07-1923:11gfredericks:)#2016-07-1923:12gfredericksspeaking of which I'd love to hear if anybody has any thoughts about string generators and unicode#2016-07-1923:13gfredericksthere are a couple test.check tickets about it and I have no idea what the best idea as#2016-07-1923:13gfrederickss/as/is/#2016-07-1923:13gfredericksit's a question of what a reasonable distribution is#2016-07-1923:15seancorfield
boot.user> (s/exercise ::m/email)
(["\"󍋩\"@[897.7.55.01]" "\"󍋩\"@[897.7.55.01]"] ["\"ńŸ²®ņ½†·\"@R.V.zZl" "\"ńŸ²®ņ½†·\"@R.V.zZl"] ["\"ó®©£ē˜žó¼”\"@[562.0.2.073]" "\"ó®©£ē˜žó¼”\"@[562.0.2.073]"] ["\"󄼧󄅲ń»®ˆń¼ š\"@[8.158.2.927]" "\"󄼧󄅲ń»®ˆń¼ š\"@[8.158.2.927]"] ["\"ń‘™¾Ę¾\"@4stFz.Tkc43.gZ4j.HRs-.mYf.VbC" "\"ń‘™¾Ę¾\"@4stFz.Tkc43.gZ4j.HRs-.mYf.VbC"] ["\"ń§°”\"@[5.32.7.10]" "\"ń§°”\"@[5.32.7.10]"] ["\"ńæ™ń‰‹¹\"@[67.50.40.3]" "\"ńæ™ń‰‹¹\"@[67.50.40.3]"] ["ń²“‰ó¦µ‡š°„½óŽ—ŖķŽš­‚ńŖ°¤ó½„.ń¢“žō‰…Žņ²„æņ€¼ƒń‡„­óŠˆ­š”žœó›æ‘.ó…·¢ć³ń³€°ó¢œņŽ­š“„æš¼¾ń§šŽ.ō‡Æ›š“³ø.ń”Œž.ń¹¦ņ¹‡Øņ¬Š¤.ń¦ˆ½.󲟙@[6.9.905.824]" "ń²“‰ó¦µ‡š°„½óŽ—ŖķŽš­‚ńŖ°¤ó½„.ń¢“žō‰…Žņ²„æņ€¼ƒń‡„­óŠˆ­š”žœó›æ‘.ó…·¢ć³ń³€°ó¢œņŽ­š“„æš¼¾ń§šŽ.ō‡Æ›š“³ø.ń”Œž.ń¹¦ņ¹‡Øņ¬Š¤.ń¦ˆ½.󲟙@[6.9.905.824]"] ["ń¢¼æóÆ²@9B.bCokjbO.6CJq2iCzy.cwIz" "ń¢¼æóÆ²@9B.bCokjbO.6CJq2iCzy.cwIz"] ["\"󐟳ń£“­óŖņ„²¹š¢ŒŽń¶«ŗšŖ‡¤ó»²ŗ\"@[447.0.87.53]" "\"󐟳ń£“­óŖņ„²¹š¢ŒŽń¶«ŗšŖ‡¤ó»²ŗ\"@[447.0.87.53]"])
#2016-07-1923:16gfredericksemails are pairs?#2016-07-1923:16seancorfieldexercise produces pairs#2016-07-1923:16seancorfieldHereā€™s the regex
(def email-regex
  "Sophisticated regex for validating an email address."
  (-> (str "(([^<>()\\[\\]\\\\.,;:\\
#2016-07-1923:17gfredericksthose are some really good email addresses#2016-07-1923:17seancorfield(I donā€™t remember where we got that from and, yes, I know you canā€™t really validate email addresses correctly with "just" a regex but this has been pretty good for us so far)#2016-07-1923:17seancorfieldBut Iā€™m very impressed that I could just throw that at test.chuck and it worked right out of the box!#2016-07-1923:18gfredericks:D#2016-07-2006:27bbloomthose generated emails are amusing šŸ™‚ iā€™ve noticed that generated tests tend to have pretty wacky looking data - i guess thatā€™s good for tests, but i canā€™t help but wonder: what about ā€œexample dataā€? anybody taken a crack at that? would be cool to lorum ipsum up a full database based on specs!#2016-07-2007:04robert-stuttafordi second that, @bbloom#2016-07-2008:48rickmoynihanSometimes specs seem to dereference themselves... e.g. (s/def ::k (s/keys :req [::foo ::bar])) dereferences ::foo ::bar but I've noticed that for example (s/merge ::k ,,,,) blows up... instead you need to bind :k to a real var... i.e. it won't dereference the keyword into the specs value. I'm just wondering if this is by design? And what the reason is... and how to know what to expect#2016-07-2010:19rickmoynihanactually this does make a lot of sense... sorry being stupid... keys/`:req` requires knowledge of the keys themselves... and clearly giving keywords symbol semantics would be insane#2016-07-2010:24rickmoynihanI guess you have to rely on the doc strings and specs specs... to know the difference between a spec and a keyword... I guess it's kinda like the distinction between quoted and unquoted symbols... I've only just got started with spec - so knowing when to use which isn't entirely clear to me#2016-07-2010:25mpenetis there a version of #(partial instance? %) in core or spec?#2016-07-2010:28mpenetapparently not#2016-07-2010:28mpenetI guess this could even simply be another arity of instance?, given how common this is#2016-07-2010:28rickmoynihanyeah I've written that one more than a few times#2016-07-2010:29mpenetex: (instance? Cluster), which would be a partial on instance?/2#2016-07-2010:30mpenet@alexmiller: would you consider something like this, I'd gladly send a patch for it ?#2016-07-2011:36Alex Miller (Clojure team)No thanks, has been considered by Rich#2016-07-2012:47gfredericksare the lower-level details of clojure.spec, in particular the implementation of s/keys, meant to be public so that users/libraries can use them to write new kinds of specs?#2016-07-2012:48gfredericksit might just amount to implementing a protocol, in which case the question is if the protocol is meant to be public#2016-07-2013:19Alex Miller (Clojure team)I would say that you should consider all implementation details of spec to be just that - subject to change without notice#2016-07-2013:21Alex Miller (Clojure team)spec is meant to provide a relatively complete set of compositional parts, extensible (at the unit level) via predicates#2016-07-2013:22Alex Miller (Clojure team)which is not to say that extending via the protocol is necessarily wrong, just that weā€™re not committed to maintaining that api, so use at your own risk (like any other Clojure ā€œinternals")#2016-07-2013:22Alex Miller (Clojure team)the public api is whatā€™s in the api docs#2016-07-2013:24codonnell@alexmiller: is there a reason with-gen does not pass along the unformer function to its return value?#2016-07-2013:25Alex Miller (Clojure team)donā€™t know - example?#2016-07-2013:25mpenetare there improvement coming to prevent having to manually create non existing kw namespaces?#2016-07-2013:26codonnell
;; conform works, unform fails, gen works
(def date-spec
  (s/with-gen
    (s/conformer date-conformer date-unformer)
    (fn [] (s/gen inst?))))
#2016-07-2013:26Alex Miller (Clojure team)@mpenet maybe#2016-07-2013:27Alex Miller (Clojure team)@codonnell: thx, Iā€™ll pass along to Rich#2016-07-2013:27codonnellno problem#2016-07-2014:41mpenetand next in my wish list of stuff that will not happen for sure is a partially applied version of satisfies? šŸ˜†. Probably an opportunity for a lib with helpers like these#2016-07-2014:44Alex Miller (Clojure team)knock yourself out#2016-07-2019:08gfrederickson that note#2016-07-2019:10codonnellthat would be a fantastic name#2016-07-2019:11gfrederickswell I'm doing it right now#2016-07-2019:15soloistany idea how to write a spec to check an aggregate value of a collection? eg: (s/valid? (= 5 (partial reduce +))) or something like that.. nvm mybad: (s/conform #(= 5 (reduce + %)) [2 3])#2016-07-2019:18gfrederickshttps://github.com/gfredericks/schpec#2016-07-2019:28gfrederickslvh: ^ I finally created schpec#2016-07-2019:28lvhawesome!#2016-07-2019:28lvhI just saw an email notification from the corner of my eye#2016-07-2019:29lvh"whatever other feature you miss from plumatic/schemaā€ šŸ’Æ#2016-07-2019:30lvhitā€™s distracting to see how many layers deep you can go ā€” I was trying to do something useful with swagger specs and now Iā€™m explaining to clojure.spec how json schema works so I can generate json schemata so I can generate specs from them so I can generate instances of those specs and then see if they match the original json schema#2016-07-2019:30lvhitā€™s specs all the way down šŸ™‚#2016-07-2019:31gfredericksa shed to put bike sheds in#2016-07-2019:53bhaumanI've got some bikes for that shed ...#2016-07-2020:07Alex Miller (Clojure team)@jjcomer: read the instrument docs carefully - "Instruments the vars named by sym-or-symsā€, "Opts for symbols not included in sym-or-syms are ignored.ā€, etc - I think that right away rules out your -3 and -4 examples. Iā€™ll look closer at -2. Some of this stuff really makes most sense when instrumenting a ns or all and passing an opts map that sets up the env you want.#2016-07-2020:18Alex Miller (Clojure team)for -3, if you replace
`inc-it
with
[`inc-it `print-it]
I think that works
#2016-07-2020:20Alex Miller (Clojure team)for -2 and -4, interestingly if you do that instrument and check outside a defn, just at the repl, they work#2016-07-2020:22Alex Miller (Clojure team)(that is, donā€™t print)#2016-07-2020:22Alex Miller (Clojure team)which implies things about compilation and may be a problem#2016-07-2020:37Alex Miller (Clojure team)yeah, when the compiled class is loaded, the var has not yet been altered by instrument and thatā€™s what is loaded into the class constants and used at runtime - the instrumented version of the var is not used#2016-07-2020:42Alex Miller (Clojure team)nah that doesnā€™t make sense - check takes the symbol, not the var and resolves at runtime#2016-07-2020:46kendall.buchananI suspect Iā€™m doing something wrong, but Iā€™m reliably getting errors like this:#2016-07-2020:46kendall.buchananjava.util.concurrent.ExecutionException: java.lang.ClassCastException: app.business.klass_test$eval75458$fn__75459 cannot be cast to clojure.lang.MultiFn, compiling:(clojure/test/check/clojure_test.cljc:95:1)#2016-07-2020:46kendall.buchananFrom stest/check.#2016-07-2020:47kendall.buchananItā€™s typically when I restart the REPL and try to run check from my test files (without first navigating over and evaluating the functions it calls). Note Iā€™m requiring the namespaces first.#2016-07-2020:47kendall.buchananThoughts?#2016-07-2020:57Alex Miller (Clojure team)I havenā€™t seen that - looks like it points to https://github.com/clojure/test.check/blob/master/src/main/clojure/clojure/test/check/clojure_test.cljc#L96#2016-07-2020:58kendall.buchananThere isnā€™t a danger to calling stest/check within deftest is there?#2016-07-2020:59Alex Miller (Clojure team)shouldnā€™t be#2016-07-2020:59Alex Miller (Clojure team)is lein or other build tooling involved?#2016-07-2020:59Alex Miller (Clojure team)lein hacks up a lot of the clojure.test stuff#2016-07-2021:00kendall.buchananItā€™s certainly possible ā€“ Iā€™m using weavejesterā€™s eftest.#2016-07-2021:00Alex Miller (Clojure team)does it happen if you donā€™t?#2016-07-2021:01kendall.buchananIt happens if I simply eval the form, yes...#2016-07-2021:01kendall.buchananBut itā€™s a fairly large project ā€“ itā€™s possible something else is intruding.#2016-07-2021:02Alex Miller (Clojure team)eftest is definitely also mucking with clojure.test/report#2016-07-2021:02kendall.buchananYeah, itā€™s a touch thing ā€“ if I go to the test namespace, require everything, and run a single form, it works.#2016-07-2021:02kendall.buchananBut if I restart the REPL, go to the namespace, eval everything all at once, it falls apart.#2016-07-2021:03kendall.buchananK, thanks. Helps to know itā€™s likely in my environment. Iā€™ll report back when I find it.#2016-07-2021:03kendall.buchananBut yeah, only happens on check.#2016-07-2021:04Alex Miller (Clojure team)both eftest and check are bashing the same clojure.test hook for reporting and that seems to be in the realm of the error youā€™re seeing#2016-07-2021:04Alex Miller (Clojure team)so Iā€™m going to suspect that combination as the most likely probable cause#2016-07-2021:05Alex Miller (Clojure team)I guess not ā€œcauseā€, I donā€™t understand why the two together fails but I suspect thatā€™s involved#2016-07-2021:06kendall.buchananOkay, great insights. Iā€™ll start by disabling eftest.#2016-07-2021:06Alex Miller (Clojure team)if you can get a reproducible case that fails with both, feel free to file a jira and maybe one or the other can be made more tolerant/harmonious#2016-07-2021:13kendall.buchananOkay...#2016-07-2021:13kendall.buchananDisabling eftest altogether had no effect:#2016-07-2021:14kendall.buchananThatā€™s after taking any reference to eftest out of project.clj, and running lein test on its own.#2016-07-2021:14kendall.buchananIā€™ll keep poking around.#2016-07-2021:16Alex Miller (Clojure team)are you explicitly requiring clojure.test.check.clojure-test?#2016-07-2021:17Alex Miller (Clojure team)maybe to use defspec?#2016-07-2021:17kendall.buchananMade myself a couple helpers to simplify tests with check.#2016-07-2021:22Alex Miller (Clojure team)unless youā€™re loading it, itā€™s not clear to me why clojure.test.check.clojure-test is getting called at all. nothing in core calls it. Your definition above wonā€™t call it, yet you are compiling it. are you AOTā€™ing?#2016-07-2021:22kendall.buchananSeeing if I can create a reproducible lein project...#2016-07-2021:22kendall.buchananNo AOT.#2016-07-2021:30kendall.buchananCool, just reproduced it.#2016-07-2021:30kendall.buchananVery well could be lein, cause I have a barebones example.#2016-07-2021:34kendall.buchanan@alexmiller: https://github.com/kendagriff/check#2016-07-2021:36kendall.buchananIā€™m happy to post to JIRA too if you feel like itā€™s a legit reproduction.#2016-07-2021:37Alex Miller (Clojure team)gimme a sec to look at it#2016-07-2022:27Alex Miller (Clojure team)@kendall.buchanan: looks like this has nothing to do with spec per se#2016-07-2022:28Alex Miller (Clojure team)clojure.test.check.clojure-test assumes that clojure.test/report is a multimethod (which it is in core) and modifies one of itā€™s cases - this is loaded as a side effect when clojure.test.check is loaded via gen via spec.test/check#2016-07-2022:29Alex Miller (Clojure team)however, Leiningen monkeypatches core and replaces that clojure.test/report function with something thatā€™s not a multimethod#2016-07-2022:29kendall.buchananI noticed the project threw off another error about gen being missingā€¦ until I added the test.check dependency.#2016-07-2022:29Alex Miller (Clojure team)so lein test and test.check.clojure-test appear to be incompatible afaict#2016-07-2022:29Alex Miller (Clojure team)you can turn off the monkey patching in lein though#2016-07-2022:30Alex Miller (Clojure team)with :monkeypatch-clojure-test false in your project.clj#2016-07-2022:30Alex Miller (Clojure team)@gfredericks: ^^ you aware of any of this?#2016-07-2022:30kendall.buchananInteresting. Iā€™m assuming lein has documented the trade-off of turning it off?#2016-07-2022:31Alex Miller (Clojure team)I think you then lose ā€œlein retest"#2016-07-2022:31kendall.buchananNot sure Iā€™d ever have known to look to lein for thisā€¦ is it appropriate to introduce this into spec docs as a ā€œgotchaā€? (just curious how you guys deal with things like thisā€¦)#2016-07-2022:32Alex Miller (Clojure team)maybe, I was not aware of it before. the master version of test.check actually doesnā€™t load clojure-test anymore, so you wouldnā€™t even see this come up unless you included it explicitly#2016-07-2022:33Alex Miller (Clojure team)I gotta go, but will think about it more tomorrow#2016-07-2022:33kendall.buchananOkay, I am loading explicitly in my project as wellā€¦ good point.#2016-07-2022:33kendall.buchanan@alexmiller: Thanks much for spending so much time on that.#2016-07-2022:33kendall.buchananBig help.#2016-07-2022:33Alex Miller (Clojure team)thx for the good and simple repro! that helped enormously#2016-07-2022:34kendall.buchanannp#2016-07-2022:34Alex Miller (Clojure team)@jjcomer I havenā€™t forgotten yours either - still looking at it#2016-07-2022:35jjcomer@alexmiller: thanks :) #2016-07-2022:38gfredericksalexmiller: I've been using lein-test with test.check.clojure-test for years, so I assume it's something subtler than a full incompatibility. I wasn't aware of this before, no, so I'll look into it#2016-07-2022:41kendall.buchanan@alexmiller: One final note: turning off monkeypatching fixed it. Turning eftest back on broke it again.#2016-07-2022:42kendall.buchananItā€™s a great start, though, to at least know where it breaks down.#2016-07-2022:47gfrederickskendall.buchanan: I'm trying to reproduce it now#2016-07-2022:52gfrederickskendall.buchanan: any tips beyond "using test.check and lein test"?#2016-07-2022:52kendall.buchanangfredericks: Did you clone the repo above?#2016-07-2022:53gfredericksoh no, didn't see it#2016-07-2022:54kendall.buchananThat should be everything you need: https://github.com/kendagriff/check#2016-07-2022:54kendall.buchananI think the odd thing about stest/check, though, is it requires test.check, explicitly.#2016-07-2022:55kendall.buchananOtherwise you get errors saying generators are missing.#2016-07-2022:55gfrederickswhat do you mean by "explicitly"?#2016-07-2022:55kendall.buchananAdded to the list of dependencies.#2016-07-2022:55kendall.buchananItā€™s a function that canā€™t run without the user having to introduce another dependency.#2016-07-2022:57kendall.buchanan@seancorfield seems to have identified the same thing in a slightly different context: http://dev.clojure.org/jira/browse/CLJ-1936#2016-07-2022:59gfredericksstest/check is explicitly about generative testing, right?#2016-07-2022:59kendall.buchananYes, it tests function specs.#2016-07-2022:59gfredericksI'm not sure how it could make sense to run that without test.check available, given the underlying "test.check is optional" design decision#2016-07-2022:59kendall.buchanan(Itā€™s the only way to test the return values of functions, afaik.)#2016-07-2023:00kendall.buchananOkay, I figured as much.#2016-07-2023:03gfrederickskendall.buchanan: I still don't quite understand how this happens (or rather, why it doesn't happen all the time), but I can at least tell you it's fixed on test.check master#2016-07-2023:04gfredericksI'm hoping to make a new release soon#2016-07-2023:04kendall.buchananWow, thatā€™s fantastic.#2016-07-2023:05kendall.buchananThatā€™s great news, thansk.#2016-07-2023:06gfredericksoh actually I get it#2016-07-2023:06gfredericksI think lein does the monkeypatching after all the requires get loaded#2016-07-2023:06gfredericksso this is only a problem if you (or something) loads the test.check.clojure-test namespace later (at runtime)#2016-07-2023:06kendall.buchanan@gfredericks: Just curiousā€¦ if stest/check is all about generative testing, why is it in clojure.spec.test? Does it not make more sense to move it to test.check (making one aware of the other, but not in both directions)?#2016-07-2023:07gfrederickskendall.buchanan: that might be possible; but spec might be too tied together with test.check to make that practical#2016-07-2023:07kendall.buchananMakes sense.#2016-07-2023:37gfrederickskendall.buchanan: http://dev.clojure.org/jira/browse/TCHECK-113#2016-07-2023:38gfredericksI'm not planning to investigate it anymore (I considered closing the issue immediately after creating it) until I hear that it's a problem for someone on test.check master somehow#2016-07-2023:38gfredericksI figured I'd create the ticket to document the problem for future historians though#2016-07-2101:15gfredericks@alexmiller here's an interesting question about spec naming conventions in namespaces that exist primarily to provide specs: https://github.com/gfredericks/schpec/pull/1/files#r71633898#2016-07-2101:15gfredericksi.e., is there a difference between the real specs and the specs describing args to the functions to make specs?#2016-07-2101:16gfredericksI'm imagining maybe namespacing the function arg/ret specs one level deeper, using the name of the function as the last component of the namespace#2016-07-2101:16gfrederickswhich of course immediately brings up the old "aliasing a non-code namespace"#2016-07-2101:17gfredericksproblem#2016-07-2101:20donaldballIf s/keys allowed anonymous specs for :req-un and :opt-un, that would be the obvious choice, but absent that, using the fn name as the last component of the spec namespace is p reasonable#2016-07-2101:21gfredericksoh hi donaldball #2016-07-2101:21donaldballhowdy, stranger#2016-07-2101:22gfredericksdonaldball: would you say the main point of these specs is combining range-checks with type-agnosticism?#2016-07-2101:28donaldballI think of them as more describing domains in the math sense, but weā€™re probably saying the same thing#2016-07-2101:31bhaumansooner or later we are going to have a record like map spec that doesn't require namespaced keys#2016-07-2101:37donaldballIā€™ve written a few score specs now and am tentatively concluding that Richā€™s encouragement towards namespaces keys is tremendously valuable for describing domain model maps, and not very useful, maybe even a bit abrasive, when describing function option maps or kwargs.#2016-07-2101:39bhaumanI agree completely, did you see my suggestion on the PR?#2016-07-2101:40donaldballThatā€™s an interesting idea, I havenā€™t tried it in that way yet#2016-07-2101:41bhaumana bit verbose but it feels like it fits the problem better#2016-07-2101:42donaldballYeah, itā€™s more in the spirit#2016-07-2101:42bhaumanit allows duplicate keys but ...#2016-07-2101:42donaldballIf you had a bunch of fns that shared many of the same options, maybe a spec registry would be called for#2016-07-2101:42bhaumanabsolutely, in fact it would be shame not to#2016-07-2101:44donaldballkwargs do allow duplicate keys anyway, so your formulation may be more correct#2016-07-2101:45donaldball(even though itā€™s probably a usage bug when it happens)#2016-07-2101:45bhaumanwell its straightforward enough to add a check for that#2016-07-2101:46bhaumanbut a bit cumbersome ... really asking for a macro at that point#2016-07-2101:49gfredericksschpec.bhauman#2016-07-2101:50bhaumanthe notorious bad boy ns#2016-07-2101:50bhaumanoh you're using that ...#2016-07-2101:52bhaumanall the gen functions return (repeat 'figwheel)#2016-07-2102:01gfredericksbhauman: so the whole point of your comment code is to avoid the namespacing-requirement of keys*?#2016-07-2102:01gfredericksI bet that spec conforms a lot more awkwardly than keys* does :/#2016-07-2102:17bhaumanagreed#2016-07-2102:20bhaumanbut I'm just trying to use the whats available to avoid registering in situations that don't merit it#2016-07-2102:20gfrederickswe could make a macro that names them in the background with gensyms or something#2016-07-2102:21bhaumanbut you could also make a macro that doesn't require the registration#2016-07-2102:21gfredericksby using keys* or by avoiding it?#2016-07-2102:22bhaumanby avoiding it#2016-07-2102:23bhaumanbut I don't really see whats wrong with the regex formulation#2016-07-2102:24gfredericksless readable, conforms weird#2016-07-2102:24gfredericksI think that's it#2016-07-2102:24gfrederickswell more readable in one sense I guess#2016-07-2102:24gfrederickskeys* requires you to go somewhere else for the rest of the spec#2016-07-2102:25bhaumanand a macro could fix that#2016-07-2102:25bhaumanthe conforming, and readability ... but this isn't a big deal at all#2016-07-2102:25bhaumanI'm just interested because this has come up a bunch for me#2016-07-2102:26bhaumanI've been writing a spec for leiningen#2016-07-2102:26gfredericksyou can change the conforming with a macro?#2016-07-2102:26gfredericksI'm still hazy on all the details of clojure.spec#2016-07-2102:26bhaumanyou mean the final conformed output?#2016-07-2102:27bhaumanwith a conformer yes#2016-07-2102:27gfredericksman I gotta learn this stuff better#2016-07-2102:27bhauman(conformer (fn [x] (if (= x "one") 1 ::spec:invalid))#2016-07-2102:28gfredericksso you could take the sequence of pairs that your style of regex creates and conform that further into a map?#2016-07-2102:29bhaumanyes sireee#2016-07-2102:29bhaumanyou can do ANYTHING#2016-07-2102:30gfredericksthis has been another episode of Turing Completeness with Bhauman #2016-07-2102:30gfredericksI'm interested in how useful conformers are for dealing with transforming things in and out of json and jdbc#2016-07-2102:32gfredericksI'd like to be able to precising describe the idiosyncratic shape of some funky json at the same time as describing the pristine clojure-friendly equivalent and automatically get functions to translate between them#2016-07-2102:32gfredericksditto for goofy jdbc types#2016-07-2102:32bhaumanwell it can be done#2016-07-2102:32gfredericksthat's all I wanted to hear#2016-07-2102:32bhaumanand lets face it ... it should be done and done now#2016-07-2102:33gfredericksschpec.bijections#2016-07-2102:33gfrederickss/precising/precisely/#2016-07-2103:03seancorfieldclojure.spec FTW: as weā€™re specifying more of our domain model and rewriting our validation logic on top of s/conform, weā€™re discovering some interesting edge cases that we hadnā€™t considered before, as well as uncovering inconsistencies between how we treat certain pieces of data, in different parts of our application. Itā€™s very enlighteningā€¦ and itā€™s also making us feel a lot more confident in our updated validation code!#2016-07-2103:05seancorfieldNot all of our validation is suitable for clojure.spec thoā€™: we have several places where the validation is contextual, based on current data from the environment, but all of the basic "shape" validation is much nicer now.#2016-07-2106:06bbloomsurely clojure.spec vNext will have context sensitive logic variables šŸ˜‰#2016-07-2106:09bbloomunrelated: Iā€™m all for the openness of maps etc, but Iā€™ve run in to some real bugs where typos or misplaced data gets ignored. Is there anything currently or planned for identifying ā€œextraā€ data? Iā€™ve found it incredibly useful to log unknown data in the past.#2016-07-2106:10bbloomAs an example from a Go program I worked on recently, we were parsing a Toml config file and our tester lost a few hours to a flag that was in the wrong section. So I added a call to this ā€œUndecodedā€ method: https://github.com/BurntSushi/toml/blob/99064174e013895bbd9b025c31100bd1d9b590ca/decode_meta.go#L113 ā€” logged all the undecoded keys and he never had this problem again#2016-07-2106:11seancorfield@bbloom: I think the only solution right now is s/and with a set operation on the keys of the map?#2016-07-2106:11bbloom@seancorfield: gotcha, thanks. was hoping not to treat it as invalid only to discover that it exists#2016-07-2106:13seancorfieldYeah, if you're dealing with just :req keys you're OK but once you get into optional key territory it's tricky šŸ˜ž#2016-07-2106:13bbloomsomething like s/conform but like s/unconformed that returns paths to all the data that was ignored#2016-07-2106:13bbloomthat Go code just returns a collection of paths#2016-07-2106:39mpenetanother case like this in our context: maps as database records, an invalid map-entry could cause a query execution exception, invalid column etc (a map missing a key because of the typo is valid too, which doesn't help)#2016-07-2106:42mpenet@seancorfield: do you have an example of this keys+and code?#2016-07-2106:43mpenetI guess this can be done with map-of as well#2016-07-2106:43mpenet(map-of #{:foo :bar} ...)#2016-07-2106:52bfabry@mpenet: I think it'd be like (s/and (s/keys :req-un [::foo]) #(every? #{:foo} (keys %)))#2016-07-2106:53mpenetI see#2016-07-2107:08borkdudeGood 15 minute screencast by @stuarthalloway: http://blog.cognitect.com/blog/2016/7/13/screencast-spec-leverage#2016-07-2108:23mpenet@seancorfield: wouldn't this fail https://github.com/clojure/java.jdbc/blob/f9fe3dde5b4bc5f1a5b5a79ffb5374fad3377b0b/src/test/clojure/clojure/java/test_jdbc.clj#L31 ?#2016-07-2108:23mpenet" Instruments the vars named by sym-or-syms, a symbol or collection of symbols, or all instrumentable vars if sym-or-syms is not specified."#2016-07-2108:25mpenetI guess you need to call ns-publics and then pass the coll to instrument, or just call instrument without arg#2016-07-2110:58mpenetmaybe this is something that changed recently#2016-07-2111:17mpenet
(s/explain-str (s/cat :statements (s/+ string?)
                                  :type keyword?)
                    [["q"] :logged])
--> "In: [0] val: [\"a\"] fails at: [:statements] predicate: string?\n"
#2016-07-2111:17mpenetany idea of what I am doing wrong here?#2016-07-2111:19mpenetthis is part of my attempt to debug https://github.com/mpenet/alia/blob/feature/specs/modules/alia-spec/src/qbits/alia/spec.clj#L296-L299 (the rest of the specs seem fine so far)#2016-07-2111:34mpenetreplacing (s/+ string?) by (s/coll-of string? :min-count 1 or (s/spec (s/+ string?)) works, but I am not sure why "+" failed here#2016-07-2111:53mpenet"When regex ops are combined, they describe a single sequence. If you need to spec a nested sequential collection, you must use an explicit call to spec to start a new nested regex context." okay#2016-07-2113:19gfredericks@bbloom: regarding discovering typos/etc in open maps, I've been pondering the idea of a pair of schemas -- the normal one, for which violations are a bug/error-condition/whatever, and a more restrictive one, for which violations are merely suspicious, and might warrant logging, etc.#2016-07-2113:22gfredericksE.g. "this string is probably a uuid", "this map probably has no unknown keys", ...#2016-07-2113:29gfredericks@bbloom: your conform/unconform idea suggests that you could just roundtrip the data and do a diff, but that would only work with map keys and not general mistakes I think#2016-07-2113:29mpenet@gfredericks: something that could be nice for speck would be a schema->spec converting function/macro. When you have large'ish Schemas (ex multi-level maps) it get tedious/verbose fast. I might actually give it a try if I get some free time this week end#2016-07-2113:30gfredericksBy "schema" you mean something like plumatic/schema's api where the schema is the same shape as the data?#2016-07-2113:30mpenetyes#2016-07-2113:31mpenetI meant plumatic/schema schemas#2016-07-2113:31gfredericksYeah I definitely imagined that would show up#2016-07-2113:31mpenetsince it's so wide spread atm#2016-07-2113:31gfredericksOh you mean their whole API?#2016-07-2113:31gfredericksOr as much as possible at least#2016-07-2113:31mpenetno, I mean just an utility to convert schemas from their format to spec#2016-07-2113:31mpenetsome of their api is hard to port to spec#2016-07-2113:32mpenetex all the coercion stuff
#2016-07-2113:32gfredericksA 1-time dev utility or a runtime thing?#2016-07-2113:32mpenetwell I guess if you do the latter the former spawns in a couple of lines#2016-07-2113:33gfredericksThe former can be less robust#2016-07-2113:34mpenetIn my case I am mostly concerned about the ton of map schemas we have in our projects, would be nice to be able to port these without spending ages on it and laying a bug field in the process#2016-07-2113:34mpenetI could just not bother as well, and use both in parallel#2016-07-2113:37gfredericksJust punt on certain things and point the user to where it's broken#2016-07-2113:37mpenetthe example of nested maps is a good one I think, a very concise schema from plumatic Schema can end up being a tons of lines of mostly s/def's for k/v#2016-07-2113:38mpenetcould end up being a more concise way to write map specs as well#2016-07-2113:38gfredericksIt definitely would be#2016-07-2113:42EdHi ... I've just been looking at spec (alpha10) and am confused by some results I'm seeing ...#2016-07-2113:43Edthe docs from fdef suggest that it should instrument the function referred to#2016-07-2113:43Edbut it doesn't seem to#2016-07-2113:43gfredericksNot by itself#2016-07-2113:44gfredericksYou have to call c.s.test/instrument#2016-07-2113:44Edand that will permanently replace the var with a function that checks it's args?#2016-07-2113:45Edthe docs for fdef say:#2016-07-2113:45EdOnce registered, function specs are included in doc, checked by instrument, tested by the runner clojure.spec.test/run-tests, and (if a macro) used to explain errors during macroexpansion.#2016-07-2113:47gfredericks"Checked by instrument" is a reference to the function I mentioned above#2016-07-2113:48gfredericksThe wording is confusing though since that's not obvious#2016-07-2113:48bhauman@gfredericks: I have a strict-keys macro that uses a dynamic variable to set it's level of strictness#2016-07-2113:48gfredericks@bhauman: interesting#2016-07-2113:48Edok ... so I need also need to include clojure.spec.test in my production code to have specs checked at runtime?#2016-07-2113:48bhauman:ignore, :warn, :blowup#2016-07-2113:49gfredericks@bhauman: I'm in the middle of pushing more customization into plumatic/schema to support this kind of thing#2016-07-2113:49Ed@gfredericks: thanks#2016-07-2113:50bhaumanI think a better solution is to have both the dynamic variable with a spec level override to force a certain level for certain specs#2016-07-2113:50gfredericks@l0st3d: I think the intention is for you to do more explicit checks if you want production checking#2016-07-2113:51gfredericks@bhauman: cuz e.g. defproject is always open but certain submaps are more restrictive? #2016-07-2113:53Ed@gfredericks: fair enough ... just thought that the docs suggested it would do those checks for me if I included the fspecs ... just trying to work out the api atm i think šŸ˜‰ .. thanks for your help#2016-07-2113:53bhaumanjust that you would want certain maps to never allow extraneous members#2016-07-2113:53bhaumanthere is actually a really interesting case for misspellings#2016-07-2113:53bhaumanand still having an open map#2016-07-2113:54bhaumanhaving a level of verification that does a fuzzy-selection of the set of keys#2016-07-2113:55bhaumanso the provided keys are checked with a distance to the spec'ed keys#2016-07-2113:56bhaumanThis could frankly be done all the time for something like (s/keys ) where a warning will be generated#2016-07-2113:57bhauman@bbloom: ^#2016-07-2113:57bhaumanbut being able to set a dynamic var to adjust this level of checking seems appropriate#2016-07-2114:10sundbpiā€™m struggling with a recursive map spec. what iā€™m trying to do is something like this:#2016-07-2114:10sundbp
(s/def ::node-content ::series)
(s/def ::node-inputs ::graph-seq)
(s/def ::graph-node (s/keys :req [::node-content ::node-inputs ::label ::stream]))
(s/def ::graph-seq (s/coll-of ::graph-node))
#2016-07-2114:10sundbpi donā€™t quite get how i can have a key spec referring to having a required key that is related to itself..#2016-07-2114:12sundbpI canā€™t find any examples of a recursive spec of a map#2016-07-2114:45Alex Miller (Clojure team)
(s/def ::label keyword?)
(s/def ::children (s/coll-of ::node))
(s/def ::node (s/keys :req [::label] :opt [::children]))
(s/conform ::node {::label :a ::children [{::label :b} {::label :c}]})
=> #:user{:label :a, :children [#:user{:label :b} #:user{:label :c}]}
#2016-07-2114:45Alex Miller (Clojure team)thereā€™s a simple example - I canā€™t quite work out what youā€™re trying to do#2016-07-2115:54sundbpthanks. iā€™ll check it out#2016-07-2115:58sundbpfirst time i tried something like that i thought i got a complaint about the spec not existing.. think was because i didnā€™t have ::graph-seq and ::node-inputs in the right order.#2016-07-2115:58sundbpsimple mistake#2016-07-2115:58sundbpfine if i move those 2 around#2016-07-2116:12seancorfield@mpenet: My understanding is that, despite the docstring, you can use instrument to turn on instrumentation for an entire namespace using that call.#2016-07-2116:13mpenetHmm I tried but it didn't seem to do anything.#2016-07-2116:13mpenetit returned [], which would seem to indicate it instrumented nothing#2016-07-2116:14mpenetI am not at work anymore, but I guess I ll check the souce next chance I get#2016-07-2116:21mpenetseems it might have changed here: https://github.com/clojure/clojure/commit/a4477453db5b195dd6d1041f1da31c75af21c939 at least that's what the docstring would suggest#2016-07-2116:22mpenet(didnt try it)#2016-07-2116:22seancorfieldYeah, something definitely isnā€™t working right now ā€” I changed an fdef spec and the tests still passā€¦ investigating.#2016-07-2116:22seancorfieldIt definitely used to work.#2016-07-2116:23mpenet@alexmiller: would it be a bug or is it an intentional change?#2016-07-2116:23mpenet^ instrument no longer working with a ns argument#2016-07-2116:24Alex Miller (Clojure team)thatā€™s intentional - use st/enumerate-namespace to produce a list of syms in an ns#2016-07-2116:24mpenetok makes sense#2016-07-2116:27seancorfieldHmm, I missed that change in the release notes @alexmiller#2016-07-2116:27Alex Miller (Clojure team)alpha8#2016-07-2116:27seancorfieldAlthough now I canā€™t get my fdef specs to work at all#2016-07-2116:27Alex Miller (Clojure team)iirc#2016-07-2116:28Alex Miller (Clojure team)for check or instrument?#2016-07-2116:28Alex Miller (Clojure team)I have just tracked down a problem with check#2016-07-2116:28seancorfieldWhen I call fdef, the spec does not subsequently show up on doc...#2016-07-2116:29Alex Miller (Clojure team)how are you calling it? expects a fully-qualified symbol#2016-07-2116:29Alex Miller (Clojure team)actually, it resolves, so not necessarily fully-qualified#2016-07-2116:32seancorfieldI had (s/fdef drop-table-ddl ā€¦) after referring in :all and that used to work but doesnā€™t now. I changed it to (s/fdef clojure.java.jdbc/drop-table-ddl ā€¦) and it works now.#2016-07-2116:41Alex Miller (Clojure team)yeah, if you use a bare symbol for def or fdef, it will treat that as <current-ns>/sym#2016-07-2116:41Alex Miller (Clojure team)so itā€™s not going to pick up refer's#2016-07-2116:44seancorfieldThat changed at some point. This code used to work.#2016-07-2116:44seancorfield(not a big deal but the silent "failure" is disturbing)#2016-07-2116:51Alex Miller (Clojure team)fdef stuff changed around 6/7 - there were some major rewrites of it in there#2016-07-2116:52Alex Miller (Clojure team)you can use st/instrumentable-syms to verify that things are speced maybe?#2016-07-2116:56sundbpwith these changes - if you want to run your clojure.test tests with specs instrumented and checked for fdefā€™s, whatā€™s the proposed setup?#2016-07-2116:59Alex Miller (Clojure team)youā€™ll need to turn on instrumentation#2016-07-2117:00Alex Miller (Clojure team)so you can do that per-test, in a fixture, etc#2016-07-2117:02sundbpif one would like to turn on instrumentation for ā€œeverythingā€ in a fixture - does one have to manually enumerate the NS to then do instrumentable-syms and finally instrument? i.e. thereā€™s nothing that just turns on instrumentation for all NS in project?#2016-07-2117:06Alex Miller (Clojure team)just (st/instrument) is supposed to do that#2016-07-2117:07Alex Miller (Clojure team)thatā€™s like the old instrument-all#2016-07-2117:07Alex Miller (Clojure team)and macro fdefs are always instrumented in macroexpansion#2016-07-2117:16seancorfieldHereā€™s what I ended up with in java.jdbc:
(try
  (require 'clojure.java.jdbc.spec)
  (require 'clojure.spec.test)
  (let [syms ((resolve 'clojure.spec.test/enumerate-namespace) 'clojure.java.jdbc)]
    ((resolve 'clojure.spec.test/instrument) syms))
  (println "Instrumenting clojure.java.jdbc with clojure.spec")
  (catch Exception _))
#2016-07-2117:16seancorfieldAnd I had to qualify all the symbols in clojure.java.jdbc.spec fdef calls in order to get those working again.#2016-07-2117:16seancorfieldLife on the bleeding edge ā€¦ šŸ™‚#2016-07-2117:17seancorfieldThanks @mpenet for the heads up on that ā€” I hadnā€™t noticed it was broken!#2016-07-2117:24fentonin my app is use GPS data. Normally my data structure looks like: {:lat 33.3 :lng -129.3}. With spec things are looking like: {:pc.api/latitude 33.3 :pc.api/longitude -129.3}. I'm finding it less fun to type that everywhere. I guess I could alias my namespace to [pc.api :as a] and change latitude and longitude to lat/lng then get {:a/lat 33.3 :a/lng -129.3}. R others doing/finding something similar? I wonder if having keywords as short as lat/lng leads to being unclear? Thoughts?#2016-07-2117:29seancorfieldYou can use unqualified keywords in maps if you want @fenton#2016-07-2117:29mpenet:) @seancorfield i looked a bit at clj.jdbc to spec parts of alia, that s how i spotted this#2016-07-2117:30seancorfieldIā€™m cutting 0.6.2-alpha2 with those updates.#2016-07-2117:30fenton@seancorfield: do you mean un-qualified?#2016-07-2117:33fenton@seancorfield: I guess I was suggesting the ns qualified keywords seem a bit of a pain to type everywhere...maybe I should look into using un-qualified keywords....not sure the implications off the top of my head... I'm thinking, how do you use unqualified keywords in other files? The way I'm using specs is to put them into a third *.cljc file that is shared by my front and backends. So dont i have to use qualified keywords in that case?#2016-07-2117:33seancorfieldYes, sorry, typo.#2016-07-2117:33seancorfieldFixed šŸ™‚#2016-07-2117:33fentonI keep my specs in an external library...does that make un-qualified an issue or can i still have them defined elsewhere?#2016-07-2117:33seancorfieldWell, you can do #::a{:lat 33.3 :lng -129.3}#2016-07-2117:34fenton@seancorfield: okay that looks a bit better....#2016-07-2117:34seancorfieldA map does not need qualified keys in order to be used with spec.#2016-07-2117:35seancorfield(s/keys :req-un [::lat ::lng]) will conform {:lat 33.3 :lng -129.3}#2016-07-2117:35fenton@seancorfield: oh really? how does spec use it then?#2016-07-2117:35fentonoh...really...cool!#2016-07-2117:35seancorfieldhttps://clojure.org/guides/spec#_entity_maps gives examples.#2016-07-2117:35fentoni guess i missed that, i'll go check it again...#2016-07-2117:36seancorfieldScroll down to where it shows :req-un being used.#2016-07-2117:36fentonok...#2016-07-2117:37fentoni'll give that a try....that'll clean up my code nicely i think.#2016-07-2117:37fentonthanks!#2016-07-2118:25fenton@seancorfield: I'm unfamiliar with the #::a{:lat 3.3 :lng 3.3} syntax. i.e. the pulling of the namespace out in front of the map. is there some documentation about that somewhere?#2016-07-2118:34seancorfieldhttp://dev.clojure.org/jira/browse/CLJ-1910#2016-07-2118:44Alex Miller (Clojure team)and more officially http://clojure.org/reference/reader#_maps#2016-07-2119:00Alex Miller (Clojure team)@jjcomer I can explain what youā€™re seeing now btw. spec.test/check returns a lazy sequence of test results per sym. In your test, you are not realizing those results, then immediately uninstrumenting them, then later realizing (actually running the check on the uninstrumented functions). If you wrap a doall around your two calls to check, that addresses why youā€™re not seeing the instrumentation.#2016-07-2119:01Alex Miller (Clojure team)while check does document this laziness, I admit it was a surprise to me.#2016-07-2119:02jjcomerGotcha. Thanks for the debug :)#2016-07-2119:02Alex Miller (Clojure team)and then the other thing was not including the sym youā€™re replacing in the instrument list#2016-07-2119:05jjcomerAwesome, I'll give it another go this afternoon. Thanks again#2016-07-2119:09Alex Miller (Clojure team)np, thx for the repro#2016-07-2122:50seancorfieldIf you define a spec with a custom generator (using s/with-gen), does clojure.spec filter the generated values using the spec itself, or does it trust that the generator will only ever produce conforming values?#2016-07-2122:51seancorfield(I ask because I have a spec with a very complex validation predicate and so I have to write a custom generator and Iā€™m not certain whether the generator Iā€™ve written is only going to produce conforming values by itselfā€¦)#2016-07-2123:14Alex Miller (Clojure team)It does not trust and always re-checks the custom gen#2016-07-2123:15Alex Miller (Clojure team)It's not filtering though - it will error if the gen produces a bad value#2016-07-2123:19glv@alexmiller: in http://clojure.org/reference/reader#_maps, an example of the #:: behavior would be helpful. I think itā€™s accurate as it is, but not particularly clear.#2016-07-2123:21seancorfieldThanks @alexmiller thatā€™s good to know. I couldnā€™t produce a bad value in 10,000,000 generations so I think Iā€™ll trust it as working for now šŸ™‚#2016-07-2123:21Alex Miller (Clojure team)@glv Will consider#2016-07-2202:20lvhWhatā€™s the preferred way to check if a coll has only unique elements?#2016-07-2202:21lvhJust a pred? Iā€™d like the generators to be efficient.#2016-07-2202:35codonnell@lvh: I think coll-of and every use clojure.test.check.generators/vector-distinct under the hood#2016-07-2202:36lvhah; that solves part of the problem#2016-07-2202:36codonnellrather, they use vector-distinct if they get :distinct true#2016-07-2202:37lvhah! I missed that opt#2016-07-2202:37lvhdoes coll-of take the same opts as every?#2016-07-2202:38codonnellyes#2016-07-2202:39codonnellthis is the entire body of the coll-of macro: backtick before ( ---> (every ~pred ::conform-all true #2016-07-2202:40codonnelldamn, how do you get a backtick in a code block -_-#2016-07-2203:12lvhah, awesome#2016-07-2203:12lvhI should really just read the source of that ns#2016-07-2203:33lvhI ran some test.check specs, but even with only 100 samples Iā€™m getting actual: java.lang.OutOfMemoryError: GC overhead limit exceeded. I guess recursive data structures can grow big.#2016-07-2203:40lvhIs there something extra weird that goes on when you test against (for-all [ā€¦] true)? I was expecting that to trivially pass.#2016-07-2203:40lvh(that seems like it would be the normal behavior once your code works.)#2016-07-2203:46glv@lvh is a generator in control of the size (depth or breadth) of the generated structure? The current integer generators in spec grow very quickly. By the 20th test, you're probably in 8-digit range.#2016-07-2204:06madstapIs there a way to say that if there are any numbers, the string also needs to be there, but if there aren't, it's optional? (s/def ::xs (s/cat :str (s/? string?) :nums (s/* number?) :key keyword?))#2016-07-2204:31madstap
;; so this should be valid
  (s/valid? ::xs ["s" 2 3 4 :k])

  ;; And this
  (s/valid? ::xs ["s" :k])

  ;; but not this
  (s/valid? ::xs [2 3 4 :k])
#2016-07-2205:24bfabry@madstap: (s/or :regex1 (s/cat ...) :refex2 (s/cat ...)) seems simplest to me#2016-07-2205:45Alex Miller (Clojure team)
(s/def ::xs
  (s/cat :pre (s/alt :opt1 (s/cat :str string? :nums (s/* number?))
                     :opt2 (s/? string?))
         :key keyword?))
#2016-07-2207:52mpenetWhat do you think makes more sense for a lib with optional specs: ship with specs in a separate namespace or have specs in a separate repo even. former requires macro hackery to allow to run/use with clj1.9- latter is just a dependency.#2016-07-2213:00mpenetany idea why this fails when trying to validate with it:
(s/def :foo  (s/fspec :args (s/cat :err #(instance? ExceptionInfo %))
           :ret any?))
#2016-07-2213:00mpenet
(s/valid? ::foo (fn [x] 1)) -> ExceptionInfo Unable to construct gen at: [:err] for: (instance? clojure.lang.ExceptionInfo %)  clojure.core/ex-info (core.clj:4724)
#2016-07-2213:03mpenetoh I see on the guide#2016-07-2213:04codonnell@mpenet: regarding your earlier question, can you not use the port someone made of spec to clojure 1.8?#2016-07-2213:04mpenetit was more of a general question when publishing oss#2016-07-2213:05codonnelloops, I missed your "specs in a separate namespace" option#2016-07-2213:06mpenetabout my recent issue, I wonder why gen is coupled like this to preds#2016-07-2213:07mpenetI am mostly interested in instrumentation in that case, yet I have to specify a generator apparently#2016-07-2213:08codonnellI would guess that valid? checks validity by generating some arguments using the :args generator and then checking that the return values match your :ret and :fn predicates#2016-07-2213:12mpenetapparently#2016-07-2213:17mpenetso instrumentation works without having to specify more here. good#2016-07-2213:19codonnellthat's good to know#2016-07-2213:20mpenetwell actually no it doesn't#2016-07-2213:20mpenetdamnit#2016-07-2213:22codonnellit worked for me here:
=> (defn foo [& exs] (map str exs))
=> (s/fdef foo :args (s/cat :err #(instance? clojure.lang.ExceptionInfo %)) :ret any?)
=> (stest/instrument `foo)
=> (foo 1 2 3)
ExceptionInfo Call to #'user/foo did not conform to spec:
In: [0] val: 1 fails at: [:args :err] predicate: (instance? clojure.lang.ExceptionInfo %)
:clojure.spec/args  (1 2 3)
:clojure.spec/failure  :instrument
:clojure.spec.test/caller  {:file "form-init8739029786060363099.clj", :line 990, :var-scope user/eval90084}
  clojure.core/ex-info (core.clj:4724)
=> (foo (ex-info "test" {}))
("clojure.lang.ExceptionInfo: test {}")
#2016-07-2213:23codonnellhow did it fail for you?#2016-07-2213:23mpenetnot the same in my case, it's a function that takes another fn as argument, the fn passed fails to gen#2016-07-2213:24mpenetso no s/fdef but s/fspec#2016-07-2213:24codonnelloh, I see#2016-07-2213:24mpenet
(s/def ::alia.execute-async/error
  (s/fspec :args (s/cat :err (instance-pred clojure.lang.ExceptionInfo))
           :ret any?))
#2016-07-2213:25mpenet
(s/valid? ::alia.execute-async/error (fn [x] :meh))
#2016-07-2213:25codonnellit tries to call valid? on the function returned rather than doing something like propogate the instrumentation to the returned function#2016-07-2213:26mpenetsame when it's on the test suite via instrumentation of the parent fn#2016-07-2213:26mpenethttps://github.com/mpenet/alia/blob/feature/specs/modules/alia-spec/src/qbits/alia/spec.clj#L311-L315#2016-07-2213:29mpenetI understand it's necessary when running valid? on the single spec, but via instrumentation not really#2016-07-2213:30codonnellI tend to agree.#2016-07-2213:30mpenetit probably is slow too#2016-07-2213:31mpenet@alexmiller: any thoughts on this?#2016-07-2213:35mpenetIn my mind instrumentation should "only" be pre/post assertions based on specs. no gen involved (same as plumatic/schema actually)#2016-07-2214:03lvh@glv: Yes, I suppose so.#2016-07-2214:04lvhHereā€™s what I have now that blows up at 50: https://gist.github.com/lvh/243576d4e825d3792d4cd1bb8d8c39d7#2016-07-2214:04lvhat line 36, youā€™ll see ::properties is a map-of keywords to ::schemas#2016-07-2214:04lvh(so recursive there)#2016-07-2214:04lvhitā€™ll be worse when I add arrays, because those are also recursive json schemas#2016-07-2214:09glvOh ā€¦ no, it doesnā€™t look like a generated value is in controlling the depth ā€¦ in fact, nothing is controlling it except chance.#2016-07-2214:10glvSo on line 36, thereā€™s a 1-in-7 chance that the generator will recur and add another level to the structure ā€¦ but at line 32, thereā€™s a 50% chance. Itā€™s easy to imagine the process rolling the dice just right so that the structure grows until it fills the JVMā€™s memory allocation, and that seems to be happening.#2016-07-2214:16glvThe problem for writing the specs is this: youā€™ve accurately captured the semantics for the purposes of verification, but when generating for tests, you want to add some additional constraints. I ran into this when testing a 2D grid ā€¦ in production code I donā€™t want to arbitrarily restrict the allowable sizes, but in my specs, I was ending up with grids that had many millions of cells, which was slow, failure-prone (because I also got out-of-memory errors) and also pointless (because a bunch of tests with 25x25 grids will catch any problems ā€” the size of the grid isnā€™t really the important factor).#2016-07-2214:16glvSo I ended up with this spec:
;; restrict grid sizes for testing, but allow larger grids in production.
(s/def ::grid-dimen     (s/with-gen (s/and integer? #(>= % 2))
                                    #(s/gen (s/int-in 2 25))))
#2016-07-2214:17glvIt doesnā€™t enforce an upper bound for validation, but it does for generation.#2016-07-2214:18glvYou might need something similar, but rather than limiting a size (because you donā€™t have one) you can alter the probabilities, so that the generator for ::additional-properties only chooses to recur 10% of the time, say, instead of 50%.#2016-07-2214:52Alex Miller (Clojure team)when running check etc you can supply generator overrides#2016-07-2214:53Alex Miller (Clojure team)same for exercise, gen, and instrument#2016-07-2214:54Alex Miller (Clojure team)this allows you to override the generator with a more specific one at test time#2016-07-2214:54Alex Miller (Clojure team)also see s/*recursion-limit*#2016-07-2214:56mpenetI saw this. But shouldn't it be possible to avoid generators usage for instrumentation?#2016-07-2214:57mpenetsince it's all predicates, just wrap args/ret#2016-07-2215:00Alex Miller (Clojure team)you can do so with the replace/stub functionality in instrument#2016-07-2215:00Alex Miller (Clojure team)or maybe Iā€™m not understanding your question#2016-07-2215:01mpenetI don't know if you read my issue earlier#2016-07-2215:01Alex Miller (Clojure team)the fspec thing?#2016-07-2215:01mpenetyes#2016-07-2215:02mpenetI imagined that instrumentation would work without having the need to stub anything since it's all specified, I didn't expect it to run "gen" on arguments (hundreds of calls)#2016-07-2215:03Alex Miller (Clojure team)there is still some unfinished work in this area#2016-07-2215:03mpenetoki, glad to hear that, so in the final design "gen" wouldn't be required by instrumentation code#2016-07-2215:03Alex Miller (Clojure team)this is same as http://dev.clojure.org/jira/browse/CLJ-1936#2016-07-2215:04mpenetah indeed, I tried to look at jiras, I missed that one I guess#2016-07-2215:19glv@lvh: just to test my hypothesis, try changing ::additional-properties to this and see if that helps:
(s/def ::additional-properties
  (s/with-gen
    (s/or
      :implicit-additional-properties boolean?
      :explicit-additional-properties ::schema)
    #(sg/fmap (fn [d10]
                (sg/generate (s/gen (if (= d10 1)
                                        ::schema
                                        boolean?))))
              (sg/choose 1 10))))
#2016-07-2215:21glvYouā€™ll get a nested schema as the generated value only 10% of the time.#2016-07-2215:21gfredericksalexmiller: ugh that "open intervals" email makes me wish I'd made the bounds opts on those numeric generators named #{:< :<= :> :>=} :/#2016-07-2215:21glv(sg is aliased to clojure.spec.gen)#2016-07-2215:22gfredericksalexmiller: I'd consider deprecating the old opts and adding those four if you think that'd be useful for spec#2016-07-2215:24Alex Miller (Clojure team)I donā€™t think anything needs to change with it, itā€™s fine#2016-07-2215:29gfredericksindependently I also halfway regret how :NaN? and :infinite? interact with :min and :max#2016-07-2215:30gfredericksor don't interact rather#2016-07-2215:30gfredericksbut that's harder to change without technically breaking#2016-07-2216:10rickmoynihanHmmm... any tips for debugging stackoverflow exceptions in specs?#2016-07-2216:16rickmoynihanalso is it possible to reset the spec registry?#2016-07-2216:16rickmoynihanpretty sure I've heard people ask about this before#2016-07-2216:32mpenetMaybe (reset! #'clojure.spec/registry-ref (atom {}))#2016-07-2216:32mpenet(Untested)#2016-07-2216:50rickmoynihanI've probably got a mistake in my specs...#2016-07-2218:39rickmoynihanI had a typo along the lines of by mistake (s/def foo ::foo)#2016-07-2218:40rickmoynihanIs it possible to call an s/fdef'd function and validate its :ret spec?#2016-07-2218:40rickmoynihanI saw the guide said it wasn't supported because that's for testing#2016-07-2218:41Alex Miller (Clojure team)the only place thatā€™s checked by spec is during check#2016-07-2218:41Alex Miller (Clojure team)you can of course obtain and check it yourself#2016-07-2218:42Alex Miller (Clojure team)something like (s/valid? (:ret (s/get-spec 'user/foo)) ret)#2016-07-2218:42rickmoynihanahh thanks#2016-07-2218:42Alex Miller (Clojure team)or s/assert#2016-07-2218:43rickmoynihannice - I'd missed that one#2016-07-2218:53xcthulhu@alexmiller: As I opined on reddit, I don't see any deep reason for why you can't just have in clojure.spec a schema transpiler:
(spec/def ::json-api
     (spec/schema
          {:id       uuid?
           :text     str?
           :rating   (int-in 0 5)}))
This would make migrating a lot less annoying.
#2016-07-2218:54Alex Miller (Clojure team)Clojure is not going to provide a schema transpiler#2016-07-2218:54xcthulhuWhy not?#2016-07-2218:54xcthulhuNIH?#2016-07-2218:54Alex Miller (Clojure team)b/c we have other things to do#2016-07-2218:54Alex Miller (Clojure team)people are more than welcome to make one#2016-07-2218:55Alex Miller (Clojure team)would we also create a truss transpiler?#2016-07-2218:55Alex Miller (Clojure team)and a herbert transpiler?#2016-07-2218:55Alex Miller (Clojure team)etc?#2016-07-2218:56Alex Miller (Clojure team)things in core are forever#2016-07-2218:56xcthulhuI suppose. IDK, clojure.spec.gen/fmap is in there which is awkward.#2016-07-2218:57xcthulhuI don't really grok the logic behind what gets in and what doesn't#2016-07-2218:57Alex Miller (Clojure team)why?#2016-07-2218:57Alex Miller (Clojure team)gen is just a dynamically loaded skin around test.check#2016-07-2218:58xcthulhuWell, so there's tiny little monad in a framework where everyone did a great job of dodging that sort of thing.#2016-07-2218:58Alex Miller (Clojure team)you can ignore itā€™s monadic nature if you like#2016-07-2218:58Alex Miller (Clojure team)it does what it does#2016-07-2218:59xcthulhuAll I'm saying is that great pains were made to wrap test.check, but the lesser pain of wrapping schema type syntax was ignored for some reason.#2016-07-2219:01xcthulhuIt's your product, everyone picks and chooses which wheels they want to reinvent I suppose.#2016-07-2219:05Alex Miller (Clojure team)test.check is a Clojure contrib library, following the same contributor agreement, license, and dev methodology as Clojure itself (to make things like this possible)#2016-07-2219:06Alex Miller (Clojure team)Schema is not (and I mean nothing negative towards Schema in this regard, they are just much different from a core perspective)#2016-07-2219:07xcthulhuOkay, so I guess we just need a clojure.contrib.spec.utils library.#2016-07-2219:07Alex Miller (Clojure team)spec was designed to solve a set of problems, not to replace Schema (even though it solves an overlapping set of problems)#2016-07-2219:08xcthulhuWith schema and a dynamically loaded test.check/let#2016-07-2219:08xcthulhuSince that would be super nice#2016-07-2219:08Alex Miller (Clojure team)let is tricky as itā€™s actually a macro#2016-07-2219:08Alex Miller (Clojure team)Stu did the work on gen, but I suspect thatā€™s the only reason itā€™s not there#2016-07-2219:09Alex Miller (Clojure team)The Clojure contrib libs are primarily standalone libs without external deps (there are exceptions, but thatā€™s the ideal). There is not going to be a contrib lib related to schema. But there is nothing stopping someone from creating a lib to do what you suggest.#2016-07-2219:10Alex Miller (Clojure team)if we think something is fit for spec, it will go into spec, not into a lib#2016-07-2219:12xcthulhuLooking here, it looks like you guys rope in bind: https://github.com/clojure/clojure/blob/master/src/clj/clojure/spec/gen.clj#L92#2016-07-2219:12xcthulhuI'll go try and write the little let macro for you#2016-07-2219:13Alex Miller (Clojure team)bind is a function#2016-07-2219:13xcthulhuI know#2016-07-2219:13xcthulhuBut let is just like the do notation in Haskell, wrapping bind#2016-07-2219:13xcthulhuI'll do it for you this weekend.#2016-07-2219:13xcthulhuOkay?#2016-07-2219:14Alex Miller (Clojure team)I think let would basically have to be copied into gen rather than dynamically loaded#2016-07-2219:14xcthulhuYup.#2016-07-2219:14xcthulhuSo I'll write it and post it here for you.#2016-07-2219:14Alex Miller (Clojure team)you can file a jira if you like, but no guarantees for anything#2016-07-2219:15xcthulhuNah, I'll just post it here in a gist or whatever unless you need me to sign something#2016-07-2219:15xcthulhuIt's like 15 lines tops#2016-07-2219:15Alex Miller (Clojure team)we donā€™t take contributions via slack :)#2016-07-2219:16Alex Miller (Clojure team)sign the CA, file a jira, supply a patch#2016-07-2219:17xcthulhuOh man red tape my favorite#2016-07-2219:17Alex Miller (Clojure team)http://insideclojure.org/2015/05/01/contributing-clojure/#2016-07-2219:18xcthulhuOkay, I'll post it here for people to playtest and then do the patch, since that's a lot of tedium for 15 lines of code#2016-07-2220:12mpenetSeems like fmap appears twice in lazy-combinators call fyi#2016-07-2220:22glvJust for fun, Iā€™ve been prototyping a little library to generate EBNF-ish syntax diagrams from specs. Delving into the spec internal representation has been ā€¦ eeeenteresting. šŸ˜•#2016-07-2220:31xcthulhu@mpenet: Sounds like someone else is going to be signing something, issuing a JIRA ticket with a simple patch and waiting days and days for a fix#2016-07-2220:34xcthulhu@alexmiller: https://gist.github.com/ef92987c8160a796b62c95e207ee0ad4 (fixed a typo)#2016-07-2220:35xcthulhuIt's admittedly 16 lines....#2016-07-2220:42bbrinckAny ideas as to why clojure.test/check would return the empty list?
user=>  (require '[clojure.spec :as s])
nil
user=> (s/fdef my+
  #_=>         :args (s/tuple number? number?)
  #_=>         :ret symbol?
  #_=>         :fn #(= (:ret %) (apply + (:args %))))
user/my+
user=> (require '[clojure.spec.test :as stest])
nil
user=> (stest/check 'my+)
()
user=>
Am I missing a step here?
#2016-07-2220:44bbrinckI have the following function defined (defn my+ [x y] (+ x y))#2016-07-2220:45glvUse a backquote instead:
(stest/check `my+)
#2016-07-2220:45glv(That yields a fully namespace-qualified symbol.)#2016-07-2220:45bbrinckah, thank you!#2016-07-2220:52Alex Miller (Clojure team)@mpenet thx, Iā€™ll try to get that fixed the next time someone is paying attention#2016-07-2221:18xcthulhu> Just for fun, Iā€™ve been prototyping a little library to generate EBNF-ish syntax diagrams from specs. Delving into the spec internal representation has been ā€¦ eeeenteresting. @glv: This is cool. Does it emit EBNF that instaparse can digest? I would absolutely love if I could shake SQL tables out of specs. But that's super hard and probably will never happen.#2016-07-2221:22Alex Miller (Clojure team)why not model your data and shake specs and tables out of that?#2016-07-2221:25glvNo, but itā€™s super basic right now. Iā€™m more concerned with deciphering the structures and building code that can navigate them. Itā€™ll be easy to play with the details of what gets generated later. (Plus, Alex said that those internals are still likely to change.)#2016-07-2223:26bsimacan I spec tagged literals?#2016-07-2223:26bsimaI'm trying to spec a config file, like this: https://github.com/juxt/aero#profile#2016-07-2223:38bsimanvm, the problem I'm having is that aero does its own eval of the config file, so I can't really get access to the data before the literals are processed (even if I do spec the config file before I pass it to aero, I don't have aero's data readers loaded, so #profile and such is unrecognized). The solution is to spec the data after aero processes it and replaces the tagged literals#2016-07-2300:06codonnellI'm using clojure.spec to validate responses from a particularly poorly written API, and explain-data is seriously my new best friend. :+1:#2016-07-2300:42lvhglv: Thanks! That did appear to make things better, but this is still pegging my CPU so Iā€™ll work at it a little more šŸ™‚#2016-07-2300:59glvYeah ā€¦ in my experiments it helped some, but not dramatically. #2016-07-2301:05lvhpart of the problem is that json schema is naturally extremely recursive#2016-07-2301:05lvhbut the ones that actually exist are wide, not deep#2016-07-2301:05lvhthe problem is that that implies statefulness in the recursion, IIUC#2016-07-2301:05lvhI want to recurse a lot high in the tree, less so at the bottom of he tree#2016-07-2301:07lvhIs there a shorthand for saying "no other keysā€ in a way thatā€™s automatically generator efficient? I realize that these are normally intended to be open for extension, but this is an existing spec, and it is not open for extension#2016-07-2301:24glvI would expect test/check to peg your CPU, though. For this kind of thing it's definitely CPU-bound. If you're not getting out-of-memory errors anymore, that's an improvement.#2016-07-2301:43Alex Miller (Clojure team)You can override generators at paths#2016-07-2301:43Alex Miller (Clojure team)Could let you use different gens at different levels#2016-07-2301:44glvWow, I've totally missed that. Doc pointer?
#2016-07-2301:53Alex Miller (Clojure team)Was added recently, not sure there are any docs other than the api#2016-07-2301:54Alex Miller (Clojure team)Any place that takes a gen override map can take a vector of keys#2016-07-2301:54glvAh, interesting. Will check into that. Thanks!#2016-07-2301:54Alex Miller (Clojure team)Although I don't remember now if those are spec keys or path keys or both#2016-07-2301:55glvSome help you are. ;-)#2016-07-2301:55Alex Miller (Clojure team)Yeah#2016-07-2302:16Alex Miller (Clojure team)It has to be the path keys#2016-07-2321:39dominicm@bsima: You can access the reader literals in aero 0.4.0, but spec'ing after is okay too#2016-07-2322:56nhaIs it possible to coerce the content of an edn file ? My use case: coerce a configuration map (ex. coming from environ where all values are strings to be able to override them from env vars). It seems that edn/read-string does not accept namespaced keywords. Using load-file seem to lead to problems when trying to aot.#2016-07-2401:11Alex Miller (Clojure team)Namespaced keywords should be fine in edn#2016-07-2401:11Alex Miller (Clojure team)Autoresolved keywords are not part of edn though (as there is no namespace context)#2016-07-2401:12Alex Miller (Clojure team)Do you have an example?#2016-07-2401:34lvhWhat exactly does calling spec with a map do? map-spec-impl suggests I can do that, but specā€™s docstring only mentions predicates and regex specs; I guess a map can behave like a predicate since itā€™s an IFn?#2016-07-2401:56lvhIs there an easy way to get a keys thatā€™s not open for extension? Iā€™m digging into map-spec-impl and just extracting the generator bit isnā€™t super trivial.#2016-07-2402:01Alex Miller (Clojure team)in short, no#2016-07-2402:01Alex Miller (Clojure team)you can s/and with a restriction on the key set#2016-07-2402:03lvhyeah; I got that part; trying to fix the generator though#2016-07-2402:03lvhI guess I can fmap over the gen I get from keys, and then dissoc the noise it comes up with I donā€™t want#2016-07-2402:04lvh(I understand the general argument for having extra keys; it just doesnā€™t make sense for what Iā€™m doing)#2016-07-2402:07lvh@alexmiller: Thanks! Much appreciated šŸ™‚#2016-07-2402:41Alex Miller (Clojure team)yeah, def leverage the existing specs as much as possible for gen#2016-07-2416:41lvhWhen using with-gen, is there some kind of automatic spec verification, or are generators assumed to be correct#2016-07-2416:48lvhMy excl-keys macro ostensibly works, but merging it doesnā€™t seem to.
(defmacro excl-keys
  "Like [[s/keys]], but closed for extension."
  [& {:keys [req-un opt-un req opt] :as keys-spec}]
  (let [bare-un-keys (map (comp keyword name) (concat req-un opt-un))
        all-keys (set (concat bare-un-keys req opt))]
    `(let [ks# (s/keys 
(s/def ::a (excl-keys :opt-un [::b] :req-un [::c]))
(s/def ::b string?)
(s/def ::c keyword?)
(gen/sample (s/gen ::a))
(s/def ::x (excl-keys :opt-un [::y] :req-un [::z]))
(s/def ::y integer?)
(s/def ::z rational?)
(gen/sample (s/gen ::x))
(s/def ::mix (s/merge ::a ::x))
(gen/sample (s/gen ::mix))
Samples for ::a, ::x work fine, merging less so. I guess Iā€™ll dive into how s/merge builds generators šŸ™‚
#2016-07-2418:45xcthulhuHey guys, I've been working on a patch to port test.check/let to clojure.spec.gen#2016-07-2418:46xcthulhuSince we all have fmap blues from time to time.#2016-07-2418:46xcthulhuHere's the unit-test vector I want to push to clojure.test-clojure.spec:#2016-07-2418:46xcthulhu
(deftest gen-let-macro
  (s/def ::a (s/spec keyword?))
  (s/def ::b (s/int-in 7 42))
  (let [t (s/tuple keyword? string?)
        m (s/keys :req [::a ::b])]
     (are [spec generator]
        (s/valid? spec (gen/generate generator))
        t (gen/let [x :abc, y "efg"] [x y])
        t (gen/let [x ::a, y string?] [x y])
        t (gen/let [[_ y] t] [::a y])
        t (gen/let [] [keyword? string?])
        t (gen/let [] t)
        t (gen/let [[x y] (gen/let [] t)] [x y])
        m (gen/let [{a ::a, b ::b} m] {::a a, ::b b})
        m (gen/let [a keyword?] {::a a, ::b ::b})
        m (gen/let [] {::a keyword?, ::b 7})
        m (gen/let [] {::a keyword?, ::b #{7 8 9}})
        int? (gen/let [i 'clojure.test.check.generators/int] i)
        #{'abc} (gen/let [x 'abc] x))))
#2016-07-2418:47xcthulhuThat test vector should give a pretty good idea of how gen/let would work.#2016-07-2418:47xcthulhuMy idea is that it's really annoying having to coerce specs, predicates and constants to generators so it just does it automagically#2016-07-2418:49xcthulhuI'll wait for @alexmiller to tell me if he likes it or not before I attempt a patch#2016-07-2419:25mike_ananevhi there! how to specify :ret value for fdef if function returns void?#2016-07-2419:26xcthulhunil?#2016-07-2419:28mike_ananev:ret nil? or :ret #(nil? %) ?#2016-07-2419:30xcthulhuI think :ret nil? but I'm not 100%#2016-07-2419:31xcthulhu#(nil? %) is probably wrong#2016-07-2420:11xcthulhu@alexmiller: Since you're a category theory fan, I could generalize let from generators to specs if we model specs as having type (a -> Bool, Gen a) (Ć” la Haskell)#2016-07-2420:24Alex Miller (Clojure team)@mike1452 either will work, I'd use the shorter one#2016-07-2420:25Alex Miller (Clojure team)@lvh re with-gen, the generator is not trusted and all generated samples will be verified by checking the spec as well#2016-07-2420:28Alex Miller (Clojure team)@mike1452: sorry read further back - functions are never void in Clojure, only comes up in some interop cases#2016-07-2503:19lvhI guess thereā€™s no way to override that? The issue Iā€™m running into is that merge assumes the key specs it gets are open for extension; so keys-excl merged with anything else fails to produce anything, because the keys-excl part of the spec that checks that the map only contains specific keys will fail on all the keys being added by anything thatā€™s merged in#2016-07-2507:35mike_ananev@alexmiller: last call in my clojure function is java (interop) fn call which returns void. So in clojure if java fn returns void we see nil, if i understand correctly.#2016-07-2511:41Alex Miller (Clojure team)Yep#2016-07-2512:56xcthulhu@lvh: Instead of merge, have you tried to make an intersection operation?#2016-07-2512:57xcthulhuSometimes the intersection of two keys-excl won't work either of course, since you have certain mandatory keys.#2016-07-2513:01mpenet"closed" key sets seems like a common request, hopefully Rich reconsiders his choice here. I get that he doesn't want to make it the default, but simply providing a way to spec this can't hurt#2016-07-2513:22Alex Miller (Clojure team)There are no plans to revisit that afaik#2016-07-2513:24Alex Miller (Clojure team)@lvh you might try going down the path of treating your map as a coll-of tuple entries instead (make sure to use :into {}) #2016-07-2513:25Alex Miller (Clojure team)Then merge that with s/keys to pick up the attribute checks#2016-07-2513:28mpenetIt's super easy to please everybody with a system with "closed" keysets by default (you can just add a k/v spec with any? if you want to "open" the map), but that apparently is out of the question. I personally do not understand the logic here. Never saw anybody complain about the "strictness" of prismatic/Schema for these things for instance.#2016-07-2513:31mpenetthen again, I don't consider the openness of records a good thing either#2016-07-2514:02lvh@xcthulhu: Iā€™m not sure I understand. As in, remove keys?#2016-07-2514:03lvhI mean, I guess I could, but that seems very gauche. What would that look like? I have a base spec that everything conforms to, and some stuff thatā€™s specific per type (i.e. multi-spec)#2016-07-2514:30xcthulhu@lvh:Yeah, you would remove keys. If a map-spec isn't open under extension, it may be open under specialization.#2016-07-2514:43xcthulhuSo provided that your map-specs s1 and s2 that have the form (keys-excl :opt [...] :opt-un [...]) then you could have
(defn intersection
  [s1 s2]
  (spec/with-gen (spec/and s1 s2)
    #(fmap (fn [[a b]] (select-keys a (keys b)))
           (spec/gen (spec/tuple s1 s2)))))
#2016-07-2514:44xcthulhuAnd you'd have the rule:
(valid? (intersection s1 s2) m)
; if and only if
(and (valid? s1 m) (valid? s2 m))
#2016-07-2516:38ghadi@mpenet: search slack history for @richhickey expounding on closed keysets making brittle systems#2016-07-2517:54mpenetIt's kind of in the same class as static vs dynamic lang/systems imho: it depends, sometimes exactitude is necessary. My point was that allowing both would do no harm, it would be optional. #2016-07-2518:02mpenetI guess that ship has sailed anyway, and likely not open for debate as usual.#2016-07-2518:41ghadithat's a pessimistic attitude#2016-07-2518:41ghadiit's not open for debate (IMHO) because it's a bad idea, and the core team can justify why#2016-07-2518:43ghadiim guessing no one went back to the original discussion. This comes up often enough that it's probably worth documenting why it's not a good idea#2016-07-2518:48ghadihttps://clojurians-log.clojureverse.org/clojure-spec/2016-06-09.html#2016-07-2520:41glvThatā€™s a pretty good discussion, and includes a not-too-bad way to accomplish it if you really feel you must.#2016-07-2603:50wildermuthnAnyone have experience using clojure.spec to spec out macros for use in clojurescript?#2016-07-2603:51wildermuthnI get odd errors relating to clojure.spec not being found when I run clojure 1.9.#2016-07-2603:51wildermuthnMight just be my setup.#2016-07-2604:08glvUse the cljs.spec namespace in Clojurescript. #2016-07-2616:29nhaI have been trying to use clojure.spec to coerce edn (config) files. EDN prevents autonamespacing keywords ( I get that now, thanks alexmiller ), but allow custom #readers. Clojure (well read-string and load-file) allow autonamespacing but prevent custom readers. So it seems that my only option is to explicitly have the keyword namespace every time in my configuration file, is that correct ? I am asking because I find it a bit cumbersome/repetitive to have to have these in my file.#2016-07-2616:35Alex Miller (Clojure team)Yes, you'll need full namespaces in edn config files#2016-07-2616:36Alex Miller (Clojure team)You could also write specs on unqualified keys with req-un and opt-un#2016-07-2616:37nhaAh I see - not super familiar with spec yet. Will look into those keys, thanks šŸ™‚#2016-07-2616:37Alex Miller (Clojure team)That's in s/keys#2016-07-2616:39Alex Miller (Clojure team)Either the normal core reader fns or clojure.edn reader fns should be able to read tagged literals though with ##2016-07-2616:40Alex Miller (Clojure team)You just need to bind around the call to set up the readers#2016-07-2616:41nhaAh I see - I did not knew that. So all is fine šŸ™‚ I will go with clojure.spec (s/keys :req-un [...]) for now.#2016-07-2702:32lvhHm, I donā€™t really understand how to use retag#2016-07-2702:33lvhI mean, I can use it with a keyword, but now I have something a bit more complex: a schema is either a reference (`{:$ref ā€œsome-string"}`) or a schema with a type#2016-07-2702:34lvhI was hoping to teach the multimethod about that, since s/or doesnā€™t seem to work#2016-07-2702:35lvh(when I s/or, it complains that it canā€™t find an appropriate method, which is true I guess; I wasnā€™t expecting it to even try that branch)#2016-07-2702:41lvhI tried doing with with s/or this way:
(s/def ::schema
  (s/or ::reference
        (fn [m] (contains? m :$ref))
        ::direct-schema
        (fn [m] (contains? m :type))))
#2016-07-2702:41lvhWhich doesnā€™t work because spec doesnā€™t know how to generate schemas from that pred.#2016-07-2702:43lvhI can (s/and map? ā€¦) but that doesnā€™t really make it easier to generate maps#2016-07-2702:49lvhI guess I can just use map? and it can later specify with keys ā€” that appears to sorta work šŸ™‚#2016-07-2715:39xcthulhu
(s/def ::$ref uuid?)
(s/def ::type keyword?)
(s/def ::schema (s/or ::reference (s/keys :req-un [::$ref]) ::direct-schema (s/keys :req-un [::type])))
#2016-07-2715:39xcthulhu@lvh: I think you are supposed to do it in the style above.#2016-07-2715:53michaeldrogalisHello. Is it possible to assert properties about keys and values in a map using s/keys without using the registry?#2016-07-2716:12xcthulhuNot that I know of.#2016-07-2716:12codonnell@michaeldrogalis: You can put keys into s/keys that aren't in the registry, and the spec will just check for their existence. It is not possible to provide inline specs to validate map values.#2016-07-2716:13michaeldrogalis@codonnell: Thanks for the confirmation.#2016-07-2716:14codonnellno problem#2016-07-2716:19Alex Miller (Clojure team)http://blog.cognitect.com/blog/2016/7/26/clojure-spec-screencast-testing#2016-07-2716:19Alex Miller (Clojure team)new screencast about check and instrument#2016-07-2716:34lvhAwesome, thank you!#2016-07-2716:39michaeldrogalis@alexmiller Neato šŸ™‚#2016-07-2717:51jonathanjAre there any thoughts on using spec for something like web form validation? Specifically having some way to generate user-readable messages from failed predicates, i.e. more fine-grained than "Name is wrong"#2016-07-2717:51jonathanj(I don't have much experience with spec, apologies if this is either a really frequent question or a really stupid one.)#2016-07-2718:15dominicm@jonathanj: IMO, the output would be okay for parsing for a web form. However, spec doesn't play too well with cross validation, e.g. "is this email taken in the database?"#2016-07-2718:16dominicmI also found specifying equal fields difficult, with meaningful output, so you'd have a few special cases to handle outside of s/keys#2016-07-2718:25jonathanjAre you referring to the error data when you say output, @dominicm? I don't see how the error data would be suitable for a user-facing form, you need a layer of translation since arbitrary predicate names are unlikely to be of value to users in most circumstances. #2016-07-2718:26dominicmMy apologies, suitable for translating to user information.#2016-07-2718:28jonathanj@dominicm: what about involving multiple values in a validation? We have a case where the SSN needs to be validated against the gender of the selected title (don't ask), would this be written as a separate spec that combines the others, with and perhaps?#2016-07-2718:31jonathanjAnd perhaps more interestingly, would the error data allow me to determine all the participants in the failing validation?#2016-07-2718:31dominicmUser forms have plenty of awkward cases, domain logic runs rampant here. I think and would be appropriate yes. You want to ensure the SSN, gender and selected title are all valid before doing the overall validation I assume? That would be s/and. I don't know the api too well though, my interest dwindled when I tried to use spec for web forms and found limitations in spec.#2016-07-2718:32dominicmI did find that the output error data was very thorough though, and you could derive a lot of information from it.#2016-07-2718:33jonathanj@dominicm: You're quite right about awkward cases for user forms, did you go another route in the end?#2016-07-2718:34dominicmI started writing my own validation library.#2016-07-2718:34jonathanjIs the code public?#2016-07-2718:34dominicmNot yet šŸ˜ž#2016-07-2718:34dominicmI should open source my attempts.#2016-07-2718:34dominicmhttps://github.com/logaan/vlad I can highly recommend vlad though.#2016-07-2718:35jonathanjI've already written two (JS) validation libraries and they are plagued with reinventing existing predicates. I was hoping to find a better way of layering this on something else. #2016-07-2718:36dominicmvlad has a lot of useful predicates built in. https://github.com/weavejester/valip/blob/master/src/valip/predicates.clj I sometimes take advantage.#2016-07-2718:37jonathanjMy experience was that a Boolean is not a good enough result for explaining errors to users, but having to decompose the data after the predicate failed was too much duplication of effort #2016-07-2718:38dominicmNope, you must attach additional semantics to it. That's where vlad comes in.#2016-07-2718:39dominicmI just realised that the README does not do it justice.#2016-07-2718:39dominicmhttps://github.com/logaan/vlad/blob/master/src/vlad/core.cljc#L117 this is the definition of the present? checker#2016-07-2718:40jonathanjSkimming the readme, vlad seems worth further investigation, thanks!#2016-07-2718:41jonathanj@dominicm: Nice, reasonably succinct. #2016-07-2718:43dominicmIt's the result of much comparison, I'm glad you can benefit from my research šŸ™‚#2016-07-2719:32bsimadoes anyone know what this error means?#2016-07-2719:32bsimahttps://gist.github.com/bsima/042554a1a61ce18adad0ebaae60a833f#2016-07-2720:02seancorfield@bsima: Which version of Clojure? That works for me on Alpha 10.#2016-07-2720:02bsima1.9-alpha10#2016-07-2720:02bsimaĀÆ\(惄)/ĀÆ#2016-07-2720:05bsimaI think a lein clean fixed it#2016-07-2720:12seancorfieldlein clean solves so many evils šŸ˜ˆ#2016-07-2720:52kendall.buchanan@gfredericks: Just following up real quick on that issue last week where explicit loading of test.check and clojure.spec.test/check lead to MultiFn errors. You mentioned master seemed to fix it. Any thoughts on when the next release will be cut?#2016-07-2721:11gfredericksProbably depends on whether it's still painful for @alexmiller #2016-07-2721:11Alex Miller (Clojure team)It is#2016-07-2721:21martinklepsch@dominicm: interesting, didn't know about vlad, looks cool#2016-07-2721:32gfredericks@kendall.buchanan: I'll prioritize getting a good release together ASAP#2016-07-2721:33kendall.buchanan@gfredericks: Thank you. I donā€™t mean to put pressure on you, or anything, but I do sure appreciate it.#2016-07-2721:34gfredericksNo worries#2016-07-2721:48Alex Miller (Clojure team)Iā€™m willing to do a release, just would prefer to do as few as possible as it will take me at least an hour to figure out how again#2016-07-2721:49Alex Miller (Clojure team)@gfredericks: a good change log wrt to things likely to affect spec would also be helpful to have#2016-07-2721:50gfredericks@alexmiller: like to send to you folks or for permanent public consumption? #2016-07-2721:52Alex Miller (Clojure team)public, not looking for anything special for us, just to have a place to start analyzing impact#2016-07-2721:52gfredericksCool#2016-07-2807:43mpenet@gfredericks: chuck.gen/string-from-regex is nothing short of magic, impressive#2016-07-2809:42stammiHi! Maybe someone can spot my mistake in this:#2016-07-2809:42stammi(s/def ::number-between-one-and-zero (s/and number? #(< % 1) #(> % 0)))#2016-07-2809:42stammi(s/conform ::number-between-one-and-zero "-13")#2016-07-2809:43stammigives me valid#2016-07-2809:44stammioh forget that please...#2016-07-2809:44stammi...gnarf#2016-07-2812:56Alex Miller (Clojure team)Check out s/int-in or s/double-in#2016-07-2813:29mpenetsmall attempt at specing the latest ring specification -> https://github.com/mpenet/ring-spec/blob/master/src/clj/qbits/ring_spec.clj#2016-07-2813:49mike_ananevhi! if i have (s/def ::decimal #(instance? BigDecimal %)) how to make (s/exercise ::decimal) working? I suppose a should create custom generator in spec, but can't find how to do it. thanx#2016-07-2813:57mpenetI guess something like (gen/fmap #(BigDecimal. %) gen/nat)#2016-07-2813:58mpenet(s/def :decimal (s/spec #(...) :gen (constantly (gen/fmap #(BigDecimal. %) gen/nat)))) (untested)#2016-07-2813:59mpenet@mike1452: super handy cheat-cheat -> https://github.com/clojure/test.check/blob/master/doc/cheatsheet.md#2016-07-2814:30mike_ananev@mpenet: šŸ˜ž doesn't work. I've tried similar ways and unfortunately can't get result. i'm dumb :((#2016-07-2814:31mpenetseems to work here#2016-07-2814:32mpenet
(s/def ::decimal (s/spec #(instance? BigDecimal %)
                         :gen (constantly (gen/fmap #(BigDecimal. %) gen/nat))))
#2016-07-2814:32mpenetexercise returns -> ([0M 0M] [0M 0M] [1M 1M] [2M 2M] [1M 1M] [0M 0M] [4M 4M] [1M 1M] [2M 2M] [5M 5M])#2016-07-2814:32mpenetetc#2016-07-2814:33mpenetwell, you need to throw in the decimal part actually ex (str % ".0")#2016-07-2814:34mpenetyou could generate pairs for the part after .#2016-07-2814:35mike_ananevthanx. can you show your gen namespace from require section?#2016-07-2814:35mpenetgen is clojure.test.check.generators#2016-07-2814:36gfredericksMaybe it's a good time to get a good bigdec and bigint and ratio generator in test.check#2016-07-2814:36mpenetthere is ratio I think, not big* tho#2016-07-2814:37mike_ananevhave trioubles with gen namespace#2016-07-2814:37mike_ananevAlias gen already exists in namespace arina-aaa.atomix.spec, aliasing clojure.spec.gen#2016-07-2814:38mike_ananev(:require [clojure.spec :as s] [clojure.test.check.generators :as gen])#2016-07-2814:39mike_ananevAh! REPL restart helps!#2016-07-2814:39mike_ananevthanx!#2016-07-2814:39mpenetšŸ™‡#2016-07-2814:46gfredericksOh right. I meant a good ratio generator :)#2016-07-2815:53michaeldrogalisI'm a little caught up on s/def being a macro. Is it possible to do something like:
(let [x :my/spec-name]
  (s/def x string?))
#2016-07-2815:54michaeldrogalisThe problem seems to be that def expands, and the first argument retains the value of the symbol x, not the value that it refers to (`:my/spec-name`)#2016-07-2816:20featalionyou can do it with another macro:
(let [x :user/the-x]
  `(s/def ~x string?))
;;=> (clojure.spec/def :user/the-x clojure.core/string?)
#2016-07-2817:34stammithanks @alexmiller. that looks better indeed#2016-07-2909:50mpenethow can I spec the return spec of a function that depends on the args it received : ex ring-spec normal handler vs async-handler: (if args.length == 3 : ret then is nil else response map)?#2016-07-2909:50mpenetI guess I can hack something with :fn, but it doesn't feel right#2016-07-2909:53mpeneta "multiple arity" version of fspec could make sense here. it would feel more natural maybe#2016-07-2909:54mpenet(s/fspec (:args ... :ret) (:args :ret))#2016-07-2909:56mpenetor maybe I can just write 2 fspec and just wrap them with (s/or ..)#2016-07-2912:22gfredericksYou could make the ret spec nilable and describe that relationship in the :fn as you mentioned. Doesn't seem like that much of a hack to me#2016-07-2912:23mpenetI think I prefer the s/or with the 2 fspecs, makes the relation between args/ret clearer#2016-07-2912:24mpenetI came up with this in the end https://github.com/mpenet/ring-spec/blob/master/src/clj/qbits/ring_spec.clj#L81-L111#2016-07-2913:49Alex Miller (Clojure team)@mpenet: the whole purpose of :fn is to describe relationships between args and ret#2016-07-2913:51mpenetgot it, so not really good to discern 2 arities of a function like I was doing (I had one "giant" fspec describing both arities/return types)#2016-07-2913:51Alex Miller (Clojure team)seems perfectly reasonable to cover each possibility in the :ret (just tell the truth about what it can be)#2016-07-2913:51Alex Miller (Clojure team)then in :fn you can declare that the :ret you got matched the :args that were sent#2016-07-2913:52Alex Miller (Clojure team)you get conformed values of :args and :ret in :fn, so it should be relatively easy to check#2016-07-2913:53mpenetYes I saw this, but wouldn't it be more work for gen down the road having all these branches? I mean vs. (s/or (s/fspec) (s/fspec))#2016-07-2913:53mpenetI guess it depends what kind of functions we re talking about#2016-07-2913:53Alex Miller (Clojure team)it just picks one or the other#2016-07-2913:53Alex Miller (Clojure team)thatā€™s why I have a computer :)#2016-07-2915:00mpenetany teaser of what's coming in the next alpha?#2016-07-2915:14gfredericksa fully-featured dependent type system#2016-07-2916:30arohner@gfredericks: you joke, but: https://github.com/arohner/spectrum#2016-07-2916:31gfredericks@arohner: oh hey good!#2016-07-2916:32arohnerIt still needs a whole lot of polish, but Iā€™m convinced its sound#2016-07-2916:33mpenetwow, impressive. will kick tires for sure#2016-07-2916:36arohneritā€™s still basically at developer preview stage#2016-07-2916:36arohnerbasic tests pass, and simple files will check / not-check as appropriate, but nowhere near ready for production code#2016-07-2922:34Alex Miller (Clojure team)@mpenet I have some of the nastier core macro specs ready and some of them might land in the near future#2016-07-2923:30lvhgfredericks: Preferred schpec namespace for maps that are closed for extension and its generators? keys?#2016-07-2923:30lvhAlso, weighted-or#2016-07-2923:31lvh(weighted-or:
(defmacro weighted-or
  "Like [[s/or]], but with weights."
  [& weighted-options]
  (let [[names specs ws] (apply map vector (partition 3 weighted-options))
        sums (reductions + ws)]
    `(s/with-gen
       (s/or 
)
#2016-07-2923:58lvhIs there a pred for large integers?#2016-07-2923:58lvhLike biginteger or bigint#2016-07-3000:00lvhI canā€™t seem to find long? either#2016-07-3000:01eggsyntaxint? will capture all fixed-precision integers. It would still be nice to have a bigint? pred so that the generator specifically produces bigints.#2016-07-3000:02lvhah, gotcha#2016-07-3000:02lvhthanks šŸ™‚#2016-07-3000:02gfredericks@lvh I don't mind the top level namespace for things that are pretty useful/general#2016-07-3000:03lvhgfredericks: OK works for me#2016-07-3001:10Alex Miller (Clojure team)Doesn't integer? map to that?#2016-07-3001:10Alex Miller (Clojure team)Oh you want just bigints#2016-07-3001:13seancorfield(s/and int? (s/conformer bigint)) perhaps?#2016-07-3001:14seancorfield
boot.user=> (s/exercise (s/and int? (s/conformer bigint)))
([-1 -1N] [0 0N] [0 0N] [0 0N] [4 4N] [4 4N] [-12 -12N] [-45 -45N] [4 4N] [-63 -63N])
#2016-07-3001:15seancorfieldBTW @alexmiller I think the docstring on s/conformer is very confusing about its intent...#2016-07-3001:15seancorfield"takes a predicate function with the semantics of conform i.e. it should return either a (possibly converted) value or :clojure.spec/invalid, and returns a spec that uses it as a predicate/conformer"#2016-07-3001:17Alex Miller (Clojure team)Because?#2016-07-3001:18seancorfieldBecause the function argument it takes isnā€™t really a "predicate function"ā€¦ that makes it sound like it should be something like int?...#2016-07-3001:19seancorfieldā€¦perhaps if it said "takes a conforming function i.e., it should return either a (possibly converted) value or :clojure.spec/invalid"ā€¦?#2016-07-3001:21Alex Miller (Clojure team)Maybe#2016-07-3001:24seancorfield(predicate is pretty much otherwise used for functions that return truthy / falsey values in the Spec docs I think?)#2016-07-3001:26seancorfield& and and take predicates, fdefā€™s :ret, merge, spec#2016-07-3001:28seancorfield(although there is mention in a couple of places that some of those "predicate" contexts may also conform the valueā€¦ so itā€™s not entirely clear whatā€™s intended)#2016-07-3001:31seancorfield@lvh @eggsyntax I tested and it looks like you can use (s/and (s/double-in ā€¦) (s/conformer bigdec)) if you wanted BigDecimals ā€” and (s/conformer rationalize) if you wanted ratiosā€¦ and the example above produces big ints#2016-07-3001:32seancorfieldOh, simpler: (s/exercise bigdec?) ā€” didnā€™t realize there was a direct generator for that#2016-07-3001:33seancorfieldand (s/exercise ratio?) ...#2016-07-3001:33seancorfieldSo bigint? is kind of the only missing one...#2016-07-3001:35eggsyntaxOh, interesting. I was thinking something roughly like (s/with-gen int? (gen/fmap #(bigint %) int?)) (that's just offhand & may have errors; I don't have a repl at hand).#2016-07-3001:48seancorfieldUsing s/conformer seems simplerā€¦ but now Alex has made me question whether thatā€™s entirely valid šŸ˜ø#2016-07-3001:48seancorfieldj/k#2016-07-3001:50seancorfieldWe use s/conformer as a way to map input field formats to domain model formats along with specs for input fields. Weā€™re now using specs as part of input validation and as part of our domain model testing.#2016-07-3001:52eggsyntaxNice. I haven't really played with s/conformer yet (although I'm certainly making use of s/conform).#2016-07-3001:53lvh@seancorfield: great, thanks šŸ™‚#2016-07-3001:54lvh@alexmiller: Iā€™m OK with any integer; I didnā€™t realize integer? had been introduced and was using int?#2016-07-3002:02Alex Miller (Clojure team)integer? is old, int? is the new one#2016-07-3009:15dryewoIt seems that :ret is not checked at all currently:
(defn foo [x] x)
(s/fdef foo
  :args (s/cat :x int?)
  :ret string?)
(stest/instrument `foo)
(foo 1)
#2016-07-3009:16dryewowhat am I doing wrong?#2016-07-3009:17dryewoif I put string? into :args, it throws as expected.#2016-07-3009:17dryewoIā€™m using alpha10#2016-07-3009:28dryewoHowever, when I run (stest/check foo)`, it complains about wrong :ret. Should it be this way, only checked while testing?#2016-07-3009:30dryewoOh, it seems to be a feature: https://groups.google.com/forum/#!msg/clojure/RLQBFJ0vGG4/K-5Rp6QXCgAJ#2016-07-3009:30dryewookay, never mind#2016-07-3017:44seancorfield@dryewo: yes, instrument is to test that functions are called correctly, check is to test that functions are implemented correctly. #2016-07-3103:00gfredericksthat's a nice explanation#2016-07-3103:15seancorfield(Iā€™m just repeating what Stu said in the latest Cognitect clojure.spec 'cast! šŸ™‚ )#2016-07-3118:47gfredericks@alexmiller: the call to generate here undermines test.check determinism (similar to CLJ-1949): https://github.com/clojure/clojure/blob/f374423053b75b7b484ffbc8b49b2ede9d92e406/src/clj/clojure/spec/test.clj#L155#2016-07-3118:47gfredericksin particular if you're running check while instrumenting with stubs#2016-07-3118:48gfredericksI don't think there's as straightforward a fix as in CLJ-1949 though#2016-07-3118:49gfredericksprobably the cleanest thing would be a private dynamic var somewhere for registering quickcheck-driven generation fns#2016-07-3118:49gfredericksit could use gen/generate as the default when you're not actually running a test.check test#2016-07-3118:54gfredericksshall I make a ticket?#2016-07-3119:02Alex Miller (Clojure team)Sure#2016-07-3119:16gfredericksACK#2016-07-3122:43lvh@gfredericks: FYI, I made a bunch of progress on the JSON Schema thing, knocking a few remaining ones down; the really really hard one is references (idk if youā€™re familiar with JSON schema, but basically you have a base URL and URL-y internal/external references, so you need a way to resolve things like #../../xyzzy/0 or whatever). Dunno if itā€™s going to be of a size where itā€™s interesting for spec, but weā€™ll see#2016-07-3122:44lvhAre you interested in like, mostly-working-but-missing-stuff impls, or would you rather just pull in an entire impl when itā€™s done, or rather have it live in its own package#2016-07-3122:44lvhItā€™s currently about 250 lines of impl and 220 of test; thatā€™ll probably go up to 300#2016-07-3122:45lvhit doesnā€™t have any particularly weird deps, but to do it Right(TM) you need test-chuckā€™s regex generator#2016-08-0103:37gfredericks@lvh I was pondering having an "experimental" namespace, so we could probably start it there (c.g.schpec.expermental.json-schema)#2016-08-0103:38gfredericksschpec depending on chuck is fine and probably inevitable#2016-08-0109:09rickmoynihan@lvh: @gfredericks: Sorry, just seen the messages above about JSON Schema / spec... curious what you're doing exactly, it sounds interesting and useful!#2016-08-0109:46mpenetsame here, I'd love to see the work in progress#2016-08-0110:12mpenetI started to dump utils I am using in our projects in https://github.com/mpenet/spex, I might make a couple of PR if there's an agreed "spec utils" project in the works (and interest)#2016-08-0113:38lvhIā€™m writing a fn that takes a JSON Schema and produces an equivalent spec#2016-08-0114:42dominicm@lvh: How possible would the reverse be do you think?#2016-08-0114:50gfredericksshould be possible for a subset of specs presumably#2016-08-0115:24lvh@dominicm: Trickier, but like gfredericks said possible for some subset of specs; the biggest problem is that you presumably need to statically determine what a particular spec pred means, and then see how much of that you can express in JSON Schema#2016-08-0115:25lvhitā€™s not going to be very nice json schema (probably larger, decent amount of duplication) but as long as you treat it like a compilation product and donā€™t care about bidirectional access that seems fine#2016-08-0115:25dominicmYeah, that was my thoughts also.#2016-08-0115:25gfredericksthe fact the predicates are stored as symbols that can't be reliably mapped to vars seems problematic for doing this kind of thing robustly#2016-08-0115:25dominicm^^#2016-08-0115:26dominicmCould you say :foo/bar is this part of JSON schema?#2016-08-0116:03rickmoynihana symbol is a namespace qualified value too though right? 'clojure.core/int#2016-08-0116:06gfredericksNot necessarily#2016-08-0116:07gfredericksThe mapping from symbols to vars is entirely relative to the current namespace setup#2016-08-0116:08gfredericksYou can setup your namespace so that clojure.core/int resolves to something entirely different#2016-08-0116:08gfredericksBut in any case I think spec would only store int? generally#2016-08-0116:10rickmoynihansurely a macro could resolve it to its bound value though by inspecting ns-map etc... though#2016-08-0116:33arohner
(s/form (s/spec int?))
clojure.core/int?
#2016-08-0116:34arohneryes, you could set up your namespace so that int? redirects to something else, but spec does store the ns#2016-08-0116:37eggsyntax@lvh: @seancorfield -- I just had occasion to write a spec for bigints, and was surprised to discover that (int? (bigint 1)) returns false. So I went with:
(s/def ::bigint (s/with-gen #(instance? clojure.lang.BigInt %)
                  (fn [] (gen/fmap #(bigint %) (s/gen int?)))))
Just as an FYI followup.
#2016-08-0116:38lvhWhat does integer? do?#2016-08-0116:39lvhI find the two pretty confusing still :)#2016-08-0116:41eggsyntaxint? has to be fixed-precision. integer? just has to represent an integer, I guess. So I may be able to simplify that šŸ™‚#2016-08-0116:42eggsyntaxYes indeed, this works:
(s/def ::bigint (s/with-gen integer? 
                  (fn [] (gen/fmap #(bigint %) (s/gen int?)))))
#2016-08-0116:42eggsyntaxThanks @lvh, I would have missed that.#2016-08-0116:45seancorfield@eggsyntax: and why doesnā€™t (s/def ::bigint (s/and integer? (s/conformer bigint))) work for you?#2016-08-0116:45seancorfield(all I changed from my original suggestion was to use integer? instead of int? there)#2016-08-0116:46seancorfieldI guess the question is should (s/conform ::bigint 1) be 1N or :clojure.spec/invalid?#2016-08-0116:46eggsyntaxYeah, good question. For my purposes, sure. But I could imagine going the other way in another context.#2016-08-0116:47eggsyntaxThe conformer version works too, & is nicer actually, I'll go with that šŸ˜„#2016-08-0116:48eggsyntaxI mainly just meant to illustrate that it had to be integer? rather than int?#2016-08-0117:05seancorfieldYour predicate ā€” #(instance? clojure.lang.BigInt %) ā€” is not the same as integer? thoā€™. Your predicate rejects 1. That was my question.#2016-08-0117:06seancorfield(and (s/exercise integer?) doesnā€™t seem to generate bigints in my quick tests)#2016-08-0117:07seancorfieldclojure.spec encourages us to be specific šŸ™‚#2016-08-0117:56gfredericksw.r.t. generating bigints, I ran into a bug once that didn't surface until the bigints were larger than Double/MAX_VALUE#2016-08-0117:58gfrederickswhich makes it interesting to consider how large of a bigint a general bigint generator should test by default#2016-08-0117:58gfrederickspresumably at least bigger than Long/MAX_VALUE#2016-08-0117:58gfredericksbut Double/MAX_VALUE is quite large#2016-08-0118:02dominicm@gfredericks: I thought some of the automatic tests were around max/min values?#2016-08-0118:03gfredericksyou're asking about default generator behavior?
#2016-08-0118:03gfrederickstest.check doesn't have anything like that currently besides favoring small numbers#2016-08-0118:03gfredericksand nan/infinity#2016-08-0118:03dominicmAh right. I was operating based on conference talks. That's unusual.#2016-08-0118:05gfredericksI'm not sure what the best way to do that is#2016-08-0118:05gfrederickse.g., if Double/MAX_VALUE should be tested frequently, does that mean the value right below it should be too? or just make that one special? and should it be easy to "shrink" to Double/MAX_VALUE from something more random? or should we always shrink downward?#2016-08-0118:06glv@gfredericks: I donā€™t know if you saw the discussion I started a couple of weeks ago about the growth rate on the default spec generators ā€¦ but the conclusion was that we definitely need integer generators for sizes (that grow slowly and have reasonable bounds) and generators for values used in mathematical functions (that grow rapidly and as large as they can get). Those already exist in test.check (although theyā€™re not labeled/documented for those uses) but you have to jump through more hoops to get sensible bounds in spec.#2016-08-0118:06gfredericksI think I saw that, yeah#2016-08-0118:06gfredericksI wish I could just rename a bunch of generators#2016-08-0118:06gfredericksA docstring overhaul could be good at least#2016-08-0118:08xcthulhu@gfredericks: You probably weren't considering supporting goog.math.Integers, but if you were you might be interested to know that multiplication is broken for them https://github.com/google/closure-library/issues/703#2016-08-0118:09gfredericksxcthulhu: ha, I fixed a couple division bugs in that class a while back#2016-08-0118:10xcthulhuOh man thank you I did notice that division works!#2016-08-0118:12gfredericksI've definitely thought about Integer but I'm not sure how it ought to fit since it's pretty 2nd class in the cljs arithmetic world#2016-08-0118:12gfrederickscould definitely add some stuff in test.chuck for it though#2016-08-0118:13xcthulhuWell... they don't work... when I need bigintegers I use SJCL and am generally disappointed.#2016-08-0118:13gfrederickswhat doesn't work?#2016-08-0118:13xcthulhuMultiplication#2016-08-0118:13gfredericksthat might be a recent bug#2016-08-0118:13xcthulhuI ran into that particular bug when I was implementing modular inverses.#2016-08-0118:13gfredericksI did a bunch of property-based testing on Integer in https://github.com/gfredericks/exact#2016-08-0118:13gfrederickswhich is where I ran into the division bugs#2016-08-0118:14gfredericksso either I tested on an earlier version w/ the mult bugs or else they're hard to find#2016-08-0118:15xcthulhuCool maybe I should port the modular inverse code I wrote since it makes great tests.#2016-08-0118:15xcthulhuI didn't have time to spelunk goog.math.Integer to figure out the problem.#2016-08-0118:16gfredericksI'll probably take a look at it; I've already been through their contribution process#2016-08-0118:21xcthulhuThis is a cool library btw. Would you ever want an is-prime? function? Miller-Rabin is rather fast provided you accept the generalized Riemann hypothesis is true.#2016-08-0118:28gfrederickswell you should probably call it probable-prime? then; but I'm not opposed to a namespace for number theory stuff if you find that useful#2016-08-0119:45glv@gfredericks: Yeah, I think for test.check a documentation update is fine ā€¦ the right generators are there, itā€™s just not obvious to a newcomer how different those two use cases are.#2016-08-0119:45glvā€¦ or at least, to this newcomer.#2016-08-0120:01gfredericksyeah; communicating how sizing works in general is kind of difficult#2016-08-0121:21pvinishello. i want to have something like
:nav {:bla 1
      :foo [{:ok true
             :sure 0}]}
how would i do the s/def on that?
#2016-08-0121:30lvhIs there a way in clojure.spec to specify xor of schemas? JSON Schema has it. Specifically, thereā€™s a way to say that something must match all these schemas, or at least one of them, or exactly one of them (the latter being the xor Iā€™m talking about).#2016-08-0121:31pvinisalso, what happens if i have a key like :ok in some other place, and its a string there? how do i s/def that..?#2016-08-0121:31lvhI guess Iā€™ll just do it with and and a predicate#2016-08-0121:32gfredericks@lvh: add that to schpec#2016-08-0121:32lvhsounds good#2016-08-0122:37lvhis there a way for a pred that fails to communicate something about why it failed? specifically, iā€™m implementing xor as described above (aka one-of), Iā€™d like to say the name of the spec that failed the pred#2016-08-0122:42eggsyntaxSpecifically the predicate itself? For a spec it's just (s/explain ::the-spec failing-input)#2016-08-0122:43lvhyeah; the predicate is checking that something is invalid against a particular spec#2016-08-0122:43lvhand spec doesnā€™t appear to support negation natively right now#2016-08-0122:44eggsyntaxFor a pred you can do (s/explain (s/spec my-pred) failing-data)#2016-08-0123:26lvheggsyntax: Sorry, Iā€™m not being very clear. I know how to get specā€™s opinion on why the data is bad; Iā€™m trying to improve what s/explain produces when you give it failing data#2016-08-0123:59eggsyntax@lvh: ah, gotcha. Maybe you can get more detail by splitting it into multiple subspecs? Dunno, just guessing.#2016-08-0202:39lvh@gfredericks: I donā€™t know if youā€™re interested in adding me as a contributor to schpec, but Iā€™d be happy to add e.g. CI and review PRs.#2016-08-0202:40lvhexcl-keys PR is up: https://github.com/gfredericks/schpec/pull/6#2016-08-0202:40gfredericks@lvh that'd be cool#2016-08-0202:53jstokesis there a test runner for specs now? the documentation for fdef references clojure.spec.test/run-tests but I can't seem to find that function#2016-08-0204:18seancorfield@jstokes: The short answer is "no". Rich has talked about this on the mailing list and in the Cognicast and it sounds like they want clojure.spec to be test tool neutral. You can already use it with clojure.test, Expectations, etc however you want so you can use your existing test runner setup if you want. But clojure.spec is kind of different to regular tests...#2016-08-0204:19seancorfield...you may well want to (instrument) your application while running your normal test suite or even while just developing...#2016-08-0204:20seancorfield...but generative tests don't always fit in with fast-running "unit" style tests so you probably don't want to just run those in amongst your regular tests.#2016-08-0213:13jstokes@seancorfield: thanks! that makes a lot of sense#2016-08-0213:15jstokesi noticed that some of my specs do take a while to run, so probably dont want that included in my normal cider-test workflow#2016-08-0213:19glvWhat Iā€™ve always wanted with test.check is a nice way to run the generative tests as part of the usual unit-test suite, but with a much smaller number of tests. And then run them separately with the full deal.#2016-08-0213:21glv(1000 tests is nicely exhaustive; 25 is a decent sanity check.)#2016-08-0213:21glvI suspect thereā€™s a way to do that that I havenā€™t figured out yet.#2016-08-0213:23minimal@glv: you can use times and use an env var to change from the default : https://github.com/gfredericks/test.chuck/blob/master/src/com/gfredericks/test/chuck.cljc#2016-08-0213:40glvAh, nice. Thanks!#2016-08-0214:46jstokeshow would you use test.chuck with specs? clojure.spec.test/check returns the check results, not something you can plug in to defspec, right?#2016-08-0215:03gfredericksyou'd probably reimplement something similar that passes num-tests to clojure.spec.test/check#2016-08-0215:51jstokesah ok, i didnā€™t know if that was already in place#2016-08-0215:51jstokesthanks#2016-08-0215:52gfrederickssomething like that could be added to schpec#2016-08-0216:48lvhIā€™m writing stuff that produces (s/and ā€¦) specs and generates a pred for it; Iā€™d like to make the error message more useful. Right now it just says something silly like:
{:clojure.spec/problems
             ({:path [:test-schema.root/item-0],
               :pred (subset? (set (keys %)) all-props),
               :val {:a 1},
               :via [:test-schema/root :test-schema.root/item-1],
               :in []})}
all-props is a set. Itā€™d be much more useful if it showed the actual set. s/and inspects the form, so I canā€™t quite figure out how to do that ā€” eval doesnā€™t quite work because then you get the ugly syntax-quoted form
#2016-08-0216:50lvhhow can I communicate the value of all-props for when it fails? Write my own Spec impl?#2016-08-0221:52bsimawhy does def work here even though it's excluded in the ns form? https://github.com/clojure/clojure/blob/d920ada9fab7e9b8342d28d8295a600a814c1d8a/src/clj/clojure/spec.clj#L10-L19#2016-08-0222:14gfredericksbsima: def is a special form, you can't exclude it#2016-08-0222:14bsimammm ok#2016-08-0222:54glvIn that case, does having it in the exclude clause have any effect?#2016-08-0223:11gfredericksnot that I know of#2016-08-0223:11gfredericksexclude is silent when the thing doesn't exist#2016-08-0303:03scriptorright, def doesn't exist in clojure.core, it's part of the base language#2016-08-0303:22gfredericksYou could imagine a not-too-different clojure where special forms are namespaced and excludable#2016-08-0312:50rickmoynihanI'm curious what people think about the differences/benefits/relationships between: - Generative Property Based Testing - Pairwise Testing In particular is anyone combining the two... It seems to me that you could/should use generative tests to provide instance data within certain equivalence classes of values and then use pairwise testing to generate arguments as test pairs to apply to the function under test.#2016-08-0313:43borkdudeclojure.spec and music generation - has anyone tried it ? šŸ˜‰#2016-08-0313:46eggsyntax@borkdude: not I, but I've written some evolutionary music programs, and I think that'd be a really cool approach -- evolve organisms whose phenotype is a spec & generate music from the spec.#2016-08-0313:47eggsyntaxCarin Meier has already some genetic programming w/ spec.#2016-08-0313:47eggsyntaxhttp://gigasquidsoftware.com/blog/2016/07/18/genetic-programming-with-clojure-dot-spec/#2016-08-0313:47borkdude@eggsyntax: thanks for the read!#2016-08-0314:26lopalghostAnyone using spec to parse and validate tabular data?#2016-08-0315:06mpenet@alexmiller: Should I fill a ticket to have :conform-keys be turned into :conform-keys? (option in map-of), also :distinct-> :distinct?, there are possibly other bool options like these in spec, didn't check all of them#2016-08-0315:22rickmoynihanlopalghost: not yet - but I totally plan too#2016-08-0315:22rickmoynihanlopalghost: I think specs and their regexes etc are a natural fit for validating columns/rows of data#2016-08-0315:35lopalghost@rickmoynihan: I've been using it to read unfriendly excel spreadsheets, it's really helpful Was wondering what sort of approaches people might be taking#2016-08-0315:43Alex Miller (Clojure team)@mpenet you can, not sure on the likelihood of that changing#2016-08-0315:44rickmoynihanlopalghost: I plan on doing the same thing - as it seems a natural fit... curious to hear about your experiences though#2016-08-0318:56thegeezis clojure.spec/conform meant to be used in production code (and not only when developing/testing/instrumenting)? For spec/assert it is mentioned explicitly that is should be a no-op and disabled in production, same as with instrumenting and such. Does the same hold for spec/conform? Will the run-time impact be to high to use spec/conform generously everywhere?#2016-08-0319:02lvhmy understanding is that conform is expected at the edges where you take input#2016-08-0319:02lvhit has the benefit of e.g. automatically classifying multiple options for you (s/or) via a map key#2016-08-0319:03lvhalso, you donā€™t care about expensive there because a) youā€™re doing it once b) the alternative is potentially consuming random garbage and having it break somewhere down the road#2016-08-0319:11thegeezthanks, I hadn't considered the "one time" aspect of using conform. I was thinking of using conform multiple times on the same value but that doesn't work as conform will most likely transform the value#2016-08-0319:28donaldballFor a couple of maps I want to specify, there are some derived values therein which Iā€™d like to specify (and generate) correctly. For example {:a 1 :b 2 :a+b 3}. This is easy to express with (s/and (s/keys ā€¦) #(some-predicate)) but of course the generator essentially never passes the such-that filter#2016-08-0319:29donaldballHas anyone found any good patterns for this situation?#2016-08-0320:43gfredericksdonaldball: running the default generator through gen/fmap would make it easy enough to overwrite key like the one you showed#2016-08-0320:43gfredericksI don't think clojure.spec makes it easy to modify a generator though#2016-08-0320:55donaldballIā€™ve kinda settled on having two specs, one which specifies the map with only independent values, and another which specifies the map including derived values#2016-08-0321:09gfredericksgen/fmap should help in that case too#2016-08-0322:28donaldballYea, itā€™s not hard to add on, Iā€™m just wondering if anyone else has been writing specs for maps with internal value consistency requirements and, if so, if thereā€™s a declarative approach to expressing them#2016-08-0402:31mishagood morning. is there a short answer to
are clojure.spec's - a replacement for {:pre ... :post ...} condition-map parameters?
? (assuming sparse usage of the latter)
#2016-08-0402:35gfredericks@misha from my perspective, mostly yes#2016-08-0402:36misha@gfredericks: thanks#2016-08-0402:54Alex Miller (Clojure team)@thegeez you can definitely use conform to destructure values if thatā€™s useful (particularly useful in macros when decoding syntax). Cost is not free of course but will get better.#2016-08-0402:58Alex Miller (Clojure team)@gfredericks itā€™s easy to add a generator with s/with-gen. There are combinations of gen/bind and gen/fmap that can help you work from a model of your data towards generated values. The next (I think) screencast from Stu is going to talk about this more.#2016-08-0403:00gfredericks@alexmiller: but it's not easy to use with-gen to modify the default generator, unless I'm missing something#2016-08-0403:03Alex Miller (Clojure team)It could be easier :)#2016-08-0403:05Alex Miller (Clojure team)But if you have just the base spec, you can pull its gen and feed it into fmap#2016-08-0403:08gfredericksYep#2016-08-0416:43calvisis there any easy way to spec a map where the required keywords depend on the values in the map? > e.g. if (:a m) is 5 then m should also have a :b keyword I realize this can be done with manually defined predicates and then manually defined generators#2016-08-0416:46eggsyntaxSeems like you could cobble it together with s/or, defining each possible combination separately (&, in your example, presumably two separate specs for the options for :a). But thatā€™s hardly elegant...#2016-08-0416:47eggsyntaxie an ::a-5 predicate and a general ::a version#2016-08-0416:48eggsyntaxwith one alternative containing the former as well as the :b keyword, and the other alternative containing the latter and not containing :b.#2016-08-0416:48calvisIā€™m specifically asking for an elegant way, before I go and write a macro that does it#2016-08-0416:48eggsyntaxYeah, just brainstorming.#2016-08-0416:49calvisit seems like this would be a pretty common thing to do at least#2016-08-0418:28bsimaIs it preferred to put specs in a separate namespace, like how clojure.java.jdbc does it? --> https://github.com/clojure/java.jdbc/blob/f9fe3dde5b4bc5f1a5b5a79ffb5374fad3377b0b/src/main/clojure/clojure/java/jdbc/spec.clj#2016-08-0418:30gfredericks@bsima: I think java.jdbc does it to be backwards compatible with older versions of clojure#2016-08-0418:34seancorfield@bsima: FWIW, weā€™re following a similar pattern at World Singles where our specs are in one namespace and we pull that into other namespaces as needed ā€” but thatā€™s partly because nearly all of our specs so far are very data-focused rather than function-focused.#2016-08-0418:34seancorfieldWe havenā€™t yet decided how that will play out when weā€™re defining system boundary APIs on top of those specs.#2016-08-0418:35seancorfieldAnd, yes, java.jdbc needs to support Clojure back to 1.4.0, as @gfredericks says.#2016-08-0418:37bsimain a recent cognicast rich hickey mentioned using specs as a way to version a codebase, like "if your code matches this set of spec keys then we didn't break anything with this new release" (paraphrasing)#2016-08-0418:38bsimaso, I think that's an interesting way to approach specs, and an argument for separating specs from code#2016-08-0418:39bsimamost of the specs i've written so far have been fdefs, it almost feels like racket's contract system#2016-08-0419:53Alex Miller (Clojure team)for another data point, we are currently working on putting clojure.core specs in the namespace clojure.core.specs#2016-08-0419:54Alex Miller (Clojure team)not saying that following the pattern is the only way or best way to do it for anyone elseā€™s particular situation though#2016-08-0421:14rickmoynihanFWIW - I've been experimenting with the specs in a sidecar namespace, but with the fdefs in the foo namespace beneath the functions... not been using spec much yet though - so not sure how it'll work out#2016-08-0421:40bbrinckI've been placing the fdef right above the function definition. I find that's the most valuable place from the perspective of providing documentation. I like seeing the specs, the arg names, and the docstring all in one place.#2016-08-0421:41bbrinckbut i suspect that specs for common data types will go in a different namespace that can be used in many namespaces#2016-08-0421:53rickmoynihanyou can do that? I'd assumed because fdef takes a symbol for the function its annotating that it would have to come after the definition of the function#2016-08-0422:03bbrinck@rickmoynihan: yeah it works for me:
(s/fdef my-new-fn
        :args (s/cat :int integer?))
(defn my-new-fn [x]
  (+ x x))

(stest/instrument `my-new-fn)

(comment
  (= 2 (my-new-fn 1))    ; => true 
  (= 2 (my-new-fn nil))  ; spec error
  )
#2016-08-0422:05rickmoynihanright enough: https://github.com/clojure/clojure/blob/d920ada9fab7e9b8342d28d8295a600a814c1d8a/src/clj/clojure/spec.clj#L294#2016-08-0422:05rickmoynihanagree that they're better above the definition#2016-08-0422:22seancorfieldJust remember that if your fdefs are elsewhere, you need a qualified symbol name (see https://github.com/clojure/java.jdbc/blob/master/src/main/clojure/clojure/java/jdbc/spec.clj where I updated this for Alpha 10 ā€” unqualified symbols worked in an earlier Alpha if you just referred them in unqualified).#2016-08-0502:08lvh@gfredericks: You saw the PR I put up for schpec a few days ago, right? Iā€™d like to contribute all 3 things I have tickets open for, but Iā€™d prefer to do them sequentially for minimal git exposure#2016-08-0502:09gfredericks@lvh yep, I just didn't have any free time the last week; I think this weekend is promising#2016-08-0502:10gfredericksI can probably turn them into a proper release#2016-08-0502:13lvhsure, no problem šŸ™‚ not complaining, just wanted to make sure you knew it existed#2016-08-0502:13lvhI will be in Seoul next week with mixed amounts of time available on this#2016-08-0502:14gfredericksACK#2016-08-0510:06kurt-yagramHey, would it be considered bad practice to put spec/def inside a top-level let?#2016-08-0510:08mpenetI think so yes, if you need let scoped specs you can use s/spec I think#2016-08-0510:09mpenets/def mutates the spec registry so a bit hairy in a let imho#2016-08-0510:15kurt-yagramalright... thx#2016-08-0513:36glvJust curious ā€¦ what would you be defining in the let that the spec would need? I can see maybe putting it in a letfn if you needed a particularly hairy :fn and wanted to compose it from smaller pieces ā€¦#2016-08-0515:47mpenet(s/def ::something (letfn [(yourpred? [x[ ...) (yourgen? [] ...)] (s/spec yourpred? :gen yourgen?)) is nice also for the explain#2016-08-0515:47mpenetat least it shows something more meaningful than an anonymous fn#2016-08-0515:57glvYep, you could put the letfn either place.#2016-08-0516:08donaldballIā€™ve been using that pattern like itā€™s going out of style for my specs of e.g. maps with dependent values#2016-08-0516:15donaldballHas anyone been able to articulate a reason for or work around the MultiFn errors that often occur when running stest/check?#2016-08-0516:15donaldballE.g.
CompilerException java.lang.ClassCastException: cider.nrepl.middleware.test$report cannot be cast to clojure.lang.MultiFn, compiling:(clojure/test/check/clojure_test.cljc:95:1) 
#2016-08-0516:55gfredericks@donaldball: yes, you just have to explicitly require clojure.test.check.clojure-test#2016-08-0516:56donaldballAwesome, thanks#2016-08-0516:59donaldballAlas, I still get that exception#2016-08-0517:14gfredericks@donaldball: see TCHECK-113#2016-08-0517:45Alex Miller (Clojure team)@donaldball I think I added the lein workaround as a comment there to turn off clojure.test monkeypatching#2016-08-0519:19donaldballFWIW the lein workaround is sufficient to get lein test to not barf, but I needed to explicitly require clojure.test.check.clojure-test in my test namespaces for the check to work in my cider repl#2016-08-0519:19donaldballĀÆ\(惄)/ĀÆ#2016-08-0519:53gfredericksI'll take this as more evidence that a test.check release would be helpful#2016-08-0600:07lvhHas anyone worked on a datalog spec?#2016-08-0600:46eggsyntax@lvh: Iā€™ve been doing a spec for our datomic data, but Iā€™m generating specs from our (somewhat customized) schema definition vectors, not from the datalog itself. In retrospect I might have derived them from the datalog in the interest of greater generality.#2016-08-0600:49eggsyntaxWe have some custom code that takes things like:
[[:ti/install-type :ti/Location []
  "A location in the real world that can be the source or destination of a shipment"
  {:id [:uuid :identity "Unique ID for the location"]
   :name [:string :fulltext "The name of the loaction"] ...]]
and turns them into vanilla datomic schema defs.
#2016-08-0601:15lvhYep, makes sense :)#2016-08-0606:24levitanongHi all, is there a way to use multi-spec with a multimethod that takes two arguments instead of one? Iā€™d like to be able to dispatch on a key/value outside of the map being mult-specā€™d. Another way of describing the problem is: I have some map of the shape:
{:message/uuid uuid
 :message/action :some-action
 :message/payload {}}
:message/payload changes its shape based on :message/action.
#2016-08-0606:27levitanongAs far as I can tell, multi-specs only allow you to dispatch on a key/value inside of the data structure. i.e. it would only be able to do the following:
{:message/uuid uuid
 :message/payload
   {:payload/action :some-action
   ;; other data here
}}
This is what Iā€™m currently using, but I think it is more appropriate to place the :action as part of the message, and not the payload.
#2016-08-0620:24seancorfield@levitanong: consider the message as a whole -- although it has the same three keys in every case, the spec for :message/payload would be different in each arm of the multi spec.#2016-08-0623:06atrocheI started on a custom formatter for :cljs.spec/problems in cljs-devtools if anyone wants to carry on the work: https://github.com/gfredericks/schpec/issues/2#issuecomment-238005568#2016-08-0623:07atrocheiā€™ve done the small amount of plumbing required, and whatā€™s left is to experiment with making it look nice#2016-08-0702:48levitanong@seancorfield: thanks for replying :) so you're saying that I should multi spec the message as a whole? If that is so, then how would I change the spec for the :message/payload for each arm of the multi spec? Is there a way to alias a spec locally? Like if I have two different specs, ::warning-payload and a ::user-joined-payload, how would I be able to assign ::warning-payload to :message/payload inside the (defmulti message :warning ...) body, and likewise for the :user-joined branch?#2016-08-0702:52seancorfieldI haven't looked at the multi-spec stuff. That's just what my intuition said ought to work...#2016-08-0702:59seancorfieldBased on reading the documentation just now, it looks like you could have each method return an s/and of the message spec (with payload just specified as map? perhaps), along with a type-specific predicate for the payload value. #2016-08-0703:01seancorfieldI'm on my phone otherwise I'd crack open a REPL and try to produce a working spec for you. #2016-08-0703:57levitanongHmmā€¦ Wouldnā€™t that just end up specā€™ing a message against both the message spec and the payload spec?#2016-08-0703:58levitanongoh wait, I think I see what youā€™re saying#2016-08-0705:14seancorfieldYup, that's along the lines of what I was thinking. I don't know how it would hold up for generative testing -- I suspect you'd need to provide s/with-gen based on the ::pin-with-latlng spec -- but it seems reasonable for validation.#2016-08-0705:52levitanongI havenā€™t even gotten to generative testing yet šŸ˜›#2016-08-0705:54levitanongI hope rich hickey adds a way to decouple spec map keys from the specs of the keys themselves. I imagine I wonā€™t be alone with this use case.#2016-08-0705:55levitanongi.e. something like (s/def some-map-spec (s/keys :req [:some/key1 :some/key2] :req-dec {:some/key3 :spec-for-key-3}))#2016-08-0705:55seancorfieldWell, that's why you use namespaced keys -- even for non-qualified keys.#2016-08-0705:56seancorfieldI think that if you have :foo/bar that's a very specific entity in your application, but :bar is not.#2016-08-0705:56levitanongthe predicate with s/valid? works, but it seems hacky#2016-08-0705:57seancorfieldSo you can use :quux/bar and :what/bar as different specs for :bar in different contexts#2016-08-0705:57levitanonghmm. would you say then that Iā€™m being too specific with how iā€™m namespacing the :message/payload?#2016-08-0705:58seancorfieldIf you're using a qualified keyword, it should represent one very specific entity in your domain.#2016-08-0705:58seancorfieldWe have :ws.domain.member as a prefix for attributes that belong to "Member" entities.#2016-08-0705:58seancorfieldso things are unique in our domain#2016-08-0705:59seancorfield:message/payload is not very specific#2016-08-0706:00levitanongah#2016-08-0706:01seancorfieldIt's interesting... clojure.spec is very opinionated. And it speaks to how the Clojure/core team feel the language should be used.#2016-08-0706:01seancorfieldAnd if you use it that way, life is easy. If you fight it, life is hard.#2016-08-0706:01levitanonghahaha#2016-08-0706:04seancorfieldAs a guideline, any time I find something feels "hard" in Clojure, I ask here what I'm doing wrong (because it nearly always means I am doing something wrong). And the correct idiom is always easier...#2016-08-0706:04levitanongAny hint on what Iā€™m doing wrong? šŸ˜„#2016-08-0706:06levitanongCould it be that iā€™m structuring the shape of my messages wrong?#2016-08-0706:06seancorfieldI don't know that you're doing anything wrong but you might consider whether :message is the right prefix for your keywords... Is that specific enough?#2016-08-0706:06seancorfieldWhy is your payload nested?#2016-08-0706:07seancorfieldHaving it at the top-level feels a better fit#2016-08-0706:08seancorfield:message/uuid, :message/action are common and then the other keys would be dependent on the action -- why nest it?#2016-08-0706:09levitanongThe idea is to have it as a generic way to pass information between clients and the server through a websocket. The sender of the message identifies what it would like the recipient to do with the information provided. Sometimes, a payload is not necessary for some specific actions#2016-08-0706:10seancorfieldIs the uuid an inherent part of the data itself? Or is it part of some wrapper concept?#2016-08-0706:10levitanongSure, I could bring everything to the top level, but it feels neater to structure it this way. So that everything on the top level is something to do with the messageā€™s metadata: its uuid, who sent it, the timestamp, what action should be done, etcā€¦ And then the payload is just a whatever context is needed to perform the action.#2016-08-0706:11levitanongthe UUID is part of the wrapper. Itā€™s a way to facilitate some optimistic update error handling.#2016-08-0706:12levitanongor some other debugging purpose#2016-08-0706:12seancorfieldIf you have a dynamic payload, the discriminant has to be part of that, otherwise it isn't semantically consistent#2016-08-0706:12seancorfieldThat's why I suggested considering the message as a whole.#2016-08-0706:12levitanongSo you mean it should be :payload/action?#2016-08-0706:12levitanongah, or bringing everything up to the message#2016-08-0706:13seancorfieldIf it's really UUID + message then action should be part of the message#2016-08-0706:13seancorfieldIt depends on whether the message means something independent of the UUID.#2016-08-0706:13seancorfield(although maps are open so adding a UUID is OK anyway)#2016-08-0706:15seancorfieldThe real issue is that moving action outside payload makes it impossible to analyze payload on its own -- it has no type#2016-08-0706:15levitanongAs an aside: > If you have a dynamic payload, the discriminant has to be part of that, otherwise it isn't semantically consistent > The real issue is that moving action outside payload makes it impossible to analyze payload on its own -- it has no type These seem like very important concepts to learn. Is there a resource where one learns things like this?#2016-08-0706:16seancorfieldI don't know... it just seems... intuitive...?#2016-08-0706:16levitanongšŸ˜#2016-08-0706:16seancorfieldNot helpful, I know...#2016-08-0706:17seancorfield...but what is the atomic element here?#2016-08-0706:17levitanongI do suppose that the truly important parts here are the :message/action and the contents of :message/payload.#2016-08-0706:18seancorfieldSo "payload" is important and it must have an inherent discriminant (i.e., action must be part of it)#2016-08-0706:18levitanongI see...#2016-08-0706:18seancorfieldAnd message is uuid + payload#2016-08-0706:19levitanongI seeeee#2016-08-0706:19levitanongokay yeah it makes a lot of sense now#2016-08-0706:19levitanongAnd with that framework, if I were to rename the keys, I would rename payload to action, and action to type.#2016-08-0706:21seancorfieldWell, "payload" would be a map that contained "action" (as a type / discriminant) and a various keys that depend on the action ... and then it's an open question whether your "message" is uuid + payload map or whether it's a payload map with a :uuid key added#2016-08-0706:22seancorfield(i.e., nested or not)#2016-08-0706:23levitanongOkay, I think I have enough information to proceed confidently. šŸ˜„#2016-08-0706:23levitanongThanks so much, @seancorfield!#2016-08-0706:24seancorfieldGlad that was helpful... sometimes I'm not sure I'm being helpful šŸ™‚#2016-08-0706:24levitanongIā€™m sure you always are šŸ˜„ cheers! šŸ»#2016-08-0706:25levitanong(helpful, I mean)#2016-08-0706:25seancorfieldI'll raise a Snake Dog IPA to your šŸ» šŸ™‚#2016-08-0716:12gfredericksokay I'm going to try to slog through a schpec release today#2016-08-0716:37lvh@gfredericks: my wife seems bent on getting me to Portage Park (& assocā€™d pool) today so you might have a window betwixt merging xor and me putting up weighted-or šŸ˜‰#2016-08-0716:39gfredericksACK#2016-08-0716:40gfredericksmaking two releases isn't terrible either#2016-08-0716:49lvhsure šŸ˜„#2016-08-0721:30gfredericksI'm thinking putting a test runner in schpec#2016-08-0721:30gfredericksit would allow you to add metadata to functions you want tested, where you could also set up stubs/mocks/whatever#2016-08-0721:30gfredericksso theoretically you wouldn't need a separate test namespace if you didn't have other tests#2016-08-0721:30gfredericksis that silly or dumb?#2016-08-0813:34levitanongHello, I may have found a typo in the spec guide. http://clojure.org/guides/spec#_spec_ing_functions > The second :ret predicate takes as input the conformed result of the first predicate and verifies that start < end. The :ret value > spec is also an integer. Finally, the :fn spec checks that the return value is >= start and < end. I believe ā€œThe second :ret predicateā€ should be ā€œThe second :args predicateā€.#2016-08-0813:52donaldballI like putting my s/fdef before the fn definition, but when I have a :fn argument which needs a reference to the fn itself (e.g. to specify that the fn is idempotent, or maybe inverts on a given arg), itā€™s a forward reference and thus I must declare the symbol first. Would it be reasonable for the :fn arg to be called with a reference to the function under spec in its map arg?#2016-08-0814:13Alex Miller (Clojure team)@levitanong: fixed, thx#2016-08-0814:13levitanongHurrah! glad to help#2016-08-0814:15Alex Miller (Clojure team)@donaldball: what do you mean by ā€œreference to the functionā€ ? do you mean a key that refers to the function? what would the value be?#2016-08-0814:17donaldballIn this case, simply avoiding the necessity of declaring a forward reference to the function being specified, since s/fdef precedes defn#2016-08-0814:20Alex Miller (Clojure team)can you give me an example of what that would look like? I donā€™t think this is something we would do.#2016-08-0814:23donaldball
(declare convert)

(s/fdef convert
  :args ...
  :ret ...
  :fn (fn [{:keys [args ret]} map]
        ...
        (= ret (convert ...))))

(defn convert
  [...]
  ...)
#2016-08-0814:24Alex Miller (Clojure team)but what would your replacement look like?#2016-08-0814:24donaldballYou could add the function-under-spec to the map given to the :fn callback fn#2016-08-0814:25Alex Miller (Clojure team)it seems like a smell that you would be calling the function in the :fn spec#2016-08-0814:26donaldballThe other possible use case I was ruminating about the other day would be higher-order fns to specify that e.g. the specified fn is associative, commutative, has a given identity value#2016-08-0814:26donaldballMaybe these cases shouldnā€™t be specified here, but a priori it doesnā€™t seem unreasonable#2016-08-0814:50glvTo me, at least, that starts to seem overspecified, and kind of feels like ā€œthe function does what it doesā€. (I know you mentioned some cases where it would make more sense than that.) Iā€™ve been wrestling with the tension of how exhaustively to specify functions ā€¦ thereā€™s a point where it starts to feel a bit ridiculous. Thereā€™s still a place for prose documentation, and I think thereā€™s also still a place for traditional test.check property tests to deal with some properties.#2016-08-0814:53gfredericksyeah those things definitely smell a lot more like normal test.check tests#2016-08-0815:06glvThat tension has always existed in contract systems. How do you specify ā€œwhat the function is supposed to doā€ without simply restating the (possibly incorrect) code in the function? For some things thatā€™s fairly easy, but for other things it becomes very difficult. In such cases, the typical practice is to include some simple sanity checks, and use traditional testing methods to validate correctness. It seems to me like clojure.spec raises the bar for what makes sense to put in the spec (or contract) ā€¦ but the bar is still there.#2016-08-0815:06donaldballSo far to summarize that best practices for the :fn arg of s/fdef is strictly to specify some aspects of the relationship between specific args and return values, not properties of the function more broadly?#2016-08-0815:11glvThe rule of thumb Iā€™ve got so far is that I stop trying to make the fdef more thorough when the validation goal starts to overwhelm the usefulness of the spec for internal documentation purposes. (In other words, I donā€™t want the spec to be so complex that a programmer trying to work in the file has to stop and really think through complex details in the fdef to figure out how the thing is supposed to work.)#2016-08-0815:12glvI donā€™t know if thatā€™s the best practice or not, but for something that claims ā€œyou get a lot of different uses out of the same thingā€ that kind of balance strikes me as important.#2016-08-0815:39Alex Miller (Clojure team)yes#2016-08-0815:40Alex Miller (Clojure team)complicated :fn should (sometimes) also raise questions about whether your function is doing too much#2016-08-0912:16xcthulhu@gfredericks: You are the man for doing this - https://github.com/google/closure-library/pull/741#2016-08-0912:21gfredericks@xcthulhu: I can't resist a good arithmetic bug#2016-08-0912:24xcthulhuOkay I'm going to have to try to fix one now too. Google Closure doesn't handle UTF-8 properly, so every time I've tried to use it for taking SHA256 of unicode it's been borked.#2016-08-0912:25xcthulhuAlso I was curious how to speed up division, the way GMP does it is by divide and conquer reducing the problem to Mƶller-Granlund division: https://github.com/SaberMod/gnu-gmp/blob/faf7424576910eca222b0ddaea6bbe490e10e70e/mpn/generic/dcpi1_div_qr.c#2016-08-0915:48calvis
(s/def ::basically-an-integer
  (s/or :exact integer?
        :inexact #(and (number? %)
                       (zero? (- (int %) %)))))
(s/def ::basically-a-positive-integer
  (s/and ::basically-an-integer pos?))

user> (s/conform ::basically-an-integer 1.0)
;; => [:inexact 1.0]
user> (s/conform ::basically-a-positive-integer 1.0)
;; => ClassCastException   [trace missing]
running into this error, I realize itā€™s because s/and passes along the conformed value. is there an alternative to s/or or s/and that doesnā€™t add/pass along conformed information? would rather not have to think about the conformed value of the earlier spec when the spec essentially means ā€œitā€™s safe to call pos? on it"
#2016-08-0915:59eggsyntax@alexmiller: realizing that there hasn't been an alpha release in nearly a month makes me wonder: would you say that spec has pretty much stabilized at this point? Not that I'm requesting any sort of commitment to that, of course; just curious what your sense is.#2016-08-0916:19Alex Miller (Clojure team)no#2016-08-0916:19Alex Miller (Clojure team)Rich has just been busy/ooo#2016-08-0916:20Alex Miller (Clojure team)We have a ton of stuff queued up :)#2016-08-0916:20eggsyntaxMakes sense, thanks! Looking forward to checking it out šŸ˜„#2016-08-0916:21Alex Miller (Clojure team)@calvis Rich has suggested having both a flowing and non-flowing version of and and I think itā€™s likely that will eventually happen (probably and-> for flowing and and for non-flowing, although that changes the current meaning of and)#2016-08-0916:22calvis@alexmiller: yeah, that would be helpful. canā€™t really keep track which of my specs are orā€™s and which aren't#2016-08-0916:22Alex Miller (Clojure team)one option here is to use s/conformer in ::basically-a-positive-integer to unwrap#2016-08-0916:23Alex Miller (Clojure team)(s/def ::basically-a-positive-integer (s/and ::basically-an-integer (s/conformer val) pos?))#2016-08-1017:57Alex Miller (Clojure team)http://blog.cognitect.com/blog/2016/8/10/clojure-spec-screencast-customizing-generators#2016-08-1017:57Alex Miller (Clojure team)^^ new screencast on using models to make custom generators with fmap and bind#2016-08-1018:08eggsyntaxReally looking forward to this one šŸ˜„#2016-08-1018:17Alex Miller (Clojure team)Iā€™ll write a blog that expands on a few things in this too (eventually)#2016-08-1019:04eggsyntax^ Good call IMO. Just watched it, and it's great -- especially good for me, because we've been thinking lately about totally restructuring our domain model to be useful for spec as well as other stuff -- but I'm not sure I'd recommend it to someone as their 1st encounter with custom generators.#2016-08-1112:47rickmoynihan@alexmiller: super useful screencast thanks again! One thing I've not quite grokked yet is the full relationship between spec and test.check... I understand it's a dev time dependency for the generators... but should you ever be using raw clojure.test generator functions? Or should you use spec as a wrapper or use both?#2016-08-1112:48rickmoynihanI noticed that spec lazy-loaded test.check underneath too... not sure what the purpose of that is#2016-08-1113:00Alex Miller (Clojure team)spec.gen relies on test.check and exposes most of its generator functions#2016-08-1113:01Alex Miller (Clojure team)This anything that uses generators will require a dependency on test.check (we expect that to primarily be a test time dependency)#2016-08-1113:02Alex Miller (Clojure team)spec.gen wraps those functions to dynamically load them though, allowing you to build generators without incurring a load-time dependency on test.check#2016-08-1113:03Alex Miller (Clojure team)So you should always prefer using spec.gen as that lets you use specs in other ways without incurring that dependency#2016-08-1113:04Alex Miller (Clojure team)Maybe this itself would be a good blog topic :)#2016-08-1114:00borkdudeWhat is the reason :ret is not checked by t/instrument?#2016-08-1115:49gfredericksman we need some kind of canonical link for that question#2016-08-1115:54glvt/instrument is designed to help you ensure that youā€™re calling functions correctly, according to spec. Checking that functions (yours or someone elseā€™s) are implemented correctly is a different job, handled by st/check.#2016-08-1115:56minimalCould pin that ^#2016-08-1116:01andrewhrnot sure how well this will play with our history constraints#2016-08-1116:01gfredericksamend it with "please re-say&pin this message each morning"#2016-08-1116:02minimalI see plenty of old things pinned on other channels#2016-08-1116:02seancorfield@rickmoynihan: we use test.chuck for additional generators and we've taken the same approach of loading it dynamically as a test-only dependency /cc @alexmiller #2016-08-1116:02glvYeah, I think pinned stuff sticks around.#2016-08-1116:03gfredericks@seancorfield: you just generated a bunch of lazy vars?#2016-08-1116:03andrewhrtheres is some documentation update planned regarding this issue @alexmiller? Something the community could help you with? Sorry if the docs were already amended with this point, I didn't re-read them lately#2016-08-1116:06rickmoynihanalexmiller: thanks for the enlightening answer... And I think it would be another great blog topic šŸ™‚#2016-08-1116:08rickmoynihanseancorfield: yeah test.chuck is awesome (thanks gfredericks btw) how do you load it as a test time dependency only? Do you mean just through a dev/test profile? Or do you mean you're reusing core.specs lazy loading things somehow.#2016-08-1116:11ghadire :ret it in fact says it in the guide -- I think people have a false expectation of the tool to behave in a certain symmetric (for lack of a better term) way. But the real distinction is about development (`instrument` - :ret) vs test (`check` + :ret) time. As you know dev time vs test time is a little blurrier in a REPL.#2016-08-1116:12ghadiif you think of it (and teach it to people!) in terms of that distinction, it becomes intuitive#2016-08-1116:23Alex Miller (Clojure team)@andrewhr sorry, what issue are you referring to? (several things in this channel)
#2016-08-1116:24Alex Miller (Clojure team)the guide is updated with each alpha so should be in sync#2016-08-1116:25andrewhr@alexmiller: about :ret in s/instrument, @glv's answer https://clojurians.slack.com/archives/clojure-spec/p1470930899001878#2016-08-1116:26Alex Miller (Clojure team)guide discusses it, docstrings I think are up to date. where else should it be?#2016-08-1116:27andrewhrgreat! It's a pretty common question and we've not sure if the docs were up to date.. sorry for my laziness in not checking there before asking you#2016-08-1116:27Alex Miller (Clojure team)I started working on a reference page for spec which will be kind of somewhere between the docstrings and the guide, eventually Iā€™ll get around to finishing that#2016-08-1116:28andrewhrlooks promising! thanks for working on that šŸ™‚#2016-08-1116:48seancorfield@rickmoynihan: we wrap stuff we need to use in a function like this
(defn fn-string-from-regex
  "Return a function that produces a generator for the given
  regular expression string."
  [regex]
  (fn []
    (require '[com.gfredericks.test.chuck.generators :as xgen])
    (let [string-from-regex (resolve 'xgen/string-from-regex)]
      (string-from-regex regex))))
/cc @gfredericks
#2016-08-1116:49seancorfieldIn my build.boot file I have
(deftask testing-context
  "Provide main testing context."
  []
  (merge-env! :dependencies '[[com.gfredericks/test.chuck "0.2.7" :scope "test"
                               :exclusions [*/*]]
                              [javax.servlet/servlet-api "2.5" :scope "test"]
                              [org.clojure/test.check "0.9.0" :scope "test"]])
  identity)
#2016-08-1116:50seancorfieldthat task is chained in ahead of any testing tasks, to make the libraries available, but scoped so they donā€™t end up in artifacts (in case a build test accidentally got chained in with a testing task).#2016-08-1118:07borkdudehow do you say in spec: map of keywords and arbitrary values#2016-08-1118:08borkdude(s/map-of keyword? (constantly true))?#2016-08-1118:11Alex Miller (Clojure team)any?#2016-08-1118:13borkdudethanks!#2016-08-1118:16borkdudesorry if I'm asking the obvious, but why are there both or and alt? alt is used in sequences, but why isn't or used there?#2016-08-1118:23Alex Miller (Clojure team)this is covered briefly in the guide btw#2016-08-1118:24Alex Miller (Clojure team)alt describes alternatives that are in the context of a sequence#2016-08-1118:24Alex Miller (Clojure team)a spec with alt can thus be reused in different regex op contexts#2016-08-1118:25Alex Miller (Clojure team)whereas or represents a spec for a single value#2016-08-1118:25Alex Miller (Clojure team)there are situations where either would give you the same result but then there are cases where they would not#2016-08-1118:26Alex Miller (Clojure team)itā€™s better to start from the direction of what youā€™re specā€™ing - if it is syntax in a sequential context, then use alt, otherwise use or#2016-08-1118:27Alex Miller (Clojure team)(s/conform (s/* (s/alt :n number? :k keyword?)) [5 :a 10 :b 20 :c])#2016-08-1118:28Alex Miller (Clojure team)you are describing the structure / syntax of a sequential structure there#2016-08-1118:28Alex Miller (Clojure team)so use s/alt#2016-08-1118:28Alex Miller (Clojure team)(note that you canā€™t use s/or here)#2016-08-1118:29borkdude@alexmiller: thanks for explaining - I tried to find it in the guide, but couldn't understand it as clear as you explain it now#2016-08-1118:29Alex Miller (Clojure team)(s/conform (s/or :n number? :k keyword?) 100)#2016-08-1118:29Alex Miller (Clojure team)here youā€™re describing a single value - use s/or#2016-08-1118:30borkdudehmm, wait#2016-08-1118:30donaldballIs there any reason to avoid using (s/tuple ā€¦) for the :args value of s/fdef?#2016-08-1118:30borkdude
repl=> (s/conform (s/* (s/alt :n number? :k keyword?)) [5 :a 10 :b 20 :c])
[[:n 5] [:k :a] [:n 10] [:k :b] [:n 20] [:k :c]]
repl=> (s/conform (s/* (s/or :n number? :k keyword?)) [5 :a 10 :b 20 :c])
[[:n 5] [:k :a] [:n 10] [:k :b] [:n 20] [:k :c]]
#2016-08-1118:31borkdudeboth just work#2016-08-1118:32Alex Miller (Clojure team)both can work#2016-08-1118:32Alex Miller (Clojure team)but regex forms like cat are preferred#2016-08-1118:32Alex Miller (Clojure team)b/c the conformed result will tag the arguments (which tuples donā€™t)#2016-08-1118:33Alex Miller (Clojure team)again, this is syntax (the syntax of a function) so thatā€™s a better match#2016-08-1118:37borkdudehmm:
repl=> (s/conform (s/* (s/or :n (s/* number?) :s (s/* string?))) [[1 2 3]])
[[:n [1 2 3]]]
repl=> (s/conform (s/* (s/alt :n (s/* number?) :s (s/* string?))) [[1 2 3]])

StackOverflowError   clojure.core/implements? (core_deftype.clj:539)
#2016-08-1118:39Alex Miller (Clojure team)nice :)#2016-08-1118:39borkdudedid I break something? šŸ˜‰#2016-08-1118:40Alex Miller (Clojure team)I canā€™t spot anything immediately that would make that result acceptable :)#2016-08-1118:42Alex Miller (Clojure team)looks like a bug to me#2016-08-1118:42Alex Miller (Clojure team)I would not expect that value to actually satisfy that spec though#2016-08-1118:42Alex Miller (Clojure team)you are describing sequences like [1 2 3 ā€œaā€ ā€œbā€ 4 5 6 ā€œdā€ ā€œeā€]#2016-08-1118:43borkdudeyeah, weird right?#2016-08-1118:43Alex Miller (Clojure team)whereas the first s/or spec is for sequences like [[1 2 3] [ā€œaā€ ā€œbā€] [4 5 6] [ā€œdā€ ā€œeā€]]#2016-08-1118:43Alex Miller (Clojure team)if you can file a jira, that would be useful#2016-08-1118:43borkdudeah, then I found a good example to explain the difference?#2016-08-1118:44Alex Miller (Clojure team)really, I donā€™t think I would use s/or to describe either of those two kinds of inputs#2016-08-1118:45Alex Miller (Clojure team)well, it might make sense, it really depends on semantics#2016-08-1118:45borkdudeHow would you describe
[[1 2 3] [ā€œaā€ ā€œbā€] [4 5 6] [ā€œdā€ ā€œeā€]]
?
#2016-08-1118:45borkdudeJira ticket coming btw#2016-08-1118:47Alex Miller (Clojure team)(s/conform (s/* (s/spec (s/alt :n (s/* number?) :s (s/* string?)))) [[1 2 3] ["a" "b"] [4 5 6]])#2016-08-1118:47borkdudehttp://dev.clojure.org/jira/browse/CLJ-2002#2016-08-1118:48Alex Miller (Clojure team)or you could do (s/conform (s/* (s/or :n (s/coll-of number?) :s (s/coll-of string?))) [[1 2 3] ["a" "b"] [4 5 6]])#2016-08-1118:48Alex Miller (Clojure team)depends#2016-08-1118:48borkdudeI would choose the second probably too#2016-08-1118:48Alex Miller (Clojure team)thx for the ticket#2016-08-1118:50borkdude
(s/conform (s/* (s/spec (s/alt :n (s/* number?) :s (s/* string?)))) [[1 2 3]])
;;=> [[:n [1 2 3]]]
#2016-08-1118:50borkdude@alexmiller: so was the bug that I missed the surrounding s/spec?#2016-08-1118:51borkdudeThe guide says: "If you need to spec a nested sequential collection, you must use an explicit call to spec to start a new nested regex context. "#2016-08-1118:53Alex Miller (Clojure team)yes#2016-08-1118:53Alex Miller (Clojure team)well I mean Iā€™d say that was your error#2016-08-1118:54Alex Miller (Clojure team)but itā€™s a bug that you got a stackoverflow :)#2016-08-1118:54borkdudegarbage in, garbage out šŸ˜‰#2016-08-1118:54Alex Miller (Clojure team)it should have just said invalid#2016-08-1118:54Alex Miller (Clojure team)it wasn't garbage though, just not accurate#2016-08-1119:45stathissiderisis there an equivalent of declare but for specs?#2016-08-1120:39bfabry@stathissideris: you don't need to declare keywords before using them#2016-08-1121:16rickmoynihanseancorfield: Thanks... at first the require in that function makes me wince... but it's actually a nice trick being able to include the generator in the spec and use the spec in production context without the generator dependency. Nice! xgen/string-from-regex is too nice to use! šŸ™‚#2016-08-1121:33seancorfieldI use runtime require quite a bit. Very useful for conditional dependencies etc. #2016-08-1121:46donaldballI seem to recall that s/atom-of was bandied about shortly after clojure.spec was released. Is that something we should anticipate in core or has it been discarded as a bad idea?#2016-08-1121:49donaldballMy current use case calls for like an s/promise-of, but Iā€™m curious what the thinking is wrt specs for async values#2016-08-1122:01Alex Miller (Clojure team)it was never bandied about by anyone from core :)#2016-08-1122:01Alex Miller (Clojure team)I think itā€™s a bad idea#2016-08-1122:01Alex Miller (Clojure team)IMHO#2016-08-1200:40donaldballCurious if you could elaborate on why. I can see it being an impractical spec to validate outside of narrow test scenarios, but the documentation benefit seems worthwhile even so#2016-08-1202:10Alex Miller (Clojure team)because spec is about data and atoms are about state#2016-08-1202:11Alex Miller (Clojure team)it makes sense to me to spec the functions used to update an atom though#2016-08-1202:11Alex Miller (Clojure team)this is just my own thoughts btw, not sure if rich or stu would agree so take it with that perspective#2016-08-1206:36borkdudeI guess people could use set-validator! with a normal spec? Does anyone ever use it?#2016-08-1206:41mpenetI used to do exactly that with prismatic Schema. Works well#2016-08-1211:46Alex Miller (Clojure team)Yes#2016-08-1211:54borkdudeIt's all so... composable#2016-08-1217:55donaldballAm Iā€¦ doing something wrong here?#2016-08-1217:55donaldball
(clojure.spec.gen/vector clojure.spec.gen/char-alphanumeric 5)
AssertionError Assert failed: First arg to vector must be a generator
(generator? generator)  clojure.test.check.generators/vector (generators.cljc:446)
#2016-08-1217:58donaldballsorry, I see now, these vars turned into fns, presumably to facilitate lazy-loading#2016-08-1502:11lvhReminder: https://github.com/gfredericks/schpec exists and you might like it and it could use some eyeballs šŸ™‚#2016-08-1502:11lvhIā€™ll probably end up contributing regex-str spec at some point because I gotta write it anyway#2016-08-1502:19lvhWhat would be an appropriate name for a spec for a string that matches a regex?#2016-08-1502:34gfredericksWould re-matches and re-find be weird?#2016-08-1502:36lvhMy first attempt was matches-re?#2016-08-1502:37lvhI wonder how Iā€™d write the generator for has-re? or whatever; so far I only care about the entire re#2016-08-1502:37lvhI guess I could generate prefix string, re string, suffix string in parallel and then fmap str/join#2016-08-1502:45lvhhuh, with-gen has a branch for if the spec is a regex#2016-08-1502:45lvhoh, wait, that means clojure.spec generalized regex, not str regex šŸ™‚#2016-08-1509:34sandbagsPerhaps this is obvious but I just noticed that cljs.spec doesnā€™t seem to define an assert macro a la clojure.spec#2016-08-1509:34sandbagsseems relatively unlikely to be an oversight so I am wondering what I am missing#2016-08-1509:38sandbagsI guess i can provide my own version of assert*#2016-08-1510:38brabsterIf I want to always validate a function's input against an fdef spec, do I need to use the clojure.spec.test namespace? Or does the namespacing suggest that eg. clojure.spec.test/instrument is only for use in development and tests and I shouldn't use it otherwise?#2016-08-1510:50borkdude@brabster: good question.#2016-08-1510:52borkdude@brabster: Maybe you can separate out the args spec and refer to it in the fdef - and then call s/valid? in your function (or hook it up with :pre)?#2016-08-1510:54brabster@borkdude: you're right, I did that but it's quite a lot of extra typing if the individual arg specs aren't really reusable anywhere - and when I use a spec in a precondition the error message doesn't give any details of why the validation failed (just gives the name of the spec)#2016-08-1510:56borkdudehmm, I see#2016-08-1510:56brabsterplus I seem to frequently end up with what feels like duplication like (s/cat :foo ::foo :bar ::bar :baz ::baz) when I break the specs out - maybe i'm just writing bad clojure or something šŸ™‚#2016-08-1510:57brabster(tbh I'm struggling a little to work out when/when not to use namespaced keywords too :P)#2016-08-1510:57borkdudeIt's quite annoying to get the arguments as a vector that you can validate yourself, isn't it?#2016-08-1510:58borkdudewithout repeating#2016-08-1510:58brabsterYep#2016-08-1510:58borkdudeSay we have:
(s/def ::my-fn-args (s/cat :n number? :s string?))
(s/fdef my-fn :args ::my-fn-args)
(defn my-fn [n s] (if (s/valid? ;; now what?
#2016-08-1510:58borkdudeI get your point. Following the discussion.#2016-08-1510:59borkdudeWhat I'm also a bit uneasy with is that you have to repeat the argument names and that there's a possible source of duplication error.#2016-08-1511:01brabsterI imagine there are plans to integrate spec more deeply with core stuff like defn so maybe this is a bit painful because it's early days#2016-08-1511:01borkdudeIt would be cool if you could write a spec and then the function using the spec:
(defn my-fn ::my-fn-args ...)
Kind of like that.
#2016-08-1511:02brabsterLooks good to me#2016-08-1511:03brabsterWith the option to disable at runtime if you want to trade off the safety for slightly faster code#2016-08-1511:03brabsterWhat might be even nicer is combined spec conform and destructuring if that's possible#2016-08-1511:05brabsterThanks for the input @borkdude, more confident I'm not missing something really obvious now šŸ™‚#2016-08-1511:06borkdude@brabster: No problem, I'm also just beginning to see the dimension that spec adds to Clojure. Pretty excited about it.#2016-08-1515:02Ed@borkdude: seems really useful ... maybe it should do the conform so you specify something like (defn my-fn [{:keys [destructuring form]} ::my-fn-args] ...) and you could get the destructured pieces in your arguments?#2016-08-1515:32borkdudeI don't know yet, but something with less boilerplate would be awesome#2016-08-1515:35borkdude
(defn my-fn #[n number? :s string?] ...)  
without having to destructure vs
(defn my-fn [{:keys ....]] (s/cat :n number? :s string?) ...)
#2016-08-1515:37borkdudedon't know if it makes sense šŸ™‚#2016-08-1516:00brabsterlooks like a good start on the idea to me!#2016-08-1516:04borkdudeIs it possible to get the keys out of an s/cat without using exercise?#2016-08-1516:23Alex Miller (Clojure team)you can call s/form and grab the odd elements#2016-08-1516:25Alex Miller (Clojure team)(->> (s/cat :a string? :b keyword?) s/form (drop 1) (partition-all 2) (map first))#2016-08-1518:13borkdudeJust for fun: https://gist.github.com/borkdude/8d69f326b3d363a0b34e7b949b039992#2016-08-1519:58arohnerborkdude: if youā€™re doing serious parsing, you may also want to look at spectrum:#2016-08-1519:59arohnerhttps://github.com/arohner/spectrum#2016-08-1519:59arohnerspectrum.conform/parse-spec returns defrecords for the vast majority of specs. 100% coverage is a goal#2016-08-1521:44burbmaHaving an error specing a function that takes a seq of maps. Hereā€™s a gist with a cljc and error. https://gist.github.com/mmb90/e2e33cb526ed8c25549edbdb6e48f711#2016-08-1521:45burbmaIt seems to think it should be getting a map instead of a seq of maps. Is there a detail Iā€™m missing?#2016-08-1523:09seancorfield@mmb90: s/* is a regex for zero or more of these specs ā€” so youā€™re saying name-unicorns takes zero or more arguments, each of which should match the ::unicorn spec.#2016-08-1523:09seancorfieldI think you want s/coll-of for a collection of ::unicorn entities.#2016-08-1523:20smniel@seancorfield: The s/* is inside of an s/cat pulling out the first argument, so I thought the spec said that name-unicorns takes one argument which is a sequence of zero or more ::unicorns entities.#2016-08-1523:20seancorfieldThe s/cat is for the entire :args sequence.#2016-08-1523:21seancorfieldIn s/fdef you specify :args as a sequence ā€” as if you always invoked the function with apply.#2016-08-1523:24smnielYes. I understood that aspect, but I didnā€™t understand the way s/cat and s/* were combining to only specify one sequence. I was understanding as if the s/* were wrapped in s/spec which is required to turn a regex-op into a full spec or predicate.#2016-08-1523:24smnielDoes that sound correct?#2016-08-1600:05Alex Miller (Clojure team)Yes#2016-08-1607:33borkdude@arohner: do you have any examples how to use the result of parse-spec?#2016-08-1613:04sandbagsHas anyone come across a spec for Hiccup markup? Iā€™m just getting started and not sure Iā€™m up to trying to figure out if & how to express it.#2016-08-1613:08sandbagsat the moment I am copping out and using vector? šŸ™‚#2016-08-1613:08mpenetstart from a tag, then attributes then how they relate to each other (and themselves). should be relatively easy#2016-08-1613:09sandbagsthanks Max but Iā€™m in a hurry and not yet up to the nested spec stuff, if the answer is ā€œnoā€ thatā€™s fine too#2016-08-1617:42burbmaIn clojure I can do (gen/generate (s/gen int?)) ;; => 1094 or (gen/generate (s/gen boolean?)) ;; => false. In clojurescript I can do (gen/generate (s/gen int?)) ;; => -308. But in clojurescript I canā€™t do (gen/generate (s/gen boolean?)) ;; => Error: Unable to construct gen at: [] for: function cljs$core$boolean_QMARK_(x){return (x === true) || (x === false);}]. Whatā€™s happening in clj vs cljs to make this difference?#2016-08-1617:56seancorfieldSounds like no one has written the generator for boolean? in cljs yet.#2016-08-1617:56seancorfieldAre you on the very latest cljs version? If so, file a JIRA ticket.#2016-08-1618:06burbmaIā€™m on 1.9.216.#2016-08-1620:33Alex Miller (Clojure team)agreed, that should work#2016-08-1620:46seancorfieldI see specā€™ing of ns has hit master snapshot now @alexmiller ? My build just broke šŸ™‚ This is a good thing ā€” turns out we had (import ā€¦) in a test namespace instead of (:import ā€¦)#2016-08-1620:52seancorfieldOh, and a require instead of :require in another file...#2016-08-1620:55Alex Miller (Clojure team)yep#2016-08-1620:55Alex Miller (Clojure team)thatā€™s definitely the most common kind of failure I found in ns in the wild#2016-08-1620:57Alex Miller (Clojure team)there are a number of popular libraries that currently fail with the new specs - Iā€™ve filed PRs on all the ones I found and theyā€™ve all been merged, but afaik most do not yet have new released versions#2016-08-1620:57Alex Miller (Clojure team)Iā€™ll have some more guidance on some of this after we release an alpha#2016-08-1621:00seancorfieldWow, Iā€™m running into more and more of those ns bugs!#2016-08-1621:03seancorfieldWell, this is why we always do multi-version testing against master and our current selected release version!#2016-08-1621:09seancorfieldWe had a missing : in six files in all.#2016-08-1621:10Alex Miller (Clojure team)well I found instances in both Clojure and ClojureScript too so youā€™re in good company :)#2016-08-1710:56stathissideris@seancorfield: (require instead of (:require also breaks tools.namespace#2016-08-1710:56stathissiderisor makes it unreliable#2016-08-1712:06Alex Miller (Clojure team)I actually found a case of it in an old version of tools.namespace too (long since fixed)#2016-08-1714:00ikitommiHi. Back trying to integrate spec into http api-libs for (border) runtime validation, but need to figure out how to do selective conforming based on runtime parameters (which is needed to do this right). Wrote a small gist of how this is done with Schema - https://gist.github.com/ikitommi/17602f0d08f754f89a4c6a029d8dd47e. Any clues how to do this with spec? Writing conformers that read dynamic vars? Wait for the next alpha? šŸ˜‰#2016-08-1716:58sparkofreasonShort of writing a custom generator, is there a way to limit the depth of a generated tree from a recursive spec?#2016-08-1717:01gfredericks@dave.dixon why do you need to limit it?#2016-08-1717:01gfredericksthere might be a var for that actually#2016-08-1717:01gfredericksbut I'm still curious why that need comes up#2016-08-1717:04sparkofreason@gfredericks: I'm getting stack overflow exceptions. Actually not sure if that's the real issue anyway, since I only get the when I wrap the spec in with-gen to try and limit the number of entries.#2016-08-1717:10sparkofreason@gfredericks: Come to think of it, they way I'm specifying the tree probably isn't right, since there's an additional restriction on the leaves. Guessing that means I need to write my own generator anyway.#2016-08-1717:13gfrederickstest.check has a helper for recursive generators#2016-08-1717:14gfredericksyou have to be careful with sizing though :/#2016-08-1717:14gfrederickssizing and recursion have a weird relationship#2016-08-1717:29sparkofreason@gfredericks: using clojure.spec/*recursion-limit*. Sometimes I get answer from exercise, sometimes a stack overflow.#2016-08-1717:36gfredericksthe *recursion-limit* var and the test.check recursion helper are mutually exclusive#2016-08-1717:37gfredericksthe former only applies to the built-in generator clojure.spec gives you for for recursive specs#2016-08-1717:37gfredericksI need to write some documentation on sizing in test.check#2016-08-1718:36ikitommirelated to my runtime-coercion question. did a spike about using dynamic var to define the environment for the conformations: * specs define a special ā€œtypeā€ predicate as a first predicate. Allows type-based conformations & allows easy generation of api-docs (we are doing the spec->swagger, https://github.com/metosin/ring-swagger/issues/95) * type predicates know how to conform value in different environment (should be recursive): string, json & edn formats get all different treatment. Using a dynamic var sounds like a bad idea, but coudnā€™t figure out any other way to do this. Any comments?#2016-08-1808:47magnetophonI have a spec for a sorted set of positive integers in a wide range (say 0 to 1000000). I want those to be keys in a map. how do I spec/def that map? (I'm pretty new to clojure, so I hope I'm using the right terms here)#2016-08-1808:48magnetophonIn the normal use-case, there will only be up to a few dozen keys in the map.#2016-08-1812:30nha(Not sure if this is the right place to ask) When using clojure.spec with ClojureScript, is it able to figure out which keys are being used in order to minimize the build size ?#2016-08-1812:52Alex Miller (Clojure team)@magnetophon: does map-of satisfy?#2016-08-1813:10Alex Miller (Clojure team)@nha I donā€™t know, but I would guess not. @dnolen would know.#2016-08-1813:14dnolen@nha Dead Code Elimination (DCE) is purely a Google Closure thing#2016-08-1813:14dnolenstill eliminating keys is hardly going to save you anything at all - so I wouldnā€™t be all that worried about it#2016-08-1813:15dnolenkeywords are subject to constant optimization, i.e. single allocation and global usage#2016-08-1813:15dnolenthis also aids DCE#2016-08-1813:15Alex Miller (Clojure team)I think probably the question is whether the spec definition and registration for an unused key can be eliminated#2016-08-1813:16nhaIt was unclear, yes that is what I meant.#2016-08-1813:17dnolenah ok not currently, since itā€™s stored dynamically in an atom#2016-08-1813:19nhaAh I see. It may not be an intended use, I just spec'ed some server-side functions and would like to reuse some of these for validation/coercion on the client side.#2016-08-1813:19dnolen@nha I donā€™t see what that has to do with your question#2016-08-1813:20nhaWell embedding the validation/coercions fns for my DB (for instance) is not something I would like to have on my frontend code.#2016-08-1813:21dnolenso split those apart#2016-08-1813:21dnolenweā€™re not going to automate any of that stuff for you#2016-08-1813:21nhaOk separate namespaces then.#2016-08-1813:22nhaGet it, thanks for your quick answers šŸ™‚#2016-08-1813:22dnolennp#2016-08-1815:46magnetophon@alexmiller it partly does, but can spec ensure that each element in the set has one in the map, and the other way around?#2016-08-1815:48magnetophon@alexmiller maybe I should be using other data structures altogether. I'm learning closure while designing this thing.#2016-08-1815:57Alex Miller (Clojure team)to create a spec that makes assertions about a set and a map together, you will either need to spec a data structure that combines them or if you are passing them both as args together, that spec could be on the fdef args of that function#2016-08-1815:57Alex Miller (Clojure team)both are possible, Iā€™m just not sure what youā€™re doing from the description#2016-08-1816:05magnetophon@alexmiller Thanks! Do you have an example of the combined data structure way of doing it? The guide explains how to combine structures with s/keys, but I didn't see how to assert anything about the combination.#2016-08-1816:06Alex Miller (Clojure team)You're talking about both a map and a set so those would have to be combined into something else to talk about them together#2016-08-1816:06Alex Miller (Clojure team)Or maybe I totally misunderstand what you mean#2016-08-1816:07Alex Miller (Clojure team)You can combine arbitrary specs with s/and#2016-08-1816:09magnetophon@alexmiller I'll push some code and link you to it. one sec.#2016-08-1816:13magnetophon@alexmiller https://github.com/magnetophon/openloop/blob/master/src/openloop/states.clj#L101 is the sorted set, and #L115 is the map. how do I spec that each element has a corresponding element in the other structure?#2016-08-1817:31Alex Miller (Clojure team)That question only makes sense in some context where both are present#2016-08-1817:32Alex Miller (Clojure team)Either both parts of the same data value or both args in a call to the same function#2016-08-1817:33magnetophon@alexmiller you mean, I need to define a data-structure (or function, but let's go with DS for now) that combines both, right?#2016-08-1817:39Alex Miller (Clojure team)Where is the code that's going to use these things together? What does that need? It feels like you are doing things that are disconnected from the actual problem you are solving. Specs don't exist in a vacuum.#2016-08-1817:43magnetophonThere is no code yet. I'm brainstorming on how to model my problem by writing specs for the needed data structures. I'm learning clojure at the same time. Am I going about this the wrong way?#2016-08-1817:47magnetophonI have a proof of concept working. now I want to add a ton of features, but I'm trying to do so in an organized way, that won't lead to too many re-writes.#2016-08-1817:52magnetophonI thought if I have the data-structures and the states of the FSM that will be the brain of the thing, the rest of the code will be relatively easy.#2016-08-1817:53magnetophon@alexmiller Does that make sense?#2016-08-1818:03magnetophonI'm manipulating audio on disk into repeating loops in RAM. The data-structure in question is supposed to represent where in the disk-file each part of the RAM loop came from. I thought I'd use a sorted set for the indexes in the loop where I switch to another part of the source file, so that when I want to replace part of the audio with something else, I can easily find out which nodes to replace. The map contains, for each section, the rest of the data needed to construct the actual audio (so IOW: where does the audio come from for this section, how do I crossfade it, etc). I hope I'm making myself clear.#2016-08-1818:19seancorfield@magnetophon: It may help to think about whether the "sorted set of indexes" is just an implementation detail of you algorithm or whether itā€™s an inherently important data structure in its own right?#2016-08-1818:19seancorfieldIn other words, do you derive the set from the main data structure "internally", just for that piece of work, or are you passing it around "externally" along with the main data structure?#2016-08-1818:21seancorfieldIf the answer is "internal", then I donā€™t think you need to spec it, although you may spec functions that get passed both the set and the main DS (per Alexā€™s comment about specā€™ing via fdef).#2016-08-1818:21seancorfieldIf the answer is "external", then the best solution is to pass around a higher level DS that contains both the sorted set and the "main" DS as elements ā€” and have the spec for that higher level DS (since then it can check that the set element is consistent with the other DS element).#2016-08-1818:21seancorfieldDoes that help?#2016-08-1818:30Alex Miller (Clojure team)+1 :)#2016-08-1818:34magnetophon@seancorfield thanks. Needed some time to process that. I think the sorted set is an implementation detail. I might not even need it at all.#2016-08-1818:37magnetophonI thought I'd need it for performance, but I think I won't: I will be adding and removing indexes regularly, and for deciding which ones to remove or update I'll have to look at the value of the indexes. In practice there will be at most a few hundred indexes, so It'll be quick enough to go over them.#2016-08-1818:38magnetophon@alexmiller and @seancorfield Many thanks for your time and insight!#2016-08-1819:56bbrinckI've got a variable length vector (some elements are required, some are optional). I'm using cat to create a spec to check it, which works well. But when I want to generate examples, cat generates lists. Is there a way spec a vector this way?#2016-08-1820:00bbrinckI can achieve what I want with with-gen, but I believe it requires that I always have two specs - one for the list, as defined with cat and then another to wrap the default generator#2016-08-1820:11Alex Miller (Clojure team)Yeah, this is a common problem (regex syntax in vector) and a reasonable workaround#2016-08-1820:11Alex Miller (Clojure team)We have talked about creating vcat for this particular case#2016-08-1820:22bbrinck@alexmiller Glad to know I'm not totally missing something šŸ˜„. Thanks for the help. vcat would be very useful in my application.#2016-08-1820:36seancorfieldCanā€™t you just use coll-of with :into [] for that?#2016-08-1820:36seancorfieldAh, youā€™re talking about heterogeneous vectors, not homogeneous...#2016-08-1821:28Alex Miller (Clojure team)Yeah. Comes up a lot in macro syntax.#2016-08-1904:30lvhWhatā€™s the standard tactic for taming optionally recursive data structures? E.g. I have a foo that may also contain a foo under ::child; sampling the generator takes a long time because it generates huge foos#2016-08-1913:06gfrederickshuge foos because of the recursion?#2016-08-1913:18lvhYep.#2016-08-1913:19lvhRealistically the number of recursions for real data is like, uh, 1. Most applications wonā€™t do the recursion beyond that 1 level, but some might. Generating ā€œdeepā€ trees is fine and a useful test, but Iā€™d prefer to have it generate wider trees instead (because now generation is taking a while). Iā€™m not super sure yet if itā€™s actually taking a while or if itā€™s just CIDERā€™s repl view thatā€™s super slow or both.#2016-08-1915:22Alex Miller (Clojure team)have you messed with https://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec/*recursion-limit* at all?#2016-08-1916:55sparkofreasonHaving a similar issue, trying to control size of a tree. I don't think *recursion-limit* is working for me. Exercising the spec below sometimes gives results that violate the recursion limit, or even generate a stack overflow.#2016-08-1917:05sparkofreasonSeems a lot better behaved if I redefine as follows. Not sure if this is expected:#2016-08-1917:12sparkofreasonAt least I'm not getting stack overflow errors with this case, though trees appear to sometimes be fairly deep regardless of *recursion-limit*, but maybe that's what the docs mean by it being a "soft limit".#2016-08-1918:34bbrinckWhat do I need to require to use exercise-fn? With clojurescript 1.9.225 and test.check 0.9.0 I get the following error: #object[Error Error: Var clojure.test.check.generators/->Generator does not exist, clojure.test.check.generators never required]. This is in a namespace where I've required [cljs.spec :as s :include-macros true], [cljs.spec.test :as st :include-macros true] and [clojure.test.check.generators :as gen :include-macros true].#2016-08-1923:02lvhalexmiller: I have not; messing with a global dynamic var seems inappropriate, I want to affect this specific recursiveness#2016-08-1923:21Alex Miller (Clojure team)@lvh do it in a bindings#2016-08-2005:18kahunamooreI just posted a question to the Clojure ML - has anyone had any issues with danielz/system or stuartsierra/component when used with the latest alpha11?#2016-08-2005:21kahunamooreIā€™m using lein (not boot) if that makes a differenceā€¦ I started with a fresh closp template generated code which ran fine. Then I ran lein ancient upgrade on the project (using :allow-all) and now Iā€™m getting errors related to (require vs (:require ā€¦ afaikā€¦#2016-08-2005:55Alex Miller (Clojure team)I replied on list - there are bugs in system, I have filed PRs, and they have been merged. There are also upstream problems in monger, which system pulls in.#2016-08-2014:11eraserhdIs there a way to enable fdef return checking?#2016-08-2014:11eraserhdI thought there was, but can't find it.#2016-08-2015:24gfredericks@eraserhd: w.r.t. instrumentation, no -- the advice is to use generative testing to test the impl of functions#2016-08-2015:30eraserhdAh, OK.#2016-08-2015:30eraserhdI'm kind of using clojure.spec for debugging, though I guess I could do that.#2016-08-2015:31gfredericks@eraserhd probably not hard to write a helper aimed at your use case, and you could throw it into https://github.com/gfredericks/schpec if you want#2016-08-2015:32eraserhdHrmm#2016-08-2015:34eraserhdI have a weird tree data structure made out of vectors and numbers that has positional information in it. I did this mostly because I've written too much C. In Clojure, it's neither actually efficient, nor semantic. (You have to compute offsets for information.)#2016-08-2015:34eraserhdI'm using clojure.spec to migrate it to maps.#2016-08-2015:36eraserhdI was happy to discover I can actually specify the whole thing with spec. Though the conformed values aren't useful.#2016-08-2015:49eraserhd(It was a terrible terrible idea.)#2016-08-2019:29lvh@gfredericks: Is there anything I can do to help https://github.com/gfredericks/schpec/pull/10 land? My first thought was ā€œcontribute review timeā€, but the two other open PRs seem to be blocked on upstream responding to feedback.#2016-08-2019:33gfredericks@lvh I have some minor concerns but I think they're all improvements that can be made later without breaking anything. Go ahead and merge it, I'll release later today#2016-08-2019:34lvhcool, thanks!#2016-08-2019:34gfredericksSorry I had mistakenly thought everything was blocked on others#2016-08-2019:35lvhno worries, Iā€™ve maintained open sores projects, I get stuff falling through the cracks šŸ™‚#2016-08-2022:14gfredericks@lvh schpec 0.1.0 exists now#2016-08-2022:15lvh@gfredericks: awesome, thanks; I'll ship more features your way#2016-08-2104:01lvhIā€™m trying to run stest/check for my entire coebase. Itā€™s failing, but I canā€™t tell which thing is failing. All I get is:
ExceptionInfo Couldn't satisfy such-that predicate after 100 tries.  clojure.core/ex-info (core.clj:4725)
#2016-08-2104:02lvhItā€™s also not clear how I run this as part of clojure.test; puting it in the toplevel of a ns doesnā€™t work (`lein test` doesnā€™t pick it up) ā€” wrapping it in a deftest makes lein test see that ns as a test ns, but doesnā€™t seem to run the actual checking (no exception)#2016-08-2104:02lvhI only get the exception when manually evaling stest/check in a repl#2016-08-2105:24seancorfieldThe generative testing behind check is / should be very different from your unit testing with clojure.test...#2016-08-2105:25seancorfield... you should see something in the stacktrace explaining which identifies which spec's generator is unable to satisfy the predicate ...#2016-08-2105:41seancorfield... as you're writing specs, you probably want to exercise them to make sure the generators are working as expected: that would help you find problems like the one above as you created each spec. Not sure whether that's helpful advice @lvh?#2016-08-2109:12robert-stuttaford@lvh that particular error means that your generator can't produce something that matches your spec. i had this when i tried to s/and two s/keys together; it would generate for the first keys and never find anything that satisfied the second keys#2016-08-2111:39lvhrobert-stuttaford; yep; I know what the error means, but I have more than 1 spec#2016-08-2111:40lvh@seancorfield: Iā€™ve manually arrowed it down to a particular spec; and gen/sample + s/gen works fine#2016-08-2111:41lvh@seancorfield: Iā€™ve used test.check before: is the generative testing behind check different enough to not want to run it as part of lein test? Because thatā€™s how I run my normal generative tests#2016-08-2111:41lvhAlso, judging by the Guide, just running all the tests for all the namespaces is an explicitly supported feature#2016-08-2112:14lvhIs there any intention to make named? publicly available as a predicate such that s/and can maybe take advantage of if? Ideally also not-named šŸ™‚#2016-08-2113:34Alex Miller (Clojure team)Use ident?#2016-08-2113:34Alex Miller (Clojure team)That was added to core (named? really needs to be replaced in that code)#2016-08-2113:35Alex Miller (Clojure team)And #(not (ident? %)) #2016-08-2113:35Alex Miller (Clojure team):)#2016-08-2114:19gfredericksoooh found another thing that clojure allowed before specs were added: (fn a/b [])#2016-08-2114:20gfrederickswhich can easily be accidentally coded in situations like this: https://github.com/palletops/lein-shorthand/blob/06699b908df43a3029d6ef91b7591f17017a0405/src/com/palletops/shorthand.clj#L13#2016-08-2114:36Alex Miller (Clojure team)Yeah, I fixed a bunch of those, nearly always in a sloppy macro#2016-08-2114:37Alex Miller (Clojure team)algo.monads is actually broken on alpha11 due to a similar problem but with defn#2016-08-2117:03seancorfield@lvh I'm basing those comments on what Rich and others have said about generative testing not being part of "normal" (fast) unit testing. #2016-08-2117:05seancorfieldLike you, in the past, what little generative testing we've done has been okay in our regular Expectations-based test suite. But with the much more extensive use of generative testing in spec.test/check that is not a realistic approach. Unit tests should give near-immediate feedback and take "seconds" to run overall. Generative testing takes much longer so it needs a different approach. #2016-08-2118:03lvhSure, that makes sense; I still want to run the tests as part of a test suite even if it isn't the fast, rapid-feedback one though :)#2016-08-2121:25lvhIā€™m seeing a lot of failures with stest/check like:
6. { :failure clojure.lang.ExceptionInfo: No fn to spec #:clojure.spec{:failure :no-fn}, :sym clojure.core/ns, :spec 
defn and ns not having fns seems like not my problem though ā€” Iā€™m guessing the real solution is to explicitly specify namespaces?
#2016-08-2121:37Alex Miller (Clojure team)Looks buggy to me#2016-08-2122:29lvh@alexmiller: Iā€™m guessing that means ā€œyou should go file a bug?ā€ šŸ™‚#2016-08-2122:30lvhAlso probably something for core.typed to stop using versions of core.contracts that pull in old versions of core.unified that no longer work šŸ™‚#2016-08-2123:42atrochehas anyone been working on a macro that combines defn and fdef, in a similar way to schemaā€™s defnk?#2016-08-2202:49lvhatroche: nope but that sounds like a good idea šŸ™‚#2016-08-2202:50lvhany plans for a with macro for instrument/unstrument? That seems like an obvious thing to want šŸ™‚#2016-08-2202:50lvhIā€™m also wondering how to use check in a test suite#2016-08-2202:52lvhsomething like:
(deftest ^:generative check
  (stest/check (stest/enumerate-namespace 'obrysec.keychain)))
doesnā€™t do anything useful (doesnā€™t fail when itā€™s supposed to)
#2016-08-2202:53atrochedoes check have a return value you can make assertions on?#2016-08-2202:53atrochestandard clojure.test is assertions i mean#2016-08-2203:00Alex Miller (Clojure team)check returns results that you need to assert on
#2016-08-2203:01Alex Miller (Clojure team)@lvh yes bug, and yes on core.typed#2016-08-2203:01Alex Miller (Clojure team)@lvh the with instrument is interesting#2016-08-2203:02lvhMaybe I just havenā€™t seen the other use cases; it just seems obvious that Iā€™d use it in a deftest#2016-08-2203:03Alex Miller (Clojure team)You mean on instrument ? It's useful just to turn on instrumented specs at the repl#2016-08-2203:13lvhHm, OK, that makes sense#2016-08-2203:13lvhI filed that issue, Iā€™ll file another underā€¦ core.typed I guess#2016-08-2203:17lvhDone šŸ™‚#2016-08-2204:42seancorfield@lvh Have you seen how clojure.java.jdbc deals with instrument in the test namespace?#2016-08-2204:52seancorfieldFor stest/check, you could look at stest/summarize-results which prints a nice summary of the success/failure and produces a hash map you can easily assert about (using is, for example).#2016-08-2204:54seancorfieldWe use Expectations, and with test.check we (expect (more-of {:keys [...]} ...) (property-based checks go here))#2016-08-2205:05seancorfield(or you can (map stest/abbrev-result (stest/check 'my-ns/foo)) and make assertions on each of those results)#2016-08-2212:07brabsterI'm having a bit of pain with namespaced keywords - I have code that defines specs as namespaced keywords and expects to see those keywords used in calls to its functions - but it's super easy for a client to accidentally typo a new keyword into my namespace instead of referring to one that I've defined, and I can't catch the mistake if the keyword I've defined is optional#2016-08-2212:09brabsterso I defined a map key as ::s3/bucket-name, and in the client code mistyped ::s3/ucket-name - it seems like a really obvious dumb error but I'm not sure I can do anything to catch it without starting to custom-code validation?#2016-08-2212:10brabsteris there anything I can do in clojure to prevent keywords being defined into my namespace from outside (i.e. making that an error) or in spec to say that only the keys I've defined are valid - or have I misused something?#2016-08-2212:27donaldballFor the latter, (s/and (s/keys ā€¦) (s/map-of #{allowed-keys} any?)) is a possibility#2016-08-2212:45eraserhd@atroche I've always wanted a macro like this. Preferably allowing for with-test as well.#2016-08-2213:33brabster@donaldball nice, hadn't thought of that - only problem being the need to duplicate union of :req and :opt into it to achieve the effect... but helpful, thanks!#2016-08-2213:34donaldballCould probably wrap it up in a nice macro#2016-08-2213:36brabsteryeah - it doesn't seem like it would be a terrible idea to just have a flag or something on s/keys that made any keys that aren't in the union file#2016-08-2213:38brabsterI get the argument for the leniency about composing functions to build up maps from the rationale, but it seems like there's a case for the final step in such a process where I need to assert that I've produced a valid result.#2016-08-2214:41bbrinck@lvh @seancorfield Re: unit/generative tests, with my existing test.check tests, I've had luck using something like times (https://github.com/gfredericks/test.chuck#times) to run a small percentage of generative tests in my normal, fast feedback loop and then run the full amount less often.#2016-08-2214:42bbrinckI haven't yet tried that approach with the clojure.test generative testing, but it's been a useful technique so far#2016-08-2218:49jstokesim trying to come up with a way to define a spec for a map, and also a spec for key/value pairs in that map#2016-08-2218:50jstokesit also seems like something like this doesnā€™t work, which makes me think Iā€™m headed in the wrong direction#2016-08-2218:53angusiguessYou could define an entry as
(s/or :a ::a :b ::b :c ::c)
#2016-08-2218:54angusiguessThat would get you the values only, you might have to do something with tuples.#2016-08-2218:55angusiguess
(s/tuple #{::a} int?)
#2016-08-2218:55angusiguessMight start to get a little redundant#2016-08-2221:21Alex Miller (Clojure team)Entries are collections and maps can be treated as collections of entries#2016-08-2221:22Alex Miller (Clojure team)tuple is probably the best match for an entry#2016-08-2221:24Alex Miller (Clojure team)So you can spec a map as a coll-of an or of tuples#2016-08-2221:24Alex Miller (Clojure team)If you do so, you should also merge it with s/keys to get validation of values#2016-08-2303:10amacdougallI've been following http://clojure.org/guides/spec, and trying to apply it to some maze-building code I'm toying with. The core data structure is a grid (a two-dimensional vector) of cells (x, y, and a set of open exit directions). I have some specs which seem entirely reasonable to me:
(spec/def ::x integer?)
(spec/def ::y integer?)
(spec/def ::direction #{::n ::ne ::e ::se ::s ::sw ::w ::nw})
(spec/def ::exits (spec/coll-of ::direction :kind set?))
(spec/def ::cell (spec/keys :req [::x ::y ::exits]))
(spec/def ::row (spec/coll-of ::cell))
(spec/def ::grid (spec/coll-of ::row))
And then I have this pretty basic function, with a pretty basic fspec:
(defn grid-contains?
  "True if the supplied x and y coordinates fall within the grid boundaries."
  [grid x y]
  (and (<= 0 y) (<= 0 x)
       (< y (count grid))
       (< x (count (nth grid y)))))

(spec/fdef grid-contains?
  :args (spec/cat :grid ::grid :x ::x :y ::y)
  :ret boolean?)
But when I take the guide's advice and do this:
(require '[clojure.spec.test :as stest])

(stest/check `grid-contains?)
...I use 350% CPU for fifteen minutes and counting. What gives? Is there some unbounded complexity in my specs? Or is test.check just being very thorough?
#2016-08-2303:12amacdougallI did try this:
(stest/check `grid-contains? {:clojure.spec.test.check/opts {:num-tests 10}})
... which seems to be the right way to specify the options, according to https://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec.test/check, but it didn't make a difference.
#2016-08-2303:19amacdougall(spec/exercise ::grid) does fine, so I want to think that Clojure is able to generate correct test data without any issues. ĀÆ\(惄)/ĀÆ#2016-08-2303:20amacdougallFinal note: when I first ran stest/check, it hit null errors which revealed bugs (yay?). Once I fixed those bugs, I ended up in this situation.#2016-08-2303:58lvhCan I use spec to spec out a protocol? Just fdef the protocolā€™s method separately??#2016-08-2304:23seancorfield@lvh I seem to recall a JIRA issue around protocol functions, but it may be only for instrument...?#2016-08-2306:58brabster@amacdougall my first thought is maybe your spec is generating enormous grids under test which is chewing up cpu walking around it#2016-08-2307:13brabster@amacdougall just added a print to your fn and ran (stest/check `grid-contains? {:num-tests 1}), getting values like 32174406 809896496 -13491 -1 -3980 -2856 193086473 -53862035#2016-08-2307:15brabsterWould guess it's the (nth grid y) that's trying to walk along really long linked lists or something - out of time now but hope that helps!#2016-08-2310:45lvhhttp://dev.clojure.org/jira/browse/CLJ-1941 maybe#2016-08-2311:20magnetophonwhat's wrong with the above?#2016-08-2311:22magnetophonthe generator works, the s/def works if I comment out the with-gen parts, but the whole fails with: "Couldn't satisfy such-that predicate after 100 tries."#2016-08-2311:24minimalsort turns maps into lists of map entries#2016-08-2311:26magnetophon@minimal thanks.#2016-08-2311:26magnetophonwhat would be the proper way?#2016-08-2311:32minimalare you trying to gen a sorted-map? @magnetophon#2016-08-2311:32magnetophonyes#2016-08-2311:33minimali think this works (gen/sample (gen/fmap (fn [a] (apply sorted-map (flatten (sort a)))) (gen/map (s/gen int? ) (s/gen int?))))#2016-08-2311:35minimal
({} {0 -1} {0 -1} {-1 -1, 0 -4, 7 -3} {-1 -4, 0 -5, 1 -1} {-2 -2, -1 0, 0 3, 3 7} {-1 6, 0 -4, 24 -12} {-16 -21, -7 0, -4 16, -1 0, 0 -1, 1 -10} {-32 -7, 0 -23, 92 -13} {-13 1, -3 -1, -2 0, -1 1, 6 51, 14 11})
#2016-08-2311:37magnetophon@minimal It does! thanks! now I just need to decrypt that, but I'll manage.#2016-08-2311:37minimalJust turning it into what sorted map expects (sorted-map k1 v1 k2 v2 ā€¦)#2016-08-2311:40magnetophon@minimal I'm very new to clojure and lisp, so I'll have to study a bit to really get it, but I'll be fine.#2016-08-2312:41magnetophon@minimal Hmm, I sort of get what you're doing, and can use that with most types as the val of the map, but when I use (the type I need )[https://github.com/magnetophon/openloop/blob/master/src/openloop/spec.clj#L79] I get:
No value supplied for key: 0
#2016-08-2312:42magnetophonthe weird thing is: when I comment out the :src-index, it works. when I leave that in, but uncomment :length, it also works.#2016-08-2314:22amacdougall@brabster: Thanks for the pointerā€”maybe if I refine my spec to declare that the grid should never be more than, say, 10000x10000, and also specify that it must be made of vectors (for faster indexed access), test.check will be a bit less ambitious. On the other hand, should we expect such a system to generate arbitrarily large collections up to the literal limit of the integer type? I certainly see the benefit, but if a single test takes such a long time to run, how could you run a whole suite without a Bitcoin-mining level of resources?#2016-08-2314:29amacdougallLast night before giving up, I did change the grid spec to include :kind vector?:
(spec/def ::row (spec/coll-of ::cell :kind vector?))
(spec/def ::grid (spec/coll-of ::row :kind vector?))
But it didn't seem to make a difference. My understanding was that nth on a vector is O(1), because it is effectively (v n), and although I don't understand all the voodoo at a glance, https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/PersistentVector.java seems to confirm that.
#2016-08-2314:33amacdougallTo put it more generally: should developers expect to hand-tune specs to keep generative testing from trying to simulate the entire possible range of inputs? On reflection, the answer is yes: that's what generative testing is for. But in practice, generative testing on strings generates a bunch of garbage strings, it doesn't generate million-character strings. So why is this test trying to generate a 193,086,473-column grid? ...end of musing.#2016-08-2314:48magnetophon@minimal ah, found the problem: flatten flattens recursively, so when I have an uneven number of elements I get an error.#2016-08-2314:49minimalcool, sorry was afk for a bit#2016-08-2314:50magnetophonnp, you've been very helpful.#2016-08-2314:51minimalnp#2016-08-2315:08amacdougallI'm afraid that setting explicit bounds on my specs doesn't make a difference. I updated the relevant specs to limit both coordinates and grid dimensions:
(spec/def ::x (spec/and integer? (partial > 1000)))
(spec/def ::y (spec/and integer? (partial > 1000)))
(spec/def ::row (spec/coll-of ::cell :kind vector? :max-count 1000))
(spec/def ::grid (spec/coll-of ::row :kind vector? :max-count 1000))
...and added a prn to grid-contains? to see what test.check is trying to do... and I get this:
#'user/grid-contains?
user=> (stest/check `grid-contains? {:num-tests 1})
"grid-contains? grid 0 0"

OutOfMemoryError GC overhead limit exceeded  clojure.lang.PersistentVector$TransientVector.persistent (PersistentVector.java:568)
Perhaps there's something I could do to work around this, but I think for now I'll skip the generative testing.
#2016-08-2315:14amacdougallThe other main use of fdef is to instrument a namespace during development/test, which forces spec checking of all function calls, right? That seems like it would still be quite beneficial even without full-spectrum generative testing. šŸŒˆ#2016-08-2316:59raymcdermottHi guys, I want to do some checking of JSON data with spec, seems like it will be OK - any gotchas?#2016-08-2317:37magnetophon@minimal I found a solution ^^#2016-08-2317:40magnetophonwhen I s/conform some ok values it works as expected. when I try to conform values with a duplicate key, I don't get an error saying that, but instead I get:
Unmatched delimiter: )
#2016-08-2317:41magnetophonshould I report that as a bug in clojure, or am I doing it wrong?#2016-08-2317:43magnetophonHere is the code, and some sample data,if someone wants to try it out: https://github.com/magnetophon/openloop/blob/master/src/openloop/spec.clj#L79#2016-08-2317:50minimal@magnetophon cool! Yep (into (sorted-map)) is better. Not sure about the error#2016-08-2317:54magnetophon@minimal a clearer example of the error is:
(s/def ::test-type
  (s/map-of int? int?))
(s/conform ::test-type {41 1, 42 2}) ; ok
(s/conform ::test-type {42 1, 42 2}) ; unmatched delimiter: )
#2016-08-2317:54bbrinck@amacdougall Keep in mind that instrument will only check :args, not :ret or :fn specs#2016-08-2317:57bfabry@magnetophon you should be getting an exception from the reader creating a literal map with duplicate keys#2016-08-2317:57minimal@magnetophon you canā€™t have map literals with duplicate keys anyway. The unmatched delimiter must be a symptom of that#2016-08-2318:01bfabryI mean.. you also can't have maps with duplicate keys either. if you try and construct one one of the keyval pairs will disappear#2016-08-2318:01magnetophon@bfabry @minimal thanks. It's just that I recently read the clojure spec page, and wanted to try out the promise of usefull error messages.#2016-08-2318:02bfabrytry (s/conform ::test-type {:a 2, 31 3})#2016-08-2318:03bfabrythe thing is spec is never coming into play when you're trying to detect duplicate keys in a map. the map data structure just doesn't support that, and the clojure reader also doesn't support that. so it gets chucked out way before spec ever checks it#2016-08-2318:04magnetophonI was hoping to get a nice sorry, the keys are not unique message from spec. oh well... šŸ™‚#2016-08-2318:04bfabryI'm kind of surprised the reader didn't give you that error#2016-08-2318:04bfabrybecause that's what I get#2016-08-2318:05bfabry
user=> {1 1 1 1}

user=> clojure.lang.LispReader$ReaderException: java.lang.IllegalArgumentException: Duplicate key: 1
     java.lang.IllegalArgumentException: Duplicate key: 1
#2016-08-2318:07magnetophon@bfabry yeah I get that too, but not when I do it in spec#2016-08-2318:07bfabryok but you're not doing it "in spec"#2016-08-2318:07bfabrythe error happens pre spec#2016-08-2318:08bfabryyou literally cannot create a map with duplicate keys, so there's no way for spec to look at a map that has duplicate keys and give you an error about it#2016-08-2318:09magnetophon@bfabry I mean I don't get a usefull error from the reader when the faulty map is inside a conform or explain function.#2016-08-2318:10minimalI get
(s/conform ::test-type {42 1 42 2}) 
     java.lang.IllegalArgumentException: Duplicate key: 42
clojure.lang.LispReader$ReaderException: java.lang.IllegalArgumentException: Duplicate key: 42
             java.lang.RuntimeException: Unmatched delimiter: )
clojure.lang.LispReader$ReaderException: java.lang.RuntimeException: Unmatched delimiter: )
#2016-08-2318:10bfabryyeah that's a bit odd, but I don't think it's related to spec#2016-08-2318:10bfabry
user=> (identity {1 1 1 1})

clojure.lang.LispReader$ReaderException: java.lang.IllegalArgumentException: Duplicate key: 1
     java.lang.IllegalArgumentException: Duplicate key: 1
clojure.lang.LispReader$ReaderException: java.lang.RuntimeException: Unmatched delimiter: )
             java.lang.RuntimeException: Unmatched delimiter: )
user=>
#2016-08-2318:11minimalNo itā€™s not spec#2016-08-2318:11bfabryit's just because you've blown up inside the reader, so there's unconsumed input#2016-08-2318:11minimalOnce the reader fails, all bets are off and it gets more errors#2016-08-2318:12magnetophon@minimal when I run that line, I get the ) error.#2016-08-2318:12bfabrysomething about your environment is eating the first error, I guess#2016-08-2318:12minimalYeah#2016-08-2318:13minimalItā€™s a distracting error but the duplicate key error is the useful one. So it's confusing if itā€™s missing from your output#2016-08-2318:14magnetophon@bfabry @minimal ah, I was looking in the wrong place. both errors are there. sorry for the noise.#2016-08-2318:15bfabrynp#2016-08-2318:15minimalno worries.#2016-08-2318:19magnetophonWhat is meant by
By default map-of will validate but not conform keys because conformed keys might create key duplicates that would cause entries in the map to be overridden. If conformed keys are desired, pass the option `:conform-keys true'.
in the spec guide?
#2016-08-2318:21magnetophonafaik conform means to add metadata like the keys of a map. What do they mean by conforming keys? And why would that create duplicates?#2016-08-2318:28bfabryconforming means modifying the data to fit the spec#2016-08-2318:28bfabry
boot.user=> (s/conform (s/map-of int? (s/or :k keyword? :i int?)) {1 :foo 2 3})
{1 [:k :foo], 2 [:i 3]}
#2016-08-2318:28bfabryso you see there the values were conformed#2016-08-2318:28bfabryto tell you whether the value conformed to the :k branch or the :i branch of the spec#2016-08-2318:30bfabrybut you can use custom conformers, so hence the statement above#2016-08-2318:41Alex Miller (Clojure team)by default the keys in the conformed value of the map will be identical#2016-08-2318:42Alex Miller (Clojure team)but you can use :conform-keys true in the call to map-of to allow that to happen#2016-08-2318:44Alex Miller (Clojure team)
user=> (s/conform (s/map-of (s/or :i int? :s string?) int?) {5 5, "a" 10})
{5 5, "a" 10}
user=> (s/conform (s/map-of (s/or :i int? :s string?) int? :conform-keys true) {5 5, "a" 10})
{[:i 5] 5, [:s "a"] 10}
#2016-08-2318:45Alex Miller (Clojure team)and the reason not to do that automatically is that when you are altering the keys of the map, you might end up producing the same key and effectively erasing some of your transformed input, so we err on the side of safety#2016-08-2318:56magnetophonthanks. so it only makes sense if you have a branch in your key spec, and you want to know which branch was taken?#2016-08-2318:56magnetophonthat is not clear (to me) from the guide.#2016-08-2321:56Alex Miller (Clojure team)It only matters if your key spec conforms to something other than the original value#2016-08-2321:57Alex Miller (Clojure team)Given the common map keys - strings, longs, keywords, symbols - those are all likely to conform the same and it's a nonissue#2016-08-2321:57Alex Miller (Clojure team)So in most cases you can ignore this entirely#2016-08-2406:42tgkIā€™m sure my googling skills are just sub-par but I canā€™t seem to find an obvious way of running (clojure.spec.test/test) from lein e.g. as part CI integration#2016-08-2406:42tgkMaybe everyone has switched to boot while I wasnā€™t looking šŸ˜‰#2016-08-2407:01tgkThe approach Iā€™m taking now is to have a spec-test namespace loaded in dev environments and run -main using lein run -n spec-test/-main (or an alias) and have -main run clojure.spec.test/check and analyse the result. I guess this might be the obvious way of doing this.#2016-08-2407:54magnetophonIs it expected behavior that s/unform doesn't descend into vectors? IOW:
(s/def ::test-type (s/coll-of (s/cat :test-val int? )))
(s/conform ::test-type (s/unform ::test-type (s/conform ::test-type (gen/generate (s/gen ::test-type )))))
gives an error because the outer conform gets passed :test-val plus the actual value.
#2016-08-2408:08magnetophon@alexmiller I just noticed your response. Thanks, all clear now.#2016-08-2512:11nicolaHi all, i think this question already was answered, but - how to restrict collection of keys in map, i.e. no extra keys?#2016-08-2512:18tgk@nicola: You could write a predicate to cover that case and do (s/and (s/keys ā€¦) pred)#2016-08-2512:19tgkI assume there isnā€™t a standard way of doing it to be liberal in what you should be able to receive as data#2016-08-2512:19tgk(seems to be the Clojure philosophy)#2016-08-2512:32nicolaI've already read this thread - https://clojurians-log.clojureverse.org/clojure-spec/2016-06-09.html šŸ™‚ We want to use clojure.spec instead of json schema and we have many cases of miss-typed keys, so validating this would be helpful. :closed true would be nice, but let's start from custom pred.#2016-08-2512:39nicolaThen another question: s/and report errors of first failed predicate, could i compose two predicates - so both be applied, even if first failed?#2016-08-2512:40minimalThere is a excl-keys in https://github.com/gfredericks/schpec#2016-08-2513:07nicola@minimal: same problem with excl-keys, it uses internally s/and šŸ˜ž#2016-08-2513:08nicolaI want report both - extra-keys and ok-keys validation problems#2016-08-2513:09minimalYouā€™ll probably have to come up with your own custom version#2016-08-2513:10nicolaDo you mean own version of s/keys? šŸ˜ž#2016-08-2513:21minimal@nicola I guess so if it doesnā€™t report problems how you want#2016-08-2513:22minimalYou could use the clojure.set namespace to give more interesting reporting#2016-08-2513:23minimal
(defn keys-diff [expected provided]
  (let [expected (set expected)
        provided (set provided)
        missing  (set/difference expected provided)
        extra    (set/difference provided expected)]
    {:missing-keys missing
     :extra-keys extra}))
#2016-08-2513:25nicolathx @minimal , i've got it. It's not problem to write extra-keys predicate, but force validation and reporting both extra-keys and s/keys#2016-08-2513:26nicolai.e. error-greedy s/and#2016-08-2513:26minimalyeah could be useful#2016-08-2513:51xcthulhuAt one point someone here was trying to convert specs to BNF grammars - what came of that?#2016-08-2513:54xcthulhuThat could be incredibly valuable - it would be very useful if you could compile a spec into a .proto file, or even C++ code for a JNI interface.#2016-08-2514:01potetmHas anyone had experience setting up clojure.spec.test/check to run automatically (e.g. via lein test)?#2016-08-2514:03potetmI know you can wrap it in a (deftest foo (testing ...)), but the output of an is failure isn't as verbose as the output of check.#2016-08-2514:35bbrinck@potetm I use something like this (might need to be updated for latest alpha): https://gist.github.com/alexanderkiel/931387c7a86c1879b2267ad064067af7#2016-08-2514:38potetm@bbrinck Thanks! That'll do!#2016-08-2517:51sattvikAre there any examples of non-trivial uses of instrument, esp. showing how :spec, :stub, :gen, and :replace are used?#2016-08-2519:03Alex Miller (Clojure team)@sattvik have you read http://clojure.org/guides/spec ?#2016-08-2519:05sattvikYes, I have, and it has been really useful. Itā€™s just weā€™re trying to figure out how to use instrument and check together for some functions that may not be specced or only partially specced.#2016-08-2519:07sattvikThe videos on ClojureTV have also been useful, but a more thorough example that shows how to use some of the arguments to instrument to deal with things that might otherwise be considered part of a ā€œtest fixtureā€.#2016-08-2519:09Alex Miller (Clojure team)http://clojure.org/guides/spec#_combining_code_check_code_and_code_instrument_code has one example and I will write a blog on it eventually (but prob not for a while)#2016-08-2519:09Alex Miller (Clojure team)I think Stu might have a screencast in mind for it, but not sure#2016-08-2519:10sparkofreasonI've been getting this exception a lot lately when running check: "Additional data must be non-nil." Any thoughts on why?#2016-08-2519:11Alex Miller (Clojure team)that shouldnā€™t ever happen :)#2016-08-2519:12Alex Miller (Clojure team)it basically means that conform failed, but explain didnā€™t find any problems so ex-info was created with a nil map#2016-08-2519:13Alex Miller (Clojure team)unfortunately this exception happens while throwing the useful exception so itā€™s hard to diagnose#2016-08-2519:13Alex Miller (Clojure team)but itā€™s definitely a bug whatever it is#2016-08-2519:13Alex Miller (Clojure team)a bug in Clojure that is#2016-08-2519:13Alex Miller (Clojure team)and if you can figure out what it is, Iā€™d love a jira on it#2016-08-2519:28Alex Miller (Clojure team)itā€™s not too hard to hack Clojure itself to dump more info when this happens if you have the interest in doing so#2016-08-2519:31Alex Miller (Clojure team)FYI, Iā€™m doing a half day spec workshop at the Conj this year - tickets available now https://ti.to/cognitect/clojure-conj-2016#2016-08-2520:19sparkofreasonIf I can come up with a consistent minimal repo, I'll file a ticket.#2016-08-2520:22Alex Miller (Clojure team)thx#2016-08-2520:41sparkofreasonI'm going to venture a guess, though, based on some recent debugging, which is that it seems to occur when I have a function that returns another spec'ed function, and the returned function throws an exception while spec is checking it. Just a guess, will try to see if that stick consistently when I have a little time.#2016-08-2520:59Alex Miller (Clojure team)hmm, well I guess thatā€™s possible too (although Iā€™d put that one on you :)#2016-08-2521:08sparkofreasonApache library šŸ˜ž#2016-08-2521:20Alex Miller (Clojure team)ok, Iā€™ll put it on Apache :)#2016-08-2521:36shaun-mahood@alexmiller: Any expectation of previous spec knowledge for your Conj workshop, or can we come in blind?#2016-08-2521:36Alex Miller (Clojure team)nope#2016-08-2521:36Alex Miller (Clojure team)I will expect basic Clojure knowledge :)#2016-08-2521:38shaun-mahoodPerfect, I should be able to handle that šŸ™‚#2016-08-2521:39Alex Miller (Clojure team)for sure#2016-08-2521:54sebastianpoeplauThis might be a stupid question, but can anyone give me a hint how to spec the following function:
(defn my-apply [f & args]
  (apply f args))
(This is a simplified version of what I actually have.) My problem is in expressing that (count args) has to be equal to the number of parameters that f takes...
#2016-08-2600:38gfredericksThat sounds hard.#2016-08-2602:11Alex Miller (Clojure team)the hard part is that you donā€™t know how many params f takes#2016-08-2602:12Alex Miller (Clojure team)I donā€™t know that itā€™s profitable to spec such a thing at that level#2016-08-2605:53rymndhngis there a way to take a map schema of un-namespaced keys, and promote them to namespace qualified ones? -- I'm imagining something like:
(s/def ::id int?)
(s/def ::config (s/keys :req-un [::id]))
(s/some-conform-like-function ::config {:id 10})
;; -> {::id 10}
#2016-08-2608:42Alex Miller (Clojure team)this is kind of an aside and I assume you did this really for readability, but youā€™ll never see ::id printed - autoresolved kws are only read, never printed but you could certainly use a conformer to do this#2016-08-2608:43Alex Miller (Clojure team)with conform#2016-08-2608:47Alex Miller (Clojure team)
(defn qualify-keys [m]
  (zipmap (map #(keyword (str *ns*) (name %)) (keys m)) (vals m)))

(s/conform (s/and ::config (s/conformer qualify-keys)) {:id 10})
;;=> #:user{:id 10}
#2016-08-2614:27vikeriThis has probably already been asked, but can I somehow pprint the error messages thrown by (spec.test/instument)? They are pretty much impossible to read for what Iā€™m doing right now.#2016-08-2617:54Alex Miller (Clojure team)Not yet#2016-08-2619:39sveriHi, seems I missed a few updates and instrument-all is not in clojure.spec anymore. What is the idiomatic approach to enable the specs during tests with the latest alpha release?#2016-08-2619:42Alex Miller (Clojure team)(clojure.spec.test/instrument)#2016-08-2619:45sveri@alexmiller Ah, there it is and without arguments it instruments the ns, right?#2016-08-2619:45Alex Miller (Clojure team)it instruments everything in the registry#2016-08-2619:45Alex Miller (Clojure team)when you pass it no args#2016-08-2619:45sveriEven better, thank you very much#2016-08-2619:45Alex Miller (Clojure team)same thing for unstrument and check#2016-08-2620:32fenton(cross-posted from clojurescript...but no answer there): do others find it difficult to spec functions that use core-async for backend comms? This seems where stubbing would be very helpful, that is to stub out backend service calls. However since core-async based functions only return channels, not request responses...how do other people tackle this issue?#2016-08-2620:42dnolen@fenton thereā€™s no story for that yet#2016-08-2620:43fenton@dnolen ok...#2016-08-2620:45fentonok, i'll nut it over with some clojure devs here at our Vancouver, Canada, meetup...see what we come up with.#2016-08-2621:26fentonmaybe this came up before, but wondering if this is being thought about. There is some overlap in clojure.spec'ing and datomic schema spec'ing. Could these be unified?#2016-08-2621:53Alex Miller (Clojure team)eventually I expect that spec will be more intimately involved in both core.async and Datomic#2016-08-2621:54Alex Miller (Clojure team)but thatā€™s a ways off#2016-08-2621:58fenton@alexmiller cool. glad to hear that is the direction tho! šŸ™‚#2016-08-2622:38mjhamrickIs there a better way to do the following? I'm figuring the registry-ref is private for a reason, but I couldn't come up with another way to temporarily override the value of a spec. My usecase is that during runtime, it is okay for a spec to be nilable, but not during test time. Looking for any suggestions for this. I did try writing the specs separately, but that ended up with (quite a bit) of duplication since the spec I need to redef is nested a few levels down.
(defmacro with-spec-redefs
  [& forms]
  `(let [previous-registry# (s/registry)]
     (let [result# (do 
#2016-08-2715:58sattvik@fenton I believe exercise returns tuples of [val conformed-val].#2016-08-2716:02sattvik@mjhamrick I think the way to do that is via instrument. When you instrument, you can pass something like (stest/instrument some-ns/some-fn {:spec {`some-ns/some-fn (s/fspec ā€¦)}})`#2016-08-2814:30lvhIā€™m trying to use regex ops and running into some problems. I have two data structures naturally represented by a regex, the main one is of the shape: a*c, in my map data structure in order to get the item at that location, the get-in path is something like (ab)*c ā€” it seems that the problem is that nested regex operators ā€œmergeā€; so now for a fn that takes a*c and returns (ab)*c, I canā€™t specify its :args as (s/cat :a ::a) ā€” does that make sense?#2016-08-2814:32lvhLooks like thatā€™s intentional; found it in the guide šŸ™‚ > When regex ops are combined, they describe a single sequence. If you need to spec a nested sequential collection, you must use an explicit call to spec to start a new nested regex context. For example to describe a sequence like [:names ["a" "b"] :nums [1 2 3]], you need nested regular expressions to describe the inner sequential data:#2016-08-2814:50Alex Miller (Clojure team)Yes#2016-08-2814:50Alex Miller (Clojure team)That's intended#2016-08-2814:51lvhMakes sense ā€” youā€™d want to be able to parse both, and break them out šŸ™‚#2016-08-2816:36sveriHi, how do I spec functions with optional arguments?#2016-08-2820:34lvhsveri: usually you use regex ops to describe the args as they would be given to apply#2016-08-2820:34lvhso .e.g you can use spec/?#2016-08-2822:07potetmHow would you spec 2 arguments where one arg's validity depends on the other arg.#2016-08-2822:08potetmFor example: (fn [i-am-a-map i-am-a-key-in-the-map] ...)#2016-08-2822:31gfrederickspotetm: you can use s/and#2016-08-2822:32potetm@gfredericks perfect; just figured that out šŸ™‚#2016-08-2901:03potetmnext question: is there an ad hoc way to supply a :gen for function :args to stest/check?#2016-08-2901:05potetmI see you can s/def the args, then supply a gen. I'm not opposed to the notion that you must name something to use it elsewhere (even if that elsewhere is a test), but I thought there might be some sugar out there.#2016-08-2901:06sattvikI think you can do it with instrument.#2016-08-2909:16sveri@lvh Thanks, I just took ? It works as expected šŸ™‚#2016-08-2913:17lvhIs there any chance of a with-instrument macro, and/or a clojure.test + clojure.spec.test.check deftest macro a la defspec with test.check? (Too many similarly named words; I feel like I screwed that up)#2016-08-2913:34sattvikWe just started really looking into clojure.spec last week, and while we are still figuring out how to best leverage it, I definitely agree that a clojure.test integration and a with-instrument macro sound like useful additions.#2016-08-2914:58sveriHm, I am not sure, but, so far I would want to have everything instrumented during my tests#2016-08-2914:58sveriThats the first thing I do in my tests, turn on instrumentation#2016-08-2915:08sattvikYes, in general, you want to have everything instrumented. However, if you are using stubs or overriding specs, you may only want to do that within the scope of a few checks.#2016-08-2916:30xcthulhuHey! I'm trying to cryptographically sign data structures in Clojure/ClojureScript. The approach I am taking is to serialize my structures into a canonical form - namely recursively sorting the keywords of every hash-map in the structure, prior to serializing with transit. (1) Is there already something that does this for me?#2016-08-2916:30xcthulhu(2) Maybe this isn't the best practice, is there a better way?#2016-08-2916:31xcthulhu(3) If there isn't a better way, how do you enforce that a data structure is canonical in spec?#2016-08-2916:36lvh@sveri +1 on also wanting instrumentation with stubs/overrides as sattvik mentioned#2016-08-2916:36lvh@xcthulhu Hello I am a cryptographer#2016-08-2916:36lvhSolving a similar problem, turns out#2016-08-2916:37lvhFortunately I donā€™t particularly care about canonicalization or deterministic signatures#2016-08-2916:37lvhIā€™m personally using nippy, but I donā€™t think that has any particular canonical form guarantees#2016-08-2916:38lvh@xcthulhu How do you keep the hash-map structure in the serialized form whilst being sorted?#2016-08-2916:39xcthulhuSerialized form comes after the thing is canonical. Yeah, I've gone to pretty extreme lengths to keep all of my signatures deterministic.#2016-08-2916:40xcthulhuI'm doing ECC in ClojureScript and I don't believe I can get a reasonable secure RNG without resorting to RFC 6979#2016-08-2916:40lvhSure; when you say canonicalizing a data structure the first things that come to mind are ASN.1 C(X)ER#2016-08-2916:40xcthulhuYeah... I've implemented a fragment.#2016-08-2916:40xcthulhuI could go whole-hog. I was thinking of just using transit#2016-08-2916:41lvhFortunately RFC 6979 + ed25519 is how youā€™d want to do it anyway regardless of how much you care about that determinism šŸ™‚#2016-08-2916:41lvhHow do you ship the ClojureScript?#2016-08-2916:42xcthulhuRight now, I'm just publishing a JAR - https://github.com/Sepia-Officinalis/secp256k1#2016-08-2916:42lvh(The devilā€™s advocate argument against JS crypto usually goes by how you ship the source to begin with)#2016-08-2916:42lvhOh; OK ā€” why SECP256k1?#2016-08-2916:42xcthulhuBecause I started this when I was doing consulting for digital currency and it's what everyone uses because of BitCoin#2016-08-2916:42lvhEither way this is not a #spec discussion anymore so maybe we should move to #crypto šŸ™‚#2016-08-2916:43xcthulhuI should switch it to ed25519#2016-08-2919:57lfn3@fenton A thing Iā€™ve done previously with schema is wrap a channel with a validation to check anything put into the channel conforms to the schema. This causes exceptions to be thrown where youā€™re putting the unexpected value onto the channel which is really useful. Sure you could do the same thing with spec if you wanted to.#2016-08-2919:58lfn3Is there a way to attach a doc-string to a spec?#2016-08-2919:58fenton@ifn3 that sounds like a great use of specs#2016-08-2920:19Alex Miller (Clojure team)@lfn3 re doc string, not currently but feel free to vote on http://dev.clojure.org/jira/browse/CLJ-1965#2016-08-2920:20Alex Miller (Clojure team)@lvh re with-instrument, if you wanted to log a jira, that would help us remember this idea (seems quite reasonable)#2016-08-2920:21lvhwill do šŸ™‚#2016-08-2920:50lfn3@fenton https://gist.github.com/lfn3/0aaf4c420df206484372e9b7bbbe6c6f <- demo of using specs to validate things put on channels.#2016-08-2922:01agif I have a vector of maps:
(def accounts
  [{:name "Foo"} {:name "Bar"} {:name "Baz"}])
How do I define a spec that confirms to be one of the names, e.g.:
(s/def ::account-name ,,,)
: so when I do:
(gen/generate (s/gen ::account-name))
it would generate something like :account-name ā€œBazā€
#2016-08-2922:03bfabry@ag (s/def ::account-name (set (map :name accounts)))#2016-08-2922:04agalright thanks!#2016-08-2922:09emrehanHello all. I'm stuck at defining a map with undefined keys with clojure.spec. For instance, I'm defining a map with keyword keys and string values. I can define it as below with Schema library:
(require '[schema.core :as schema])

(def Data {schema/Keyword schema/Str})
(schema/validate Data {:x "str"})
> {:x "str"}
(schema/validate Data {:x 0})
> ExceptionInfo Value does not match schema: {:x (not (matches-some-precondition? 0))}  schema.core/validator/fn--8583 (core.clj:155)
How can I define such a map with clojure.spec?
#2016-08-2922:22bfabry@emrehan map-of https://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec/map-of (s/def ::a-map (s/map-of keyword? string?))#2016-08-2922:24emrehanthanks so much @bfabry šŸ™‚ I must have been missed this map-of function.#2016-08-2922:26bfabrynp#2016-08-2922:26agwhat if I have a collection and want a spec that satisfies for one of the items in the collection? e.g.
(def accounts
  [{:name "Foo" :type :foo}
   {:name "Bar" :type :bar}
   {:name "Baz" :type :baz}])

(s/def ::foo-or-bar-or-baz #_(any of those))
#2016-08-2922:28bfabry@ag not sure what you're saying, could you give a (s/valid ::account-name ...) => true example? also is accounts known at spec definition time?#2016-08-2922:45agso when I do (gen/generate (s/gen ::foo-or-bar-or-baz)) it would give me one of the maps of accounts vector#2016-08-2922:47bfabryassuming accounts is known at spec definition time, and accounts is reasonably small, then (s/def ::foo-or-bar-or-baz (set accounts))#2016-08-2922:59emrehanIs there a nice way to combine to clojure.spec definitions? I have a definition that must conform two defined schemas and I defined it as below:
(s/def ::spec3 (s/and #(s/valid? ::spec1 %) #(s/valid? ::spec2 %)))
Is there a better way to define this?
#2016-08-2923:00bfabrys/and accepts specs#2016-08-2923:01bfabryso (s/and ::spec1 ::spec2) is fine#2016-08-2923:02bfabrythat's actually not that clear from the docstring is it? it just says "spec-forms"#2016-08-2923:09emrehanI didnā€™t look at the spec but I tried that form. It didnā€™t work, maybe Iā€™m missing something.#2016-08-2923:10bfabryif ::spec1 conforms then the conformed value will be passed to ::spec2 instead of the original value#2016-08-2923:11bfabrymaybe that's your issue?#2016-08-2923:13bfabryif it is you could (s/and any? ::spec1 ::spec2) I suppose#2016-08-2923:15emrehanthanks for the tip. this behaviour is not obvious though. let me try that#2016-08-2923:17bfabryI think s/and conforming with the first spec has tripped a few people up so the docstrings are probably being worked I imagine. iirc the guide does cover it though#2016-08-2923:19agis there equivalent of clojure.test.check.generators/let? in clojure.spec?#2016-08-2923:25emrehanHereā€™s a simplified example#2016-08-2923:25emrehan
(s/def ::type (s/or :string string? :number number?))
(s/def ::a-map (s/map-of keyword? ::type))

(s/def ::id int?)
(s/def ::with-id (s/keys :req-un [::id]))

(s/def ::metadata (s/and #(s/valid? ::a-map %) #(s/valid? ::with-id %)))
(s/valid? ::metadata {:id 1 :n 1 :s "dsa"})
> true 

(s/def ::metadata (s/and ::a-map ::with-id))
(s/valid? ::metadata {:id 1 :n 1 :s "dsa"})
> false

(s/def ::metadata (s/and any? ::a-map ::with-id))
(s/valid? ::metadata {:id 1 :n 1 :s "dsa"})
> false
#2016-08-2923:28emrehanIt seems to me that s/and passes the return value of (s/conform ::a-map {:id 1 :n 1 :s "dsa"}) to the second conform. However, the when s/or is conformed the return value is different from the input.#2016-08-2923:31bfabryyes, conforming means changing the value to explain how it is true given the spec, ie if the spec has a branch#2016-08-2923:33bfabryI think maybe s/and has changed so it successively conforms the values now#2016-08-2923:34bfabryyes, it does#2016-08-2923:34bfabry
boot.user=> (s/conform any? {:id 1 :n 1 :s "dsa"})
{:id 1, :n 1, :s "dsa"}
boot.user=> (s/conform (s/and any? ::a-map) {:id 1 :n 1 :s "dsa"})
{:id [:number 1], :n [:number 1], :s [:string "dsa"]}
boot.user=> (s/conform (s/and any? ::a-map ::with-id) {:id 1 :n 1 :s "dsa"})
:clojure.spec/invalid
#2016-08-2923:37bfabryanyway, you have options. if you want to preserve the conforming behaviour, I would just re-order your predicates so that ::with-id comes first. if you want to do away with it then creating a new predicate with #(s/valid? ::a-map %) just around the spec that you don't want to conform anymore seems sensible#2016-08-2923:38bfabryOH#2016-08-2923:38bfabryor you could use the new merge function#2016-08-2923:38bfabrywhich is specifically for map specs#2016-08-2923:38bfabryboot.user=> (s/valid? (s/merge ::a-map ::with-id) {:id 1 :n 1 :s "dsa"}) true boot.user=> (s/conform (s/merge ::a-map ::with-id) {:id 1 :n 1 :s "dsa"}) {:id 1, :n 1, :s "dsa"}#2016-08-2923:39bfabrytotally forgot about that#2016-08-2923:40bfabrymerge appears to conform using the last spec. is that right @alexmiller ?#2016-08-2923:40bfabry
boot.user=> (s/conform (s/merge ::with-id ::a-map) {:id 1 :n 1 :s "dsa"})
{:id [:number 1], :n [:number 1], :s [:string "dsa"]}
#2016-08-2923:41Alex Miller (Clojure team)Yes#2016-08-2923:42Alex Miller (Clojure team)Merge does not flow conformed values like and, it's more like union#2016-08-2923:43agis it possible to have ā€œinlinedā€ spec in keys I donā€™t think itā€™s working for me.
(s/def ::foo (s/keys :req [(s/def ::bar string?)]))
#2016-08-2923:43ag
(g/generate (s/gen ::foo))
#2016-08-2923:44bfabryI think something named def should always be at the top level#2016-08-2923:44bfabryand in that particular case I don't think it will work because keys is a macro#2016-08-2923:45bfabryso the keys macro is passed the form '[(s/def ::bar string?)] instead of [::bar]#2016-08-2923:46Alex Miller (Clojure team)@bfabry s/and has always flowed conformed values. We may change it to not flow and add s/and-> still as both are sometimes useful#2016-08-2923:47bfabryah, cheers, not sure where I ever got the idea that only the first did#2016-08-2923:52Alex Miller (Clojure team)@ag s/keys always requires registered keys - that's part of the design that is talked about at http://clojure.org/about/spec#2016-08-2923:52agokā€¦ thanks..#2016-08-2923:56aghow do I create a spec for uuid? this doesnā€™t seem to work:
(s/def ::id g/uuid)
#2016-08-2923:57agI donā€™t want to use java.util.UUID/randomUUID I may have to make it later work on cljs side too#2016-08-2923:59bfabry@ag there's a uuid? predicate in core
#2016-08-2923:59agoh, okā€¦ thanks!#2016-08-3000:13agoh no.. I donā€™t think I know how to solve it with s/keysā€¦ the problem where I need to have a spec for a map where values picked up from a vector and associated values are together#2016-08-3000:16agso this is how I am generating a map with test.check.generators:
(g/let [acnt (g/elements ledger-accounts)]
      (g/hash-map
        :id                    (uuid-gen)
        :account-name          (-> acnt :account-name g/return)
        :account-type          (-> acnt :account-type g/return)
ā€¦
how can I do the same thing with spec?
#2016-08-3000:17aga map where related values pulled out of a vector?#2016-08-3000:21bfabry@ag are you using this spec for validation as well as generation?#2016-08-3000:21agright#2016-08-3000:21agI am planning#2016-08-3000:21agI donā€™t have a spec yet šŸ™‚#2016-08-3000:22agI am playing with clojure.spec, and I feel utterly stupid ;(#2016-08-3000:24agshould I be separating spec and generation parts?#2016-08-3000:24bfabrywell it depends#2016-08-3000:24bfabrybut it seems unlikely that you'll know all of the values of something like :account-name before runtime#2016-08-3000:26bfabryhave you read the guide?#2016-08-3000:26bfabryalso, I reckon the clojure spec screencasts by stuart holloway here https://www.youtube.com/channel/UCaLlzGqiPE2QRj6sSOawJRg might help. particularly the last one on customising generators#2016-08-3000:28agthatā€™s the thing, account-names are predefined. and types are predefined. I need to have a map with bunch of keys along with name and type where name comes from that vector and type is the associated type (from the same vector)#2016-08-3000:35bfabryok, honestly it's strange enough that there's probably not a way to express it and still get generators so you'll probably need to specify the generator like you did above, and do the spec portion separate#2016-08-3000:39bfabryI'd probably write the spec as
(s/def ::account-name (set (map :account-name leger-accounts)))
(s/def ::account-type (set (map :account-type ledger-accounts)))
(s/def ::id uuid?)

(s/def ::your-map (s/and (s/keys :req-un [::account-name ::account-type ::id] :gen YOUR-GENERATOR) #((set (select-keys ledger-accounts :account-name :account-type)) (select-keys % :account-name :account-type))))
#2016-08-3000:49Alex Miller (Clojure team)@ag it sounds like you are expressing what have been called hybrid maps#2016-08-3000:49Alex Miller (Clojure team)Which you can spec with a merge of keys and every of tuples of key-value pairs#2016-08-3000:49Alex Miller (Clojure team)I've given some examples here in the past#2016-08-3000:51agYeah Iā€™m watching Stuā€™s screencast ā€œcustomizing generatorsā€, looking into gen/bind, gen/tuple etc.#2016-08-3001:25agso why for simple predicates generator wonā€™t work:
(s/def ::more-than-five #(< 5 %))
(s/conform ::more-than-five 6)
(s/exercise ::more-than-five)

clojure.lang.ExceptionInfo: Unable to construct gen at: [] for: :user/more-than-five
#2016-08-3001:26gfredericks@ag that would be a hard thing to support in general#2016-08-3001:28gfredericksif you do (s/and integer? #(< 5 %)) it will at least make an attempt#2016-08-3001:28gfredericksbecause clojure.spec has a registry of generators for some of the built-in predicates#2016-08-3001:29agoh, okā€¦ thanks!#2016-08-3001:30gfredericksnp#2016-08-3001:47agonce again, how do I create s/def with a custom generator? I think I saw it somewhere?#2016-08-3002:03sattvikI beleive it is (s/def ::foo (s/with-gen spec gen))#2016-08-3002:22ag@sattvik Thanks!#2016-08-3007:24agerhmā€¦ how do I refer to a spec s/deffed in other namespace? I am trying to put spec in .cljc file, is that possible at all?#2016-08-3007:45agI think I found itā€¦ so if spec is in foo.clj then require [foo :as f] ā€¦ (s/exercise ::f/my-spec#2016-08-3007:46agstill canā€™t figure out how to refer to spec (and use it without having to prefix it with namespace)#2016-08-3010:25emrehanthanks all; clojure.spec/merge is exactly what I was looking for. I even read the whole API but I was quite sleepy.#2016-08-3010:26emrehanHow can I contribure to clojure.spec? docs/dev/testing/benchmarking?#2016-08-3012:16jannisHi. Using the latest ClojureScript version, I'd like to get all checkable syms (with cljs.spec.test/checkable-syms, filter them and call clojure.spec.test/check on each of them individually). However, if I do something like (doseq [sym syms] (st/check sym)) it tells me sym is unresolvable. This works fine in Clojure but in ClojureScript, am I supposed to pass in something else than symbols?#2016-08-3015:57spinningtopsofdoomI'm looking to make a spec that validates nested maps (e.g.` {:value {:foo :bar :key :value}}`).#2016-08-3015:57spinningtopsofdoomSo far I'm only able to get one level of nesting via spec/keys#2016-08-3015:59sattvikYou can use two different specs: one for the inner map and then use it in the outer map.#2016-08-3016:00sattvikThere are other options, but I think thatā€™s one of the easiest.#2016-08-3016:00sattvik
(s/def ::value :req-un [::foo ::key])
(s/def ::outer :req-un [::value])
#2016-08-3016:03spinningtopsofdoomI also have the need to have different specs for value (e.g. http://clojure.org/guides/spec#_multi_spec) . For example {:value {:foo :bar}} and {:value {:key :value}}#2016-08-3016:05sattvikWell, in that case, I think you could do with more specs:#2016-08-3016:06sattvik
(s/def ::foo-value (s/keys :req-un [::foo]))
(s/def ::key-value (s/keys :req-un [::key]))
(s/def ::value (s/or :foo-value ::foo-value :key-value ::key-value))
(s/def ::outer (s/keys :req-un [::value]))
#2016-08-3016:07sattvikjava.jdbc has such an example: https://github.com/clojure/java.jdbc/blob/master/src/main/clojure/clojure/java/jdbc/spec.clj#L27-L42#2016-08-3016:08sattvikHowever, not all of those specs are fully specified, e.g. ::user.#2016-08-3017:12agso how come this
(s/def ::my-map (s/cat :id string?))
(gen/generate (s/gen ::my-map))
#2016-08-3017:12aggenerates: ("pn7zcHi1WDk830ZltM5L5Yl82E0Ulā€)#2016-08-3017:12agand not a map {:id ā€œsomenonsenseā€} ?#2016-08-3017:14djwhitts/cat specifies a sequence. ":id" in that code is specifying a label rather than a map key#2016-08-3017:15agso what should I use instead, if I donā€™t want to use s/keys?#2016-08-3017:17bfabryyou should use s/keys#2016-08-3017:18djwhitthmm... I'm not sure (just learning spec myself). any reason you can't use s/keys?#2016-08-3017:19agbecause it doesnā€™t allow ā€œinline specsā€ e.g. s/keys (s/def :id string?)#2016-08-3017:20djwhittI think spec is intentionally trying to steer people away from inline specs like that#2016-08-3017:21djwhittyou could make a separate spec for the key and use req-un to allow unnamespaced keys if that's what you're looking for#2016-08-3017:21bfabry@ag giving every spec a fully qualified name is one of the core design goals of spec, things are going to get difficult if you try not to follow it#2016-08-3017:22agmkay#2016-08-3017:34agguys, I need a good spec for values of milliseconds after the Unix epoch. i.e.:`clj-time.coerce/to-long`#2016-08-3017:34agcan you help?#2016-08-3017:49bfabry@ag (int-in? 0 Long/MAX_VALUE)?#2016-08-3017:53agmmm thatā€™s too simple, I need to make sure values are of reasonable datetime range, something like from 1900 to 2100 maybe#2016-08-3017:58bfabrywell unix epochs don't go down to 1900#2016-08-3017:58bfabryso that's out#2016-08-3017:58bfabry(int-in? 0 (c/to-long #inst "2100-01-01"))#2016-08-3018:16Alex Miller (Clojure team)should really use int-in if youā€™re defining a spec (not just the predicate int-in?), as then you will get generator support#2016-08-3018:18Alex Miller (Clojure team)and there is a new fn for extracting the ms - inst-ms in core#2016-08-3018:20Alex Miller (Clojure team)so I would say (s/int-in 0 (inst-ms #inst "2100-01-01"))#2016-08-3018:23Alex Miller (Clojure team)assuming youā€™re cool with the year 2100 problem#2016-08-3018:24Alex Miller (Clojure team)inst-ms calls into the Inst protocol, which is extended to both Date and (if you use JDK 8), Instant. Could also be extended to a JodaTime Instant.#2016-08-3018:27agI donā€™t understand why this is failing:
(s/def ::account-name string?)
(s/def ::account-type keyword?)
(s/def ::description string?)

(s/valid? (s/keys :req [::account-name ::description ::account-type])
  {:account-name "pre-fund" :account-type :internal :description  "Pre-Fundā€})
#2016-08-3018:28sattvik:req requires namespace-qualified keys.#2016-08-3018:28Alex Miller (Clojure team)it has them ^^#2016-08-3018:28sattvikYou may want to use :req-un instead.#2016-08-3018:28Alex Miller (Clojure team):: will autoresolve and fully qualify#2016-08-3018:28Alex Miller (Clojure team)oh you mean in the data#2016-08-3018:28Alex Miller (Clojure team)yes#2016-08-3018:28Alex Miller (Clojure team)sorry! :)#2016-08-3018:29agoh, okā€¦ thanks a lot!#2016-08-3018:35agSorry for bugging you with bunch of noob questions. Trying to solve real problem with spec. If I donā€™t get this right sooner, would be asked to stop my experiments.#2016-08-3018:36Alex Miller (Clojure team)noob questions are good#2016-08-3018:36Alex Miller (Clojure team)I asked Rich a lot of noob questions at the beginning too :)#2016-08-3018:37agis there a way to pass original value (I dunno of spec) to custom-generator function, when itā€™s created with s/with-gen?#2016-08-3018:38agsomething like (gen/generate (s/gen (s/with-gen ::account-balance-updated #(gen-account-balance-updated %))))#2016-08-3018:39agI guess I can refer to ::account-balance-updated from inside the gen-account-balance-updated, yet thinking if thereā€™s more ā€œgenericā€ way#2016-08-3018:41agehā€¦ I guess I can just pass it as is#2016-08-3018:41agnevermind#2016-08-3018:41sattvikWell, you can do something like use gen/fmap or gen/bind.#2016-08-3018:43sattvikThough, I am not sure that really answers your questionā€¦#2016-08-3018:47Alex Miller (Clojure team)@ag in short, no#2016-08-3018:48Alex Miller (Clojure team)but you can define both the spec and the spec-with-custom-gen to do so#2016-08-3018:48agso I have a spec for a map structure, I need to use custom generator function where I would generate all the fields of that map, except of few selected, those should come from a predefined variable#2016-08-3018:49Alex Miller (Clojure team)itā€™s easiest to use gen/fmap and source it with (s/gen (s/keys ::a ::b)), then in the function just merge with the constant map#2016-08-3018:50Alex Miller (Clojure team)where ::a and ::b are the variable parts#2016-08-3018:52eraserhdWe are having a weird problem: https://gist.github.com/eraserhd/0aca4172b34c3c64f2e17f8c9107174d#2016-08-3018:54Alex Miller (Clojure team)@ag
(s/def ::a int?)
(s/def ::b int?)
(s/def ::c string?)
(s/def ::m (s/keys :req [::a ::b ::c]))
(s/def ::m (s/with-gen (s/keys :req [::a ::b ::c]) (fn [] (gen/fmap #(merge {::c "xyz"} %) (s/gen (s/keys :req [::a ::b]))))))
(gen/sample (s/gen ::m))
;; (#:user{:c "xyz", :a -1, :b 0} #:user{:c "xyz", :a 0, :b -1} #:user{:c "xyz", :a -1, :b 0} #:user{:c "xyz", :a -1, :b -1} #:user{:c "xyz", :a 0, :b 0} #:user{:c "xyz", :a -10, :b 1} #:user{:c "xyz", :a 3, :b -2} #:user{:c "xyz", :a -1, :b 40} #:user{:c "xyz", :a 45, :b -2} #:user{:c "xyz", :a -2, :b -7})
#2016-08-3018:56Alex Miller (Clojure team)@eraserhd whatā€™s the point of that gen/tuple?#2016-08-3018:56sattvikCould it be that the tuple is adding an extra layer of being a collection, i.e. instead of returning [task] it is returning [[task]]?#2016-08-3018:56Alex Miller (Clojure team)seems like you just need vector in that case#2016-08-3018:58Alex Miller (Clojure team)or you might just want to fmap with vector inside the fn#2016-08-3018:58eraserhd@alexmiller Er, we found out that spec conforms the generator's results to the spec.#2016-08-3018:59eraserhdWe had something with multiple values in the tuple, but deleted code until we found out where there was a problem.#2016-08-3018:59eraserhd(except that we didn't know that part)#2016-08-3019:02Alex Miller (Clojure team)yes, spec does not trust the generator#2016-08-3019:03Alex Miller (Clojure team)even custom gens must produce values valid for the spec#2016-08-3019:23sattvikHmmmā€¦ so whatā€™s the best way to create fixtures (for lack of a better term) with spec? This could range anywhere from setting up an environment within which to run the tests to testing that the output from the function under test matches a value generated from a replaced/stubbed function. check seems really useful for relatively pure functions, but having to orchestrate a lot via instrument seems clunky. Does anyone have any recommendations?#2016-08-3019:32Alex Miller (Clojure team)if you want fixtures and assertions, use a testing framework which has them ?#2016-08-3019:32Alex Miller (Clojure team)that is, wrap a clojure.test around instrument+check#2016-08-3019:32Alex Miller (Clojure team)instrument could go in a fixture too if it applies broadly#2016-08-3019:36sattvikYes, Iā€™ve done a bit of that. I have gotten it to work, but I was a bit unsatisfied with the result. I felt that the end result was a bit clunky and harder to understand/maintain compared to just using a traditional deftest.#2016-08-3019:37sattvikI will continue to experiment with it, but I was curious to see if anyone else had any experience with non-trivial testing using spec.#2016-08-3019:38Alex Miller (Clojure team)what was unsatisfying?#2016-08-3019:49sattvikWell, I am still a bit of a spec beginner (have been using it for less than a week), so there are things I may be missing. Some things that could help include: 1. More documentation around instrument, especially with examples of using the various options. 2. It would be nice to be able to call instrument with just options rather than (instrument (instrumentable-syms) opts). 3. The lack of a with-instrumentation macro makes setting up ā€˜fixturesā€™ harder. It was a simple one to write, but it would be nice to have it baked in. 4. Overall, specifying things through instrument doesnā€™t read as well. When this sort of thing is written out long-hand in code, it is easier to understand how the flow works. Instrumentation is sort of like defining callbacks before you need to use them, which removes some of the context from them. 4. The last one it is a little bit harder to describe. Instrumenting is very good about modifying things that will happen during the execution of the function, but sometimes I want to do something around the execution of the function. I can do that by specifying a test function and running spec on that, but it would be nice if there was a more direct way of doing that with spec.#2016-08-3019:58Alex Miller (Clojure team)1. for sure :) more will be coming eventually. 2. did you find clojure.spec.test/enumerate-namespace? it helps with building sym lists. 3. there is a ticket for that that I was just looking at. there are questions - feel free to weigh in. http://dev.clojure.org/jira/browse/CLJ-2015 4. I can see what you mean, prob worth seeing more examples 4. thatā€™s interesting. sounds like a check fixture? I assume you meant ā€œrunning checkā€, not ā€œrunning specā€ above#2016-08-3019:59gfrederickswoah CLJ tickets have exactly caught up with the calendar year#2016-08-3020:00Alex Miller (Clojure team)now if we can just slow down to 1 per year, that will always be true#2016-08-3020:01gfrederickssounds doable#2016-08-3020:01Alex Miller (Clojure team)thx#2016-08-3020:08sattvikRegarding (4), you are right. I set up a method to handle (is (checks? foo/bar {ā€¦}))` where the last two arguments are just used to invoke stest/check. I first started by putting my fixture in a separately defined test function that invoked the the function to test. It works, but adds a bit of noise (also there is not a way to automatically inherit the function spec of the original). In a case where the fixture is transparent, i.e. takes the same args/returns the same result, I was able to write a macro that too the var nameā€™s symbol and a higher-order function that served as the test fixture. That macro dereferenced the var giving the old value a local binding, and with-redefed the var to invoke the fixture (which took the old binding as an argument and returned a new function that behaved like the old one plus the fixture logic).#2016-08-3020:08sattvikI had to do with-redef because I couldnā€™t :replace the function under test.#2016-08-3020:14sattvikThat technique works when the fixture is transparent, but perhaps I want to create a fixture that has a different signature than the function I am testing. For example, I might be testing that a function will eventually invoke a stubbed function that will return generated data. I want to check that the return value of the function under test satisfies a predicate that is partly dependent on the data that comes from the stubbed function, not just the inputs.#2016-08-3020:15sattvikThis is all doable right now by creating such a function and properly specifying the test function and instrumenting the third-party function.#2016-08-3020:15sattvikThis is all doable right now by creating such a function and properly specifying the test function and instrumenting the stubbed function.#2016-08-3020:16sattvikI suppose it might make more sense with an exampleā€¦#2016-08-3022:08aghow can I create generate based on predefined vector? letā€™s say I have a vector [{:name ā€œAnnaā€}{:name ā€œDavid}] ..etc, I need a generator that creates vector with the same amount of elements, with added fields letā€™s say :age and :height?#2016-08-3022:15agI guess Iā€™ll just wrap things into gen/return and for#2016-08-3022:30gfredericks@ag so you want a generator that completes the maps?#2016-08-3022:31gfredericksYou could use gen/vector with a fixed length and gen/hash-map#2016-08-3022:40agI need to create a bunch of accounts with some predefined fields and some randomly generated fields, and need to create bunch of other structures associated#2016-08-3022:44gfredericks(gen/fmap #(map merge % predefined) (gen/vector (gen/hash-map ...) (count predefined))) @ag would something like ā† that work?#2016-08-3022:50aglemme tryā€¦ thanks!#2016-08-3100:12agerrrā€¦ so I have spec like this:
#?(:clj (defn uuid-str-gen [] (gen/return (str (java.util.UUID/randomUUID))))
   :cljs (defn uuid-str-gen [] (gen/return (str (random-uuid)))))

(s/def ::uuid (s/with-gen (s/and
                            string?
                            #(re-matches uuid-regex %)
                            uuid-str?)
                uuid-str-gen))
for uuid vals. A bit bulky, I couldnā€™t find anything better. my problem now, if another spec includes it it always generates same uuids. How do I fix that?
#2016-08-3100:13ag
(gen/sample (s/gen ::uuid))
("06cd3a07-98e0-4ee2-bce8-fa0b9e96d9e2" "06cd3a07-98e0-4ee2-bce8-fa0b9e96d9e2" "06cd3a07-98e0-4ee2-bce8-fa0b9e96d9e2" "06cd3a07-98e0-4ee2-bce8-fa0b9e96d9e2" "06cd3a07-98e0-4ee2-bce8-fa0b9e96d9e2" "06cd3a07-98e0-4ee2-bce8-fa0b9e96d9e2" "06cd3a07-98e0-4ee2-bce8-fa0b9e96d9e2" "06cd3a07-98e0-4ee2-bce8-fa0b9e96d9e2" "06cd3a07-98e0-4ee2-bce8-fa0b9e96d9e2" "06cd3a07-98e0-4ee2-bce8-fa0b9e96d9e2ā€)
`
#2016-08-3100:15agoopsieā€¦ thereā€™s gen/uuid I guess I should be using that#2016-08-3100:15agI need only the sting though#2016-08-3100:22agso why this is not working:
(s/exercise (s/with-gen string? (gen/fmap str (gen/uuid))))
#2016-08-3100:28agnvmdā€¦ got it
(s/exercise (s/with-gen string? #(gen/fmap str (gen/uuid))))
#2016-08-3100:31gfredericks:)#2016-08-3112:00borkdudeWas there s/instrument-all, which got removed later on?#2016-08-3112:01borkdudeI don't see it in my REPL, but I've seen it in a talk: https://www.dropbox.com/s/w1ktte022j5xy4o/Screenshot%202016-08-31%2014.00.08.png?dl=0#2016-08-3112:03minimal@borkdude just call instrument with no args#2016-08-3112:04borkdudeaah ok#2016-08-3112:04minimalit got changed at one point#2016-08-3112:33sveriAnybody experimented with using spec for form / input validation?#2016-08-3112:36martinklepsch@sveri I did#2016-08-3112:36martinklepsch@sveri very rough but maybe it helps https://gist.github.com/martinklepsch/e1366008c5a478b33c00d324314da4fd#2016-08-3112:43sveri@martinklepsch thanks, so basically its: 1. validate input with spec 2. try to parse result and display some useful text#2016-08-3112:44martinklepschright. Do all of that on a form level and do some basic dirty? tracking#2016-08-3112:45sveriOk, thank you very much, I will try some stuff šŸ™‚#2016-08-3112:58Alex Miller (Clojure team)Rich checked in a round of perf improvements to master yesterday which fixed some hot spots#2016-08-3114:22mschmeleWhen I try to run check on a function that Iā€™ve specā€™d with fdef, it fails with the cause ā€œUnable to construct gen at: [] for : clojure.spec$ā€¦"#2016-08-3114:22mschmeleam I missing something obvious?#2016-08-3114:25mschmelehttps://github.com/mschmele/CSPOC#2016-08-3114:26mschmeleCode is up there for anybody who wants to read through it#2016-08-3114:29minimal@mschmele you should (generally) use the predicates on their own instead of wrapping in an anonymous function. Otherwise you won.t get generators for the functions that have them.#2016-08-3114:30minimale.g. instead of (s/def ::id #(string? %)) use (s/def ::id string?)#2016-08-3114:31mschmeleOkay, I just saw an example that was formatted that way and it seemed to work. Changing that now#2016-08-3114:31minimalalso you should just use the specs inside fdef, no need for assert or valid which will probably give strange results#2016-08-3114:32mschmeleso something like#2016-08-3114:32minimaland you need to use s/cat for arg lists#2016-08-3114:32mschmele
:args [::accounts ::id]
#2016-08-3114:33minimalno#2016-08-3114:33minimalmore like :args (s/cat :acc ::accounts :id ::id)#2016-08-3114:34mschmeleah, thank you#2016-08-3114:35minimalhttp://clojure.org/guides/spec#_sequences#2016-08-3114:43mschmele@minimal thanks!#2016-08-3114:46minimalnp#2016-08-3121:48agis there a concise way to get a keyword? s/def but the one that always generates short keywords? because this looks awful;
(s/def ::keyword (s/with-gen keyword? (fn [] (gen/fmap #(keyword %)
                                               (gen/such-that #(and (> 10 (count %))
                                                                 (< 0 (count %)))
                                                 (gen/string-alphanumeric) 1000)))))
#2016-08-3122:24sattvikCould you do as part of the spec itself?#2016-08-3122:24sattvikMaybe (s/and keyword? #(< (count (name %)) 10)?#2016-08-3122:24sattvik@ag ^#2016-08-3122:28bfabry@ag
(s/def ::short-keyword (s/with-gen keyword? (fn [] (gen/fmap #(keyword %)
                                               (gen/such-that #(and (> 10 (count %))
                                                                 (< 0 (count %)))
                                                 (gen/string-alphanumeric) 1000)))))

(s/def ::keyword ::short-keyword)
(s/def ::keyword2 ::short-keyword)
#2016-08-3122:30bfabry@sattvik doing it that way makes the spec validate that the keyword is short, which you may not want, and also makes for slow generators#2016-08-3122:40sattvikWell, if the idea is to accept any keyword but only generate short ones during testing, then supplying the generator at check time is probably what you want. test.check has things like c.t.c.generators/resize that can fix the generator to a manageable size, e.g. 10.#2016-08-3122:40bfabrythis is true#2016-08-3122:40sattvikUnfortunately, thatā€™s not one of the functions clojure.spec mirrors.#2016-09-0100:26sattvikCurious. When working with higher-order functions, instrumentation will result in the execution of a function argument repeatedly. Which, isnā€™t great when the function has side effects.#2016-09-0100:29gfredericksyou probably need to stub things?#2016-09-0100:32sattvikEhā€¦ itā€™s kind of a weird situation. I have a function that performs some assertions for testing. It is invoked by the function under test. Under normal circumstances, my function with assertions is only invoked once per test. However, as the function under test has its parameters specified, the conformance checker ends up running my test function scores of times.#2016-09-0100:34sattvikI think one workaround is to instrument the function under test with less demanding specification (something like fn?). In the end, I just use a mutable cell to hold the data I want to check and have my test code update that cell. I can count on the last invocation of the function to be the one I care about.#2016-09-0100:35gfredericksyou also get to override specs#2016-09-0100:35gfredericksso maybe you could test it in two ways#2016-09-0100:35gfredericksonce where you have the function doing the assertion and you make sure it gets called only once, and another test without that where the spec gets exercised#2016-09-0100:36sattvikWell, doesnā€™t overriding specs only be done when using check? I am not doing that. I am only instrumenting for input argument validation.#2016-09-0100:39gfredericksthat might be true#2016-09-0101:40seancorfieldIs there any sort of standard pattern for specā€™ing a function that curries its arguments?
(defn my-fn
  ([a] (fn [b] (my-fn a b)))
  ([a b] ā€¦ a ā€¦ b ā€¦))
#2016-09-0101:41seancorfieldSpecifically, how to handle the fact that :ret depends on whether you provided one or two arguments...#2016-09-0101:45sattvikHmmā€¦ good question. My guess is that you would use alt for the different return types and then an fn to tie them together to the passed args. Iā€™m not sure if there is a better way, though.#2016-09-0102:17seancorfield@gfredericks all of a sudden, Iā€™m getting this exception from test.check ā€” any pointers? clojure.lang.Compiler$CompilerException: java.lang.ClassCastException: clojure.lang.AFunction$1 cannot be cast to clojure.lang.MultiFn, compiling:(clojure/test/check/clojure_test.cljc:95:1)#2016-09-0102:19seancorfield@sattvik yeahā€¦ that feels kinda ugly thoā€™ā€¦ Iā€™m hoping thereā€™s a cleaner way...#2016-09-0102:23sattvikHmmā€¦ I wonder if it is possible to do (s/or :unary (s/fspec ā€¦) :binary (s/fspec ā€¦))#2016-09-0102:28gfredericks@seancorfield that smells like this one thing hold on#2016-09-0102:29gfredericks@seancorfield http://dev.clojure.org/jira/browse/TCHECK-113#2016-09-0102:30seancorfieldPerfect! Thanksā€¦ Iā€™ll add that Leiningen setting!#2016-09-0102:31seancorfield(Iā€™m expanding the clojure.spec coverage for java.jdbc and ran into that for the first time)#2016-09-0102:32gfredericksthis is all my fault for not having a new test.check release ready yet#2016-09-0104:49seancorfieldDon't sweat it. test.check is already great!#2016-09-0104:52seancorfieldMaking progress with spec'ing java.jdbc. Some good, some frustrating simple_smile #2016-09-0107:26sveriHi, I have trouble specing & args. What I want is (f [a1 a2 & ax]) where ax can be zero or more (s/*) and it must have an even number of elements. This is what I thought should work: (s/* (s/and any? #(even? (count %)))) But it does not, instead it works for (f 1 2 3) but not for (f 1 2 3 4)#2016-09-0108:00sveriIs it because spec takes the args vector apart and checks every element of the vector?#2016-09-0112:43Alex Miller (Clojure team)The s/and there is in the spot for each element of the arg vector so that doesn't make sense#2016-09-0112:45Alex Miller (Clojure team)(s/& (s/* any?) #(even (count? %)))#2016-09-0112:45Alex Miller (Clojure team)Is one solution - s/& applies a predicate in addition to a regex match#2016-09-0112:46Alex Miller (Clojure team)Or you could make the pairs explicit in the regex if that makes sense#2016-09-0112:47Alex Miller (Clojure team)(s/* (s/cat :x1 any? :x2 any?))#2016-09-0112:48Alex Miller (Clojure team)Or actually since you require at least 2 args that should be + not *#2016-09-0112:49Alex Miller (Clojure team)That's probably preferred, esp if you can give better names to x1 and x2#2016-09-0112:49sveriZero / 2 / 4 / ...args are fine#2016-09-0112:50sveriI just wonder, if I look at: (s/ (s/cat :x1 any? :x2 any?)) I would assume it should have two arguments, not more, not less. But, I think its the s/ around it, that makes it work for more inputs, true?#2016-09-0112:53Alex Miller (Clojure team)Yes - it's just like regex#2016-09-0112:53Alex Miller (Clojure team)It's a repetition of pairs#2016-09-0112:54Alex Miller (Clojure team)@sattvik: you can override specs in instrument to use a simpler spec#2016-09-0112:55Alex Miller (Clojure team)@seancorfield re currying, you can use a :fn spec if you need to make an assertion about the relationship between :args and :ret#2016-09-0113:01sveri@alexmiller Thank you very much šŸ™‚#2016-09-0113:30mschmeleIs it generally considered bad practice to allow functions to have nilable return values?#2016-09-0113:30mschmeleIt makes having functions that pass spec tests quite a bit easier, but I feel like it kinda defeats the purpose to some extent#2016-09-0113:49sattvik@mschmele Itā€™s perfectly reasonable, if nil means something like ā€˜not foundā€™ or ā€˜emptyā€™.#2016-09-0113:50sattvik@alexmiller Thatā€™s true. Itā€™s just that I was a little surprised (but perhaps I shouldnā€™t have been) that conforming a function argument involved repeated invocations of the function.#2016-09-0113:51mschmeleThatā€™s what Iā€™ve been using it for in simpler functions like get-from-id for example. I guess my real question (and I probably should have been more specific), applies to more complex functions like transfer which would take two accounts that canā€™t be nil#2016-09-0113:52mschmeleDespite having validation in the function, Iā€™m having trouble getting it to pass spec check#2016-09-0116:09seancorfield@alexmiller Yeah, I know I can deal with currying via :fn but that still makes for a fairly complex spec ā€” Has there been any discussion of making it easier to spec multi-arity functions?#2016-09-0116:19Alex Miller (Clojure team)Not in regards to this#2016-09-0116:20Alex Miller (Clojure team)My experience has been that it is comparatively rare for the ret spec to rely on the args arity#2016-09-0116:21Alex Miller (Clojure team)And when it does :fn is the way to talk about that constraint#2016-09-0116:25seancorfieldWell, with a curried function, the :ret will either be a function (of the remaining argument(s)) or a result...#2016-09-0116:26seancorfieldSo
(defn foo
  ([a] (fn [b] (foo a b)))
  ([a b] (* a (inc b))))
=> :ret would be int? in the two arg case but some fspec in the one arg case
#2016-09-0116:27seancorfieldIf fspec / fdef supported multi-arity, this could be neater...#2016-09-0116:30Alex Miller (Clojure team)Well I'd say curried functions are not idiomatic in Clojure :)#2016-09-0116:30seancorfieldšŸ˜#2016-09-0116:31Alex Miller (Clojure team)But the main place I've run into this is with the core seq functions with transducer arity#2016-09-0116:31seancorfieldWhat about multi-arity functions in general?#2016-09-0116:32Alex Miller (Clojure team)Usually most arities call into a canonical arity and have the same ret spec#2016-09-0116:32seancorfieldTheyā€™re still pretty messy to spec out, even if :ret is fixed.#2016-09-0116:32Alex Miller (Clojure team)I have not found multiple arities at all difficult to spec#2016-09-0116:32Alex Miller (Clojure team)They're often quite easy to talk about in regex #2016-09-0116:34seancorfieldAs long as the arities all extend the base case in order, yesā€¦ but thereā€™s quite a bit of real world code out there which doesnā€™t follow that model (or is that also non-idiomatic?).#2016-09-0116:36Alex Miller (Clojure team)Even if not in same order, regex can easily describe with ?#2016-09-0116:37Alex Miller (Clojure team)I'm not discounting what you're saying, but I have not found that to be an issue in my own experience#2016-09-0116:38Alex Miller (Clojure team)In general, I am more commonly surprised at how well regex specs work for args#2016-09-0116:41robert-stuttafordyou guys are having a ball, aren't you. can't wait to get stuck in myself!#2016-09-0116:41mschmele@sattvik copied from above, because I forgot to @ you earlier šŸ˜ Thatā€™s what Iā€™ve been using it for in simpler functions like get-from-id for example. I guess my real question (and I probably should have been more specific), applies to more complex functions like transfer which would take two accounts that canā€™t be nil Despite having validation in the function, Iā€™m having trouble getting it to pass spec check#2016-09-0116:43seancorfield@alexmiller Fair enough. I held off specā€™ing some of the multi-arity java.jdbc stuff at first since it didnā€™t seem "easy" (although it may yet prove to be "simple"). Iā€™ll take another run at it soon (next week probably) and report back.#2016-09-0116:46seancorfield@alexmiller My last Q for the morning (I promise!): is there an idiomatic way to spec something that is treated as truthy / falsey, when it might not be strictly true or false. I tried #{true false nil} before realizing wonā€™t work facepalm and (s/nilable #{true false}) "works" but seems a bit ā€¦ I guess that treating an any? argument as a pseudo-boolean is probably a bit sketchy but ...#2016-09-0117:03Alex Miller (Clojure team)I donā€™t think (s/nilable #{true false}) works either for false (for same reason as the set)#2016-09-0117:04Alex Miller (Clojure team)so you need some kind of fn to do it so pick your favorite function that matches those 3 values :)#2016-09-0117:05Alex Miller (Clojure team)I guess (s/nilable boolean?) works#2016-09-0117:06Alex Miller (Clojure team)Iā€™ll go with that :)#2016-09-0117:07donaldball(s/def ::boolish (s/or :truthy (complement #{false nil}) :falsey #{false nil})) ?#2016-09-0117:09Alex Miller (Clojure team)no set with falsey values in it is going to be useful :)#2016-09-0117:10Alex Miller (Clojure team)s/nilable has been made significantly better performing this week from Richā€™s commits in master btw#2016-09-0117:12seancorfieldAh, yes, of course false wonā€™t work either. facepalm again šŸ™‚#2016-09-0117:17seancorfield(it hadnā€™t failed in testing yet butā€¦)#2016-09-0117:19seancorfieldThe specific case is for ::as-arrays? in java.jdbc where itā€™s intended to be true / false or :cols-as-is but in reality nil is acceptable and common when passing defaulted options around (and, of course, it really accepts any? and just treats it as a boolean).#2016-09-0117:20sattvikFor the multi-arity stuff, the following doesnā€™t work, but it might be nice if it did:
(defn foo
  ([a] (fn [b] (foo a b)))
  ([a b] (* a (inc b))))

(s/def foo
  (s/or :unary (s/fspec :args (s/cat :arg int?)
                        :ret (s/fspec :args (s/cat :arg int?)
                                      :ret int?))
        :binary (s/fspec :args (s/cat :arg1 int?
                                      :arg2 int?)
                         :ret int?)))
#2016-09-0117:26seancorfield
(s/fdef foo
  (:args (s/cat :a int?) 
   :ret  (s/fspec :args (s/cat :b int?) 
                  :ret int?))
  (:args (s/cat :a int? :b int?) 
   :ret  int?))
That was along the lines of what I was thinking...
#2016-09-0117:26seancorfieldWhich matches defn for multi-arity functions.#2016-09-0117:56Alex Miller (Clojure team)bleh#2016-09-0117:56Alex Miller (Clojure team):)#2016-09-0118:34Alex Miller (Clojure team)
(s/fdef foo
  :args (s/cat :a int? :b (s/? int?))
  :ret (s/or :val int? :fun (s/fspec :args (s/cat :arg int?) :ret int?))
  :fn (fn [m]
        (= (-> m :ret key)
           (if (-> m :args :b) :val :fun))))
#2016-09-0118:37Alex Miller (Clojure team)I think Rich would say about :ret here that it should simply state the truth - it can either be a number or a function#2016-09-0118:38Alex Miller (Clojure team)and :fn can add an arg-dependent constraint#2016-09-0118:39Alex Miller (Clojure team)I think itā€™s unlikely we would extend to either of the two suggestions above#2016-09-0118:40Alex Miller (Clojure team)this kind of dependent constraint is the whole reason to have :fn#2016-09-0118:41Alex Miller (Clojure team)if you felt the need, I think you could create a macro that automatically created a spec for a curried fn#2016-09-0118:52seancorfieldHmm, yeah, that might well be worth doingā€¦ Iā€™ll have a think about that...#2016-09-0118:54seancorfieldA variant of fdef for which you give :args, :ret, and :fn -- and then also an indication of which curried variants you needā€¦ and then it automatically generates the fdef spec from that...#2016-09-0118:54Alex Miller (Clojure team)I guess youā€™d need to specify the shape of every curried result#2016-09-0118:55seancorfieldThat would be the hard part since you canā€™t use s/? for all those args, only for the last one.#2016-09-0118:56Alex Miller (Clojure team)you can stack em up#2016-09-0118:57seancorfield(s/? (s/cat ā€¦))#2016-09-0118:58Alex Miller (Clojure team)yeah, itā€™s gross looking - youā€™d want to generate it out of an s/cat or an s/cat with noted optional parts or something#2016-09-0118:58Alex Miller (Clojure team)I donā€™t think curried fns are common enough to do any of this :) but if youā€™re looking for a puzzle to play with ā€¦#2016-09-0118:59seancorfieldI tend to curry functions quite a bit ā€” to avoid partial all over the place ā€” but it is almost always currying just a two arg function.#2016-09-0119:02Alex Miller (Clojure team)well maybe thatā€™s a simplifier#2016-09-0119:02Alex Miller (Clojure team)the shape I had above could be made generic#2016-09-0119:06seancorfieldI may just remove the currying in java.jdbc since Iā€™d be shocked if anyone actually leverages it in client code (given your comment about it being non-idiomatic). java.jdbc doesnā€™t use the curried form of as-sql-name internally and I donā€™t know why anyone would externally. And I wouldnā€™t expect the non-curried form of quoted to be used by anyone either (the one-argument form is actually the common, useful arity).#2016-09-0119:06seancorfieldI hadnā€™t thought hard about either of those until I sat down to try to spec them out.#2016-09-0119:10seancorfieldclojure.spec definitely makes you question your design choices šŸ™‚#2016-09-0119:32Alex Miller (Clojure team)indeed#2016-09-0119:33patrkrisAny ideas on how to organize a project's specs? For instance, if I have a domain model for my application that describes a customer entity, would I then create the spec in a com.example.customer namespace and use ::name for defining a spec for the customer's name? Or would I benefit from centralizing specs in a dedicated namespace, and in that spell out the fully namespaced keyword, i.e. com.example.customer/name?#2016-09-0119:35Alex Miller (Clojure team)Iā€™d say both are fine :)#2016-09-0119:38patrkrisYeah. That's what I thought you'd say. šŸ˜‰ But then I am imagining scenarios where a customer means different things in different contexts. It may be one thing in the domain model and another thing in a HTTP request handler. So it might be okay to have :com.example.customer/name and :com.example.resources.customer/name? Does that look wrong?#2016-09-0119:46Alex Miller (Clojure team)you can alias specs (s/def :com.example.customer/name :com.example.resources.customer/name)#2016-09-0119:46Alex Miller (Clojure team)so you can have both#2016-09-0119:47Alex Miller (Clojure team)has to be done explicitly of course so ymmv#2016-09-0119:47Alex Miller (Clojure team)weā€™ve talked about a version of s/keys that would separate the map keys from the specs rather than requiring them to be the same keyword, not sure if that will pan out#2016-09-0120:31manderson+1 to separating map keys from specs ^^^#2016-09-0120:55Alex Miller (Clojure team)the spec part would still be a qualified keyword (not inline specs), just to be clear#2016-09-0120:56Alex Miller (Clojure team)this was considered as an alternative to :req-un too#2016-09-0121:05seancorfieldIt took me a while to internalize that :req-un could have :my.foo/bar as a spec for the key :bar independent of the namespace you define the keys spec in. #2016-09-0121:07seancorfield(Because I didn't read the docs closely enough apparently)#2016-09-0121:08seancorfieldSo I'm not sure why you'd want to separate the specs from the keys at this point?#2016-09-0121:08seancorfield(Or am I still misunderstanding the issue?)#2016-09-0201:33seancorfield@gfredericks does the maven clojure plugin do the same monkey-patching of clojure.test that lein does? http://build.clojure.org/job/java.jdbc-test-matrix/453/CLOJURE_VERSION=1.9.0-alpha11,jdk=Sun%20JDK%201.6/console <ā€” seems to be the same exception as I got testing locally#2016-09-0201:55gfredericks@seancorfield um#2016-09-0201:56seancorfieldNot sure how to tell Mavenā€™s plugin not to do the Leiningen naughtiness šŸ™‚#2016-09-0201:56gfrederickscom.theoryinpractise.clojure.testrunner is your culprit#2016-09-0201:56gfredericksI don't know about maven clojure, but it smells like something similar#2016-09-0201:57gfredericksthis seems a lot harder to work around given the constraints#2016-09-0201:57seancorfieldhttps://github.com/talios/clojure-maven-plugin/blob/develop/src/main/resources/default_test_script.clj#L55
#2016-09-0201:57seancorfieldIt rebinds the report function#2016-09-0201:58gfredericksdo you get to choose an alternate version of the clojure-maven-plugin if you want?#2016-09-0201:59seancorfieldI could override it for my project I think, yes (this is for java.jdbc)#2016-09-0201:59seancorfieldIs there a version that is compatible with test.checkā€™s clojure_test stuff?#2016-09-0201:59gfredericksalternately, any tactic you can think of that lets you require test.check.clojure-test prior to that line running will fix it#2016-09-0201:59gfredericksno I was just imagining forking it :)#2016-09-0201:59seancorfieldHahahaā€¦ ok...#2016-09-0201:59gfredericksso a user.clj could work if you can keep it out of the release jar#2016-09-0202:00seancorfieldWould that run with Maven?#2016-09-0202:00gfredericksit runs when clojure boots up#2016-09-0202:00gfredericksuser.clj is a pretty reliably way to slip something in before just about anything else happens#2016-09-0202:01seancorfieldPretty sure that doesnā€™t work with Boot? (more an FYI butā€¦)#2016-09-0202:01gfrederickswhy not?#2016-09-0202:03seancorfieldI donā€™t rememberā€¦ but it was discussed in #boot a while back...#2016-09-0202:04gfredericksprobably something about their magical space age classloader thing#2016-09-0202:05seancorfieldAnd how do you get it to be loaded for Maven running Clojure?#2016-09-0202:05gfredericks@seancorfield heck for that matter adding a (:require clojure.test.check.clojure-test) to any of your namespaces should also work#2016-09-0202:06gfredericksputting the user.clj on the classpath means that clojure.core reads it at then end of its loading#2016-09-0202:06gfrederickse.g. src/test/resources/user.clj or whatever#2016-09-0202:09seancorfieldAh, yeah, that worked...#2016-09-0202:15talios@seancorfield you can define/give your own test runner script to clojure-maven-plugin if you need to#2016-09-0202:16seancorfieldDynamically requireing that namespace when my test namespace loads seems to do the trick ā€” and I can remove the Leiningen monkey-patch setting as well.#2016-09-0202:16talios
<configuration>
  <testScript>src/test/clojure/com/jobsheet/test.clj</testScript>
</configuration>
#2016-09-0202:16taliosin the pom#2016-09-0202:17seancorfieldI have this now in my test ns:
(def with-spec? (try
                  (require 'clojure.java.jdbc.spec)
                  (require 'clojure.spec.test)
                  ;; require this to workaround rebinding of report multi-fn
                  (require 'clojure.test.check.clojure-test)
                  (let [syms ((resolve 'clojure.spec.test/enumerate-namespace) 'clojure.java.jdbc)]
                    ((resolve 'clojure.spec.test/instrument) syms))
                  (println "Instrumenting clojure.java.jdbc with clojure.spec")
                  true
                  (catch Exception _
                    false)))
#2016-09-0202:17seancorfieldWorks with Leiningen and Maven!#2016-09-0202:18taliosSweet - PRs welcome to improve that default test runner script as well.#2016-09-0202:18taliosif needed#2016-09-0215:08kurt-yagramI have a spec that looks like this:
(s/def ::value <???>)
(s/def ::type #{"type-date", "type-weirdo", "no-value"})
(s/def ::ent (s/keys :req-un [::type] :opt-un [::value]))
(s/def ::ents (s/coll-of ::ent))
Depending on the type, the value needs to be different, e.g., for type-date, the value should be a date. I know there are multimethods that can, and probably should, be used, but I just can't get it right. What comes at the <???>, so I can have different predicates for value depending on the value of type?
#2016-09-0215:12Alex Miller (Clojure team)have you looked at s/multi-spec?#2016-09-0215:12Alex Miller (Clojure team)there is an example in the guide http://clojure.org/guides/spec#2016-09-0215:19spinningtopsofdoomI ran into this problem two days ago you want to make N :value specs. (e..g.`(s/def :my/value <???>)`, (s/def :other/value <???>), (s/def :one.more/value <???>) and then dispatch those :value's with a multimethod#2016-09-0215:20kurt-yagramyeah, I'm looking at it. but it seems to be somewhat different. So, I could do:
(defmulti ent-type ::type)
(defmethod ent-type "type-date" [_]
   (s/keys :req-un [::value])
(s/def ent-type (s/multi-spec ent-type ::type))
So I day value is required for "type-date". But that still doesn't solve the problem. I must be missing something...
#2016-09-0215:21kurt-yagram@spinningtopsofdoom Allright, and I don't need to care about the namespaces? - I mean, the map contains just 'value', no namespaces. I'll give it a few tries.#2016-09-0215:21Alex Miller (Clojure team)instead of using ::value, you could define many :foo1/value :foo2/value :foo3/value specs#2016-09-0215:22kurt-yagramšŸ™‚#2016-09-0215:22Alex Miller (Clojure team)each defmethod would use a different one in :req-un#2016-09-0215:22kurt-yagramoh, allright#2016-09-0215:25spinningtopsofdoom@alexmiller is there any movement on having spec/keys take a map of keywords and specs (e.g.)
(spec/def :one-map (spec/keys :req-un {:value <one spec>})
(spec/def :other-map (spec/keys :req-un {:value <other spec>})
So that you don't have to have registered specs for spec/keys
#2016-09-0215:25Alex Miller (Clojure team)nothing yet#2016-09-0215:26Alex Miller (Clojure team)well, it will never take inline specs#2016-09-0215:27Alex Miller (Clojure team)we might possibly loosen the constraint between key name and spec name, but itā€™s part of the design that s/keys doesnā€™t use inline specs and I donā€™t expect that to change#2016-09-0215:27Alex Miller (Clojure team)the idea is to encourage defining semantics for attributes#2016-09-0215:29spinningtopsofdoomThe constraint between key name and spec name is what I would like to loosen. So then my example would be
(spec/def :one-map (spec/keys :req-un {:value ::one-spec})
(spec/def :other-map (spec/keys :req-un {:value ::other-spec})
Correct?
#2016-09-0215:30Alex Miller (Clojure team)yeah, something like that has been mentioned, but I do not know whether weā€™ll end up doing it or not#2016-09-0215:32spinningtopsofdoomWell I'll make a Jira ticket for that if it's not already there, then. Thanks for the clarification.#2016-09-0215:35Alex Miller (Clojure team)I do not know of a jira ticket for this#2016-09-0215:40gerstreeHi everybody, very new to specs, but have already a good part implemented. Now I need a little help on something that is probably very easy, but I'm stuck. I have a def that verifies a url to be a s3 url (s/def ::s3-url #(str/starts-with? % "s3"). Using it to check a single argument works. Now I have a function with 2 arguments (from-url and to-url) of which 1 needs to conform to that spec. Can anyone hint me for the solution.#2016-09-0215:44gerstreeThis is what I have:
(s/def ::valid-url is-valid-url?)
(s/def ::s3-url #(str/starts-with? % "s3"))

(defn sync
  "sync an s3 folder with a local folder, this works both ways"
  [from-url to-url])

(s/fdef sync
        :args (s/and (s/cat :from-url is-valid-url? :to-url is-valid-url?)
                     ??? this is where I get lost ???))
#2016-09-0215:47Alex Miller (Clojure team)
(s/fdef sync
        :args (s/cat :from-url is-valid-url? :to-url is-valid-url?))
#2016-09-0215:48Alex Miller (Clojure team)I guess Iā€™m also wondering what the difference is between is-valid-url?, ::valid-url and ::s3-url#2016-09-0215:48Alex Miller (Clojure team)given that you have specs, I would actually use the specs in the sync fdef#2016-09-0215:49gerstreeGood one, I will have to clean that up#2016-09-0215:49Alex Miller (Clojure team)(s/fdef sync :args (s/cat :from-url ::s3-url :to-url ::s3-url))#2016-09-0215:49Alex Miller (Clojure team)something like that#2016-09-0215:50gerstreeThe function will be called either (sync "" "") or (sync "" "")#2016-09-0215:51gerstreeWhat I am trying to spec is that either url is an s3 url#2016-09-0215:51Alex Miller (Clojure team)right so you could have something like:
(s/def ::file-url #(str/starts-with? % ā€œfile://ā€œ))
(s/def ::s3-url #(str/starts-with? % ā€œs3://ā€œ))
(s/def ::aws-url (s/or :file ::file-url :s3 ::s3-url))
ā€¦ then use ::aws-url in the sync fdef
#2016-09-0215:52Alex Miller (Clojure team)or whatever is appropriate#2016-09-0215:52Alex Miller (Clojure team)oh, you want a constraint across the args!#2016-09-0215:52gerstreeYes, is that beyond what spec is meant for?#2016-09-0215:53Alex Miller (Clojure team)No, that's fine!#2016-09-0215:54Alex Miller (Clojure team)You can do it with s/and like you were or you can do that in the :fn spec too#2016-09-0215:54Alex Miller (Clojure team)fn is used for constraints between args and ret or also across args#2016-09-0215:55Alex Miller (Clojure team)Only the args spec is checked in instrumentation though so that might be what you want#2016-09-0215:55gerstreeAh, been reading about it all day and understood the args / ret, but of course that works for across args as well.#2016-09-0215:56Alex Miller (Clojure team)Yeah fn gets the conformed version of both args and ret#2016-09-0215:56gerstreeThat might be the easiest way to accomplish it.#2016-09-0216:16gerstreeAlso read about 10 times that stest/instrument only does :args, now I know that is true šŸ˜‰#2016-09-0216:30gerstreeI have the :fn version working. I'm still curious about a solution in the :args part, where I got stuck to begin with. That way, if people use my sync function they can simply (stest/instrument `sync) and work from there.#2016-09-0216:39Alex Miller (Clojure team)
(s/fdef :args (s/and (s/cat :from-url ::aws-url :to-url ::aws-url)
                   (fn [{:keys [from-url to-url]}]
                     (or (s3-url? from-url) (s3-url? to-url)))))
#2016-09-0216:39Alex Miller (Clojure team)something like that#2016-09-0216:40Alex Miller (Clojure team)the second function in the s/and receives the conformed version of the first part of the and#2016-09-0216:41Alex Miller (Clojure team)so that will be a map with :from-url and :to-url keys#2016-09-0216:43gerstreeWorks like a charm and is actually simple enough#2016-09-0216:43gerstreeLearned a lot today, thanks for helping me get through that last part!#2016-09-0216:49Alex Miller (Clojure team)np#2016-09-0217:44kurt-yagramis it possible to use `s/multi-spec' on different specs? Something like - but this doesn't work:
(defmulti m [::type ::op])
(defmethod m ["date-type" "do"] [_]
  (s/keys ...))
...
(s/def .... (s/multi-spec m [::type ::op]))
#2016-09-0217:58Alex Miller (Clojure team)that defmulti definition looks wrong - it takes a dispatch function not a vector#2016-09-0218:07Alex Miller (Clojure team)(defmulti m #(vector (::type %) (::op %))) maybe?#2016-09-0218:07Alex Miller (Clojure team)or itā€™s a great opportunity to use juxt :)#2016-09-0218:08Alex Miller (Clojure team)(defmulti m (juxt ::type ::op))#2016-09-0218:09kurt-yagramoh, cool... thx!#2016-09-0218:10Alex Miller (Clojure team)and then the retag value is not valid either - should either be a tag or a fn of generated value and dispatch-tag#2016-09-0222:34lvhhuh; are instrumentation exceptions supposed to escape clojure.test assertions?#2016-09-0222:35lvhoh, wait, never mind; only some of them are outside assertions šŸ™‚#2016-09-0300:44agcan anyone tell me ETA of 1.9 stable? I already have a piece of code that depends on clojure.spec, unfortunately my team did not approve my PR (bunch of alpha-phobics), that makes it a bit inconvenient for me personally.#2016-09-0300:45Alex Miller (Clojure team)No eta#2016-09-0300:47agnot even approximation? before Christmas, 2nd quarter next year etc.?#2016-09-0300:47Alex Miller (Clojure team)I'm sure we will try to hit some milestone before the conj but may just be beta (feature freeze)#2016-09-0300:48Alex Miller (Clojure team)We have a couple other things in flight that we haven't even talked about yet#2016-09-0300:49Alex Miller (Clojure team)We are not currently working towards any concrete deadline though#2016-09-0300:50agyeah, okā€¦ I am so excited though. Spec turns out to be a massive feature of the platform. I really like it.#2016-09-0301:24Alex Miller (Clojure team)Thanks#2016-09-0311:47wagjo@ag there is a backport of specs for 1.8, https://github.com/tonsky/clojure-future-spec#2016-09-0319:10ag@wagjo yeah, I switched to backport. Worked as a charm.#2016-09-0322:34lvhAre you generally expected to add enough info to s/fdef such that running check on that fn works? Is that considered the ā€œdefaultā€ state? Iā€™m writing reasonably gnarly generators to make that work, because I have two args with related keys (and if you picked args at random, youā€™d almost ceratinly end up with garbage)#2016-09-0322:47lvhHm. I wonder if itā€™s OK for an fdefā€™s :args to have a generator that uses the fn being fdefā€™d (a lot of functions can clearly be run with a ā€œbase caseā€ if you will; e.g. if you have something that conjs a bunch of stuff together, you might conj a bit ahead of time and assert that it doesnā€™t throw away previously conjā€™d things)#2016-09-0322:47lvhI think so, but with the apparently suggested pattern of fdefing before the defn itself, you run into a compile error; so Iā€™m wondering if I should just declare it and be done with it or what šŸ™‚#2016-09-0323:40lvhFYI, got a weird error when running clojure.spec.test/check + deftest through CIDER (and only CIDER): https://github.com/clojure-emacs/cider/issues/1841 Might be interesting for upstream to look into since just because CIDER ran into it (or rather; I only ran into it with CIDER) doesnā€™t mean itā€™s necessarily CIDER-specific šŸ™‚#2016-09-0401:52hiredmanit likely is, cider is likely redefing the report function in clojure test with function that is not a multimethod#2016-09-0402:54Alex Miller (Clojure team)Yes this is the lein monkey patching#2016-09-0402:54Alex Miller (Clojure team)There are issues in both lein and test.check about it and it's fixed in test.check master#2016-09-0402:55Alex Miller (Clojure team)You can disable lein monkey patch to avoid#2016-09-0402:56Alex Miller (Clojure team)Well I guess it could be a cider variant of the same problem, not sure#2016-09-0403:14gfrederickssomebody filed a bug report a few months back, I think to test.chuck, wherein I convinced them that cider was monkeypatching the wrong way#2016-09-0403:16gfredericksno nevermind#2016-09-0403:16gfredericksit was probably fireplace too#2016-09-0417:53flyboarderHello, just a quick question with the new spec system. Where do I place my specs? In the same namespace as the things they specify are declared? And how do I go about making them compatible with older versions of clojure? I am imagining my lib being used in 1.7 and 1.8 as well.#2016-09-0418:03flyboarderAh I see there are back ports for spec#2016-09-0419:21gfredericksOther libs have put the specs in a separate ns so users on older versions can ignore them#2016-09-0420:42lvhmeh; stest/check now basically looks like it hangs forever and I donā€™t know which thing itā€™s choking on#2016-09-0420:43lvhitā€™d be nice if there was a debug mode or something where it told me what it was working on#2016-09-0420:47lvhItā€™s definitely helpful in finding bugs though šŸ™‚#2016-09-0420:48lvhalso in making my laptop not very suitable for holding on a lap šŸ˜„#2016-09-0509:57ikitommi@alexmiller did you consider using Records instead of just doing reify for the Spec protocol? For example and-spec-impl could return clojure.spec.AndSpec record? Would it have negative effect on performance? Records might be easier to understand and adding extensions would be easier. Trying to cover the JSON Schema generation for the core Specs. Extending records with a new protocol would be easy (the way we are doing the same with Schema).#2016-09-0510:02ikitommiPushed out a experimental version of the dynamic conformations (based on runtime data). Would like to get feedback on that, now using dynamic var as the conform doesnā€™t take any optional parameters to do this. https://github.com/metosin/spec-tools#2016-09-0511:59mishagreetings, is there an equivalent to clojure.spec/def, for defining many specs? like:
(s/def ::first-name string?)
(s/def ::last-name string?)
->>
(s/def-many
  ::first-name string?)
  ::last-name string?)
#2016-09-0512:08madstap
(defmacro def-many [& key-spec-pairs]
  (cons 'do (for [[k spec] (partition 2 key-spec-pairs)]
              `(s/def ~k ~spec))))
#2016-09-0512:21misha@madstap šŸ™‚ is there any popular use case, of using return value of s/def?#2016-09-0617:03nopromptare there any explanations of how to properly implement clojure.spec.Spec?#2016-09-0617:13noprompti ask because i'm interested in integrating a small parser combinator library i wrote with clojure.spec so i can hook into to the conform, explain, etc. goodness without having to do it manually post-hoc.#2016-09-0617:18nopromptit seems like a useful thing since, on occasion, one might be interested in validating/conforming nested (or unnested) textual data.#2016-09-0617:44Alex Miller (Clojure team)@noprompt for the moment, weā€™re considering all that to be implementation details#2016-09-0617:44Alex Miller (Clojure team)and subject to change without notice#2016-09-0617:45Alex Miller (Clojure team)in general, spec is designed to primarily be extensible via predicates (at the bottom) and wrapping with macros (at the top - particularly s/conformer and s/& are tools for this)#2016-09-0617:50noprompt@alexmiller i see. well, i'll continue to hold out then for things to stabilize. i've at least familiarized myself a bit more with the internals (implementing Spec myself) so that can't be a bad thing. is there something similar to conformer for explain? that would be nice in my case because the parser combinators do produce failure data of where in the string the parse failed which could be appended to the path.#2016-09-0617:51noprompttl;dr explainer?#2016-09-0618:00Alex Miller (Clojure team)no, not currently#2016-09-0618:01Alex Miller (Clojure team)although in next alpha, the explain-out fn is a dynvar you can swap#2016-09-0618:01Alex Miller (Clojure team)thatā€™s a bigger hammer though#2016-09-0618:03nopromptyeah. that sounds like too big of a hammer. some way to implement an explain for a pred would be nice. i think i can get by with conformer for now. i hope in the future the protocols will stabilize for this kind of thing.#2016-09-0700:25lvh+1; by the way; I have nothing to contribute to that other than ā€œI can explain a problem better than my consumers can read Clojureā€ šŸ˜„#2016-09-0701:19aghow do I create an enum spec? Value that conforms that is in a given set?#2016-09-0701:20sparkofreason(s/def ::set-spec #{:value1 :value2 :value3})#2016-09-0701:20sparkofreasonIt's just the set itself.#2016-09-0701:20agyeah but ohā€¦ I got itā€¦ (gen/generate (s/gen (s/and #{:foo :boo})))#2016-09-0701:21agI was using s/exercise its output confused me#2016-09-0701:22agthanks#2016-09-0702:20bbrinck@ag I think (gen/generate (s/gen #{:foo :boo})) is equivalent#2016-09-0715:18jannisOut of curiosity: Is there a way to say "in this s/cat spec, please omit this item and this item from the map resulting from s/conform"? One example (s/def ::myspec (s/cat :first-operand number? :_ #{:plus :minus} :second-operand number?)) -> (s/conform ::myspec [15 :plus 10]]) -> {:first-operand 15 :second-operand 10} (with the :_ key indicating "please don't include this in the result")?#2016-09-0715:20jannisI'm asking because I'm parsing a small language with clojure.spec and there are some parts in the language that I don't care about later on. And the simpler the output of s/conform the better - in this case.#2016-09-0715:26Alex Miller (Clojure team)there is no way to automatically omit syntactic elements like that, but you can use a conformer to get the same effect#2016-09-0715:27Alex Miller (Clojure team)something like (s/conform (s/and ::myspec (s/conformer #(dissoc % :_)) [15 :plus 10]])#2016-09-0715:28Alex Miller (Clojure team)you can put the conformer inside the spec too of course, just wrapping here as an example#2016-09-0715:29jannisNice šŸ™‚#2016-09-0715:29jannisThat's pretty awesome.#2016-09-0715:29jannisI assume I can't have multiple :_s in an s/cat?#2016-09-0715:31Alex Miller (Clojure team)it probably wouldnā€™t throw an error, but I havenā€™t tried it#2016-09-0715:31Alex Miller (Clojure team)theyā€™d probably all just override the prior as it parsed - you wouldnā€™t be able to unform that but doesnā€™t seem like you care about that anyways#2016-09-0715:32Alex Miller (Clojure team)(`conformer` can also take an unform function to support both directions)#2016-09-0715:33jannisYep, later :_s override earlier occurences in the resulting map.#2016-09-0715:34jannisCool. I'm not sure I'm going to use this but it could make my life easier.#2016-09-0716:02joshgIs there an idiomatic way to define :args and :ret when defining a function without a separate fdef declaration? It would be easy to write a macro to do this, but is it considered bad practice?#2016-09-0716:03joshg(similar to schemaā€™s s/defn)#2016-09-0716:04Alex Miller (Clojure team)spec does not provide this (and does not intend to)#2016-09-0716:05Alex Miller (Clojure team)but youā€™re welcome to do so :)#2016-09-0716:06joshgthanks#2016-09-0716:09joshg@alexmiller Iā€™m curious why spec :args and :ret are not defined in defn like pre and postconditions?#2016-09-0716:14Alex Miller (Clojure team)Several reasons#2016-09-0716:15Alex Miller (Clojure team)Rich talks about the notion of an independent registry at http://clojure.org/about/spec#2016-09-0716:16Alex Miller (Clojure team)That is, we don't need to hang everything off vars (and there are downsides to doing so, that affect bytecode size and startup time)#2016-09-0716:16Alex Miller (Clojure team)Also, from the purposes of API evolution, it is useful for the specs to be independent from the vars#2016-09-0716:17joshgGot it. Thanks for the explanation.#2016-09-0716:19joshgthe ā€œAPI evolutionā€ sound interesting. I assume thatā€™s yet to be announced?#2016-09-0716:27Alex Miller (Clojure team)Rich talked about some of that on the Cognicast episode he did#2016-09-0716:28joshgwhen he was talking about Postel's Law and API versioning?#2016-09-0716:28Alex Miller (Clojure team)But the idea is that if the specs are independent from the functions you can talk (programmatically) about whether one api subsumed another#2016-09-0716:29joshgthat makes sense#2016-09-0721:44Alex Miller (Clojure team)1.9.0-alpha12 is out, mostly spec related things https://groups.google.com/forum/#!topic/clojure/lQ5beZB6QYE#2016-09-0721:44Alex Miller (Clojure team)in particular, Rich did a big perf pass and improved times of most things#2016-09-0722:20noprompt@alexmiller have you talked to or has rich mentioned anything by the way of the explainer concept we talked about yesterday? it'd be really nice to have something to pair with conformer to produce explanation data for custom conformers!#2016-09-0723:08Alex Miller (Clojure team)No#2016-09-0802:36bbloomi just want to say: iā€™ve been writing javascript with flowtype for the past ~2 weeks. Itā€™s a very nice type system and itā€™s very well implementedā€¦. and I absolutely hate it. I miss clojure šŸ˜ž#2016-09-0802:51seancorfield@bbloom No, no, you must be mistaken. Type systems are awesome šŸ™‚#2016-09-0802:52seancorfieldIā€™m finding new things to use clojure.spec for almost every day, I have to sayā€¦#2016-09-0802:54bbloomi think iā€™ve hit more false positives about NPEs from flowtype in two weeks than iā€™ve had ACTUAL NPEs in a year of Go#2016-09-0802:59seancorfieldSo itā€™s telling you "This will likely NPE" but the code wouldnā€™t?#2016-09-0803:03bbloombasically it says something like ā€œpotentially null or undefinedā€ b/c itā€™s doing a map lookup or something, but i know with 100% confidence by construction that value is in the map#2016-09-0803:04bbloomor similar#2016-09-0803:05bbloomyou wind up having to employ strategies all over the place:#2016-09-0803:05bbloom1) making tons of extra structure types for every possible state of things#2016-09-0803:05bbloomand 2) using invariant(whatever, ā€˜whatever may not be nullā€™) which basically is just a glorified manual throw npe#2016-09-0803:36seancorfieldMust admit, when we were using core.typed at work, that was one of the problems we had: nil was contagious and then we kept getting complaints about code not accepting nil (when we knew nil wouldn't flow to that point),#2016-09-0803:37shaun-mahood@seancorfield: Have you run into anything like that with spec yet?#2016-09-0805:38seancorfield@shaun-mahood no, but thatā€™s because spec isnā€™t "contagious" in the same way that type systems tend to be: specā€™ing one piece of code doesnā€™t ripple out into other pieces of code.#2016-09-0805:42shaun-mahood@seancorfield: That's awesome - I ran into similar issues with schema in a very limited manner and it really bothered me. I know you'll put spec through a lot more than I ever will so I've been using your notes and comments about as sort of my real-world expectations. #2016-09-0805:45seancorfieldI'm surprised you hit that with Schema since it's really just runtime assertions. #2016-09-0806:22shaun-mahoodWell, it's more likely that I did something stupid or am remembering different problems then - I really didn't put a whole ton of effort into figuring out exactly what was going on and it was quite a while ago. I probably just didn't know what I was doing as well as I though I did :)#2016-09-0813:27jannisIs it a deliberate difference between Clojure and ClojureScript that ClojureScript's clojure.spec.test/check only accepts a literal collection of syms instead of a code-generated value (e.g. (let [syms (filter in-my-namespaces? (st/checkable-syms))] (st/check syms)) will not work as it raises a Unable to resolve symbol: syms error.#2016-09-0814:24jeroenvandijkQuestion about multi-spec, is it meant/possible to use with something else than a map? I cannot find examples and after a lot of trying it seems the multi-spec implementation doesnā€™t call the multi-method like one would expect#2016-09-0814:32jeroenvandijkah ok, i had a clear moment. I was a victim of multimethod + multi-spec reloading. It is possible with the right dispatch function#2016-09-0815:09jeroenvandijkFYI https://gist.github.com/jeroenvandijk/2748b6af74c6ba6f8fd7dffaed5cc390#2016-09-0815:24Alex Miller (Clojure team)right#2016-09-0815:26Alex Miller (Clojure team)retagging is discussed in the docstring - itā€™s for gen and can be a fn as well#2016-09-0815:27Alex Miller (Clojure team)@jannis prob a question @dnolen would have to answer re check#2016-09-0815:28jannis@alexmiller Ok, I'll check with him#2016-09-0816:21jannisMight as well do it here. @dnolen: ping šŸ˜‰#2016-09-0816:35Timis anyone using this: https://github.com/tonsky/clojure-future-spec ?#2016-09-0816:47jeroenvandijk@tmtwd yep, we use it. Not in something we deployed in production (yet). But I have used it quite a lot now and it works well#2016-09-0816:48Timhm#2016-09-0816:49jeroenvandijkIs anybody doing things with clojure.spec/unform at the moment? I asked something on the mailinglist about using it for data migrations, but no answer yet#2016-09-0817:04Alex Miller (Clojure team)not many from my impression (given the number of bugs I am aware of but no one has mentioned yet :)#2016-09-0817:04Alex Miller (Clojure team)data migration is one interesting use case for it though#2016-09-0817:07jeroenvandijkthanks, for confirming the use case šŸ™‚ Iā€™ll see if I can make it work#2016-09-0818:21dmarjenburghHello. With multi-spec, is there a way to generate values conforming to only a specific dispatch-value? For example, in the :event/event example in http://clojure.org/guides/spec, the generator generates all events. What is the best way to generate events of only a certain type?#2016-09-0818:42bfabry@dmarjenburgh you can get the spec (and hence the generator) for a specific dispatch value by just invoking the multi-method, (event-type {:event/type :event/search})#2016-09-0818:51dmarjenburghAh, ofcourse. Thanks! šŸ™‚#2016-09-0818:52bfabrynp#2016-09-0913:03Alex Miller (Clojure team)The validation benchmark has been updated with alpha12 showing the spec improvements: https://muhuk.github.io/validation-benchmark#2016-09-0914:19ghadiPretty massive improvements#2016-09-0914:26Alex Miller (Clojure team)yes, should help a lot with runtime use cases#2016-09-0914:31Alex Miller (Clojure team)https://imgur.com/a/l9Hc3 shows the before#2016-09-0915:20otfromalexmiller: that's great. Means I'll use it to conform stuff in my sparkling spark jobs. :-D Thx!#2016-09-0915:20Alex Miller (Clojure team)generally, itā€™s faster than schema, so if schema was fast enough for you, it should be sufficient#2016-09-0915:24otfromalexmiller: that was my alternative so I'm very happy. :-D#2016-09-0915:32seancorfieldI'm going to have to study the commit(s) that are responsible for that massive speed up -- very curious about how that was achieved! šŸ˜ø #2016-09-0915:34Alex Miller (Clojure team)lot of changes. one change of possible importance is that a spec using another registered spec will ā€œcompileā€ in that definition, so changes to the upstream spec (during dev) now require the downstream spec to be reloaded#2016-09-0915:34Alex Miller (Clojure team)so itā€™s a little less dynamic than prior#2016-09-0915:34Alex Miller (Clojure team)this does not affect recursive specs as delays are used#2016-09-0915:41otfrom
> (s/conform (s/coll-of int? :type set?) #{0 1})
#{0 1}
> (s/conform (s/coll-of int? :type set?) [0 1])
[0 1]
(s/conform (s/coll-of int? :type set?) [0 "e"])
:clojure.spec/invalid
#2016-09-0915:42otfromnot sure why I'm not getting invalid on the 2nd one#2016-09-0915:43otfromah, I think I need :into#2016-09-0915:52otfrom
> (s/conform (s/coll-of int? :into #{}) [0 1])
#{0 1}
#2016-09-0915:52otfromseems to work#2016-09-0915:52jannisI'm hitting odd behavior with clojure.spec and I'm not sure it's me or a bug in clojure.spec. Here's the example: https://gist.github.com/Jannis/f23a2ecf350b401745d190b02bbb619c What I would expect is no surrounding [] around [:nested ...] in the second case. It seems that ::link-value spec is clashing with the ::query spec. If I change ::link-value to e.g. (s/tuple symbol? keyword?) I don't get the extract []. The odd thing is that despite the "clash", [:link ...] never appears in the output... any ideas?#2016-09-0915:59Alex Miller (Clojure team)@otfrom :type isnā€™t a thing#2016-09-0915:59Alex Miller (Clojure team)itā€™s :kind#2016-09-0916:00Alex Miller (Clojure team)if only you had the spec specs, it would tell you that :)#2016-09-0916:06Alex Miller (Clojure team)@jannis so the one youā€™re asking about is
(s/conform ::query '[a b [c d]])
;;=> [[:single [:simple a]]
;;    [:nested {:parent [:simple b], :children [[:single [:simple c]]]}]]
#2016-09-0916:07Alex Miller (Clojure team)?#2016-09-0916:07Alex Miller (Clojure team)the outermost [ ] is from ::query#2016-09-0916:08Alex Miller (Clojure team)the [ ] around :single and :nested are due to the s/alt tagging#2016-09-0916:08otfrom@alexmiller thx. Mostly just flailing around in the library trying to learn it (and getting dumb things wrong) šŸ˜‰#2016-09-0916:12jannis@alexmiller The one I'm asking about is [[:single [:simple a]] [[:nested ...]]] - with the [[:nested ..]] instead of [:nested ..].#2016-09-0916:16jannisWhat's puzzling me there is that the other two examples don't have the extra [] around [:nested ...] - and that I can make it disappear in the second case if I change the :link-value spec that should have no impact here.#2016-09-0916:16jannis(This is Clojure 1.9.0-alpha11 by the way)#2016-09-0916:17Alex Miller (Clojure team)oh, this reminds me of something#2016-09-0916:18Alex Miller (Clojure team)http://dev.clojure.org/jira/browse/CLJ-2003?focusedCommentId=43552&amp;page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-43552#2016-09-0916:18Alex Miller (Clojure team)that example in the comment also has something similar#2016-09-0916:19Alex Miller (Clojure team)Iā€™ve been looking at some regex conform results that seem off to me - although most of those relate to use of s/?#2016-09-0916:20jannisThe output certainly looks similar šŸ˜‰#2016-09-0916:25jannisIn this case I think s/conform may not know which of [::query :single :link] and [::query :nested :children] to pick. It decides on :nested instead of :single but surrounds it with the extra [] for some reason.#2016-09-0916:26jannisCould that be what happens in case of ambiguity?#2016-09-0916:30Alex Miller (Clojure team)I donā€™t think thatā€™s whatā€™s happening - I think it has to do with the guts of the regex derivatives and when it decides to create the nested context#2016-09-0916:36jannisYou know the internals, I don't. šŸ˜‰ Is there anything I can do to help investigate the problem (if it is one at all)?#2016-09-0917:06Alex Miller (Clojure team)no, Iā€™m looking at some of this stuff already#2016-09-0917:06Alex Miller (Clojure team)if you wanted to boil it down to a simpler repro and file a jira, that would be helpful#2016-09-0917:06Alex Miller (Clojure team)just to make sure that it does get addressed and not just ā€œin my headā€ :)#2016-09-0920:55fentonhow do I spec that a map should look like: {:db/id <something>}. I don't own the db namespace so am not sure how to spec for that.#2016-09-0920:56bfabry@fenton you don't need to own a namespace to refer to it#2016-09-0920:57Alex Miller (Clojure team)(s/keys :req [:db/id])#2016-09-0920:57Alex Miller (Clojure team)and then separately (s/def :db/id <something>)#2016-09-0920:58Alex Miller (Clojure team)which could come from a lib or could be something you build#2016-09-0920:59fenton(s/def :db/id (s/tuple #{:pc.api/name} string?)) (s/def ::kwm (s/keys req [:db/id]))#2016-09-0921:00fenton{:db/id [:pc.api/name "JCZ4vAlwyUl6r37PeVJ"]} is what i'm going for#2016-09-0921:00fentonrepl> (gen/generate (s/gen :pc.api/kwm)) {}#2016-09-0921:01fentonpcbe.http> (s/valid? :pc.api/kwm {:db/id [:pc.api/name "orange"]}) true#2016-09-0921:01fentoncool...#2016-09-1100:02ghufranIā€™m going through the spec tutorial on http://clojure.org . Is there any difference between using spec/or and using spec/alt ? It seems to work the same for the example from the tutorial: ` (s/def ::config-alt (s/* (s/cat :prop string? :val (s/alt :s string? :b boolean?)))) ;;=> :testspec.core/config-alt (s/conform ::config-alt [ "-server" "foo" "-verbose" true "-user" "joe"]) ;;=> [{:prop "-server", :val [:s "foo"]} {:prop "-verbose", :val [:b true]} {:prop "-user", :val [:s "joe"]}] (s/def ::config-or (s/* (s/cat :prop string? :val (s/or :s string? :b boolean?)))) ;;=> :testspec.core/config-or (s/conform ::config-or [ "-server" "foo" "-verbose" true "-user" "joe"]) ;;=> [{:prop "-server", :val [:s "foo"]} {:prop "-verbose", :val [:b true]} {:prop "-user", :val [:s "joe"]}]`#2016-09-1100:14Alex Miller (Clojure team)Yes (although there are situations where they can yield the same result)#2016-09-1100:15Alex Miller (Clojure team)in this particular example we are describing the structure of a sequential thing and regex ops are being used to specify that structure#2016-09-1100:15Alex Miller (Clojure team)s/alt is the proper choice here as it is the regex op version#2016-09-1100:16Alex Miller (Clojure team)it matters in the context where it is used - when used inside another regex op, it does not start a new nested context, rather it matches elements out of the current context#2016-09-1100:16Alex Miller (Clojure team)however s/or is effectively opaquely matching a value#2016-09-1100:16Alex Miller (Clojure team)so an example where alt works but or does not is:#2016-09-1100:20Alex Miller (Clojure team)
user=> (s/def ::a (s/alt :i int? :s (s/cat :a string? :b string?)))
:user/a
user=> (s/conform ::a [1])
[:i 1]
user=> (s/conform ::a ["a" "b"])
[:s {:a "a", :b "bā€}]
#2016-09-1100:20Alex Miller (Clojure team)here an alt by itself implies a sequential context but thatā€™s not true with s/or#2016-09-1100:21Alex Miller (Clojure team)but the nested s/cat does (b/c also a regex op):#2016-09-1100:21Alex Miller (Clojure team)
user=> (s/def ::o (s/or :i int? :s (s/cat :a string? :b string?)))
:user/o
user=> (s/conform ::o 5)   ;; note 5 is a bare value, not in a collection
[:i 5]
user=> (s/conform ::o ["a" "b"])
[:s {:a "a", :b "bā€}]
#2016-09-1100:23Alex Miller (Clojure team)and you can nest an alt inside another regex without requiring a new sequential context:#2016-09-1100:23Alex Miller (Clojure team)
user=> (s/conform (s/cat :k keyword? :nested-a ::a) [:foo 5])
{:k :foo, :nested-a [:i 5]}
user=> (s/conform (s/cat :k keyword? :nested-a ::a) [:foo "a" "b"])
{:k :foo, :nested-a [:s {:a "a", :b "b"}]}
#2016-09-1100:23Alex Miller (Clojure team)here ::a is embedded inside something else but just describes more elements of the sequence#2016-09-1101:06ghufranThanks @alexmiller! The guide to spec is great so far, will keep working through it!#2016-09-1206:16bretIf I write a spec like:
(s/def :in/data (s/and
                  (s/keys :req-un [:in/id] :opt-un [:in/more])
                  (s/map-of #{:id :more} nil)))
My instinct is to not duplicate the keys and modify this to be:
(def req-keys [:in/id])
(def opt-keys [:in/more])

(defn unk
  "Returns ns unqualified keys for (possibly) qualified ones."
  [& ks]
  (map #(-> % name keyword) ks))

(s/def :in2/data (s/and
                   (s/keys :req-un req-keys :opt-un opt-keys)
                   (s/map-of (set (apply unk (concat req-keys opt-keys))) nil)))
which fails due to the nature of the s/keys macro. I understand the low level cause of the error. However, I want to make sure Iā€™m not missing something fundamental about the intention. I did find this on the google group https://groups.google.com/forum/#!searchin/clojure/s$2Fkeys%7Csort:relevance/clojure/mlMYUrPVdso/ATklLgpGBAAJ so, possibly, Iā€™m not completely alone in my instincts. But there was no meaningful reply.
#2016-09-1207:51jeroenvandijkIā€™ve found a case where conform -> unform -> conform leads to an invalid result. This is the case with the clojure.core.specs/defn-args spec. See https://gist.github.com/jeroenvandijk/28c6cdd867dbc9889565dca92673a531 Should I file a JIRA issue?#2016-09-1212:24jmglovSorry for asking such a basic question, but what is the recommended way to test spec'd functions in unit tests (i.e. by running lein test)?#2016-09-1212:24jmglovI like the idea of combining unit and generative tests as per https://dpassen1.github.io/software/2016/09/07/from-repl-to-tests#a-better-way#2016-09-1212:26jmglovBut I'm not really sure how to get c.s.test/check to hook into the (deftest ... (checking ...)) style.#2016-09-1212:26jmglovRTFM links welcome. šŸ˜‰#2016-09-1212:29otfromjmglov: I'd be happy with figuring out that workflow too#2016-09-1212:32tgkYeah, I was puzzled with this as well. I ended up writing a very small namespace for it and creating a lein alias for running the specs#2016-09-1212:32jmglovUsing enumerate-namespace?#2016-09-1212:34tgkUsing clojure.tools.namespace.repl: https://gist.github.com/tgk/be0325e7b78bc692ad6c85ef6aca818d#2016-09-1212:34tgkThe file is only on dev path by the way šŸ™‚#2016-09-1212:37jmglovInteresting. I didn't realise that test/check had a zero-arity form. Really useful!#2016-09-1212:38jmglovDoes it actually find all specs in all namespaces in your classpath, or something?#2016-09-1212:39jmglovI don't see you specifying any namespaces under test, or requiring them in.#2016-09-1212:39tgkYes, as long as they have been evaluated#2016-09-1212:39jmglovWow.#2016-09-1212:39tgkAh yes, thatā€™s where refresh comes into the picture šŸ™‚#2016-09-1212:39jmglovWould your approach actually run all the specs for dependencies as well?#2016-09-1212:40jmglovOr just stuff in your src?#2016-09-1212:40jmglovThat's nifty!#2016-09-1212:41jmglovAll the same, it would be great to find an approach that would allow me to drop spec generative tests into my standard clojure.test files.#2016-09-1212:41jmglovI haven't found anything with Google, but stemming is really fighting me on this one. šŸ˜‰#2016-09-1212:44jmglovMaybe I need to roll my own checking macro, like this guy did: http://blog.colinwilliams.name/blog/2015/01/26/alternative-clojure-dot-test-integration-with-test-dot-check/#the-alternative#2016-09-1212:49tgkHmm yes, I donā€™t think specs for dependencies would be run. I canā€™t see how they would šŸ™‚#2016-09-1212:49jmglovThat's good. šŸ™‚#2016-09-1212:49tgkI found it very hard to find anyone whoā€™d hooked it into tests
#2016-09-1212:49jmglovSo refresh simply evals everything in your source directories?#2016-09-1212:54Alex Miller (Clojure team)using spec.test/check or spec.test/instrument will pick up any specā€™ed fns that have been loaded and added to the registry, so it depends completely on what code youā€™ve loaded#2016-09-1212:55jmglovThanks, Alex!#2016-09-1212:55jmglovAlso, thanks for the spec Guide. I finished reading it, and it is really excellent!#2016-09-1212:56Alex Miller (Clojure team)@bret I personally would prefer your first spec (although looks like you missing the kw namespaces on the map-of and you probably want s/merge instead of s/and)#2016-09-1212:57jmglovOK, switching gears for a second, I'm obviously doing something silly, but I'm not sure what. I'm trying to spec out the input coming in from some JSON, and I have some code like this:
(ns wtf
  (:require [clojure.spec :as s]
            [clojure.spec.test :as test]))

(s/def ::contract_type_id pos-int?)
(s/def ::product (s/keys :req-un [::contract_type_id]))
(s/fdef exclude-products
  :args (s/cat :products (s/coll-of ::product)
               :excluded-contracts (s/coll-of ::contract_type_id))
  :ret (s/coll-of ::product)
  :fn #(<= (-> % :args :products) (-> % :ret)))

(defn- exclude-products [products excluded-contracts]
  (letfn [(excluded? [product]
            (some #{(:contract_type_id product)} excluded-contracts))]
    (remove excluded? products)))
#2016-09-1212:58jmglovMy exclude-products function should do something this:
wtf> (exclude-products [{:contract_type_id 1} {:contract_type_id 2}] [1])
({:contract_type_id 2})
#2016-09-1212:59jmglovThings look good with exercise-fn:
wtf> (s/exercise-fn `exclude-products 1)
([([{:contract_type_id 2} {:contract_type_id 2} {:contract_type_id 2} {:contract_type_id 1}] [2 2 1 1 1 1 1 2 1])
  ()])
#2016-09-1213:00jmglovBut check completely rejects my entire worldview:
wtf> (test/check `exclude-products)
({:spec #object[clojure.spec$fspec_impl$reify__14244 0x2c221658 "
#2016-09-1213:01jmglovCan anyone shed light on what I'm doing wrong?#2016-09-1213:01Alex Miller (Clojure team)@jeroenvandijk yes, please file a jira. this is where we are hurting for an s/vcat which Rich and I have talked about several times.#2016-09-1213:05Alex Miller (Clojure team)@jmglov it looks to me like your :fn spec is wrong and should be comparing count of each thing?#2016-09-1213:06jmglovdoh#2016-09-1213:06jmglov@alexmiller Of course you are right. Thanks for pointing out my idiocy! šŸ™‚#2016-09-1213:07Alex Miller (Clojure team)well I wouldnā€™t go that far. :) fwiw, Iā€™ve done the same.#2016-09-1213:10jmglovOh, so much better! Now summarize-results is showing me a bug in my code or spec. šŸ™‚#2016-09-1213:11bret@alexmiller I guess my point/observation is that the second one is not allowed at all. That was puzzling at first. I've missed s/merge all this time. Iā€™ll look at that and see if that changes anything.#2016-09-1213:16jeroenvandijk@alexmiller thank, will do#2016-09-1213:17Alex Miller (Clojure team)@bret not sure what you mean by your point/observation, sorry#2016-09-1213:23jeroenvandijk@alexmiller Iā€™ve created the issue here http://dev.clojure.org/jira/browse/CLJ-2021?focusedCommentId=43842#comment-43842 Not sure what else to add#2016-09-1213:27bret@alexmiller I probably shouldn't start writing at midnight on Sunday night. :) I guess it boils down to, is the reason this works
(s/keys :req-un [::k1 ::k2])
=> #object[clojure.spec$map_spec_impl$reify__13426 0x1b824394 "
and this doesnā€™t
(def rks [::k1 ::k2])
=> #'onenine.core/rks
(s/keys :req-un rks)
CompilerException java.lang.IllegalArgumentException: Don't know how to create ISeq from: clojure.lang.Symbol, compiling:(/Users/brety/dev/personal/onenine/src/onenine/core.clj:69:1) 
merely a consequence of the s/keys macro implementation or is this intended to not be valid?
#2016-09-1213:27Alex Miller (Clojure team)well, both#2016-09-1213:28Alex Miller (Clojure team)s/keys is a macro and expect a concrete list of keys#2016-09-1213:28Alex Miller (Clojure team)so thatā€™s as intended#2016-09-1213:28Alex Miller (Clojure team)and the reason most of the spec creating fns are macros is to capture forms for reporting#2016-09-1213:29jmglovDoes anyone know how to make test/check work on private functions? I've tried #'full.namespace/foo, but I get java.lang.IllegalArgumentException: Don't know how to create ISeq from: clojure.lang.Var.#2016-09-1213:29jmglovLeaving off the #' tells me the function is not public, which I already know. šŸ˜‰#2016-09-1213:30Alex Miller (Clojure team)we might in the future have a fn entry point for s/keys with the caveat that you may lose some of the explain reporting capability#2016-09-1213:30bretThat is what I suspected (the reporting aspect) but wanted to confirm.#2016-09-1213:31Alex Miller (Clojure team)@jmglov I donā€™t think that was considered in the design (and Iā€™m not sure whether it should be)#2016-09-1213:32jmglovYeah, I'm a bit naughty, really.#2016-09-1213:32lvhOn a related note: I find myself regularly wanting validation of runtime-defined specs, where I learn e.g. the structure of the JSON in this particular REST API at runtime. This is for presumably obvious reasons, not very convenient right now.#2016-09-1213:32jmglovThis is always a tough call. I want to use private functions to communicate to my clients that they are not part of the interface, but I also want to be able to unit test them. šŸ˜•#2016-09-1213:33lvh(Everything ends up being evalā€™d, which I guess is fine?)#2016-09-1213:33bretIt would be nice in some case, I think, to be able to define keys once and combine them in different ways ways when building specs. At least, as I think about repeating information in the spec(s) and trying to reduce that.#2016-09-1213:33bretBut I donā€™t have enough time in spec to be sure.#2016-09-1213:34jmglovI might just go the foo.bar.internal route, where everything in the internal ns is public and can be tested, but is pretty clearly not for client consumption.#2016-09-1213:34Alex Miller (Clojure team)@lvh yeah, I understand that as a use case, not sure how common that will be in general though#2016-09-1213:34jmglovUsing a namespace-level docstring to warn off potential troublemakers, of course.#2016-09-1213:34Alex Miller (Clojure team)yeah#2016-09-1213:35Alex Miller (Clojure team)@bret well thatā€™s exactly the point of having the registry#2016-09-1213:40lvhalexmiller: If you give Clojure programmers a feature it seems like a matter of time before theyā€™ll try to express as much of it as data, and then itā€™s not a long stretch until that data isnā€™t available at compile time šŸ˜‰#2016-09-1213:40lvhI might be able to get around it and just move more stuff into compile-time-land#2016-09-1213:41Alex Miller (Clojure team)specs are data in s-expr form#2016-09-1213:42Alex Miller (Clojure team)we havenā€™t released it yet, but I have a spec for spec forms#2016-09-1213:43Alex Miller (Clojure team)(which revealed a lot of bugs in s/form :)#2016-09-1213:45lvhnice; I would very much like that#2016-09-1213:45lvhsince a hypothetical awful person might want to construct specs at runtime and have better feedback about why they donā€™t work šŸ˜‰#2016-09-1213:46Alex Miller (Clojure team)oh, I donā€™t think youā€™re awful :)#2016-09-1213:46Alex Miller (Clojure team)itā€™s reasonable#2016-09-1213:46Alex Miller (Clojure team)just not the primary use case we were working to support#2016-09-1213:47bret@alexmiller Ok, this will help me.
;; I want to check that an input map's keys are valid
;;   where the keys are unqualified, some required, some optional,
;;   and not allow keys outside that set.

; This is straight forward
(s/def :in/data (s/and
                  (s/keys :req-un [:in/id] :opt-un [:in/more])
                  (s/map-of #{:id :more} nil)))

; but I'm (kind of) repeating information.
;
; If I write

(def req-keys [:in/id])
(def opt-keys [:in/more])

(defn unk
  "Returns ns unqualified keys for (possibly) qualified ones."
  [& ks]
  (map #(-> % name keyword) ks))

(s/def :in2/data (s/and
                   (s/keys :req-un req-keys :opt-un opt-keys)
                   (s/map-of (set (apply unk (concat req-keys opt-keys))) nil)))

; I have not repeated the key values but s/key doesn't allow it.
What is the proper way to write the spec where I not repeating information? I didnā€™t initially see a way to piece it together from ā€˜smallerā€™ specs since the args in map-of is really just a set used as a predicate.
#2016-09-1213:48bretI could be missing something fundamental.#2016-09-1213:49Alex Miller (Clojure team)Iā€™d say generally that Rich believes in open maps and thatā€™s why this is not a feature provided out of the box#2016-09-1213:49Alex Miller (Clojure team)and that I think your first example is what I would do if I was doing it#2016-09-1213:49Alex Miller (Clojure team)(although nil is not a valid spec there - you want any?)#2016-09-1213:51Alex Miller (Clojure team)and I would use s/merge instead of s/and#2016-09-1213:51Alex Miller (Clojure team)which I think would gen better#2016-09-1213:53bretSo, s/merge can be used for combining more than s/keys (`s/map-of` in this case)?#2016-09-1214:00Alex Miller (Clojure team)s/merge is used to combine (union) map specs#2016-09-1214:00Alex Miller (Clojure team)it differs in not flowing conformed results like s/and and also in being better at gen'ing#2016-09-1214:00bretI get the open map approach and generally like it. One thought I had, that really relates to the reporting aspect, is that s/keys supports ā€˜andā€™/ā€˜orā€™ combinations of key vectors. So, keeping the form used for reporting as close to literal boolean expressions of literal key vectors is not a bad thing. Ok, thanks, this helps.#2016-09-1214:22rickmoynihanwhat's the best way to spec that something satisfies? a protocol?#2016-09-1214:27rickmoynihanobviously you can just use the (partial satisifies? Protocol) predicate... but is there a way for implementers to hook in and extend the generator to generate types that satisfy it? I could imagine that if implementers spec'd their constructing functions, you could get this almost for free.#2016-09-1214:37Alex Miller (Clojure team)nothing built-in#2016-09-1214:54jmglovHere's another fun one. Using the fixed version of the same spec as previously, I can use stest/check on it in my REPL:
kpcs.product-catalog.internal-test> (first (stest/check 'kpcs.product-catalog.internal/exclude-products))
{:spec #object[clojure.spec$fspec_impl$reify__14244 0x2c8e87a5 "
#2016-09-1214:56jmglovHowever, when I try to use it in a test, I get an exception. Here's what I'm trying to do:
(ns kpcs.product-catalog.internal-test
  (:require [clojure.spec.test :as stest]
            [clojure.test :refer [deftest is testing]]
            [kpcs.product-catalog.internal :as internal]))

(deftest exclude-products
  (testing "Specs"
    (let [res (first (stest/check 'kpcs.product-catalog.internal/exclude-products))]
      (is (= java.lang.String (type res)))))
#2016-09-1214:57jmglovAnd I get this:
java.util.concurrent.ExecutionException: java.lang.ClassCastException: clojure.lang.AFunction$1 cannot be cast to clojure.lang.MultiFn, compiling:(clojure/test/check/clojure_test.cljc:95:1)
 at java.util.concurrent.FutureTask.report (FutureTask.java:122)
    java.util.concurrent.FutureTask.get (FutureTask.java:192)
    clojure.core$deref_future.invokeStatic (core.clj:2290)
    clojure.core$future_call$reify__9352.deref (core.clj:6847)
    clojure.core$deref.invokeStatic (core.clj:2310)
    clojure.core$deref.invoke (core.clj:2296)
    clojure.core$map$fn__6856.invoke (core.clj:2728)
    clojure.lang.LazySeq.sval (LazySeq.java:40)
    clojure.lang.LazySeq.seq (LazySeq.java:56)
    clojure.lang.LazySeq.first (LazySeq.java:71)
    clojure.lang.RT.first (RT.java:682)
    clojure.core$first__6379.invokeStatic (core.clj:55)
    clojure.core/first (core.clj:55)
...
#2016-09-1214:59jmglovAny ideas?#2016-09-1215:00jmglovTo be clear, if I REPL into my kpcs.product-catalog.internal-test and run the code above, it works. If I run lein test, I get the exception.#2016-09-1215:00jmglovI don't see what should be different between the two.#2016-09-1215:09Alex Miller (Clojure team)thatā€™s a problem with lein testā€™s monkeypatching#2016-09-1215:09Alex Miller (Clojure team)itā€™s fixed in (not yet released) next version of test.check#2016-09-1215:09Alex Miller (Clojure team)but you can disable lein monkeypatching to fix#2016-09-1215:10Alex Miller (Clojure team):monkeypatch-clojure-test false#2016-09-1215:10jmglovGreat, thanks!#2016-09-1215:10Alex Miller (Clojure team)that will disable lein retest but otherwise should not affect what youā€™re doing#2016-09-1215:11jmglovJust tried it, and it works perfectly.#2016-09-1215:18Alex Miller (Clojure team)you are not the first person to encounter it :)#2016-09-1215:22jmglovThank goodness for that!#2016-09-1215:27jmglov@otfrom @tgk Here's a cheap hack to fit check into my standard tests:
(deftest exclude-products
  (testing "Specs"
    (let [result (-> (stest/check 'kpcs.product-catalog.internal/exclude-products)
                     first
                     :clojure.spec.test.check/ret
                     :result)]
      (if (true? result)
        (is result)
        (is (= {} (ex-data result)))))))
#2016-09-1215:28otfromjmglov: thx!#2016-09-1215:28jmglovI'll make a checking function out of it and throw it in a test lib. The output is decent enough with the humane-test-output plugin. šŸ™‚#2016-09-1215:29jmglovIf it's useful, you're welcome. Otherwise, I'm sorry for such a disgusting kludge! šŸ˜‰#2016-09-1215:48mike_ananevHi there! Sorry for very dumb question. How to define spec for function with variable args?#2016-09-1215:56bahulneelHi guys, I've just started using clojure.spec and was wondering how to use clojure.spec.test/check with clojure.test#2016-09-1215:56bahulneelsorry, missed the msg from @jmglov#2016-09-1216:38Alex Miller (Clojure team)@mike1452 (s/fdef myf :args (s/cat :map-params (s/? map?))) will take both 0 and 1 (map) arg#2016-09-1216:38Alex Miller (Clojure team)you can replace map? with something more specific too of course#2016-09-1217:20bfabryit seems awkward that core/defn has a special syntax for arity dispatch but spec/fdef doesn't provide one for speccing/testing#2016-09-1217:21bfabrydoesn't really affect me as I never use arity dispatch but I could see it sucking if you previously used it a lot#2016-09-1217:25Alex Miller (Clojure team)they are doing different things. most multi-arity functions share param definitions across arities and merging them works very nicely for this in most cases.#2016-09-1217:28bfabrysure, for the :args, but it doesn't make the :fn for say map less readable?#2016-09-1217:45Alex Miller (Clojure team)sure, although I think thatā€™s an unusual case#2016-09-1217:55bfabrytrue. I guess the common case would be reduce. smaller arity providing default value#2016-09-1218:57ikitommiIs there a reason why if-let and when-let canā€™t return :clojure.spec/invalid?#2016-09-1219:01hiredmanwell that is problematic#2016-09-1219:04hiredman (if-let [a 1] '::s/invalid) works if you need a work around#2016-09-1219:05ikitommi@hiredman cool, thanks. didnā€™t know that works too.#2016-09-1220:26Alex Miller (Clojure team)@ikitommi heh, thatā€™s fun#2016-09-1220:28Alex Miller (Clojure team)thereā€™s actually a ticket related to this#2016-09-1220:28Alex Miller (Clojure team)http://dev.clojure.org/jira/browse/CLJ-1966#2016-09-1221:07seancorfieldI added the if-let example above to that ticket. I suspect people will run into this in more and more situations as they try to write conformers.#2016-09-1221:13Alex Miller (Clojure team)and as there are more specā€™ed things#2016-09-1221:21agI need a spec that would generate vectors of values taking random elements from a predefined list, e.g: [:foo :bar] [:foo] [:baz :bar] []ā€¦ etc.#2016-09-1221:21aghow?#2016-09-1221:24Alex Miller (Clojure team)(s/coll-of #{:foo :bar :baz} :kind vector?)#2016-09-1221:27Alex Miller (Clojure team)you can also use the other options on coll-of to set :min-count, :max-count, :count constraints on the spec or :gen-max to cap what the generator will produce#2016-09-1221:28agrightā€¦ http://clojure.org/guides/spec#_collections#2016-09-1221:28agthanks!#2016-09-1306:55jmglov@bahulneel I'm working on upgrading a production project to Clojure 1.9 and spec right now, so I'll keep this channel informed of any new stuff I figure out with the clojure.test integration.#2016-09-1306:56jmglovGiven that clojure.spec.test/check takes a quoted symbol, I shouldn't even need a macro to accomplish what I want. A plain 'ol function should do the trick. šŸ™‚#2016-09-1310:00bahulneel@jmglov thanks, I'll keep my ear to the ground#2016-09-1311:18jmglovI'm having trouble getting clojure.spec.test/check to run a non-default number of tests. From reading the docs and the code, it looks like I should be able to do this:
(stest/check 'kpcs.event.processor/schema-version {:num-tests 1})
But it always runs 1000 tests:
({:spec #object[clojure.spec$fspec_impl$reify__14244 0x7b0d682d "
#2016-09-1311:18jmglovWhat am I missing?#2016-09-1311:22jmglovHrm... I realise that I'm dealing with a namespaced key. This doesn't work, either:
(stest/check 'kpcs.event.processor/schema-version {:clojure.test.check.stc/opts {:num-tests 1}})
#2016-09-1311:24jmglovOK, finally figured it out:
(stest/check 'kpcs.event.processor/schema-version {:clojure.spec.test.check/opts {:num-tests 1}})
({:spec #object[clojure.spec$fspec_impl$reify__14244 0x7b0d682d "
#2016-09-1311:24jmglovI've never seen the ::foo/bar-style keywords before, only ::bar.#2016-09-1311:25jmglovWhy does ::stc/opts in the clojure.spec.test namespace expand into :clojure.spec.test.check/opts?#2016-09-1311:49Alex Miller (Clojure team)clojure.spec.test sets up an alias of ā€˜stc to ā€˜clojure.spec.test.check#2016-09-1311:50Alex Miller (Clojure team)the ::stc/opts just says to create a fully-qualified keyword using the local alias stc so that expands#2016-09-1312:46jmglov@alexmiller Thanks for the explanation!#2016-09-1317:42uwocould someone point me to the reasoning behind fspec requiring :ret and fdef not?#2016-09-1317:47Alex Miller (Clojure team)I think they both used to and fdef was relaxed#2016-09-1317:47Alex Miller (Clojure team)fspec prob should be too#2016-09-1317:47Alex Miller (Clojure team)if you file a jira enhancement weā€™ll look at it#2016-09-1317:47Alex Miller (Clojure team)(in other words, I think no reason :)#2016-09-1317:48uwošŸ™‚ thanks#2016-09-1317:54esp1fyi :ret is apparently required for fspec in alpha12#2016-09-1317:56esp1
(s/explain (s/fspec :args empty?) (fn [] nil))
Success!
=> nil
(s/valid? (s/fspec :args empty?) (fn [] nil))
=> false
#2016-09-1318:16Alex Miller (Clojure team)just to be sure, thatā€™s the same thing @uwo said above right?#2016-09-1318:16uwoyes#2016-09-1318:17Alex Miller (Clojure team)ok, Iā€™m agreeing thatā€™s wrong if it wasnā€™t clear :)#2016-09-1318:19esp1yup just pointing out that itā€™s required now. in previous alphas :ret was actually not required on fspec, but in alpha12 that restriction is enforced#2016-09-1318:20jannisIs there a way to refer to a spec that is defined later in the same file? E.g. (s/def ::foo (s/with-gen ::bar ...)) ... (s/def ::bar ...) ?#2016-09-1318:22jannisI may be getting an unspecified behavior, where sometimes the spec is found and sometimes it isn't, resulting in conform returning a tagged, conformed value and sometimes returning the untagged original value.#2016-09-1318:22jannisGranted, the second spec is more complex than in this example but the first is pretty much that.#2016-09-1318:24jannis(describe ::foo) says :clojure.spec/unknown.#2016-09-1318:29darwinclojurescript related question, Iā€™m working on a library which uses clojure.spec to describe a data structure using s/*, the data structure can be used statically during compilation in macros, or during runtime in cljs. all is nice and shiny on clj side[1], but on cljs side, I want native js arrays to be treated as collections for the purpose of speccing, but unfortunatly s/* does not treat them such. Ended up writing custom predicates for native array case[2], just wondering if there is a better way to teach s/* to walk native js arrays the same way as cljs vectors. [1] https://github.com/binaryage/cljs-oops/blob/master/src/lib/oops/sdefs.clj [2] https://github.com/binaryage/cljs-oops/blob/master/src/lib/oops/sdefs.cljs#2016-09-1318:33jannisFunny. (s/def ::foo (s/with-gen (s/and ::bar) ..)) makes it work at all times.#2016-09-1318:37Alex Miller (Clojure team)@jannis I donā€™t know of any issue with that. however, one of the perf changes in alpha12 means that one spec gets compiled into anotherā€™s definition (via delay) so when you change (re s/def) a spec at the repl, downstream specs need to be re-s/defā€™ed as well#2016-09-1318:37Alex Miller (Clojure team)is there any chance you are seeing this behavior at the repl?#2016-09-1318:37Alex Miller (Clojure team)maybe when you made it ā€œworkā€ you just caused a new s/def that was needed#2016-09-1318:38Alex Miller (Clojure team)@esp1 I donā€™t know of any change that would have altered this behavior in alpha12, but thatā€™s possible (or that it changed at some point during the alphas)#2016-09-1318:40esp1actually previously i was using tonskyā€™s future-spec port of spec to 1.8, so possibly it was something specific to that, not sure#2016-09-1318:44jannis@alexmiller: If I load the namespace that these specs are in I get it in the REPL. If I reload the same namespace, it disappears. And I get it when running tests outside the REPL (using boot test).#2016-09-1318:45jannisI also get it when defining them in the REPL: https://gist.github.com/Jannis/d249d6d21e92f6bca7736c79eb008b49#2016-09-1318:45jannisThis is alpha11 though. I can't use alpha12 at the moment because it fails requiring DataScript due to an invalid ns expression in it.#2016-09-1318:50Alex Miller (Clojure team)I canā€™t even run the first line of that with a bare alpha12 repl, not sure about earlier#2016-09-1318:51Alex Miller (Clojure team)I realize this is slimmed down, but I donā€™t think you should expect to with-gen over a spec that doesnā€™t exist yet#2016-09-1318:52jannisThese specs are for a language that is recursive (think: Om Next query expressions). How would I include the recursive nature in the spec other than refering to the top-level spec in a part of it?#2016-09-1318:53jannisThe top-level spec needs the sub-spec, the sub-spec needs the top-level spec. Hmm...#2016-09-1319:04Alex Miller (Clojure team)yeah, thatā€™s fine#2016-09-1319:04Alex Miller (Clojure team)Iā€™m talking about with-gen in particular#2016-09-1319:05Alex Miller (Clojure team)you can provide gen overrides separately too of course. gen with recursion is generally hard.#2016-09-1319:05jannisHow do I do it separately?#2016-09-1319:16Alex Miller (Clojure team)fns like instrument take an override map#2016-09-1319:16Alex Miller (Clojure team)from name (or path) to generator#2016-09-1319:31jannisAh, yes. So that would also work with clojure.spec.test/check. I'd prefer to have all generators defined along with the actual spec so it's as reusable as possible (e.g. in combination with fdef args in functions other people may write). I'll see what the best solution is. Thanks!#2016-09-1319:37bhaganyI'm trying to understand why nested calls s/cat don't operate the way I expect - to illustrate:
cljs.user> (s/explain (s/cat :x number? :y number? :z number?) [0 0 0])
Success!
nil
cljs.user> (s/explain (s/cat :coords (s/cat :x number? :y number? :z number?)) [[0 0 0]])
In: [0] val: [0 0 0] fails at: [:coords :x] predicate: number?

nil
In my actual situation, I'm trying to reuse the inner s/cat as both the return value of a fn, and an argument to another fn. Also, it does work the way I expect if I use (s/tuple number? number? number?), but I'd like to be able to conform it and get a map. Am I missing something?
#2016-09-1319:40seancorfield(s/explain (s/cat :coords (s/spec (s/cat :x number? :y number? :z number?))) [[0 0 0]]) => Success!#2016-09-1319:40bfabry@bhagany the regex operators assume concatenation by default, you need to wrap nested calls in s/spec#2016-09-1319:41bhaganyaha, thanks @seancorfield and @bfabry!#2016-09-1319:41bfabrycovered towards the bottom of this section http://clojure.org/guides/spec#_sequences#2016-09-1319:43bhaganyI'd read it, but I think I have not yet internalized that cat is a regex operator. thanks again#2016-09-1320:05Alex Miller (Clojure team)any set of nested regex ops (`cat`, alt, ?, +, *, &, keys*) describes the structure of a single sequential context#2016-09-1320:06Alex Miller (Clojure team)which is just like regex on a string#2016-09-1320:07Alex Miller (Clojure team)strings are different in that a single character in a string canā€™t also be a string (but that is true of elements in a Clojure sequential collection)#2016-09-1320:08hiredmanthe video of dnolen's talk on parsing with derivatives / spec is up https://www.youtube.com/watch?v=FKiEsJiTMtI#2016-09-1322:14gfredericks@alexmiller: test.check already supports collections distinct by custom key-fns#2016-09-1322:15gfredericksE.g. gen/vector-distinct-by#2016-09-1322:21gfredericks@jmglov: https://github.com/gfredericks/schpec exists in case you don't want to bother managing a whole library for that function#2016-09-1323:07Alex Miller (Clojure team)@gfredericks nice#2016-09-1323:40wilkesAre there any conventions around where to put specs? In the namespace, in a special whatever.spec namespace, etc..#2016-09-1323:46seancorfield@wilkes "it depends" is the answer to that.#2016-09-1323:46wilkesšŸ™‚#2016-09-1323:47seancorfieldAt World Singles, our specs are very data centric so they mostly live in their own namespace and we pull them into other namespaces as needed, and then our tests pull them in to instrument / check the code.#2016-09-1323:48seancorfieldIn clojure.java.jdbc all the fdefā€™s are in a separate namespace so the code still works with pre-1.9.0 versions of Clojure.#2016-09-1323:49seancorfieldBut if you know youā€™re only going to use 1.9.0 onward and youā€™re mostly specā€™ing functions, rather than data, Iā€™d imagine it would be convenient to put fdefā€™s next to their defnā€™s in the same namespace.#2016-09-1323:53wilkes@seancorfield Thanks, that helps. I was expecting this to be at least in flux until people have spent some time with specs in their projects.#2016-09-1323:56Alex Miller (Clojure team)Clojure itself is putting specs in clojure.core.specs#2016-09-1400:17hiredmanit seems like that could end up mixing testing code in with, production code (for lack of a better description)#2016-09-1400:17hiredmanis the idea that people will do similar lazying loading kind of stuff like clojure.spec does for test.check?#2016-09-1400:24seancorfield"mixing testing code in with production code" ā€¦ how? Itā€™s not the case with how WS is using spec at the moment.#2016-09-1400:25seancorfieldOur spec namespaces have just specs in them ā€” which our (production) code uses for explicit validation and conforming of data at each "system" boundary (input; domain; persistence).#2016-09-1400:26seancorfieldIn addition, our test code can pull in the specs and instrument and/or check code. Any test-specific code would live in the test (expectations) namespace, not the main (production) namespace.#2016-09-1400:27seancorfield(itā€™s certainly possible Iā€™ve missed some subtlety here butā€¦)#2016-09-1400:27hiredmando you have specs with custom generators?#2016-09-1400:30hiredmanor, I guess I should say, do you define your own predicates with generators? it seems like that would depend at least on test.check, so you would either need to do lazy loading or have a runtime dependency on test.check#2016-09-1400:31seancorfieldAh, gotcha...#2016-09-1400:31hiredmanI haven't used spec in anger, so I dunno how it plays out, just something I wonder about#2016-09-1400:32bfabry@hiredman you can specify the generators in the spec.test/check or instrument call, instead of with the spec itself#2016-09-1400:32seancorfieldWe lazy-load via the functions that return generators (and do so in a separate ns):
;; copyright (c) 2016 world singles llc

(ns ws.spec.generators
  "Helpers for our specs that need custom generators.
  Dynamically loads the namespaces as needed so we can compiler
  this without test dependencies.")

(defn fn-string-from-regex
  "Return a function that produces a generator for the given
  regular expression string."
  [regex]
  (fn []
    (require '[com.gfredericks.test.chuck.generators :as xgen])
    (let [string-from-regex (resolve 'xgen/string-from-regex)]
      (string-from-regex regex))))
#2016-09-1400:32seancorfieldOoh, typo. compiler instead of compile...#2016-09-1400:32hiredmanah#2016-09-1400:35seancorfieldWe have a few with-gens that leverage stuff in clojure.spec or code thatā€™s already available in production, otherwise we defer to our ws.spec.generators "proxy" namespace.#2016-09-1400:36seancorfieldBut, yeah, ensuring those sort of test artifacts only get loaded when weā€™re testing was something we had to navigate at first...#2016-09-1400:36seancorfieldIn our build.boot we have
(deftask testing-context
  "Provide main testing context."
  []
  (merge-env! :dependencies '[[com.gfredericks/test.chuck "0.2.7" :scope "test"
                               :exclusions [*/*]]
                              [javax.servlet/servlet-api "2.5" :scope "test"]
                              [org.clojure/test.check "0.9.0" :scope "test"]])
  identity)
#2016-09-1400:37seancorfieldand thatā€™s only used by our test tasks#2016-09-1400:39bnoguchiI have been having trouble specā€™ing an function arg to another function where the former can be either a function with arity 0 or a function with arity 1 (both not variadic functions). Any ideas?
(s/explain (s/fspec :args (s/alt :arity-0 (s/cat) :arity-1 (s/cat :arg1 any?)) :ret any?) (fn []))
; val: (nil) fails predicate: (apply fn),  Wrong number of args (1) passed to: user/eval16844/fn--16845
#2016-09-1400:44bfabry@bnoguchi your spec seems to be working. you said the function should accept 1 or 0, your function only accepts 1#2016-09-1400:45hiredmanI think you need to pull the alt out#2016-09-1400:45bfabry@bnoguchi fwiw there's a briefer way to describe that spec: (s/cat :arg1 (s/? any?))#2016-09-1400:46bnoguchi@bfabry Itā€™s written to alternatively accept 1 or 0.#2016-09-1400:46bfabry@bnoguchi your spec is, but the anonymous function (fn [] ) only accepts 0 arguments#2016-09-1400:46hiredman(s/explain (s/alt :a0 (s/fspec :args (s/cat) :ret any?) :a1 (s/cat :a1 any?)) (fn []))#2016-09-1400:46bfabry
boot.user=> (s/explain (s/fspec :args (s/cat :arg1 (s/? any?)) :ret any?) (fn []))
val: (nil) fails predicate: (apply fn),  Wrong number of args (1) passed to: user/eval2135/fn--2136
nil
boot.user=> (s/explain (s/fspec :args (s/cat :arg1 (s/? any?)) :ret any?) (fn [a]))
val: () fails predicate: (apply fn),  Wrong number of args (0) passed to: user/eval2260/fn--2261
nil
boot.user=> (s/explain (s/fspec :args (s/cat :arg1 (s/? any?)) :ret any?) (fn [& a]))
Success!
nil
#2016-09-1400:47bnoguchi@bfabry That is not what Iā€™m after, however#2016-09-1400:47hiredmanoh, whoops#2016-09-1400:47bnoguchii.e., I donā€™t want to spec a variadic function. Perhaps this form will help#2016-09-1400:47hiredman
(s/explain (s/alt :a0  (s/fspec :args (s/cat) :ret any?) :a1 (s/fspec :args (s/cat :a1 any?) :ret any?)) (fn []))
#2016-09-1400:48hiredmanif you have two fspecs, with the alt outside, then it seems like you get he behavior you want#2016-09-1400:49hiredmanI think that makes sense, but it is hard to write the distinction in prose#2016-09-1400:49hiredmana function that takes 0 or 1 arg vs a function that takes 0 args, or a function that takes 1 arg#2016-09-1400:49bfabryit's not that it's variadic, it's that a variadic function satisfies the spec. so does a function that is arity 1 or 0
boot.user=> (defn foo
       #_=>  ([] nil)
       #_=>  ([a] nil))
#'boot.user/foo
boot.user=> (s/explain (s/fspec :args (s/cat :arg1 (s/? any?)) :ret any?) foo)
Success!
nil
#2016-09-1400:51hiredmanoh, am I am just wrong, and not paying close enough attention#2016-09-1400:56hiredman
user=> (s/explain (s/or :a0  (s/fspec :args (s/cat) :ret any?) :a1 (s/fspec :args (s/cat :a1 any?) :ret any?)) (fn []))
Success!
nil
user=> 
#2016-09-1400:56hiredmanthere we go, of course it needs to be s/or not s/alt#2016-09-1400:56hiredman(outside of matching the arglist#2016-09-1400:56hiredman)#2016-09-1400:57bfabryoh I'm sorry you wanted to spec either one or the other, not a function that expects both. in that case yes you need two fspec's (two different fiunction specs) not one fspec (a single function that does two things)#2016-09-1400:57bnoguchi
(defn arity [f] (-> f class .getDeclaredMethods first .getParameterTypes alength))

(defn fn-expecting-a-fn-arg-either-0-or-1-arity [func] (if (zero? (arity func)) (func) (func 1)))

(s/fdef fn-expecting-a-fn-arg-either-0-or-1-arity :args (s/cat :fn (s/fspec :args (s/or :arity-0 (s/cat) :arity-1 (s/cat :arg1 any?)) :ret any?)) :ret any?)

(require '[clojure.spec.test :as stest])

(stest/instrument `fn-expecting-a-fn-arg-either-0-or-1-arity)

(fn-expecting-a-fn-arg-either-0-or-1-arity (fn []))
throws
ExceptionInfo Call to #'user/fn-expecting-a-fn-arg-either-0-or-1-arity did not conform to spec:
val: (#object[user$eval16875$fn__16876 0x5fa44d5a "
#2016-09-1400:58hiredmanyou can't put the choice (alt or or) in the args for the function, because that means the function needs to satisfy both#2016-09-1400:58bfabry@hiredman's example is correct, you want to s/or two s/fspec's#2016-09-1400:59hiredmanyou have to spec the possible arguments distinctly in different f/specs#2016-09-1401:03bnoguchi@hiredman Yeah I think youā€™re right#2016-09-1401:04bnoguchiThis works
(s/fdef fn-expecting-a-fn-arg-either-0-or-1-arity :args (s/alt :arity-0 (s/fspec :args (s/cat) :ret any?) :arity-1 (s/fspec :args (s/cat :arg1 any?) :ret any?)) :ret any?)
#2016-09-1411:53jetzajacHello here! Iā€™m trying to use spec with a DataScript app. And I want to validate several different things: 1) tx-data in a map form 2) results of a pull 3) entities In these 3 flavours of data keywords may have different sorts of values. like itā€™s ok to have an int as a value of ref attr in tx-data, not in a pull. Itā€™s ok to miss some attrs in pull result, not in a tx-data with negative id (creating a new entity, we have a notion of required attrs). And the question is if we can have different spaces of specs? like separate registries and somehow refer them in spec and make it narrow down to this space. Say, space of tx-data and pulls that have to be validated differently. Not sure if I described the demand clear, bit hope I could find somebody who does these kind of things or be clarified why itā€™s impossible or is a bad idea. Thanx.#2016-09-1412:04jetzajacnot sure if I like the idea to have one (private) atom for registry. why not a dynamic binding?#2016-09-1412:16jmglov@bahulneel @otfrom This is what I'm using in tests so far:
(defn- check [function num-tests]
  (if num-tests
    (stest/check function {:clojure.spec.test.check/opts {:num-tests num-tests}})
    (stest/check function)))

(defn checking
  ([function]
   (checking function nil))
  ([function num-tests]
   (testing "Schema"
     (let [result (-> (check function num-tests)
                      first
                      :clojure.spec.test.check/ret
                      :result)]
       (if (true? result)
         (is result)
         (is (= {} (ex-data result))))))))

(defn fails-spec [f & args]
  (try
    (let [res (apply f args)]
      (is (instance? Exception res)))
    (catch Exception e
      (is (contains? (ex-data e) :clojure.spec/problems)))))
#2016-09-1412:20jmglovAnd then tests can look like this:
(deftest plus
  (checking 'foo/plus)
  (testing "Adding 0"
    (is (= 1 (foo/plus 1 0))))
  (testing "Invalid input"
    (fails-spec foo/plus 1 :forty-two)))
#2016-09-1412:27jmglovIs there an "any" spec?#2016-09-1412:27bahulneel@jmglov I've found that i need to extend the (if (true? result) ... to
(cond
          (true? result) (t/is result)
          (nil? (ex-data result)) (t/is (= {} result))
          :else (t/is (= {} (ex-data result))))
#2016-09-1412:28jmglov@bahulneel Thanks!#2016-09-1412:29jmglovI see that @esp1 pointed out that :ret is required for fdef now, and I'm trying to spec a function for which the return value is effectively Unit.#2016-09-1412:30jmglovSo I don't care what it returns, only that spec is happy with it. šŸ˜‰#2016-09-1412:31bahulneel@jmglov What about a predicate like (constantly true)#2016-09-1412:31bahulneel(def any? (constantly true))#2016-09-1412:35jmglovProbably shouldn't call it any?, though. šŸ˜‰#2016-09-1412:35jmglov
user> any?
#function[clojure.core/any?]
#2016-09-1412:35bahulneelyeah, that was more for illustration, best to make it explicit and not hide behind a name#2016-09-1412:36jmglovI actually like naming it. I think that makes the intention quite clear. I'll probably just call it unit.#2016-09-1412:36jmglovClear enough to me.#2016-09-1412:36bahulneelsure#2016-09-1412:38jmglovThanks for that! It's a nice, simple solution.#2016-09-1412:41bahulneel@jmglov no problem#2016-09-1412:42bahulneel@jmglov any plans to throw that testing code into a little library to house test helpers?#2016-09-1412:43jmglov@bahulneel I put it in its own namespace, but I'm not planning to lib it up, no.#2016-09-1412:43jmglovNot sure the Clojure community needs yet another library of utility functions. šŸ˜‰#2016-09-1412:44jmglovI figure best practises will emerge as more people start mixing specs into their tests.#2016-09-1412:44bahulneelfair enough. Maybe a GIST would do the trick, at least then it's updatable and discoverable#2016-09-1412:45jmglovI'll stick it in a Gist.#2016-09-1412:45jmglovHaha, great minds think alike.#2016-09-1412:45jmglovI'll do that.#2016-09-1412:47bahulneelalso if you put a namespace declaration at the top then, in theory, you could include it as a git submodule in your source tree#2016-09-1412:55jmglov@bahulneel Here it is: https://gist.github.com/jmglov/30571ede32d34208d77bebe51bb64f29#2016-09-1413:08bahulneel@jmglov thanks, I've added it as a submodule and then added the directory to my test-paths#2016-09-1413:08jmglovNinja!#2016-09-1413:08jmglovPlease ping me as you find improvements.#2016-09-1413:09bahulneelwill do#2016-09-1413:21madstapThere is already a spec utils library that @gfredericks made... https://github.com/gfredericks/schpec#2016-09-1413:22madstapJust throwing that out there, maybe useful to collect things like these in one place#2016-09-1413:40jmglov@madstap Thanks for the heads up!#2016-09-1413:40jmglovhttps://github.com/gfredericks/schpec/issues/14#2016-09-1414:52bahulneel@jmglov I noticed that you need to require [clojure.test.check.clojure-test] so that it's loded as some useful multimethods are added#2016-09-1416:01kkruitIs there any reason to not use instrument for validation if the behavior I want is for the function to throw an error if the data passed is invalid?#2016-09-1416:01kkruit(in production)#2016-09-1416:02Alex Miller (Clojure team)no (as long as it meets your perf concerns)#2016-09-1416:04kkruitI'm thinking it wouldn't be less performant than doing a verify?, throw, and explain, would it?#2016-09-1416:06kkruitI can run some tests...#2016-09-1416:09Alex Miller (Clojure team)with alpha12, itā€™s pretty fast but test it and see#2016-09-1416:09kkruitWill do. Thanks!#2016-09-1416:20kkruitand by verify? i meant valid?...#2016-09-1421:40kkruitSo I'm looking at using spec in place of schema for a rest api and am running into a question about the best way to handle something. I have something like this:#2016-09-1421:43kkruitI like using :ns/map-key in combination with :req-un like that but I don't like how long the namespace is/could get any thoughts? Am i shoehorning :ns/map-key somewhere where it shouldn't go?#2016-09-1421:44arohner@kkruit you can alias namespaces using :require#2016-09-1421:45arohner
(:require [my.controller.spec.update :as c])  (s/def ::c/body ā€¦))
#2016-09-1421:46kkruitdoh! I was trying to do:#2016-09-1421:46kkruit(:require [my.controller.spec.update :as c]) (s/def :c/body ā€¦))#2016-09-1421:47kkruitThanks @arohner#2016-09-1421:48arohnerthat would look for an unaliased namespace named ā€˜c, rather than resolving the alias ā€˜c#2016-09-1421:48arohnernp#2016-09-1422:44stuartmitchellIs this a known issue
(def common-attributes [::id ::type ::from-date ::to-date ::style])

(s/def ::common
  ;; defines the common parts of an activity
  (s/keys :req-un common-attributes))
Gives this compiler error
----  Could not Analyze  src/day8/plan8/spec/activity.cljs   line:24  column:3  ----

  Don't know how to create ISeq from: clojure.lang.Symbol

  22  (s/def ::common
  23    ;; defines the common parts of an activity
  24    (s/keys :req-un common-attributes))
        ^--- Don't know how to create ISeq from: clojure.lang.Symbol
  25  
#2016-09-1422:45seancorfields/keys is a macro so it doesnā€™t evaluate its arguments (so this is by design)#2016-09-1422:46stuartmitchellstrangely this works
(def common-attributes [::id ::type ::from-date ::to-date ::style])

(s/def ::common
  ;; defines the common parts of an activity
  (s/keys :req-un (concat common-attributes)))
#2016-09-1422:46stuartmitchellbut will it fail later on?#2016-09-1422:46seancorfieldIt does not do what you think it does.#2016-09-1422:47seancorfield(Iā€™m a bit surprised it conforms to specā€™s specs ā€” are you trying this on Alpha 12 or an earlier version?)#2016-09-1422:49stuartmitchell1.8.0 I'll bump it up#2016-09-1422:51stuartmitchellbut how should we compose specs i.e. I want spec ::b to be spec ::a with some extra keys#2016-09-1422:51stuartmitchellthis is what I wanted
(def common-attributes [::id ::type ::from-date ::to-date ::style])

(s/def ::common
  ;; defines the common parts of an activity
  (s/keys :req-un common-attributes))

(s/def ::tv
  ;; defines the attributes for a tv activity
  (s/keys :req-un (concat common-attributes [::daymask ::tarp ::cpt ::rate
                                              ::market])))
#2016-09-1422:54seancorfieldOh, so youā€™re using the backport of clojure.spec rather than the official version?#2016-09-1422:55seancorfieldYou want clojure.spec/merge for merging key specs.#2016-09-1422:55stuartmitchellahh thank you#2016-09-1422:55seancorfield
boot.user=> (doc s/merge)
-------------------------
clojure.spec/merge
([& pred-forms])
Macro
  Takes map-validating specs (e.g. 'keys' specs) and
  returns a spec that returns a conformed map satisfying all of the
  specs.  Unlike 'and', merge can generate maps satisfying the
  union of the predicates.
#2016-09-1422:56seancorfield(note: also a macro)#2016-09-1423:01stuartmitchellThx, I also tried the concat version on 1.9.0-alpha12 and it didn't give a compiler error however I am using clojurescript as well so someone should probably try it on vanilla java, if indeed it should fail spec's specs#2016-09-1423:12Alex Miller (Clojure team)there are no included spec specs#2016-09-1423:13Alex Miller (Clojure team)Iā€™ve made some but they are a bit out of date and unclear whether we will release them. there are some recursion issues with checking specs in spec too. :)#2016-09-1423:32seancorfield@alexmiller Am I right that (s/keys :req-un (concat common-attributes)) is going to create a spec where the two required keys are the symbols concat and common-attributes? And that will error out when usedā€¦?
#2016-09-1500:21kennyCan you set the :default method for a multi-spec? Just adding a multimethod with :default for the dispatch value results in no method when calling explain. Small example:
(defmulti event-type :event/type)
(defmethod event-type :event/search [_]
  (s/keys :req [:event/type :event/timestamp :search/url]))
(defmethod event-type :default [_]
  (s/map-of any? any?))

(s/def :event/event (s/multi-spec event-type :event/type))
(s/explain-data :event/event
                {:event/type      :foo
                 :event/timestamp 1463970123000
                 :search/url      ""})
=> #:clojure.spec{:problems [{:path [:foo], :pred event-type, :val {:event/type :foo, :event/timestamp 1463970123000, :search/url ""}, :reason "no method", :via [:event/event], :in []}]}
#2016-09-1500:30hiredmanit looks like no, because multi-spec re-implements the multimethod dispatch logic without :default#2016-09-1500:34hiredmanI think https://github.com/clojure/clojure/blob/master/src/clj/clojure/spec.clj#L888 would need a (or (.-defaultDispatchValue mm) ...)#2016-09-1500:38hiredmanI suppose the default stuff would be tricky to handle well for generative testing#2016-09-1500:51kennyIs there reasoning behind this?#2016-09-1500:54kennyBehind not handling :default in defmethod ^#2016-09-1500:54kennyOr has it just not been discussed?#2016-09-1501:09Alex Miller (Clojure team)@seancorfield that would be my guess too#2016-09-1501:10Alex Miller (Clojure team)@kenny if :default doesnā€™t work, it should be made to#2016-09-1501:10Alex Miller (Clojure team)at least as far as my thinking atm, so jira appreciated#2016-09-1501:10Alex Miller (Clojure team)I did put a patch in alpha12 to make hierarchies work again, but I donā€™t recall testing :default stuff#2016-09-1501:12kennyOh shoot, I'm still on alpha11 because of the hash changes. I'll test it out in alpha12 and if it doesn't work I'll file a jira issue.#2016-09-1501:15kenny@alexmiller Yes it is fixed in alpha12. Thanks!#2016-09-1506:11curlyfryHow "safe" am I in using spec in ClojureScript? Is it subject to any large changes in the future?#2016-09-1509:43jmglovAny tips for writing a spec for the string version of a positive int? Here's what I'm doing now:
(defn- pos-int-string? [s]
  (try
    (pos-int? (Integer/parseInt s))
    (catch Exception _
      false)))

(s/def ::id pos-int-string?)
#2016-09-1509:43jmglovI assume that will need a custom generator though, right?#2016-09-1509:43jmglovAny more idiomatic way to do it?#2016-09-1511:08flipmokidCould spec be used to parse a vector such as '[1 + 3 * 4] and conform the input in such a way so that operator precedence is preserved?#2016-09-1514:16jmglovThe first time you see a function spec actually catching invalid input data is absolutely wonderful!#2016-09-1514:33Alex Miller (Clojure team)@curlyfry spec is in alpha and subject to change (but probably unlikely to change dramatically, more additive)#2016-09-1514:38jmglovIs there a way to use conform to drop all map keys not explicitly mentioned in an s/keys spec?#2016-09-1514:39jmglovLet's say I have something like this:
(s/def ::foo string?)
(s/def ::bar int?)
(s/def ::thingy
  (s/keys :req-un [::foo]
          :opt-un [::bar])
#2016-09-1514:41Alex Miller (Clojure team)you could use something like (s/and (s/keys ā€¦) (s/conformer #(select-keys % #{::foo ::bar}))#2016-09-1514:41jmglovI'd like to do this:
> (s/conform ::thingy {:foo "yup", :bar 42, :baz [1, 2, 3]})
{:foo "yup", :bar 42}
#2016-09-1514:42curlyfry@alexmiller: Thanks!#2016-09-1514:43jmglov@alexmiller Wow! You guys thought of everything!#2016-09-1514:43jmglovBTW, it is possible to run spec.test/check on a private function; my failure to do so was operator error. šŸ™‚#2016-09-1514:46Alex Miller (Clojure team)No#2016-09-1514:46Alex Miller (Clojure team)Right now at least#2016-09-1514:46Alex Miller (Clojure team)No one has filed an enhancement for that yeah afaik#2016-09-1514:47Alex Miller (Clojure team)I'm not sure whether it should be supported or not#2016-09-1514:47jmglovWell, it works. šŸ™‚#2016-09-1514:49jmglov(stest/check 'foo.bar/baz) is just fine, even if baz is defn-'d.#2016-09-1514:50jmglovI personally think it should not be disallowed, since Clojure generally doesn't try too hard to save you from yourself with respect to "private" stuff in a namespace.#2016-09-1514:50jmglove.g. you can happily call a private function as long as you grab its var.#2016-09-1518:02uwohow might I spec a map whose keys I do not know, but whose values all have the same shape?#2016-09-1518:03bfabry@uwo map-of#2016-09-1518:03uwodoh. thanks#2016-09-1518:03bfabrynp#2016-09-1518:52richiardiandreaI have a little problem with this small test:
(defn all-divisions []
  {:test nil})

(s/fdef all-divisions
        :args empty?
        :ret #(do (log/error "here") (not (data-format/nil-keys? %))))

:: repl
(test/instrument `all-divisions) ;; => [app.db.queries/all-divisions]
(all-divisions) ;; => {:test nil}
Am I spec-ing it right? (I don't see "here" in the logs)
#2016-09-1519:17bfabry@richiardiandrea instrument does not check :ret or :fn#2016-09-1519:17richiardiandreaoh#2016-09-1519:18richiardiandreaonly :args ?#2016-09-1519:18bfabryyup#2016-09-1519:18bfabry:ret and :fn are checked during clojure.spec.test/check checks though#2016-09-1519:18richiardiandreaand what do I need to use to check :ret at runtime while developing ?#2016-09-1519:18richiardiandreaah ok#2016-09-1519:18richiardiandreatnx @bfabry I did not know that#2016-09-1519:19bfabrynp#2016-09-1520:32richiardiandreaso if I want to instrument all the instrumentable symbols in my project, should I run! on instrumentable-syms the function instrument?#2016-09-1520:33Alex Miller (Clojure team)No just call instrument with no args#2016-09-1520:34Alex Miller (Clojure team)Also be aware of enumerate-ns#2016-09-1520:34richiardiandrea@alexmiller cool, sorry it was right there in the docs#2016-09-1520:35richiardiandreareading through it now#2016-09-1522:17smwoh my godā€¦ s/exercise is so cool.#2016-09-1609:40jmglovI have a couple of timestamp string specs that require custom generators. I'm using a function to make the generators with the following function spec:
(s/fdef make-timestamp-gen
        :args (s/cat :min-year pos-int?
                     :max-year pos-int?
                     :formatter #(instance? DateTimeFormatter))
        :ret clojure.test.check.generators/generator?)
I'd really like to spec the function, but that would require adding org.clojure/test.check to my non-dev dependencies, which feels a little yucky. Is there another way to ensure that the return value of the function is a generator? clojure.spec.gen doesn't have a generator? function.
#2016-09-1611:34jmglovI have a data structure like this:
{:schema {:version "v1"}
 :payload {:type "v1" ...}}
The schema version can either be "v1" or "v2", and the contents of the payload vary based on this. Is there any good way to write a custom generator that will match up version and payload? I can write the conformer reasonably easily, but I can't think of a good way to do the generator.
#2016-09-1611:57jmglovOK, I came up with something that feels only mildly hackish:
(defn- basic-gen []
  (->> (s/gen (s/keys :req-un [::schema]))
       (gen/fmap (fn [{{:keys [version]} :schema :as event}]
                   (let [payload-spec (payload-spec-for-schema version)]
                     (assoc event :payload (gen/generate (s/gen payload-spec))))))))

(s/def ::event (s/with-gen
                 (s/keys :req-un [::schema
                                  ::payload])
                 basic-gen))
#2016-09-1611:58jmglovIs this OK, or is there a better way?#2016-09-1613:17jmglovWriting a custom generator / conformer pair has led me down a merry path of learning a lot more about spec.#2016-09-1613:17jmglovI have what I hope is the ultimate question in this particular journey:#2016-09-1613:20jmglovGiven a custom conformer, how to I make the s/explain output meaningful? At the moment, my conformer just says:
user> (s/conform ::event intentionally-bogus-event)
:clojure.spec/invalid
user> (s/explain ::event intentionally-bogus-event)
val: :kpcs.spec.event/basic fails predicate: :clojure.spec/unknown
#2016-09-1616:09hiredmanre: the v1 vs v2 thing, that sounds sort of like what multi-spec does#2016-09-1616:21hiredmanre: :clojure.spec/unknown, my guess is it is something related to the Specize protocol, the fall through Object case for that protocol uses unknown as the name for the predicate#2016-09-1616:38jmglov@hiredman I remember reading about multi-spec now that you bring it up. I think you're right there!#2016-09-1616:39jmglovAny clue what I would need to do to solve the :clojure.spec/unknown problem?#2016-09-1616:40jmglovOr maybe multi-spec would sort that for me anyway. šŸ™‚#2016-09-1616:40hiredmanyeah, maybe#2016-09-1616:42Alex Miller (Clojure team)@jmglov thatā€™s a known issue that Rich and I are working on#2016-09-1616:42Alex Miller (Clojure team)just hasnā€™t been finished yet#2016-09-1616:42jmglovNice!#2016-09-1617:17uwoIā€™ve instrumented the following spec and it catches errors except for the optional parameter. Any advice?
(s/def ::skip integer?)
(s/def ::limit integer?)
(s/def ::request-options (s/keys :un-opt [::skip ::limit]))
(s/fdef request!
  :args (s/cat :field-path ::field-path
               :query-data ::query-data
               :options (s/? ::request-options)))
#2016-09-1617:18uwoI can provide any value for skip/limit and it wonā€™t complain#2016-09-1617:20bfabry@uwo :opt-un not :un-opt#2016-09-1617:20uwojeeze, Iā€™m on a roll with stupid questions. thanks!#2016-09-1617:21bfabryhaha it's all good. it'd be nice if spec caught those, but I imagine having spec spec'd would actually be a rather difficult problem#2016-09-1618:57Alex Miller (Clojure team)I have spec specs but using them presents some problems#2016-09-1619:46liamddoes anyone know of any guides that go beyond explaining how spec works and show how to integrate it into a real project?#2016-09-1619:53hiredmanyou know, there isn't a non-alpha version of clojure released with spec yet#2016-09-1619:54liamdi know, but how is it intended to be used? what do i do with my specs in an actual codebase?#2016-09-1619:55hiredmanlike where to put specs in relation to what they are specing?#2016-09-1619:56liamdyeah i found this
(defn person-name
  [person]
  {:pre [(s/valid? ::person person)]
   :post [(s/valid? string? %)]}
  (str (::first-name person) " " (::last-name person)))
#2016-09-1619:57liamdand i see the instrument function#2016-09-1619:58liamdis the idea that you have them as like accessories to your test suite or that they would run on deployed code?#2016-09-1619:58hiredmanI think spec can be split in to 1. descriptions of data 2. ways to use those descriptions#2016-09-1619:58liamdyeah so 2 is where i'm foggy#2016-09-1619:59hiredmanright#2016-09-1619:59hiredman#2 can be basically anything, but spec provides a few things out of the box#2016-09-1620:00hiredmana. generating data that matches the description b. checking if data matches the description#2016-09-1620:00hiredmanand it actually does b. in a few different ways#2016-09-1620:01hiredmanone way it can check that data matches is you can call instrument#2016-09-1620:01hiredmanand instrument will instrument your functions with checks (and some of those checks rely on generating data, so you really shouldn't instrument out side of testing)#2016-09-1620:02hiredmananother way is using valid? like in that person-name function#2016-09-1620:07hiredmaninstrument is very similar to sort of the classic idea of what you can do with assertions, assert all these expensive pre and post condition checks in development, then compile with assertions turned off for production#2016-09-1620:07hiredman(not that I have ever seen a clojure build that turns off assertions, but I am sure they exist)#2016-09-1620:08seancorfield@liamd I think itā€™s mostly too early for standard patterns of usage to have settled down (and I donā€™t think thereā€™s One True Way to use specs anyway). Have you read the http://clojure.org Guide? And watched Stu Hallowayā€™s webcast demos of it? Also the Cognicast with Rich talking about the motivations for it is good listening.#2016-09-1620:08liamdi've listened to the cognicast and the generative testing was what sounded most useful to me#2016-09-1620:08liamdand yeah i've read the guide#2016-09-1620:08liamdi'll check out the webcast#2016-09-1620:08bfabryas an aside, instrument doesn't do what would normally be considered :post checks. It only checks the :args section of an fdef#2016-09-1620:08liamdmaybe it's because i'm thinking of spec as an answer to the lack of typing? are those related concerns?#2016-09-1620:09seancorfieldAt World Singles, weā€™re specā€™ing data primarily and using conform as part of our validation and transformation across application layer boundaries. Weā€™re using instrument to augment our testing. Weā€™re starting to work with the generative side more at this point but we havenā€™t settled into a good flow for it yet.#2016-09-1620:10hiredmanmaybe more like a database schema than what you would typically think of as a type#2016-09-1620:10seancorfield@liamd Rich is pretty clear that spec != type system and should not be viewed the same way#2016-09-1620:12liamdso basically my thinking is: - in a statically typed system you can make some assumptions about your passed in args since they wouldn't compile otherwise - without static typing we can just make those assumptions anyway but we might have to debug and get ugly errors down the line - or we could do some defensive programming and handle incorrect args gracefully (by some standard, a nice error or whatever)#2016-09-1620:12richiardiandreaA side question: how folks use exercise?#2016-09-1620:12bfabryit's not a type system, but it's clearly geared at solving some of the same problems in a different way, as well as solving some problems that type systems don't. documentation, correctness, better errors#2016-09-1620:12liamdso does spec just make that third point easier? do a conform and then handle malformed inputs in every function without have to rewrite ugly validation logic everywhere#2016-09-1620:13shaun-mahood@liamd: 2 interesting resources for the intersection of spec and typing https://www.indiegogo.com/projects/typed-clojure-clojure-spec-auto-annotations#/ https://github.com/arohner/spectrum#2016-09-1620:13bfabry@liamd instrument and clojure.spec.test are attempts to make us more confident about point 2#2016-09-1620:13hiredmanI would say no#2016-09-1620:14hiredmanif you have a system where you are calling conform in every function on all your arguments, something has gone way wrong#2016-09-1620:14seancorfield@liamd Weā€™re finding spec to be a better way to write our existing validation and transformation code since it abstracts out the specification part "as data". We donā€™t view it as "defensive programming".#2016-09-1620:14liamdsomething like:
(let [my-x (s/conform my-x-spec x)]
  (if (= my-x :clojure.spec/invalid)
        (do-something)
        (proceed)))
#2016-09-1620:15liamd@shaun-mahood i'll take a look#2016-09-1620:15liamdthanks!#2016-09-1620:15hiredmanlike, where are your functions getting passed all these malformed inputs from?#2016-09-1620:15liamdit's a wild world out there#2016-09-1620:15hiredmanat an edge, like a rest api, sure#2016-09-1620:16hiredmanif all your functions are exposed an all passed arbitrary input from the outside world, sure#2016-09-1620:16liamdi was just thinking "couldn't we try to analyze the specs at compile time" and then i clicked spectrum#2016-09-1620:17liamdlooks neat#2016-09-1620:17liamdbut i mean isn't that the strength of type systems#2016-09-1620:17hiredmantype systems are going to need help at the edge too#2016-09-1620:17liamdwe can make assumptions about the data we're handling#2016-09-1620:18liamdright, parsing json in any type system is sticky#2016-09-1620:18hiredmantype systems can only analyze data they see, random data coming from the edge isn't going to be seen at compile time#2016-09-1620:19arohner@liamd no, spectrum canā€™t handle that. It can prove that you havenā€™t validated the data though#2016-09-1620:19seancorfieldSpec can also enforce data "structure" in a way that type systems cannot. It can specify valid ranges of numbers, relationships between parts of a data structure, patterns in strings and so on.#2016-09-1620:20arohner@seancorfield there are workarounds for that in spectrum, in some cases#2016-09-1620:20hiredman@seancorfield: that is just opening the door and waiting for someone to step in and start talking about dependent types#2016-09-1620:20arohnerbecause spectrum can assert that the spec gets validated at runtime#2016-09-1620:20bfabry@liamd spec isn't going to start down the path of compile time checking other than for macros. at the end of the day clojure is a dynamic language. spec is an attempt to give us stronger confidence about what we're passing around at test and dev time using instrument and check. It also includes tools to validating the shape of things at runtime in production but imo I think that's more meant for things that are at high risk of being invalid (boundaries) rather than things that are likely fine#2016-09-1620:20seancorfield@hiredman Hahahaā€¦ trueā€¦ but we donā€™t have those in very many languages and almost none that are in common production usage!#2016-09-1620:21seancorfieldBesides, if folks want a type system, then go use a statically typed language. Clojure is not that language.#2016-09-1620:21hiredmanI think the big thing about spec is the language of terms (regular clojure expressions) and the language of descriptions (specs) are the same, so you can easily get access to specs at runtime, and write some interpreter for them to do whatever you want#2016-09-1620:22liamdi see. so it should be though about as "a set of tools that helps mitigate the downsides of a dynamic language during your dev work flow"#2016-09-1620:22seancorfieldI donā€™t see those as "downsides" šŸ™‚#2016-09-1620:22hiredmancalling them downsides reflects a particular stance that is not universal#2016-09-1620:23liamdwell surely spec was developed as a solution to some perceived problem?#2016-09-1620:23liamdmaybe i'm wrong in thinking that that problem is rooted in lack of typing#2016-09-1620:23bfabryit also gives you more confidence than any static type systems I'm aware of because predicates can be arbitrary#2016-09-1620:23richiardiandreaA very nice talk about this was the last one by Felleisen at Clojure/West#2016-09-1620:24hiredman@liamd: static typing is a solution to a similar set of problems#2016-09-1620:25hiredmanbut it is sort of like saying you have a problem P, and solutions A and B, and saying solution B is there to fix a lack of solution A#2016-09-1620:25hiredmanno, B is there to fix P#2016-09-1620:25arohnerthereā€™s significant overlap between spec and static types, but neither completely subsumes the other#2016-09-1620:26arohnerthere are bugs that can be caught by static types that are hard to catch w/ spec, and vice versa#2016-09-1620:26bfabrystatic type systems give a lot of confidence about correct calls (probably more than spec) but provide no tools for regular data validation. they're also limited in what they can describe regarding correctness#2016-09-1620:28hiredmanin most typed languages, the language of terms and the language of types are distinct#2016-09-1620:29liamdi see#2016-09-1620:29hiredmantypes end up being kind of their own langauge that exists largely at compile time#2016-09-1620:29hiredmanspecs are more like a database schema, they exist in the system and you can poke and fiddle with them live#2016-09-1620:29liamdso due to them being just plain old clojure we can ingest the specs and do something with them#2016-09-1620:30liamdis there a way to get the spec of something at run time?#2016-09-1620:30hiredmanthings don't inherently have a spec#2016-09-1620:30liamdright#2016-09-1620:30hiredman(another different from types)#2016-09-1620:30arohneryes, you can get spec definitions at runtime#2016-09-1620:30liamdwe just have things and specs and we can smash them into each otehr if we want#2016-09-1620:31arohner(s/form (s/spec ::foo))#2016-09-1620:31liamdwould it make sense to use specs as part of your logic? like if this conforms to my Person spec do one thing else do some other thing#2016-09-1620:31bfabrytotally#2016-09-1620:32liamdor i could see in web dev using them to validate incoming json#2016-09-1620:32bfabryconvenient dsl for expressing that kind of logic#2016-09-1620:32liamdit'd be nice if explain could map back to a json error#2016-09-1620:33liamdi don't know if api consumers want to deal with ;; val: 42 fails spec: ::suit predicate: #{:spade :heart :diamond :club}#2016-09-1620:34arohners/explain-data#2016-09-1620:34liamdah, there it is#2016-09-1620:34liamdvery nice#2016-09-1620:50seancorfield@liamd As I noted above, at World Singles, we are using spec to replace our custom validation and transformation logic so we can have separate, comprehensive specifications we can show to and discuss with product owners / stakeholders, and it removes a lot of bespoke code.#2016-09-1620:51seancorfieldWe are actually using spec for three layers: input, domain, persistence and conforming between them. The validate-and-conform aspect of spec is very nice.#2016-09-1620:53liamdis that running only during dev or when you deploy?#2016-09-1621:06seancorfieldItā€™s a core part of our production code, calling conform.#2016-09-1621:08seancorfieldWe also use instrument in testing, and weā€™re using some generative testing too (`check`). As I said above, we havenā€™t yet settled onto a final workflow for the latter since generative testing can be slow and we want "unit tests" to be fast (so we can run them easily while writing code).#2016-09-1714:26petterikTwo talks on clojure.spec have just been uploaded to youtube: Strange Loop 2016: Stuart Halloway - "Agility & Robustness: Clojure spec" https://www.youtube.com/watch?v=VNTQ-M_uSo8 ClojuTRE 2016: Arne Brasseur - "Introduction to clojure.spec" https://www.youtube.com/watch?v=-MeOPF94LhI#2016-09-1720:04surreal.analysisChicago Clojure also recently had a meeting on Spec, which was posted to Youtube a couple weeks ago - https://vimeo.com/181231654#2016-09-1814:01wilkesCan someone post the slides for Chelimskyā€™s intro?#2016-09-1819:11lvhHm; are there any specs somewhere that someone is collecting for datomic/datascript datoms?#2016-09-1822:47Oliver GeorgeHere's a quick script which generates simple clojure.spec definitions for a database. https://gist.github.com/olivergeorge/468464ce82b8da486736fe725a4b6ff8#2016-09-1822:48Oliver GeorgeNothing particularly magic in there. JDBC makes it easy to inspect types for table columns.#2016-09-1822:49Oliver GeorgeAn interesting extension would be to interpret relations and generate optional keys#2016-09-1822:50Oliver George(My use case is generating dummy data for a re-frame UI which is fed from a pull api (which in turn mirrors a db schema)#2016-09-1906:08jasonjcknis there an inverse of s/conform ?#2016-09-1906:10jasonjcknfound it#2016-09-1908:21jeroenvandijk@jasonjckn yes, itā€™s s/unform#2016-09-1908:22jeroenvandijk(itā€™s not perfect yet, see http://dev.clojure.org/jira/browse/CLJ-2021?focusedCommentId=43842#comment-43842)#2016-09-1917:28jasonjckn@jeroenvandijk thanks! I ran head smack right into that issue yesterday šŸ™‚#2016-09-1917:28jasonjckn@jeroenvandijk but used s/conformer to get around it#2016-09-1917:28Alex Miller (Clojure team)btw, that issue is imo a problem with conform, not unform :)#2016-09-1917:29jasonjcknaside from that issue is making macro writing a joy#2016-09-1917:29Alex Miller (Clojure team)are you using conform to destructure input?#2016-09-1917:29jasonjcknyes#2016-09-1917:29Alex Miller (Clojure team)I havenā€™t seen too many people doing that yet but our presumption has been that it would remove a lot of fragile gross code#2016-09-1917:29jasonjckn
(let [ast (s/conform ::defui forms)
        ast2 (ast-transform ast)
        out (s/unform ::defui ast2)
#2016-09-1917:29jasonjckntotal joy šŸ™‚#2016-09-1917:30Alex Miller (Clojure team)cool, thatā€™s great to hear about#2016-09-1917:30jasonjcknhere's my ast transformer still toying with this code
(defn ast-transform [ast]
  (letfn [(walk-fn [xf]
            (-> xf
                (match->
                 {:protocol (_ :guard #(static-symbs (name %)))}
                 (assoc :static? 'static)

                 {:method-name 'render :args [this om-props] :body body}
                 (assoc :args [this]
                        :body `[(let [~(s/unform ::cs/binding-form om-props)
                                      (om.next/props ~(s/unform ::cs/binding-form this))]
                                  
#2016-09-1917:30jasonjcknthe pattern is walking + match->#2016-09-1917:30jasonjcknI implemented match->#2016-09-1917:32Alex Miller (Clojure team)cool#2016-09-1917:38patrkrisIs clojure.spec a suitable tool for validating JSON formatted HTTP request bodies? I am asking because the pervasive use of namespaced keywords specs seems to conflict with that#2016-09-1917:39patrkrisI mean using namespaced keywords as object keys in JSON seems unnatural#2016-09-1917:50Alex Miller (Clojure team)you can use s/keys with :req-un and :opt-un for that#2016-09-1917:51Alex Miller (Clojure team)many people are and itā€™s been discussed a number of times here - you might find it in the back chat#2016-09-1917:52patrkrisThat was what I thought about doing myself. But then I started thinking about nested maps and whether that would end up posing a problem. Probably not šŸ™‚ You mentioned some time ago that you were thinking about adding a way to "rename" keys in s/keys. Any development on that?#2016-09-1917:54Alex Miller (Clojure team)nope. up to Rich whether that idea comes back, so Iā€™m not sure if it will.#2016-09-1921:28liamdiā€™m not able to import clojure.spec.gen into my clojurescript project? any idea why?#2016-09-1921:29liamd
No such namespace: clojure.spec.gen, could not locate clojure/spec/gen.cljs, clojure/spec/gen.cljc, or Closure namespace ā€œclojure.spec.genā€
#2016-09-1921:32ggaillardHi @liamd, according to the current clojure.spec guide, it seems you need to add [org.clojure/test.check "0.9.0"] to your :dev profile in your project.clj
:profiles {:dev {:dependencies [[org.clojure/test.check "0.9.0"]]}}
Or with boot
(set-env!
 :dependencies '[[org.clojure/test.check "0.9.0" :scope "test"]])
Did it solve your problem ?
#2016-09-1921:32hiredmanlooking at the clojurescript repo, that namespace doesn't exist#2016-09-1921:33hiredmanit doesn't seem to have been ported#2016-09-1921:33liamdi did add test.check to my dependencies#2016-09-1921:34liamdnot under dev for now, but that shouldnā€™t matter right?#2016-09-1921:34ggaillardHo my bad, sorry. Try with [cljs.spec.impl.gen :as gen]#2016-09-1921:37ggaillardMaybe more information here : http://dev.clojure.org/jira/browse/CLJS-1696#2016-09-1921:38liamdthat seems to have gotten rid of the import error, but iā€™m still not sure which things to use to get spec generation working#2016-09-1921:39liamdnow itā€™s
clojure.test.check.generators/simple-type-printable does not exist, clojure.test.check.generators never required
#2016-09-1921:39liamdso looks like i need to require clojure.test.check.generators#2016-09-1921:39liamdah got it!#2016-09-1921:43ggaillardYes same for me, sometime I need cljs.spec.impl.gen, sometime clojure.test.check.generators. But this code
(gen/sample (s/gen int?))
effectively require clojure.test.check.generators :as gen and cljs.spec :as s
#2016-09-2000:27Oliver GeorgeThis is fun. Using a pull-query to generate specs means I can generate test data for UI views based on the pull query they use. https://gist.github.com/olivergeorge/da6487fc60c67b79ba082bc807072e43 This still requires table and field specs based on something like this: https://gist.github.com/olivergeorge/468464ce82b8da486736fe725a4b6ff8#2016-09-2000:29danielcomptonIs there a JIRA issue open for spec coercions (in the same way that schema does them)? Iā€™m aware of conform, but that isnā€™t quite the same#2016-09-2001:20Alex Miller (Clojure team)there is no plan to add that#2016-09-2001:20Alex Miller (Clojure team)Rich sees them as separate concerns#2016-09-2008:00otfromI'm presuming I'm doing something wrong and that there is a function in core that will do this, but this bit of code is working for me to integrate spec tests into my normal clojure.test flow#2016-09-2008:00otfrom
(defn passes? [sym]
  (-> sym
      stest/check
      first
      :clojure.spec.test.check/ret
      :result))

(t/deftest conformer-testing
  (t/testing "Checking the conformer"
    (t/is (true? (passes? `sut/my-specced-func)))))
#2016-09-2008:01otfromI've elided the bit where I define the :args :ret and :fn for my-specced-func#2016-09-2008:02otfromis there something that does the same as passes? and is true? really the right way of putting this into t/is ? Success and failure both return truthy values which is why I'm using it atm#2016-09-2008:11jmglov@otfrom Here's what @bahulneel and I came up with: https://gist.github.com/jmglov/30571ede32d34208d77bebe51bb64f29#2016-09-2008:22jmglovI have a record that implements a protocol. How do I spec this?#2016-09-2008:53jmglovWhen I try to gen/sample some deeply nested spec, I get a "Couldn't satisfy such-that predicate after 100 tries." exception with no mention of the spec or predicate in question. Is there a good way to figure out which spec caused this?#2016-09-2008:56jmglovSo far, my only hope is a binary search by running gen/sample on the sub-specs until I find the offender. šŸ˜ž#2016-09-2009:04bahulneel@jmglov I had this problem also. I had to eyeball my specs and try exercising individual ones. In the end, it's usually ones that use and and limit the valid input to some "shape". To fix these I use with gen, an example I stole is here: https://github.com/nwjsmith/datomic-spec/blob/master/src/datomic_spec.clj#L6#2016-09-2009:05jmglov@bahulneel Yes, that was exactly it. My offending one turned out to be (s/and map? empty?). I changed it to #{{}}. šŸ™‚#2016-09-2009:06bahulneelnice#2016-09-2009:09bahulneel@jmglov it's worth noting that, in that case (s/and empty? map?) will also work fine#2016-09-2009:09jmglovOf course! That's actually nicer to read, I think.#2016-09-2009:09jmglovThanks!#2016-09-2009:10bahulneelthe only issue is that the explanation would fail empty? before map? so the message may not be as useful#2016-09-2009:11bahulneel
user> (clojure.spec/explain (clojure.spec/and empty? map?) [1])
val: [1] fails predicate: empty?
nil
user> (clojure.spec/explain (clojure.spec/and empty? map?) [])
val: [] fails predicate: map?
nil
user> (clojure.spec/explain (clojure.spec/and empty? map?) {1 2})
val: {1 2} fails predicate: empty?
nil
user> 
#2016-09-2009:12bahulneelBut it's better than the alternative:
user> (clojure.spec/explain #{{}} {1 2})
val: {1 2} fails predicate: :clojure.spec/unknown
nil
user> (clojure.spec/explain #{{}} [])
val: [] fails predicate: :clojure.spec/unknown
nil
user> 
#2016-09-2009:14bahulneelThis works better:
user> (clojure.spec/def ::empty-map #{{}})
:user/empty-map
user> (clojure.spec/explain ::empty-map {1 2})
val: {1 2} fails spec: :user/empty-map predicate: #{{}}
nil
user> 
#2016-09-2009:15bahulneel(also naming (s/and empty? map?) gives a similar level of clarity)#2016-09-2009:23mpenetany known attempt to generate json-schema from specs?#2016-09-2009:24mpenetthere's a ring-swagger fork that tried, but it's not there yet#2016-09-2009:29ikitommi@mpenet would we easy with records, https://groups.google.com/forum/m/#!topic/clojure-dev/9EjTvxPPaGQ#2016-09-2009:39mpenetI am not really interested in the conform part right now#2016-09-2009:39mpenetI just need generating decent json-schemas from specs#2016-09-2009:42jmglov@bahulneel Nice!#2016-09-2010:03ikitommiok, not there yet. It would be easy to cook up a working prototype of the conversion from existing spikes, but still gaps to make it robust and extendable. Good thing that, there are smarter people here :)#2016-09-2010:05ikitommiSpectrums parse-spec might be a good way for it.#2016-09-2010:06mpenetIsnt it possible just to gen from s/form?#2016-09-2010:06mpenetIt s a one time thing so perf doesnt matter#2016-09-2010:11ikitommiHmm... there was one version with s/form & core.match, can't recall what was the problem with it. Maybe just not finished.#2016-09-2010:14ikitommiactually, it could be extended too, with multimethod on the spec symbol. Ping @miikka. #2016-09-2011:40mpenetI have something along these lines already, even tho the multimethod is kinda useless ultimately, it can be closed#2016-09-2012:09bhagany@jmglov: regarding protocol implementations - I read (on the mailing list, I think), that directly specā€™ing protocol fns wasnā€™t supported, and the recommendation was to make the protocol fns ā€œprivateā€, provide wrapper fns, and spec those. This worked fine for me, especially because I had some common logic that fit nicely in the wrapper fns.#2016-09-2012:19jmglov@bhagany I'm not trying the spec the protocol functions themselves, but rather an argument to a function that should implement a certain protocol. i.e.
(defprotocol Database
  (read-thing [this id])
  (store-thing [this thing]))

...

(defn do-stuff [db updates]
  (let [thing (database/read-thing (:id updates))]
    (-> thing
        (merge updates)
        database/store-thing)))
I want to spec do-stuff something like this:
(s/fdef do-stuff
        :args (s/cat :db #(implements? Database %)
                     :updates (s/keys :req [::thing/id]
                                      :opt [::thing/name]))
        :ret any?)
Of course implements? is not a real function, but this illustrates what I'm trying to do, I hope.
#2016-09-2012:26mpenetyou can use satisfies?#2016-09-2012:32jmglov@mpenet Exactly what I needed! Thanks!#2016-09-2012:37mpenetit gets more tricky if you want to add gen support#2016-09-2012:38jmglovNo need for now, thank goodness.#2016-09-2015:29bhaganyIā€™m trying to run ā€œfreeā€ generative tests in cljs, using doo, but it seems as though the registry is not being populated in my test build. Iā€™m importing a namespace that defines some named specs, and I can run the functions in that namespace, but clojure.spec.test/check and checkable-syms are both empty. However, in my figwheel repl, the registry is populated as I would expect. Any ideas what I might be missing?#2016-09-2015:33bhaganyI should also add that clojure.spec.test/enumerate-namespace does see the fns that are specā€™d in my test build#2016-09-2015:34bhaganyand also that I can (s/valid? ::some/data-spec data) in my test build. It seems like only fn specs are missing.#2016-09-2015:43husain.mohssenPossibly stupid question: why is s/fdef not called s/defn ?#2016-09-2016:05Alex Miller (Clojure team)because thatā€™s not what Rich named it#2016-09-2016:17husain.mohssengood enough for me šŸ™‚ all hail Master Rich šŸ™‚ (I was asking in case there was some subtlety Iā€™m missing)#2016-09-2019:40liamdhaving a hard time wrapping my head around writing generators, how would i write a generator for a set of distinct strings?#2016-09-2019:40hiredmanthere is a function, it might be called one-of#2016-09-2019:41hiredmanhttps://github.com/clojure/test.check/blob/master/src/main/clojure/clojure/test/check/generators.cljc#L272-L289#2016-09-2019:41hiredmanoh, sorry, no wrong one#2016-09-2019:42hiredmanhttps://github.com/clojure/test.check/blob/master/src/main/clojure/clojure/test/check/generators.cljc#L337-L348#2016-09-2019:42hiredmanelements#2016-09-2019:42liamdi tried (gen/fmap set (gen/vector gen/string))#2016-09-2019:43hiredmanoh, I misunderstood#2016-09-2019:43hiredmanthere is a generator for sets#2016-09-2019:43hiredmanso (gen/set gen/string)#2016-09-2019:45liamdexcellent that did it#2016-09-2019:45hiredmanyou may need to do some voodoo connect that to spec, I haven't looked too much at how test.check and spec are tied together#2016-09-2019:45liamdi just had to do with-gen#2016-09-2019:45liamdreally easy#2016-09-2019:48liamdhereā€™s another one: any idea how to generate long strings?#2016-09-2019:48liamdsay > 200 chars?#2016-09-2019:53hiredmanif you run more generative test iterations, the size of the strings will get larger, but it would likely be easier to create a special purpose larger string generator#2016-09-2019:54hiredmanwhich maybe some combination of resized and the string generator would help you do#2016-09-2019:54hiredmanhttps://github.com/clojure/test.check/blob/master/src/main/clojure/clojure/test/check/generators.cljc#L233-L240#2016-09-2019:55hiredman(gen/set (gen/resized 200 gen/string))#2016-09-2020:03liamdresized looks good#2016-09-2020:05bfabrythere's sooooo much stuff in the clojure.test.check.generators namespace, trying to internalise all that is the one thing I'm not looking forward to when we start using spec#2016-09-2020:08liamdthe docs donā€™t mention everything so you gotta dig into the sources#2016-09-2020:08liamdthat said, using spec with devcards is kicking ass right now for building om ui's#2016-09-2020:22Alex Miller (Clojure team)in many cases itā€™s easier to just gen from a spec#2016-09-2020:22Alex Miller (Clojure team)(s/gen (s/coll-of string? :kind set?))#2016-09-2020:24Alex Miller (Clojure team)@liamd ^^#2016-09-2020:24liamdfor some specs i get {:message "Couldn't satisfy such-that predicate after 100 tries.", :data {}}#2016-09-2020:24Alex Miller (Clojure team)if you are andā€™ing specs with fairly restrictive filters, youā€™re likely to run into this#2016-09-2020:25Alex Miller (Clojure team)this is discussed (plus making custom generators) in http://clojure.org/guides/spec if you havenā€™t seen that yet#2016-09-2020:25liamdyeah iā€™ve read that over, seems the solution is to write your own generators and then use with-gen which works great
#2016-09-2020:26liamdiā€™m already re-using a good amount of generators for things like certain numbers and strings with various properties#2016-09-2020:26liamdone thing is that error wonā€™t tell me which generator itā€™s talking about#2016-09-2020:26liamdwhich isā€¦ confusing#2016-09-2020:26Alex Miller (Clojure team)yes, I think thatā€™s worth improving#2016-09-2020:27Alex Miller (Clojure team)I donā€™t think anyone has actually filed a ticket about it, but I would be in favor of improving that#2016-09-2020:28liamdis that specific to the cljs implentation or spec in general?#2016-09-2020:29liamdor i guess thatā€™s test.check#2016-09-2020:29Alex Miller (Clojure team)it is, but I suspect it could be wrapped somehow in spec#2016-09-2020:30Alex Miller (Clojure team)havenā€™t looked at it#2016-09-2020:31liamdyeah i suppose youā€™d want to tie it back to your namespaced key#2016-09-2020:51madstapI'm having a problem with generators.
(def col-gen
  (gen/fmap #(->> % (apply str) str/upper-case keyword)
    (gen/vector gen/char-alpha)))


(s/def :cell/col (s/and simple-keyword? (comp (set util/alphabet) name)))
This throws an error `java.lang.AssertionError Assert failed: Arg to vector must be a generator (generator? generator)` How can I generate a vector of characters from the alphabet?
#2016-09-2020:54madstapOther question, how can I make a generator from a set?#2016-09-2020:56hiredmanwhat namespace is gen?#2016-09-2020:56madstapclojure.spec.gen#2016-09-2020:56hiredmanI bet you need to invoke gen/char-alpha#2016-09-2020:57hiredmanthe things in clojure.spec.gen aren't actually generators, they are 0 argument functions that you invoke to get a generator#2016-09-2020:57madstapThanks, that worked#2016-09-2021:01madstapAnd how can I make a generator from a set?#2016-09-2021:02hiredmanyou have a set of elements and you want a generator to pick elements from it?#2016-09-2021:03madstapYeah#2016-09-2021:03hiredmangen/elements#2016-09-2021:03Alex Miller (Clojure team)or of course just (s/gen #{:a :b :c})#2016-09-2021:04Alex Miller (Clojure team)people do not leverage gens from specs enough imo#2016-09-2021:04Alex Miller (Clojure team)that should be your first choice#2016-09-2021:04Alex Miller (Clojure team)then stuff in spec.gen, then stuff in test.check#2016-09-2021:05Alex Miller (Clojure team)or test.chuck etc#2016-09-2021:06hiredmanso (s/gen (s/spec number?)) instead of gen/number or whatever?#2016-09-2021:06Alex Miller (Clojure team)just (s/gen number?)#2016-09-2021:07Alex Miller (Clojure team)but yes#2016-09-2021:07ggaillard@alexmiller about that case, is it normal for
(clojure.test.check.generators/sample (cljs.spec/gen #{true false}))
to always be
(true true true true true true true true true true)
?
#2016-09-2021:07Alex Miller (Clojure team)well any set with falsey elements is going to be a bad time#2016-09-2021:08Alex Miller (Clojure team)thatā€™s kind of a special case - it will return falsey elements which is the signal that the value doesnā€™t match the spec#2016-09-2021:08Alex Miller (Clojure team)in this case (s/gen boolean?) will be better#2016-09-2021:08ggaillardOh, I get it !#2016-09-2021:09Alex Miller (Clojure team)and if you want to support nil,true,false then (s/gen (s/nilable boolean?))#2016-09-2021:12ggaillardThank you ! I had some bad time with (s/gen boolean?) in ClojureScript, throwing an error. I figured it out with with-gen and (gen/elements #{true false}). I don't know if it's a known bug or not btwā€¦#2016-09-2021:13Alex Miller (Clojure team)sounds like a bug#2016-09-2021:14Alex Miller (Clojure team)you might ask in #clojurescript to see what @dnolen thinks#2016-09-2021:17dnolen@ggaillard it should work, make sure you are on the latest 1.9.229, if it still doesnā€™t work, file an issue in JIRA#2016-09-2021:19ggaillardYou are quick! I didn't had time to post in #clojurescript šŸ™‚ Thank you, I'll check that.#2016-09-2021:34ggaillardI was using 1.9.216, 1.9.229 fixed it. Sorry for that šŸ˜•#2016-09-2022:56devnThe clojure.spec/nilable docs say it takes a pred, but it also seems to work if it's passed a spec. Is that behavior I can rely on?#2016-09-2102:22gardnervickersHey folks, if I had a key in a map that I want to constrain to be a function that conforms to a spec, how would I go about that? I thought I would be able to pass the results of s/fdef to (s/def ::my-key ā€¦) but that does not seem to be the case.#2016-09-2102:25hiredmanfspec#2016-09-2102:27gardnervickersOh jeeze, missed that one thanks#2016-09-2102:27hiredmanI am sort of surprised that the result of fdef didn't work though#2016-09-2102:49Alex Miller (Clojure team)It returns the result of def not fspec#2016-09-2102:49Alex Miller (Clojure team)Fdef is def + fspec#2016-09-2102:49bfabryah, so you could use (s/spec ::my-key) @alexmiller ?#2016-09-2102:50bfabrywait#2016-09-2102:50bfabrythat makes no sense#2016-09-2102:50bfabryso what does fdef define?#2016-09-2102:50Alex Miller (Clojure team)Same as def but the key is a symbol not a keyword#2016-09-2102:51Alex Miller (Clojure team)It registers, not defines#2016-09-2102:51Alex Miller (Clojure team)Or at least that's how I think about it#2016-09-2102:51bfabryoh, so you could use the symbol 'my-function in place of the spec? or at least (s/spec 'my-function)#2016-09-2102:51Alex Miller (Clojure team)Yes - fully qualified though#2016-09-2102:52bfabrymakes sense#2016-09-2102:52Alex Miller (Clojure team)So back tick #2016-09-2102:52hiredman
user=> (defn foo [x] x)
#'user/foo
user=> (s/fdef foo :args (s/cat :arg any?) :ret any?)
user/foo
user=> (s/def :foo/bar user/foo)
:foo/bar
user=> (s/valid? :foo/bar foo)
true
user=> 
#2016-09-2102:52bfabry@hiredman I think that will just use foo as a predicate#2016-09-2102:53hiredmanoh, did I flip that around?#2016-09-2102:53hiredmanno, I think that is right#2016-09-2102:53bfabrylike, (s/def :foo/bar user/foo) == (s/def :foo/bar clojure.core/identity)#2016-09-2102:54hiredmanOh#2016-09-2102:55bfabrybuuut I think (s/def :foo/bar 'user/foo) would use the spec created by fdef#2016-09-2102:56hiredmanright#2016-09-2102:56hiredmanlike regular def the thing being def'ed(registered) is evaluated#2016-09-2102:57hiredmanhah, I can tell that changed it because valid? takes longer to return because of the generative checking of functions#2016-09-2102:59bfabrydamn, so a lot of the core functions take a single arity it turns out, but anyway yes
oot.user=> (s/def :foo/bar 'boot.user/foo)
:foo/bar
boot.user=> (s/valid? :foo/bar foo)
true
boot.user=> (s/valid? :foo/bar clojure.string/replace-first)
false
#2016-09-2113:31kestrel7@alexmiller some feedback. I created a generator that did not conform to the spec (doh!). The generator contained the such-that predicate. When I tried creating a sample from the generator I got this error:
ExceptionInfo Couldn't satisfy such-that predicate after 100 tries.  clojure.core/ex-info (core.clj:4725)
I assumed that it referred to my custom generator but that was a red herring because in fact spec must be using such-that to ensure that the generated value conforms to the spec, and it was this such-that that generated the failure, not the one in my custom generator. Code (with the problem corrected but showing the such-that in my generator:
(defn mod11-checkdigit
  "Calculate the checkdigit see "
  [n]
  (let [x (->> (map #(Integer/parseInt (str %)) (take 9 n))
               (map * (range 10 1 -1))
               (reduce +))
        y (mod x 11)
        c (- 11 y)]
    (cond (== 10 c) nil
          (== 11 c) 0
          :else c)))

(def nhs-number-gen
  "Generates a valid NHS number"
  (gen/fmap #(str (+ (* 10 %) (mod11-checkdigit (str %))))
            (gen/such-that #(mod11-checkdigit (str %))
                           (gen/choose 100000000 999999999))))

(defn nhs-number?
  "Returns true if passed a valid nhs number else returns false"
  [n]
  (and (string? n) (= 10 (count n)) (= (str (mod11-checkdigit n)) (str (last n)))))

(s/def ::nhs-number (s/with-gen nhs-number?
                                (fn [] nhs-number-gen)))

#2016-09-2113:33kestrel7It would be nicer if the error thrown due to the generated value being non-conformant with the spec stated this.#2016-09-2113:46patrkrisSay I have a namespace with two functions create-component-a and create-component-b. Each function takes a map with options. How would I go about spec'ing those option maps in the same namespace as the functions? Let's say that each map takes an :id (or something else) attribute, but they're semantically different. If I want to use the ::keyword shortcut to create a namespaced keyword, I can't call them both ::id. I could create a namespace for each type of option map, alias them and use something like ::a-options/id and ::b-options/id, but that seems wrong.#2016-09-2114:10Alex Miller (Clojure team)@kestrel7 you are correct in your assessment - spec doesnā€™t trust custom generators and re-confirms they match the spec. Agreed that it would be nice to know the cause here more explicitly (kind of a special case of the general desire to know more about which part of a generator failed to generate). Happy to see a jira for enhancement.#2016-09-2114:11Alex Miller (Clojure team)@patrkris your last suggestion is what I would do there (you donā€™t have to alias of course - you can just use fully-qualified too)#2016-09-2114:11Alex Miller (Clojure team)why does that seem wrong?#2016-09-2114:13patrkris@alexmiller: I don't know. Maybe it shouldn't šŸ™‚. It's probably just from being used to define a schema (with herbert) in the same namespace as the function relying on it. But using the fully qualified version also feels a little wrong, because then you have hardcoded the namespace. It possibly just something I need to get used to#2016-09-2114:15patrkrisI thought about whether allowing the use of "subnamespaces" in aliases would be useful. For instance ::a-options/id without having an ::a-options alias. But I can imagine that it would turn out to be problematic.#2016-09-2114:16patrkrisEhr... by "subnamespaces" I meant being able to say ::order.delivery/type without having a order.delivery alias created#2016-09-2114:19Alex Miller (Clojure team)well, I donā€™t think weā€™ll do that :)#2016-09-2114:21patrkrisBut it's not definite no? šŸ˜‰ I'm kiddin'#2016-09-2114:21kestrel7@alexmiller as requested http://dev.clojure.org/jira/browse/CLJ-2025 - thanks for the great work.#2016-09-2114:22Alex Miller (Clojure team)thanks!#2016-09-2115:21kkruitIs there a way to require that in {:a "a" šŸ˜› "b" :c "c"} :a is required and šŸ˜› or :c is required and still have the other be optional? I've tried this using s/keys but cant figure out how i'd do it and I can't see how i could use s/map-of and still require :a...#2016-09-2115:24minimal(s/def ::mymap (s/keys :req-un [::a (or ::b ::c) ]))#2016-09-2115:27kkruitoh man! I tried s/or... Thanks!#2016-09-2115:28minimalthe or is syntax in the keys macro rather than clojure.core/or or spec/or#2016-09-2115:28kkruitoh, I was just thinking. how is that working as clojure.core/or ... guess it's not haha#2016-09-2115:29kkruitthanks again!#2016-09-2115:29minimalI didnā€™t notice it at first either#2016-09-2118:48sparkofreasonSome functions, like exercise, allow you to override generators by name. Is there any mechanism to do the same with conformers?#2016-09-2118:49leongrapenthin@alexmiller I think a good solution to aliasing keyword and symbol namespaces not existing in the codebase like :human.programmer/language would be (defalias hp human.programmer). So :hp would still be :hp, but ::hp/language would resolve to :human.programmer/language . This would overwrite aliases created via require. Should I write a tickent?#2016-09-2118:49Alex Miller (Clojure team)@dave.dixon no#2016-09-2118:50Alex Miller (Clojure team)@leongrapenthin alias already exists#2016-09-2118:50Alex Miller (Clojure team)the problem with it is that it requires the ns you are aliasing to exist#2016-09-2118:51leongrapenthinRight#2016-09-2118:51Alex Miller (Clojure team)we are already considering loosening that constraint or auto-creating the ns#2016-09-2118:51Alex Miller (Clojure team)I donā€™t need a ticket for it#2016-09-2118:51leongrapenthininteresting. auto-creating seems like a good strategy#2016-09-2118:52Alex Miller (Clojure team)that is by far easier given the existing implementation of Namespace etc#2016-09-2118:52leongrapenthinyeah#2016-09-2118:52leongrapenthinI just realized that I could use create-ns in combination with alias#2016-09-2118:53leongrapenthinSo there is a good workaround#2016-09-2118:53Alex Miller (Clojure team)yes that works fine now#2016-09-2118:54leongrapenthinDK about Cljs though#2016-09-2119:17Alex Miller (Clojure team)have you looked at multi-spec?#2016-09-2119:17Alex Miller (Clojure team)has that kind of a feel to me#2016-09-2119:58mandersonI keep running into the same problem where I have two maps that have the same named key, but with different specs. For example: two maps that both have a key called name, where in the first it is a string?, but the second is int? because it is more of an ID. How do I specify this so that the specs can be different, but the field name the same in both maps? Only solution I see is to change name to something else (like person-name) and just living with the name change, but that doesn't work especially if modeling a domain I don't control. Thoughts?#2016-09-2120:11bfabry@manderson :opt-un and :req-un in s/keys#2016-09-2120:13mandersonNot sure I'm following... I still have to define a spec called ::name in two different ways. Yes, I could use or, but that muddies the definition since they are actually separate specs.#2016-09-2120:17bfabry@manderson sorry I should've been clearer, define a spec called :my.app.person/name and a spec called :my.app.service/name or whatever makes semantic sense#2016-09-2120:20mandersonOk, so using the namespacing. That makes sense and should work for most of my cases (often translating from a format like JSON where there aren't namespaces). I suppose in the case where namespaces are expected, it would probably already be modeled correctly from the beginning (hopefully). Thanks @bfabry#2016-09-2120:20bfabryno worries#2016-09-2121:40seancorfield@manderson To (hopefully) clarify: if youā€™re specā€™ing with :req-un and :opt-un then you are using simple (non-namespaced) keys in your maps, but you can use a namespaced spec name / key name there to distinguish the two specs.#2016-09-2121:42seancorfield
(s/def ::person (s/keys :req-un [:person/name]))
(s/def ::service (s/keys :req-un [:service/name]))
Both person and service specs are maps with a required :name key, but the specs used to validate those keysā€™ values will be different for ::person and ::service ā€” does that help?
#2016-09-2121:42seancorfield(it was something that I did not get on the first two or three reads of the Guide and it was Alex that set me straight on that)#2016-09-2123:19stuartmitchellHi I am beginning a large multi-developer project that will use spec extensively I was wondering if there is any advice for how to place specs in namespaces. I was thinking of having a namespace structure like
project-name.spec
                 -common
                 -thing1
                 -thing2
...
and then requiring these namespaces in the actual code, probably when doing s/fdef below the defn this will require lots of map access to be ::thing1/id or ::thing1/date etc is this a good idea or should I be trying to keep all specs within the namespaces they deal with so ::id can be used when possible
#2016-09-2123:48hiredmantying your data model (names of attributes) to the way your code is currently organized seems like a bad idea#2016-09-2123:49hiredmanI would recommend picking good names for things in your maps, and sure namespace them, but don't think you have to use the same namespace names as your code#2016-09-2123:50hiredman:: is a real pain when moving code around#2016-09-2200:03stuartmitchellso you are saying :thing1/id (non aliased namespace) used everywhere we were thinking that, but it doesn't seem to look like the examples (which I assume are best practice)#2016-09-2200:14stuartmitchellI guess you loose the uniqueness of the keywords though.#2016-09-2200:14hiredmanwhat uniqueness?#2016-09-2200:16hiredmanin five years on a large clojure project we had several shakeups where namespaces were changed or renamed in mass, a long with regular day to day code churn#2016-09-2200:16stuartmitchellfor instance if i define :spec/thing1 in library code it could conflict with someone else defining :spec/thing1#2016-09-2200:17hiredmanuse a better namespace then#2016-09-2200:17hiredmanmy.orgs.spec/thing1#2016-09-2200:18stuartmitchellyeah good point, I was just pondering what the differences would be#2016-09-2200:27hiredmandata is something that can be exchanged with other systems, stored long term, etc, so using names that reflect the organization of code in one codebase isn't the best#2016-09-2212:50jmglov@skjutare Hi!#2016-09-2212:50jmglovJust noticed you here. šŸ˜‰#2016-09-2212:58jmglovI think I'm running into an infinite loop with a custom generator. I need an atom holding a map with specific keys, which seems pretty straightforward:
(s/def ::foo (s/map-of ::foo/id ::foo/event))
(s/def ::bar (s/map-of ::bar/country ::bar/event))
(s/def ::db (s/keys :req [::foo
                          ::bar]))

;; Give these predicates names so spec's error messages are clear
(defn- atom? [x] (instance? clojure.lang.Atom x))
(defn- deref-db? [x] (s/valid? ::db @x))

(s/def ::db-atom (s/and atom? deref-db?))
So far, so good:
user> (s/explain ::db-atom {::foo {}, ::bar {}})
val: #:user{:foo {}, :bar {}} fails spec: :user/db-atom predicate: atom?
nil
user> (s/explain ::db-atom (atom 42))
val: #atom[42 0x702dec31] fails spec: :user/db-atom predicate: deref-db?
nil
user> (s/explain ::db-atom (atom {::foo {}, ::bar {}}))
Success!
nil
#2016-09-2213:01jmglovHowever, using a custom generator is getting me into a world of trouble:
(s/def ::db-atom (s/with-gen
                   (s/and atom? deref-db?)
                   (fn [] (gen/fmap atom (s/gen ::db)))))
When I try to generate a value, my REPL becomes unresponsive for awhile and then eventually crashes with a really odd stacktrace:
user> (gen/generate (s/gen ::db-atom))
...
#2016-09-2213:03jmglovCan anyone spot my error?#2016-09-2213:36mandersonThanks @seancorfield, just saw your reply. That does make sense. I think I need to rethink the way I organize my specs. Right now, I have everything in a single namespace and now I'm wondering if I should break these into sub-namespaces. I know that I can still collect my specs in 1 namespace and just add different namespaces to the keys. Not sure which is the best approach. How do you group your specs? I'm still trying to get a feel for the proper balance of the right amount of spec without becoming cumbersome.#2016-09-2213:41mandersonHa, funny. I just actually read back through the rest of the messages and see @hiredman making some good suggestions to my very question... sounds like best practice is to de-couple top level namespace of a file from namespace of a keyword. So, all could be grouped in a file, but with explicit namespaces for keys instead of ::.#2016-09-2213:50patrkris@manderson: @hiredman talked about specs for data that might be exchanged between systems. If you want to spec data used only inside your system, creating namespaces for that may be feasible. For example, if you have a component in a com.example.payments-processor with a factory function that takes an options map, you could potentially spec it inside com.example.payments-processor.options, and alias that namespace wherever you need to create the options map. Hope I make sense.#2016-09-2213:52mandersonYep, that makes sense. I suppose there is no one right answer. Like I said, I'm still getting the feel for what works and what doesn't. We're building our first real app with this (as I imagine most are!) and running into pain points with name conflicts and a large spec file, so I've been thinking of how to best maintain it.#2016-09-2213:55patrkrisYes. I have been thinking a lot about this too. The problem is that you can't use aliased namespaces in specs if there isn't actually a file containing that namespace. In those cases you need to spell out the whole namespace of the spec, which can be verbose.#2016-09-2213:56mandersonHm. that's a good point.#2016-09-2213:59mandersonIt's kind of similar to the question of lots of fn's in a single namespace so only 1 to require, or fn's spread over ns's, which is more modular, but adds complexity. With spec, you would have to keep up with more ns's, but then you could alias them for simpler inline usage... It seems to encourage simplifying your model domain as much as possible in terms of defining it, which is probably a good thing.#2016-09-2216:12seancorfieldThere are workarounds for the alias issue that let you introduce aliases for non-existent namespaces. #2016-09-2216:15seancorfield@manderson: we group our data specs into files organized for what they spec, primarily, with almost no code in those files. Then we require them into namespaces that use those specs (i.e., where the code / functions are).#2016-09-2216:16seancorfieldThat's probably not a very helpful answer without you knowing a lot more about our codebase šŸ˜ø #2016-09-2217:03mandersonThanks @seancorfield, that's similar to what we're doing now, but it's a single spec file for a new app with a changing domain so it's been a bit of a pain. I think it's just going to take some trial and error to work out the kinks.#2016-09-2217:13seancorfieldYeah, weā€™re still evolving our approach. One of the issues for us, particularly, is that our domain model spec is the most extensive part and weā€™re working to automate the validation and conformance from our API spec to our domain model spec, so weā€™re moving toward dynamic derivation of API specs from domain model specs, which I hadnā€™t considered at firstā€¦ so thatā€™s changing the dependencies between our spec namespaces as well as how we use the specs in the first place. And we havenā€™t even gotten very far into specā€™ing functions at this point (specā€™ing the domain model and doing data validation and transformation has proved much more valuable for our initial focus).#2016-09-2217:21kkruitI've noticed that, for specs that are specific to your application, clojure src is using them in a sub-namespace e.g. all the specs for clojure.core are in clojure.core.specs. This has seemed to work okay for me.#2016-09-2217:25kkruit@patrkris I've been using create-ns to make namespaces for that issue.#2016-09-2217:28seancorfieldThatā€™s nicer than the workaround I was using, from Alex. I think Iā€™ll switch to that @kkruit ā€” thanks!#2016-09-2217:33kkruitno problem#2016-09-2219:00mpenetI have something somewhat similar used there: https://github.com/mpenet/alia/blob/master/modules/alia-spec/src/qbits/alia/spec.clj#2016-09-2219:01mpenetit's just an util fn, but maybe that's what you're refering to @seancorfield#2016-09-2219:02seancorfieldThatā€™s a big file @mpenet ā€” are you referring to a particular part of it?#2016-09-2219:03mpenetright: https://github.com/mpenet/alia/blob/master/modules/alia-spec/src/qbits/alia/spec.clj#L152-L153#2016-09-2219:03mpenetit's just a wrapper for create-ns+alias#2016-09-2219:12seancorfieldAh, from another project...#2016-09-2219:13mpenetit's a 3 line fn, as i said, create-ns + alias#2016-09-2219:13seancorfieldSince your alias is so long, it took me a while to realize it even was an aliasā€¦ and the args are the opposite way round to alias which made it look even stranger:
(x/ns-as 'qbits.alia.cluster-options.ssl-options
       'cluster-options.ssl-options)
as opposed to
(alias 'cluster-options.ssl-options (create-ns 'qbits.alia.cluster-options.ssl-options))
#2016-09-2219:14seancorfieldGiven that alias will probably be modified to auto-create the ns, Iā€™m preferring alias directly since I can probably just remove the create-ns call at some point.#2016-09-2219:14mpenetyou have a point#2016-09-2219:16mpenetI just wanted to get rid of "qbits.alia" if I recall, since I used this single file for the whole project I still like to keep some ns'ing#2016-09-2219:46nicolaWe had discussion on clojure-russia hangout about clojure.spec & namespaced keys and there is an idea by @prepor: we have reader macro to do ::key => :my.ns/key , we have sugar to do (require ... :as a) ::a/key => :my.required.ns/key. But there are a lot of cases where we need something like ::.subns/key => my.ns.subns/key. For real world deeply nested data-structures create a lot of namespaces just for fully qualified keys for spec looks like overhead...#2016-09-2221:12patrkris@seancorfield: what was that workaround you mentioned earlier? > Thatā€™s nicer than the workaround I was using, from Alex. I think Iā€™ll switch to that @kkruit ā€” thanks!#2016-09-2221:13seancorfield@patrkris: https://clojurians.slack.com/archives/clojure-spec/p1474565192002064#2016-09-2221:14patrkris@seancorfield: I mean the one from Alex#2016-09-2221:16seancorfieldOh, that was to use in-ns twice which is what something in core does if I recall. #2016-09-2221:17seancorfieldI doubt it's still available in the scroll back and I don't remember what code he linked to. #2016-09-2221:21seancorfieldOk you made me curious so I searched the source code: https://github.com/clojure/clojure/blob/a1c3dafec01ab02fb10d91f98b9ffd3241e860c0/src/clj/clojure/spec/test.clj#L17 @patrkris #2016-09-2223:55noprompthas anyone been able to bend spec to parse and conform strings?#2016-09-2300:03devn@noprompt: what mad science are you up to now? :)#2016-09-2300:09noprompt@devn my problem is that i want to conform nested string data so i don't need to do it in two steps. it's possible to use conformer but when there's a problem the error message is really, really bad (and there's nothing available in the public api for custom explanations). i'd like to simply write a custom spec against the protocols but alexmiller, in a previous discussion, has cautioned against that.#2016-09-2300:10nopromptbasically i'm at a crossroads where it's either write the data portion of the parser by hand, forgoing spec, or bend spec to do my bidding.#2016-09-2300:56kennyHow do you spec a function with optional args that should be conformed to a map?
(defn foo
  [& {::keys [bar]}]
  ;...
  )
#2016-09-2300:59kidpollohmmm sounds interesting and actually useful @noprompt It might actually also help my use case#2016-09-2301:00kidpollowhen an input param for example is a string but the string contains an elastic search query#2016-09-2301:00kidpolloan extra ste would need to be made to validate the syntax of the string#2016-09-2301:05kennyThis is a more concrete example:
(s/conform (s/and #(even? (count %))
                  (s/conformer (partial apply hash-map))
                  (s/keys :req [::foo]))
           [::foo true])
There must be a better way of handling this for optional map args in fns
#2016-09-2301:09bfabry@kenny keys* is for this#2016-09-2301:09bfabryhttps://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec/keys*#2016-09-2301:09kennyAh! Perfect!#2016-09-2301:09kennyThanks!#2016-09-2301:09bfabryI knew I'd seen a specific function for it but it took me ages to find that again#2016-09-2301:42Alex Miller (Clojure team)@noprompt I will stick with my caution in trying to use spec regex to parse strings - that's not what it's designed for and I a normal regex is going to be a better choice. Why cant you use a regex predicate?#2016-09-2314:18djpowellwhat is the recommended way to make a spec for a constant string?#2016-09-2314:22donaldballI put it in a set#2016-09-2314:23djpowellooh - gen/return maybe?#2016-09-2314:26djpowellah yeah, a set seems to work ok#2016-09-2315:15Alex Miller (Clojure team)yes, a set #{ā€œfooā€}#2016-09-2317:10mishagood day! is there a way to spec a function, which I expect to get as an argument? like "I expect function of 2 arguments: int and string"#2016-09-2317:12mishaso far I see only s/fdef, which expects a symbol identical to existing function name.#2016-09-2317:13mishas/fspec!#2016-09-2317:24bfabry^ yes šŸ™‚#2016-09-2317:26bfabryalso, something which I think is a bit not clear from the docs, fdef adds an fspec to the registry under the fully qualified name of the function, so if you've written an fdef for foo, you can use the symbol `foo in place of a spec#2016-09-2318:07spiedenanyone know how to spec a map by predicates over its keys and values? really expected this to work:
(s/explain (s/+ (s/cat :foo string? :bar keyword?)) {"hi" :bye})
In: [0] val: ["hi" :bye] fails at: [:foo] predicate: string?
#2016-09-2318:08bfabry@spieden map-of#2016-09-2318:09spieden@bfabry thanks! ok, now just for academic interest:
(s/explain (s/+ (s/cat :foo string? :bar keyword?)) [["hi" :bye]])
#2016-09-2318:10spiedenah looks like i want ā€œevery"#2016-09-2318:10bfabrya map is also I believe a sequence of tuples, not a single long sequence of key,val,key,val#2016-09-2318:11spiedenyeah i guess i was expecting the regular expression to match multiple instances of [ā€œhiā€ :bye] within the sequence#2016-09-2318:11bfabryso this works
boot.user=> (s/explain (s/+ (s/spec (s/cat :foo string? :bar keyword?))) {"hi" :bye})
Success!
nil
#2016-09-2318:11spiedenooo#2016-09-2318:12bfabryas does this
boot.user=> (s/explain (s/+ (s/tuple string? keyword?)) {"hi" :bye})
Success!
nil
#2016-09-2318:12bfabrybut like I said, the simplest way to spec it is
boot.user=> (s/explain (s/map-of string? keyword?) {"hi" :bye})
Success!
nil
#2016-09-2318:13spiedenthatā€™s what iā€™m going with. thanks#2016-09-2321:11noprompt@alexmiller i can't use regex in my case because the strings i'm parsing are context-free.#2016-09-2321:12Alex Miller (Clojure team)so use a parser like instaparse?#2016-09-2321:13noprompt@alexmiller i probably would but unfortunately the story for clojurescript isn't solidified yet on that account.#2016-09-2321:15Alex Miller (Clojure team)@bfabry I would not use s/+ for that example above but (s/every (s/tuple string? keyword?)) - thatā€™s basically (minus things like :kind and :into) what map-of translates to#2016-09-2321:15noprompteven still, the story for plugging into explainability with conformer is just not good enough. yes, the result does tell what string is wrong but i'm not able to say where in the string something is wrong.#2016-09-2321:15nopromptto be sure, i'm not complaining about spec. it's already saved me a huge amount of work. if i could tap into explainability it'd be a closed case for me.#2016-09-2321:16Alex Miller (Clojure team)not sure what to tell ya - youā€™re asking for things spec is unlikely to provide ĀÆ\(惄)/ĀÆ#2016-09-2321:17nopromptwhy wouldn't spec want to expose hooks to explainability?#2016-09-2321:17Alex Miller (Clojure team)because one of the ideas in spec is to make explainability generic#2016-09-2321:18nopromptthat presupposes that such an idea is possible.#2016-09-2321:18Alex Miller (Clojure team)I havenā€™t talked with Rich about this specifically so this is just my read on it#2016-09-2321:19noprompt(not saying it's not)#2016-09-2321:19Alex Miller (Clojure team)when you get to a predicate the explanation is: the predicate failed#2016-09-2321:19Alex Miller (Clojure team)if you want more detail, make finer-grained predicates and combine them#2016-09-2321:20Alex Miller (Clojure team)if you want that + some sort of string parsing, I think youā€™re into the territory of custom specs#2016-09-2321:20nopromptright, however, sometimes the context is a little more granular e.g. failure alone is not enough for reporting. sometimes there's additional context for why the predicate failed.#2016-09-2321:20nopromptcan we get a promise to stabilize the protocols? šŸ™‚#2016-09-2321:21Alex Miller (Clojure team)now? no#2016-09-2321:21noprompthahah#2016-09-2321:21Alex Miller (Clojure team)certainly not in alpha#2016-09-2321:21noprompti meant when 1.9 is stable.#2016-09-2321:21noprompthonestly, that would be phenomenal for my use case.#2016-09-2321:21Alex Miller (Clojure team)I donā€™t have an answer on that#2016-09-2321:22Alex Miller (Clojure team)we can provide guidance when things move past alpha#2016-09-2321:22nopromptchances are most folks won't be interested in implementing the protocols and will just use "stock" spec. a handful of people, like myself, would probably greatly appreciate and benefit from the protocols being stable.#2016-09-2321:23nopromptawesome.#2016-09-2321:23nopromptwould this conversation be better on the dev mailing list?#2016-09-2321:23Alex Miller (Clojure team)doesnā€™t make any difference to me#2016-09-2321:23nopromptfair enough.#2016-09-2321:24Alex Miller (Clojure team)I read both :)#2016-09-2400:35seancorfield@alexmiller I was just looking at https://github.com/clojure/clojure/commit/7ff4c70b13d29cf0a1fc7f19ba53413a38bb03d5 and noticed nonconforming which I had not previously seen in the APIā€¦ and now itā€™s deliberately undocumentedā€¦ I have several places where I have to conform with or and then use (conformer second) to "undo" the conformed result to get back what I need. Is there any likelihood that nonconforming would be "restored" to the documented API as something usable by developers? Or was it always intended to be an internal implementation detail and it just accidentally leaked?#2016-09-2400:36Alex Miller (Clojure team)It was accidentally public and stands a good chance of being removed#2016-09-2400:36seancorfield(I note that there actually is a test added in that commit for the nonconforming function, even thoā€™ that commit also removes it from the documentation)#2016-09-2400:36seancorfieldhttps://github.com/clojure/clojure/blob/7ff4c70b13d29cf0a1fc7f19ba53413a38bb03d5/test/clojure/test_clojure/spec.clj#L166#2016-09-2400:36Alex Miller (Clojure team)Well in my diligent way I added a test for the thing I fixed#2016-09-2400:37agguysā€¦ why when I have something like this:
(s/def ::mgen (s/with-gen
               (s/and not-empty map?)
               #(gen/map (s/gen ::keyword-with-question-mark) (s/gen boolean?))))

(gen/generate (s/gen ::mgen))
it would generate a map with multiple keysā€¦ but when used in prop/for-all always just a single key-value?
#2016-09-2400:37seancorfieldDo you think "flattening" out an or is a good use case for having something like nonconforming thoā€™?#2016-09-2400:38Alex Miller (Clojure team)But now that nilable doesn't use nonconforming Rich is considering removing it#2016-09-2400:38Alex Miller (Clojure team)You can still easily do that with a conformer on val#2016-09-2400:38seancorfieldTrue, for a simple spec. It gets harder with more complex specs.#2016-09-2400:39Alex Miller (Clojure team)Well, rely on it at your peril :)#2016-09-2400:40seancorfieldSo far thatā€™s the only case Iā€™ve run into thoā€™, where I need s/or in order to combine specs but I really donā€™t want that branch in the result. (s/and (s/or :a ::some-pred :b ::other-pred) (conformer second)) seems a bit of a mouthful šŸ˜ø#2016-09-2400:41Alex Miller (Clojure team) @ag test.check is a bit different from spec in that spec wraps generators in a no-arg function - maybe that's causing the disconnect?#2016-09-2400:42Alex Miller (Clojure team)@seancorfield: advice from us is: use conformers very sparingly#2016-09-2400:47aghmmā€¦ I still have no clue how to deal with that.#2016-09-2400:47agwrap into something like gen/fmap?#2016-09-2400:50agusing overrides? second param of s/gen ?#2016-09-2400:51Alex Miller (Clojure team)I don't totally understand what you're trying to do #2016-09-2400:53agI have a spec that generates a map (with gen/map), when used in gen/generate - it generates maps with multiple keys. but when used with test-checkā€™s prop-all and quick-check it always generates maps with single key/value pair#2016-09-2400:57Alex Miller (Clojure team)Hmm, not sure#2016-09-2401:03seancorfield@alexmiller My use case ā€” and itā€™s cropped up in a few specs Iā€™ve needed at work ā€” is that we have situations where we have some data that is either an X or a string that converts to an X (partly because of legacy constraints) but we want to conform the data to X in both cases and donā€™t care whether it was initially an X or a string.#2016-09-2401:05seancorfieldNow, you can certainly argue "Donā€™t do that!" but having it baked into the spec and being able to just (s/conform ::my-spec data) is nicer than having (if (string? data) ā€¦) everywhere that uses it.#2016-09-2406:14plexusis this a bug or a feature?#2016-09-2406:14plexus
(s/valid? #{:clojure.spec/foo} :clojure.spec/foo) ;;=> true
(s/valid? qualified-keyword? :clojure.spec/foo) ;;=> true
(s/valid? #{:clojure.spec/invalid} :clojure.spec/invalid) ;;=> false
(s/valid? qualified-keyword? :clojure.spec/invalid) ;;=> false
#2016-09-2408:24seancorfieldIt's a feature. :clojure.spec/invalid is special -- it only ever satisfies s/invalid?.#2016-09-2408:56mpenetOdd, seems like an impl. quirk#2016-09-2416:17borkdudeIt would be nice if t/instrument also gave a warning when the :ret spec doesnā€™t conform. I know spec only tests this with t/check, but is there a reason the first one isnā€™t in spec (optionally) right now? Seems useful for dev purposes.#2016-09-2416:25borkdudeIf I could programmatically insert a post-condition on a function that would get me far#2016-09-2417:23borkdudelittle hack: https://gist.github.com/borkdude/38336a9ef2eba8ee0b3c9536fda28233#2016-09-2420:03Alex Miller (Clojure team)@borkdude: you can use s/assert to state postconditions if you like#2016-09-2420:04Alex Miller (Clojure team)They can then be turned off and even optionally compiled out#2016-09-2420:22borkdude@alexmiller ah yes, like in the snippet#2016-09-2421:18jfntnIf I have defined a spec for a value like :datomic/tempid, how can I reuse that in a map spec thatā€™d say that the :db/id key should have a value that conforms to :datomic/tempid?#2016-09-2421:54jfntnI think I got something usable! The point is that I want to define a map spec for an entity, and reuse that spec in describing the inputs to a tx and the outputs from a read. The trick is that the read and write specs should be identical with the exception that :db/id takes on different values#2016-09-2421:55jfntnNot sure thatā€™s a good way to do this, would appreciate some feedback#2016-09-2422:56mishawhat am i doing wrong?
;; [org.clojure/clojure "1.9.0-alpha12"]
;; [org.clojure/clojurescript "1.9.229"]

=> (s/def :foo/bar
  (s/fspec :args
    (s/cat
      :id number?
      :type keyword?)))
:foo/bar

=> (s/spec :foo/bar)
#object[cljs.spec.t_cljs$spec32342]

=> (s/conform :foo/bar #(identity {}))
#object[TypeError TypeError: Cannot read property 'call' of undefined]

=> (s/explain-data :foo/bar '#(identity {}))
{:cljs.spec/problems [{:path [], :pred ifn?, :val (fn* [] (identity {})), :via [:foo/bar], :in []}]}
#2016-09-2422:58misha(I want to register a function spec to a keyword, to be able to "say" later: this map contains :foo/bar key, and its value is a function of 2 args: number and keyword)#2016-09-2422:59mishaor just to conform any other function to this function signature#2016-09-2423:10misha
=> (defn yoyo [f] (s/explain-data :foo/bar f))

=> (yoyo map)
#object[TypeError TypeError: Cannot read property 'call' of undefined]
#2016-09-2500:00mishahowever in clojure it works:
(s/explain-data :foo/bar #(identity {}))
=> #:clojure.spec{:problems [{:path [], :pred (apply fn), :val (0 :A), :reason "Wrong number of args (2) passed to: sandbox/eval7829/fn--7830", :via [:foo/bar], :in []}]}
#2016-09-2505:48Oliver GeorgeAnyone here use devcards? I'm experimenting with how cards could be initialised based on spec/gen data. Would love any opinions/insights. Ticket: https://github.com/bhauman/devcards/issues/112 and gist https://gist.github.com/olivergeorge/82a20dd03fd86e82ab9b0f3959590f3f#2016-09-2505:50Oliver GeorgePerhaps my use case is special but it seems like something which might be useful to others.#2016-09-2505:55Oliver GeorgeThe other fiddly bit was working with the function sym. I needed to resolve the sym to the current local var value but also keep the sym to call (s/get-spec ...). In the end I used (var). Not sure if there's downsides to that I'm missing.#2016-09-2509:33stathissideris@olivergeorge itā€™s been done before šŸ™‚ https://juxt.pro/blog/posts/generative-ui-clojure-spec.html#2016-09-2510:09Oliver GeorgeThanks @stathissideris #2016-09-2510:43Oliver GeorgeI think that was my inspiration. I've added a bit more finessing about showing interesting random data (overrides) and controls (seed & size) which might make for a more useful UI dev tool. #2016-09-2510:46Oliver GeorgeNeeds some helper generators. I never leave home without my lorem ipsum generator. #2016-09-2613:13Oliver Georgehttps://github.com/olivergeorge/devcards-vs-clojure-spec#2016-09-2614:22kestrel7Is it possible to define a spec that depends on multiple keys within a map? e.g. ::date-1 must be greater that ::date-2#2016-09-2614:45bcbradleyi'm having issues with setting up clojure.spec#2016-09-2614:47bcbradleyi'm getting "could not locate clojure/test/check/generators__init.class or clojure/test/check/generators.clj on class path"#2016-09-2614:47bcbradleywhat does that mean?#2016-09-2614:54thegeez@bcbradley that might be due to this: http://clojure.org/guides/spec#_project_setup#2016-09-2614:54bcbradleythankyou!#2016-09-2614:58bcbradleyi'm still getting the error šŸ˜•#2016-09-2615:01bcbradleyhere is my lein#2016-09-2615:01bcbradley(defproject sand "0.0.1-SNAPSHOT" :description "FIXME: write description" :profiles {:dev {:dependencies [[org.clojure/test.check "0.9.0"]]}} :dependencies [[org.clojure/clojure "1.9.0-alpha12"]] :javac-options ["-target" "1.6" "-source" "1.6" "-Xlint:-options"] :aot [sand.core] :main sand.core)#2016-09-2615:02thegeezIf you are using Leiningen, you can try to put the dependency at the top level. So put it next to [org.clojure/clojure ...] in :dependencies#2016-09-2615:02bcbradleyyou mean don't use a dev profile?#2016-09-2615:03misha@bcbradley clj or cljs ?#2016-09-2615:03bcbradleyclojure#2016-09-2615:03bcbradleythe function causing the issue is exercise-fn#2016-09-2615:03thegeezYeah, just to check if that works. You might be using the test.check function in a way that does not fall under leiningen dev profile#2016-09-2615:04thegeezor in a place, rather than a way#2016-09-2615:04bcbradley(defproject sand "0.0.1-SNAPSHOT" :description "FIXME: write description" :dependencies [[org.clojure/clojure "1.9.0-alpha12"] [org.clojure/test.check "0.9.0"]] :javac-options ["-target" "1.6" "-source" "1.6" "-Xlint:-options"] :aot [sand.core] :main sand.core)#2016-09-2615:04bcbradleystill no luck
#2016-09-2615:05bcbradleyi can use clojure.spec/fdef#2016-09-2615:05bcbradleybut can't use clojure.spec/exercise-fn#2016-09-2615:15thegeezhmm curious, it works on my machine šŸ˜‰#2016-09-2615:17bcbradleyhere is what i'm looking at#2016-09-2615:22seancorfield@bcbradley Youā€™re AOTā€™ing that namespace so youā€™re forcing all those specs ā€” including exercise-fn ā€” to run at compile time.#2016-09-2615:23bcbradleyi see#2016-09-2615:23bcbradleyi'm sorry for being so new at this šŸ˜ž#2016-09-2615:23bcbradleythe AOT is an acronym i assume#2016-09-2615:23bcbradleywhat does it stand for?#2016-09-2615:23seancorfieldAhead Of Time (compilation) ā€” relating to the :aot in your project.clj#2016-09-2615:25seancorfieldStill, having the exercise-fn call at the top-level of your code means it will be executed whenever that ns is run ā€” put it inside -main so it only runs when you run the -main...#2016-09-2615:25seancorfieldSame with generate.#2016-09-2615:25bcbradleyok, i was wondering though#2016-09-2615:26bcbradleyif i wanted to make a library how would i test the code? Would i need a main?#2016-09-2615:26seancorfieldYouā€™d normally put test code in a separate ns and have it run via one of the test runners.#2016-09-2615:27seancorfieldIn your project tree, youā€™ll see a test folder next to src.#2016-09-2615:27seancorfieldHmm, no, in your case you wonā€™t ā€” did you create that project manually?#2016-09-2615:28seancorfield(Iā€™m not familiar with how NightCode creates projectsā€¦ but normally lein new app myapp will create a test folder as well as a src folder)#2016-09-2615:28bcbradleyi just chose "create new console project"#2016-09-2615:28bcbradleyi've used lein before#2016-09-2615:28bcbradleybut importing a lein project into nightcode gave me some problems#2016-09-2615:29seancorfieldIā€™m surprised (and a bit disappointed) that NightCode doesnā€™t create a test folder tree for all projects šŸ˜ž#2016-09-2615:29thegeezI think this is an issue with nightcode. A top level exercise-fn with :gen-class and aot would works#2016-09-2615:29bcbradleyi've had trouble finding a good ide for clojure that i like#2016-09-2615:30seancorfield@thegeez Well, if test.check is part of the regular :dependencies yes, but you donā€™t want that code running at compile time or ns load time...#2016-09-2615:30bcbradleyi don't wanna use emacs because its 2016, i don't like eclipse because its sluggish and smells of enterprise, i don't like intellij + cursive because i don't want to pay for it, I don't like netbeans because netbeans. Light table isn't working with clojure 1.9-alpha12#2016-09-2615:30bcbradleyit looks like nightcode is my only choice unless i want to rock it in vim#2016-09-2615:31seancorfieldThe majority of Clojure devs use a form of Emacs according to the annual "State of Clojure" surveys šŸ™‚#2016-09-2615:31bcbradleyi'd prefer not to develop carpal tunnel before i turn 30#2016-09-2615:32bcbradleyjabbing aside, even after removing the AOT and pulling it into main i get some issues#2016-09-2615:34thegeez@seancorfield yes that setup is not ideal, but more meant to isolate the problem#2016-09-2615:35thegeez@bcbradley now you have the name my-index-of and a spec for my-index-of2#2016-09-2615:36seancorfield@bcbradley Your spec has my-index-of2 but the function ā€¦ yeah, what @thegeez said šŸ™‚#2016-09-2615:36bcbradleyyou are right!#2016-09-2615:36bcbradleythat fixed it xd#2016-09-2615:37bcbradleyi know i'm probably being a pest, but can you explain the difference between clojure.spec and any normal library like clojure.set?#2016-09-2615:37bcbradleyi mean in terms of pitfalls,#2016-09-2615:37bcbradleylike for instance, i didn't know that i "shouldn't" invoke certain methods or macros at the top level#2016-09-2615:37bcbradleynormally i wouldn't invoke anything at the top level in an application, but i figure heck#2016-09-2615:37bcbradleyif it works on a repl#2016-09-2615:37bcbradleyi'm just trying to learn#2016-09-2615:38bcbradleyi need to know what NOT to do#2016-09-2615:44seancorfieldIn general you donā€™t want any executable code at the top-level of your namespace.#2016-09-2615:46seancorfieldWhatā€™s your language background, prior to Clojure? (so weā€™ll have a frame of reference for what youā€™re familiar with)#2016-09-2615:48seancorfieldAnd regarding the difference between clojure.spec and "normal" libraries, probably the big one is that thereā€™s a bunch of "testing" stuff in clojure.spec that belongs in test code, where test.check gets brought in via a :dev profile (for REPL and running tests, as opposed to "production" line code). Does that help?#2016-09-2615:49seancorfieldThe specs themselves ā€” and valid? and conform ā€” are intended for production code, but the generative testing stuff (`generate`, exercise, check) is intended for either interactive use or for running as part of test code.#2016-09-2616:02bcbradleysorry i was away, my language background: C, C++, Java, Python, Javascript and then Clojure#2016-09-2616:03bcbradleyi've got 8 years in C, 6 years in C++, 3 years in python, 2 years in javascript and 1 year in clojure#2016-09-2616:04bcbradleyi'm in my senior year for software engineering and would like to get a job where the primary language is clojure, or at the very least some other functional oriented job with scala or erlang or whatever#2016-09-2616:04bcbradleyi've seen enough of stateful programs to know they are a pitfall#2016-09-2616:04seancorfieldšŸ˜„#2016-09-2616:04bcbradleyand i'm not just espousing things i've heard#2016-09-2616:05bcbradleyi've had time to think about it for myself#2016-09-2616:06bcbradleymy experience with clojure is probaby summarized best as follows:#2016-09-2616:06bcbradleyi know how to use map, reduce, apply partial blah blah blah, i've been doing it for a while#2016-09-2616:06bcbradleywhen i moved from javascript to clojure i did it for elegance#2016-09-2616:06seancorfield'k so a Clojure namespace produces a class, and all the top-level forms become static initializers in that classā€¦ if you think of it in Java / C++ terms, those static initializers get run when a class is loaded, long before any class instance is createdā€¦ so the same caveats apply.#2016-09-2616:07bcbradleyah i see#2016-09-2616:07bcbradleyi don't think i'm actually familiar with static initializers#2016-09-2616:08bcbradleywell a search on stack overflow was fast enough#2016-09-2616:08bcbradleygood to know thats how clojure namespaces work#2016-09-2616:10bcbradleyso basically what you are saying is that if I have a bunch of top level forms that depend on some other class already being loaded, but it hasn't been loaded, i'll get a "class nil" error or some such#2016-09-2616:11bcbradleythat makes perfect sense#2016-09-2616:50bfabry@bcbradley cursive is free for non-commercial use, fyi#2016-09-2616:52bfabryand in a commercial setting, I can't imagine any company would care about the cost#2016-09-2617:05bfabryit's also rapidly catching up with the the perpetual yak emacs šŸ˜›#2016-09-2617:20seancorfieldSome of us really like yaks...#2016-09-2617:33dominicmMy vim shaving is unmatched in fun vim #2016-09-2619:10Alex Miller (Clojure team)1.9.0-alpha13 is out https://groups.google.com/d/msg/clojure/QWPUWG9BwbE/9a7ymJb9AQAJ#2016-09-2619:11Alex Miller (Clojure team)which is mostly fixes and improvements to nilable, which is also probably a little faster#2016-09-2621:57bcbradleyi really like nightcode for some reason#2016-09-2621:57bcbradleyidk its just not that pretentious#2016-09-2621:57bcbradleyits not one thing pretending to be another. Its just a clojure ide. Thats it.#2016-09-2712:48kurt-yagramhow to s/fdef functions with optional named args?
(s/fdef myfun 
  :args (s/cat :first string? <???>))
(defn myfun "" [first & {:keys [named]}]
   ...)
so what should be at <???> to add a spec for the named argument?
#2016-09-2712:54madstap(s/? (s/cat :named-k #{:named} :named boolean?))#2016-09-2712:54kurt-yagramGot it. Makes sense.#2016-09-2714:51Alex Miller (Clojure team)no, donā€™t do that - this is what s/keys* is for#2016-09-2714:52Alex Miller (Clojure team)
(s/def ::named boolean?)
(s/fdef myfun :args (s/cat :first string? :opts (s/keys* :req-un [::named])))
#2016-09-2714:53Alex Miller (Clojure team)this allows you to specify any number of kwarg options at the end in any order#2016-09-2715:42madstapRight, I had missed s/keys*, thanks for the clarification. Sometimes it seems like you guys think of everything, probably all that hammock time.#2016-09-2715:45kurt-yagramah, ok... thanks!#2016-09-2800:07jfntnI canā€™t find a good solution to write different specs that all have a :db/id key but with different values#2016-09-2800:08danielcompton@jfntn what do you mean by ā€˜different valuesā€™?#2016-09-2800:09jfntnWell one would be say a ::temp-entity spec thatā€™d check that :db/id is a datomic/tempid, and the other would be an ::entity spec that would check that :db/id is an entity id (long)#2016-09-2800:10jfntnSo I need to attach multiple specs to the same namespaced key, but s/or wonā€™t do since the point is to be able to assert whether an entity is temporary or not...#2016-09-2800:11jfntnI found a weird trick using multiple slashes in the spec name and using :req-un in the spec, but those donā€™t seem to really be supported since Iā€™m getting intermitent analyzer errors and the keys wonā€™t compile in cljs#2016-09-2800:15bhaganyI think in this case, I would use s/or, and use the conformed value to say whether itā€™s a temp id or not#2016-09-2800:16bhaganyitā€™s an extra step, but I donā€™t see another way to make it work#2016-09-2800:18bhaganyif youā€™re spec-ing a function that needs to take a temp entity, or something like that, you could always make a predicate function, temp-entity? or some such, that conforms its argument and checks which branch s/or took#2016-09-2800:32jfntn@bhagany thatā€™s a good idea!#2016-09-2800:33jfntnNot as direct, but Iā€™m able to express what I wanted, which is a reusable keys spec and combine it with the temp and final specs#2016-09-2806:17misha@jfntn
-(s/def :datomic.tempid/idx int?)
+(s/def :datomic.tempid/idx neg-int?)
#2016-09-2806:42przemekHi, while trying to migrate from schema to spec I've come across an issue. Following the guide I was able to define specs for maps, but I think I have a case that is not covered. Basically the same key i.e. :id or :name may be an attribute of different entities. If I define ::name then the only way to use a different type definition for :name is to use namespaces which puts me in sort of OO style when I keep module per "noun".... Is there not a way to decouple key from value so that I can define i.e. ::plan-name type, but use :name in my map?#2016-09-2806:50misha@przemek what about (s/keys :req-un [...] :opt-un [...]?#2016-09-2806:55mishabasically, you'd have multiple namespaced specs for name: :foo/name, :bar/name; and then use them in map specs, but inside :req-un (or :opt-un), with the only requirement of keeping (name kw) identical#2016-09-2806:59misha
(s/def :foo/id int?)
(s/def :bar/id string?)
(s/def :foo/thing (s/keys :req-un [:foo/id]))
(s/def :bar/thing (s/keys :req-un [:bar/id]))

(s/valid? :foo/thing {:id 1})
=> true
(s/valid? :foo/thing {:id "one"})
=> false

(s/valid? :bar/thing {:id "one"})
=> true
(s/valid? :bar/thing {:id 1})
=> false
#2016-09-2807:02przemekhell yes šŸ™‚ That looks like what I was looking for. I makes sense to document more how namespacing looks in action. I am gonna try this one in a moment. thanks#2016-09-2807:11przemekit works. I guess for Datomic that :xxx/yy pattern is heavily used (I have not used it though). thanks again!#2016-09-2807:53jmglovIs there anyone here who can shed any light on my customer generator for an atom holding a map? https://clojurians.slack.com/archives/clojure-spec/p1474549130002028#2016-09-2807:53jmglovOr is this something better posted to the mailing list?#2016-09-2808:22przemekAnother question. Given this definition (s/def ::email (s/and string? #(re-matches email-regex %))) I am not able to generate a sample getting "Couldn't satisfy such-that predicate after 100 tries".#2016-09-2808:25przemekOk. Solved. Found a paragraph in the guide on custom generators.#2016-09-2810:42madstap@przemek https://github.com/gfredericks/test.chuck has a string-from-regex generator#2016-09-2810:50przemek@madstap thx, will look at that. I managed to build a custom gen, but this is more elegant#2016-09-2813:16jfntn@misha good catch, thanks!#2016-09-2901:52richiardiandreaHello all! A question, let's say I have:
(s/def ::event-common (s/keys :req [::nonce ::payload]))
I would like to declare an event that has fixed ::type:
(s/def ::command-event (s/merge ::event-common -- what do I put here? --))
#2016-09-2901:53richiardiandrea(I was also wondering if I can use s/and with my ::event-common def)#2016-09-2903:59Alex Miller (Clojure team)@richiardiandrea not entirely clear what you want to do. maybe: (s/def ::command-event (s/merge ::event-common (s/keys :req [::type]))) ?#2016-09-2904:01Alex Miller (Clojure team)or are you asking because they canā€™t both be satisfied that way? I suspect this is the kind of thing where youā€™re happier using s/multi-spec#2016-09-2904:12richiardiandrea@alexmiller tnx for answering, I have a multi-spec in place so that I can check if the event conforms to my expected :types. Re-reading my question I see that maybe I am looking at it from the wrong angle..I will clarify my own ideas first and ask again#2016-09-2904:17richiardiandreaI think I wanted to be sure that a function returns only a certain :type of events. So I need to combine the ::common-event spec with a predicate...probably an s/and will suffice.#2016-09-2904:29Alex Miller (Clojure team)Yeah s/and is fine#2016-09-2906:32seancorfieldSo I still keep running into situations where I have a spec for a domain level entity that is, say, a set of keywords and then the spec for the input level entity is a set of strings that are the names of the keywords... so my gut says to spec the input as (s/and string? (s/conformer (comp keyword str/lower-case)) ::domain-entity) with a generator of (fn [] (g/fmap name (s/gen ::domain-entity)))#2016-09-2906:33seancorfieldBut @alexmiller has said to avoid s/conformer so I suspect this is an anti-pattern, even tho' it's cropping up over and over again.#2016-09-2906:35seancorfieldThe other (anti-)pattern I hit is a domain spec for an int (int-in or whatever) and the input spec is string? and parse to long and the domain spec, again with a generator of fmap str over the domain spec generator...#2016-09-2907:26mpenetThinking about using a set of specs just for json/http params coercion here, then pass the result to domain specs, I think I prefer to keep both clearly separate.#2016-09-2907:28mpenetmakes it a bit easier to reuse also maybe#2016-09-2907:29mpenetanyway, still have a ton of Schema code in our codebases, and still undecided on if/how to port it so far.#2016-09-2911:03mpenetis there an equivalent of describe/form that will resolve specs recursively?#2016-09-2911:04mpenetI mean resolve specs from the registry when it's composed from these#2016-09-2911:05mpenet-> apparently not#2016-09-2911:33mpenet(probably another anti-)pattern I encountered: setting a default value if nil when conforming, it's useful at input level too#2016-09-2911:37mpenet
(defn default [spec default]
  (s/conformer
   #(-> spec s/nilable (s/conform %) (or default))))
#2016-09-2911:53mpenetanother annoying quirk, the following won't compile on alpha13: (defn foo [x] :clojure.spec/invalid)
ExceptionInfo Call to clojure.core/defn did not conform to spec:
:clojure.spec/args  (foo [x] :clojure.spec/invalid)
  clojure.core/ex-info (core.clj:4725)
#2016-09-2911:58mpenetmakes it impossible to write a declared conformer ...#2016-09-2912:01mpenetwell you can create an alias first (def invalid :clojure.spec/invalid) and use it instead of :clojure.spec/invalid in the code, but that's kind of ugly#2016-09-2916:49jeroenvandijkQuestion, do people add docs to specs? Iā€™m making specs for some configuration data and I want to add meta data/documentation to individual specs to give more background information about the specific options#2016-09-2918:18seancorfieldI donā€™t think you can add docs to specsā€¦?#2016-09-2918:55Alex Miller (Clojure team)you canā€™t#2016-09-2918:56Alex Miller (Clojure team)but feel free to vote on the idea here http://dev.clojure.org/jira/browse/CLJ-1965#2016-09-2922:28jrheardfirst time spec user here, trying to wrap my head around the best way of using namespaced keywords, never used ā€˜em before whatā€™s the more correct pattern: having my keywords have namespaces that match my projectā€™s namespaces, like :myapp.system.collision/foo? or having my keywords have namespaces that correspond to arbitrary concepts, unrelated to my appā€™s file/namespace structure, like :collision/foo?#2016-09-2922:38jrheardso far iā€™m going with the latter, and it feels pretty good, but iā€™d just like to confirm with the experts šŸ™‚#2016-09-2922:48jrheardalso - do you folks have any favorite example github repos that use spec, like, completely idiomatically? iā€™m switching my project over from schema, and some parts of the transition are straightforward, but other parts just feel like they could be done in a variety of different ways and iā€™d love to have a few canonical examples to look at and learn from#2016-09-2922:52bfabryit's probably too early for that to exist =/#2016-09-2922:53bfabryfwiw I'm leaning toward the latter of your two options as wel#2016-09-2923:08seancorfield@jrheard Weā€™re creating new namespaces specifically for the data structure specs, and then requiring them into code namespaces as we need them. In a few places we have both data/specs and code. Itā€™s whatever seems most appropriate.#2016-09-2923:09seancorfieldBTW, if folks have any thoughts on my comments posted late last night https://clojurians.slack.com/archives/clojure-spec/p1475130747003048 (specifically about the pros and cons of using s/conformer)#2016-09-2923:10seancorfield(if you click that link, youā€™ll see two more follow-on comments)#2016-09-3001:20aghey guysā€¦ anyone here using spec and prop. based testing in clojuresript? I have encountered strange problem. (s/gen) within a test that works perfectly on my machine, on CI machine (same java, lein, node and phantomjs versions) says ""Couldn't satisfy such-that predicate after 100 tries.ā€,#2016-09-3001:21agis it possible that a certain spec would work on one machine and fail on other?#2016-09-3001:22agitā€™s not like random. I run it many times. it passes on my machine and it ALWAYS fails on CI machine#2016-09-3001:22agother tests work fine though#2016-09-3001:22agĀÆ\(惄)/ĀÆ#2016-09-3001:56danielcomptonwhat kind of spec is it?#2016-09-3002:15agohā€¦ ohā€¦ itā€™s a bit nastyā€¦ I need to generate iso-date strings and I did this:
(s/def :date/iso (s/with-gen (s/and string?
                               #(re-matches iso-date-regex %))
                   #(gen'/for [d (gen'/datetime)]
                      #?(:clj  (time-f/unparse local-date-formatter d)
                         :cljs (.format (js/moment (js/Date. d)))))))
#2016-09-3002:15agusing test.chuck#2016-09-3002:38bfabry@ag my first guess would be a different timezone setting between CI and your pc#2016-09-3006:26danielstocktonI know this has probably been asked 1000 times but what is the prevailing preference for where to put specs? In their own parallel namespace or inline?#2016-09-3006:27stathissiderismy impression is inline for function specs, different namespace for ā€œdomainā€ specs#2016-09-3006:27seancorfieldIt depends.#2016-09-3006:29seancorfieldBut yeah, I think @stathissideris comment reflects what is becoming the most likely best practice at this point. It's what we're doing at World Singles so far...#2016-09-3006:29danielstocktonand "domain" spec means specs which are likely to impact more than one namespace and be reused a lot?#2016-09-3006:30seancorfieldData structure specs, as opposed to function specs.#2016-09-3006:36danielstocktonKnow of any open source projects that are already using it, that I could take a look at?#2016-09-3006:40seancorfieldclojure.java.jdbc? šŸ™‚#2016-09-3006:42danielstocktonThanks. Looks like they follow the some.namespace and some.namespace.spec convention instead#2016-09-3006:43danielstocktonI was thinking an advantage of in-lining is that it provides some documentation, but am I right that spec adds something to the docstring anyway?#2016-09-3006:45danielstocktons/they/yourself/ i see#2016-09-3006:45bfabryfdef specs are added to the output of doc ya#2016-09-3006:46danielstocktonOk, I might be leaning towards a separate namespace in that case#2016-09-3006:47seancorfieldInlining fdef makes sense for documentation, but not for portability across versions. def for data structures in a separate namespace makes sense regardless.#2016-09-3006:48seancorfieldWe've generally found if we start to def a data structure spec in a namespace with functions, it's gets messy ... keeping the data structure specs separate is more flexible...#2016-09-3006:50danielstocktonI'm worried things will be messy with function specs too, it seems the documentation advantage is somewhat negated by the docstring#2016-09-3006:55seancorfieldNot sure what you mean...? Specs just add to the docstring...#2016-09-3007:08danielstocktonI mean that if it's added to the docstring then it isn't much of an advantage having it inline as well.#2016-09-3007:09seancorfieldRight, yes, but the docstring is only updated if you've loaded the ns with the spec so you might as well have the fdef inline (IMO)#2016-09-3007:10bfabryyeah.. different documentation requirements. documentation for people reading/maintaining the code, documentation for people using the function#2016-09-3007:20yendaHi, I am trying to use spec to conform the arguments of a function: there is at most 4 arguments with 4 distinct types. I want to conform them into a map but the solution I came up with is not fully compliant because it allows 2 arguments of the same type. Is there a way to make sure there isn't 2 arguments of the same type ?#2016-09-3007:20seancorfieldNot sure I follow... can you share some code?#2016-09-3007:21yenda@seancorfield yes sorry you are too quick šŸ˜„#2016-09-3007:21seancorfieldšŸ™‚#2016-09-3007:27seancorfieldYour coll-of is a map so each :message item overwrites any prior item before :distinct can be checked.#2016-09-3007:28seancorfieldAnd if you used :into [] you would still accept those two :message arguments because they are different values so they are distinct.#2016-09-3007:28seancorfieldThe :distinct flag says that all the conformed values must be distinct -- which they are.#2016-09-3007:29seancorfieldYou'd need your ::argument spec to conform to just the type, not the type and value.#2016-09-3007:34seancorfieldHmm, that doesn't quite work either:
(s/def ::argument (s/and (s/or :message string? :app-data map? :type keyword? :variables vector?) (s/conformer first)))
(s/def ::arguments (s/coll-of ::argument :distinct true :into [] :max-count 4))
#2016-09-3007:34seancorfield
(s/conform ::arguments ["helo {a} hello"  "hello" ::random-log {:c '(inc c)}])
[:message :message :type :app-data]
Not what I expected.
#2016-09-3007:35yenda@seancorfield if I have 2 identical arguments the predicate distinct? is triggered so the distinct check happens before the into {}#2016-09-3007:36yendahowever this not really what I want I could remove it, what I would like is to have distinct types for the arguments not distinct arguments#2016-09-3007:37seancorfieldYeah, I think you're right about the :distinct check... it would check distinct values before the conforming.#2016-09-3007:37bfabrythere's something from a very long time ago whispering in my ear that this is an irregular grammar, and so you're going to need a custom predicate to validate it. but I could be wrong#2016-09-3007:38bfabrys/irregular/not context free/#2016-09-3007:38seancorfieldSo you need s/and on ::arguments to force it to be distinct types.#2016-09-3007:39seancorfield
(s/def ::arguments (s/and (s/coll-of ::argument :into [] :max-count 4) (s/coll-of keyword? :distinct true)))
#2016-09-3007:39seancorfield
(s/conform ::arguments ["helo {a} hello"  "hello" ::random-log {:c '(inc c)}])
:clojure.spec/invalid
#2016-09-3007:40seancorfield
(s/conform ::arguments ["helo {a} hello" ['b] ::random-log {:c '(inc c)}])
[:message :variables :type :app-data]
#2016-09-3007:40seancorfieldAnd that's with
(s/def ::argument (s/and (s/or :message string? :app-data map? :type keyword? :variables vector?) (s/conformer first)))
#2016-09-3007:41seancorfieldSo each argument needs to conform to its type (throwing away the value) and then the arguments collection needs to flow the conformed arguments into a distinct collection#2016-09-3007:43yendawow thank you it works but it also makes me realize I still have a lot of work to do on spec šŸ™‚#2016-09-3007:48seancorfieldYeah, it's radically changing how we approach several types of problems. #2016-09-3007:49yendaoh ok I understand now because I didn't realize that I don't get the values anymore while conforming#2016-09-3007:51yendathe function I don't get is conformer#2016-09-3007:55seancorfieldTook me a while too. conformer uses its function argument to transform the data rather than just being a true/false predicate. #2016-09-3007:55seancorfieldBut Alex says it's an anti-pattern so be cautious. #2016-09-3008:20yendawell I'll play a bit to see if I can keep my conformed map and still have the validation on distinct keywords#2016-09-3008:21yendabtw do I misunderstand unform ? I thought this call (s/unform ::x (s/conform ::x data-to-conform)) would return data-to-conform but it just returns the conformed data#2016-09-3008:30yenda@seancorfield is the explanation on why it is an anti-pattern available somewhere ?#2016-09-3008:32yendabelow is the final spec I made thanks to your input that matches my needs, I could eventually just do the map transform outside of the spec iff conformer is an anti-pattern#2016-09-3008:39jmglov@ag Here's how I generate date strings:
(require '[clj-time.core :as t]
         '[clj-time.format :as f])
(import '(org.joda.time.format DateTimeFormatter))

(defn parseable-timestamp
  "Returns a spec for a timestamp string can be parsed with the specified
   datetime formatter."
  [formatter]
  (s/and string?
         (fn [timestamp-str]
           (try
             (f/parse formatter timestamp-str)
             true
             (catch Exception _
               false)))))

(defn make-timestamp-gen
  "Returns a generator for a timestamp string between the minimum year and
   maximum year (exclusive) that can be parsed with the specified datetime
   formatter."
  [min-year max-year formatter]
  (fn []
    (let [year-gen (s/gen (s/int-in min-year max-year))
          month-gen (s/gen (s/int-in 1 12))
          day-gen (s/gen (s/int-in 1 28))
          hour-gen (s/gen (s/int-in 0 23))
          m-s-gen (s/gen (s/int-in 0 59))]
      (->> [year-gen month-gen day-gen hour-gen m-s-gen m-s-gen m-s-gen]
           (map #(gen/fmap vector %))
           (apply gen/cat)
           (gen/fmap #(->> (apply t/date-time %)
                           (f/unparse formatter)))))))

(def timestamp-formatter (f/formatters :date-time-no-ms))

(s/def ::timestamp
  (s/with-gen
    (parseable-timestamp timestamp-formatter)
    (make-timestamp-gen 2000 2050 timestamp-formatter)))
#2016-09-3013:26mpenetasking again: I'd like to generate documentation from specs, is there a way to resolve/expand specs from (s/form ...) (or other) ?#2016-09-3013:28mpenetI guess I can hack this walking the (s/registry) + (s/form ...), but that seems a bit "hairy"#2016-09-3013:41mpenet(it's for front-end team, so I cant just spit keywords, i need to resolve the leafs of specs to stuff they understand)#2016-09-3013:53mpenetmight be a good fit for some specter juggling#2016-09-3014:02Alex Miller (Clojure team)are you talking about a ā€œdeepā€ version of s/form?#2016-09-3014:03mpenetyes#2016-09-3014:03Alex Miller (Clojure team)yeah, weā€™ve talked about providing that but it doesnā€™t exist right now#2016-09-3014:04mpenetthat'd be nice, it's should be easy enough to write, but that's something one would expect out of the box imho#2016-09-3014:04Alex Miller (Clojure team)I have specs for spec forms (kind of, modulo a number of bugs in s/form) which could help a lot#2016-09-3014:04mpenetsounds good#2016-09-3014:04Alex Miller (Clojure team)because you could conform the spec and then pick the parts per form that you need#2016-09-3014:05Alex Miller (Clojure team)I will probably get around to this soon-ish as I need it for other things#2016-09-3014:05mpenetI didn't actually think about approaching it that way, neat#2016-09-3014:05Alex Miller (Clojure team)having specs for specs opens up all sorts of things#2016-09-3014:06Alex Miller (Clojure team)for example, automated testing of spec by generating specs from spec specs, then checking conformance on data generated for the generated spec#2016-09-3014:06mpenetthat's spec inception#2016-09-3014:06mpenetoh since you're here, I have another question about the conformer + :clojure.spec/invalid issue I mentioned yesterday#2016-09-3014:07Alex Miller (Clojure team)yes, it feels like that :)#2016-09-3014:07mpenethttps://clojurians.slack.com/archives/clojure-spec/p1475150010003090#2016-09-3014:07Alex Miller (Clojure team)can you repeat? I canā€™t keep up the slacks#2016-09-3014:07mpenetit used to compile (I think with alpha10)#2016-09-3014:07Alex Miller (Clojure team)yeah, thatā€™s been logged#2016-09-3014:07mpenetprobably a result of the recent optimisations#2016-09-3014:07mpenet(guess)#2016-09-3014:07Alex Miller (Clojure team)would have changed as of alpha 11 or 12#2016-09-3014:07mpenetoki, good to hear#2016-09-3014:08Alex Miller (Clojure team)I mean changed as in ā€œstarted failingā€#2016-09-3014:08mpenetI couldn't find a ticket about it in jira#2016-09-3014:08Alex Miller (Clojure team)Iā€™m not sure there is a good solution to it#2016-09-3014:08mpenetoh, isn't this considered a bug?#2016-09-3014:09Alex Miller (Clojure team)http://dev.clojure.org/jira/browse/CLJ-1966 is basically the same probelm#2016-09-3014:09mpenetit makes writing conformers a bit tricky, I basically have to (def invalid :clojure.spec/invalid) and use this in the body of the functions I am using#2016-09-3014:12Alex Miller (Clojure team)yeah, but wait for a spec on def to break that too :)#2016-09-3014:12mpenetšŸ˜ž#2016-09-3014:12Alex Miller (Clojure team)itā€™s a broader problem#2016-09-3014:12Alex Miller (Clojure team)one possible solution would be to do something like is done in the reader where a reader is handed a token that it can return to indicate a special case#2016-09-3014:13Alex Miller (Clojure team)so rather than being expected to return ::s/invalid, you are given a token that you can return (and that can be a per-instance (Object.))#2016-09-3014:13Alex Miller (Clojure team)something like that#2016-09-3014:14mpenetok#2016-09-3014:15Alex Miller (Clojure team)that still doesnā€™t solve the problems around core specs though#2016-09-3014:15Alex Miller (Clojure team)it needs a longer conversation with Rich and he hasnā€™t had the time to have it#2016-09-3014:15mpenetany hint of what might come in the next alpha?#2016-09-3014:28Alex Miller (Clojure team)whatever we fix or add next :)#2016-09-3014:30Alex Miller (Clojure team)CLJ-2024, CLJ-2027, and CLJ-2026 all have patches that are ready for Stu and Rich to look at so I expect those to be in the next alpha#2016-09-3014:30Alex Miller (Clojure team)(presuming they like them)#2016-09-3014:51mpenetActually walking the specs isn't really possible in my case: if you do (s/def ::foo ::bar) (s/form ::foo) will have expanded ::bar to a predicate so I cannot check my stop point (since I don't want to expand down to the last bits, I have a set of specs that are the surface I want to expose to the front-end people)#2016-09-3014:53mpenet
(s/def ::foo any?)
(s/def ::bar ::foo)
(s/form ::bar)
=> clojure.core/any?
#2016-09-3014:54mpenet((s/registry) ::bar) works in that case actually#2016-09-3014:54mpenetnevermind#2016-09-3019:01jrheardiā€™ve got a somewhat complex map spec that iā€™m trying to s/exercise, and iā€™m getting the dreaded "Couldn't satisfy such-that predicate after 100 tries.ā€ error. what are the primary things i should be looking for when tracking down the root cause?#2016-09-3019:01jrheardiā€™m familiar with s/and and how you should eg do int? first, followed by even?, etc#2016-09-3019:02jrheardis naive s/and usage the primary trigger for this error, or are there other categories of spec misuse that often cause it?#2016-09-3019:02jrheard(if itā€™s helpful, the spec iā€™m trying to exercise is https://github.com/jrheard/voke/blob/spec/src/voke/specs.cljs#L74 )#2016-09-3019:02jrheardiā€™ve only got one s/and in here, and itā€™s a (s/and number? pos?), so i feel like there must be some other category of thing iā€™m doing totally incorrectly#2016-09-3019:14jrheardhm, seems to be specific to :component/input#2016-09-3019:15jrheardthis guyā€™s the culprit: https://github.com/jrheard/voke/blob/spec/src/voke/specs.cljs#L22#2016-09-3019:17jrheardthe documentation on :kind says: "Note that if :kind is specified and :into is not, this pred must generate in order for every to generate.ā€; and adding :into #{} fixes the issue. guess i need to sit down and take some time to understand the semantics of :kind and :into. thanks all ā¤ļø#2016-09-3019:26jrheard(if you click on those links after coming back from lunch, pretend that the (s/coll-of) calls with :kind specified do not have a corresponding :into; thatā€™s the state the file was in when i linked it, and thatā€™s what the problem was šŸ™‚ )#2016-09-3019:32seancorfield@jrheard Late to your spec party but, yeah, naĆÆve s/and has been my primary trigger for that error ā€” Iā€™ve taken to using s/with-gen quite a lot to "help" clojure.spec exercise stuff.#2016-09-3020:04danielstocktonI s/fdefed a function but the doc string doesn't contain my spec. Did I miss something?#2016-09-3021:28kennyIs this the expected behavior?
(require '[clojure.core.specs :as clj-specs])
(def c (s/conform ::clj-specs/defn-args '(t
                                           [x y]
                                           x)))
=> #'boot.user/c
(s/unform ::clj-specs/defn-args c)
=> (t (x y) x)
Shouldnā€™t the args be unconformed to a vector, not a list?
#2016-09-3021:36kenny
(let [args '(t
              [x y]
              x)
      args-s ::clj-specs/defn-args
      c (s/conform args-s args)]
  (s/conform args-s (s/unform args-s c)))
=> :clojure.spec/invalid
#2016-09-3021:42bfabryhmmmmm#2016-09-3021:43bfabry::arg-list seems to use s/and not s/coll-of#2016-09-3021:43kennyYeah: https://github.com/clojure/clojure/blob/master/src/clj/clojure/core/specs.clj#L67#2016-09-3021:44bfabrystill, wouldn't flowing back through vector? cause it to be turned into a vector? shrug#2016-10-0100:09jrheardwhatā€™s the recommended way of actually running stest/check tests?#2016-10-0100:09jrheardiā€™m in a cljs project, using lein-doo and cljs.test#2016-10-0100:10jrheardshould i be using (stest/summarize-results (stest/check `foo/bar)), and verifying that thereā€™s no :check-failed key in the returned map? or is there a better way?#2016-10-0100:15jrheardeg https://github.com/pieterbreed/ansible-inventory-clj/blob/9857f22ccd92e21b629d5d79907e4c266bdcbc52/test/ansible_inventory_clj/core_test.clj#L79#2016-10-0109:31decoursinHow can I duplicate a spec?
(s/def ::my-spec number?)
(s/def ::a ::my-spec)
(s/def ::b ::my-spec)
This fails with:
=> #error {
 :cause "Assert failed: k must be namespaced keyword or resolvable symbol\n(c/and (ident? k) (namespace k))"
 :via
 [{:type clojure.lang.Compiler$CompilerException
   :message "java.lang.AssertionError: Assert failed: k must be namespaced keyword or resolvable symbol\n(c/and (ident? k) (namespace k)), compiling:(ann.clj:63:1)"
   :at [clojure.lang.Compiler$InvokeExpr eval "Compiler.java" 3657]}
  {:type java.lang.AssertionError
   :message "Assert failed: k must be namespaced keyword or resolvable symbol\n(c/and (ident? k) (namespace k))"
   :at [clojure.spec$def_impl invokeStatic "spec.clj" 315]}]
 :trace
 [[clojure.spec$def_impl invokeStatic "spec.clj" 315]
  [clojure.spec$def_impl invoke "spec.clj" 312]
  [clojure.lang.AFn applyToHelper "AFn.java" 160]
  [clojure.lang.AFn applyTo "AFn.java" 144]
...
#2016-10-0112:52Alex Miller (Clojure team)I canā€™t reproduce that - that should certainly work#2016-10-0112:52Alex Miller (Clojure team)what version are you using?#2016-10-0113:14puzzlerunform is not properly reversing vectors that have been conformed, e.g.
(spec/unform :clojure.core.specs/defn-args (spec/conform :clojure.core.specs/defn-args '(f [x] x)))
=> (f (x) x)
#2016-10-0113:14puzzlerI checked that the spec "knows" that the args must be a vector.#2016-10-0113:15puzzlerI'm trying to make a defn-like macro by using spec to conform with the defn spec, then manipulate, then unform.#2016-10-0113:25Alex Miller (Clojure team)yeah, this is a known issue with the spec as itā€™s not enforcing the vector part#2016-10-0113:26Alex Miller (Clojure team)we are talking about creating an s/vcat as doing regex + vector is tedious right now#2016-10-0113:27Alex Miller (Clojure team)and itā€™s relatively common in macro syntax#2016-10-0113:27Alex Miller (Clojure team)this is a good example of it#2016-10-0113:28Alex Miller (Clojure team)you can make it work using something like (s/and (s/cat ā€¦) (s/conformer vec)) but like I said, itā€™s tedious#2016-10-0113:30puzzlerSo you mean I should write my own more verbose version of the defn-args spec rather than use the one built into core?#2016-10-0113:31Alex Miller (Clojure team)well you could wrap defn-args too#2016-10-0113:33Alex Miller (Clojure team)part of being in alpha means things arenā€™t done#2016-10-0113:34puzzlerThe problematic piece is :arg-list, which is a component of the :defn-args spec:
(s/def ::arg-list
  (s/and
    vector?
    (s/cat :args (s/* ::binding-form)
:varargs (s/? (s/cat :amp #{'&} :form ::binding-form))))
#2016-10-0113:34puzzlerSo could I just "patch" arg-list?#2016-10-0113:37puzzlerI guess if I s/def :clojure.core.specs/arg-list in my library, that will overwrite the spec for other consumers of my library as well, so maybe I shouldn't do that.#2016-10-0113:55Alex Miller (Clojure team)maybe you could make the patching a separate (temporary) step#2016-10-0115:27decoursin@alexmiller Thanks for having a look. It's working now. I'm not sure, if it comes up again, I'll let you know#2016-10-0120:12jrheardiā€™m having a lot of trouble getting specā€™s generative testing to work#2016-10-0120:12jrheardwhen i call (stest/check `foo/bar), and the output is [], how should i interpret that empty list?#2016-10-0120:14jrheardi get [] from calling (stest/check) on each fdefā€™d function in this file, eg https://github.com/jrheard/voke/blob/spec/src/voke/input.cljs#L85 , and donā€™t really know where to go from here. has anyone seen this behavior? is there something obvious that iā€™m doing wrong?#2016-10-0120:15jrheardexample code:#2016-10-0120:16jrheard
(deftest generative
  (let [output (stest/check `input/intended-directions->angle)]
    (print "yo")
    (print output)
    (print (stest/summarize-results output))))

;; lein doo phantom test output:

Testing voke.input-test
yo
[]
{:total 0}
#2016-10-0120:19jrheardrelevant specs are in https://github.com/jrheard/voke/blob/spec/src/voke/specs.cljs#2016-10-0122:52lvhHm, going from -alpha11 to -alpha12/-alpha13 I get a pretty unhelpful NPE when calling explain on some data. clojure.spec portion of the stacktrace:
spec.clj:  864  clojure.spec/spec-impl/reify
                  spec.clj:  150  clojure.spec/conform
                  spec.clj:  782  clojure.spec/map-spec-impl/reify
                  spec.clj:  150  clojure.spec/conform
                  spec.clj:  731  clojure.spec/dt
                  spec.clj:  727  clojure.spec/dt
                  spec.clj: 1081  clojure.spec/explain-pred-list
                  spec.clj: 1118  clojure.spec/and-spec-impl/reify
                  spec.clj:  198  clojure.spec/explain-data*
                  spec.clj:  209  clojure.spec/explain-data
                  spec.clj:  202  clojure.spec/explain-data
#2016-10-0122:52lvhI dunno if anyoneā€™s seen anything like that or if itā€™s worth producing a minimal sample#2016-10-0122:53lvhwhich suggests that pred in spec-impl is nil#2016-10-0122:59lvhDid anything change about how registry-ref works?#2016-10-0123:04lvh(The weirdest thing that code path does is (with-redefs [s/registry-ref (atom @@#'s/registry-ref)] ā€¦), which it needs to because itā€™s generating specs so I donā€™t want to have tests fail because of stale state)#2016-10-0123:19lvhThat does not look like itā€™s part of the problem.
#2016-10-0123:36agI am dealing with strangest bug I cannot reproduce anywhere else: it works everywhere except Circle CI. Basically this spec:
(require '[com.gfredericks.test.chuck.generators :as gen']
         '[clojure.spec :as s])

(def iso-date-regex #"^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}-\d{2}:\d{2}$")

(s/def ::date-iso (s/with-gen (s/and #(re-find iso-date-regex %) string?)
                   #(gen'/for [d (gen'/datetime)]
                      (.format (js/moment (js/Date. d))))))
It works everywhere but on CI it says:
ERROR in (test-datetime-string) (:)
Uncaught exception, not in assertion.
expected: nil
  actual: #error {:message "Couldn't satisfy such-that predicate after 100 tries.", :data {}}
#2016-10-0123:38agfirst I thought itā€™s due to incompatible version of phantomjs. and then version of Leineingen, and then I disabled any parallelism on CI. and still canā€™t figure it out. I thought maybe I could replicate in a Docker container - still nothing#2016-10-0123:39agAnyone, any ideas?#2016-10-0123:39lvhhm, thatā€™s weird: it would only make sense if the generator portion is being ignored#2016-10-0123:40agbut why would that happen only in Circle CI and nowhere else?#2016-10-0123:40lvhmaybe you should create a test PR that s/gen + gen/samples that spec#2016-10-0123:40lvhthat could be a few reasons, including profile.clj files or whatever#2016-10-0123:40lvhs/gen + gen/samples will tell you what it thinks the generator is for that spec#2016-10-0123:41lvhhow did you determine it wasnā€™t an incompatible version of phantomjs?#2016-10-0123:42agI am replacing phantom with the right version#2016-10-0123:43gfredericksLEIN_NO_USER_PROFILES=1 can be helpful for debugging things that only happen in ci#2016-10-0123:45lvhIā€™ve seen a lot of problems with Phantom where phantom versions insisting they were the same, although that regex does look innocuous#2016-10-0123:45lvh1.9.8 is like seventeen different webkits#2016-10-0123:45gfredericks Also have you rerun the ci? That "such-that" error could also just be highly improbable#2016-10-0123:46lvhWouldnā€™t the such-that error never happen with with-gen? I thought that with-gen did not check that generated values match the spec?#2016-10-0123:48lvh(that may have changed in recent alphas, or not be true in JS, or I might be misremembering entirely of course)#2016-10-0123:49gfredericksI don't know where the such-that came from actually#2016-10-0123:50lvhwell, Iā€™m thinking of the spec by itself#2016-10-0123:50lvh(s/and #(re-find iso-date-regex %) string?)#2016-10-0123:50lvhIIRC s/and uses the first arg to determine type#2016-10-0123:50lvhso it wonā€™t even be able to guess ā€œstringā€ let alone ā€œstring that incidentally matches this regex"#2016-10-0123:51lvh(Sorry, to be less obtuse: I think the such-that is coming from the s/and)#2016-10-0123:52gfredericksOh I didn't see an and#2016-10-0123:53gfredericksIt shouldn't matter though since it's wrapped in with-gen#2016-10-0123:55lvhexactly#2016-10-0123:56lvh@ag: Can you please dump the full test ns somewhere#2016-10-0123:56lvhideally a reproducible sample#2016-10-0200:02jrheard@gfredericks - have you ever seen s/check output the empty list like i mentioned above? sorry to bug you directly, iā€™ve just seen your name around test.check-related things, figured you might have an answer off the top of your head šŸ™‚#2016-10-0200:02jrheardiā€™m a complete newbie to generative testing, so for all i know this is normal well-understood behavior, but iā€™m having trouble making heads or tails of it#2016-10-0200:03gfredericksI expect it's a clojure.spec-related thing#2016-10-0200:04gfredericksI've seen that happen when I've used it but can't remember why#2016-10-0200:04jrheardcool, iā€™ll keep poking at it#2016-10-0218:36lvhI found my problem from yesterday: between -11 and -12, a nil spec changed meaning#2016-10-0218:36lvhostensibly from meaning any? to silent failure#2016-10-0218:37lvhI appreciate that there might be a bootstrapping failure, but that seems like it could be found at spec definition time and not with an NPE thrown from a reify#2016-10-0218:43bfabrygenerated values are still checked against the spec, I'm guessing there's something in the environment causing .format to return a differently formatted date#2016-10-0218:52lvhoh, huh; I wonder if thatā€™s now or Iā€™m just horribly misremembering#2016-10-0218:52lvh
(s/def ::a integer?)
(s/def ::b (s/with-gen string? #(s/gen ::a)))
(sg/sample (s/gen ::a))
(sg/sample (s/gen ::b))
.. confirms
#2016-10-0218:52lvhI guess .format isnā€™t awfully specific#2016-10-0218:53bfabrythere's probably some dumb version of the JS engine that returns "Monday the 2nd of Feburary 10:13PM Australian eastern daylight savings time"#2016-10-0218:57lvhright#2016-10-0218:57lvhMoment.js takes explicit format strings though, so hopefully easy to recover from#2016-10-0218:57lvh@ag ^#2016-10-0223:11Alex Miller (Clojure team)@lvh There was a bug introduced with conforming nilable in 12 that is fixed in 13#2016-10-0223:13lvh@alexmiller In -alpha11, a nil spec was effectively any? afaict#2016-10-0223:13lvhthatā€™s gone in -12 and -13, which broke my tests; but really it was just exposing something that was a bug anyway#2016-10-0223:15lvhthis is not a problem unless youā€™re generating specs, which I guess is the part of clojure.spec Iā€™m exercising a little more that most folks šŸ˜„#2016-10-0223:15lvh(my generated specs included nils by accident; that was actually a bug)#2016-10-0223:54jrheard@alexmiller - have you seen situations where (stest/check `foo/bar) returns []? what type of mistake does that return value usually indicate? iā€™m having a hard time figuring out what causes it - it seems to return [] sometimes and eg [{:spec #object[cljs.spec.t_cljs$spec8685], :clojure.test.check/ret {:result true, :num-tests 1000, :seed 1475452376922}, :sym voke.input/intended-directions->angle}] other times, the behavior varying from run to run when none of the code has changed#2016-10-0300:10gfredericksthat sounds pretty weird#2016-10-0301:27jrheardi think tomorrow iā€™m just gonna have to learn how to use checkout dependencies and start adding printfs to spec.test code so i can see whatā€™s going on in there#2016-10-0302:21bhagany@jrheard fwiw, Iā€™m eager to hear what you learn. I tried getting doo to run my cljs spec tests a few weeks ago, and hit a wall.#2016-10-0304:02jrheardwell, for starters i think iā€™ve found a small issue with the cljs.spec documentation#2016-10-0304:03jrheardhttps://github.com/clojure/clojurescript/blob/master/src/main/cljs/cljs/spec/test.cljc#L210 says that options maps passed to (s/check) should look like {:clojure.spec.test.check/opts {:num-tests 100}}#2016-10-0304:03jrheardbut actually, they should look like {:clojure.test.check/opts {:num-tests 100}}#2016-10-0304:36jrheardi havenā€™t managed to reproduce this [] return value issue using my local checkout of clojurescript, so maybe this behavior has been fixed on master? will poke at it some more in the morning#2016-10-0304:55seancorfieldAre you sure about the options map @jrheard ? When I was trying that on Clojure, it definitely needed to be :clojure.spec.test.check/opts (even thoā€™ no such namespace exists).#2016-10-0304:56seancorfieldIf it really is the latter, as you say, then thatā€™s a bug in my opinion ā€” ClojureScript should follow Clojure there I think?#2016-10-0306:56decoursinHow to override keys in (s/merge ..)? My failing attempt:
(s/def :my-ns/a string?)
(s/def :my-other-ns/a int?)
(gen/generate (s/gen (s/merge (s/keys :req-un [:my-ns/a])
                              (s/keys :req-un [:my-other-ns/a]))))
=> ExceptionInfo Couldn't satisfy such-that predicate after 100 tries.  clojure.core/ex-info (core.clj:4725)
#2016-10-0307:18misha@decoursin: what do you expect merged spec to be?#2016-10-0307:20misha"Int or string"?#2016-10-0307:23decoursinEither one, make a choice. Always take the first or always take the second.#2016-10-0307:25decoursinprobably the same way clojure.core/merge works which is the second (or last one.)#2016-10-0307:25decoursin
(merge {:a 3} {:a 4})
=> {:a 4}
#2016-10-0307:27misha
Takes map-validating specs (e.g. 'keys' specs) and
returns a spec that returns a conformed map satisfying all of the
specs.  Unlike 'and', merge can generate maps satisfying the
union of the predicates.
does not sound like it can satisfy int? and string? with a same value on a single key
#2016-10-0307:29bfabryI think in terms of usefulness the way it's behaving atm is probably best, though I could see an argument for it behaving like core.merge#2016-10-0307:30decoursinYeah I think it should behave like core.merge. I'd rather see that what @misha described to be called like union or something#2016-10-0308:22decoursinHow could I do this then? Have one key override another#2016-10-0308:42mishawhat is your use case? or rather "problem"#2016-10-0311:31mpenetis there a way to validate against a (dynamic) subset of keys in a map? ex I define an exhaustive spec for a map (with mixed req/opt), and at runtime I want to be able to validate against a subset of it#2016-10-0311:32mpeneta bit in the flavor of s/merge, I'd like s/select-keys (kind of)#2016-10-0311:33mishayou can give names to those subsets, and just merge those in uber-map-spec. no?#2016-10-0311:34mpenetI dont want to define as many subsets are they are key combos no#2016-10-0311:34mpenetex with Schema you can just do (since map specs are just Maps) :
(defn validate-subset
  [schema value]
  (-> (select-keys schema (keys value))
      (s/validate value)))
#2016-10-0311:35mpenetseems like s/select-keys could be a nice addition actually, thoughts @alexmiller ?#2016-10-0311:38mpenetjust putting every key as optional is not an option either#2016-10-0311:42mishagiven spec's "global" nature, I'd go and name all the combos. also multi-spec might be a good fit too#2016-10-0311:42mpenetthat's a lot of combos for a 20 key map for instance ... it'd be horrible#2016-10-0311:43mpenetI'll check multi-spec, but my gut feeling it doesn't fit that problem either#2016-10-0311:43mishadepending on how you'd define combos out of those 20#2016-10-0311:44mishayou might have only 2-3 "domain-valid" combos.#2016-10-0311:44mpenetdoesn't matter#2016-10-0311:44mpenetin my case the user can update any of them and I cannot pass all of it#2016-10-0311:45mpenet(there are concurrent update issues for the "pass the whole thing" case)#2016-10-0311:45mishanot enough information then#2016-10-0311:46mpenetit's a common enough problem imho, any REST api would hit it for partial update for instance#2016-10-0311:47mpenetthe least horrible (yet it sucks) imho would be to call valid? against every key in the set but yuk. Or just have a s/keys spec with all as optional for updates, but that's a lot of duplication I'd like to avoid#2016-10-0311:47mishathen probably I'd have 20 different specs defined, and constructed map spec on demand based on incoming map's keys#2016-10-0311:48misha20 for possible keys, + maybe some for key combinations, like "if that one present, those 2 are required".#2016-10-0311:51mishaa collection of specs for possible keys should not necessarily be a ready to use map spec.#2016-10-0311:51mpenetnot going to do that, that's truly awful, not to mention for 20 fields (the example here but some of our records have more than this) that's a large number of specs (a lot more than 20)#2016-10-0311:52mishanot everything can be solved with oneliners kappa#2016-10-0311:52mpenetit's not the point, you don't have a solution to this, no biggie#2016-10-0311:53mpenetAt the moment there's no elegant way to solve this with spec.#2016-10-0312:29misha@mpenet how about this?
(s/def :foo/bar int?)
(s/def :foo/baz string?)

(def schema #{:foo/bar :foo/baz})

(defn validate-subset
  [schema m]
  (let [ks (filterv schema (keys m))]
    (s/valid? (s/keys :req `[
(validate-subset schema {:foo/bar 1 :foo/baz "y"})
=> true
(validate-subset schema {:foo/baz "y"})
=> true
(validate-subset schema {:foo/baz 1})
=> false
#2016-10-0312:35mpenetThat's more or less what I mentioned earlier, but I dont like it. Having 1 separate spec with all opt keys is probably nicer#2016-10-0313:46danielstocktonIs there a way to instrument everything at once, rather than individual functions?#2016-10-0313:47Alex Miller (Clojure team)Just call instrument with no args#2016-10-0313:52danielstocktonIs there a good way to hook this into the test runner? I imagine it can be quite helpful to know not only what tests failed but which functions got unexpected data.#2016-10-0313:55mpenetI think you can just call instrument at top level of your test#2016-10-0313:55danielstocktonI suppose, i'd have to do it in every namespace#2016-10-0313:56danielstocktonMight also be useful to turn it on whenever starting lein repl or in a dev environment#2016-10-0314:09mpenet@alexmiller (s/valid? #{false} false) => false that's what you mention on github?#2016-10-0314:09mpenetit does look like a bug from the outside (without considering/knowing how it's implemented under the hood)#2016-10-0314:14mpenetI guess its (#{false true} false)#2016-10-0314:14Alex Miller (Clojure team)Yes that's what I meant. It's not a bug, just a consequence of using sets as specs#2016-10-0314:14mpenetok#2016-10-0314:14Alex Miller (Clojure team)Sets with falsey values won't work #2016-10-0314:15Alex Miller (Clojure team)So don't do that#2016-10-0314:15Alex Miller (Clojure team)You've used a function that returns false for what you consider to be a valid value #2016-10-0314:16mpenetyep, got it#2016-10-0314:26mpenetactually about my s/select-keys proposal earlier, why not making s/keys spec an c.l.Associative instance and allow us to compose this stuff with normal core fn?#2016-10-0314:27mpenetmaybe it's crazy talk#2016-10-0315:35Alex Miller (Clojure team)the problem with that is that we capture the key set for describe and form, so you would lose that ability#2016-10-0315:36Alex Miller (Clojure team)thatā€™s the reason itā€™s a macro and not a function that takes data now#2016-10-0315:42Alex Miller (Clojure team)@jrheard no, havenā€™t seen that#2016-10-0315:59mlimottehi. I'm just starting to play with clojure.spec. I'm ok with some basic specs and validation that I've tried. But having trouble with even a trivial example of specing a higher-order fn. Here's what I'm seeing:
(def my-spec (s/fspec :ret string?))
=> #'user/my-spec
(s/conform my-spec (fn [j] (str j)))
IllegalArgumentException No implementation of method: :specize* of protocol: #'clojure.spec/Specize found for class: nil  clojure.core/-cache-protocol-fn (core_deftype.clj:568)
#2016-10-0316:04Alex Miller (Clojure team)(s/fdef my-spec :ret string?) will be better for you right now#2016-10-0316:05Alex Miller (Clojure team)sorry, I misread that first def as s/def, let me read again#2016-10-0316:06Alex Miller (Clojure team)so the issue here is that to verify that the function youā€™ve passed is valid, it will generate args based on the :args spec for the function and invoke it#2016-10-0316:06Alex Miller (Clojure team)but youā€™ve passed no args spec#2016-10-0316:08mlimottetrue. I was going with a simple example -- just validating that it is a fn that returns a string. I did try with args before, and was getting different errors, so I was trying to simplify the example.#2016-10-0316:08mlimottei'll try w/ args again#2016-10-0316:08Alex Miller (Clojure team)although I am seeing some weird stuff on this path#2016-10-0316:09mlimotte(s/def my-spec (s/fspec :args (s/tuple integer?) :ret string?)) => user/my-spec (s/conform my-spec (fn [j] (str j))) IllegalArgumentException No implementation of method: :specize* of protocol: #'clojure.spec/Specize found for class: nil clojure.core/-cache-protocol-fn (core_deftype.clj:568)#2016-10-0316:09mlimottešŸ˜ž same thing, even with args.#2016-10-0316:14Alex Miller (Clojure team)ah, so conform takes a qualified keyword or symbol#2016-10-0316:14Alex Miller (Clojure team)
(s/conform `my-spec (fn [j] (str j)))
#2016-10-0316:14Alex Miller (Clojure team)will fully-qualify my-spec#2016-10-0316:14jrheard@seancorfield yeah, very sure- compare clojureā€™s https://github.com/clojure/clojure/blob/master/src/clj/clojure/spec/test.clj#L19 vs cljsā€™s https://github.com/clojure/clojurescript/blob/master/src/main/cljs/cljs/spec/test.cljs#L19 [and also iā€™ve verified interactively]#2016-10-0316:15Alex Miller (Clojure team)@jrheard the idea in both of those is to pass a map of options through to test.check - I think the underlying option keys differ in clj vs cljs test.check#2016-10-0316:16Alex Miller (Clojure team)so there may be a disconnect between clj vs cljs and docs here#2016-10-0316:16jrheardyeah, i think the cljs docs need to be updated to reflect the disconnect - https://github.com/clojure/clojurescript/blob/master/src/main/cljs/cljs/spec/test.cljc#L211#2016-10-0316:16jrheardi was using :clojure.spec.test.check/opts as the docs recommend, but nothing was happening#2016-10-0316:16Alex Miller (Clojure team)I suspect so - prob should file a jira (and if you like a patch!)#2016-10-0316:16mlimotteahh! it does say that in the spec guide! Maybe it would be helpful for the conform and valid? docstring(s) could also highlight that requirement.#2016-10-0316:16jrheardsounds good, itā€™ll be my first of both of those šŸ™‚ time to go read the contributing guidelines again#2016-10-0316:17Alex Miller (Clojure team)@mlimotte seems like there should be a better error message in this case#2016-10-0316:17Alex Miller (Clojure team)I will log that and add a patc hfor it#2016-10-0316:19mlimotteyep. that would help. here's another question. What's a recommended way to spec the requirement that the fn should be a one-arg fn? I tried:
(s/def my-spec (s/fspec :args (s/tuple identity) :ret string?))
(s/def my-spec (s/fspec :args #(= (count %) 1) :ret string?))
#2016-10-0316:20mlimottewhich results in
ExceptionInfo Unable to construct gen at: [0] for: identity  clojure.core/ex-info (core.clj:4617)
ExceptionInfo Unable to construct gen at: [] for: 
#2016-10-0316:21Alex Miller (Clojure team)(s/def my-spec (s/fspec :args (s/cat :j any?) :ret string?)#2016-10-0316:21jrheard(iā€™ve made a jira account, username jrheard - alex, are you the right person to bug to add whatever necessary permissions to that account, or should i go bother someone else?)#2016-10-0316:22Alex Miller (Clojure team)you can add tickets with the starting permissions#2016-10-0316:22jrheardawesome#2016-10-0316:24mlimottewhat is any?, AFAICT that is not in clojure.core nor clojure.spec#2016-10-0316:24Alex Miller (Clojure team)itā€™s in core#2016-10-0316:24lvhItā€™s new in Clojure#2016-10-0316:24Alex Miller (Clojure team)itā€™s just (constantly true)#2016-10-0316:25lvhhm; I figured it might take only 1 arg#2016-10-0316:25Alex Miller (Clojure team)actually it does#2016-10-0316:25Alex Miller (Clojure team)I was more sketching than saying the actual def :)#2016-10-0316:25Alex Miller (Clojure team)
(defn any?
  "Returns true given any argument."
  {:tag Boolean
   :added "1.9"}
  [x] true)
#2016-10-0316:26lvhah, gotcha šŸ˜„#2016-10-0316:27mlimottei see. I'm using clojure-future-spec w/ clojure 1.8, so this works: (s/def my-spec (s/fspec :args (s/cat :j clojure.future/any?) :ret string?))#2016-10-0316:28Alex Miller (Clojure team)ah#2016-10-0316:28mlimotteas does: (s/def my-spec (s/fspec :args (s/tuple clojure.future/any?) :ret string?)) Was your choice of s/cat over s/tuple just stylistic, or is there something more significant?#2016-10-0316:28Alex Miller (Clojure team)@jrheard I added edit permissions for you as well in jira#2016-10-0316:29jrheardthanks!#2016-10-0316:31jrheardopened CLJS-1808 , will submit a patch later today#2016-10-0316:31Alex Miller (Clojure team)thanks#2016-10-0316:32Alex Miller (Clojure team)@mlimotte tuple is fine too. we usually use regex ops to describe args. Args are effectively syntax and thatā€™s what regex ops in spec are the best match for.#2016-10-0316:33Alex Miller (Clojure team)one benefit of cat is that you will get named parts for conforming the parts whereas you will get just indexes for tuple#2016-10-0316:34Alex Miller (Clojure team)so that affects both the conform result (which youā€™ll see in the :fn spec) and the explain errors#2016-10-0316:34Alex Miller (Clojure team)generally I find being able to tie the spec to the arg name helpful#2016-10-0316:35mlimottegot it. thanks for the help.#2016-10-0317:32Alex Miller (Clojure team)@mlimotte fyi, http://dev.clojure.org/jira/browse/CLJ-2032#2016-10-0317:32mlimottethx#2016-10-0317:43jrheard@alexmiller - i just added a couple patches to the ticket. is there anything else i need to do, or should i just wait and expect to get a response at some point in the future? iā€™m asking you this not because iā€™m in an incredible hurry to have someone immediately look at this jira issue this very second; i just want to make sure iā€™m not missing some ā€œnotify the maintainersā€ step before i close all these tabs šŸ™‚ thanks!#2016-10-0317:47Alex Miller (Clojure team)@dnolen: ^^#2016-10-0317:48Alex Miller (Clojure team)For cljs-1808#2016-10-0317:48mlimotteCould there be a problem with s/tuple? I have the code below, which bombs w/ the error below, but if I change the '(s/tuple integer?)` to (s/cat :a integer?) than it works.
(defn myfn [a] (str a))
(s/fdef myfn :args (s/tuple integer?) :ret string?)
(stest/instrument `myfn)
(myfn 1)
I get
=> #'user/myfn
=> user/myfn
=> [user/myfn]
ExceptionInfo Call to #'user/myfn did not conform to spec:
val: (1) fails at: [:args] predicate: vector?
:clojure.spec/args  (1)
:clojure.spec/failure  :instrument
:clojure.spec.test/caller  {:file "form-init1342783443404146057.clj", :line 4, :var-scope user/eval30918}
  clojure.core/ex-info (core.clj:4617)
#2016-10-0317:50bfabry@mlimotte s/tuple specifically expects its input to be a vector, the :args list is not a vector#2016-10-0317:51bfabrys/cat just validates a sequence, which the :args list is#2016-10-0317:53mlimottei see. in retrospect, kind of obvious from the error message. i should have gotten that.#2016-10-0317:55dnolen@jrheard just assign it to me I will look at it when I have time#2016-10-0317:55jrheardgreat, will do, thanks!#2016-10-0321:15tokenshiftCan s/conform handle coercion? For example, could I have an s/def that defines a ::uuid to be either a (valid) string or a java.util.UUID, and always conforms it to a java.util.UUID?#2016-10-0321:28lvhtokenshift Check out ā€œconformer"#2016-10-0321:29tokenshiftIā€™d pass a conformer (the result of s/conformer) to s/def, e.g. (s/def ::foo (s/conformer #(do-stuff %)))?#2016-10-0321:31tokenshiftAnswered my own question; yep, that works#2016-10-0321:31tokenshiftThanks!#2016-10-0321:33tokenshiftNow I have to decide if thatā€™s a good pattern, or if itā€™d be better to keep input coercion completely separate from specs#2016-10-0321:38lvhI think itā€™s fine to conform to a specific object#2016-10-0321:38lvhconforming is not something that you want in your hot loop though IIUC; itā€™s a measurable performance impac#2016-10-0321:38lvht#2016-10-0321:39lvhbut at the edges? sure; youā€™re gonna do that anyway, might as well have it be declarative#2016-10-0321:39tokenshiftHmm, thatā€™s a good point#2016-10-0321:39tokenshiftIf itā€™s only at the edges, then I can assume the input will be a string (for a web service, at least)#2016-10-0321:40tokenshiftThough if the backend requires a UUID, Iā€™d probably still want to coerce it as early as possible.#2016-10-0321:40tokenshiftI guess if the coercion is lossless, itā€™s safe; but if itā€™s lossy, Iā€™d want to have an explicit coercion step.#2016-10-0321:48tokenshiftHeading out#2016-10-0321:48tokenshiftThanks for the help!#2016-10-0400:10jrheardheh, enabling (s/instrument) on all instrumentable vars does not do wonders for my gameā€™s performance#2016-10-0400:10jrheardinvaluable for finding the 7 random spots where iā€™d goofed when converting to namespaced keywords, though šŸ™‚#2016-10-0400:11jrheardiā€™m not complaining, i absolutely expected this degree of perf degradation when instrumenting like every function in this tight loop, it was just funny to go from 60fps to 0.5 šŸ˜„#2016-10-0400:15jrheardinstrumenting only at the edges brings me back up to good perf, nice#2016-10-0400:21agguys, you remember me struggling with this: https://clojurians.slack.com/archives/clojure-spec/p1475364998003541#2016-10-0400:22jrheardyeah!#2016-10-0400:22agso it turned out that my regex is bad#2016-10-0400:22agit does not take into account timezones#2016-10-0400:22agthat minus sign sometimes is a plus sign#2016-10-0400:23agalthough I think it would be helpful if spec told me why exactly it couldnā€™t satisfy the predicate#2016-10-0400:25jrheardi worked with a guy once whose most famous quote around the office was: <jlatt> dates suck <jlatt> f&@# time#2016-10-0400:25jrheardiā€™ve had similar feelings when running into the ā€œCouldnā€™t satisfy such-that predicate after 100 triesā€ error - i wish there was some way it could tell me how iā€™ve messed up#2016-10-0400:25agalso I think test.chuck generator does not generate values with random timezones. maybe it should#2016-10-0400:25jrheardiā€™m not sure how it could do that though#2016-10-0413:07minimal@danielstockton :args (s/cat :arg1 (s/and ::query (s/+ (s/or :db map? :coll vector?)))#2016-10-0413:20danielstocktonHas this changed? http://clojure.org/guides/spec#_spec_ing_functions#2016-10-0413:20danielstocktonArgh, I think I want s/cat instead of s/and actually#2016-10-0413:21danielstocktonnow i see that s/and is adding two specs#2016-10-0413:21danielstocktonyep, thanks @minimal, somehow i was blind to that#2016-10-0422:23seancorfieldWe officially have clojure.spec-based code in production now ā€” Clojure 1.9.0 Alpha 13 and all of our REST API search code relies on spec for validation and conformation!#2016-10-0423:08jcsims@seancorfield any plan to write about it?#2016-10-0423:35seancorfield@jcsims I submitted a proposal to Clojure/conj about itā€¦ so if that gets accepted, Iā€™ll have to write about it. If it doesnā€™t get accepted, Iā€™ll probably still write it up for my blog, but in a very different format I expect.#2016-10-0423:36seancorfieldPart of my concern is that core to a lot of what weā€™re doing with spec is something that @alexmiller said is essentially an anti-pattern (heavy use of s/conformer).#2016-10-0423:36seancorfieldThat, and we are still at times struggling mightily with how to apply spec to some situations...#2016-10-0423:36jcsimswell, looking forward to it either way :+1:#2016-10-0507:02mpenetIs there a workaround to http://dev.clojure.org/jira/browse/CLJ-2033 for un-namespaced keys ?#2016-10-0507:02mpenetI just hit this too#2016-10-0507:04mpenetit makes s/merge a bit useless tbh, not sure why it's not considered a bug#2016-10-0507:05mpenetI guess a workaround would be to do the composition at a higher level, and not rely on spec/merge for merging specs like this, which is... odd#2016-10-0507:14mpenetand the fact that it's macros all the way down makes this workaround also a bit hairy#2016-10-0508:43jmglovIs it possible to spec protocols in any way?#2016-10-0508:43jmglovOr objects that satisfy them?#2016-10-0511:08mpenetI think it's encouraged to wrap protocol fn invocation, so you can instrument/spec the wrapper fn. otherwise if you mean spec predicate it's just #(satisfies? %), possibly with (s/spec pred :gen ... ) if you want to have the gen part#2016-10-0511:43odinodinAnyone used spec with DataScript? Need tips on how to deal with the fact that entities are not maps, and specifically how to get the same functionality as clojure.spec/keys but for Datascript entities#2016-10-0511:53misha@odinodin what do you mean by "not maps"? pull pretty much returns maps.#2016-10-0511:54mishawhat exactly do you want to cover with spec?#2016-10-0511:54odinodinI have a function that takes DataScript entities#2016-10-0511:55odinodinI'd rather avoid creating maps from entities, and just use entities directly#2016-10-0511:56mishaoh, you mean thing returned by d/entity? no idea : )#2016-10-0511:57odinodinšŸ™‚#2016-10-0512:29Yehonathan SharvitIs it possible to define a spec that references itself in the definition - like in Context-Free Grammars?#2016-10-0512:30Alex Miller (Clojure team)Sure, via it's registered keyword#2016-10-0512:30Alex Miller (Clojure team)Recursive and mutually recursive specs are fine#2016-10-0512:31Yehonathan SharvitCould you share an example?#2016-10-0512:32Alex Miller (Clojure team)There are several in clojure.core.specs - destructuring is recursive#2016-10-0512:33Alex Miller (Clojure team)Some of the ns stuff with prefix lists , etc#2016-10-0512:36Alex Miller (Clojure team)::prefix-list there is self-recursive#2016-10-0512:37Alex Miller (Clojure team)::binding-form can be ::seq-binding-form or ::map-binding-form, which can both include ::binding-form#2016-10-0512:38Yehonathan Sharvitthx#2016-10-0512:38Yehonathan SharvitIā€™m looking at it#2016-10-0512:40Alex Miller (Clojure team)A really good example to try is just a simple tree of leaf and branch#2016-10-0512:41Alex Miller (Clojure team)Where branches can contain either leaf or branch#2016-10-0513:49mlimotteWhat's the best way to trigger clojure.spec tests with lein test? I.e. I have a namesapce with (stest/check (stest/enumerate-namespace 'wwai.common.util.instant)), I want it run those checks when I do lein test. I'm really using expectations, but if I see how to make it work with standard test, I should be able to adapt from their.#2016-10-0516:48ag@mlimotte clojure.test.checkā€™s defscpec worked for me#2016-10-0516:52jrheard@ag iā€™d love to see an example of that on github if you have one, np if not, just figured iā€™d check šŸ™‚#2016-10-0518:08mlimotteI came up with this hack, which kind of works, although the output in the error case, isn't very friendly:
(defmacro is-result-ok?
  [result]
  `(is (not (:failure (first ~result)))))

(deftest auto
  (doseq [spec-fn-sym (stest/enumerate-namespace 'wwai.common.util.instant)]
    (is-result-ok? (stest/check spec-fn-sym))))
#2016-10-0518:09mlimotteI looked at defspec, but didn't see how to make it work with stest/check.#2016-10-0518:20jrheardme neither!#2016-10-0518:51luxbockwhat's the idiomatic way to express lack of arguments for a function with fdef?#2016-10-0518:52luxbockjust empty? should do I guess#2016-10-0518:53bfabry@luxbock (s/cat)#2016-10-0518:54luxbockwhat's the benefit over just using empty?#2016-10-0518:55bfabrynothing afaik, it's just that s/cat is the normal way to start specifying :args#2016-10-0518:56luxbockalright#2016-10-0519:07luxbockI would've expected this to throw: https://gist.github.com/luxbock/532a19f75553ae938c0a998e3be47851#2016-10-0519:11bfabryinstrument does not check return values#2016-10-0519:13luxbockI see#2016-10-0519:15bfabryyou're probably looking for clojure.spec.test/check#2016-10-0519:16bfabryit runs the function a bunch of times with generated args (in this case always nothing) and checks the result conforms to :ret and that any supplied :fn relationship between :args and :ret is true#2016-10-0519:17bfabry
boot.user=> (s/def ::number int?)
:boot.user/number
boot.user=> (s/def ::number-list (s/coll-of ::number :min-count 1))
:boot.user/number-list
boot.user=> (s/fdef foobar
       #_=>         :args (s/cat)
       #_=>         :ret ::number-list)
boot.user/foobar
boot.user=> (defn foobar [] [])
#'boot.user/foobar
boot.user=> (clojure.spec.test/check `foobar)
({:spec #object[clojure.spec$fspec_impl$reify__13891 0x222df61e "
#2016-10-0519:20luxbockthanks, yeah the function which I simplified here probably won't need a spec at all, but it's good to know this for the future#2016-10-0520:02jasonjcknhow do I specify that ::start_date must start before ::end_date in a map#2016-10-0520:02jasonjcknso far I have (s/keys :req-un [::start-date ::end-date]) but I need to add in a predicate#2016-10-0520:02jasonjcknthe relationship between these two#2016-10-0520:07jrheardperhaps you could write a constructor function rather than making instances of that map by hand, and itā€™s got an (s/fdef) that encodes that information?#2016-10-0520:08jrhearddoesnā€™t 100% solve the problem though#2016-10-0520:09mlimottemaybe you could use s/and and a predicate, something like: (s/and (s/keys :req-un [::start-date ::end-date]) #( date-before? (:start-date %) (:end-date %)))#2016-10-0520:09jrheardoh interesting#2016-10-0520:09jrheardi like that better#2016-10-0520:10jasonjckncool thanks, testing now#2016-10-0520:38luxbockwhat is the use case of the optional argument unf to conformer?#2016-10-0520:40bfabry@luxbock I'm guessing it's for a custom unformer#2016-10-0520:42luxbockah right, yeah I get it now#2016-10-0520:44bfabry
boot.user=> (s/def ::thing (s/conformer {1 "foo"}))
:boot.user/thing
boot.user=> (s/unform ::thing (s/conform ::thing 1))

java.lang.IllegalStateException: no unform fn for conformer
boot.user=> (s/def ::thing (s/conformer {1 "foo"} {"foo" 1}))
:boot.user/thing
boot.user=> (s/unform ::thing (s/conform ::thing 1))
1
#2016-10-0521:47jasonjcknis there a way to include doc string with the spec?#2016-10-0521:47jasonjckne.g. (s/and (s/keys :req-un [::start-date ::end-date]) "start date must come before end date" #( date-before? (:start-date %) (:end-date %)))#2016-10-0521:48jasonjckn(and retrieve the doc string in explain-data)#2016-10-0522:05jrheardhas anyone else had trouble doing eg
(stest/instrument (stest/enumerate-namespace ā€˜my.ns))
in cljs?
#2016-10-0522:06jrheardi see a stacktrace with error messages like java.lang.RuntimeException: No such namespace: stest, compiling:(/private/var/folders/zl/bh7pbyz95rg7pmcvcc_f_kdm0000gn/T/form-init2230077429322923281.clj:6:19); gonna dig into it a bit, just curious if this is known / expected / if anyone else has dealt with this#2016-10-0522:17jrheardif i do eg
(let [syms (stest/enumerate-namespace 'voke.events)]
    (stest/instrument syms))
then i get
Caused by: clojure.lang.ExceptionInfo: java.lang.RuntimeException: Unable to resolve symbol: syms in this context
feels like i must be doing something wrong
#2016-10-0522:18jrheardperhaps related to the (eval) call in https://github.com/clojure/clojurescript/blob/master/src/main/cljs/cljs/spec/test.cljc#L110 ?#2016-10-0522:22jrheard@dnolen am i doing something wrong / is this known/expected?#2016-10-0522:23dnolen@jrheard yes wonā€™t work, not intended to work#2016-10-0522:23jrheardcool, good to know#2016-10-0522:23jrheardshould i just instrument vars by hand, naming one at a time, or is there some better way?#2016-10-0522:23dnoleninstrument is a macro - must take symbols that reference vars directly#2016-10-0522:23dnolenjust skip the locals stuff#2016-10-0522:23jrheardi donā€™t follow#2016-10-0522:24dnolenremove the let binding and it will work#2016-10-0522:24dnoleni.e. do this inline#2016-10-0522:24jrheardwhen i do (stest/instrument (stest/enumerate-namespace 'my.ns)), i get an error message complaining that stest doesnā€™t exist#2016-10-0522:24jrheardis that what you mean by doing this inline, or am i misunderstanding?#2016-10-0522:24dnolenso thatā€™s a bug, file an issue in JIRA#2016-10-0522:24jrheardgotcha, will do! thanks!#2016-10-0522:25dnolenthanks, to be clear, do this in the CLJS JIRA#2016-10-0522:25jrheardyou got it#2016-10-0522:30jrheard( CLJS-1811 )
#2016-10-0523:58Oliver GeorgeToday's random spec experiment: Can you define an SQL schema from a clojure.spec definition?#2016-10-0523:58Oliver GeorgeHere's what I came up with: https://gist.github.com/olivergeorge/27e9fced404f5ddef371ea3376456f2f#2016-10-0523:59Oliver George
(do
  (s/def ::name string?)
  (s/def ::address (varchar 256))
  (s/def ::age integer?)
  (s/def ::gender #{"M" "F"})
  (s/def ::example-table (s/keys :req-un [::name ::age ::gender]
                                 :opt-un [::address]))

  (spec->sql-table ::example-table))
#2016-10-0523:59bfabryoooh I should bookmark this. I'm definitely going to need spec->avro schema at some point#2016-10-0523:59Oliver GeorgeProduces
[["name" "varchar2(4000)" "not null"]
  ["age" "int" "not null"]
  ["gender" "varchar2(1)" "not null"]
  ["address" "varchar2(256)"]]
#2016-10-0600:00Oliver George@bfabry no promises that it's the most elegant approach. I'm looking forward to seeing how others work with spec forms.#2016-10-0600:16mattlyheh, I'm actually working on something to define graphql forms from specs definitions#2016-10-0600:17jrheardunrelated: youā€™re using graphql? dā€™you like it?#2016-10-0600:17jrheardi havenā€™t used it myself but the talks iā€™ve seen on it are pretty compelling#2016-10-0600:20Oliver George@mattly that's on my radar too. We are replacing our simple "datomic style" pull api with graphql soon. Here's what I came up with generating recursive pull specs from clojure.spec forms: https://gist.github.com/olivergeorge/9d822447e4838092d07138ea51af5782#2016-10-0600:21mattlyoh neat#2016-10-0600:21Oliver George(again, no promises that it's a good approach... really want to see other peoples code and learn some effective techniques)#2016-10-0600:22mattlyI'm going to keep my pull-style api, but aim to hookup a graphql parser to it#2016-10-0600:23Oliver GeorgeThat makes sense.#2016-10-0600:24mattlymostly because some of my query endpoints, I need expressive arguments#2016-10-0600:26Oliver GeorgeWe bolted a "pull" arg to our REST endpoints which was a practical step forward at the time. Same old REST params etc but more flexibility around what data we return. Now that GraphQL is looking fairly stable we're intrested in standardising on it's interface. That decouples our frontend from the backend which should allow us some flexibility.#2016-10-0600:27mattlyI'm fairly lucky in that mine is a greenfield project#2016-10-0600:28Oliver GeorgeNice. We're in the enterprise space so typically the backend tech stack is dictated to us.#2016-10-0600:29mattlyI may try to abstract out my resolver into a library#2016-10-0600:30mattlyit has a separate planning stage to help with the n+1 problem#2016-10-0600:30Oliver George@mattly I'd love to see what you come up with.#2016-10-0600:33Oliver GeorgeI'm not sure it's related but I did some thinking around translating graph queries into flat sql queries.#2016-10-0600:34Oliver GeorgeThis paper gave me the idea: http://homepages.inf.ed.ac.uk/slindley/papers/shredding.pdf#2016-10-0600:37mattlyeh, removed the link, there's a bug in that#2016-10-0600:37mattlyI need to properly extract it#2016-10-0600:38Oliver GeorgešŸ™‚#2016-10-0607:35mikeb@olivergeorge any thoughts about reversing direction and generating spec definitions from a sql schema?#2016-10-0609:30Oliver George@mikeb I've had some fun with that actually. It's easy to produce something for each column in a database but the relations are trickier.#2016-10-0609:31Oliver GeorgeHere's some related code. https://gist.github.com/olivergeorge/468464ce82b8da486736fe725a4b6ff8#2016-10-0609:32Oliver GeorgeThe slightly fiddly bit is that you can't really do it dynamically. At least I can't work out how to have my script generate and run macros. In the end I generated some forms and committed them as code.#2016-10-0615:04Yehonathan SharvitI was looking into clojurescript code and I saw that clojure/core/specs.clj from clojure is not there. Why clojure/core/specs.clj hasnā€™t been ported into clojurescript?#2016-10-0615:10Yehonathan Sharvit@dnolen is it on purpose?#2016-10-0615:11dnolenJust haven't gotten to it yet#2016-10-0615:14Yehonathan Sharvitwould you be interested in a patch?#2016-10-0615:22mlimotteI see this in the spec guide: "check also takes a number of options that can be passed to test.check to influence the test run, as well as the option to override generators for parts of the spec, by either name or path." Any examples of the "path" option for generator overrides?#2016-10-0615:25jrheardiā€™d love to see an example of generator override usage, path or no#2016-10-0615:28mlimotte@jrheard here's a basic example with override by name:
(defn foo [x] (println "FOO" x) (inc x))
(s/fdef foo :args (s/cat :x ::x))
(s/def ::x clojure.future/int?)
(stest/check `foo
             {:gen          {::x #(gen/return 10)}
              ::stestc/opts {:num-tests 1}})
#2016-10-0615:29jrheardthanks!#2016-10-0615:29jrheardgen comes from clojure.test.check, not from spec, right?#2016-10-0615:30mlimotteBut not sure how to do it when the spec is defined like this:
(s/fdef foo :args (s/cat :x clojure.future/int?))
#2016-10-0615:30mlimotte[clojure.spec.gen :as gen]#2016-10-0615:31jrheardnice, thx#2016-10-0615:43dpiatek@mlimotte I think you need to do (s/fdef foo :args (s/cat :x (s/spec clojure.future/int?)))#2016-10-0615:46mlimotteThe foo spec above seems to work even w/out (s/spec ...). I'm still not sure how to do a generator override for this case where the spec is not named, though.#2016-10-0615:48dpiatekOh, I see - sorry, misread the conversation!#2016-10-0615:49mlimottenp#2016-10-0618:34naomariki'm sorry about this naive question: if i were to spec a map that has strings as keys or even another type, i would have to convert them to keywords first before passing them into spec?#2016-10-0618:42mlimottei don't think so, do you have an example of what you're trying to do?#2016-10-0618:43bfabry@naomarik no. you need keywords for keys to use s/keys but there's still lots of other predicates that you can use to validate maps, including writing your own#2016-10-0618:46naomariki've been struggling to find an example for this, let's say just validating something simple like {"type" "tweet" "user" {"type" "registered"}}. I would know how to go about doing this from reading the official guide if these were keywords, but not like this#2016-10-0618:48mlimottei think you're saying that those are required keys. So, like bfabry said, you can't use s/keys. you have to write your own predicates to do that.#2016-10-0618:49bfabry@naomarik it depends on what you want to validate. if you know what keys you're going to get in advance and what their semantics are and which ones are required then I'd recommend converting them to keywords. it just makes sense. if the keys are dynamic then I would leave them as strings and validating other properties about the map#2016-10-0618:55mlimotte@naomarik if it's just required keys, a simple predicate isn't too bad:
(s/def ::my-map (s/and map? #(every? (partial contains? %)  #{"type" "user"})))
(s/valid? ::my-map {"type" "tweet" "user" {"type" "registered"}})
=> true
(s/valid? ::my-map {"type" "tweet" "NotUser" {"type" "registered"}})
=> false
If it's more complicated and you need all the features of s/keys, than writing your could be a hassle.
#2016-10-0618:57naomarikah i see#2016-10-0618:57naomariks/keys is a lot more convenient at that point#2016-10-0618:58bfabryyeah like I said, if you know what keys your map contains I would convert them to keywords. for lots and lots of good reasons including that you get nice spec tools for working with them#2016-10-0619:00naomariki always write all my maps as keywords, but is there a discussion online why namespaced keywords are enforced so i can be enlightened?#2016-10-0619:01bfabrythere's been a few, I think maybe the cognicast podcast with rich hickey talking about spec might be the best bet. they're not enforced though#2016-10-0619:02bfabrys/keys has corresponding :req-un and :opt-un options that you can use for un-namespaced keywords#2016-10-0619:05mpenetYeah but some stuff is brittle with un- keys ex broken conforming with s/merge#2016-10-0619:06bfabryeh?#2016-10-0619:06mpenetSee recent discussion about it in history#2016-10-0619:07mpenetClj-1981#2016-10-0619:08bfabryalex's comments seem to indicate that's confusion over how conforming works with merge, and isn't related to namespaced keys#2016-10-0619:09mpenetWrong ticket, my google fu failed me#2016-10-0619:09bfabryhttp://dev.clojure.org/jira/browse/CLJ-2033#2016-10-0619:10bfabryhe's saying only the last spec in the merge conforms#2016-10-0619:10bfabryohh wait no I understand#2016-10-0619:10bfabryok, but there's no way to fix that#2016-10-0619:11mpenetNever saw the point in ns keys personally. #2016-10-0619:11bfabryyou get behaviour for free when using namespaced keys. but it's just not possible if they're not namespaced#2016-10-0619:11bfabrywell. this would be one of the points. you get conforming for free#2016-10-0619:12bfabryI think perhaps alex's example hasn't explained what's happening properly. this might help
boot.user=> (require '[clojure.spec :as s])
nil
boot.user=> (defn convert [n] (if (double? n) n (double n)))
#'boot.user/convert
boot.user=> (s/def ::value (s/conformer convert))
:boot.user/value
boot.user=> (s/conform (s/keys) {::value 5})
#:boot.user{:value 5.0}
boot.user=>
#2016-10-0619:13bfabryso even though s/keys doesn't specify ::value as req or opt, because ::value is namespaced and has a spec associated it automatically gets conformed and validated#2016-10-0619:15bfabryso it's not that merge behaves differently with namespaced keys, it's that s/keys behaves differently with namespaced keys (it gives you free stuff if they're namespaced)#2016-10-0619:19Yehonathan SharvitSomething weird about s/and:
user=> (s/def ::ff (s/cat :start integer? :end integer?))
:user/ff
user=> (s/def ::gg (s/* ::ff))
:user/gg
user=> (s/explain-str ::gg [4 7 2 1])
"Success!\n"
user=> (s/def ::ff (s/and (s/cat :start integer? :end integer?)))
:user/ff
user=> (s/explain-str ::gg [4 7 2 1])
"In: [0] val: 4 fails spec: :user/ff predicate: (cat :start integer? :end integer?)\n"
#2016-10-0619:20Yehonathan SharvitWhy in the second case - after adding s/and the vector doesnā€™t conform?#2016-10-0619:31seancorfieldRegex.#2016-10-0619:32seancorfieldYour ::gg spec is zero or more integer-followed-by-integer patterns.#2016-10-0619:32seancorfieldBecause s/* and s/cat combine as a regex sequence.#2016-10-0619:33seancorfieldWhen you insert s/and you isolate the the pair spec so ::gg becomes zero or more pairs#2016-10-0619:33seancorfieldi.e., [[4 7] [2 1]]#2016-10-0619:36Yehonathan SharvitWhat is the way not to isolate the pair?#2016-10-0619:36Yehonathan SharvitMy use case is that I need extra-validation#2016-10-0619:36Yehonathan Sharvit
(s/def ::ff (s/and (s/cat :start integer? :end integer?)
                   #(< (:start %) (:end %))))

(s/def ::gg (s/* ::ff))
#2016-10-0619:37Yehonathan SharvitMakes sense?#2016-10-0619:37Yehonathan Sharvit@seancorfield#2016-10-0619:40seancorfieldSo you want a sequence of integers that is even in length and when you partition that sequence into pairs, each pair is orderedā€¦?#2016-10-0619:40Yehonathan Sharvitexactly!#2016-10-0619:42seancorfieldSo (s/and seq-of-ints length-is-even every-partitioned-pair-is-ordered)#2016-10-0619:43mpenet@bfabry Yep, thats what I meant. The fact that ns keys are so deeply rooted in spec makes this stuff (merge for instance) counter intuitive. #2016-10-0619:46Yehonathan Sharvit@seancorfield But I want to do it in an idiomatic clojure.spec way#2016-10-0619:46bfabryagain, I'd say it's (s/keys) that's counter-intuitive, not s/merge, but yeah possibly the automatic conform/validate nature of namespaced keywords could be called out more#2016-10-0619:46Yehonathan SharvitActually, my real need is to build a spec for sequence of integers that are palindromes...#2016-10-0619:48Yehonathan SharvitI tried:
(s/def ::short-palindrome 
  (s/and (s/cat :a integer?
                :rest ::palindrome
                :a integer?)
         #(= (:a1 %) (:a2 %))))

(s/def ::palindrome (s/* ::short-palindrome))
#2016-10-0619:48mpenetI know, I never said merged was bugged, but that s one of the places where this behavior shows through in a bad way#2016-10-0619:49Yehonathan SharvitBut it only accepts nested sequences like [[1 [ 2 2] 1]] but not flat sequences like that: [1 2 2 1]#2016-10-0619:53Yehonathan Sharvithow would you write the spec for palindrome @seancorfield ?#2016-10-0619:56seancorfieldFor a sequence that is a palindrome? (s/and (s/coll-of integer?) #(= % (reverse %)))#2016-10-0619:56seancorfield(untested)#2016-10-0619:56Yehonathan Sharvittesting it...#2016-10-0619:58Yehonathan SharvitIt works (except for empty sequences)#2016-10-0619:58Yehonathan SharvitBut...#2016-10-0619:58Yehonathan SharvitI would like to write it like a context-free grammar#2016-10-0619:59Yehonathan SharvitMaybe thatā€™s not the idea with clojure.spec @seancorfield ?#2016-10-0619:59seancorfieldYou probably want to ask @alexmiller ...#2016-10-0620:08Yehonathan SharvitNow, Iā€™m trying to implement the grammar for algebraic expressions - as described here https://en.wikipedia.org/wiki/Context-free_grammar#Algebraic_expressions:
Here is a context-free grammar for syntactically correct infix algebraic expressions in the variables x, y and z:

S ā†’ x
S ā†’ y
S ā†’ z
S ā†’ S + S
S ā†’ S - S
S ā†’ S * S
S ā†’ S / S
S ā†’ ( S )
#2016-10-0620:08Yehonathan SharvitDo I have a chance to succeed @seancorfield and @alexmiller ?#2016-10-0620:10Alex Miller (Clojure team)I would have written something like this for palindromes (& is much better than and here as it stays in regex)#2016-10-0620:10Alex Miller (Clojure team)
(s/def ::pal 
  (s/alt :0 (s/cat)
         :1 int?
         :n (s/& (s/cat :a int? :b ::pal :c int?) (fn [{:keys [a c]}] (= a c)))))
#2016-10-0620:12Alex Miller (Clojure team)you can use the same approach for your algebraic expressions#2016-10-0620:12Yehonathan Sharvitgreat#2016-10-0620:12Yehonathan SharvitIā€™m trying it now#2016-10-0620:13Yehonathan Sharvit::pal works well#2016-10-0620:13Yehonathan Sharvitworking on ::algebraic-expression...#2016-10-0620:16Yehonathan Sharvit@alexmiller whatā€™s the exact meaning of (cat) with no arguments?#2016-10-0620:16Yehonathan SharvitIs it like empty??#2016-10-0620:17Alex Miller (Clojure team)yes#2016-10-0620:17Alex Miller (Clojure team)itā€™s a sequential collection with no contents#2016-10-0620:17Alex Miller (Clojure team)but itā€™s a regex so will compose better with other regex ops#2016-10-0620:18Alex Miller (Clojure team)in particular, it will be treated in the same sequential context here, not be a separate (nested) collection#2016-10-0620:19Yehonathan Sharvitthx#2016-10-0620:21seancorfieldAh, yes, I still forget about s/&...#2016-10-0620:22Yehonathan Sharvit@alexmiller Now, Iā€™m having stackoverflow issues#2016-10-0620:22Yehonathan SharvitšŸ˜ž#2016-10-0620:23Yehonathan Sharvit
(s/def ::arithmetic
  (s/alt
    :var int?
    :plus (s/cat :a ::arithmetic :op #{"+"} :b ::arithmetic)))

(s/explain-str ::arithmetic [1 "+" 1])
#2016-10-0620:24Alex Miller (Clojure team)yeah, that's not going to work#2016-10-0620:24Alex Miller (Clojure team)there are other typical approaches to writing left-recursive grammars like this#2016-10-0620:26Yehonathan Sharvitleft-recursion is not (yet) supported in clojure.spec?#2016-10-0620:28Alex Miller (Clojure team)this is just a consequence of how the regex logic works#2016-10-0620:31Yehonathan Sharvityou mean the deriv and accept-nil? functions#2016-10-0620:31Yehonathan Sharvit- that implement the ideas of the ā€œParsing with derivativesā€ paper ?#2016-10-0620:36Alex Miller (Clojure team)spec regex walks all viable alternatives in parallel - in this case, that becomes an ever-growing set#2016-10-0620:36Alex Miller (Clojure team)but you can rewrite those kinds of grammars like#2016-10-0620:36Alex Miller (Clojure team)
(s/def ::ar (s/cat :a int? :r (s/? (s/cat :op #{"+ā€} :b (s/alt :i int? :e ::ar)))))
#2016-10-0620:38Alex Miller (Clojure team)in reality, a) these problems donā€™t come up that often and b) prefix-factoring is an option#2016-10-0620:39Yehonathan SharvitBy ā€œprefix-factoringā€ you mean that the recursion appears last or that it doesnā€™t appear first?#2016-10-0621:01seancorfield@viebel you can see why I deferred to Alex on the "context-free grammar" aspect of clojure.spec, eh? šŸ™‚#2016-10-0621:02Yehonathan SharvitYeah! Itā€™s hard stuff#2016-10-0621:02Yehonathan SharvitNow, Iā€™m struggling with the parentheses#2016-10-0621:02Yehonathan Sharvit
(s/def ::my-int (s/* #{\0 \1 \2 \3 \4 \5 \6 \7 \8 \9 \x \y \z}))
(s/def ::ar (s/alt :operation (s/cat :a ::my-int      
                                     :r (s/? (s/cat :op #{\+ \*  \- \/}
                                                    :b (s/alt :i ::my-int :e ::ar))))
                   :parentheses (s/cat :o #{"("}
                                       :b (s/alt :i ::my-int :e ::ar)
                                       :c #{")"})))
#2016-10-0621:03Yehonathan SharvitIt works for (s/explain-str ::ar (seq "x+(y*x)ā€))#2016-10-0621:03Yehonathan SharvitBut with (s/explain-str ::ar (seq "(y*x)+xā€)) I get:
In: [5] val: ("+" "x") fails spec: :my.spec/ar predicate: (alt :operation (cat :a :my.spec/my-int :r (? (cat :op #{"+" "*" "-" "/"} :b (alt :i :my.spec/my-int :e :my.spec/ar)))) :parentheses (cat :o #{"("} :b (alt :i :my.spec/my-int :e :my.spec/ar) :c #{")"})),  Extra input
#2016-10-0621:04Yehonathan Sharvit@alexmiller please help (again) šŸ˜°#2016-10-0621:17Alex Miller (Clojure team)Sorry I'm off duty for the night :)#2016-10-0621:22Yehonathan SharvitNo problem: Actually Iā€™m in Israel and itā€™s 12:30 AM - Iā€™m too tired to continue with this tough topic. Weā€™ll catch up later. If you find something please mention me on slack.#2016-10-0621:45joshgIs there an idiomatic way to define a spec for a constant value? Something like (s/def ::some-constant #(identical? % "foo"))?#2016-10-0621:45Yehonathan SharvitYou can use set like this: #{ā€œfooā€}#2016-10-0621:46Yehonathan Sharvitbecause sets behave like functions#2016-10-0621:46joshgThat works, thanks#2016-10-0621:46bfabryyes, sets are the idiomatic way to do that, sets also work as generators (while #(identical? % "foo") does not)#2016-10-0702:26richiardiandreaHello folks! I know how to spec a map with keys which I don't care the values of: (s/map-of #{:mid :seq} any?) thanks to Alex, but is there a way standard pred to say "any non-nil value" or I must create my own predicate?#2016-10-0702:28richiardiandreaI tried (complement nil?) but then the generator fails#2016-10-0702:29bfabry@richiardiandrea probably some?#2016-10-0702:29richiardiandreaoh right, let me try#2016-10-0702:30bfabry
boot.user=> (s/def ::some some?)
:boot.user/some
boot.user=> (s/exercise ::some)
([{} {}] [{} {}] [(()) (())] [{([[]]) ([])} {([[]]) ([])}] [(-1/2 false) (-1/2 false)] [{{{{{} (-1.5)} ([\=])} []} [{}]} {{{{{} (-1.5)} ([\=])} []} [{}]}] [([:N4oa:*I+-:0:+c8_68:!!-6L]) ([:N4oa:*I+-:0:+c8_68:!!-6L])] [[[([true])]] [[([true])]]] [{} {}] [[()] [()]])
boot.user=>
#2016-10-0702:30richiardiandreayes it looks like that's what I was looking for thanks a lot @bfabry !#2016-10-0702:30bfabrynp#2016-10-0703:12seancorfieldLooks like some? always generates a collection thoā€™?#2016-10-0703:14bfabryHUH, that's weird#2016-10-0703:14seancorfield@richiardiandrea what about (s/and any? (complement nil?)) ? That generates regular values that arenā€™t nilā€¦
boot.user=> (s/exercise (s/and any? (complement nil?)))
([{} {}] [[] []] [[] []] [([false]) ([false])] [[()] [()]] [[] []] [{[[:Q5B+B:-vS7!+_2a:14s+?:Wd:?:E3+cY7:7q9n_V:+T*_] [L?lQDc.poz2z._rRq+9*Le._?.o.Pq!San9?!.uP36/y?+-i_]] {[] {}}} {[[:Q5B+B:-vS7!+_2a:14s+?:Wd:?:E3+cY7:7q9n_V:+T*_] [L?lQDc.poz2z._rRq+9*Le._?.o.Pq!San9?!.uP36/y?+-i_]] {[] {}}}] [{[] ()} {[] ()}] [[:AE+4CW!863.Rt_??z4!.ob*6.*M.J*+7!.?Wpz.RH?.k+L-L?-2_8.Dz_5E3zaY.!2GNEsM/N51M0Sn7 6 5] [:AE+4CW!863.Rt_??z4!.ob*6.*M.J*+7!.?Wpz.RH?.k+L-L?-2_8.Dz_5E3zaY.!2GNEsM/N51M0Sn7 6 5]] [[{[3 \s] [true], (s) (1/6)} {() (:?tLe9B2-66t.--s6+?8E*.*GVek-7!Y.++5BT3TQ6.x0Al3z.*7sAkrq8.*h9.a+P+.qz6?6?/KT-4!m+_4D -8), {\t v-Y+u+D+vk-.K?XwW-8?1?.+!529+.DaEb.erp7.hl?+??ma+t.-ZiN+y+.VC63aicOOe7.!226W+RFO.e1*??7/-q*.} {-5/3 \?}}] [{[3 \s] [true], (s) (1/6)} {() (:?tLe9B2-66t.--s6+?8E*.*GVek-7!Y.++5BT3TQ6.x0Al3z.*7sAkrq8.*h9.a+P+.qz6?6?/KT-4!m+_4D -8), {\t v-Y+u+D+vk-.K?XwW-8?1?.+!529+.DaEb.erp7.hl?+??ma+t.-ZiN+y+.VC63aicOOe7.!226W+RFO.e1*??7/-q*.} {-5/3 \?}}]])
#2016-10-0703:15seancorfieldHmm, looking at those, most seem to be collections tooā€¦?#2016-10-0703:15bfabrysomething weird going on there https://github.com/clojure/clojure/blob/master/src/clj/clojure/spec/gen.clj#L133#2016-10-0703:16seancorfieldAh, I think itā€™s down to how any? generates stuff...#2016-10-0703:16bfabrythere's no reason some? should be only collections, and I think that behaviour might be common#2016-10-0703:17seancorfieldLooks like any? generates nil or a collectionā€¦
boot.user=> (->> (s/exercise any? 100) (map second) (remove coll?))
(nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil)
#2016-10-0703:18bfabryyeah seems like whatever any-printable is always generates a coll#2016-10-0703:23bfabryI can't be bothered digging but that seems like not-incorrect-but-pretty-undesirable#2016-10-0703:26seancorfield10,000 iterations donā€™t produce anything that isnā€™t nil or a collection:
boot.user=> (->> (s/exercise any? 10000) (map second) (remove coll?) (remove nil?))
()
#2016-10-0703:26seancorfield(dang, I love that you can do stuff like that so easily in the REPL!)#2016-10-0703:29bfabryseems like the behaviour is inherited from clojure.test.check.generators/any-printable#2016-10-0703:34bfabryyuuup
boot.user=> (remove coll? (remove nil? (repeatedly 1000 #(clojure.test.check.generators/generate clojure.test.check.generators/any-printable))))
()
#2016-10-0703:41seancorfieldThat probably needs ā€¦ enhancing ...#2016-10-0704:10richiardiandreaFor generating maps only I was thinking of using an s/and but for a first approximation of this spec some? is perfect#2016-10-0704:10richiardiandreaThat is good investigation though#2016-10-0707:26preporHello. ring-spec had just released. Imagine that library was named not "ring", but "com.stuartsierra.ring". For example, https://gist.github.com/prepor/c9a3ac1c5746146671b55631a543a6e4#file-spec-clj-L107. There are a lot of noise, aren't it?#2016-10-0707:36robert-stuttafordiā€™ve never understood reverse domain name packages#2016-10-0707:36robert-stuttafordi get the rationale, but it almost never pans out in practice. company names suffice, i find#2016-10-0707:44misha@prepor
(require '[clojure.spec :as s])
(create-ns 'com.this.looks.really.inconvenient)
(alias 'inco 'com.this.looks.really.inconvenient)

(s/def ::inco/bar int?)
=> :com.this.looks.really.inconvenient/bar

(s/valid? ::inco/bar 1)
=> true

(s/valid? :com.this.looks.really.inconvenient/bar 1)
=> true
#2016-10-0707:47prepor> iā€™ve never understood reverse domain name packages this is the only way to guarantee name clashes absence#2016-10-0707:47prepor@misha nice. will it work with aot?#2016-10-0707:48mishakappa no idea#2016-10-0707:50preporyep. i think dynamic ns juggling is controversial way...#2016-10-0710:13borkdudeIā€™m amazed spec can parse the last case:
(s/def ::rhs (s/cat :lo (s/? symbol?)
                    :r (s/? (s/alt :rsh (s/cat :op #(= % 'RSHIFT)
                                               :ro int?)
                                   :or (s/cat :op #(= % 'OR)
                                              :ro symbol?)
                                   :not (s/cat :op #(= % 'NOT)
                                               :ro symbol?)
                                   ))))

(s/conform ::rhs '[bn RSHIFT 2])
(s/conform ::rhs '[cj OR cp])
(s/conform ::rhs '[lx])
(s/conform ::rhs '[NOT ax])
I expected this not to work because NOT is also a symbol and that none other alt matches the rest
#2016-10-0711:07borkdudeIs there a conform variant that throws like s/assert but returns the conformed value?#2016-10-0712:23Alex Miller (Clojure team)No but it's easy to make one#2016-10-0712:23Alex Miller (Clojure team)@prepor should work fine with aot#2016-10-0712:24Alex Miller (Clojure team)@seancorfield: if you filed a ticket on that any? some? generator business I would take a look at it. I keep forgetting to do so#2016-10-0712:57prepor@alexmiller ok, thank you#2016-10-0714:26lvh@borkdude btw, consider #{ā€˜RSHIFT}#2016-10-0714:26lvh@borkdude Itā€™s shorter, generally considered more idiomatic, and spec can inspect it#2016-10-0714:27borkdude@lvh my current spec:
(s/def ::val (s/alt :name symbol? :value int?))
(s/def ::binary-operator #(#{'LSHIFT 'RSHIFT 'AND 'OR} %))
(s/def ::binary-expression (s/cat
                            :left-operand ::val
                            :operator ::binary-operator
                            :right-operand ::val))
(s/def ::not (s/cat :not #(= % 'NOT) :operand ::val))
(s/def ::lhs (s/alt :simple-value ::val
                    :binary-expression
                    ::binary-expression
                    :not ::not))
(s/def ::rhs symbol?)
(s/def ::expr (s/cat :lhs ::lhs :arrow #(= % '->) :rhs ::rhs))
#2016-10-0714:27lvhthat works too#2016-10-0714:27lvhstill gotta fix the #{ā€™NOT} if you want generation to work though#2016-10-0714:27borkdudeah#2016-10-0714:27lvhalso: arrow#2016-10-0714:27borkdudeoh wow, I didnā€™t even think of generation#2016-10-0714:28borkdudemust. try.#2016-10-0714:28lvhborkdude Itā€™s pretty cool. Iā€™m currently generating specs for swagger from the swagger json-schema definition, which I then use the generate swagger instances, from which I then generate instances of that swagger schema, which I then validate against the original swagger#2016-10-0714:29borkdudešŸ˜„#2016-10-0714:29borkdudewhat reads better, '#{LSHIFT RSHIFT AND OR} or quotes before all symbols?#2016-10-0714:30lvherrr#2016-10-0714:30lvhI dunno; Iā€™d go with quotes before syms#2016-10-0714:35borkdudeIt works. How do I get s/cat to generate vectors instead of whatever it generates now?#2016-10-0714:44borkdudeHow do I get a generator that only generates lower case alphabetic strings?#2016-10-0714:55nwjsmith
(gen/not-empty
  (gen/fmap
    (fn [s] (str/replace s #"[A-Z0-9]" ""))
    gen/string-alphanumeric))
#2016-10-0714:56nwjsmiththat's a really naive generator, but it'll get you there#2016-10-0715:07borkdudeThanks! I needed symbols of one or two character, so I got this now:
(s/with-gen symbol?
                  #(gen/fmap (fn [[i j]]
                               (symbol (string/lower-case (str i j))))
                             (gen/tuple (gen/char-alpha) (gen/char-alpha))))
#2016-10-0715:09borkdudehow do I get s/exercise to generate vectors instead of seqs for s/cat?#2016-10-0715:41nwjsmithShould work with :kind, no?
(s/exercise (s/coll-of (s/cat :a nat-int? :b string?) :kind vector?))
#2016-10-0715:53nwjsmithOtherwise, you might want to check out s/tuple#2016-10-0716:54seancorfield@alexmiller https://clojurians.slack.com/archives/clojure-spec/p1475843093004536 http://dev.clojure.org/jira/browse/CLJ-2036#2016-10-0716:55bfabry@seancorfield @alexmiller it actually seems like there's already a ticket on test.check http://dev.clojure.org/jira/browse/TCHECK-111#2016-10-0716:56seancorfieldThanks @bfabry ā€” cross-linked#2016-10-0721:55jrheardha, hadnā€™t seen ffirst before, thx for the tip#2016-10-0721:57jrheardhadnā€™t seen advent of code either, will have to take a look#2016-10-0800:31jrheard(update: itā€™s pretty fun! :D)#2016-10-0803:30richiardiandreaA question, I can open a JIRA bug as well, shouldn't exercise-fn accept custom generators?#2016-10-0809:33Alex Miller (Clojure team)Yes#2016-10-0815:22jrheard@borkdude just caught up - https://github.com/jrheard/advent-of-code/blob/master/src/advent_of_code/day_7.clj#2016-10-0815:23jrheardthis is fun, donā€™t know why i didnā€™t do it when it came out! looking forward to the 2016 edition#2016-10-0815:23jrheard[assuming i can solve the next 18ā€¦]#2016-10-0815:23borkdudegreat!#2016-10-0815:24jrheardi like your use of s/conform, it felt like that was super relevant re: parsing but i guess i just didnā€™t reach for it; gonna take another look and see how you accomplished it#2016-10-0815:26jrhearddelayā€™s neat too, havenā€™t used that before - thanks for sharing your solution, v educational! šŸ˜„#2016-10-0822:01jrheards/conform came super in handy for day 8 šŸ™‚ https://github.com/jrheard/advent-of-code/blob/master/src/advent_of_code/day_8.clj#2016-10-0822:02jrheardspec rules#2016-10-0907:01bensuI defined an fdef spec for a macro and Iā€™m testing it with (is (thrown? Exception (macroexpand ā€˜(my-macro bad-args))))#2016-10-0907:02bensuwhen I run it in the repl lein with-profile +test repl the test passes#2016-10-0907:02bensubut when I run it with lein test the arguments are not checked and the test fails#2016-10-0907:03bensuHas anybody hit a similar problem?#2016-10-0907:17hiredmanis your macro code actually being loaded? macroexpand doesn't throw an error if my-macro doesn't exist and isn't defined as a macro#2016-10-0907:29bensuit was getting loaded#2016-10-0907:29bensuthe problem turned out to be related: it was loaded but not recognized by macroexpand because I was not using a syntax-quote#2016-10-0907:29bensu@hiredman thank you for your help#2016-10-0909:17Yehonathan SharvitSometimes s/conform and s/unform are not inlined. For instance:
(s/def ::my-spec (s/cat :op (s/? (s/cat :a integer?))))
(->> (s/conform ::my-spec '(1))
     (s/unform ::my-spec))
It returns ((1))
#2016-10-0909:17Yehonathan SharvitThere is an extra nesting#2016-10-0909:18Yehonathan SharvitLive demo here: https://bit.ly/2efyHkI#2016-10-0909:18Yehonathan SharvitIs it a bug in clojure.spec @alexmiller ?#2016-10-0912:17sanderis there a way to get a spec that (s/conform ::my-spec v) will always conform to? I want to fspec a function saying that it takes a spec-destructured value as an argument, and returns another spec-destructured value#2016-10-0912:22Yehonathan Sharvitdid you try s/any??#2016-10-0912:22Yehonathan Sharvit@sander#2016-10-0912:23sanderI should rephrase, I want a specific spec that only accepts values in the shape that (s/conform ::my-spec v) returns#2016-10-0912:30Yehonathan SharvitInteresting question...#2016-10-0912:44Alex Miller (Clojure team)@viebel yes, generally anytime conform unform returns a different result, that's a bug. This particular one is already logged.#2016-10-0912:44Yehonathan Sharvitjira?#2016-10-0912:47Alex Miller (Clojure team)@sander no, other than saying what you just said as a custom predicate - call s/valid? on the result with another spec#2016-10-0912:47sanderok, should be easy enough, thanks!#2016-10-0912:51Alex Miller (Clojure team)@viebel http://dev.clojure.org/jira/browse/CLJ-2003#2016-10-0912:54Yehonathan Sharvitthx @alexmiller#2016-10-0912:54Alex Miller (Clojure team)Rich and I talked about this one on Friday in particular - it's tricky#2016-10-0912:55Yehonathan SharvitWhy is it tricky?#2016-10-0912:56Alex Miller (Clojure team)Conceptually tricky about what an optional regex should return in different contexts, how it nests, etc#2016-10-0912:58Yehonathan SharvitIn my specific case, I was able to workaround this bug using s/conformer. But I still donā€™t understand fully how s/conformer works. Could you explain in simple words how does s/conformer work?#2016-10-0912:58Yehonathan SharvitThe docstring is a bit obscure...#2016-10-0912:59Alex Miller (Clojure team)It acts as a predicate but applies an arbitrary transformation in the conformed value #2016-10-0913:00Yehonathan SharvitWhat is the simplest use case for conformer?#2016-10-0913:00Alex Miller (Clojure team)Applying an arbitrary transformation to the conformed value #2016-10-0913:01Alex Miller (Clojure team)It is primarily a tool for building more complex spec variants#2016-10-0913:03Alex Miller (Clojure team)It's generally recommended to use sparingly as embedding it in a spec means that you are overriding how spec conforms for all future consumers of your spec#2016-10-0913:03Yehonathan Sharvitwith s/and?#2016-10-0913:03Alex Miller (Clojure team)Yes#2016-10-0913:04Alex Miller (Clojure team)Like if you used s/or but didn't want a tagged result#2016-10-0913:05Alex Miller (Clojure team)When you do that you throw away info about how something was conformed#2016-10-0913:05Alex Miller (Clojure team)One use might not need it but other possible future uses may#2016-10-0913:06Alex Miller (Clojure team)So baking it into your public spec takes away that option#2016-10-0913:11Yehonathan SharvitThx#2016-10-0913:11Yehonathan SharvitIā€™ll play with that...#2016-10-0913:45Yehonathan Sharvit@alexmiller another use case would be when one wants to match regex with vectors#2016-10-0913:45Yehonathan SharvitThe naive code would be:
(s/def ::vec-of-int-and-str (s/cat :0 integer?
                                   :1 string?))

(->> (s/conform ::vec-of-int-and-str [1 "a"])
     (s/unform ::vec-of-int-and-str))
#2016-10-0913:46Alex Miller (Clojure team)There will be a different solution for that#2016-10-0913:46Alex Miller (Clojure team)But yes#2016-10-0913:46Yehonathan Sharvitbut it returns: (1 "aā€) instead of [1 ā€œaā€]#2016-10-0913:47Yehonathan SharvitSo the fix is:
(s/def ::vec-of-int-and-str (s/and
                              vector?
                              (s/conformer identity vec)
                              (s/cat :0 integer?
                                     :1 string?)))
#2016-10-0913:47Yehonathan SharvitIs there a better solution for that?#2016-10-0913:54Alex Miller (Clojure team)Not yet but it's coming#2016-10-0913:55Yehonathan SharvitIā€™m curiousā€¦ :relaxed:#2016-10-0915:31jrheardnoob question, iā€™m sure, re: conformer - i reached for conformer when i was operating on data that looked like ā€œ123ā€, but i wanted (s/conform ::my-spec) to parse the integer such that it ended up being 123. what should i use instead of s/conformer in situations like that, for coercion of this sort?#2016-10-0915:31jrheardi felt like i read about something that would solve this problem idiomatically in the rationale or guide, but couldnā€™t find it when i went back to look for it#2016-10-0916:30seancorfield@jrheard: yes I have a lot of cases where I have strings as input but I want them conformed to integers (for API input specs). But @alexmiller has told me a couple of times that is "not idiomatic" and to avoid conformers like that :(#2016-10-0916:45jrheardhm, i donā€™t understand - the strings have to be turned into integers at some point, right?#2016-10-0916:46jrheardis it more idiomatic to have a secondary, non-spec transformation layer that happens after s/conform?#2016-10-0916:47jrheardiā€™m sure @alexmiller has good reasons for saying that, but i donā€™t understand what spec users are actually supposed to do here#2016-10-0917:44Alex Miller (Clojure team)The downside is that once you enshrine a conversion into your spec, you have made a conformance decision for all future consumers that has removed information. If you understand this ramification and are willing to live with the consequences, then that's up to you.#2016-10-0917:46Alex Miller (Clojure team)There are many ways to apply conversions - you should be aware of when you are conflating conformance and conversion though#2016-10-0918:01jrheardi think i follow those statements, but a ā€œin situations like that, try doing this insteadā€ example would be supremely helpful šŸ™‚#2016-10-0919:40Alex Miller (Clojure team)There's not one answer#2016-10-0919:41Alex Miller (Clojure team)You can apply post transformations#2016-10-0919:41Alex Miller (Clojure team)You can apply a second spec#2016-10-0919:41Alex Miller (Clojure team)You can apply a different spec in one case /cdn-cgi/l/email-protection#2016-10-0919:41Alex Miller (Clojure team)Etc#2016-10-0919:45Alex Miller (Clojure team)And in some cases conformers might be the right answer#2016-10-1000:21jrheardgotcha, makes sense. thanks!#2016-10-1016:32Yehonathan Sharvithttps://twitter.com/viebel/status/785518288012992512#2016-10-1017:19juliobarrosI'd like to implement a multi step data pipeline where each step's "spec" is slightly modified and based on the previous step's spec. For example at step 1 an address has some number of fields and at step 2 it has the same fields plus geo coordinates. Or in step 1 it can have optional string geo coordinates and after step 2 it has to have numeric ones. It seems like spec requires/encourages you to duplicate base address fields in two separate specs (rather than build on a base spec) and possibly have different names for the string geo coords and double geo coords (since they are registered under the same name). This appears to be more straight forward in schema where I can modify the schema map as I go through the pipeline. Am I reading this right? Am I missing something? What's the best spec based approach to this? How would you approach this? Thanks in advance.#2016-10-1017:31hiredmanif your changes are purely additive between pipeline steps, you can compose specs using and#2016-10-1017:35hiredmanmy perspective on specs is you should treat them like defining columns in a sql database. if I had a column named date in a sql database, would I want the scheme to allow strings or dates? most likely not, I would have different columns for strings and dates.#2016-10-1017:41juliobarrosThanks @hiredman the sql table analogy is a good way to think about it. #2016-10-1017:57mattlyI think specs that coerce in a conformer are useful, but I've taken to naming them with a ! suffix#2016-10-1017:59mattlyor well, naming the function that the conformer calls with a ! suffix and typically only using them where absolutely necessary
#2016-10-1018:19jrheardseems like a reasonable pattern, thanks for sharing#2016-10-1018:23Yehonathan Sharvit@mattly donā€™t forget to pass the 2nd param to s/conformer - otherwise s/unform will fail#2016-10-1018:23mattlyheh#2016-10-1018:23mattlyin this case I'm basically using it to validate environment variables#2016-10-1023:23bbloom@hiredman indeed the table/column analogy is interesting, but itā€™s not the whole story. iā€™ve been thinking about ā€œrefinementā€ (for lack of a better word) a bunch lately#2016-10-1023:24bbloomspecs (and for that matter: types) have an inherit problem when modeling time#2016-10-1023:24bbloomfor a simple example, consider before/after validation of some property#2016-10-1023:25bbloomyou can have :unvalidated/foo and :validated/foo, but it seems like a weird way to go about that if they are going to be identical? to each other in all cases#2016-10-1023:25bbloomof course, you can always model the validation as coercion, such that you get the validated value or nil, so having two keys makes sense - and indeed thatā€™s what iā€™d probably do with spec#2016-10-1023:26bbloomthe real problem comes in when you talk about nested structure#2016-10-1023:27bbloomyou donā€™t want to do O(N) allocations to validate a structure#2016-10-1023:27bbloomrenaming all the keys, losing generality of functions#2016-10-1023:31hiredmanI don't think I follow#2016-10-1023:32hiredmanwhen I said sql column above, that is because I am more familiar with using sql, but obviously the best analogy is a datomic attribute#2016-10-1023:33hiredmanso stick all your maps in datomic so you can scrub forward and backwards in time on them šŸ™‚#2016-10-1023:33bbloomheh, sorry, let me try to explain again#2016-10-1023:34bbloomso iā€™m talking about a more general problem, but i think this neatly specifies one specific instance of it: http://blog.ezyang.com/2013/05/the-ast-typing-problem/#2016-10-1023:35bbloomthe ast typing problem is double bad b/c they arenā€™t modeling things with extensible records#2016-10-1023:35bbloombut the notion of having two recursive structures that are subtle distinct at different stages in a pipeline is a common theme#2016-10-1023:36bbloomit sucks to have to do O(N) work (eg copy a whole AST) in order to make some small changes to a subset of nodes#2016-10-1023:36hiredmanisn't that the same argument against immutable datastructures?#2016-10-1023:37bbloomnot quite, itā€™s more related to the argument in favor of covariant arrays in java/c# šŸ™‚#2016-10-1023:38bbloomeg you have an array of integers and need an array of objects, so why should you have to copy every integer in order to satisfy the type checker? obviously you donā€™t have to do that with spec, but itā€™s still a problem if you have a recursive structure and use different keys for different specs#2016-10-1023:38hiredmanI mean, your options are to write novelty in place, or accrete it#2016-10-1023:40bbloomsure, youā€™re totally right - this problem is usually easily avoidable in spec thanks to the dynamic instrumentation and extensible records#2016-10-1023:42bbloomi just think its interesting to think about refinement of specifications without having to introduce new names#2016-10-1023:42bbloomthereā€™s definitely use cases to have ā€œchangedā€ structures vs ā€œextendedā€ structures, but yeah - go with the latter whenever possible#2016-10-1108:02danielstocktonI'm calling explain on a spec and it's returning nil, why would that be?#2016-10-1108:06danielstocktonanswer: i need to use explain-str, explain just prints to out#2016-10-1108:59borkdudeInteresting clojure.spec + datomic question: https://stackoverflow.com/questions/39971727/using-clojure-spec-with-datomic-entities#2016-10-1109:05dm3how would you create a spec for the following: 1) ns store - defines ::state as (s/keys :req [::data ::metadata]) where ::data is a map? 2) ns x produces a ::state with concrete ::data How would the ::x-state spec be defined? (s/and ::store/state #(s/assert ::x-data (::store/data %))) doesn't look right...#2016-10-1109:48danielstocktonOn it's own, [1 1] conforms to the ::pattern spec#2016-10-1114:52samedhi
(ns example.paths
  (:require
   [cljs.spec :as s]
   [cljs.spec.impl.gen :as gen]
   [cljs.spec.test :as test]))

(s/def ::keyword-or-string
  (s/with-gen
    (s/or :keyword keyword?
          :string string?)
    #(s/gen #{"a" "b" :c :d :e/z})))

(def path (s/coll-of ::keyword-or-string
                     :min-count 1
                     :kind vector?))

(s/def ::path
  (s/with-gen
    path
    (fn []
      (s/gen
       (gen/such-that
        #(-> % count (< 5))
        (s/gen path))))))
#2016-10-1114:52samedhi
#object[Error Error: Unable to construct gen at: [] for: [object Object]]
Error: Unable to construct gen at: [] for: [object Object]
#2016-10-1114:53samedhiSo I am trying to reduce the size of generated ::paths to at most be of length 5. It isnā€™t part of the actual spec for a ::path, but it is something I would like when generating, any idea what I am doing wrong?#2016-10-1115:00jrheard@samedhi there is a 20% chance that what iā€™m about to say is relevant, but iā€™ll say it just in case#2016-10-1115:00jrheardiā€™ve run into trouble with s/coll-of specs that define a :kind but not an :into#2016-10-1115:00jrheardtry adding :into [] and see if it does anything#2016-10-1115:01jrhearditā€™s probably irrelevant, but who knows#2016-10-1115:01jrheardthis is definitely a thing if your :kind is set?, you need to define :into #{} for generators to work afaict#2016-10-1115:05samedhi
(ns example.paths (:require [cljs.spec :as s] [cljs.spec.impl.gen :as gen] [cljs.spec.test :as test]))

(s/def ::keyword-or-string
  (s/with-gen
    (s/or :keyword keyword?
          :string string?)
    #(s/gen #{"a" "b" :c :d :e/z})))

(gen/generate (s/gen ::keyword-or-string)) => :d

(def path (s/coll-of ::keyword-or-string
                     :min-count 1
                     :kind vector?
                     :into []))

(gen/generate (s/gen path)) => [ā€œaā€ :c ā€œbā€ ā€œbā€ ā€œbā€ ā€œbā€ ā€œaā€]

(s/gen
 (gen/such-that
  #(-> % count (< 5))
  (s/gen path)))
=> (errors with)
#object[Error Error: Unable to construct gen at: [] for: [object Object]]
Error: Unable to construct gen at: [] for: [object Object]
...
#2016-10-1115:10samedhi@jrheard is this what you mean? In this case I am trying to return a vector of keywords or strings, so I set :kind to vector? and :into to []#2016-10-1115:14jrheardyeah, thatā€™s what i mean; looks like it wasnā€™t relevant, sorry šŸ™‚#2016-10-1115:22samedhiHey, no problem, I appreciate it.#2016-10-1115:40thegeez@samedhi should that be gen/generate instead of s/gen around the gen/such-that?#2016-10-1115:42samedhi@thegeez, actually, that does workā€¦ Let me play a bit with it.#2016-10-1115:43samedhiOn my first post, 2 paste up, I had.#2016-10-1115:43samedhi
(s/def ::path
  (s/with-gen
    path
    (fn []
      (s/gen
       (gen/such-that
        #(-> % count (< 5))
        (s/gen path))))))
#2016-10-1115:44samedhiI think s/with-gen requires a generator as the second argument...#2016-10-1115:45samedhiI think I am basing it off of this from http://clojure.org/guides/spec#_explain#2016-10-1115:45samedhi
(s/def ::kws (s/with-gen (s/and keyword? #(= (namespace %) "my.domain"))
               #(s/gen #{:my.domain/name :my.domain/occupation :my.domain/id})))
#2016-10-1115:46samedhi> Note that with-gen (and other places that take a custom generator) take a no-arg function that returns the generator, allowing it to be lazily realized.#2016-10-1115:47samedhiSo it sounds like I want a zero-args function that returns a generator.#2016-10-1115:47samedhiKind of seems like that is what I have aboveā€¦ right?#2016-10-1115:48samedhiMaybe gen/such-that is already a generator or maybe it is a spec?#2016-10-1115:48thegeezremove the s/gen around the gen/such-that#2016-10-1115:49thegeezgen/such-that is already a generator indeed#2016-10-1115:51samedhi@thegeez You got it!#2016-10-1115:51samedhi
(s/def ::path
  (s/with-gen
    path
    (fn [] (gen/such-that #(-> % count (< 5)) (s/gen path)))))
#2016-10-1115:55samedhiThanks, that helped me out a lot, data seemed to just take forever on generating without this. (though that may actually be my formater and not the actual generation). Fixed loading issues.#2016-10-1117:09samedhi
(s/def ::path
  (s/with-gen
    path
    (fn [] (gen/fmap #(vec (take 5 %)) (s/gen path)))))
#2016-10-1117:10samedhiIs actually slightly better, as it never fails to generate under the 100 try limit.#2016-10-1117:18lvh@alexmiller Suggestion: a special s/cat, primarily intended for specifying args in fdef, that automatically calls s/spec on each of its arguments#2016-10-1117:18lvh@alexmiller I feel like a lot of people have bumped their toe on the nested regex gotcha#2016-10-1117:20Alex Miller (Clojure team)I think thatā€™s unlikely something we would add#2016-10-1117:21Alex Miller (Clojure team)and in general, I donā€™t find that that is (usually) what you want#2016-10-1117:24Alex Miller (Clojure team)it seems more common from specs Iā€™ve written to have s/cat of s/coll-of for functions. For macros, you sometimes have this but I find itā€™s really important in that case to develop the awareness of when you are really needing to drop nested levels - thatā€™s critical to build reusable parts, and often for making workable recursive specs#2016-10-1118:56danielstocktonI think just making this clearer in the guides, in the specing functions section, would help#2016-10-1118:57danielstocktonI found where it's mentioned now, but not when I was trying to work out what was wrong#2016-10-1118:58danielstocktonYou could argue I should have read the guide more thoroughly#2016-10-1121:15bhaganyfor me, it was just forgetting that cat is a regex operator. I didnā€™t feel like the guide could have been much clearer about it.#2016-10-1208:04jmglovIs it possible to spec functions defined in a protocol?#2016-10-1208:04jmglovOr the implementations thereof in a defrecord or reify?#2016-10-1208:16hiredmanyou can fdef a protocol function, and you'll be able to validate it, but instrument may behave oddly#2016-10-1208:18danielstocktonI found instrument doesn't work, I make my protocol functions a simple proxy to another function that I can spec#2016-10-1210:31danstoneHas anyone had any success spec'ing high level components, such as ring handlers? I would like to spec boundaries rather than implementation details in an app I'm working on and it seems like this will be a ton of work.#2016-10-1211:38jmglov@danielstockton That's what I'm doing for now. It would be really nice if instrument worked, though. šŸ™‚#2016-10-1211:38jmglov@alexmiller Is that something worth submitting as a feature request? Or is unlikely to be considered?#2016-10-1211:39jmglov@hiredman So I can do something like this?
(defprotocol Foo
  (bar [this]))

(s/fdef bar
        :args (s/cat :x int?
                     :y int?)
        :ret string?)

(defrecord MyFoo [_]
  Foo
  (bar [_] (str a "+" b))
#2016-10-1211:40danielstocktonIt was asked before on the google group: https://groups.google.com/forum/#!topic/clojure/f068WTgakpk#2016-10-1213:11Alex Miller (Clojure team)I'm not sure it's possible to do#2016-10-1213:12Alex Miller (Clojure team)I'm certainly not opposed to it#2016-10-1214:52jfntnIs there a working equivalent of (s/map-of ::k ::v :kind sorted?) for sorted maps?#2016-10-1215:13Alex Miller (Clojure team)There is sorted-map? predicate#2016-10-1215:13Alex Miller (Clojure team)I think? On phone so going from memory#2016-10-1215:22jfntnHmm I donā€™t think so, at least in alpha12?#2016-10-1216:55clojuregeekTrying to use spec, and have a problem with setting up my spec ... basically i want style? to be one of those two values (for now :one :two) ... but it is not working, i've been studying the docs and i can't figure out where i am doing something wrong .#2016-10-1216:56clojuregeekhttps://gist.github.com/rubygeek/065c2f32147cf09f5148d9fd4f7668c3#2016-10-1216:57jrheard@clojuregeek - i think you want to either do (def style? #{:one :two}) and (s/def ::style style?), or just (s/def ::style #{:one :two})#2016-10-1216:57jrheardi donā€™t think (s/def) expects you to give it a symbol like style? as the first argument, i think it expects a namespaced keyword like ::style#2016-10-1216:58jrheardalso you have ::style?? in there at one point, which is probably confusing things as well#2016-10-1216:58bfabryyes, you've registered the spec under the symbol style? instead of under the keyword ::style?#2016-10-1216:59clojuregeekok let me try some more ..#2016-10-1217:00clojuregeekok i think maybe it was extra ? on style in job not a keyword when i defined style? spec#2016-10-1217:00clojuregeekthanks guys :+1:#2016-10-1217:03clojuregeeknow if i can only stop typing confirm instead of conform šŸ™‚#2016-10-1306:52tianshuIt seems clojure.spec/assert always return original data in clojurescript.#2016-10-1306:53tianshu
(s/assert (s/keys :req-un [::a]) {:b 10})
return {:b 10}
#2016-10-1309:05Oliver George@doglooksgood asserts aren't enabled by default. Try (s/check-asserts true).#2016-10-1309:05Oliver Georgehttps://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec/check-asserts#2016-10-1309:17yendais there a recommended place to put (s/check-asserts true) ?#2016-10-1309:46jeroenvandijkIs this a bug?
(def my-keys [:my.ns/some-key])
(s/def :my.ns/ok (s/keys :req-un ~my-keys)) 
;=> :my.ns/ok
(s/def :my.ns/not-ok (s/keys :opt-un ~my-keys))
; CompilerException java.lang.AssertionError: Assert failed: all keys must be namespace-qualified keywords
#2016-10-1309:47jeroenvandijksome specific behaviour for :opt-un, Iā€™ll try to dig further#2016-10-1311:50jeroenvandijkAfter looking a bit further I fell into a macro trap. I guess at most it is inconsistent behaviour#2016-10-1312:30jeroenvandijkThis terrible hack does what I want
(defmacro dynamic-keys [& opts]
  (let [args (mapcat (fn [[k v]]
                       [k v])
                     (partition 2 opts))]
    `(eval (list `s/keys 
#2016-10-1312:30jeroenvandijkIs there a better way?#2016-10-1312:43odinodinwhat is the preferred convention for fdef specs, should they come before or after the function they spec?#2016-10-1312:45darwin@jeroenvandijk you could write a macro which emits my-keys#2016-10-1312:46darwinbut your solution is fine, under given circumstances, I think it leads to more-readable code#2016-10-1312:47darwininstead of eval you could use ns-resolve and var-get, if you expected just a symbol as arg, I think#2016-10-1312:48darwineval could be a weapon of mass destruction šŸ™‚#2016-10-1312:49immoh@jeroenvandijk You donā€™t necessarily need macros. Iā€™ve been using these to create specs dynamically:#2016-10-1312:50jeroenvandijkah yeah could just use eval, slightly better#2016-10-1312:51jeroenvandijkThanks @darwin and @immoh#2016-10-1320:17mlimotteAre there any examples of how to instrument with a :spec override? For example:
(defn foo [x] (inc x))
(s/def ::x clojure.future/pos-int?)
(s/fdef foo :args (s/cat :x ::x))
; Attempt #1
(stest/instrument `foo {:spec {`foo {:args (s/cat :x zero?)}}})
(foo 1)
=> 2
; Attempt #2
(stest/instrument `foo {:spec {::x zero?}})
(foo 1)
=> 2
#2016-10-1320:47clojuregeekI want to say that my function returns a vector of 1 or more jobs (a map) is this correct?
30 ā”‚ (s/def ::job (s/keys :req [::type ::meta]
  31 ā”‚                      :opt [::payload]))
  32 ā”‚
  33 ā”‚ (s/fdef sample-jobs
  34 ā”‚   :args int?
  35 ā”‚   :ret (s/+ ::job))
#2016-10-1321:25mlimotte@clojuregeek I think you need to use (s/coll-of ...) for :ret#2016-10-1321:27clojuregeekhmm .. i would expect a better doc string?
kraken-consumer.job/sample-jobs
       ([] [x])
       Spec
         args: int?
         ret: (every :kraken-consumer.job/job :clojure.spec/cpred #function[kraken-consumer.job/fn--12178] \
       :clojure.spec/kind-form nil :clojure.spec/conform-all true)
#2016-10-1321:35mlimotteFound the answer to my instrument with :spec question above. Should be done like this:
(defn foo [x] (inc x))
(s/def ::x clojure.future/pos-int?)
(s/fdef foo :args (s/cat :x ::x))
(stest/instrument `foo {:spec {`foo (s/fspec :args (s/cat :x zero?))}})
(foo 1)
=>
ExceptionInfo Call to #'user/foo did not conform to spec:
In: [0] val: 1 fails at: [:args :x] predicate: zero?
:clojure.spec/args  (1)
:clojure.spec/failure  :instrument
:clojure.spec.test/caller  {:file "form-init5306699344209214826.clj", :line 5, :var-scope user/eval24489}
  clojure.core/ex-info (core.clj:4617)
#2016-10-1321:37clojuregeekusing + has a decent doc string
175 ā”‚ kraken-consumer.job/sample-jobs
 176 ā”‚ ([] [x])
 177 ā”‚ Spec
 178 ā”‚   args: int?
 179 ā”‚   ret: (+ :kraken-consumer.job/job)
 
#2016-10-1322:25danielcomptonIs there a way to spec a sequential collection? s/coll-of throws a massive error if you pass a map to it, as it tries to check each mapentry against a spec and displays a failure for each mapentry#2016-10-1322:26jrheards/map-of might be useful for you#2016-10-1322:26danielcomptonI want my spec to be a sequence of values, each conforming to a spec#2016-10-1322:27danielcomptonIf I pass a map, then each mapentry gets evaluated against that spec instead#2016-10-1322:27jrheardhuh!#2016-10-1322:27jrheardwould you mind pasting an example?#2016-10-1322:29jrheards/cat is a good way of speccing sequences, but youā€™re probably already familiar with it so i hesitate to mention it#2016-10-1322:29jrheardi havenā€™t seen the behavior you describe re: coll-of and maps#2016-10-1322:29jrheardam very curious šŸ™‚#2016-10-1322:30danielcomptonI thought about s/cat but didn't think it was for this use case? I could be wrong though#2016-10-1322:30jrheardif iā€™m reading this right, youā€™re passing in {a-map}, and not [{a-map}] - could that be the issue?#2016-10-1322:31jrheardrereading, iā€™ve probably missed something šŸ™‚#2016-10-1322:31danielcomptonI know what the issue is, I'm saying that the spec error message telling me that wasn't very useful when I used s/coll-of, so I was wondering if there was something else?#2016-10-1322:43Alex Miller (Clojure team)coll-of seems like what you want although I'm not entirely positive I understand what you're doing#2016-10-1322:53danielcomptonI want my spec to be a sequential? collection of ::request-maps. I define ::request-map, then create (s/def ::request-maps (s/coll-of ::request-map)). If I pass "abc" and check it against my spec, I get
{:cljs.spec/problems
 [{:path [],
   :pred coll?,
   :val "abc", 
   :via [:day8.re-frame.http-fx/request-maps], 
   :in []}]}
as it fails on the coll? predicate. This is a pretty easy to understand problem. However if I check a single map against this spec, I get a very large error, because the map is treated as a sequence, and each MapEntry in the map is checked against (and fails) my spec ::request-map.
#2016-10-1322:55danielcomptonI can change my spec to (s/and sequential? (s/coll-of ::request-map)) but this seems like a fairly common use case, and wondered if there was another way to spec this more precisely?#2016-10-1323:19hiredmanuse the regex ops, + or *#2016-10-1323:19hiredman(s/* ::request-map)#2016-10-1323:20hiredman(I mean, I don't know, that is just a suggestion, seems nicer than coll-of)#2016-10-1323:27danielcomptonThat's a bit better, thanks. It short circuits earlier, but the error message is still not very illuminating:
{:cljs.spec/problems
 [{:path [],
   :pred map?,
   :val [:method :post],
   :via
   [:day8.re-frame.http-fx/request-maps
    :day8.re-frame.http-fx/request-maps
    :day8.re-frame.http-fx/request-map
    :day8.re-frame.http-fx/request-map], 
   :in [0]}]}
#2016-10-1323:28danielcomptonIt still treats a map as a sequence of map entries#2016-10-1323:38Alex Miller (Clojure team)I think coll-of is the best match for what you are trying to say#2016-10-1323:40Alex Miller (Clojure team)Using s/* by itself without other regex ops is less good#2016-10-1323:41Alex Miller (Clojure team)As it conveys a sequential with more interesting internal structure#2016-10-1323:41danielcomptonDoes it make sense to include maps as valid coll-of's?#2016-10-1323:41Alex Miller (Clojure team)Yes#2016-10-1323:41Alex Miller (Clojure team)They are a collection of tuples #2016-10-1323:41Alex Miller (Clojure team)And that's very useful#2016-10-1323:42danielcomptonIs there scope to add a sequential-of? or something similar?#2016-10-1323:42Alex Miller (Clojure team)No, you could use :kind though#2016-10-1323:43Alex Miller (Clojure team):kind sequential?#2016-10-1323:43danielcomptonNice!#2016-10-1323:43danielcomptonThat's wha I was after#2016-10-1323:43Alex Miller (Clojure team)You may also want :into []#2016-10-1323:44Alex Miller (Clojure team)Can't remember how it will conform without that#2016-10-1323:44Alex Miller (Clojure team)That might already be the default#2016-10-1323:44danielcomptonI think so#2016-10-1323:45Alex Miller (Clojure team)That's used by gen too if you care about that so might double check that too#2016-10-1323:46danielcomptondocs seem to suggest an :into is needed as well here, as I don't know if sequential? can generate?#2016-10-1401:39Alex Miller (Clojure team)I don't know - try it? You can specify a generator too#2016-10-1402:02danielcomptonyep, generating works fine with :kind sequential?#2016-10-1415:04robertofyi, snippets slow down slack a lot. Maybe we should default to creating posts instead of snippets.#2016-10-1415:29samedhiI keep having to with-gen or overspecify my generator in clojurescript as generation just takes too long and eventually times out.#2016-10-1415:29samedhiSpecifically coll? as a :args always causes me to eventually time out#2016-10-1415:29samedhiAnything clever to fix this?#2016-10-1415:30samedhiOh, I mean when using clojure.spec.test/check.#2016-10-1415:37samedhi
> (defn fx [coll] coll)
> (s/fdef fx :args (s/cat :coll coll?))
> (stest/summarize-results (stest/check `fx))
#2016-10-1415:37samedhiAs an example of timing out in cljs.#2016-10-1415:39Alex Miller (Clojure team)Have you tried using the :gen-max option?#2016-10-1415:39samedhinope#2016-10-1415:39Alex Miller (Clojure team)For spec'ing collections that is#2016-10-1415:40samedhiyeah, I saw one example of that in the spec getting started guide, forgot about it.#2016-10-1415:40Alex Miller (Clojure team)You could use (s/coll-of any? :gen-max 3)#2016-10-1417:00spiedeniā€™m looking at doing form validation using spec. any thoughts on mapping from an explain to a slightly more human readable messages to put under fields? canā€™t find anything to put metadata on#2016-10-1417:01spiedenmaybe just a map from spec keys to messages would do#2016-10-1417:06spiedenhmm, looks like iā€™d need to key off the pred symbol actually
user=> (s/explain-data (s/keys :req [::foo]) {::foo ""})
#:clojure.spec{:problems ({:path [:user/foo], :pred not-empty, :val "", :via [:user/foo], :in [:user/foo]})}
#2016-10-1417:25samedhi@alexmiller Thank you for your help. I played around with it since then. Actually, the real issue was that I was including a spec in the :ret part of my s/fdef. This causes the value in :ret in the :fn part of the same s/fdef to be the spec-conformed-return version of the :ret, not the function-value-return version of the return. Kept causing failures that I just couldnā€™t understand. šŸ™‚#2016-10-1417:27samedhitldr; If your spec does not conform to the identity of the value passed to it, you should think about what you are doing when using it in s/fdefā€™s :ret#2016-10-1417:31spiedenhrm, wrapping it in s/and allows you to retrieve the ā€œgroundā€ spec:
cljs.user=> (cljs.spec/explain-data (cljs.spec/keys :req [::foo]) {::foo ""})
{:cljs.spec/problems {[:cljs.user/foo] {:pred not-empty?, :val "", :via [:cljs.user/foo], :in [:cljs.user/foo]}}}
cljs.user=> (cljs.spec/def ::foo (cljs.spec/and ::not-empty))
:cljs.user/foo
cljs.user=> (cljs.spec/explain-data (cljs.spec/keys :req [::foo]) {::foo ""})
{:cljs.spec/problems {[:cljs.user/foo] {:pred not-empty?, :val "", :via [:cljs.user/foo :cljs.user/not-empty], :in [:cljs.user/foo]}}}
#2016-10-1417:32spieden(difference is in :via values)#2016-10-1419:21samedhi@spieden Thanks you for letting me know, I did reach that solution as well. It all makes pretty good sense, it just caught me by surprise that :ret also effected the :ret in :fn.#2016-10-1520:53samedhihttps://gist.github.com/samedhi/3ec2d55f2a9b554c74737b347a911210#2016-10-1520:53samedhi(above) Having a bit of confusion about when specs can be used in :ret and when they can be used in spec/and.#2016-10-1520:55gfredericks@samedhi the :fn on line 24 is not supposed to be a spec#2016-10-1520:56gfredericksit's supposed to be a function that compares the args and the return value#2016-10-1520:56gfredericksand you can leave it off if you don't have anything obvious to check#2016-10-1520:56gfredericksthat might not be your problem, I'm just guessing#2016-10-1520:59samedhi@gfredericks Looking at https://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec/fdef, seems like a spec should be ok.#2016-10-1520:59samedhiOh, you mean within it#2016-10-1521:00samedhiYes, it does state it takes a spec, but it should ā€œcontain predicates"#2016-10-1521:00samedhiinnnteresting#2016-10-1521:01gfredericksI would wager "A spec of the relationship..." is a typo, unless things have changed recently#2016-10-1521:01gfredericksI'll check on that actually#2016-10-1521:01gfrederickswhat's the latest clojure version#2016-10-1521:01samedhi1.9.0-alpha13 I think?#2016-10-1521:02gfrederickslooks like there's 13#2016-10-1521:02samedhiYeah, I just checked and saw that. šŸ™‚#2016-10-1521:04samedhi> Note that :fn specs require the presence of :args and :ret specs to conform values, and so :fn specs will be ignored if :args or :ret are missing.#2016-10-1521:05samedhiUgh, that line alone would have saved me hooours. I donā€™t know what you can do for people who donā€™t read the api docs (sorry), but figuring this out on my own just hurt.#2016-10-1521:06samedhiMaybe a warning if you include a :fn but not :args and :ret#2016-10-1521:13gfrederickslooks like the s/and thing does work, and I don't know why#2016-10-1521:13gfredericksso TIL I guess#2016-10-1521:15samedhi@gfredericks Sounds good, yeah, no idea. Thanks for you help. TIL I should always read the docs right after going through the getting started section šŸ˜¬#2016-10-1605:21samedhi
> (->> (with-meta #{} {:I :am-lost}) (s/conform (s/coll-of strings?)) meta)
{:I :am-lost}
> (->> (with-meta #{} {:I :am-lost}) (s/conform (s/coll-of string?)) meta)
nil
#2016-10-1605:22samedhiI get the behavior, but I wanted to fdef check the metadata after a function call in the :fn. However, the :fn only has the conformed return values, so I donā€™t have the metadata. Any ideas how to do this. Or would you just say I shouldnā€™t include it in metadata?#2016-10-1613:53ggaillardHello everyone! I am going crazy on a cljs.spec strange behavior. I use (s/fdef) and (cljs.spec.test/instrument) in my dev mode to catch non-conform arguments. For a strange reason my spec-ed functions keep beeing called as if a generator was forever running. I have cljs.spec.test in my dependencies but I don't call it anywhere at the momentā€¦ any idea why ?#2016-10-1615:37jetzajacHi! How do I spec named arguments of a function/marco?#2016-10-1616:10Alex Miller (Clojure team)cat#2016-10-1616:35gfredericksjetzajac: s/keys?#2016-10-1616:35gfredericksdepends on what you meant by "named arguments" I guess#2016-10-1617:16jrheard@ggaillard i have the same problem using cljs.spec.test/instrument! iā€™m glad itā€™s not just me#2016-10-1617:16jrheardi canā€™t reproduce it reliably though#2016-10-1617:16jrheardi really hope youā€™re able to figure out whatā€™s going on and open a jira issue#2016-10-1617:16jrhearditā€™s been driving me nuts#2016-10-1617:20ggaillardI'm not crazy! Thank you! ^^ I can't manage to reproduce it too. I'm building a re-frame app and this situation always appear after a page refresh, when React tries to render some spec-ed Reagent components. I searched deeply into tens of stacktraces but found nothing particular šŸ˜ž#2016-10-1617:22jrheardfwiw this happens for me in https://github.com/jrheard/voke, which has very few dependencies#2016-10-1617:22jrheardand does not include re-frame#2016-10-1617:23jrheardbut again i canā€™t figure out whatā€™s going on, mainly because the behavior is inconsistent so coming up with a repro is maddening/difficult#2016-10-1617:39Alex Miller (Clojure team)@jetzajac: oh if you mean kwargs then cat and a nested keys* for the options#2016-10-1617:50jetzajac@alexmiller: thanx! this is what i was looking for#2016-10-1619:15jrheard@ggaillard btw while youā€™re using cljs.spec.test, be wary of http://dev.clojure.org/jira/browse/CLJS-1808 and http://dev.clojure.org/jira/browse/CLJS-1812 ; fixes for both are awaiting review by @dnolen#2016-10-1619:15jrheard(the first one's just a documentation issue)#2016-10-1619:18ggaillardI struggled with the first one (ended up looking the source). Thanks a lot for the second one !#2016-10-1710:53akielHi. Is there something like s/assert which is executed every time? I like to reserve s/assert for checks I want potentially disable in production. But I have some checks I like to execute in all cases.#2016-10-1711:04NiclasHow would one go about specing a function that takes a callback that may take any kind of argument? For ex:
(defn myfunc [k cb]
  (do-something-async k (fn [res err]
                          (check-err err)
                          (cb res err))))
#2016-10-1711:05akielWhy takes the callback any kind of argument?#2016-10-1711:07NiclasIf do-something-async in this example is an un-speced library function where I only care about specing functions in my project and not external ones#2016-10-1711:09akielBut if you know what cb should look like, it would be beneficial to fail fast at your function.#2016-10-1712:31Alex Miller (Clojure team)You can use fspec to spec a function arg and any? to spec an arg that takes any value#2016-10-1712:32Alex Miller (Clojure team)So something like #2016-10-1712:33Alex Miller (Clojure team)(s/fdef myfunc :args (s/cat :k keyword? :cb (s/fspec :args (s/cat :res any? :err any?))))#2016-10-1715:34Niclas@alexmiller Thanks, thatā€™s what I was looking for!#2016-10-1715:45vikeriShould I be able to spec an infinite sequence as an argument to a function? I tried s/every but it got stuck in an infinite loop.#2016-10-1716:02Alex Miller (Clojure team)s/every samples so that should be able to work. Would be interested in seeing more.#2016-10-1716:35Alex Miller (Clojure team)(s/valid? (s/every int?) (range)) ;; => true#2016-10-1720:52akhudekIs there a recommended way to handling function arguments like [a b & [foo]] ?#2016-10-1720:52akhudekI suppose you could do (s/or :two-args ā€¦ :three-args ..)#2016-10-1721:09arohneralso (s/cat :a a :b b :foo (s/? foo?))#2016-10-1721:09arohnerthe choice partly depends on intent#2016-10-1723:52akhudekah, interesting thanks#2016-10-1811:48ewenHi, some spec descriptions return fully qualified symbols while others don't#2016-10-1811:49ewenFor example (s/form (s/every string?)) => (clojure.spec/every string? ...) (s/form (s/tuple string?)) => (clojure.spec/tuple clojure.core/string?)#2016-10-1811:49ewenis this the expected behavior ?#2016-10-1812:40Alex Miller (Clojure team)No and I am working on fixing all of those right now#2016-10-1812:42Alex Miller (Clojure team)This particular one is http://dev.clojure.org/jira/browse/CLJ-2035#2016-10-1812:43ewenok thank you !#2016-10-1819:55decoursinIs there a way to stub clojure.spec.test/check?#2016-10-1819:56decoursin
(clojure.spec.test/check `foo {:stub #{`bar}}) 
doesn't work
#2016-10-1820:06donaldballHas anyone used clojure.spec at runtime, e.g. to validate the arguments of fns at system boundaries?#2016-10-1820:20seancorfieldYes, thatā€™s how we use it @donaldball#2016-10-1820:21donaldballWhatā€™s your technique? Do you instrument the namespaces or fns about which you care, or manually check the args using a precondition or e.g. a custom defn macro?#2016-10-1820:22Alex Miller (Clojure team)@decoursin do that with instrument prior to the call to check#2016-10-1820:24seancorfield@donaldball We explicitly call conform and see whether we get invalid? data back, since we need to map to a set of known error codes that our API returns.#2016-10-1820:25seancorfieldIn a couple of places we use explain-data to help figure out which error code to return (but most are simply "parameter X was not valid").#2016-10-1820:26seancorfieldMost of our specs are data structure specs. We have some function specs. We call instrument when weā€™re testing, and we have a few places where we run check explicitly as well. Mostly weā€™ve been doing the latter manually (in the REPL) so far.#2016-10-1909:16NiclasHow does one set the default number of tests to run with cljs.spec.test/check?#2016-10-1912:24bhagany@looveh based on my reading of the source (specifically, https://github.com/clojure/clojure/blob/master/src/clj/clojure/spec/test.clj#L304), I donā€™t believe the default number is configurable.#2016-10-1912:31bhaganyhopefully I didnā€™t misunderstand you there - if youā€™re looking for how to specify the number of tests to run when you invoke check, youā€™d pass an options map that looks like {:clojure.spec.test.check/opts {:num-tests 500}}#2016-10-1912:44decoursin@alexmiller Thanks Alex! I'll try next opportunity#2016-10-1912:51NiclasThanks @bhagany! Yes, that was my understanding too from the docs, but the call doesnā€™t seem to take the option into account when I test it locally:
(cljs.spec.test/check
  `my-func
  {:clojure.spec.test.check/opts {:num-tests 200}}) =>

[{:spec #object[cljs.spec.t_cljs$spec16362],
  :clojure.test.check/ret {:result true,
                           :num-tests 1000,
                           :seed 1476881391908},
  :sym my-ns/my-func}]
#2016-10-1912:53minimal(st/check `my-func {:clojure.test.check/opts {:num-tests 15}})#2016-10-1912:55Niclas@minimal Yes! That did it#2016-10-1914:38bhaganyAh yes, it is different in clojurescript. I believe it's a difference in test.check iirc#2016-10-1914:42minimalYeah took me while to work it out in cljs#2016-10-1915:51jrheardhttp://dev.clojure.org/jira/browse/CLJS-1808 šŸ™‚#2016-10-1915:52jrheard@dnolen - this is what patch 1 on that issue fixes#2016-10-1915:52jrhearddeleting patch 2, pretend you never saw it#2016-10-1915:57Alex Miller (Clojure team)would be nice to get that fixed#2016-10-1916:08dnolen@jrheard applied to master#2016-10-1916:08jrheardšŸ˜® šŸ˜® šŸ˜®#2016-10-1916:08jrheardthanks! šŸ™‚ my first cljs commit!#2016-10-1916:16dnolen@jrheard nice šŸ™‚#2016-10-1920:34Timanyone using clojure-future-spec ?#2016-10-1921:23darwin@tmtwd yep, me#2016-10-1921:26liamdto use a spec in another namespace do you have to require anything?#2016-10-1921:28borkdudeyes, we are using it too at work#2016-10-1921:49jrheard@liamd - my understanding of the world is that you need to either require the namespace that the spec was defined in, or require another namespace that requires that namespace#2016-10-1921:50jrheardlike, the (s/def ::foo) line is a side effect, and so in order for you to use ::foo somewhere else, you need to ensure that that side effect was run#2016-10-1921:50jrheardthis could be a poor mental model to use, but itā€™s the one iā€™m currently using šŸ™‚#2016-10-2002:16Alex Miller (Clojure team)That's right#2016-10-2002:17Alex Miller (Clojure team)s/def updates the registry so that needs to occur before you can use the spec#2016-10-2009:12mpenethow do you wrap a function that returns values to be used as gen that don't depend and randomisation at test.check level (it's an external lib that already does this (sentence generation))#2016-10-2009:13mpenetI know I can (gen/fmap (fn [_] (some-lib/foo)) (gen/any)) but that seems a bit too convoluted#2016-10-2011:34Alex Miller (Clojure team)gen/return ?#2016-10-2011:35mpenetI don't think so gen/return will always return the same value#2016-10-2011:36Alex Miller (Clojure team)The fact that foo doesn't rely on test.checks notions of randomness is actually a problem (as it can't play in growing, shrinking, or creating reproducible results)#2016-10-2011:37mpenetI know it's a bit "dirty"#2016-10-2011:37Alex Miller (Clojure team)But I think you could also use gen/bind to do this#2016-10-2011:37mpenetbind is from generator to generator, I am not sure this works too#2016-10-2011:38Alex Miller (Clojure team)Yeah, I'm thinking of it as a gen#2016-10-2011:38mpenetI am not sure I understand, you have an example?#2016-10-2011:39Alex Miller (Clojure team)I guess its not any better than what you had with fmap#2016-10-2011:40Alex Miller (Clojure team)I mean you could directly satisfy the test.check requirements to be a generator#2016-10-2011:40Alex Miller (Clojure team)I think it's just a record with a function field#2016-10-2011:41Alex Miller (Clojure team)Something like that#2016-10-2011:41mpenetyep I saw that too, but it's almost the same as the fmap solution in the end#2016-10-2011:41mpenetit's a 2 arg func#2016-10-2011:41mpenetdoesn't feel right to use it either, the "make-gen" function is private for instance#2016-10-2011:42Alex Miller (Clojure team)Well you're starting from the position of doing something you shouldn't :)#2016-10-2011:42mpenettrue
#2016-10-2011:42Alex Miller (Clojure team)So it's going to feel wrong whatever you do#2016-10-2011:42mpenetit's probably not an uncommon need tho#2016-10-2011:43Alex Miller (Clojure team)First time I've seen it asked#2016-10-2011:43mpenetbut I can live with the "hack" for now#2016-10-2011:43Alex Miller (Clojure team)Since you're not going to use the value, I'd pick a simpler generator than any?#2016-10-2011:44Alex Miller (Clojure team)If you're doing the fmap one#2016-10-2011:44mpenetright#2016-10-2011:44Alex Miller (Clojure team)boolean?#2016-10-2011:45mpenetgood idea#2016-10-2011:45mpenetor even (gen/return nil)#2016-10-2011:45mpenetbut it doesn't make much diff#2016-10-2011:45Alex Miller (Clojure team)Yeah#2016-10-2011:45mpenetšŸ˜„#2016-10-2012:23mpenetis there a way to increase "randomness" of num generators in test.gen?#2016-10-2012:23mpenetit's often just +-1#2016-10-2012:24mpenetwell, I guess it's just a matter of gen size actually#2016-10-2012:24mpenetnevermind#2016-10-2012:25odinodinDoes anyone have any tips on how to use clojure.spec/keys on Datomic entities? Looking for a workaround for the following http://dev.clojure.org/jira/browse/CLJ-2041#2016-10-2012:37Alex Miller (Clojure team)I don't know that there is a solution to 2041 btw#2016-10-2012:38odinodincurrently hurting us bad, since we work with entities all over, treating them as maps#2016-10-2012:38Alex Miller (Clojure team)The lighter weight lookup interfaces do not have a way to iterate over entries which keys needs to do#2016-10-2012:39Alex Miller (Clojure team)I know this has been discussed re Datomic but I'm not sure where things will end up#2016-10-2012:39odinodinright, also guess conform would call assoc as well#2016-10-2021:10liamdis there a spec equivalent of annotating types using the s/defn, etc. macros in schema with :- and all that?#2016-10-2021:23liamdor do i just use fdef#2016-10-2021:31liamdalso, so the name of a spec is a key that corresponds to what it must be called in a map, but also is just the name of a thing if youā€™re using it not in a map?#2016-10-2021:45liamdso how do i spec a map where the keys are strings?#2016-10-2021:59jasonjcknuse conformer#2016-10-2022:55bfabry@liamd s/keys requires keyword keys for validating maps. if you don't have keyword keys, you'll need to use map-of, coll-of, custom predicates etc#2016-10-2022:55bfabryand yes, use fdef#2016-10-2023:32jasonjcknone of my dependencies is broken ""call to clojure.core/defn did not conform to spec""#2016-10-2023:32jasonjcknis there a way to disable spec checks?#2016-10-2023:33seancorfieldMost of the libraries "broken" by recent spec versions have already released new versions. What is breaking?#2016-10-2023:34jasonjckna few different deps, i'm trying to upgrade them now, i'll let you know if it's still broken#2016-10-2023:35jasonjcknalso fyi the error mesasge makes it very hard to know who the culprit is#2016-10-2023:35seancorfieldThis page lists the known ones: http://dev.clojure.org/display/design/Errors+found+with+core+specs#2016-10-2023:35seancorfield(and what releases have fixes)#2016-10-2023:36seancorfieldI would have expected the error to fairly clearly indicate the source file containing the illegal defn?#2016-10-2023:39seancorfieldMonger...#2016-10-2023:40seancorfieldCurrent version is 3.1.0#2016-10-2023:45seancorfieldThe problem is the format of :or [auto-connect-retry true] which should be :or {auto-connect-retry true}#2016-10-2023:46seancorfieldLooks like it was reported in early June and fixed in mid-August https://github.com/michaelklishin/monger/issues/142#2016-10-2023:48seancorfieldSo, yeah, youā€™ll need a newer version of Monger than 3.0.2 @jasonjckn#2016-10-2023:48jasonjcknkk, thanks sean#2016-10-2023:50seancorfieldI updated http://dev.clojure.org/display/design/Errors+found+with+core+specs to show 3.1.0 contains the fix for Monger 3.0.2ā€™s breakage.#2016-10-2107:25tengIf I want to make sure that a map does not contain a specific attribute, for example :user/password, how do I do that?#2016-10-2108:12pyrhola specers!#2016-10-2108:12pyrI finally bit the bullet and am moving some of my work to spec.#2016-10-2108:12pyrI have a few questions#2016-10-2108:13pyrMy first one is how to "compose" or programmatically generate specs, I have a multi spec with a lot of repetitive code#2016-10-2108:13pyrwhere each spec in the multi-spec has common members and a few things that vary#2016-10-2108:14pyrso i'd like to be able to do something along the lines of
(defn base-op [& ks] (spec/keys :req (concat [::base1 ::base2] ks)))
#2016-10-2108:15pyrThis doesn't work because spec/keys is a macro#2016-10-2108:15pyrAny hint as to how to approach this?#2016-10-2108:20pyrRight now my method is to go with macros:#2016-10-2108:21pyr
(defmacro base-op [& ks] `(spec/keys :req ~(concat [::base1 ::base2] ks)))
#2016-10-2108:22pyrThis works but doesn't feel right, if there's a better approach, happy to hear about it šŸ™‚#2016-10-2108:25mpenetI guess you can use s/and#2016-10-2108:26mpenetthere's also s/merge for this purpose, but I can't say really, haven't used it much so far#2016-10-2108:29mpenet
(s/def ::foo string?)
(s/def ::bar string?)
(s/def ::base (s/keys :req [::foo]))
(s/def ::stuff (s/keys :req [::bar]))
(s/def ::n (s/and ::base ::stuff))

(s/valid? ::n {::foo "0" ::bar "1"}) =>  true
#2016-10-2109:01mpenetI did it wrong again: (gen/large-integer* {:min 1e6 ...}) -> Doubles (my fault I know), but maybe there could be an assert (or casting) there#2016-10-2109:37pyr@mpenet, thanks!#2016-10-2109:40mpenetIs it possible to "share" a generator value for multiple keys in a map (or values in a spec) -> ex: a user profile with N fields?#2016-10-2109:43mpenettime to look up gen overrides I guess#2016-10-2111:17jetzajacHi! How do I override generators for test/check? doc says we can pass :gen parameter in there, but canā€™t find any example. passing something like (stest/check split-operation {:gen {:onair.ot/count (s/gen :onair.ot/bounded-count)}}) shows that it is not generator which is expected as a value, but what then?#2016-10-2111:18jetzajacgetting error like clojure.test.check.generators.Generator cannot be cast to clojure.lang.IFn#2016-10-2111:20jetzajacthe problem Iā€™m trying to address is actually testing for interger overflow, I donā€™t care about it, but generator for int passes huge numbers to a function, which causes it to fail with overflow exception. bounding value with predicate in spec causes a fail in conforming results =(#2016-10-2111:27jetzajaclooks like (stest/check split-operation {:gen {:onair.ot/count #(s/gen :onair.ot/bounded-count)}}) helps šŸ˜ƒ#2016-10-2118:01hiredmanclojure.spec.gen wraps clojure.test.check.generators, and the way it wraps it results in generators becoming no argument functions that return generators (this is to make the dependency on test.check optional). It can make it kind of confusing about when to use a test.check generator and when to use a function returning a generator, most notably when using the generator combinators in clojure.spec.gen#2016-10-2118:33Alex Miller (Clojure team)the answer is to always use a function returning a generator#2016-10-2118:33Alex Miller (Clojure team)I only find it to be confusing when using both spec.gen and test.check.generators at the same time#2016-10-2118:33Alex Miller (Clojure team)so I mostly try not to do that :)#2016-10-2119:37liamdiā€™m trying to s/exercise a function i specā€™d and iā€™m getting: ExceptionInfo Unable to construct gen at: [] for: what does it mean ā€œ`at: []`"#2016-10-2120:15hiredman
user=> (require '[clojure.spec :as s])
nil
user=> (s/def ::foo (constantly false))
:user/foo
user=> (s/exercise ::foo)
ExceptionInfo Unable to construct gen at: [] for: :user/foo  clojure.core/ex-info (core.clj:4725)
user=> 
#2016-10-2120:15hiredmanyou have a predicate that spec doesn't know how to find the generator for#2016-10-2120:17kennyWhere are you guys putting your clojure.spec.test/check calls? Are they in an is where you check if :result is true?#2016-10-2120:19hiredman'[]' is the path to the spec, so an empty path means the top level#2016-10-2120:24kennyWhen using test.check I would use defspec. Is there some similar integration point with clojure.spec.test?#2016-10-2120:48kennyMaybe something like this?
(defmacro defspec-test
  [name & test-check-args]
  (when t/*load-tests*
    `(def ~(vary-meta name assoc :test `(fn [] (clojure.spec.test/check 
#2016-10-2121:10liamdhm so it canā€™t find a generator for my function#2016-10-2121:11liamdthis is right beneath it:
(s/fdef option-settings->environment-variables
        :args :elb/option-settings
        :ret  :elb/option-setting
        :fn   #(= "EnvironmentVariables" (-> % :ret :option-name)))
#2016-10-2121:11liamdis that enough to give it a generator or am i missing something?#2016-10-2121:19liamdoh i should probably use exercise-fn#2016-10-2121:55liamdi followed this exactly https://asciinema.org/a/87157 to use exercise-fn and get:
IllegalArgumentException No implementation of method: :specize* of protocol: #'clojure.spec/Specize found for class: nil  clojure.core/-cache-protocol-fn (core_deftype.clj:583)
=/
#2016-10-2123:19jrheard@kenny i havenā€™t seen official guidance on this. have you seen the summarize-results function, though?#2016-10-2123:20kennyYes#2016-10-2123:20jrheardiā€™ve seen people assert on its return value in regular deftests#2016-10-2123:21kennyI wrote a version of defspec that uses abbrev-result. So you could do something like
(defspec-test qc-myfn? `myfn)
#2016-10-2123:21jrheardnice#2016-10-2123:21jrheardhavenā€™t seen abbrev-result#2016-10-2123:21kennyIt works for now. I'm curious what the official solution will be#2016-10-2123:22jrheardme too! šŸ˜„#2016-10-2123:22kennyIt's clojure.spec.test/abbrev-result#2016-10-2123:22kennyAlso combined it with clojure.spec/explain-out#2016-10-2123:23kennyclojure.test + generative tests don't fit perfectly -- it's not clear what the :expected and :actual values in the clojure.test report should be.#2016-10-2123:24kennyIn my case I used the :ret for :expected and :clojure.spec.test/val for :actual#2016-10-2216:12cddrHas anyone thought much about using spec definitions to generate avro/protocol buffer/thrift schemas (and corresponding coercion of data in those formats into clojure and back) or is that a fools errand?#2016-10-2217:04cddrI think what I really want to do is associate specs (or perhaps conformers) with "hints" that would document the low-level type implied by some conformer.#2016-10-2219:32donaldballI have had modest success using them as parts of fixed-width file parsers#2016-10-2300:52cddrDoes your approach require the generation of some sort of schema artifact representing the on-disk representation of your data?#2016-10-2322:38bfabry@cddr I fully intend to be using something that does spec->avro schema within the next 6 months. either someone else will write it and I'll use it or I'll write it#2016-10-2322:50cddrWould you want it to infer the avro type from the spec? Or would you be happy to annotate the avro part.#2016-10-2323:10bfabryprobably shoot for from spec. I'm thinking if spec can do this with generators https://github.com/clojure/clojure/blob/master/src/clj/clojure/spec/gen.clj#L128 I can do something similar with avro types#2016-10-2323:14cddr:+1:#2016-10-2407:37mpenet@bfabry: In my small experience with this stuff specs is too "open" to allow conversions like this without hitting tons of problems down the road. Prolly better to limit yourself to a subset of specs, and maybe front all this will a small DSL that would avoid people to go wild with theirs spec#2016-10-2422:35bfabry@mpenet yeah, I definitely wasn't planning on trying to create a "complete" spec->avroschema thing. just register arbitrary predicates against their avro conversion#2016-10-2510:48pviniswhat is the idiomatic way to write specs? in a spec directory? how do you run them? is there a lein spec like lein test? how do you guys do that? i guess specs are not just to be run in a repl while developing..#2016-10-2512:39Alex Miller (Clojure team)Specs are code. You can either put them inline with your other code or in a separate namespace.#2016-10-2512:40Alex Miller (Clojure team)You can use them in a variety of ways#2016-10-2512:41Alex Miller (Clojure team)If you want to use them in your production code you can call s/conform or s/valid?#2016-10-2512:41Alex Miller (Clojure team)If you want to be able to toggle them on or off you can use s/assert#2016-10-2512:43Alex Miller (Clojure team)If you want to use them during testing you can use stest/instrument (to check calls to a function) and stest/check to generatively test a spec'ed function#2016-10-2518:21donaldballI think I may have run into a bug in conform:#2016-10-2518:21donaldball
(s/def ::mapping-offset
  (s/cat :offset nat-int? :length nat-int?))

(s/def ::mapping-offsets
  (s/map-of ::mapping-offset ::mapping-offset))
#2016-10-2518:21donaldball
(s/conform ::mapping-offsets {[0 3] [4 3]}) => {[0 3] {:offset 4, :length 3}}
#2016-10-2518:22donaldballEasy enough to work around using s/tuple instead of s/cat, but figured Iā€™d mention it here. Should I file a bug report?#2016-10-2518:48Alex Miller (Clojure team)no, thatā€™s expected behavior#2016-10-2518:49Alex Miller (Clojure team)you can pass :conform-keys true as additional map-of options to get both conformed#2016-10-2518:49Alex Miller (Clojure team)default is false (as often you donā€™t need it, and thereā€™s a perf cost, and thereā€™s the possibility of conformed keys overlapping and losing data in the conformed map)#2016-10-2518:50Alex Miller (Clojure team)this is in the docstring for map-of too#2016-10-2518:51Alex Miller (Clojure team)Iā€™m assuming that you were objecting to the vals being conformed but not the keys#2016-10-2518:51Alex Miller (Clojure team)if you actually want neither conformed, you can do that with a conformer on the val spec#2016-10-2518:52Alex Miller (Clojure team)or change it to use tuple as you suggested#2016-10-2522:10Oliver GeorgeHey @alexmiller, I think there's a bug in (s/form (s/every pred ...)). (s/form (s/coll-of (s/keys :req-un [:PROJ_CONTACT/NAME]))) returns a pred of s/keys instead of clojure.spec/keys.#2016-10-2522:11Alex Miller (Clojure team)yes, thereā€™s a ticket and a patch waiting for Rich already on that#2016-10-2522:12Alex Miller (Clojure team)http://dev.clojure.org/jira/browse/CLJ-2035#2016-10-2522:13Oliver GeorgeThanks Alex.#2016-10-2522:19Alex Miller (Clojure team)that patch was super annoying with all the macrology - itā€™s hard to tell but I put a lot of work into making that as small a change as it is#2016-10-2522:19Oliver GeorgeThat code is quite tricky.#2016-10-2522:20Oliver GeorgeWhile you're about. I'm interested in finding a good approach for walking through specs to transform. In my case I'd like to use specs to produce a simple version of a datomic pull query.#2016-10-2522:20Oliver GeorgeThis is what I have https://gist.github.com/olivergeorge/8368ab3637d0ade128166de78bac7903#2016-10-2522:21Oliver GeorgeI suspect there's a slightly more artful approach. Any chance you can point me at an example.#2016-10-2522:24Alex Miller (Clojure team)so one thing that is coming (and is behind all my spec form fixes) is specs for s/formā€™s return, basically specs for spec forms, which is the path youā€™re on there#2016-10-2522:24ericfodeHas anyone tried using test.check cljs.spec and core.async together?#2016-10-2522:25Alex Miller (Clojure team)@olivergeorge so I think using spec to conform spec forms is imo a good approach#2016-10-2522:26ericfodeI am having some trouble getting a test that uses promises wired up to be the body of a property. (the example i am extending is here https://github.com/clojure/clojurescript/wiki/Testing#async-testing and here https://github.com/clojure/test.check#clojurescript)#2016-10-2522:27Oliver George@alexmiller Thanks. The spec form specs (!) sound useful. Perhaps those could be the basis for post-walk wrapper or zipper. I'm thinking that might make walking/transforming the spec tree less quirky.#2016-10-2522:28Oliver GeorgeNice to know i'm on the right track. Much appreciated.#2016-10-2522:40donaldballThanks, alexmiller, I should have checked the docstring first.#2016-10-2522:46gfredericksis (s/keys) the best way to express s/map-of-style varargs?#2016-10-2522:47gfredericksassuming my key-spec is keyword? of course#2016-10-2522:50bfabry@gfredericks s/keys* is for varargs#2016-10-2522:51bfabryassuming you meant what I thought you did by that#2016-10-2523:02gfredericksyeah I did#2016-10-2523:02gfredericksin particular I mean (s/keys*) with no args#2016-10-2523:14Alex Miller (Clojure team)Why no args?#2016-10-2523:14Alex Miller (Clojure team)I mean, that's fine but opt-un is useful on the gen side#2016-10-2600:21gfredericks@alexmiller it would be better as a s/map-of but afaik I can't use that in a regex#2016-10-2600:21gfredericksI don't have any explicit keys#2016-10-2600:23Alex Miller (Clojure team)Well then sounds good#2016-10-2600:23Alex Miller (Clojure team)Any registered keys will get checked#2016-10-2601:20gfredericksthat would be bad actually#2016-10-2601:21gfredericksI'm trying to spec gen/hash-map#2016-10-2601:21gfrederickscome to think of it I can just do (s/* (s/cat :k keyword? :gen gen?))#2016-10-2601:21gfredericksand that's perfect#2016-10-2613:34gfredericksSo I'm specing some macros#2016-10-2613:35gfredericksAnd it seems common for there to be macro args where there's a straightforward spec for the runtime value, but the macrotime form could be almost anything#2016-10-2613:36gfredericksSo my first thought is to use any?, but that's unsatisfying since it doesn't tell users anything and doesn't even try to catch problems evident at macroexpansion time#2016-10-2613:37gfredericksBut anything fancier would be somewhat complex#2016-10-2613:37gfredericksSo I'm wondering if it's worth it#2016-10-2613:37gfredericksOr if we should just expect macro specs to have a lot of any?s in them#2016-10-2613:41Alex Miller (Clojure team)yes#2016-10-2613:42moxaj@gfredericks I think your macro specs shouldn't be (can't be?) concerned about runtime semantics#2016-10-2613:42Alex Miller (Clojure team)Iā€™ve spent a fair amount of time looking at this and in general, itā€™s quite common for :ret to not be worth spec'ing#2016-10-2613:42Alex Miller (Clojure team)and in that you should just omit it#2016-10-2613:43Alex Miller (Clojure team)currently macros are skipped during check and instrument only checks :args, so there is not much value in specā€™ing :ret anyways beyond docs#2016-10-2613:44Alex Miller (Clojure team)for :args where some can be scoped but others canā€™t, I think use of any? is fine (you can see some of this in the specs Iā€™ve done in clojure.core.specs)#2016-10-2613:45Alex Miller (Clojure team)and then consider what you are expanding to - sometimes itā€™s better to spec that instead (particularly if it bottoms out at a function)#2016-10-2614:23gfredericksThat makes sense; thanks#2016-10-2615:11quollIs there a suggested approach to building specs in Clojure-1.8 code in anticipation of the 1.9 release? The ā€œalphaā€ label puts managers off systems that are rapidly going into production, but I would rather be writing specs as I write code, rather than trying to go back to older code and updating it after 1.9 comes out.#2016-10-2615:15trptcolini haven't used it, but https://github.com/tonsky/clojure-future-spec is the effort i've heard about#2016-10-2616:56seancorfield@quoll Any chance that you can persuade them that Clojure 1.9 Alpha = Clojure 1.8 + spec?#2016-10-2616:58Alex Miller (Clojure team)not to shoot down a direction towards more spec usage, but that is not exactly true and will become less so#2016-10-2616:59Alex Miller (Clojure team)1.9 is still alpha for good reasons and will not go beta until there are a few more things that have been stabilized. It is still reasonably likely that there will be significant changes in the spec internals.#2016-10-2616:59Alex Miller (Clojure team)most of that would probably not affect the api but it may affect the idea of forms as the information model for specs#2016-10-2617:00Alex Miller (Clojure team)there is a lot of concern that we get certain parts of spec ā€œrightā€ as they will be very difficult to change later#2016-10-2617:01Alex Miller (Clojure team)so if youā€™re worried about investing in something that might change, then your managers are right and you should hold off :)#2016-10-2617:02Alex Miller (Clojure team)if youā€™re worried about using alpha software because it may have known and significant bugs, you should also hold off, because there are some :)#2016-10-2617:04Alex Miller (Clojure team)1.9 beta 1 to us signals that the first part (deciding what features to include and provide) is over#2016-10-2617:04Alex Miller (Clojure team)1.9 rc1 signals that we believe the major bugs have been fixed#2016-10-2617:08Alex Miller (Clojure team)That said, I think the two approaches are to use clojure-future-spec with 1.8 or to separate your specs from your core so that you can build and deploy with 1.8 but test with 1.9 if you like#2016-10-2617:30seancorfieldUnderstood. My point was more that clojure.spec is (almost) the only new thing in 1.9 so the risk of using the current alphas is actually very low, as far as existing 1.8 behavior is concerned.#2016-10-2617:30seancorfieldI like the suggestion to have a 1.8-buildable system and an option to test against 1.9 with spec on top of that.#2016-10-2617:32seancorfieldFor us, weā€™ve used Clojure Alpha builds in production dating back to 1.3 Alpha 7 or 8 in 2011 and weā€™ve found it amazingly stable. We find the benefits of being able to leverage new features far outweighs the risks and I think weā€™ve only had to roll back one or maybe two prerelease builds in those five years.#2016-10-2617:32Alex Miller (Clojure team)@seancorfield weā€™ve had several regressions related to the changes in destructuring and namespaced map syntax#2016-10-2617:33seancorfieldYup, and there are often small regressions or breaking changes in each release but those have either not affected us or been very minor impact issues.#2016-10-2617:35seancorfieldI can certainly understand some companies being alpha-averse thoā€™. With some of the projects I maintain, Iā€™ve encountered companies who wonā€™t even consider an RC build. I find the most risk-averse companies tend to be the ones with the worst test coverage and therefore more likely to be unable to detect a regression in their own codeā€¦ šŸ˜#2016-10-2617:35seancorfieldI think the Clojure/core team has done an incredible job of stability with Clojure releases over the years.#2016-10-2617:37seancorfieldWe always run our test suite against both our regular selected Clojure release and against Clojure master snapshots to catch any new breaking changes early.#2016-10-2702:46cddrI sense that there is some profound wisdom in the "informational vs implementation" section of the spec overview but I fear it goes over my head. Is it a hint to users to not do something with spec that might seem tempting? Or is it a note to implementors to stay away from problems that cannot be solved in the general case. Could anyone provide an example of an implementation decision that should not go into a spec.#2016-10-2702:51cddrAlso it seems like most people talking about it here are using it to spec inputs/outputs of functions and macros. Is anyone using (or planning on using it) to define system boundaries? The note in the same doc about starting a dialog about semantic change seems particularly relevant to this type of usage.#2016-10-2703:07seancorfield@cddr We're using it to spec system boundaries more than function inputs/outputs.#2016-10-2703:08cddrHow do you share the specs between the two systems?#2016-10-2703:08seancorfieldWe're spec'ing our REST API at the user boundary, the domain model boundary, and the persistence boundary.#2016-10-2703:08seancorfieldRight now, those are still in a single process but that is changing.#2016-10-2703:09seancorfieldWe keep the specs mostly separate from the code. We haven't completely decided how to package that up in libraries for sharing across systems, but we expect to have the spec libraries as dependencies to both systems (everything's Clojure).#2016-10-2703:10cddrThose categories are interesting too. Is there any overlap between domain model and persistence for example?#2016-10-2703:10seancorfieldWe're mostly focusing on data -- and generators -- rather than functions.#2016-10-2703:11seancorfield@cddr We're still looking at whether we can (mostly) derive one from the other... we've had some success with that but there are... special cases... shall we say.#2016-10-2703:13seancorfieldSome parts of our API boundary match our domain model better than others šŸ™‚#2016-10-2703:14cddrYeah, it seems like it would be easy to generate a spec from some sort of schema like avro, but a bit harder to go the other way.#2016-10-2703:15cddrThanks for your thoughts. Are you going to be at the conj? I'd love to chat some more about this.#2016-10-2703:15seancorfieldWe are started with a legacy API and evolving a new one, piecemeal, so we don't have the luxury of anything like Swagger at the moment etc.#2016-10-2703:16seancorfieldYeah, I'll be there. I submitted a talk about this topic -- and it got accepted -- but partly for personal reasons I've had to withdraw it.#2016-10-2703:16cddrAh well look forward to seeing you then. šŸ™‚#2016-10-2703:16seancorfieldI love Austin šŸ™‚#2016-10-2712:19ikitommi@seancorfield @ccdr we have been writing the spec->swagger/json-schema conversion, should help in the docs part. Also, the JSON/String conforming of the specs. Trying to make those transparent. #2016-10-2712:22mpenet@ikitommi anything public ?#2016-10-2712:24mpenetalso @alexmiller has been hinting about changes in spec forms (https://clojurians.slack.com/archives/clojure-spec/p1477501089007213), I guess this might (or not) change the situation for these kind of things#2016-10-2712:25mpenetpersonally I am holding my horses until this stabilizes (I am very much interested in having something solid for swagger, json-schema & all)#2016-10-2713:24ikitommi@mpenet nothing really usable as public. There is stuff under metosin/spec-tools (second take on the ->json-schema & original spike on the dynamic conformation). But just planned to put effort on the next two weeks with these. I'm working on the spec->json-spec & spec->string-spec transformation - e.g. generating differently conforming copies of the specs for different formats/use cases. If that works, will throw the dynamic conformations into trash bin. Coercion rules in both from Schema/Ring-swagger. Were you doing the json schema -> spec side? or the same?#2016-10-2713:25ikitommiand thanks for pointing out Alexā€™s message. Would really like to hear how thing will evolve.#2016-10-2713:29mpenetSomething similar (or not), not sure. We have a thin facade that generate both specs with conformers and json-schemas, this way this shields us from changes in the underlying libs for now. Moderately happy about the current situation, hopefully the changes alex mentioned will make this stuff easier.#2016-10-2713:29mpenetgoing from spec to json-schema directly is something we tried but this was brittle, tons of possible breakage unless you set some strict rules of what you can / cannot use in spec#2016-10-2713:31ikitommisounds cool. So, you are describing something in your own way and generate specs out of those? of do you start from plain specs too?#2016-10-2713:32mpenetyes, at the edge we have our own "spec" language#2016-10-2713:32mpenetI think also onyx does something similar#2016-10-2713:32ikitommido you have something of that in public?#2016-10-2713:32mpenetbut hopefully this is temporary#2016-10-2713:33mpenetno not yet, we might release this but no promises.#2016-10-2713:34ikitommithe old (plumatic) schema -> json schema was partly brittle too, and as the json schema (and expecially the swagger version of it) is less powerfull, only parts of the Schemas could be presented in it.#2016-10-2713:35ikitommiwhat did you find brittle in the direct spec->json-schema conversion?#2016-10-2713:35mpenetindeed. same problem#2016-10-2713:36mpenetthe fact that a spec is a predicate makes it too open and hard to "understand", unless you are 100% aware of the limitations you can cause the generation of json-schema to go "wtf is that" to some validators/converters#2016-10-2713:38mpenetdunno what's best between limiting ourselves to a subset our the facade solution. the facade has the advantage of shielding us from potential dependency changes (Schema -> Spec), and allows the devs not to be afraid to break stuff in a dependency they never inspected#2016-10-2713:39mpenetseems like that could be a good solution for compojure-api & related libs imho#2016-10-2713:43ikitommiyes, at least the current spec maps are quite hard to use, was thinking of adding schema-like map -support ~ {::name string?, ::id integer?}#2016-10-2713:44ikitommiin web apis, currently we have lotā€™s of anonymous maps to describe the different paramerer sets for endpoints.#2016-10-2713:48mpenetI think the validators should be more static (for compojure-api & co), using string? integer? is not so good in that context, just keywords (:string :integer etc) that dispatch to a multimethod that knows what to make out of it and act as some sort of registry that's limited to that dsl. Allowing the user to create schemas and register/use them too in that context since they're known to be understandable by the coercion/validation chain later.#2016-10-2713:49mpenetanyway, I am curious to see what will come next#2016-10-2715:27zaneAre conformers intended to be limited in scope? As in, should I use them exclusively for minor transformations, or would it still be idiomatic to use them for larger-scale manipulation?#2016-10-2715:27zaneHope that question make sense. Sorry for being vague.#2016-10-2715:42Alex Miller (Clojure team)they should be used cautiously, particularly when using them with registered specs as you are making decisions for all consumers of your specs (for now and the future, which is long :)#2016-10-2715:42Alex Miller (Clojure team)in general, I think generic data transformation is better done explicitly with functions in normal Clojure ways#2016-10-2716:24seancorfieldThe problem we ran into is that we had a lot of specs for our API that needed to accept a "string that could be converted to something that conforms to another spec" ā€” and so your choice is either to make the spec produce the converted value (using conformer) or run that conversion twice: once to check you have a compliant string, and then again as Clojure code once you have "validated" your string input.#2016-10-2716:26seancorfieldYour other choice is to run the conversion as a "pre-spec" layer and deal with conversion failures outside specā€¦ which is a lot of boilerplate.#2016-10-2716:28seancorfieldGiven Alexā€™s dark warnings about using conformer "extensively" (which we were), we have sort of called a halt to our spec usage in order to see what falls out as the recommended approach for API specs (i.e., specs that must accept strings that essentially conform to other specs after "conversion").#2016-10-2716:29seancorfieldIf that recommended approach is "run the conversion twice", well, fair enough, but it seems a bit of a waste of effort.#2016-10-2716:56Alex Miller (Clojure team)I think perhaps you take my words more apocalyptically than they were intended :)#2016-10-2716:57zaneThanks for the exposition, @seancorfield. That's precisely the situation I'm in.#2016-10-2716:59seancorfield@alexmiller Well, you have repeatedly cautioned against (overuse of) conformer whenever this sort of pattern comes up ā€” and Iā€™m not seeing a recommendation for how to handle it otherwise.#2016-10-2717:00seancorfieldFor us, having the API spec also handle the conversion (via conformer) is the easiest way to do things. But easy != simple and I understand that it complects validation with transformation...#2016-10-2717:04Alex Miller (Clojure team)My point is and has always been: you are making a choice for all future consumers of your registered specs and you should think about the ramifications of this before you use conformers on everything#2016-10-2717:05Alex Miller (Clojure team)thatā€™s it#2016-10-2817:46drewrI'm having a time with clojure.test.spec/check#2016-10-2817:46drewr
{:spec
 (fspec :args (cat :m :runbld.store/es-opts) :ret string? :fn nil),
 :sym runbld.store/make-connection,
 :failure #error {
 :cause "clojure.test.check.generators.Generator cannot be cast to clojure.lang.IFn"
 :via
 [{:type java.lang.ClassCastException
   :message "clojure.test.check.generators.Generator cannot be cast to clojure.lang.IFn"
   :at [clojure.spec$map_spec_impl$reify__13764 gen_STAR_ "spec.clj" 821]}]
 :trace
 [[clojure.spec$map_spec_impl$reify__13764 gen_STAR_ "spec.clj" 821]
  [clojure.spec$gensub invokeStatic "spec.clj" 269]
  [clojure.spec$re_gen invokeStatic "spec.clj" 1565]
  [clojure.spec$re_gen$ggens__14211$gen__14212 invoke "spec.clj" 1554]
  [clojure.core$map$fn__6882 invoke "core.clj" 2739]
  [clojure.lang.LazySeq sval "LazySeq.java" 40]
  [clojure.lang.LazySeq seq "LazySeq.java" 49]
  [clojure.lang.RT seq "RT.java" 525]
  [clojure.core$seq__6416 invokeStatic "core.clj" 137]
#2016-10-2817:47drewrwhich comes from this fdef
(s/fdef make-connection
        :args (s/cat :m ::es-opts)
        :ret string?)
(defn make-connection
  [args]
  (http/make args))
#2016-10-2817:48hiredmanyou are using a test.check generator directly somewhere, maybe for ::es-opts#2016-10-2817:48hiredmanyou need to wrap them in a thunk#2016-10-2817:48drewryeah, I suspected that, but I don't :require test.check anywhere#2016-10-2817:49drewrah wait, I see it now#2016-10-2817:49drewryeah ::es-opts was it#2016-10-2817:50drewrI was getting that exception for a with-gen in a separate place and kept thinking it was the same place#2016-10-2817:50drewr@hiredman thanks kev!#2016-10-2817:50hiredmanšŸ™‚#2016-10-2818:04cap10morganWhat are folks' thoughts on generative testing w/ spec on stateful fns (in this case modifying an atom)? How do you setup enough variations on the state for meaningful tests? Or is the answer that I should go to test.check directly instead of relying on what clojure.spec generates?#2016-10-2818:07bhagany@cap10morgan: I havenā€™t done anything stateful yet, but if I had special generative needs, Iā€™d reach for passing a custom generator in with-gen first. If I understand you correctly, this is what youā€™re reaching for with the idea to use test.check directly.#2016-10-2818:08cap10morgan@bhagany OK, thanks. You answered my next question too. šŸ™‚#2016-10-2818:11seancorfield@cap10morgan Iā€™d try to separate the logic from the state and test the logic separately if possible.#2016-10-2818:12cap10morgan@seancorfield Hmm, that's a good idea. I need to get into the habit more often. Thanks!#2016-10-2818:13seancorfieldI was just having a similar conversation in #component šŸ™‚#2016-10-2818:15drewrhow do you figure out which spec or gen is problematic when you get Couldn't satisfy such-that predicate after 100 tries.#2016-10-2818:16drewrthe stacktrace doesn't provide any clues that I can see#2016-10-2818:19hiredmanI think alexmiller has mentioned that as a pain point, that error comes out of test.check and plumbing the spec information through from spec to test.check and back again hasn't been done#2016-10-2818:19hiredmanI would start looking at your s/ands I guess#2016-10-2818:21hiredmanif I recall, the way generators for s/and specs are constructed is you take the generator for the first spec in the s/and, and then such-that the rest of the specs (as predicates)#2016-10-2818:21hiredmanso if you have a really generic first thing, and more specific predicates afterwards, you can easily end up with a such-that that doesn't work#2016-10-2818:24drewryep, that's exactly what it was#2016-10-2818:24drewr
(s/and string?
         #(.startsWith % "http"))
#2016-10-2818:25drewrI had a string, but it was "foo"#2016-10-2818:41Alex Miller (Clojure team)there is an exactly almost the same as this in the spec guide http://clojure.org/guides/spec#2016-10-2818:42Alex Miller (Clojure team)that shows how to make a custom generator for it#2016-10-2818:45drewr@alexmiller it wasn't that I misunderstood the spec, I just couldn't find the problematic one#2016-10-2818:46drewrthanks for that doc!#2016-10-2902:03steveb8nI have zipper data structure using the techniques described in https://puredanger.github.io/tech.puredanger.com/2010/10/22/zippers-with-records-in-clojure but Iā€™m finding it hard to write a spec that validates child nodes using the protocol. I canā€™t use a map key because different impls of TreeNode (Zippable in my case) may use different keys. Iā€™ll paste an e.g. next....#2016-10-2902:04steveb8nAny suggestions would be much appreciated#2016-10-2904:04grantIs there an easy way to figure out which spec in a large nested spec that is causing "ExceptionInfo Couldn't satisfy such-that predicate after 100 tries." when using (s/exercise ...)?#2016-10-2904:25jrheardnot afaik but iā€™m not an expert#2016-10-2904:27jrheardusually itā€™s down to a string or something that has lots of constraints put on it and is difficult to generate reliably, but you probably already know about that; one gotcha that i donā€™t think is as well-known is that if you have a coll-of thatā€™s got a :kind set? but doesnā€™t have an into #{}, that can cause the same couldnā€™t-satisfy-after-100-tries behavior#2016-10-2904:28jrheardmight not be your issue, but i always feel compelled to mention it when people have this problem because it happened to me and took me a while to figure out#2016-10-2904:30grantAh, good tip on the set thing, I haven't run into that yet, but I could see making that mistake.#2016-10-2904:37seancorfieldIt doesn't really help you but I highly recommend trying to s/exercise each spec as you write it and build up complex specs in terms of other specs (not just predicates).#2016-10-2904:38seancorfieldPredicates tend to be opaque when you combine them into specs. If you define specs for those predicates, when you combine the specs, you get better error reporting.#2016-10-3021:34drewrI'm having trouble tracking down a :clojure.spec/unknown#2016-10-3021:35drewrthe :clojure.spec.test/caller, :sym, and :via all point to different things, it doesn't seem to make any sense#2016-10-3021:59cddrFolks interested in clojure-spec might be interested to know about some prior art in the pharma industry. They have an organization called "CDISC" that defines a collection of XML standards for defining clinical databases for pharma trials. Their "operational data model" has an info model quite similar to spec. It used to be private only to members but looks like they've opened it up recently: https://www.cdisc.org/standards/foundational/odm#2016-10-3022:25mac@cddr How is this different from any other XML Schema ?#2016-10-3022:27cddrThe bit that spec borrows is the way it defines "ItemDef"s independently of their containers#2016-10-3022:30mac@cddr Ok...#2016-10-3112:03tcouplandI feel like there's a missing flag on s/keys, something like 'complete' or {:allow-unspecified false} as an option. A couple of times while using s/check i've spelled something wrong, in the spec, or set a key badly in the transform code, and spec isn't yelling at me about it, despite everything being spec'd up, it comes down to this for me:
(s/def ::key integer?)
(s/def ::other integer?)

(s/def ::map
  (s/keys :req [::key]
          :opt [::other]))

(s/valid? ::map {::key 1 ::others "2"}) => true
I've clearly got this wrong (::others instead of ::other), but it's really easy to just not notice something like this, but if I could do this:
(s/def ::map
  (s/keys :req [::key]
          :opt [::other]
          :complete true))
and have spec shout if it see's any key it's not ready for, then I'd be getting more out of my spec's! Just something that's bothered me this morning as i realised I'd mistakenly pluralised a key word šŸ™‚
#2016-10-3112:04tcouplandhopefully someones going to tell me it already exists somewhere now!#2016-10-3114:00donaldballThe usual workaround is to combine the s/keys spec with an s/map-of spec#2016-10-3114:02donaldball(I agree that a :complete flag would be a welcome addition to the s/keys macro, but Rich seems committed to open maps as the preferred general data structure)#2016-10-3114:06Alex Miller (Clojure team)specs always make positive statements about data, not negative ones and we do not plan to add anything to s/keys along these lines#2016-10-3114:06Alex Miller (Clojure team)however there is something weā€™re considering that would allow making these additional assertions easier#2016-10-3114:27tcoupland@donaldball that does work, but is pretty ugly isn't it. Sound like a 'wait and see' situation#2016-10-3114:46donaldballItā€™s not a hard macro to write, a keys variant that allows a complete? flag#2016-10-3114:48tcouplandyeah i just started bashing it out šŸ™‚#2016-10-3114:48drewr
{:clojure.spec/problems
  [{:path [:args :idx],
    :pred :clojure.spec/unknown,
#2016-10-3114:49drewrcan anyone point me in the direction of tracking down why that's unknown?
#2016-10-3114:53drewrI get a caller, but I go to that line, and it's downstream from the function that :sym references šŸ¤”#2016-10-3114:53drewrso I must be misunderstanding something fundamental#2016-10-3115:23Alex Miller (Clojure team)the fact that itā€™s printing unknown there is a known issue with describing preds#2016-10-3115:23Alex Miller (Clojure team)so that part is a known bug#2016-10-3115:25Alex Miller (Clojure team)but you have the path so it should be pretty narrowly scoped to where the pred is failing?#2016-10-3115:25drewrok, that helps, so go through and perhaps exercise my specs to see if anything's funny?#2016-10-3115:25drewrI'm probably doing this the worst way, by trying to migrate from schema instead of using on a fresh codebase#2016-10-3115:26Alex Miller (Clojure team)as with most programming things, itā€™s best to start small and build up#2016-10-3117:22drewras a general comment, the jump to mandated generative testing is a bit of a leap#2016-10-3117:22drewrmaybe I'm doing something wrong, but I don't always need this#2016-10-3117:23drewrwith a lot of side-effects, traditional unit-testing can work just fine, and I just want to make sure functions deep in the call stack are getting the right arguments and returning something valid#2016-10-3117:24drewrmy impression after a couple weeks is that spec makes this some of this more difficult over, say, schema#2016-10-3117:24drewrand maybe that's the intent: lower-level, more correct, more powerful, etc., it just also means there aren't yet companion libraries to help with the porcelain#2016-10-3117:25drewrmaybe I'm just spoiled by clojure's quality... alpha is usually good enough šŸ˜„#2016-10-3117:30donaldballYou donā€™t need to buy into generative testing to use clojure.spec. instrument with traditional example-based tests is pretty useful.#2016-10-3117:31drewr@donaldball that was my early reaction as well, but I lost the ability to check return values, which I get, that's for check, but then that forces me into the generative style#2016-10-3117:32drewrhappen to have a piece of code you transitioned that you're happy with?#2016-10-3117:33donaldballAlas, nothing public, sorry#2016-10-3117:34donaldballIIRC clojure.java.jdbc uses this to good effect tho#2016-10-3117:36jrheardi also kinda wish instrument checked return values#2016-10-3119:20drewr@jrheard this thread was helpful in describing the philosophy https://groups.google.com/d/msg/clojure/JU6EmjtbRiQ/WSrueFvsBQAJ#2016-10-3119:21jrheardthanks for the link, havenā€™t seen this#2016-10-3119:21drewrand I think I agree, I just have the same desire as this guy https://groups.google.com/forum/#!msg/clojure/RLQBFJ0vGG4/E0tHqVyQBgAJ#2016-10-3119:54mattlyis there any way to view those google groups links without having a google account?#2016-10-3119:55shaun-mahood@mattly: The link works for me with no google account signed in#2016-10-3119:55drewr@mattly I brought one up in incognito#2016-10-3119:55mattlybah, ok, so it wants me to sign in with one of the google accounts I'm signed in as#2016-10-3119:56mattlywhich doesn't work because I signed into my google accounts in the wrong order#2016-10-3119:56mattly...sorry, wrong place for this rant#2016-10-3119:56drewrI use chrome profiles, separate windows, yada yada#2016-10-3121:02zane
(s/def ::select
  (s/coll-of (s/or :attribute string?
                   :subselect (s/every-kv string? ::select :count 1))))
(clojure.pprint/pprint
 (s/conform ::select
            ["1"
             {"2" ["2.1"
                   {"2.2" ["2.2.1"]}]}]))
;; => [[:attribute "1"] [:subselect {"2" ["2.1" {"2.2" ["2.2.1"]}]}]]
Why does that inner ::select clause not tagged the way the outer one is?
#2016-10-3121:29seancorfieldYou need conform-keys#2016-10-3121:31seancorfieldHmm, :conform-keys is only listed as an option for map-of but I would expect it to work here too...#2016-10-3121:33zaneWhy would I need conform keys? "2.1" is being conformed by the s/or.#2016-10-3121:34zaneIt's like conforming stops at the recursive step.#2016-10-3121:34seancorfieldNope, only works with map-of ā€” but if you change :subselect to be (s/map-of string? ::select :count 1 :conform-keys true) then you get this
[[:attribute "1"]
 [:subselect
  {"2"
   [[:attribute "2.1"] [:subselect {"2.2" [[:attribute ā€œ2.2.1ā€]]}]]}]]
#2016-10-3121:35zaneI still don't understand why :conform-keys has anything to do with this.#2016-10-3121:35zaneIt's the values that need conforming.#2016-10-3121:35seancorfieldYeah, I think I misunderstood what you were saying didnā€™t work..#2016-10-3121:36seancorfieldIs the output I got with map-of what you want?#2016-10-3121:36zaneI would just expect that in my original example "2.1" would be [:attribute "2.1"].#2016-10-3121:37zaneWhy is the recursive s/or not being conformed?#2016-10-3121:37seancorfield(Youā€™re right, you donā€™t need :conform-keys but you do need map-of, not every-kv)#2016-10-3121:37zaneIt seems like it happens, but only sometimes.#2016-10-3121:37seancorfieldPer the docs, every-kv does not conform all values ā€” map-of does.#2016-10-3121:37seancorfield"Unlike 'every-kv', map-of will exhaustively conform every value."#2016-10-3121:38zaneAhhh.#2016-10-3121:38zaneI missed that in the docs.#2016-10-3121:38seancorfield"Note that 'every' does not do exhaustive checking, rather it samples coll-check-limit elements. Nor (as a result) does it do any conforming of elements.ā€ ā€” and every-kv is like every#2016-10-3121:39zaneGot it.#2016-10-3121:44zanePhew. Thanks, Sean.#2016-10-3121:58seancorfield(I must admit, I find it counter-intuitive that every / every-kv do not in fact check every element but coll-of / map-of do)#2016-10-3121:58seancorfieldAfter all, they have every in their name so youā€™d sort of expect them to check every item...#2016-10-3121:59jrheardheh yeah#2016-10-3122:19bhaganyit is the least-intuitive thing. I wish I understood the rationale.#2016-10-3122:27zaneIt's heartening that I'm not the only one. šŸ˜…#2016-10-3122:59Alex Miller (Clojure team)"every" is an assertion of belief#2016-10-3122:59Alex Miller (Clojure team)Not a statement of what they do#2016-10-3123:24seancorfield^ rationale @jrheard @bhagany @zane#2016-11-0103:33settingheadI'd like to conform this vector [["John" 20] ["Jane" 30] ["Bob" 40]] into [{:name "John" :age 20} {:name "Jane" :age 30} {: name "Bob" :age 40}]. how should I write my spec?#2016-11-0106:23curlyfry@settinghead
(def john-jane-bob [["John" 20] ["Jane" 30] ["Bob" 40]])
(s/def ::person (s/cat :name string? :age integer?))
(s/def ::persons (s/coll-of ::person :kind vector?))
(s/conform ::persons john-jane-bob) =>  [{:name "John", :age 20} {:name "Jane", :age 30} {:name "Bob", :age 40}]
#2016-11-0115:11dominicmHow does clojure-spec work with the reloaded workflow / tools.namespace? I notice both that: 1. https://clojurians-log.clojureverse.org/clojure-spec/2016-09-09.html#inst-2016-09-09T18:34:11.000362Z 2. specs aren't/can't be, deleted or unloaded in any way. Any success with getting it to work in some way?#2016-11-0115:40zaneMy experience has been that needing to unload a spec is a rare occurrence.#2016-11-0116:46hiredmanis there something like s/keys, but instead of looking up specs via global names, I can pass it a spec for the value of each key?#2016-11-0116:51hiredmanI have these tests that result in a data structure that is sort of a log of activity, and the result of the test is determined by checking that log#2016-11-0116:53hiredmanthe log is a sequence of maps, and it has a :messages key, and at each entry in the log, I know that key should have a specific value#2016-11-0116:54hiredmanif I have to define the value globally (use s/def and s/keys) then all I can say is that it could be any of a number of different values#2016-11-0116:55hiredmanah, maybe s/map-of and s/merge#2016-11-0117:03hiredmannope#2016-11-0117:06bfabry@hiredman what do you mean "at each entry in the log", like you dynamically know what it should be or you have a list of things it should be in certain circumstances? because if the latter you could always use :req-un#2016-11-0117:07hiredman:req-un still looks up specs via a global name in the spec registry#2016-11-0117:07hiredmanI have more refined information locally than globally#2016-11-0117:08hiredmanglobally all I can say is the value for the key should be either x y or z, but locally I know exactly what the value for the key should be#2016-11-0117:11settinghead@curlyfry thanks#2016-11-0117:11bfabryright, but I meant you can define multiple versions of :messages using :req-un#2016-11-0117:11hiredmanyeah#2016-11-0117:12bfabry(s/def ::messages-1 (s/keys :req-un [:messages-1/message])) and then switch in the spec you need later on#2016-11-0117:14hiredmanI was hoping to use spec as a sort of a data regex to validate these test results, but having to globally define map validators is a pain#2016-11-0117:15hiredmanlike if you had to globally define your capture groups before you could use them in a regex engine#2016-11-0117:18bfabryI see your point#2016-11-0117:21hiredmanand if I had followed "best practices" and namespaced the :messages key, that work around wouldn't work at all#2016-11-0117:25hiredmanalso clojure.spec has a declare of map-spec, which is never defined https://github.com/clojure/clojure/blob/master/src/clj/clojure/spec.clj#L348 maybe a left over from a previous version#2016-11-0117:28hiredmanI guess I can build something using s/alt#2016-11-0118:26madstap
(s/fdef update*
    :args (s/cat :m map?
                 :k keyword?

                 ;; I want to say:
                 ;; The function needs to take at least one arg,
                 ;; the current value in the map,
                 ;; and can take zero or more extra arguments.
                 :f (s/fspec :args (s/cat :value-in-map any?
                                          :args (s/* any?)))

                 :args (s/* any?)))

  (defn update* [m k f & args]
    (apply update m k f args))

  (test/instrument)

  (update* {:a 10} :a inc)
#2016-11-0118:26madstapHow do I do this in spec?#2016-11-0118:28madstap
;;=> Call to #'foo.core/update* did not conform to spec: In: [2]
  val: (nil) fails at: [:args :f] predicate: (apply fn)
  :clojure.spec/args ({:a 10} :a #function[clojure.core/inc])
  :clojure.spec/failure :instrument :clojure.spec.test/caller
  {:file "form-init8332152340913865653.clj", :line 342, :var-scope
   foo.core/eval12509}
#2016-11-0118:31zane@madstap You're probably better off just using a function.#2016-11-0118:31madstapLike just :f ifn??#2016-11-0118:36hiredmanmaybe something with http://clojure.org/guides/spec#_multi_spec#2016-11-0118:46madstapFrom that example I'm having truble seeing how I could apply multi-spec to my problem#2016-11-0118:47madstap@hiredman could you explain?#2016-11-0118:50hiredmanmaybe not, multi-spec would only be useful if you knew the set of keys you would be pass to update*#2016-11-0118:52hiredmanI guess that is not entirely true, if you knew the set of keys entirely before hand, you could use an s/alt or s/or, the advantage using multi-spec would be that choice would be open ended#2016-11-0118:52hiredmanso easier to extend#2016-11-0118:54hiredman
(defmulti update-multi-spec (fn [m k fun & args] k))

(s/fdef update*
        :args (s/multi-spec update-multi-spec)
        :ret any?)

(defmethod update-multi-spec ::a [m k fun & args]
  (s/cat
   :map (s/keys :req [::a])
   :key #{k}
   :fun (s/fspec :args ...)
   :args ...
   ))
#2016-11-0119:09madstapI think I explained myself badly, what I want to do is say that the function passed to update needs to take at least one argument, and may take more.#2016-11-0119:17madstapYour multi-spec example helped me grok why multi-spec is useful, though, thanks#2016-11-0119:36hiredman(s/+ any?)#2016-11-0119:41madstap@hiredman Doesn't work..
(s/fdef update*
    :args (s/cat :m map?
                 :k keyword?
                 :f (s/fspec :args (s/+ any?))
                 :args (s/* any?)))

  (defn update* [m k f & args]
    (apply update m k f args))

  (test/instrument)

  (update* {:a 10} :a inc)
#2016-11-0119:42madstap
Call to #'foo.core/update* did not conform to spec: In: [2]
  val: (nil) fails at: [:args :f] predicate: (apply fn)
  :clojure.spec/args ({:a 10} :a #function[clojure.core/inc])
  :clojure.spec/failure :instrument
  :clojure.spec.test/caller {:file
                             "form-init8954135211496312533.clj", :line 478, :var-scope
                             foo.core/eval13250}
#2016-11-0119:43hiredmanyeah, because any? is generating bad inputs for the function#2016-11-0119:43hiredmanthe way test check validates the function is it generates inputs for it#2016-11-0119:44hiredmanyour function is inc, in the spec you say it can take an any?, so spec is checking that by feeding it all kinds of values#2016-11-0119:51madstapRight, what I wan't to say that this function needs to take at least one arg, but doesn't necessarily take any more args.#2016-11-0119:52madstapIn this case that passing (fn [] (rand-int 10)) to update will fail, but inc will work, as will (constantly true).#2016-11-0119:57hiredmanthe problem is when you turn instrumentation on, spec is validating inc against your fspec#2016-11-0120:00hiredmana spec that is general enough isn't going to be valid against a specific function, you will generate very general inputs that the specific function won't handle#2016-11-0120:01hiredmanI think the best you could do would be a multi-spec, with a defmethod for every function/key combination you pass to update*#2016-11-0120:26madstapGot it, thanks#2016-11-0209:55vandr0iyHi, #clojure-spec! came to ask: what's the difference between clojure.spec/map-of and clojure.spec/every-kv?#2016-11-0212:46bhagany@vandr0iy map-of checks all the entries of the map, while every-kv samples a subset of the entries#2016-11-0212:47bhaganyevery-kv is designed to be used with maps that may be large#2016-11-0212:47vandr0iynice explanation, thanks @bhagany !#2016-11-0213:06lopalghostAny thoughts on writing separate specs for input data that needs to be coerced, vs using spec/conformer?#2016-11-0213:06mpenetcoerce separately from your main specs yes, with or without spec#2016-11-0213:07mpenetit gets very messy otherwise imho#2016-11-0213:07lopalghostThat's what I'm finding to be the case#2016-11-0213:09lopalghostThe only problem is I'm duplicating my validation logic, because I want to be able to detect invalid data at the point of entry#2016-11-0213:09lopalghostI'm using spec in both cases because the generative testing is super-helpful#2016-11-0213:12mpenetyou can create predicates separately that you re-use on both specs#2016-11-0213:12mpenetbut yes, there will be duplication here and there#2016-11-0213:36drewrwhat are you all doing to cleanup after check on a fn with side effects?#2016-11-0213:36drewrit doesn't look immediately possible with :fn unless I pass context back through the return value#2016-11-0213:37drewrI suppose I could with-redefs in there, but that doesn't feel right#2016-11-0214:01Alex Miller (Clojure team)@vandr0iy additionally, every-kv will not conform its keys or values (because it doesnā€™t check all of them) whereas map-of will conform all values (and optionally all keys)#2016-11-0214:02Alex Miller (Clojure team)@drewr donā€™t write fns with side effects? :)#2016-11-0214:02Alex Miller (Clojure team)seriously though, you might look at the stub and replace options on stest/instrument#2016-11-0214:03drewryeah, would love to avoid them, but have to do things other than burn cpu šŸ”„#2016-11-0214:03drewrcool, didn't notice those opts on instrument#2016-11-0214:03Alex Miller (Clojure team)particularly for the case where you are checking a function that calls a side-effecting function#2016-11-0214:03Alex Miller (Clojure team)you can stub or replace it instead#2016-11-0214:06drewrI didn't think to look at instrument, though, because this goes farther than just args checking#2016-11-0214:07drewrseems like something that would happen with test.check... ie, a function that somewhere ends up writing a value to a db, and I need to make sure that value is deleted (or the db is removed)#2016-11-0214:07Alex Miller (Clojure team)you should instrument, then check#2016-11-0214:07Alex Miller (Clojure team)then unstrument#2016-11-0214:08drewrright now I'm doing the whole enumerate-namespace -> check -> summarize-results dance, sounds like I need to break out of that pattern#2016-11-0214:09Alex Miller (Clojure team)yes#2016-11-0214:27gfredericks@lopalghost I played around with those issues in https://github.com/gfredericks/schema-bijections; I'd love to see that sort of thing implemented for spec#2016-11-0215:01jrheardwhatā€™s the alternative to the pattern @drewr gives above?#2016-11-0215:03jrheardi havenā€™t used the stub-and-replace options alex mentions, so maybe you guys just mean that using them necessitates breaking out of the pattern he gives#2016-11-0215:03jrheardjust curious šŸ™‚#2016-11-0215:04drewr@jrheard I'll give you an example when I have one šŸ˜‚#2016-11-0215:15lopalghost@gfredericks I think spec/conformer is a pretty good solution to the problem of coercing back and forth, at least #2016-11-0215:16lopalghostThe problem is you lose information about the input, making explain and check less useful#2016-11-0215:17lopalghostI've tried writing some DSLs to generate specs, but I was never really happy with them#2016-11-0215:19gfredericksYeah, I remember an earlier conversation that ended up recommending mapping to new specs rather than just conforming#2016-11-0215:20gfredericksBut spec isn't good at, e.g., describing maps with string keys, though :/#2016-11-0215:25mpenetanything other than keyword keys (sadly)#2016-11-0215:27mpenetyou end up having to resort to weird, half working, set + map-of hacks#2016-11-0215:27zaneIt feels like passing around conformed values is discouraged. Is that correct?#2016-11-0215:29mpenetI often wonder why namespaced keys couldn't just be a collection of keywords, since that's just values#2016-11-0215:30mpenetI guess there's a good reason behind this (interning magic leading to better perf ?)#2016-11-0215:31lopalghostI don't like set + map-of because it makes explanations less clear#2016-11-0215:31mpenet[:some-ns :some-key]#2016-11-0215:31lopalghostIt might be better to coerce the keys in a separate step#2016-11-0215:31mpenet@lopalghost more importantly it breaks link between key and value#2016-11-0215:31lopalghostRight#2016-11-0215:32mpenetit just guarantees the key is here#2016-11-0215:37mpenetso yeah either coercion, or it might be better to just have a custom predicate to validate such maps#2016-11-0216:30Alex Miller (Clojure team)you can use every on maps to describe the map entries as k-v tuples#2016-11-0216:31mpenetneat, I didn't think of that one#2016-11-0216:32Alex Miller (Clojure team)like (s/every (s/or :name-entry (s/tuple #{ā€nameā€} string?) :id-entry (s/tuple #{ā€idā€} int?)) :kind map? :into {})#2016-11-0216:32Alex Miller (Clojure team)but whatever orā€™ed entry types you need#2016-11-0216:33Alex Miller (Clojure team)for non-kw key maps, this can be a good approach#2016-11-0216:33Alex Miller (Clojure team)(thatā€™s how every-kv and ultimately map-of are actually implemented)#2016-11-0219:45nwjsmithWhere can I vote for adding docstrings to specs?#2016-11-0219:48nwjsmithI think I saw it being discussed here a few weeks ago, and I though ā€œwho would ever want to do that?ā€ Turns out I am someone who would want to do that.#2016-11-0219:50jrhttp://dev.clojure.org/jira/browse/CLJ-1965#2016-11-0220:14potetmIs it appropriate to give more rationale for that ^ ticket in a comment on JIRA?#2016-11-0220:15potetmI've thought about it a little bit, and I think there's a really good justification for adding it.#2016-11-0222:21jasonjcknis there a general recommendation for versioned schema?#2016-11-0222:21jasonjcknmulti spec dispatching on version tag?#2016-11-0222:21jasonjcknwhat if i'm evolving my spec over time, deprecating fields, introducing new ones, changing existing ones.#2016-11-0305:29Alex Miller (Clojure team)@potetm feel free, although I donā€™t know that there is much need :)#2016-11-0305:30Alex Miller (Clojure team)@jasonjckn backwards compatible changes do not necessarily require a new spec (for example, s/keys will always validate all registered keys, so adding new ones does not require a spec change)#2016-11-0305:31Alex Miller (Clojure team)@jasonjckn breaking changes should mean a new spec#2016-11-0305:32Alex Miller (Clojure team)I expect this area will receive more attention as its something Rich has thought about a lot#2016-11-0305:34Alex Miller (Clojure team)and heā€™s really serious about not just ā€œchangingā€ specs but actually leaving the existing one and creating a new one (::person2, ::person3, etc)#2016-11-0305:34jrheardthe evils of ā€œusing the same name to refer to different things"#2016-11-0305:34Alex Miller (Clojure team)hello immutable data#2016-11-0305:34jrheardšŸ˜„#2016-11-0305:56ikitommihas anyone managed to do the ā€œtake a spec, walk it and transform it to another (differently conforming) specā€? I tried, but for s/keys it seems hard/impossible. Have already copied much of the clojure.spec internals into my project for this. With un-keys, itā€™s easy, with the non-un keys, itā€™s not: the key names should remain same, with a different implementation.#2016-11-0305:59ikitommiwould be nice, if there was a map-kind of Spec in clojure.spec that would allow separate keys and values - I could just swap the values in this case. Could there be such a thing under the hoods of s/keys @alexmiller?#2016-11-0318:30gfredericks@ikitommi having a consistent meaning for a namespaced keyword is one of the core ideas of clojure.spec#2016-11-0318:56zaneikitommi's question relates to my earlier one about whether conformed values are more or less intended to remain internal to the function in which they were conformed.#2016-11-0318:56zaneWhen used idiomatically, anyway.#2016-11-0319:05ikitommi@gfredericks true that, but for external input, there is a need to conform/coerce values differently, based on capabilities of the wire format. Stuarts thought was to transform raw specs into new specs, with different conformers.#2016-11-0319:06zaneHow would that preclude you from using different keys for the new specs, ikitommi?#2016-11-0319:07ikitommiwhat if I want to expose a fully-qualified key?#2016-11-0319:09ikitommifor unqualified keys, I can just create a new spec key, e.g. :mydomain/age => :JSON.mydomain/age, as both can be mapped to the use the :age key.#2016-11-0319:41kuzmin_mHi! I play with spec and found one dangerous point. I use s/fspec to specify high-order function foo with bar argument. In several cases I may call foo with bar that have some side effects. When instrument is enabled then bar will be called more than onсŠµ because spec will check bar. For instance I may write smth like that: (foo delete-user-by-id) and debug it many hours. I can check only bar arity but stest/check will not work.
(ns spec.func.example
  (:require [clojure.spec :as s]
            [clojure.spec.test :as stest]))

(defn foo [bar]
  (bar 42))

(s/def ::bar (s/fspec :args (s/cat :x integer?)
                      :ret integer?))

(s/fdef foo
  :args (s/cat :bar ::bar)
  :ret integer?)

(stest/instrument)

(foo #(do
        (println %) ;; print more than once !!!
        (inc %)))

#2016-11-0319:53drewr@kuzmin_m I've been dealing with exactly this#2016-11-0319:54drewryou might not want to use check at all, but instrument with its :replace option, and then call the function normally#2016-11-0319:58kuzmin_mYes. But people are careless. If something is breakable itā€™ll be broken.#2016-11-0320:00kuzmin_mI may forget specify replace option#2016-11-0320:00drewrnot sure what you mean#2016-11-0320:02kuzmin_mI need an extraordinary action. I need remember about s/fspec behaviour and specify instrument option.#2016-11-0320:13kuzmin_mMay we write a function that replace s/fspec by fn? when we use instrument? We donā€™t call fn and we only check an argument type in this case.#2016-11-0320:14bfabry@kuzmin_m that would make instrument significantly less useful#2016-11-0320:16bfabryfwiw, I think there should probably be more explicit handling of functions with side-effects for both fspec and fdef, and I'm hopeful that something is coming#2016-11-0320:18kuzmin_mSure! But it only about fspec, not about fdef.#2016-11-0320:20bfabrythey're kinda the same thing though, fdef as far as I understand it is just (s/def name (s/fspec blah))#2016-11-0320:23kuzmin_mYes. I just write about one case: s/fspec for function argument.#2016-11-0320:24bfabryright, I just mentioned fdef because you can use the name of something fdef'd somewhere in place of an fspec and you'll get the same behaviour. but anyway you're right that it would be on fspec where things need to be handled#2016-11-0320:25kuzmin_mright :+1: I forget about that behavior šŸ˜ž#2016-11-0320:27kuzmin_mItā€™s just alpha version. I hope that we can fix this issue.#2016-11-0321:41mrcncis there a ā€œbest practicesā€ way to run spec tests from lein?#2016-11-0321:41mrcnci found this https://gist.github.com/tgk/be0325e7b78bc692ad6c85ef6aca818d#2016-11-0323:09jrheardhave you seen summarize-results? https://github.com/clojure/clojure/blob/master/src/clj/clojure/spec/test.clj#L448#2016-11-0323:11jrheardi havenā€™t seen official guidance on the one true specifically correct invocation of clojure.spec.test/check, but when i dig around in github search i usually see people write clojure.test assertions like (is (= 0 ((summarize-results foo) :failures))) or whatever#2016-11-0323:11jrheardhandwaving here, but you get the idea#2016-11-0323:53kennyIt works well with Cursiveā€™s run tests REPL action#2016-11-0323:54kennyExample usage:
(t/defspec-test st-test-name `you-function-to-test)
#2016-11-0411:54mpenetjust wondering: why in multispec we have to repeat the dispatch fn/key in s/multi-spec call itself (since it's already in the defmulti definition)#2016-11-0411:54mpenetas seen here: http://clojure.org/guides/spec#_multi_spec#2016-11-0411:59mpenet-> retagging#2016-11-0411:59mpenetok#2016-11-0412:44Alex Miller (Clojure team)You can use multispec on cases other than keyword maps too - in that case you need a fn for retagging instead.#2016-11-0413:50mpenetyup, makes sense#2016-11-0413:51mpenetit's not obvious from the guide tho#2016-11-0413:51mpenetbut the docstring is clear about it#2016-11-0417:59sveriLets say I have two specs A and B and now I have a function that has a paramater that is the union of the two maps so like (keys ::A ::B). How would I define that spec?#2016-11-0417:59sveriI tried a (merge ::A ::B) but that does not work#2016-11-0418:01bfabry@sveri did you use core/merge or spec/merge?#2016-11-0418:02sveri@bfabry Ah, I did not know there was a spec/merge, thank you, I used the core one. With spec/merge it works šŸ™‚#2016-11-0423:12d._.bI'm feeling dumb. I have a map like
{:abc "abc" ; required key, and must have a value of "abc"
 :x {:y nil ; required key, but value can be anything
     :z 123 ; same as above}
}
#2016-11-0423:12d._.bHow do I spec this?#2016-11-0423:16bfabry(s/def :your/abc #{"abc"}) (s/def :your/x (s/keys :req-un [:your/y :your/z])) (s/def :your/y any?) (s/def :your/z any?) (s/keys :req-un [:your/abc :your/x])#2016-11-0423:16d._.b
(s/def ::abc "abc")
(s/def ::x (s/map-of :req-un [keyword? identity])
(s/def ::my-map (s/keys :req-un [::abc ::x]))
#2016-11-0423:17d._.b@bfabry thank you, i knew i was missing something#2016-11-0423:17d._.bis it possible to also use every-kv or map-of?#2016-11-0423:18bfabryanything is possible#2016-11-0423:18d._.bhaha, im just trying to understand when to reach for map-of or every-kv#2016-11-0423:18d._.band when not to#2016-11-0423:18bfabryreach for s/keys if you know what your keys will be#2016-11-0423:18d._.bin my example, i know that all of the k/v pairs will be :key any?#2016-11-0423:18d._.bunder :x#2016-11-0423:19d._.b@bfabry why do you not s/def the last line in your example?#2016-11-0423:20bfabryshrug I didn't need to refer to it by a name#2016-11-0423:20bfabryif I did I would#2016-11-0423:20bfabrygotta go#2016-11-0423:40d._.bIs there a way to def multiple things at once?#2016-11-0423:41Alex Miller (Clojure team)No#2016-11-0423:41d._.bUse for or doseq to generate the defs? What's idiomatic?#2016-11-0423:41Alex Miller (Clojure team)You can write a macro#2016-11-0423:42d._.bSomething like (doseq [k [::abc :def]] (s/def ~k any?))` ?#2016-11-0423:44d._.bI think I must just be missing something obvious#2016-11-0423:45d._.bIf I have like 20 keys which are req-un in a map, all of which should have values checked of any?, that seems like it should be more straightforward than resorting to doseq'ing to generate s/defs#2016-11-0423:48d._.bSorry for the repost but, just to reiterate:
{
 :abc "abc" ; required key, and must have a value of "abc"
 :x {:y nil ; required key, but value can be anything
     :z 123 ; same as above
     :w :wow ; same as above
     :o :out ; same as above
  }
}
#2016-11-0423:51d._.b
(s/def ::abc #{"abc"})
(s/def ::x (s/keys :req-un [::y ::z ::w ::o]))

(s/def ::y any?)
(s/def ::z any?)
(s/def ::w any?)
(s/def ::o any?)

(s/def ::entire-thing (s/keys :req-un [::abc ::x]))
#2016-11-0423:51d._.bDo I really need to write all 4 of those defs?#2016-11-0423:53d._.bI was thinking I could (s/every-kv #{::y ::z ::w ::o} any?) or something... No?#2016-11-0500:50bbloomfinally getting around to trying spec in anger#2016-11-0500:50bbloomis there a standard conform-or-throw function?#2016-11-0501:27hiredmanthere is a/assert, but you need to set a property to turn it on#2016-11-0501:35richiardiandrea@bbloom I so wanted that too, unconditionally, I could not find it and copied over s/assert and named it valid-or-throw šŸ˜„#2016-11-0501:38kenny@richiardiandrea Done šŸ™‚ https://gist.github.com/kennyjwilli/8bf30478b8a2762d2d09baabc17e2f10#2016-11-0501:39richiardiandreagreat I'll tweet you šŸ˜‰#2016-11-0512:44dominicmIs there yet a way to "scope" an error done against a whole map? Given that I have:
{:x 1
 :y [2 3 4 5]}
And want to validate that :x must be contained in :y, but I want my error message to state that it is :x that is wrong, not the whole map. How would I do it?
#2016-11-0513:02Alex Miller (Clojure team)s/and a function that checks that x is in y?#2016-11-0513:03Alex Miller (Clojure team)The error should list that function#2016-11-0513:03Alex Miller (Clojure team)As the failing predicate#2016-11-0514:17dominicm@alexmiller I'm thinking more of the :path of the error (I think that's the key)#2016-11-0515:56tianshuhow can i turn on the :ret check for clojure.spec/fdef.#2016-11-0519:47mpenetI had deal with that pb recently, If you check on test.check wiki there's a page explaining how to deal with recursive gen and how to limit gen depth/size#2016-11-0519:51mpenet(I am on my phone, not easy to find)#2016-11-0522:44bbloomis there a spec somewhere for the ns form or ā€œlibspecā€ data?#2016-11-0523:43gfredericks@bbloom I think you can poke around to find how the spec for ns is implemented#2016-11-0523:43gfredericks(doc ns) might even help with that#2016-11-0523:43bbloomduuuuh i forget that core had all the specs in there now#2016-11-0523:43bbloomthanks#2016-11-0523:43gfredericks(clojure.spec/form :clojure.core.specs/ns-clauses)#2016-11-0523:44gfredericksand clojure.spec/form#2016-11-0523:44bbloomanyway - i decided to do my own spec for a tiny subset of the complexity of the full ns form#2016-11-0523:44bbloompretty narrow use case#2016-11-0600:50hiredmanso, anyone write a spec -> gloss codec compiler yet?#2016-11-0601:42keithsparkjoyWorks fine if I define it all on one line, but include the newline and the repl chokes.#2016-11-0601:42keithsparkjoyCurious if anyone has experienced this or if itā€™s just me?#2016-11-0601:42keithsparkjoyTrying to use the new map literal syntax with clojure.spec, which is why Iā€™m asking here...#2016-11-0601:43keithsparkjoyThe error I get is java.lang.RuntimeException: Unmatched delimiter: )#2016-11-0614:55madstap@keithsparkjoy Works for me on alpha14#2016-11-0616:11keithsparkjoyOkay must be something about my environment. Thanks for trying.#2016-11-0707:36nortonHas someone an example where spec instrumentation using the :replace option works? I tried a simple example to override the ranged-rand function with one that always returns the start value ā€¦ but so far no luck.#2016-11-0718:11uwois there anyway to say that a map should contain one or the other key, besides this?
(s/def ::one-or-the-other
  #(let [ks (set (keys %))]
     (xor (ks :key1) (ks :key2))))
#2016-11-0718:13minimal@uwo you can use or in s/keys, see the docstring#2016-11-0718:13minimal
The :req key vector supports 'and' and 'or' for key groups:

  (s/keys :req [::x ::y (or ::secret (and ::user ::pwd))] :opt [::z])
#2016-11-0718:14uwo@minimal wicked, thanks!#2016-11-0718:15uwortfmftw!#2016-11-0816:09drewr@norton it's tricky... you have to make sure there's an fdef for the thing you're :replace-ing#2016-11-0816:43norton@drewr Thank you for the feedback. I posted this question (with more details) to the Google clojure mailing list with an example that is using :replace. Alex Miller responded and corrected my example. It works as expected!#2016-11-0819:38mpenets/keys question, I need to express key ::foo is optional, but if it's there ::bar key should be too, since there's no or/and in opt it's a bit tricky I think.#2016-11-0819:41mpenetI guess i ll split that in multiple s/keys#2016-11-0819:41zaneI don't think you can do all ā€” yes.#2016-11-0819:41zaneThat's what I'd do. :relaxed:#2016-11-0819:41mpenetnot sure why and/or is not supported in s/keys :opt, it would be useful#2016-11-0819:43mpenet@alexmiller might know šŸ™‚#2016-11-0819:45Alex Miller (Clojure team)Why not req (or foo (and foo bar))#2016-11-0819:45zaneI'd probably replace the #(or ā€¦ with another (s/keys ā€¦ so you get enforcement of the values for :foo and :bar.#2016-11-0819:46mpenet@alexmiller because foo is required like this#2016-11-0819:46mpenetit's should be optional, but if present there should be :bar too
#2016-11-0819:46zaneAh.#2016-11-0819:47mpenet@zane yep + s/merge support etc etc#2016-11-0819:50Alex Miller (Clojure team)I don't think merge helps at all here#2016-11-0819:50Alex Miller (Clojure team)But you could and a predicate that adds this constraint#2016-11-0819:50mpenetit doesn't, I was mentioning it in the context of a solution with keys vs s/and + pred with contains?#2016-11-0819:50Alex Miller (Clojure team)Or or 2 keys specs - one with the pair, one without#2016-11-0819:51mpenetthat's what I am going to end up using, but I wish it was possible to use or/and in opt#2016-11-0819:51Alex Miller (Clojure team)Although that may not be very useful#2016-11-0819:51Alex Miller (Clojure team)Opt is all optional - it doesn't make sense to have and/or there#2016-11-0819:52mpenetwell it'd make this example way nicer at least, there are other situations like this where it would make sense#2016-11-0819:52Alex Miller (Clojure team)Note that this is or, not xor#2016-11-0819:54mpenetto take the other example mentioned earlier: an option map that could have credentials (optionally), but if ::user is there there must be ::password too.#2016-11-0822:47bbloomuser=> (s/conform :clojure.core.specs/defn-args '(foo [{x :y}])) {:bs [:arity-1 {:args {:args [[:map {x :y}]]}}], :name foo}#2016-11-0822:47bbloomuser=> (s/conform :clojure.core.specs/defn-args '(foo [{x y}])) {:bs [:arity-1 {:args {:args [[:seq {:elems [[:seq {:elems [[:sym x] [:sym y]]}]]}]]}}], :name foo}#2016-11-0822:47bbloom@alexmiller ^ is that a bug?#2016-11-0822:49Alex Miller (Clojure team)Is what a bug?#2016-11-0822:49bbloomsorry, should have elaborated#2016-11-0822:49bbloomthe second evaluation returns {x y} with a :seq node instead of a :map node#2016-11-0822:50Alex Miller (Clojure team)Maybe?#2016-11-0822:51bbloomglad iā€™m not the only one who couldnā€™t instantly puzzle out the destructuring specs šŸ˜‰#2016-11-0822:51Alex Miller (Clojure team)Not at a computer so hard for me to look at stuff#2016-11-0822:52bbloomno worries - iā€™ll look at it a bit more and maybe file a bug - thanks#2016-11-0822:56Alex Miller (Clojure team)I guess the question is whether regex specs like cat should or do work on maps#2016-11-0822:57zaneWhat's the recommended way to handle specs that are dependent on runtime values?#2016-11-0822:57Alex Miller (Clojure team)If so, then seq-binding-form will happen to be satisfied by the map#2016-11-0822:57Alex Miller (Clojure team)@zane don't?#2016-11-0822:58Alex Miller (Clojure team)Or dynamically generate a spec#2016-11-0823:10bbloomok - iā€™m reasonably sure that is a bug & that the fix is a simple s/spec - making a patch#2016-11-0823:10bbloomthe spec splicing behavior is simultaneously useful and annoyingly subtle šŸ™‚#2016-11-0823:46Alex Miller (Clojure team)Yep#2016-11-0823:48bbloomdecided to add a s/and vector? instead, since that seemed more reliable & more accurately spec'd (i think) - if not, happy to tweak#2016-11-0823:48bbloomclj-2055#2016-11-0914:33erwin
user => (defn foo [] :clojure.spec/invalid)

ExceptionInfo Call to clojure.core/defn did not conform to spec:
:clojure.spec/args  (foo [] :clojure.spec/invalid)
  clojure.core/ex-info (core.clj:4725)
user => (defn foo [] (do :clojure.spec/invalid))
#'user/foo
#2016-11-0914:34erwinthis is maybe a repl only problem?#2016-11-0914:36erwinnope also fails in source code#2016-11-0915:36dominicmI think this is a known bug.#2016-11-0915:36dominicmI seem to remember mention of there not being much of a good solution to it in spec though.#2016-11-0915:59Alex Miller (Clojure team)@bbloom We will have some better solution for regex ops in vectors - comes up a lot in macro dsls #2016-11-0921:35yonatanelWhat is a good example of using tags other than human-readable s/explain result?#2016-11-0923:52Alex Miller (Clojure team)Gen overrides#2016-11-0923:52Alex Miller (Clojure team)Conformed values#2016-11-1009:30yonatanelIs there something like prismatic schema coercions but with clojure.spec?#2016-11-1009:35dominicmConformers?#2016-11-1010:18yonatanelCool, thanks!#2016-11-1011:51vikeriDo I always have to specify my generators with with-gen in the same spot as where I define the spec itself? Or could I assoc a generator later in say, a ns that is only used for testing? I am aware of that stest/check takes the :gen argument but I would like to have it while working in the REPL as well. I could solve this by overwriting the sdef but would like to know if there was a standard way.#2016-11-1013:23Alex Miller (Clojure team)You can use instrument to override generators at test time#2016-11-1013:24Alex Miller (Clojure team)Or repl time#2016-11-1015:11vikeri@alexmiller Great! Thanks#2016-11-1015:12vikeriDoes anyone else encounter infinite loops when trying to spec a lazy sequence even though one uses every instead of coll-of?#2016-11-1015:18jrheardin clj or cljs?#2016-11-1015:20vikericljs#2016-11-1015:21jrheardyeah, occasionally when i instrument (or is it check?) certain bits of my code in cljs, i see spec hang indefinitely#2016-11-1015:21jrheardcanā€™t reliably reproduce it#2016-11-1015:21jrheard<ā€” unhelpful sorry#2016-11-1015:21jrheardbeen a month or two since i tried to dig into it last time#2016-11-1015:30vikeriAlright, I had a spec that was failing, when I updated my spec it worked again...#2016-11-1100:20bbloomi just ran in to a situtation where (s? any?) matched the individual characters of a string#2016-11-1100:20bbloomwhich i guess is not incorrect, but itā€™s certainly not what i expected šŸ˜•#2016-11-1100:20bbloomwhatā€™s the best way to treat strings as scalars?#2016-11-1100:34bbloomoh - no maybe something else has gone wrong#2016-11-1100:35bbloomnevermind...#2016-11-1111:24joshkhnoobie question here. can spec be used to force a map into a particular shape? specifically, i'd like to drop all keys in a function's argument that aren't defined in :req-un/:opt-un#2016-11-1113:46yonatanel@joshkh Just yesterday I learned about conformer. Maybe it will help you.#2016-11-1113:50Alex Miller (Clojure team)@joshkh not much in spec that will help you (beyond using a conformer that calls select-keys or whatever way you'd do it without spec)#2016-11-1113:50joshkhi read about conform but that appeared to return the original data if it conforms, or ::invalid if it doesn't. somehow i missed conformer. thanks šŸ™‚#2016-11-1113:50Alex Miller (Clojure team)there is an extension to keys that we've talked about that might help eventually#2016-11-1113:53joshkhah, okay. it's not a show stopper for me. i can easily select-keys on the map that's passed to my function. i was just wondering if there was a way to leverage my already written specs.#2016-11-1113:53joshkhthanks @alexmiller and @yonatanel#2016-11-1117:21sophiagowondering if i can get some quick input, which might be no better than "just give it a shot incrementally, it's not all or nothing," and @alexmiller you might be well-suited to answer this if time permits: i wasn't paying attention to spec until i saw rich's sale's pitch (so to speak) last night and a lightbulb click re: debugging that symbolic algebra library with the custom numeric tower you looked at a while ago. he made several points that applied well to it: leveraging dynamic typing rather than out of laziness and accountability to stakeholders, the latter i couldn't really claim from having it "working fine" on its own as a library and with a couple clones and then breaking when i myself wrote the edge case that uses it with a library of lazy-seqs. to cut to the chase, i had to table the project and now and am weighing the time commitment of beginning to mark it up with spec vs. circling back to debug traditionally (possibly having to handroll my dynamic dispatch excluding the problem types as in math.numeric-tower). asking because you may vaguely remember the code i think it may be an ideal case because lazy-seqs both expose edge-cases in typing and are a nightmare to debug manually plus given the numeric tower is built with protocols i'm wondering how much that could cut down on my time commitment in writing tests. but then again, i'm not even sure if it would make sense to write the tests out of context in the library of making use of the library that broke it just to get the spec/explain data on it. apologies for the length, but any general advice for using spec to debug edge cases in custom typing would likely be useful given where i'm starting from. thanks!#2016-11-1117:24Alex Miller (Clojure team)spec can certainly be helpful in debugging#2016-11-1117:25Alex Miller (Clojure team)Iā€™m working on a project right now where I have specā€™ed most of the fns and run with instrument on while working at the repl and itā€™s been a huge time-saver to discover immediately when something doesnā€™t align#2016-11-1117:26Alex Miller (Clojure team)(and actually writing the specs of course clarified thinking and design significantly)#2016-11-1117:26Alex Miller (Clojure team)itā€™s impossible for me to judge how the effort involved in writing the specs compares with other debugging approaches for your particular problem#2016-11-1117:26sophiagoyeah, obviously i knew there was no answer to that question. and it's an incremental thing so i just try it out if i want#2016-11-1117:26Alex Miller (Clojure team)however, the idea with spec is that if you do take the effort to write the specs, you can reap many benefits from it#2016-11-1117:27Alex Miller (Clojure team)yes, itā€™s not all or nothing#2016-11-1117:27Alex Miller (Clojure team)Iā€™d start at the simplest, most basic functions#2016-11-1117:27Alex Miller (Clojure team)use instrument#2016-11-1117:27Alex Miller (Clojure team)try check to see if it flushes out any bugs at that layer#2016-11-1117:27sophiagobut more, advice on going that route given i've already exposed the bugs and the particular situation?#2016-11-1117:28sophiagoi guess the first question is whether to spec it at the level of the library that broke it or the broken library?#2016-11-1117:28Alex Miller (Clojure team)Iā€™d spec the library#2016-11-1117:28Alex Miller (Clojure team)but thatā€™s just my intuition#2016-11-1117:28sophiagobut i know sort of what tests to write i suppose#2016-11-1117:30sophiagolike if i know essentially creating rationals using convolution (simplifying things a bit) seems to be where it's broken then i can spec it around that, but inside the typing library#2016-11-1117:32sophiagoi suppose mainly to separate the aspects of recursive functions and lazy-seqs from the actual type conversion causing the bugs and then get that spec/explain data on those cases#2016-11-1117:34sophiagothen hopefully approach similar libraries this way from the ground up going forward. it's true how we don't use unit tests, but also...a library isn't "working fine" just because you tested it manually in the repl on its own#2016-11-1117:35sophiago(i may have another one to spec out too...good thing he made the learning curve seem slight. once he got to regex i was like, hmm benefits surely outweigh costs for me#2016-11-1117:38Alex Miller (Clojure team)be sure to read http://clojure.org/guides/spec#2016-11-1117:39sophiagoi guess i'll come back after diving in, but one thing that comes to mind initially is whether i could use it to test for casting to java numerical types. i can't remember the syntax for one talk, but i have an exact idea of which ones are safe and which are causing the problems and i know you currently just have int? and float? for colls. i assume that must be extensible somehow, though#2016-11-1117:43sophiagoactually maybe that's enough? i'd have to check the source. but if i can say (s/cat (s/and [int? float?] )) or something like that and have it return false for rations and BigInts that would satisfy my contract#2016-11-1117:46Alex Miller (Clojure team)theyā€™re just functions - use any predicate you like#2016-11-1117:46Alex Miller (Clojure team)check precisely what they mean though#2016-11-1117:46Alex Miller (Clojure team)int? is satisfied by any fixed precision Java integer type (byte, short, int, long)#2016-11-1117:47Alex Miller (Clojure team)float? is a fixed precision floating point (float, double)#2016-11-1117:48Alex Miller (Clojure team)syntax would be like (s/cat :num (s/and int? float?))#2016-11-1117:48sophiagosounds perfect then#2016-11-1117:48Alex Miller (Clojure team)but you could pull the combo out into a separate spec too: (s/def ::fixed-num (s/and int? float?))#2016-11-1117:48sophiagoby nature your two specific numerical predicates exclude all the yucky java stuff#2016-11-1117:48Alex Miller (Clojure team)then just (s/cat :num ::fixed-num)#2016-11-1117:49Alex Miller (Clojure team)I assume yucky == arbitrary precision? :)#2016-11-1117:49sophiagoratios, BigInts, BigDecimals#2016-11-1117:50sophiagodon't really have problems with the last one in this case, but the first two i would have excluded in diy dynamic dispatch#2016-11-1117:51sophiagomuch of the point of having my own ratios was preventing overflow. the canonical power series code in haskell overrides their division for that reason as well#2016-11-1117:53sophiagolike gcd to 1 == 1N ???#2016-11-1117:53sophiagono thanks#2016-11-1117:54sophiagoand when you said overriding core math was "not so easy" subtyping was my problem having done it well before you made that statement. i suspect this leads into issues with subtyping, but then...hard to tell when everything is elegant yet opaque...hence s/explain#2016-11-1217:50weiis there documentation for the new namespaced maps creation/destructuring features?#2016-11-1218:02weiworking out how to use fully-qualified keys in our maps to work with spec, without making our code too verbose. how would I create an alias for keyword namespaces without requiring the namespace like (require [foo.bar :as foo])? sometimes itā€™s impossible to require a certain namespace because of circular dependencies, but I'd still like to use an alias for creating keywords. also is there a valid syntax for referring to the current namespace in the new namespaced map syntax, something like #::{:a 1 :b 2}?#2016-11-1219:20weialso when I eval this on the repl I get RuntimeException EOF while reading and then RuntimeException Unmatched delimiter: } :
(def a #:test{:a 1
              :b 1})
#2016-11-1219:20weiwhile this works
(def b #:test{:a 1 :b 1})
#2016-11-1219:48Alex Miller (Clojure team)You can use create-ns and alias to create an alias #2016-11-1219:49Alex Miller (Clojure team)The #:: syntax should work for current ns#2016-11-1219:50Alex Miller (Clojure team)Not sure on the reader eof - which repl are you using#2016-11-1219:51Alex Miller (Clojure team)See http://clojure.org/reference/reader under Namespace Map Syntax for docs#2016-11-1220:07weithanks for the reply @alexmiller. using lein repl on 1.9-alpha14#2016-11-1220:11wei#:: not working for me in some cases
foo.bar=> #::{:a 1}

RuntimeException EOF while reading, starting at line 1  clojure.lang.Util.runtimeException (Util.java:221)

RuntimeException Unmatched delimiter: }  clojure.lang.Util.runtimeException (Util.java:221)
#2016-11-1220:22gfredericksfeels like a bug to me#2016-11-1220:22gfredericksI reproduced#2016-11-1220:35sophiagoum, is there a reason i could be getting a file not found exception for clojure/spec__init.class and clojure/spec.clj after switching to the 1.90-alpha14? i ran lein deps and even downloaded the zip in case that was necessary. do i need to run lein install in the 1.9 folder? i thought lein would have handled that#2016-11-1220:46wei@sophiago try lean clean first. if youā€™re still having problems, could you try including spec in a new project as a sanity check?#2016-11-1220:47sophiagooh you know what#2016-11-1220:48sophiagoi'm running it from a checkout dependency. so probably need to run lein install in the original directory#2016-11-1220:49sophiagohmm, ok. lein install seems to have worked, but still getting that error in the other project#2016-11-1220:49sophiagoi'll try to clean it#2016-11-1220:50sophiagohmm nope#2016-11-1220:50sophiagooh, i guess i probably just need matching versions of clojure in both? that makes sense#2016-11-1220:51gfredericksyou can only have one version of clojure at a time#2016-11-1220:51gfredericksthey don't have to match, but the dependent will use the depender's version#2016-11-1220:52sophiagoah so that should explain it#2016-11-1220:53sophiagook, all good. back to bug i'm getting started with spec to flush out šŸ™‚#2016-11-1220:53sophiagoshould have been able to think through that without talking out loud, but it's been a long week#2016-11-1220:54sophiagoprotest marching outside my door rn...#2016-11-1311:05weiwhat do I need to do to get fdef to throw errors on invalid return values?
(defn x [] "hello")
(s/fdef x :ret number?)

=> (x)
"hello"
#2016-11-1311:23yonatanelWill instrumenting a multimethod be supported? https://groups.google.com/forum/#!topic/clojure/BBWc3c40RDI#2016-11-1311:44yonatanelAlso, is there a way to require a bunch of same-namespaced keys without specifying the namespace on each one?#2016-11-1314:08yonatanelIs there a best practice for conformers, like one that can transform a string to java.util.UUID - should I call it ::uuid, ->uuid etc?#2016-11-1314:50Alex Miller (Clojure team)@wei not supported. ret and fn specs are checking that a function works, which is seen as a testing time activity supported only by stest/check#2016-11-1314:52Alex Miller (Clojure team)@yonatanel: re multimethods, certainly no opposition to it, but needs some work#2016-11-1314:55Alex Miller (Clojure team)@yonatanel: re ns keys, use create-ns to create the namespace, then alias to alias it. There is a pending patch to remove the first step.#2016-11-1314:55Alex Miller (Clojure team)Re conformers, no convention#2016-11-1318:13wei@alexmiller what would you recommend for checking inputs at runtime using spec? s/valid? inside of a :pre?#2016-11-1318:24madstap@yonatanel There's a version of alias in https://github.com/gfredericks/schpec that creates the namespace, then aliases to it.#2016-11-1318:25weinevermind, figured it out, the guide does mention :pre/:post#2016-11-1318:27weiwhat are some tradeoffs between s/check-asserts and :pre/post?#2016-11-1318:28Alex Miller (Clojure team)Check-asserts can be turned on and off or even compiled out#2016-11-1406:24ikitommiWhen creating custom Specs, what are the rules for describe*? evalā€™ing it should return the Spec?#2016-11-1406:31ikitommiCould the describe* of Specs be used as a serialization format over the wire (sending specs over Transit etc.)?#2016-11-1413:52Alex Miller (Clojure team)You should call s/form #2016-11-1413:53Alex Miller (Clojure team)That is intended to be a form that can serialized a wire (all symbols should be resolved etc)#2016-11-1413:53Alex Miller (Clojure team)There are a number of known bugs and patches on it that I have been working through so it's not 100% yet#2016-11-1413:54Alex Miller (Clojure team)s/form uses describe* internally but you shouldn't call that directly#2016-11-1414:26ikitommithanks @alexmiller and great to hear the serialization support in the scope of spec!#2016-11-1420:28hiredmanI wrote some code the other day that uses s/form to traverse specs and generates a json schema from the spec, it was way cool#2016-11-1421:41jasonjckn@hiredman gist#2016-11-1421:41jasonjckn@hiredman link to code?#2016-11-1421:43hiredmanI can't, but it is a lot like a lisp interpreter, type dispatch + dispatch on the first symbol in a seq#2016-11-1421:44hiredmanif you see a keyword, call s/form to lookup the spec, if you get a seq, dispatch on the first symbol in the seq (s/keys, s/or, whatever) if you get a symbol it is some kind of type predicate like uuid? and you map that to some type in json#2016-11-1506:03ikitommi@hiredman @jasonjckn something like https://github.com/metosin/spec-tools/blob/master/src/spec_tools/json_schema.clj?#2016-11-1520:54jasonjckn@ikitommi wonderful#2016-11-1611:59gerritI wonder if there is something built into spec already, that does the following: conform a value and throw an exception with an explanation of what violates the spec in case conform returns invalid. similar to s/assert but always on, as it should be used for internal conforming and validation, e.g. after reading from a database. I came up with this:
(defn conform-validate [spec x]
  (let [conformed (s/conform spec x)]
    (if (s/invalid? conformed)
      (throw (ex-info (str "spec violated: " (s/explain-str spec x))
                      (s/explain-data spec x)))
      conformed)))
any simpler approaches that I am missing or helper libraries that do this already?
#2016-11-1614:04Alex Miller (Clojure team)That looks good to me#2016-11-1614:04Alex Miller (Clojure team)Nothing in spec for it#2016-11-1622:05bbloom^ I ran in to precisely that need almost immediately upon trying spec. I think it would be a good addition to the lib if it had a good name and insightful docstring. Maybe s/enforce and a docstring that contrasts with assert#2016-11-1622:06bbloomalthough, to play devilā€™s advocate with myself, i found that when i did want this, i wanted to add extra data to the error via ex-info#2016-11-1623:47Alex Miller (Clojure team)Rich considered it and ended up with assert so I don't think he plans to add it afaik#2016-11-1623:48bbloomyeah, itā€™s a reasonable assumption that people would use the wrong one and/or it wouldnā€™t be particularly useful w/o wrapping it w/ extra data#2016-11-1623:50Alex Miller (Clojure team)I expect people will make exactly what they want out of the available parts#2016-11-1700:00seancorfieldWeā€™re slowly creating functions that turn explain data into a series of unique failure codes which we have as a documented part of our API. Our use case is essentially
(let [conformed (s/conform spec args)]
  (if (s/invalid? conformed)
    (render-failure (problems->failure-codes spec args))
    (render-result conformed)))
#2016-11-1711:12ikitommi@seancorfield is it possible to share what kind of code lies in the problems->failure-codes?#2016-11-1715:07ikitommiShoud (s/form (s/spec integer?)) return (clojure.spec/spec clojure.core/integer?)? now it returns just clojure.core/integer?#2016-11-1715:19Alex Miller (Clojure team)That's a valid spec#2016-11-1715:20Alex Miller (Clojure team)So, looks right to me#2016-11-1715:28ikitommihmm.. (s/spec? (s/form (s/spec integer?))) ; => nil#2016-11-1715:34seancorfield@ikitommi: it walks the problems and maps path to a code if it non-empty else it maps pred to a code (and it special-cases (contains? % key) which you get for a missing key from req/req-un. #2016-11-1715:39dergutemoritz@ikitommi That's to be expected, s/form returns a data structure, not a spec. The clojure.core/integer? you're seeing is just the symbol, not the var#2016-11-1716:11dergutemoritz@ikitommi Ah, alright, makes sense šŸ™‚#2016-11-1716:12dergutemoritzLooks like s/form is not injective!#2016-11-1716:19dergutemoritzThis is also interesting:
(s/form integer?)
; => :clojure.spec/unknown
#2016-11-1717:25yonatanelWhy does this not work?
(def not-blank? (complement clojure.string/blank?))
(s/conform not-blank? "")
IllegalStateException Attempting to call unbound fn: #'dev/not-blank?  clojure.lang.Var$Unbound.throwArity (Var.java:43)
#2016-11-1717:26gfredericksI don't know. #2016-11-1717:31yonatanelIt doesn't work when I do it in the REPL, otherwise it's fine.#2016-11-1717:38hiredmanmy guess is there is an error from the def you aren't seeing#2016-11-1717:39hiredmanlikely your repl is running with a version of clojure that doesn't have blank?#2016-11-1717:39hiredmanso the def is interning the var, but not giving it a value#2016-11-1718:14Alex Miller (Clojure team)@dergutemoritz: that's a known bug and will be fixed#2016-11-1718:31dergutemoritz@alexmiller Cool!#2016-11-1719:18Alex Miller (Clojure team)@dergutemoritz: I've done some of the work on it, just needs a bit more input from rich#2016-11-1719:37joost-diepenmaatis there a way to do something similar to the removed clojure.spec.test/instrument-all before running (but after compiling) tests with lein test?#2016-11-1719:39joost-diepenmaatoh I guess I can use fixtures for this#2016-11-1719:41bfabry@joost-diepenmaat (instrument-all) was replaced with (instrument)#2016-11-1719:42bfabryand yeah, I think (use-fixtures :once #(do (clojure.spec.test/instrument) (%)))#2016-11-1719:43joost-diepenmaat@bfabry thanks#2016-11-1719:50joost-diepenmaatyes that works fine#2016-11-1721:21Alex Miller (Clojure team)you might want to unstrument at the end#2016-11-1721:34joost-diepenmaat@alexmiller: yeah i thought so too#2016-11-1721:36joost-diepenmaat(defn with-instrument-all [t] (let [instrumented (stest/instrument)] (t) (stest/unstrument instrumented)))#2016-11-1721:36joost-diepenmaatThat's what I'm using as fixture right now#2016-11-1721:53gfrederickstry/`finally`#2016-11-1722:01hiredmanthat was my first thought too, but in theory deftest will catch any exceptions from tests, so the only exceptions could be from fixtures, but your other fixtures should already never bubble out exceptions, because fixtures have some odd edge cases when exceptions are allowed to bubble out#2016-11-1722:01hiredmanif I recall fixtures that bubble out exceptions can hang lein test#2016-11-1723:14spiedenam i abusing spec if i use it for type conversions?#2016-11-1723:15spiedene.g. an external system returns strings containing integers and i use a conformer to parse them#2016-11-1723:15spiedenitā€™s mostly convenient except that i have to use with-gen everywhere#2016-11-1723:19seancorfieldChanneling Alex: itā€™s OK to do a type conversion in a spec if youā€™re OK with all clients of that spec getting the built-in type conversion šŸ™‚#2016-11-1723:19seancorfield(but in general avoid conformers in specs)#2016-11-1723:20spiedenhmm ok#2016-11-1723:20seancorfieldFWIW, we have a bunch of specs that accept strings and conform them to non-strings, for use in a REST API.#2016-11-1723:20spiedenok thatā€™s comforting#2016-11-1723:21spiedeniā€™ll probably want to circle back and override a bunch of the generators anyway#2016-11-1723:22seancorfieldWe use this macro to wrap such specs:
(defmacro api-spec
  "Given a coercion function and a predicate / spec, produce a
  spec that accepts strings that can be coerced to a value that
  satisfies the predicate / spec, and will also generate strings
  that conform to the given spec."
  [coerce str-or-spec & [spec]]
  (let [[to-str spec] (if spec [str-or-spec spec] [str str-or-spec])]
    `(s/with-gen (s/and (s/conformer ~coerce) ~spec)
       (fn [] (g/fmap ~to-str (s/gen ~spec))))))
so you can say (s/def ::age (api-spec ->long (s/int-in 18 121))) where ->long is a conversion that produces ::s/invalid if the conversion fails.
#2016-11-1723:22seancorfieldThis also allows (s/def ::yes-no (api-spec keyword name #{:yes :no}))#2016-11-1723:24spiedencool thanks#2016-11-1806:15bbloomso instrument only works on functions right? right now iā€™m kinda wishing i could instrument a dynamic var...#2016-11-1806:15bbloomw/o having to instrument every read/write to the var manually#2016-11-1806:16bbloomi guess i could add a validator to an atom...#2016-11-1807:04bfabrygate var alterations to a single function that's specced and instrumented?#2016-11-1807:05bbloomyeah, thatā€™s what i just did#2016-11-1807:05bbloomwould be nice to avoid the indirection tho šŸ™‚#2016-11-1807:14bfabryinteresting thought. you could also use add-watch https://github.com/clojure/clojure/blob/0929d1d13d036973a03db78a5a03f29d19c9e4b2/src/clj/clojure/core.clj#L2133#2016-11-1807:15bfabryprobably not much you can do reasonably in a watcher though#2016-11-1808:01ikitommithanks @seancorfield.#2016-11-1816:30gfredericks@bbloom vars can have validators too, can't they? #2016-11-1817:51bbloom@gfredericks huh, i guess they can - i wonder why i assumed they couldnā€™t? dā€™oh.#2016-11-1817:55gfredericksBet they can't in cljs#2016-11-1910:04trissok so Iā€™ve got some specs for params of my system...#2016-11-1910:04trissthey specify the range of valid values for each of them.#2016-11-1910:05trissIā€™d like to map values between 0-127 to values with in those ranges.#2016-11-1910:05trissCan I leverage my sepcs in anyway whilst doing this?#2016-11-1910:32trisscurrently Iā€™m using this map of fns to coerce values in to the required ranges:#2016-11-1910:42trisscan I not use the specs in someway here?#2016-11-1911:01trissI guess Iā€™m wondering how I might build functionality like SuperColliderā€™s ControlSpec(http://doc.sccode.org/Classes/ControlSpec.html) around Clojureā€™s spec...#2016-11-1911:01trissany thoughts? sorry for the brain dump. hope thatā€™s ok in here#2016-11-1912:55Alex Miller (Clojure team)Aside - swap the order of the preds in ::sample#2016-11-1912:58Alex Miller (Clojure team)There are lots of ways to either use custom generators or to each piece or to create a generator that applies a function to the default generator for the overall structure (with gen/fmap)#2016-11-1912:58Alex Miller (Clojure team)Not sure what's best here but lots of options#2016-11-1912:59Alex Miller (Clojure team)There's a talk at the conj in a couple weeks about making music with spec#2016-11-1914:26trissmany thanks @alexmillerā€¦ looking forward to music talk (sadly will be digesting via youtube)#2016-11-1914:29trissI think I need to familiarise my self with generators before Iā€™ll understand what youā€™ve just suggested! many thanks though - I know what to read up on next#2016-11-1914:39trissok more direct question now, if I have:#2016-11-1914:40trisshow can I find ::everyā€˜s minimum and maximum? is it possible?#2016-11-1915:37Alex Miller (Clojure team)so there is intent to make this more visible but things are not quite in place yet#2016-11-1915:38Alex Miller (Clojure team)specifically, you can call s/form on a spec to retrieve the original form and then grab the 2nd and 3rd values in the list#2016-11-1915:38Alex Miller (Clojure team)however, these compound specs (int-in, double-in, and inst-in) do not have the original form atm#2016-11-1915:39Alex Miller (Clojure team)there are several paths towards that and Iā€™ve been discussing with Rich the best one to take but itā€™s not implemented yet#2016-11-1916:54trissah. ok that seems simple enough for an integer rangeā€¦ a lot more complex for a range of doubles...#2016-11-1920:17Alex Miller (Clojure team)If you write a spec for the double-in form, then it's just a conform away#2016-11-1921:01twashingIā€™m not sure if this is a dumb question. But Is there anyway to include clojure.spec'd functions in a generalized test suite?#2016-11-1921:01twashingI posted an SO question, to clarify: https://stackoverflow.com/questions/40697841/howto-include-clojure-specd-functions-in-a-test-suite#2016-11-1921:06twashingAdding specā€™d functions to your running test suite, is sort of discussed in this Google Group thread. But I canā€™t figure how stest/instrument is used outside of the repl, in a test suite.#2016-11-1921:06twashinghttps://groups.google.com/forum/#!searchin/clojure/Clojure$20Spec$2C$20performance$20and$20workflows/clojure/kLLvOdtPO_k/CGvAIMObCQAJ#2016-11-1921:27Alex Miller (Clojure team)check the pinned items here#2016-11-1923:38twashing@alexmiller Lol!! Too right. Ok, let me have a play.#2016-11-1923:38twashingThanks!#2016-11-2002:09amacdougallI'm stumped on this oneā€”how do I declare that a valid arg is anything fnable, not just fn?? The function params are [pred coll], and I'm not sure how it should be fspecced.#2016-11-2002:49Alex Miller (Clojure team)ifn?#2016-11-2003:03amacdougallThanks, that was the ticket!#2016-11-2013:25yonatanelDoes s/def order matter? Can I compose with yet to be defined specs?#2016-11-2013:39gfredericks@yonatanel no and yes#2016-11-2013:43yonatanelIt works for me now, at least with keyword spec (instead of predicate function)#2016-11-2013:43yonatanel@gfredericks What's the no for?#2016-11-2013:44gfredericks@yonatanel the first of your two questions#2016-11-2013:48yonatanelIs it possible to write custom explanations? At least for a conformer.#2016-11-2014:12Alex Miller (Clojure team)No#2016-11-2014:20Alex Miller (Clojure team)Regarding the earlier question on order - do note that specs delay their resolution, but cache it once they do#2016-11-2014:20Alex Miller (Clojure team)This mostly matters when working at the repl#2016-11-2014:21gfrederickswhat are the implications of that? re-deffing a spec requires re-deffing all specs that reference it?#2016-11-2014:22Alex Miller (Clojure team)Yes - if they've been used#2016-11-2014:23Alex Miller (Clojure team)This changed in the alpha that did the perf pass#2016-11-2014:23gfredericks"been used" like in s/fdef in particular?#2016-11-2014:30Alex Miller (Clojure team)No, like you've conformed with it or done something to force the delay#2016-11-2014:40yonatanelNot directly related to clojure.spec, but is there a point in namespaced keywords that are values and not keys, other than being used in datomic as enum? I'm conforming some strings to keywords and I wonder if I should conform them directly to their datomic enum form.#2016-11-2014:48Alex Miller (Clojure team)Depends on your use#2016-11-2016:09yonatanelHow can I define a spec that is like another keys spec but one of the keys is optional this time?#2016-11-2016:14Alex Miller (Clojure team)Redefine it#2016-11-2016:15Alex Miller (Clojure team)Or define the simpler one first, then s/merge with another s/keys that reqs the additional key#2016-11-2016:16yonatanel@alexmiller If the first one has the key in :opt and the second in :req, will the second one override the first?#2016-11-2016:16Alex Miller (Clojure team)There's no overriding - they are both checked and will be satisfied #2016-11-2016:17yonatanelI see#2016-11-2016:23yonatanelI can't find a reason to use s/key's :opt if s/keys checks all map keys anyway. The guide says "Weā€™ll see later where optional attributes can be useful" but I couldn't find an explanation.#2016-11-2016:47Alex Miller (Clojure team)It's in docs and it's used for gen#2016-11-2017:22yonatanelIs it possible to turn on nilable for a whole set of keys, for example if we decide to treat nil values as not existing at all in a certain case?#2016-11-2018:19Alex Miller (Clojure team)No#2016-11-2019:08yonatanelConformer to the rescue: (s/def ::no-nils (s/conformer (fn [x] (into {} (filter second x)))))#2016-11-2019:08yonatanelAlong these lines...#2016-11-2021:57Alex Miller (Clojure team)Bleh#2016-11-2021:58Alex Miller (Clojure team)With stuff like this youre just treating spec as a meat grinder rather than actually trying to describe your data.#2016-11-2022:59yonatanel@alexmiller Interesting point. What's a good use of conformer?#2016-11-2023:07Alex Miller (Clojure team)building composite specs (like keys*)#2016-11-2023:08yonatanelI'm doing some coercion from standard web api (json) to internal representation, checking that something is a string, trimming it, checking that it's not blank after trimming. Stuff like that. It looks great in clojure.spec.#2016-11-2023:09Alex Miller (Clojure team)as Iā€™ve said many times here - thatā€™s fine. but keep in mind that you are throwing away information when you use conformers like that. When you register a spec like that, you are making a decision for all future users of your spec. That may be fine, but itā€™s something to be aware of.#2016-11-2023:14zaneThere's no way to generate a spec for the conformed values of some other spec, is there?#2016-11-2023:46bbloomare people generally putting specs inline? or in separate files/namespaces?#2016-11-2100:31Alex Miller (Clojure team)@zane no, although that's been asked a lot#2016-11-2100:32Alex Miller (Clojure team)@bbloom: I've done both#2016-11-2100:34bbloom@alexmiller besides core, do you use a separate namespace for the separate file?#2016-11-2100:34bbloomi guess yes if you want to load them with a require, no if youā€™re going to load them with load-file from the main ns?#2016-11-2100:38Alex Miller (Clojure team)I think it's easier to use a separate ns#2016-11-2100:39bbloom@alexmiller one weird thing with that, spec, and namespaced keywords in general, is that i find myself doing this:#2016-11-2100:40bbloom(create-ns ā€˜mynamespace) (alias ā€˜m ā€˜mynamespace)
#2016-11-2100:40bbloomso that i can refer to things w/o a cyclic dep#2016-11-2100:41Alex Miller (Clojure team)There is a patch I wrote to auto-create #2016-11-2100:41Alex Miller (Clojure team)So just alias would be enough#2016-11-2100:41bbloomah, nice#2016-11-2100:41bbloomthat would be welcome#2016-11-2100:41Alex Miller (Clojure team)Rich hasn't looked at it yet#2016-11-2100:41bbloomĀÆ\(惄)/ĀÆ#2016-11-2100:43Alex Miller (Clojure team)CLJ-2030#2016-11-2100:44bbloomvoted.#2016-11-2100:44bbloomthx#2016-11-2100:45Alex Miller (Clojure team)Well, it's already screened, just waiting for a slice of rich time#2016-11-2100:45bbloomnot complaining - you know iā€™m in your camp on the processes šŸ™‚#2016-11-2101:43twashing@alexmiller defspec-test worked like a charm. I answered my own SO question for future reference. https://stackoverflow.com/questions/40697841/howto-include-clojure-specd-functions-in-a-test-suite/40711634#40711634#2016-11-2101:44twashingand Iā€™m hoping that such a useful function makes it into the core library.#2016-11-2101:48Alex Miller (Clojure team)Not planning on it right now#2016-11-2104:20bbloomiā€™m using spec in anger to solve a bug in an otherwise un-specā€™d namespace right now#2016-11-2104:20bbloomi havenā€™t found the bug i am looking for, but i did find three other lurking bugs#2016-11-2104:21bbloomitā€™s kinda awesome.#2016-11-2104:34twashing@alexmiller Curious as to why not. defspec-test, or something similar, seems to be low hanging fruit for adopting specā€™d function tests, into a projects overall test suite.#2016-11-2105:56kenny@twashing the one issue with the macro is it only counts generative tests as one test in the test summary (e.g. You run 1000 generative tests but the summary results only count it as one). I've been meaning to add it but haven't had time. #2016-11-2105:59kennyIf something like it isn't included in Clojure core when 1.9 is released then we can move it into a micro library instead of a gist#2016-11-2106:43twashing@kenny Right, I was thinking just that. Kewl :+1:#2016-11-2114:33alqvist@kenny Would make may day if it found its way into 1.9#2016-11-2114:40Alex Miller (Clojure team)in general, spec tests have a different shape than example based tests. I think itā€™s good to step back and think about why or if itā€™s valuable to force everything into the shape of something that clojure.test can run. This feels to me like the limits of our tool (particularly reliance on lein test to run and report everything) are affecting our ability to think about the problem. Why does there need to be only one kind of test runner? Why does there need to be only one kind of test? Generative tests are (by their nature) longer running and often donā€™t mix well with example based tests in a single suite. There is more to say on this.#2016-11-2115:00bplatzIs there a way to use conform without destructuring the values? (i.e. Iā€™d like have an s/or within the spec but not have change the original value).#2016-11-2115:42Alex Miller (Clojure team)no, but you can use a conformer of val to undo conforming an or#2016-11-2115:43Alex Miller (Clojure team)(s/and ::my-or-spec (s/conformer val))#2016-11-2115:44Alex Miller (Clojure team)Rich toyed with s/nonconforming (still there but not in docs) but I suspect it will probably get removed before final release#2016-11-2115:54dergutemoritz@bplatz You can fudge it by wrapping your or spec like this: (s/and (s/or ...) (s/conformer val))#2016-11-2115:54dergutemoritzThough this is probably dangerously close to meat grinder territory again šŸ˜„#2016-11-2115:56dergutemoritzOops, sorry, I overlooked the first reply of yours @alexmiller#2016-11-2116:12bplatzThe use case here, which Iā€™d think is a decent one, is validating and coercing JSON input - but downstream code expects data in a specific way.#2016-11-2116:13bplatzThanks for the s/and tip, Iā€™ll use that for the time being until either it gets officially supported or we just use a different mechanism to coerce the data.#2016-11-2116:51vikeriHow does one go about speccing a multimethod? Can the different defmethods have different :args?#2016-11-2116:59Alex Miller (Clojure team)I donā€™t think s/fdef works on multimethods right now#2016-11-2117:10vikeri@alexmiller Ok, but if I donā€™t use multimethods but dynamically dispatches inside the function, can I somehow define a relation between the function arguments i.e.: > If the first argument is a number, then the second should conform to :c/myspec1 , but if the first argument is a keyword then the second argument should conform to :c/myspec2?#2016-11-2117:10Alex Miller (Clojure team)there are a couple options#2016-11-2117:11Alex Miller (Clojure team)one is to write an fdef with a :fn spec which can describe a relationship between args and ret#2016-11-2117:11Alex Miller (Clojure team)another is to use s/multi-spec which can yield a different spec based on a separate multimethod#2016-11-2117:11Alex Miller (Clojure team)the latter is really ideal for functions where the spec is open for extension after the fact#2016-11-2117:12vikeriI checked into multi-spec, but then the input has to be a map right?#2016-11-2117:12Alex Miller (Clojure team)I think there are examples of both in the guide http://clojure.org/guides/spec#2016-11-2117:12Alex Miller (Clojure team)no, it can be anything#2016-11-2117:12Alex Miller (Clojure team)as long as you give it a multimethod that chooses the proper spec based on the input#2016-11-2117:13vikeri@alexmiller Ah, just like multimethods the second argument is not a keyword but a function, but in clojure a keyword can be a functionā€¦ Then thatā€™s exactly what I need.)#2016-11-2117:13Alex Miller (Clojure team)yes#2016-11-2117:14Alex Miller (Clojure team)you will likely need a custom retag function too, but thatā€™s not a big deal#2016-11-2117:14vikeriretag?#2016-11-2117:15Alex Miller (Clojure team)used during generation#2016-11-2117:15Alex Miller (Clojure team)check the docstring for the details#2016-11-2123:16paytonrulesI have a clojure-spec/clojurescript question if anybody can help. I suspect itā€™s obvious.#2016-11-2123:17paytonrulesI have the following -
(s/def ::box (s/keys :req [::c/height ::c/width ::p/x ::p/y]))
(defn- right [box]
  (+ (:width box) (:x box)))

(defn- bottom [box]
  (+ (:height box) (:y box)))

(defn intersect? [box-one box-two]
  (not
    (or
      (> (::p/x box-two) (right box-one))
      (< (right box-two) (:x box-one))
      (> (:y box-two) (bottom box-one))
      (< (bottom box-two) (:y box-one)))))

(s/fdef intersect?
        :args (s/cat :box-one ::box :box-two ::box)
        :ret boolean?)
#2016-11-2123:17paytonrulesWhat I canā€™t do is get the intersect? to actually assert when I call it incorrectly#2016-11-2123:18paytonrulesSo for instance:
cljs.user=> (require '[cljs.spec :as s])
nil
cljs.user=> (s/check-asserts?)
true
cljs.user=> (box/intersect? {} {::c/width 100})
true
#2016-11-2123:18paytonrulesIā€™m sure the problem is between brain and keyboard, but Iā€™m pretty stumped. Shouldnā€™t this go kerblooie?#2016-11-2123:20hiredmancheck-asserts? governs any s/asserts you have in the code, but you don't have any#2016-11-2123:22paytonrulesI thought fdef would instrument the function - thereby adding the asserts automagically#2016-11-2123:23hiredmannope#2016-11-2123:24hiredmanwhen testing you can run instrument, which turns on that sort of thing for testing, but you will also get bits of generative testing happening, so it is not something you generally want to turn on#2016-11-2123:26bfabry@paytonrules you want to run (s/instrument)#2016-11-2123:26paytonrulesI was going by this from fdef docs
"Once registered, function specs are included in doc, checked by instrument,
#2016-11-2123:27paytonrulesBut apparently ClojureScript doesnā€™t have the instrument function#2016-11-2123:27bfabryreally? that surprises me#2016-11-2123:28hiredmaninstrument is in clojure.spec.test (not sure what translation to that you need to do for clojurescript)#2016-11-2123:28bfabryoh yeah, sorry, instrument is in a different ns#2016-11-2123:28hiredmanfrom what I understand, always on checking is a non-goal for spec#2016-11-2123:29paytonrulesAha - it was the clojure.spec.test that I missed bit that I missed.#2016-11-2123:32paytonrulesThanks a lot. It would appear then that specā€™ing functions that you arenā€™t actively doing property based testing on, or debugging, may not be a useful endeavor.#2016-11-2123:33hiredmanhttp://clojure.org/about/spec#_using_specs#2016-11-2123:36bfabry@paytonrules I'm not sure that's true, I think there's still value in specing things and having instrumentation turned on for development/test#2016-11-2123:39paytonrulesI wonder how much slower things will be turning on instrument in development will be#2016-11-2123:40bfabryunless you're getting into speccing function arguments I don't expect instrument will be super slow in a development context#2016-11-2123:42paytonrulesMaybe to turn it on and turn it off. Iā€™m making an HTML5 game so if I spec everything itā€™ll hurt.#2016-11-2123:52hiredmanif you turn on instrument, invoking functions that are spec'ed will also cause those functions to be genertively tested against those specs#2016-11-2123:56bfabry@hiredman I'm not sure that's right? I think it only generatively tests functions that you pass as arguments, to make sure the argument satisfies the fdef#2016-11-2123:57bfabry
user=> (defn foo []
  #_=>   (println "foo!"))
#'user/foo
user=> (s/fdef foo :args (s/cat) :ret nil?)
user/foo
user=> (clojure.spec.test/instrument)
[user/foo]
user=> (foo)
foo!
nil
user=>
#2016-11-2123:57hiredmanthe exact behavior is whatever, my point is, don't turn it on in production#2016-11-2123:58hiredmanyou don't want to be running generative tests in production#2016-11-2201:52paytonrules^Thanks for the help. I donā€™t think I said so properly before as my internet time was up.#2016-11-2208:32yonatanelIs this anything to be worried about?
(s/conform (s/or) :anything)
=> :clojure.spec/invalid
(s/explain (s/or) :anything)
Success!
=> nil
#2016-11-2210:21yonatanelHere's my experiment in returning only the conformed value of an s/or spec:
(defmacro meat-grinder-or [& preds]
  (let [tags (map keyword
                  (repeatedly (count preds)
                              #(gensym "tag")))
        prepared (interleave tags preds)]
    `(s/and (s/or 
#2016-11-2212:27Alex Miller (Clojure team)On the first question, that doesn't seem right, agreed. Feel free to log it if you like#2016-11-2212:37yonatanel@alexmiller What do you mean by log it?#2016-11-2212:38Alex Miller (Clojure team)In jira#2016-11-2212:38Alex Miller (Clojure team)http://dev.clojure.org/jira/browse/CLJ#2016-11-2212:46yonatanelno https?#2016-11-2213:02Alex Miller (Clojure team)No, sorry (it's on the list)#2016-11-2217:11eggsyntaxIs there an easy way to get the keys of a map spec pulled from the registry? I feel like there must be an obvious way I'm overlooking. I can just generate one & get the keys from that, but that feels inelegant (& might be incomplete if it had optional keys).#2016-11-2217:26Alex Miller (Clojure team)you can use s/form to get the form version of it#2016-11-2217:27Alex Miller (Clojure team)when we release form specs, you could then conform with that to extract the keys#2016-11-2217:27Alex Miller (Clojure team)but you can work around that for the time being#2016-11-2217:27eggsyntaxExcellent. Thanks @alexmiller!#2016-11-2217:30eggsyntaxThat's actually going to be broadly useful for me; I had overlooked s/form until now.#2016-11-2217:49Alex Miller (Clojure team)there are a number of remaining issues with it and I have patches for most of those in the queue#2016-11-2217:50Alex Miller (Clojure team)in particular all of the coll specs and keys* have broken forms atm#2016-11-2217:50Alex Miller (Clojure team)and maybe conformer (canā€™t remember if thatā€™s been applied yet)#2016-11-2217:50eggsyntaxOK, gotcha, I'll be cautious. Still, it's a great feature!#2016-11-2217:51Alex Miller (Clojure team)those will all be fixed#2016-11-2219:05zaneIs there something I can read to better understand query caching and how to optimize queries?#2016-11-2219:54seancorfield@zane Could you elaborate? That doesnā€™t sound related to clojure.spec...#2016-11-2220:46zaneThat was indeed for the wrong channel.#2016-11-2220:46zaneMeant for #datomic.#2016-11-2220:47zaneThanks for taking my question seriously anyway, @seancorfield. šŸ˜‰#2016-11-2221:03seancorfield@zane Ah, I wondered if it was for JDBC in which case Iā€™d be happy to field itā€¦ in #clojure šŸ™‚#2016-11-2222:06rickmoynihanDoes anyone have any tips on using clojure.spec to validate some data conforms to the spec in a clojure.test? Obviously I can do (is (s/valid? ::spec data) but if its invalid I want to see the explain outputā€¦#2016-11-2222:16Alex Miller (Clojure team)we were waiting for you to do it#2016-11-2222:34Oliver GeorgeJust a quick bit of feedback. This is a circular dependency thing. I generated some specs based on my relational database schema tables became (s/keys ...) fields became simple (s/def :schem.table/field pred) one-to-many relations became (s/def :schem.table/rel (s/coll-of :schem/table)) many-to-one relations became (s/def :schem.table/rel :schem/table)#2016-11-2222:36Oliver GeorgeThe slightly fiddly bit was the relations#2016-11-2222:36Oliver GeorgeI couldn't dump these all in one file because the table specs needed to exist before the relation specs could be processed. (more specifically order mattered or it would throw an error)#2016-11-2222:37Oliver GeorgeI was hoping all (s/def ...) statements were essentially lazy but perhaps that's not the case for performance reasons.#2016-11-2222:37Oliver GeorgeI might need to produce a simple example to demonstrate this.#2016-11-2222:41rickmoynihanlol#2016-11-2222:42Oliver George(s/def ::a ::b) throws
CompilerException java.lang.Exception: Unable to resolve spec: :example.order-matters/b, compiling:(order_matters.clj:11:1) 
#2016-11-2222:45Oliver GeorgeMy work around is to do all table defs first since they reflect relations. s/keys doesn't require the req/opt specs exist before the s/def#2016-11-2222:46rickmoynihan@alexmiller: presumably the right thing to do is to extend assert-expr to operate on something like 'valid?#2016-11-2222:47bfabry@olivergeorge I believe you can avoid that by doing (s/spec ::b)#2016-11-2222:48bfabryalthough, that's not what the documentation says, so I'm probably confused#2016-11-2222:59Oliver GeorgeThanks @bfabry I did wonder if there might be something like that. I'll experiment more.#2016-11-2223:00bfabryyeah actually doesn't work at all. my bad#2016-11-2223:00Oliver GeorgeNo worries.#2016-11-2223:02Oliver GeorgeActually this works: (s/def ::a (s/and ::b))#2016-11-2223:02bfabryI wonder if there's a concise definition of which spec macros require definition and which don't#2016-11-2223:02bfabryha. neat#2016-11-2223:03Oliver GeorgeNot sure I'd say neat exactly but it'll do the trick in my case.#2016-11-2223:04Oliver GeorgeI'd be interested in hearing from @alexmiller if this is technically a bug or if "declare before use" is an implementation requirement/assumption#2016-11-2223:04Oliver George@bfabry thanks for the inspiration šŸ™‚#2016-11-2223:05bfabrylol, glad to "help"#2016-11-2223:05Alex Miller (Clojure team)I'd say bug, file a jira#2016-11-2223:05Alex Miller (Clojure team)Everything should have delays, but there are a few places missing them#2016-11-2223:06Oliver GeorgeThanks Alex. Will do.#2016-11-2223:06bfabryoh well that's concise enough, awesome#2016-11-2223:08Alex Miller (Clojure team)That's either really easy to fix or really hard :)#2016-11-2223:10Oliver GeorgešŸ™‚#2016-11-2223:15Oliver GeorgeLogged. Please tell me if it's not a well formed JIRA ticket. http://dev.clojure.org/jira/browse/CLJ-2067#2016-11-2223:16hiredmaneventually @alexmiller will release a spec for jira tickets so you'll be able to s/valid? them before submitting them#2016-11-2223:20rickmoynihanwhat do people think of this clojure.test extension for spec?
(defmethod assert-expr 's/valid? [msg form]
  (let [spec (second form)
        input-val (nth form 2)]
    `(let [value# ~form]
       (if value#
         (do-report {:type :pass, :message ~msg,
                     :expected '~form, :actual value#})
         (do-report {:type :fail, :message ~msg,
                     :expected (:clojure.spec/problems (s/explain-data ~spec ~input-val)), :actual ~input-val}))
       value#)))
Output looks like this:
FAIL in (foo-test) (common.clj:18)
expected: [{:path [],
            :pred (* :user/foo),
            :val "1",
            :via [:user/foo],
            :in []}]
  actual: "1"
#2016-11-2223:31Alex Miller (Clojure team)@olivergeorge good enough, Iā€™ll tweak as needed#2016-11-2223:55Oliver GeorgeThanks Alex#2016-11-2310:24yonatanelIf I define specs such as :event/type, it's something that will likely be defined by more projects, so is the best practice to have a fuller namespace such as :org.my.event/type? This will deviate from the shorter form of my Datomic attributes which I don't want to prefix with the organization name all over the place. (I assume specs can collide being in the same process, while in Datomic I control what I put there)#2016-11-2312:11yonatanelPerhaps the question is about specs of internal vs external data.#2016-11-2312:52Alex Miller (Clojure team)Yes, you should use a sufficiently unique namespace #2016-11-2312:53Alex Miller (Clojure team)But if you control what namespaces you're using, "sufficiently" can be pretty short#2016-11-2313:03yonatanelBeing paranoid, maybe it could be nice to have separate non-default registry instances, or attaching a spec to a key when calling s/keys somehow.#2016-11-2313:54Alex Miller (Clojure team)Not doing that #2016-11-2315:41gfredericks@yonatanel if you're paranoid then you can just use sufficiently unique namespaces#2016-11-2315:48rickmoynihanOn a similar note, whatā€™s the suggested way to spec a format you didnā€™t define, that defines the same key with different values/specsā€¦ Basically is there an alternative to using s/keys when I have two different specs that share that prefix? Or should I just move the conflicting definitions into different namespaces?#2016-11-2315:50gfredericksDifferent namespaces#2016-11-2315:50gfredericksIt just occurs to me that spec can't describe something where a namespaced keyword has varying meaning#2016-11-2315:51rickmoynihangfredericks: thatā€™s the problem I have#2016-11-2315:51gfredericksWhich I imagine is rare#2016-11-2315:51gfredericksWho subjected you to this#2016-11-2315:52rickmoynihanwell maybe I donā€™t have the problem#2016-11-2315:53rickmoynihanI mean I can work around it by defining the specs in different namespaces#2016-11-2315:53Alex Miller (Clojure team)If you have that problem, you may be doing it wrong :)#2016-11-2315:53rickmoynihanentirely possible#2016-11-2315:53rickmoynihanšŸ™‚#2016-11-2315:54Alex Miller (Clojure team)if you have unqualified keys, then you can deal with this using s/keys and :req-un with different namespaced specs#2016-11-2315:54Alex Miller (Clojure team)if you have qualified keys, thenā€¦ use the namespaces to differentiate different semantics#2016-11-2315:54rickmoynihanalexmiller: yeah I think I just need to define some new namespaces for those extra keys#2016-11-2315:57rickmoynihanout of interest is it possible to do (s/def :foo/spec ,,,) and have that work with keys :req-un?#2016-11-2315:58gfredericksLooks normal....?#2016-11-2315:59gfredericksEvery registered spec has to have a namespace#2016-11-2316:00rickmoynihangfredericks: What I mean is - is the above possible instead of having to create a new namespace e.g. can you do the above instead of needing (ns blah.foo) (s/def ::spec ,,,)#2016-11-2316:01gfredericksYes#2016-11-2316:02gfredericksCode namespaces and keyword namespaces are mostly independent#2016-11-2316:04gfrederickshttp://hacklog.gfredericks.com/2013/08/10/clojure-overloads-the-term-namespace.html#2016-11-2316:05rickmoynihanok thatā€™s what I thought, itā€™s just not working - presumably for another reason#2016-11-2316:06rickmoynihanits ok I fixed it#2016-11-2316:06rickmoynihanthanks#2016-11-2316:07rickmoynihancode blindness#2016-11-2316:08gfredericksThe eternal bug#2016-11-2316:10rickmoynihanyup - thatā€™s one thing about software development, you have to be pretty thick skinned as youā€™re continually told by the computer that ā€œyou suck! Another mistake, are you some kinda noob? Oh and another mistake, you really arenā€™t good enoughā€¦ā€ ad-infinitumā€¦ I have a hypothesis that this is why people donā€™t stick it out as a career#2016-11-2316:11rickmoynihanyou basically have to be a bit of a masochist to stick it out#2016-11-2316:16yonatanel@rickmoynihan My take on this is that you can't yet have "wonderful" mistakes when programming, unlike art where if you use some new material or do a quick sketch it can still look good and expressive.#2016-11-2316:21rickmoynihanyonatanel: yeahā€¦ programming is so precise one bit out and it's a catastrophic failure and how many trillion bits are we handling these days?#2016-11-2316:21rickmoynihanitā€™s a miracle of engineering anything works at all#2016-11-2318:08seancorfield(for me, itā€™s sheer stubbornness: ā€œI will not be beaten by an inanimate object!ā€ so I must ā€œdefeatā€ the computer šŸ™‚ )#2016-11-2318:39rickmoynihana quixotic quest if ever there was one šŸ™‚#2016-11-2318:47aengelbergThe docstring of s/def indicates that one can provide a symbol instead of a keyword as the name for a spec, but I cannot find any examples of that. What use case is that for?#2016-11-2318:49bfabry@aengelberg s/fdef is equivalent to (s/def symbol (s/fspec ...))#2016-11-2318:50bfabrysame as defn is (def symbol (fn ...)) šŸ™‚#2016-11-2318:52aengelberg@bfabry: interesting, thanks. So when would one use specs that have been set to a symbol?#2016-11-2318:54bfabrywell you're implicitly using them anytime you use instrument#2016-11-2318:54bfabryexplicitly? I dunno. I can't think of a good one off the top of my head, but that doesn't mean there's not#2016-11-2318:57bfabryyou could use the registered fspec's to spec a function to be passed in as
(s/or :this `this :that `that :the-other `the-other)
I guess. seems a little contrived
#2016-11-2318:58hiredman@aengelberg https://github.com/clojure/clojure/blob/master/src/clj/clojure/core/specs.clj#L197-L202#2016-11-2319:00hiredmanas of the 1.9 alphas, macro inputs are checked via spec, so because the spec is named with the same symbol as the macro, the ns macro's inputs are checked with that spec#2016-11-2319:03aengelbergthanks, that helps a lot#2016-11-2319:07aengelbergSuppose I have a datatype (or {:foo/name-type :keyword, :foo/name :my-foo} {:foo/name-type :string, :foo/name "my-foo"}). How can I write that in clojure.spec, if s/keys forces me to commit to one possible spec for a given key?#2016-11-2319:08bfabryfoo/name will need to be an or spec, then you can enforce the dependency using and with a predicate#2016-11-2319:09aengelbergIs the fact that :a/b/c gives me a keyword with namespace "a" and name "b/c" an implementation detail?#2016-11-2319:10aengelbergBecause I could technically leverage that with :req-un to have different versions of a namespaced key.#2016-11-2319:13hiredmanI think that is sort of a design decision baked in to spec, a namespaced key should only ever be mapped to the same kind of thing#2016-11-2319:15hiredmanspec is sort of a global schema for data in your program#2016-11-2319:16hiredmanso similar to having a sql table with a schema where you say a column has some type, you don't really say sometimes this column has this type, some times it has this other, depending on where the data comes from#2016-11-2319:16bfabryI would guess that a symbol with / in its name is undefined in terms of what will happen#2016-11-2319:18bfabry"Symbols begin with a non-numeric character and can contain alphanumeric characters and *, +, !, -, _, ', and ? (other characters may be allowed eventually)."#2016-11-2319:18bfabry'/' isn't in that list, so I'd say you're off the edge of the map if you use symbols with it in them#2016-11-2319:19aengelbergGot it, makes sense. In this case I am working with an existing program that has keys not set up with that philosophy, so I'm just trying to figure out if there's any way I can fall back to more verbose but more flexible behavior a la prismatic schema. Sounds like not.#2016-11-2319:20aengelberg@bfabry: I was referring to keywords, not symbols.#2016-11-2319:20bfabrykeywords have the same rules as symbols, but can't contain .#2016-11-2319:20hiredmanthere was a discussion about / in keywords, and it comes up over and over again because clojure allows a larger range of inputs than it should in many places#2016-11-2319:21hiredmananswer is just don't do that#2016-11-2319:21hiredmanmy answer#2016-11-2319:21bfabryyes, you can make all sorts of symbols and keywords that the documentation doesn't say is permitted, which is very unfortunate#2016-11-2319:24hiredmanhttp://dev.clojure.org/jira/browse/CLJ-1530#2016-11-2319:25aengelbergSo what you're saying is that I should use the :a/b/c keyword NOW while I can before it becomes officially unusable.#2016-11-2319:25hiredmanno#2016-11-2319:25aengelbergJust kidding. I don't in fact want to watch the world burn.#2016-11-2319:25hiredmanšŸ™‚#2016-11-2319:28aengelbergIs this intentional?
user> (s/explain #{1 2 3} 4)
val: 4 fails predicate: :clojure.spec/unknown
nil
Specifically the "unknown" part. Seems like the predicate should be more informative when it fails.
#2016-11-2319:30bfabryspecial case. if you def'd that predicate to the registry you'd get better message#2016-11-2319:30bfabry
user=> (s/def ::foo #{1 2 3 4})
:user/foo
user=> (s/explain ::foo 5)
val: 5 fails spec: :user/foo predicate: #{1 4 3 2}
nil
#2016-11-2319:30hiredman
user=> (s/explain (s/spec #{1 2 3}) 4)
val: 4 fails predicate: #{1 3 2}
nil
user=>
#2016-11-2319:30bfabrybasically, explain is not a macro#2016-11-2319:31hiredmanspec tries to transparently treat things that could be specs (like sets and predicates) as specs#2016-11-2319:32hiredmanso those generally work#2016-11-2319:32hiredmanbut there are places where it doesn't, like in the message, there, I suspect its and easy fix if you open an issue#2016-11-2319:33hiredmans/explain must be calling whatever part of the spec protocol implements s/form on the set, and the spec protocol has a fallback on object which returns the unknown keyword#2016-11-2319:34hiredmanthere may already be an issue, spec is only out in alphas#2016-11-2319:38Alex Miller (Clojure team)@aengelberg re the unknown in explain - thatā€™s a known bug and will be fixed#2016-11-2319:38Alex Miller (Clojure team)Iā€™ve worked on it a bit but waiting for some feedback from Rich#2016-11-2319:38Alex Miller (Clojure team)I should probably log an actual jira for it so I can just point people to it#2016-11-2319:39Alex Miller (Clojure team)need to unmunge the fn#2016-11-2319:41Alex Miller (Clojure team)re your prior question, depending on your goal, you can s/def :foo/name with an s/or or you could define different s/keys specs and s/or those#2016-11-2319:42Alex Miller (Clojure team)but the latter seems like it would be papering over the true range of variability in the spec for that attribute#2016-11-2319:42Alex Miller (Clojure team)if your goal is really coercion, then there may be other options (conformers) to consider#2016-11-2319:46Alex Miller (Clojure team)Filed http://dev.clojure.org/jira/browse/CLJ-2068 for the s/explain problem#2016-11-2322:19weiany best practices for ns qualifying keywords that come out of a db?#2016-11-2322:20weiin particular is there a function for adding a namespace to all bare keys in a map#2016-11-2322:23rickmoynihanOne observation whilst playing with spec is that it seems to be ā€œstaticā€ by design, in that it doesnā€™t seem to encourage or allow you to programmatically build/modify and refine specs. For example one thing I used to do with schema (especially in tests) was define a generic schema for something and then in specific tests where I want to test something more specifically e.g. the presence of a specific test value in the output, Iā€™d simply assoc a new specific value into the schema, and validate against that. Is this a fair point in understanding the proā€™s and conā€™s of spec vs other schema libraries?#2016-11-2322:23rickmoynihan(I should say it seems totally fair to me that spec trades off dynamism for something more static and comprehensible)#2016-11-2322:26hiredmanhard to say#2016-11-2322:26hiredmanthe combinators are different#2016-11-2322:26hiredmaninstead of associng in, you might s/and a new s/keys spec#2016-11-2322:27hiredmanI think there is a bias in spec towards naming things globally, vs some kind of local names or anonymous things#2016-11-2322:33hiredmanI guess I'd say, I don't think spec is particularly static, but the bias towards global names can make it feel that way some times. And the bias is just a bias, you can do that kind of stuff, you might just have to venture off the beaten path of what is provided in the box#2016-11-2322:36rickmoynihanhiredman: yeah I think youā€™re right#2016-11-2323:40aengelbergHow can you define mutually recursive specs?#2016-11-2323:45zaneI don't remember having to do anything special.#2016-11-2323:59Alex Miller (Clojure team)Just refer to them by (qualified keyword) name#2016-11-2323:59Alex Miller (Clojure team)They delay evaluation so should just work#2016-11-2406:18ikitommiRuntime defined specs/schemas have also a good use case with web apis & dynamic forms. Iā€™m thinking of creating a MapSpec -kinda Spec Record which would allow non-registered (global) specs to be used and which could manipulated as normal Clojure Map. Adding generated specs to registry at runtime might not be a good idea...#2016-11-2406:25ikitommiRelated: toyed with a simple collection spec macro that allows specs to be created from nested maps/vectors/sets. Should help to define partially anonymous map specs. For things like http query-parameters.#2016-11-2406:29ikitommiemits just recursively s/keys & s/coll-ofs.#2016-11-2407:11ikitommiHmm... but I think after runtime modification of a MapSpec, one would still need to have the full source-code of it for the s/form to work. So adding values at runtime would require both the value and itā€™s source code to be added. Removing keys would be easy, same as merging existing MapSpecs. Maybe not such a good idea after all.#2016-11-2410:20rickmoynihanikitommi: Interesting - I was having similar thoughts about the possibility of building more specialised libraries that can consume ::specs - for more dynamic use casesā€¦ e.g. you could imagine something like plumatic being able to consume specs in some circumstances- It seems youā€™re a long way ahead of me šŸ™‚#2016-11-2410:20rickmoynihanstill very much at the getting started stage with spec#2016-11-2414:24ikitommirickmoynihan: we are all still spec newbies, exiting times.#2016-11-2418:24yonatanel@ikitommi Talk to me about that runtime specs you are planning. I've just written down my requirements for this kind of thing yesterday and would love to collaborate.#2016-11-2500:08trisshi all. Iā€™ve been looking at extracting the minimum and maximum values from specs created with s/int-in and s/double-in...#2016-11-2500:08trissWhat Iā€™ve done is create a spec that recognises the formā€™s produced those functions...#2016-11-2500:09trissand then created seperate functions to pull the mins and maxs from the conformed maps that I create with these.#2016-11-2500:09trissbut the code is 50 lines long? https://gist.github.com/triss/1128bcfeb1a71c7e09ffd9cf10be0370#2016-11-2500:10trisscan anyone tell me if this is a sane approach?#2016-11-2500:11trissor am I totally bonkers?#2016-11-2502:14Alex Miller (Clojure team)So those will s/form back to their original form in the future (I know they are a mess now). That plus a spec on those forms makes this just a conform away.#2016-11-2507:16ikitommi@yonatanel plan is just to test (and built utilities) out things that we are using with Schema. For Schema we have the schema-tools library (https://github.com/metosin/schema-tools) having modified versions of the core functions for maps. In top there are special walkers, transformers and matchers. Using those both at design & runtime. At runtime, have used schemas for example for frontend form validation, and the forms/schemas can change based on the user input. With Spec, there could be either more functions/macros to manipulate the Specs (like s/merge) or a new MapSpec record which could be used like a regular map. Maybe someone has tried the map-approach already? What use cases / requirements do you have?#2016-11-2509:35slipsetSorry if this has been asked/answered already, but initial googling showed nothing#2016-11-2509:35slipsetI have a map like#2016-11-2509:35slipset
{:timestamp <jodatime instance> :value number?}
#2016-11-2509:36slipsetHow do I write a spec for the :timestamp that s/exercise understands?#2016-11-2509:37slipsetMy first attempt is
(s/def ::timestamp (partial instance? DateTime))
#2016-11-2509:38slipsetbut s/exercise doesnā€™t know how to create instances of this.#2016-11-2509:52slipsetok, so I see there is a inst? which works on java.util.Date. Might get that to work.#2016-11-2509:56slipsetwhich is part of clojure-1.9 but not of future-spec šŸ˜ž#2016-11-2510:16slipsetThis is what I ended up with:#2016-11-2510:16slipset
(s/def ::timestamp (s/with-gen (partial instance? DateTime)
                     (fn [] (gen/fmap #(DateTime. %)
                                      (gen/large-integer*
                                       {:min (c/to-long
                                              (t/minus (t/now)
                                                       (t/days 365)))})))))
#2016-11-2513:13Alex Miller (Clojure team)in 1.9, you can extend the (new) Inst protocol to DateTime - if you do so, you should be able to use s/inst-in#2016-11-2513:13Alex Miller (Clojure team)but what you have should work#2016-11-2513:26slipsetYes, I saw that, but Iā€™m just playing with future-spec by @tonsky.#2016-11-2513:26slipsetItā€™s missing some parts, like eg the much debated any?#2016-11-2513:27slipsetBut it was a good exercise writing ones own generator.#2016-11-2514:02slipsetIā€™m playing around with higher-ordered functions, so I created this:#2016-11-2514:02slipset
(s/fdef foo
        :args (s/fspec :args any?
                       :ret any?)
        :ret 3)
(defn foo [f] (f 3))
#2016-11-2514:02slipsetSo I do (stest/instrument `foo)#2016-11-2514:03slipsetand running (foo identity) gives me something like:#2016-11-2514:04slipset
ExceptionInfo Call to #ā€˜bar/foo did not conform to spec:
val: (#function[clojure.core/identity]) fails at: [:args] predicate: ifn?
:clojure.spec/args  (#function[clojure.core/identity])
:clojure.spec/failure  :instrument
:clojure.spec.test/caller  {:file "form-init100394334971998293.clj", :line 917, :var-scope bar/eval55924}
  clojure.core/ex-info (core.clj:4617)
#2016-11-2514:04slipsetwhat am I missing?#2016-11-2514:08slipset
(s/fdef foo
        :args (s/cat :function ifn?)
        :ret 3)
(defn foo [f] (f 3))
#2016-11-2514:08slipsetworks, but leaves stuff to be desired#2016-11-2514:12slipsetok, got it#2016-11-2514:12slipset
(s/fdef foo
        :args (s/cat :function (s/fspec :args (s/cat :arguments number?)
                                        :ret any?))
        :ret 3)
(defn foo [f] (f 3))
#2016-11-2521:03leongrapenthinhttp://dev.clojure.org/jira/browse/CLJ-2013 - Are there plans to fix this for 1.9? It would greatly increase error message quality. Is work on it welcome?#2016-11-2522:05Alex Miller (Clojure team)Sure#2016-11-2603:47lvhIs there a way to specify that I expect the value of a var to be valid according to a particular spec, such that it is automatically tested?#2016-11-2603:48lvhSometimes theyā€™re just data structures, but they might also be the result of a higher order function#2016-11-2604:59lvhAlso; why isnā€™t there a map-of that takes specs instead of preds?#2016-11-2605:56seancorfield@lvh map-of does take specs...?#2016-11-2605:58seancorfield(s/map-of ::foo ::bar) works...#2016-11-2606:04Alex Miller (Clojure team)@lvh you could just check s/valid? when you set the var. You can use set-validator! on vars but Iā€™m assuming youā€™re not actually changing it after you set it.#2016-11-2613:37kestrel7Is it possible to inspect a spec to get, e.g., the list of keys defined by (s/keys :req [a b c])? Iā€™ve created a spec for a map and want to retrieve the list of keys expected for that map for a purpose other than checking the spec.#2016-11-2613:57dergutemoritz@kestrel7 clojure.spec/form is what you're looking for#2016-11-2614:02kestrel7Perhaps a noob question but from that how do I extract the actual list of keys from the resulting structure, which in this case looks like the following example but could I guess be more complex? Naively I could just say (last (s/form the-spec)) but thatā€™s not going to be robust if the form contents change. (clojure.spec/keys :req [:k.specs.person/title :k.specs.person/first-name :k.specs.person/other-names :k.specs.person/last-name])#2016-11-2614:03dergutemoritzThat's the tricky bit, hehe#2016-11-2614:03kestrel7LOL#2016-11-2614:04kestrel7I guess I could use core.match somehow. It would be useful to get the spec as data thatā€™s easier to interrogate. Thanks for the help @dergutemoritz#2016-11-2614:05dergutemoritzYou can do something like that for the simple case of a single s/keys spec. However, when stuff like s/merge is involved, it gets a lot trickier#2016-11-2614:05dergutemoritzIt would be useful to have specs for those forms so you could s/conform them.#2016-11-2614:06kestrel7For the moment Iā€™m going to cheat: (def the-keys [::a ::b ::c] (s/def ::m (s/keys :req the-keys)
#2016-11-2614:06dergutemoritzI'm not sure if anyone has put effort into defining such specs already. I was gonna take a stab at that myself when I find some time#2016-11-2614:06dergutemoritz@kestrel7 That cheat won't work I'm afraid#2016-11-2614:07dergutemoritzThe keys vector needs to be literal IIRC#2016-11-2614:07dergutemoritzBecause it's processed at macro expansion time#2016-11-2614:07dergutemoritzRight, just try it in the REPL and you'll see#2016-11-2614:09kestrel7@dergutemoritz youā€™re right šŸ˜ž#2016-11-2614:10dergutemoritz@kestrel7 (apply hash-map (rest (s/form your-keys-spec))) should give you a reasonable data structure to work with for bare s/keys specs#2016-11-2614:11kestrel7@dergutemoritz Iā€™ll try that.#2016-11-2614:19kestrel7Hereā€™s my workaround for now#2016-11-2614:19kestrel7(defn spec-keys "Return the :req keys from the spec" [spec] (let [spec-form (s/form spec)] (assert (= 'clojure.spec/keys (first spec-form))) (:req (apply hash-map (rest spec-form)))))#2016-11-2614:32dergutemoritzShould do the trick#2016-11-2614:32dergutemoritzYou could destructure spec-form right there in the let but I guess that's just cosmetics šŸ™‚#2016-11-2615:13lvh@seancorfield huh! I knew that compiled but I assumed it was treating the keyword as a predicate#2016-11-2616:14dergutemoritzIs there a way to unwrap a regex op context that was previously created with s/spec?#2016-11-2616:16dergutemoritzHmm regex-spec-impl doesn't look like it offers something like that#2016-11-2618:21Alex Miller (Clojure team)I have specs for all the forms and those will be published, but I'm still working through fixing all the bugs in form first#2016-11-2618:21Alex Miller (Clojure team)With those, you will be able to conform any spec form to get at its parts#2016-11-2619:46dergutemoritzAh heck šŸ™‚ throws his Saturday afternoon hack into the trash bin#2016-11-2619:52dergutemoritzWell, I'll keep it around, perhaps I came up with some things that could be added to your implementation#2016-11-2715:17ikitommithe new specize seems to turn any function into a spec. As collections are also functions, this gives bit unintuitive results:
(s/valid? {:a 1 :b 2} :b) ; true
(s/valid? #{1 2 3} 3)     ; true
(s/valid? [1 2 3] 1)      ; true
(s/valid? [1 2 3] 4)      ; java.lang.IndexOutOfBoundsException
(s/conform > 10)          ; true
(s/conform > >)           ; #object[clojure.core$_GT_
#2016-11-2715:24ikitommishould the collections have special behavior with specize? would not turn into specs? That would help my case, I would like to identify and convert nested vectors and sets into collection specs via my coll-spec macro, but now they can be used as (strangely behaving) specs. Set seems to be the only one of those three that could act as a spec itself. But there could be s/set for it?#2016-11-2715:33Alex Miller (Clojure team)What's unintuitive about the above?#2016-11-2715:34Alex Miller (Clojure team)Any function that takes 1 arg can be used as a spec#2016-11-2715:35Alex Miller (Clojure team)Sets, maps, and vectors satisfy that#2016-11-2715:35Alex Miller (Clojure team)It's very intentional that this works#2016-11-2715:37ikitommiset as a spec means ā€œone of the valuesā€ and it has a generator. what does a vector as a spec mean?#2016-11-2715:46Alex Miller (Clojure team)Nothing very useful but it would mean "has a valid index"#2016-11-2715:47ikitommiok, thanks for the answers.#2016-11-2717:46yonatanelIs there a way to directly reuse a spec defined as multimethod implementation? I mean just one of them.#2016-11-2719:43dergutemoritz@yonatanel How do you mean "directly"?#2016-11-2719:44dergutemoritzDoes calling the multimethod with an appropriate value qualify? šŸ™‚#2016-11-2721:44yonatanelNo, I want to define a similar spec by using the one I already defined with one of the multimethod instances, without calling it with a fake map just for the dispatch value. In the end I just defined the common spec as a keyword but I thought there might be a trick to do that.#2016-11-2721:49Alex Miller (Clojure team)Why don't you do the reverse? Define the spec and then have the multimethod return it.#2016-11-2721:52yonatanel@alexmiller That's what I ended up doing.#2016-11-2814:30vikeriHmm, I have an issue with generating data for a multi-spec. It works fine for testing with s/valid? but not when using (gen/generate (s/gen ::myspec)). I have the following setup:
(defmulti myspec first)
(defmethod myspec :option/a [_] (s/cat :opt #{:option/a} :val boolean?))
(defmethod myspec :option/b [_] (s/cat :opt #{:option/b} :val string?))
(s/def ::myspec (s/multi-spec myspec first))
Then the following crashes:
(gen/generate (s/gen ::myspec))
;; #object[Error Error: :option/b is not ISeqable]
Iā€™m on cljs FYI.
#2016-11-2815:48Alex Miller (Clojure team)the problem is your retag function (first)#2016-11-2815:49Alex Miller (Clojure team)re the docs "retag is used during generation to retag generated values with matching tags. retag can either be a keyword, at which key the dispatch-tag will be assoc'ed, or a fn of generated value and dispatch-tag that should return an appropriately retagged value."#2016-11-2815:50Alex Miller (Clojure team)in this case, I think the value generated by each method spec is probably fine without modification, so you can just use (fn [val tag] val)#2016-11-2815:50Alex Miller (Clojure team)note that you canā€™t easily use the anon function syntax here as it must be a function with arity 2#2016-11-2815:51Alex Miller (Clojure team)you could sneakily use it via something like #(first %&) but I find the fn version to communicate much better#2016-11-2816:07vikeri@alexmiller Awesome, honestly I read the docs but did not quite understand the retagging partā€¦ Now it works as expected.#2016-11-2819:03yonatanelWill specs be re-resolved on Component system restart/refresh?#2016-11-2819:06zaneSpecs are re-added to the registry when you use c.t.n.r/refresh and the like, but any deleted specs will not be removed from the registry.#2016-11-2819:06zaneIf that helps.#2016-11-2819:14yonatanelThanks#2016-11-2819:39yonatanelIs this a bug? s/merge specs are not conforming in this case:
(s/def ::status (s/conformer (fn [x] (or (keyword x) ::s/invalid))))
=> :dev/status
(s/def ::a (s/keys :req-un [::status]))
=> :dev/a
(s/def ::b (s/merge ::a (s/keys :req-un [::id])))
=> :dev/b
(s/conform ::b {:id 1 :status "hi"})
=> {:id 1, :status "hi"}
(s/conform ::a {:id 1 :status "hi"})
=> {:id 1, :status :hi}
#2016-11-2819:39yonatanel"hi" should become a keyword#2016-11-2819:55Alex Miller (Clojure team)Merge does not flow conformed values like and#2016-11-2819:55Alex Miller (Clojure team)You'll get the conformed value of the last spec#2016-11-2819:58Alex Miller (Clojure team)Each spec basically has to independently conform for the merge to succeed#2016-11-2819:59Alex Miller (Clojure team)If you swap the order in your merge you should get what you want#2016-11-2820:02yonatanelDon't you want the behavior to be more like map merge? First determine the keys and then conform them.#2016-11-2820:02Alex Miller (Clojure team)No#2016-11-2820:03Alex Miller (Clojure team)That's not what merge is#2016-11-2820:06yonatanelOK. I could read the docs that way.#2016-11-2820:11yonatanelI see bare (s/keys) doesn't conform either.#2016-11-2820:12yonatanelI wonder how clear these things should be in the docs or if I'm pushing it too far.#2016-11-2821:01potetmIs there a way to spec a multimethod where you can add to the spec in an open-ended way?#2016-11-2821:01potetmJust like you can add to the multimethod in an open-ended way.#2016-11-2822:51mathpunkI had an idea to define a spec, ::item, that specifies that a thing either has an :id key or that it can have a function called id called on it. I'm not sure, though, if that's a job for protocols instead of specs.#2016-11-2822:56mathpunkThe other thing I'm struggling with is, I can use specs when I define them in the namespace in which I want to use them, but I don't understand how to require them from another namespace. I gathered that spec/def registered them globally somehow, but other-name-space/foo does not seem available.#2016-11-2823:28hiredmanit does register them globally, but if you don't load the code it doesn't happen#2016-11-2823:31Alex Miller (Clojure team)@mathpunk re the first question - you could do that but you would need some way to tell that a function can do that (and protocols is maybe the best way) - something like (s/def ::item (s/or :has-key (s/keys :req-un [:id]) :has-fn #(satisfies? HasId %)))#2016-11-2823:32Alex Miller (Clojure team)where HasId is (defprotocol HasId (id [x]))#2016-11-2823:32Alex Miller (Clojure team)re the second, as hiredman said, they are registered globally and can be referred to by (qualified-keyword) name, assuming you loaded the code that registered them#2016-11-2908:13mathpunk@alexmiller That's a very helpful example, thank you.#2016-11-2908:22mathpunk@alexmiller @hiredman: As for loading the code that registered them -- I was missing that using ::foo expands to the whole namespace name i.e. including the name of the app. The 'fully' in fully qualified šŸ™‚#2016-11-2914:32nwjsmithAre there plans to dynaload the shuffle combinator from test.check?#2016-11-2914:33nwjsmithWould a patch be welcome for such a change?#2016-11-2914:34Alex Miller (Clojure team)I think thereā€™s actually a pending patch that has that included in it#2016-11-2914:35Alex Miller (Clojure team)but yes, would be happy to add it#2016-11-2914:35nwjsmithstarts digging through Jira to upvote#2016-11-2914:36Alex Miller (Clojure team)yeah, itā€™s in http://dev.clojure.org/jira/browse/CLJ-2046#2016-11-2914:36Alex Miller (Clojure team)which is ready for Rich to look at so I expect that will happen whenever heā€™s looking at tickets next#2016-11-2914:36nwjsmithExcellent, thanks Alex#2016-11-2920:09bbloomcreate-ns and alias seem relatively useful for spec, but clojurescript doesnā€™t really support these - any good alternative other than verbose keywords?#2016-11-2920:11Alex Miller (Clojure team)prob best to ask this in #clojurescript or #cljs-dev#2016-11-2920:11bbloomok#2016-11-2920:31dnolen@alexmiller I think this is good argument for :alias at the ns form#2016-11-2920:31dnolenitā€™s already come up before#2016-11-2920:31dnolenand I believe continued usage of clojure.spec will drive the desire for such a feature#2016-11-2920:32Alex Miller (Clojure team)well youā€™re talking to the wrong person :)#2016-11-2923:26jfntnIs there a way to add meta data to spec definitions without re-implementing s/def?#2016-11-2923:41hiredmanmetadata?#2016-11-2923:43hiredmanspecs are "named" by symbols (which can have metadata attached) and keywords (which cannot)#2016-11-2923:44hiredmanbut symbols are not interned, so given a symbol 'a if you associated some metadata, another instance of that symbol will not have that metadata#2016-11-2923:45hiredmanby that I mean, even if you did re-implement s/def you would have a hard time putting metadata somewhere#2016-11-2923:48Alex Miller (Clojure team)The answer is no (for now) but it would be useful to have at least a doc string and I'd say that's still a possibility#2016-11-2923:48Alex Miller (Clojure team)As hiredman mentioned, the tricky part is where to put it#2016-11-2923:49Alex Miller (Clojure team)There are some options but none that are obviously good#2016-11-3000:29Alex Miller (Clojure team)https://github.com/prayerslayer/js.spec#2016-11-3020:02nwjsmithThereā€™s also https://github.com/settinghead/specky#2016-12-0100:15stathissiderishello, do function specs and other specs end up in different parts of the registry?#2016-12-0100:15stathissiderisfor example, I have a call like that: (foo {::foo 1})#2016-12-0100:15stathissideriscan I safely do (s/def ::foo integer?) and (s/fdef ::foo ā€¦)#2016-12-0100:17Alex Miller (Clojure team)No, those will collide (and the fdef will not be used)#2016-12-0100:17Alex Miller (Clojure team)Def puts keyword keys in the registry and fdef puts symbol keys in the registry#2016-12-0100:22stathissiderisoh so my example should really be (s/fdef foo ā€¦) instead, in which case no collision#2016-12-0100:22stathissideriscorrect?#2016-12-0100:32stathissiderisI have another one: is there a shorter form for (let [{:keys [::stats/foo ::stats/bar]} {::stats/foo 2}] foo)#2016-12-0100:33stathissideris@alexmiller btw, I enjoyed your interview for the defn podcast, quite intriguing (all the stuff about secret future plans for clojure)#2016-12-0100:33stathissideristhanks#2016-12-0100:47Alex Miller (Clojure team)@re collision - right#2016-12-0100:48Alex Miller (Clojure team)Re destructuring, yes#2016-12-0100:49Alex Miller (Clojure team)Do ::stats/keys [foo bar]#2016-12-0101:28clojuregeekLive stream from AustinClojure meeting by Stu on spec https://m.youtube.com/watch?v=dQcNAscSTSw#2016-12-0103:28bbloomiā€™m totally blown away by how i have a namespace with > 100 defns, only 4 fundamental ones of which have specs, and they help me catch tons of errors - super nice#2016-12-0103:29bbloomthe return value of instrument reminds me of just how little spec you actually need to get some value from it#2016-12-0103:29bbloomvs mandatory type checking everywhere#2016-12-0105:57naomarikthe official spec guide is very well written, had no issues consuming it to once for deep comprehension and also been able to quickly scan it for answers#2016-12-0105:57naomarikjust want to say thanks for that effort šŸ˜‰#2016-12-0109:51mishadoffHey, I experienced an issue in clojure.spec where :fn key in fdef is ignored (clojure 1.9-alpha14) Here is MVP demonstrating the problem.
(defn add [x y]
  #_(+ x y)
  2)

(s/fdef add
        :args (s/cat :x number? :y number?)
        :ret number?
        :fn #(= (+ (-> % :args :x)
                   (-> % :args :y)) (:ret %)))

(clojure.spec.test/instrument `add)
If I call (add 1 nil) I got spec error, but when I call (add 1 2) spec says everything is ok, despite the fact add always returns 2 Can anybody help me?
#2016-12-0110:06joost-diepenmaat@mishadoff AFAICT instrument only checks :args (and apparently not :ret or :fn)#2016-12-0110:06joost-diepenmaatthis is surprising to me too but thatā€™s how itā€™s documented:#2016-12-0110:07joost-diepenmaatfrom the instrument docstring#2016-12-0110:13pithyless@mishadoff @joost-diepenmaat - this has got to be FAQ #1 on clojure-spec šŸ™‚ clojure.spec.test/instrument will only check :args (itā€™s meant to throw errors if you are using the function incorrectly); clojure.spec.test/check will run generative tests and check that the function is implemented correctly.#2016-12-0110:22mishadoffGot it#2016-12-0110:22mishadoffStrange, but.. Thanks šŸ™‚#2016-12-0113:56ikitommiBlogged about Schema & Clojure Spec for Web Developers. http://www.metosin.fi/blog/schema-spec-web-devs/ hopefully everyone's spec lib got mentioned.#2016-12-0114:41nha@ikitommi the link to yada is wrong. Nice article šŸ™‚#2016-12-0115:05ikitommi@nha thanks! fixed the link.#2016-12-0115:49camdezWell, I decided to finally give spec a whirl, moved to 1.9.0-alpha14, and promptly discovered that I canā€™t get a REPL running because one of my dependencies has a non-conforming defn. šŸ˜’ Anything I can do here short of replacing / removing the (slightly busted) library?#2016-12-0115:51Alex Miller (Clojure team)Most offending libs have newer versions that fix these problems#2016-12-0115:52camdez@alexmiller: Iā€™m awareā€”you actually patched the library in question šŸ˜„ (Aleph). But a new release hasnā€™t been cut yet.#2016-12-0115:52camdezBut Iā€™m curious conceptually what the options are here.#2016-12-0115:52camdezCertainly I can cut my own release.#2016-12-0115:52Alex Miller (Clojure team)You could s/fdef a dummy spec over the defn spec#2016-12-0115:53Alex Miller (Clojure team)But might be tricky to get that loaded at the right time#2016-12-0115:53camdez@alexmiller: Gotcha. But Iā€™d basically be overriding it globally, right?#2016-12-0115:53camdezFor all defns#2016-12-0115:53Alex Miller (Clojure team)Yes#2016-12-0115:53camdezOr no?#2016-12-0115:53camdezk. Thanks.#2016-12-0115:59camdezAh, my mistakeā€”there is an alpha release of Aleph with a conforming defn. (Didnā€™t release lein ancient wouldnā€™t show that.)#2016-12-0123:08bbloomi saw some discussion yesterday about docstrings#2016-12-0123:08bbloomwould be super nice to have some mechanism for that#2016-12-0123:08bbloomiā€™ve got some metadata on some objects that iā€™d like to document better than with a comment#2016-12-0123:09Alex Miller (Clojure team)Still a possibility#2016-12-0123:10bbloomcool#2016-12-0123:12Alex Miller (Clojure team)There is not an obvious place to hang it atm, but there are some options#2016-12-0123:12bbloomsyntax: the cause of and solution to all of our problems#2016-12-0123:17Alex Miller (Clojure team)I don't mean syntax, I mean where to remember it#2016-12-0123:17Alex Miller (Clojure team)Could be easily added to s/def#2016-12-0123:18bbloomoh - i thought the (a?) challenge was no metadata on keywords#2016-12-0123:32Alex Miller (Clojure team)You don't have to provide it as metadata (or store it as metadata)#2016-12-0213:47stathissideris@alexmiller destructuring like that throws an exception: (let [{::stats/keys [foo bar]} {::stats/foo 2}] foo)#2016-12-0213:47stathissideris:stats/keys works, but thatā€™s not what I meant#2016-12-0213:48stathissiderisbecause I have this in my require [spec-provider.stats :as stats]#2016-12-0213:54dergutemoritz@bbloom The relevant ticket to vote on is http://dev.clojure.org/jira/browse/CLJ-1965 šŸ™‚#2016-12-0213:54dergutemoritz@stathissideris That should work AFAICT. What's the exception you get?#2016-12-0213:55stathissideris
spec-provider.trace> (let [{::stats/keys [foo bar]} {::stats/foo 2}] foo)
ExceptionInfo Call to clojure.core/let did not conform to spec:
In: [0 0] val: #:spec-provider.stats{:keys [foo bar]} fails spec: :clojure.core.specs/local-name at: [:args :bindings :binding :sym] predicate: simple-symbol?
In: [0 0 0] val: ([:spec-provider.stats/keys [foo bar]]) fails spec: :clojure.core.specs/seq-binding-form at: [:args :bindings :binding :seq] predicate: (cat :elems (* :clojure.core.specs/binding-form) :rest (? (cat :amp #{(quote &)} :form :clojure.core.specs/binding-form)) :as (? (cat :as #{:as} :sym :clojure.core.specs/local-name))),  Extra input
In: [0 0 :spec-provider.stats/keys] val: [foo bar] fails spec: :spec-provider.stats/keys at: [:args :bindings :binding :map :spec-provider.stats/keys :clojure.spec/pred] predicate: map?
In: [0 0 :spec-provider.stats/keys] val: [foo bar] fails spec: :spec-provider.stats/keys at: [:args :bindings :binding :map :spec-provider.stats/keys :clojure.spec/nil] predicate: nil?
:clojure.spec/args  ([#:spec-provider.stats{:keys [foo bar]} #:spec-provider.stats{:foo 2}] foo)
  clojure.core/ex-info (core.clj:4725)
#2016-12-0213:56stathissiderisIā€™m on clojure 1.9.0-alpha14#2016-12-0213:57bronsaI can't repro:
user=> (let [{::stats/keys [foo bar]} {::stats/foo 2}] foo)
2
#2016-12-0214:00dergutemoritzYeah, same here. Something must be off in your namespace @stathissideris#2016-12-0214:01stathissiderisweird, thanks for trying @bronsa and @dergutemoritz#2016-12-0214:02dergutemoritz@stathissideris You're welcome! If all else fails, try rebooting the REPL šŸ˜‰#2016-12-0214:03stathissiderisREPL reboot didnā€™t help#2016-12-0214:03stathissiderisI think Iā€™ll try in a fresh project#2016-12-0214:04bronsatry *clojure-version* to confirm you're on alpha14?#2016-12-0214:06bronsa@stathissideris hmm, do you have a ::stats/keys spec by any chance?#2016-12-0214:06dergutemoritzOooh#2016-12-0214:07dergutemoritzThat'd be nasty#2016-12-0214:07bronsayeah i can reproduce a similar exception by defining a dummy ::stats/keys spec#2016-12-0214:07dergutemoritzGood thinking @bronsa#2016-12-0214:14bronsainteresting, not sure if this was intentional but one can do this:
user=> (require '[clojure.spec :as s])
nil
user=> (s/def ::keys (s/coll-of '#{foo bar} :kind vector?))
:user/keys
user=> (let [{::keys [foo]} {}])
nil
user=> (let [{::keys [baz]} {}])
ExceptionInfo Call to clojure.core/let did not conform to spec:
In: [0 0] val: #:user{:keys [baz]} fails spec: :clojure.core.specs/local-name at: [:args :bindings :binding :sym] predicate: simple-symbol?
In: [0 0 0] val: ([:user/keys [baz]]) fails spec: :clojure.core.specs/seq-binding-form at: [:args :bindings :binding :seq] predicate: (cat :elems (* :clojure.core.specs/binding-form) :rest (? (cat :amp #{(quote &)} :form :clojure.core.specs/binding-form)) :as (? (cat :as #{:as} :sym :clojure.core.specs/local-name))),  Extra input
In: [0 0 :user/keys 0] val: baz fails spec: :user/keys at: [:args :bindings :binding :map :user/keys] predicate: (quote #{bar foo})
:clojure.spec/args  ([#:user{:keys [baz]} {}])
  clojure.core/ex-info (core.clj:4725)
#2016-12-0214:15bronsaif intentional, your issue is not a bug, otherwise I'd say it's a spec/`:clojure.core.spec/ns-keys` bug#2016-12-0214:15bronsa\cc @alexmiller#2016-12-0214:18stathissideris@bronsa OH! yes, I do have a ::stats/keys spec#2016-12-0214:20stathissiderisit doesnā€™t have to be called that, but it was the most natural name for what Iā€™m doing#2016-12-0214:21bronsayeah I'm looking at the bindings spec and I don't think that's intentional behaviour#2016-12-0214:30stathissideris@bronsa good spot!#2016-12-0214:33bronsayeah looks like a bug with s/keys :opt-un#2016-12-0214:33bronsaminimal repro:
user=> (s/def ::foo nil?)
:user/foo
user=> (s/def ::bar (s/keys :opt-un [::foo]))
:user/bar
user=> (s/valid? ::bar {:a/foo 1})
true
user=> (s/def :a/foo nil?)
:a/foo
user=> (s/valid? ::bar {:a/foo 1})
false
#2016-12-0214:35dergutemoritz@bronsa How's that wrong?#2016-12-0214:35Alex Miller (Clojure team)You need to redef bar for that to take effect#2016-12-0214:35Alex Miller (Clojure team)Specs are lazily compiled on use#2016-12-0214:36dergutemoritzNote that bit from the s/keys docstring: "In addition, the values of all namespace-qualified keys will be validated (and possibly destructured) by any registered specs."#2016-12-0214:36Alex Miller (Clojure team)And cached#2016-12-0214:36bronsa@dergutemoritz ah, then it's not a bug :)#2016-12-0214:36Alex Miller (Clojure team)Oh, that too#2016-12-0214:37bronsaleads to surprising results tho#2016-12-0214:37dergutemoritzI think this is unrelated to the keys destructuring issue @stathissideris ran into#2016-12-0214:37bronsa@dergutemoritz no, that's the same issue @stathissideris is having#2016-12-0214:37Alex Miller (Clojure team)There are some weird corners with unqualified keys #2016-12-0214:38dergutemoritzOK then I didn't get the connection, yet šŸ™‚#2016-12-0214:38bronsathe let binding destructuring specs are defined in terms of s/keys, which is why he's getting ::stats/keys validated against his ::stats/keys spec#2016-12-0214:38dergutemoritzAh, I see#2016-12-0214:38Alex Miller (Clojure team)Yeah, that's pretty subtle :)#2016-12-0214:39dergutemoritzI wonder if this is intended behavior in this particular situation, though šŸ™‚#2016-12-0214:39bronsait feels a bit weird that that happens purely because of how those specs are defined (i.e. it depends on an implementation detail)#2016-12-0214:40Alex Miller (Clojure team)We've talked about this a bit and I think this is a case where some effort could be taken to detect and warn (or maybe error)#2016-12-0214:40bronsarealistically this means using ::keys or ::strs as spec is going to lead to those issues most of the times (unless your ::keys spec is literally a coll of simple-syms)#2016-12-0214:41Alex Miller (Clojure team)It is in the ballpark of being open in your map specs or kwarg options#2016-12-0214:41dergutemoritzAn option to disable this behavior for particular keys specs might be useful, too#2016-12-0214:42Alex Miller (Clojure team)I'll think about it#2016-12-0214:42Alex Miller (Clojure team)That's already a pretty tricky hybrid map spec#2016-12-0214:42bronsacool, I'll open a ticket later to track this#2016-12-0214:45dergutemoritzGreat find everyone, I'm happy I probably won't have to run into this case myself some day and scratch my head in confusion šŸ˜„#2016-12-0215:25bronsahttp://dev.clojure.org/jira/browse/CLJ-2074#2016-12-0215:25bronsaattached a proposed solution#2016-12-0215:34Alex Miller (Clojure team)We are absolutely not doing that#2016-12-0215:35Alex Miller (Clojure team)I don't expect to make any adjustments to keys for this, but rather alter how the destructuring spec is written#2016-12-0215:36bronsafair enough#2016-12-0300:43bblooms/merge makes me so happy.#2016-12-0300:44bbloombeing able to just mix in key sets for various stages of the pipeline#2016-12-0300:44bbloombeautiful.#2016-12-0301:01bbloomthe go type system has ā€œstruct embeddingā€ which i fell in love with when working in go - lets you accomplish something similar by merging struct fields#2016-12-0318:48bbloomiā€™m not quite sure yet i grok the every vs coll-of, or every-kv vs map-of stuff yet#2016-12-0318:48bbloomis the general idea to use coll-of and map-of by default and only switch to every if your specs are too slow and you donā€™t need conformance#2016-12-0318:48bbloom?#2016-12-0318:50bbloomalso - iā€™m now running in to a situtation where a spec is not-conforming and itā€™s just consuming a ton of memory producing the explanation data#2016-12-0319:14arohner@bbloom I havenā€™t run into that situation, but thatā€™s my guess#2016-12-0319:16bbloomyeah, i get the feeling explain needs to become lazy somehow - not necessarily with thunks, but maybe with something like a ā€¦ that you can ask it to expand further#2016-12-0319:47bbloomhmm, yeah - i have a structure with LOTS of structural sharing and a ton of redundant data - not unlike the clojurescript ast#2016-12-0319:48bbloomand explain is starting to become non-useful due to the shear volume of output#2016-12-0319:48bbloomand seemingly exponential time to return#2016-12-0319:48bbloomor print or whatever#2016-12-0319:48bbloomnot quite able to pin down a reproduction yet#2016-12-0321:07Alex Miller (Clojure team)You can set a custom explain printer if that becomes useful#2016-12-0321:08Alex Miller (Clojure team)Just setting print-length will affect the default one though#2016-12-0321:08Alex Miller (Clojure team)In case that helps#2016-12-0321:09Alex Miller (Clojure team)Re your collection question, yes that's a good first approximation#2016-12-0321:09bbloomre: print-length - cool! thanks.#2016-12-0321:10Alex Miller (Clojure team)You can also set coll-check-limit to further tweak what every and every-kv check#2016-12-0321:10bbloomsuuuuuppper nit-picky feedback:#2016-12-0321:11bbloomif that first-approximation is true, that you should default to coll-of or map-of, then it seems like the more informative doc-strings should be on those, rather than every and every-kv#2016-12-0321:11bbloomfeel free to totally ignore i even said that šŸ˜›#2016-12-0321:13Alex Miller (Clojure team)Rich always thought people should default the other way#2016-12-0321:13Alex Miller (Clojure team)But in general most people seem to disagree :)#2016-12-0321:13bbloompreferring the sampling kind?#2016-12-0321:13Alex Miller (Clojure team)Yeah#2016-12-0321:13bbloomhm, i was going back and forth on it in my head#2016-12-0321:14bbloomseeming like sampling is the right thing if the specs are not intended to be used for contract-style checks#2016-12-0321:14bbloomwhich seems true#2016-12-0321:14bbloombut it feels wrong to get non-determinstic behavior by default#2016-12-0321:14Alex Miller (Clojure team)I would actually prefer that the doc for both covered all the details, but you know#2016-12-0321:14Alex Miller (Clojure team)Rich isn't big on duplication like that#2016-12-0321:14bbloomthe slippery slope of excessive doc strings - i like the short strings#2016-12-0321:15Alex Miller (Clojure team)I have a spec reference page in the works for the docs which will help too#2016-12-0321:15bbloomi wish the see also was metadata tho, so the cross referencing was more automatic#2016-12-0321:15bbloomand maybe even let you see-also a spec#2016-12-0321:15bbloomso like the options that are specified could be generated from common doc strings#2016-12-0321:15Alex Miller (Clojure team)If wishes were fishes something something #2016-12-0321:16bbloomheh, indeed - the simple strings are generally fine#2016-12-0321:16Alex Miller (Clojure team)Later, shutting down for takeoff#2016-12-0321:16bbloomaaaaannny way, i guess iā€™ll lean on the sampling ones & see how that feels#2016-12-0321:16bbloomcya, have a good flight#2016-12-0321:52paytonrulesHow do you spec a multi-method? Multi-spec seems like it should do it, but it only talks about specā€™ing a hash with types. More specifically I have the following - but it never validates the :box parameter.
(s/fdef colliding?
  :args (s/cat :invader map? :box ::box/box)
  :ret boolean?)

(defmulti colliding? :character)

(defmethod colliding? ::small [invader box]
#2016-12-0322:13Alex Miller (Clojure team)You can't right now#2016-12-0322:14Alex Miller (Clojure team)Probably fixable but I haven't looked at it yet#2016-12-0322:15paytonrulesOh really?#2016-12-0322:16paytonrulesKinda shocked - I had assumed it was me the whole time.#2016-12-0322:17paytonrulesMight wanna make it clearer in the docs that itā€™s not supported. All I could find was some confusion on multi-spec.#2016-12-0322:17paytonrulesThanks for the help tho!#2016-12-0322:20Alex Miller (Clojure team)Yeah, multimethod, protocols, primitive typed fns, and inlined fns will not work right now #2016-12-0322:21Alex Miller (Clojure team)Inlined and protocols are probably not easily fixable#2016-12-0322:37bbloomuser=> (binding [s/recursion-limit 1 s/coll-error-limit 1 s/coll-check-limit 1] (clojure.test/run-tests ā€˜ambiparse.calc-test)) That normally returns in ~1 second, but when instrumented, it seems to be infinitely looping. jstack says: https://gist.github.com/brandonbloom/dfa559f81ad1b6d0980c120d86a2e18e#2016-12-0322:37bbloomsomething seems to not be respecting the limits#2016-12-0322:37bbloomor iā€™m missing a limit#2016-12-0322:38bbloomamusingly, iā€™m working on a parser, soooo iā€™m running in to exactly the same kinds of infinite loop problems in the program iā€™m trying to debug šŸ˜‰#2016-12-0322:58derwolfeHi - I'm experimenting with spec and am getting an error that I'm not understanding
RuntimeException Var clojure.test.check.generators/large-integer is not on the classpath  clojure.spec.gen/dynaload (gen.clj:21)
. Any tip as to what could be the problem? This is happening when I'm trying to exercise a simple spec I've written.
#2016-12-0323:02bbloomlooking at the spec impl, it seems like there isnā€™t on spec checking level/depth - which might be what i want/need, not sure tho#2016-12-0323:27Alex Miller (Clojure team)It's entirely possible some spec impl is not respecting the limit#2016-12-0323:27Alex Miller (Clojure team)@derwolfe: you need test.check on your classpath#2016-12-0323:30derwolfe@alexmiller thanks!#2016-12-0410:08tslockeIā€™m guessing this comes up a lot - the fact that conformed values flow through the validation process seems to make it difficult to compose specs. Say Iā€™m writing some preconditions for a function, I have to look at the spec definitions to know if Iā€™m going to get, say, raw values, or [tag value] pairs. Am I missing something?#2016-12-0410:18tslockeOK just discovered s/unform. Adds some verbosity to some specs, but at least removes the dependency.#2016-12-0412:47tslockeHmmm doesnā€™t seem to work with s/coll-of#2016-12-0420:03gfredericks@tslocke I saw some speculation a while back that spec will eventually have two versions of s/and; not sure if that's exactly what you were referring to#2016-12-0421:14bbloomyeaaah, still not quite sure whatā€™s causing an infinite loop or otherwise such bad perf it might as well be infinite - best i can offer at the moment is these stack traces: https://gist.github.com/brandonbloom/dfa559f81ad1b6d0980c120d86a2e18e#2016-12-0421:14bbloomdefinitely related to the collection stuff tho b/c every makes it random, coll-of makes it every time#2016-12-0421:42bnoguchi@bbloom: ran into the same issue recently as well#2016-12-0421:42bbloom@bnoguchi were you able to shrink the reproduction down at all? i havenā€™t been able to yet#2016-12-0421:47bnoguchiNot quite. I had to hit the brakes on debugging it but am taking a closer look again at it this week.#2016-12-0504:43jfntnFinally have a good example of an issue I keep running into with s/keys. Here is a polymorphic payload for a couple of websocket messages:
{:msg/id :foo/msg, :msg/data { ... foo/data ... }}
{:msg/id :bar/msg, :msg/data { ... bar/data ... }}
My basic intuition is to have a multi-spec that dispatches on :msg/id :
(defmulti msg-spec :msg/id)
But then I have a problem, because methods need to deal with a polymorphic :msg/data too:
(defmethod msg-spec :foo/msg [_] (s/keys :req [:msg/data !?]))
(defmethod msg-spec :bar/msg [_] (s/keys :req [:msg/data !?]))
All I can think of is to have the multi-spec on :msg/data instead. But that dispatch function will be much more complex and will need to infer, from the :msg/data alone, the value that was right there in :msg/id. Am I missing something?
#2016-12-0504:50Alex Miller (Clojure team)You can base your polymorphism on more than just a key - it's an arbitrary function#2016-12-0504:51Alex Miller (Clojure team)So you could base it on both id and something in msg#2016-12-0504:52Alex Miller (Clojure team)Or you could do more than one level of multispec#2016-12-0504:54jfntnRight, I think I understand that and it seems to be the problem. The :msg/id is all I need to determine the type of :msg/data but itā€™s only available at that root level#2016-12-0504:55jfntnWith a multi-spec for :msg-data Iā€™d need to look at what keys are in there, and in some cases what values, just to derive a tag was just there in the parent map#2016-12-0505:14jfntnI've heard Rich's rebuttal of this kind of "contextual polymorphism" , but this example feels like something that will come up in practice, especially as people adopt qualified keywords in their apis. An ad-hoc binding of a map-key to a spec would be a bulletproof one-liner alternative to what would now have to be a complex, possibly buggy dispatch function.#2016-12-0505:50Alex Miller (Clojure team)So are you saying that :mag/data has more than one spec? That seems wrong.#2016-12-0514:38jfntnYes indeed. The websocket library is embedding the msg/tag and the msg/data in its payload. It takes a [msg/tag msg/data] and we get this map.#2016-12-0514:38jfntnSo we have multiple specs for both the msg itself (omitted other keys that need specs) and the msg/data, both of which ultimately depend on the msg/tag.#2016-12-0514:40jfntnCurious whatā€™s your intuition on why this is wrong? This is something that comes up a lot in my experience whit what are now unqualified library apis, but it seems bound to happen more as people adopt namespaced kws?#2016-12-0514:49jfntnIn a nutshell, I can define that :msg/id is either a :foo/msg or a :bar/msg and that :msg/data is either a ::foo/spec or ::bar/spec but I canā€™t enforce that relationship at the msg level#2016-12-0516:26Alex Miller (Clojure team)Qualified names should have meaningful stable semantics. Using qualified names should make this happen less if people are using sufficiently qualified names (which they should)#2016-12-0516:27Alex Miller (Clojure team)Could you not wrap an s/and around specs on both of these to add a constraint?#2016-12-0516:30Alex Miller (Clojure team)Are ::foo/spec and ::bar/spec just s/keys specs? If so, do they really need to be different or is just (s/keys) to validate all attrs sufficient?#2016-12-0517:06lvhIs the lack of docstrings on spec/def intentional?#2016-12-0517:07lvhI could of course just add the metadata to the var#2016-12-0517:13jfntnalexmiller: yes we have different s/keys specs for the data itself#2016-12-0517:43Alex Miller (Clojure team)lvh: it was not part of the initial impl. itā€™s a highly rated request in the jira system and Rich mentioned it as an idea to me long ago. itā€™s not obvious to me how to best implement it (where to put the meta) for all types of specs (when you consider things like (s/def ::a ::b) as kws donā€™t have meta). Do you have a var to add meta to?#2016-12-0517:43Alex Miller (Clojure team)jfntn: it would help me to see a more detailed example#2016-12-0517:44jfntnalexmiller, happy to put something more detailed together and ping you later today#2016-12-0517:48Alex Miller (Clojure team)cool, I will be in and out today#2016-12-0520:17lvhalexmiller: Sometimes I do by accident, yes ā€” but much to your point; thereā€™s no real vars in spec most of the time; so presumably thatā€™s not where tooling would want to go look#2016-12-0521:12thegeez
=> (s/conform
    (s/+ (s/cat :one (s/+ #{1 2 3})
                :alpha (s/? #{:a})))
    [1 2 :a 2])
[{:one [1 2], :alpha :a} {:one [2]}]
=> (s/conform
    (s/+ (s/cat :one (s/+ #{1 2 3})
                :alpha (s/? #{:a})))
    [1 2 :a 2 3])
[{:one [1 2], :alpha :a} [{:one [2 3]}]]
#2016-12-0521:13thegeezIn the second example I am surprised with the extra vector around {:one [2 3]}#2016-12-0521:14thegeezThis doesn't seem consistent with a longer example:
#2016-12-0521:15thegeez
=> (s/conform
    (s/+ (s/cat :one (s/+ #{1 2 3}) :alpha (s/? #{:a})))
    [1 2 :a 2 3 :a 2 3 :a 2])
[{:one [1 2], :alpha :a} [{:one [2 3], :alpha :a} {:one [2 3], :alpha :a} {:one [2]}]]
#2016-12-0521:19Alex Miller (Clojure team)This is a known bug#2016-12-0521:25thegeezok thanks#2016-12-0602:37settingheadnot sure if this have been asked before, suppose i have 2 cases: ["a" 1 true "b" 2 false "c" 3 true] and [["a" 1 true] ["b" 2 false] ["c" 3 true]], my understanding is (s/+ (s/*cat :first string? :second number? :third bool?)) will validate the first case. my question: how to write spec for the second case where each group is enclosed in a vector/list?#2016-12-0603:25hiredman(s/* (spec (s/cat :first string? :second number? :third bool?))#2016-12-0603:27hiredmans/spec allows spec to distinguish between the levels of sequence matching#2016-12-0603:29settinghead@hiredman great! thanks#2016-12-0605:10Alex Miller (Clojure team)another alternative would be (s/coll-of (s/tuple string? number? boolean?) :kind vector?)#2016-12-0605:11Alex Miller (Clojure team)Iā€™d probably prefer the non-regex version in this case#2016-12-0616:09gfredericksQuoting Mr. Miller on clojure-dev:#2016-12-0616:09gfredericks> That said, current versions of those libs still exist for those wanting it and we could release point updates to those if needed later.#2016-12-0616:10gfredericksā‡‘ something rich didn't address in the criticism of semantic versioning -- sometimes a one-dimensional version-space is insufficient#2016-12-0616:11gfredericksmaybe appending things to a timestamp-based versioning system would be good enough#2016-12-0616:11gfredericksversion 201612061011.2#2016-12-0616:25Alex Miller (Clojure team)no thanks :)#2016-12-0616:26gfredericks@alexmiller I'm curious what alternative you have in mind#2016-12-0616:29Alex Miller (Clojure team)I donā€™t understand what problem youā€™re trying to solve#2016-12-0616:30gfredericks@alexmiller making a "point release" off of an older version -- they exact use case you were describing in the email I quoted#2016-12-0616:30gfrederickss/they/the/#2016-12-0616:31Alex Miller (Clojure team)I donā€™t see a problem with making a point release off of an older version?#2016-12-0616:31gfredericksin semantic versioning it's clear what's going on#2016-12-0616:31Alex Miller (Clojure team)is it?#2016-12-0616:31Alex Miller (Clojure team):)#2016-12-0616:32gfredericksif we upgrade from 0.8.0 to 0.9.0, but then want to add bugfixes for users stuck on 0.8.0, we release 0.8.1 and nobody thinks that 0.8.1 is a better version of 0.9.0#2016-12-0616:32gfredericksbut if our versions are just timestamps, there's more confusion#2016-12-0616:32gfredericksrelatively clear -- the non-linearity of the releases is clearer#2016-12-0616:44Alex Miller (Clojure team)I don't think Rich was saying that we should stop using versions#2016-12-0616:53gfredericksme neither#2016-12-0616:54gfrederickshe spent a while criticizing the semantics of semantic versioning, and had a slide that suggested "YYYYMMDDHHMM" or something similar as an alternative#2016-12-0616:54gfredericksand I was thinking about how that sort of versioning system would need some sort of additional feature to support making changes for older releases#2016-12-0616:56gfredericksI'm probably communicating this poorly#2016-12-0616:58naomarikmaybe something like clojure el-capitan 201611221245#2016-12-0616:59naomarikattach a name to a major version#2016-12-0616:59gfredericksthe idea of a "major version" was one of the criticisms#2016-12-0617:01naomariki think it was cause the number is meaninglessā€¦ the name is as well but itā€™s easier to think of a feature set of android nougat vs kitkat than to refer to them as their version numbers#2016-12-0617:02naomarikand i think the lack of a clear alternative was to spawn discussions like this#2016-12-0617:14donaldballI took much of the point to be that you would ship both old and new apis in your library (assuming you didnā€™t choose to fork to a new library name when you changed apis). Thus if you needed to augment the old api with a new feature, youā€™d release a new version of the whole library.#2016-12-0617:28sparkofreasonIs it possible to attach a generator to a predicate without defining an explicit spec? For example, I want to make a predicate queue?, and generate instances of clojure.lang.PersistentQueue from a spec like (s/coll-of int? :kind queue?), similar to how it works for vector?.#2016-12-0617:34gfredericks@donaldball I'm not sure that would work in the case I brought up, where your library requires clojure 1.8 but you want to augment older versions for users that haven't upgraded to 1.8 yet#2016-12-0617:34gfredericks@dave.dixon I doubt it#2016-12-0618:20sparkofreason@gfredericks Certainly doesn't look like it. clojure.spec.gen/gen-builtins provides a nice shortcut for predicates in clojure.core. It would be nice if this was extensible.#2016-12-0618:23gfrederickshmm#2016-12-0618:42Alex Miller (Clojure team)I suspect making that extensible might also invite a world of conflicts#2016-12-0618:53kenny@alexmiller Why was that done for the predicates in clojure.core then? Is the purpose of this purely for backward compatibility? It seems like the general workflow is write your predicate, write a spec that is defined as a call to that function, include a generator on that spec, and use that spec but clojure.core varies from this workflow.#2016-12-0618:54kennyIt almost seems like you donā€™t even need predicate functions at all with spec.#2016-12-0618:55kennyA call to clojure.spec/valid? could entirely replace predicates#2016-12-0618:55kennyI think.. šŸ™‚#2016-12-0618:56Alex Miller (Clojure team)That doesn't make any sense to me :)#2016-12-0618:56Alex Miller (Clojure team)valid? works be evaluating the predicates#2016-12-0618:58Alex Miller (Clojure team)The workflow seems exactly the same to me except the generator is already provided#2016-12-0619:00kennyRight. But why is it written like that in clojure.core instead of providing actual specs for us to use?#2016-12-0619:00Alex Miller (Clojure team)Preds are specs#2016-12-0619:00kennyBut there is no way to attach a generator to a pred without defining a spec#2016-12-0619:01Alex Miller (Clojure team)Yes#2016-12-0619:01hiredmanthe generator is part of the identity of a spec#2016-12-0619:02kennyBut clojure.core is different. Iā€™m wondering why it is different#2016-12-0619:02hiredmannew generator == new spec#2016-12-0619:02Alex Miller (Clojure team)The core fns have built in mappings because that lets you build up specs in many cases without needing to write custom gens#2016-12-0619:02Alex Miller (Clojure team)Like (s/and int? even?)#2016-12-0619:04kennySo then why is there no way for us to extend that built in mapping so we can so similar things with our own predicates?#2016-12-0619:11Alex Miller (Clojure team)I was not part of that decision process so I canā€™t say what exactly went into it. but certainly you can create specs by using s/spec or s/with-gen to add custom gens#2016-12-0619:12Alex Miller (Clojure team)my suspicion is that it opens a bunch of complexities with respect to libs providing their own competing generators#2016-12-0619:12Alex Miller (Clojure team)and maybe that was just a door Rich and Stu didnā€™t want to open#2016-12-0619:14kennyYeah Iā€™m just not sure. Itā€™d be great to know if that was actually what they decided. It seems useful but I am also concerned that it could lead to complexities.#2016-12-0619:14kennyTechnically you could have Spec extend IFn so you can define your predicate on the spec. But that feels a little hacky#2016-12-0620:12danboykishow do I spec a map whose keys are strings i.e. {"foo" "bar"}?#2016-12-0620:12Alex Miller (Clojure team)(s/map-of string? string?)#2016-12-0620:13danboykisalexmiller: I wanted to make sure "foo" is present#2016-12-0620:13danboykisdo I build on top of contains?#2016-12-0620:13Alex Miller (Clojure team)#(contains? % ā€œfooā€) ?#2016-12-0620:14Alex Miller (Clojure team)I donā€™t know that I have enough info to actually answer your question :)#2016-12-0620:15danboykisI thought there was maybe some way to instrument it on top of s/keys#2016-12-0620:16danboykiscontains? works#2016-12-0620:16danboykisthanks#2016-12-0620:16hiredmanthe problem with describing things entirely in terms of predicates is you lose the structure#2016-12-0620:18danboykishiredman, yep, I was thinking if it becomes to bothersome to do contains? everywhere to convert the string keys to keywords instead#2016-12-0620:19danboykisi'm not sure how to best handle a case with a large map that has a bunch of strings for keys#2016-12-0620:20Alex Miller (Clojure team)one hack is to (s/and (s/conformer clojure.walk/keywordize-keys) (s/keys :req-un [::a]))#2016-12-0620:21Alex Miller (Clojure team)(didnā€™t try this, so could be typos there but you get the idea)#2016-12-0620:21hiredmanI have https://gist.github.com/hiredman/0668d1240e0a623904eca33fed11c45f , but I haven't hooked up generation yet#2016-12-0620:22danboykisalexmiller thanks, I'll play with it#2016-12-0620:23Alex Miller (Clojure team)@hiredman fyi, we consider the protocols internal right now and there is a reasonable chance they will still change#2016-12-0620:24hiredmansure#2016-12-0620:24hiredmanstill alpha, etc, etc#2016-12-0620:24Alex Miller (Clojure team)protocol might even go away#2016-12-0620:24hiredmanI was disappointed without much work that took to do šŸ™‚#2016-12-0620:25Alex Miller (Clojure team)the intention is that most people shouldnā€™t do it :)#2016-12-0620:43hiredmanI don't want to do it, but I also want to use spec as a data regex (including maps) without having to fiddle with the global registry#2016-12-0620:51donaldballDo you mean you want the ability to spec a map with unnamespaced keywords without registering corresponding namespaced keywords in the spec registry?#2016-12-0620:53hiredmanmaps without keyword keys exist too#2016-12-0620:57hiredmanyou can easily say, with predicates, #(and (map? %) (contains? % "foo") (uuid? (get %) "foo")), but then you run in to issues if you want to spec the data#2016-12-0621:00hiredmanyou can replace uuid? there with a call to valid? and some spec, but to conform and explain on the original spec it is a black box#2016-12-0621:01hiredmanone solution would be to replace predicates with pairs of predicates and navigators#2016-12-0622:12eraserhdHow could I compute a key for s/keys?#2016-12-0622:14eraserhd(`is-valid` is effectively (is (s/valid? ...))#2016-12-0622:16eraserhd
dev=> (s/valid? (s/keys :req [::foo]) {})
false
dev=> (let [foo ::foo] (s/valid? (s/keys :req [foo]) {}))
true
dev=>
#2016-12-0622:20bfabry@eraserhd s/keys is a macro, so you need to compute the key at/prior to macroexpansion#2016-12-0622:20bfabry
boot.user=> (defmacro computed-foo []
       #_=>   (let [foo ::foo]
       #_=>     `(s/keys :req [~foo])))
#'boot.user/computed-foo
boot.user=> (s/valid? (computed-foo) {})
false
#2016-12-0622:28eraserhdAh, eval works for this case (since it's a test and not production code)#2016-12-0623:14bbloomam i doing :ret wrong? user=> (s/fdef f :args (s/cat) :ret string?) user/f user=> (defn f [] 1) #'user/f user=> (stest/instrument `f) [user/f] user=> (f 5) ExceptionInfo Call to #'user/f did not conform to spec: In: [0] val: (5) fails at: [:args] predicate: (cat), Extra input :clojure.spec/args (5) :clojure.spec/failure :instrument :clojure.spec.test/caller {:file "form-init7158439094984280972.clj", :line 1, :var-scope user/eval100801} clojure.core/ex-info (core.clj:4725) user=> (f) 1#2016-12-0623:17gfredericks@bbloom instrument doesn't check :ret#2016-12-0623:17bbloom.... what? why?#2016-12-0623:17gfrederickssomething something instrument is for calls and generative testing is for rets#2016-12-0623:18gfredericksit does or does not make perfect sense depending on who you ask; I think there's an explanation somewhere#2016-12-0623:18gfredericksif this were irc we would just ask clojurebot for the link#2016-12-0623:18bbloomcount me in the camp that for whom that does not make any sense#2016-12-0623:19gfredericksif this were irc we would just increment a counter in clojurebot to keep track of that#2016-12-0623:20bbloomĀÆ\(惄)/ĀÆ#2016-12-0623:21bbloomfound https://groups.google.com/forum/#!searchin/clojure/$3Aret%7Csort:relevance/clojure/JU6EmjtbRiQ/WSrueFvsBQAJ#2016-12-0623:22gfredericks@bbloom my guess is the intention is when you test something, you instrument all the stuff it calls to make sure that stuff gets called correctly; checking rets is for when you're testing that function in particular, which you can do with clojure.spec.test/check, or by calling/asserting manually#2016-12-0623:22gfredericksanyhow if you hate this you could add an alternative to instrument in schpec#2016-12-0623:24bbloomugh. i KINDA understand the thought process, but i donā€™t really buy it#2016-12-0623:24bbloomi have a recursive function that has a small :ret violation deep inside it and i want to catch the error earlier than the root call from the unit test#2016-12-0623:26bronsas/assert in a postcondition ĀÆ\(惄)/ĀÆ#2016-12-0623:27bronsa(not saying I particularly like it)#2016-12-0623:33bbloomyeaaaah, iā€™ll just use s/assert#2016-12-0623:33bbloomiā€™m just annoyed that these specs werenā€™t getting run this whole time i thought they were#2016-12-0623:43bbloommy 2cents: https://groups.google.com/d/msg/clojure/jcVnjk1MOWY/LCP4ml5YDQAJ#2016-12-0700:35bfabry@bbloom I fully expect someone will write a dev lib that provides an overinstrument or whatever shortly after 1.9 is released#2016-12-0700:36bbloomitā€™s not a big deal - it was just 1) surprising and 2) not convincingly justified, despite being unsurprisingly/welcomely accompanied by a rationale#2016-12-0700:36bbloomgave me feedback, for whatever thatā€™s worth, and moved back on to writing code šŸ™‚#2016-12-0701:02tetriscodesCan someone tell me if Iā€™m off here?#2016-12-0701:02tetriscodes
(s/def :gj/coordinates (s/with-gen
                         coll?
                         #(s/gen #{(point-gen)})))
(s/def :gjpt/type (s/with-gen string? #(s/gen #{"Point"})))
(s/def :gjls/coordinates (s/coll-of :gj/coordinates))
#2016-12-0701:03tetriscodesProduces
[-7.4052159604840995, 12.824879014110294, -195933.59674337224],
			[-7.4052159604840995, 12.824879014110294, -195933.59674337224],
			[-7.4052159604840995, 12.824879014110294, -195933.59674337224]
#2016-12-0701:03tetriscodesI would think that coll-of would call a spec multiple times that has a generator and would produce unique values each time#2016-12-0701:05Alex Miller (Clojure team)what is the line that produced the output?#2016-12-0701:05Alex Miller (Clojure team)what does point-gen do?#2016-12-0701:06tetriscodes
(defn point-gen
  "Generates a valid SRID:4326 point"
  []
  [(- (rand 360) 180)
   (- (rand 180) 90)
   (- (rand 200000) 200000)])
#2016-12-0701:06tetriscodes
(s/def :gj/linestring (s/keys :req-un [:gjls/type :gjls/coordinates]))
#2016-12-0701:07Alex Miller (Clojure team)you should never use a random-generating thing in your generator#2016-12-0701:07Alex Miller (Clojure team)when you do that, you rob test.check from being able to a) make reproducible results and b) shrink on error#2016-12-0701:08Alex Miller (Clojure team)any source of randomness should come from a generator#2016-12-0701:08Alex Miller (Clojure team)so something like#2016-12-0701:08tetriscodesOk, Iā€™ll swap that with double-in#2016-12-0701:09seancorfieldalso with-gen is only going to call the function once to get the generator, right? so this gets a generator for #{(point-gen)} so it calls point-gen just once and gets #{[some, random, values]} and then just generates coords from that single-valued set.#2016-12-0701:09seancorfieldor does the function argument get called every time it needs a generator?#2016-12-0701:10Alex Miller (Clojure team)yeah, you really want to use gen/fmap or gen/bind to do this kind of thing#2016-12-0701:11Alex Miller (Clojure team)your point-gen could be (s/gen (s/tuple (s/int-in -180 180) (s/int-in -90 90) (s/int-in -20000 0)))#2016-12-0701:11Alex Miller (Clojure team)something like that#2016-12-0701:13tetriscodesOk, thank you. Iā€™ll do some refactoring#2016-12-0701:13tetriscodesIā€™m trying to generate GeoJSON with spec#2016-12-0701:13Alex Miller (Clojure team)sounds fun :)#2016-12-0701:13tetriscodesIā€™ll post to it when Iā€™m done#2016-12-0701:13tetriscodesThanks again#2016-12-0710:02nha@tetriscodes interesting šŸ™‚#2016-12-0711:41mpenet@tetriscodes I have done something similar as part of one of our internal projects (just for the subset used by twitter)#2016-12-0711:42mpenetI you need (rev)geocoding data check mpenet/sextant on github#2016-12-0711:42mpenetIt was useful for gen#2016-12-0714:14vikeriI guess Iā€™m missing something about how cat should be nested, or why does that fail?#2016-12-0714:16Alex Miller (Clojure team)If you need to nest regexs, wrap the inner in s/spec to start a new nested sequential context#2016-12-0714:51jfntnDonā€™t understand whatā€™s going on here#2016-12-0714:56vikeri@alexmiller Ok, if I donā€™t need regexes, is there some other way I could spec a vector of one or possibly two different elements? (s/tuple boolean? (s/? map?)) did not work.#2016-12-0715:05zane@vikeri: (s/or :one-element (s/tuple boolean?) :two-elements (s/tuple boolean? map?))#2016-12-0715:44Alex Miller (Clojure team)@jfntn the ::foo inside the map is not valid according to the spec you have defined for ::foo#2016-12-0715:44Alex Miller (Clojure team)(also, identity is not a valid retag function - youā€™d hit that with gen if you tried it)#2016-12-0715:54Alex Miller (Clojure team)same problem#2016-12-0715:55Alex Miller (Clojure team)the ::foo in the map is not a valid value for the ::foo multispec#2016-12-0715:56jfntnok so the problem is that the kw that registers the mspec is also present in the map, correct?#2016-12-0716:05tomcI've got the same spec returning false for s/valid? and "Success" for s/explain-str, for the same data, which I didn't expect to be possible. Is there an obvious reason this is occurring? Can someone point me to something in the docs?#2016-12-0716:07Alex Miller (Clojure team)@tomc shouldnā€™t ever happen. canā€™t say more without more info#2016-12-0716:07Alex Miller (Clojure team)@jfntn yes - s/keys will validate specs for all keys in the map#2016-12-0716:08tomc@alexmiller ok, let me verify it's not something on my end (which it almost certainly is) and if I can't figure it out I'll give you more info here. Thanks!#2016-12-0716:09jfntnMakes sense, thank you #2016-12-0716:12tomcTurns out I had a call to s/keys that didn't conform to s/key's api. If only there were some way to prevent that type of thing šŸ™‚#2016-12-0716:15Alex Miller (Clojure team)ha, yes#2016-12-0716:16Alex Miller (Clojure team)I have created specs for spec, but itā€™s somewhat problematic to actually use it without encountering an infinite loop of checking#2016-12-0716:16Alex Miller (Clojure team)that said, Iā€™d be happy to see a jira for whatever problem you had as thatā€™s something we could prevent in the code#2016-12-0716:22tomcProbably not worth it. I'm doing some whacky metaprogramming for reasons not worth getting into, and it resulted in me evaluating code like (s/keys :req-un [nil])#2016-12-0716:27Alex Miller (Clojure team)fair enough#2016-12-0718:42denikhow do I spec a hash-map where the keys are integers?#2016-12-0718:44denikah map-of!#2016-12-0719:22Alex Miller (Clojure team)Yep#2016-12-0722:25bhaumanThinking about a spec api so that tooling can register listeners for fn spec failures.#2016-12-0722:26bhaumanto clarify what I'm talking about it would be nice if this spot fired an event https://github.com/clojure/clojurescript/blob/master/src/main/cljs/cljs/spec/test.cljs#L103#2016-12-0722:26bhaumanof some sort#2016-12-0722:29bhaumanI would need something like this to reliably report spec errors in tooling like figwheel#2016-12-0722:31bhaumanin JS especially because its extremely difficult and fairly invasive to get any sort of reliable top level Exception handling#2016-12-0722:33bhaumancc/ @dnolen#2016-12-0722:52bhaumancc/ @cfleming#2016-12-0722:58lvhIs there an efficient way to ask for a string of specific length range? The obvious (s/def ::layout-row (s/and string? #(<= 8 (count %) 11))) breaks such-that generation#2016-12-0723:02dnolen@bhauman another thing to consider is thereā€™s really no harm in patching that for custom tooling#2016-12-0723:02dnolenvars exist for a reason šŸ™‚#2016-12-0723:05Alex Miller (Clojure team)@lvh you will need to dive into gen for that#2016-12-0723:05lvhThatā€™s what I figured, thanks šŸ™‚#2016-12-0723:06lvhIs there a way to figure out which spec failed to generate when I see:
java.util.concurrent.ExecutionException: clojure.lang.ExceptionInfo: Couldn't satisfy such-that predicate after 100 tries. {}
#2016-12-0723:06Alex Miller (Clojure team)no#2016-12-0723:06lvh(Iā€™m running check over the entire ns)#2016-12-0723:06lvhIs that gonna change, or is there a deep reason for that#2016-12-0723:06Alex Miller (Clojure team)the problem here is that test.check doesnā€™t have a way to do so (or doesnā€™t in the version used by spec at least)#2016-12-0723:07Alex Miller (Clojure team)basically, spec builds a big composite generator and says gimme and gets back either a sample or a failure#2016-12-0723:07Alex Miller (Clojure team)I know Gary has been making some changes to make this better in test.check but I havenā€™t had time to look at them yet#2016-12-0723:07Alex Miller (Clojure team)and itā€™s possible that we can do better if we leveraged that#2016-12-0723:08Alex Miller (Clojure team)there is certainly no desire for that to be information free :)#2016-12-0723:10lvhAh, I see šŸ™‚#2016-12-0723:10lvhYeah, so Iā€™ve made the same mistake several times now#2016-12-0723:10lvhin that I forget how to write a clojure.,test test that is actually doing stest/check#2016-12-0723:11lvhAnd just do (t/is ā€¦) and forget itā€™s lazy, and then only way later discover that the checks were just not running and Iā€™ve had a bug all along#2016-12-0723:11Alex Miller (Clojure team)yeah, Iā€™ve done that too :)#2016-12-0723:12Alex Miller (Clojure team)it is intentional that itā€™s lazy#2016-12-0723:16lvhYeah, it makes sense. Itā€™d be fine if there was an stest/deftest IMO#2016-12-0723:22Alex Miller (Clojure team)unlikely we will add that#2016-12-0723:22Alex Miller (Clojure team)in general, the lifecycle and use of generative or check-based tests is much different than example-based clojure.test tests#2016-12-0723:23Alex Miller (Clojure team)Iā€™m dubious that combining them in a single suite makes sense#2016-12-0723:51lvhHm; Iā€™ve been doing that by having lein test filter for me based on tags#2016-12-0809:28artemyarulinHello, just started to play a bit with spec. Is there any way to make it shorter? Can I somehow embed ::first-name inside ::person without separate def?
(s/def ::first-name string?)
(s/def ::last-name string?)
(s/def ::person (s/keys :req [::first-name ::last-name]))
#2016-12-0810:05patrkris@artemyarulin I don't think there is#2016-12-0810:08artemyarulinhm, ok. It kinda forces me to design everything bottom to the top#2016-12-0810:12dominicmI think that's on purpose#2016-12-0810:17artemyarulinOK, thanks#2016-12-0814:09thegeez
(s/conform (s/cat :pre (s/& (s/+ #{1 2 3})
                              (fn [r]
                                (if (= r [1 2 3])
                                  r
                                  ::s/invalid)))
                    :keyword #{:a})
             [1 2 3 :a])

=> {:pre [1 2 3], :keyword :a}
#2016-12-0814:09thegeezFor the :pre match I want to match the sequence 1 2 3 in that order as the start of a larger sequence. Is there an easier way to write the spec for the :pre part? I tried stuff like #{[1 2 3]}#2016-12-0814:15Alex Miller (Clojure team)(s/cat :1 #{1} :2 #{2} :3 #{3}) #2016-12-0814:15Alex Miller (Clojure team)Or similar with s/tuple#2016-12-0814:24thegeezBut s/tuple can't be used within a regex, right?#2016-12-0814:28Alex Miller (Clojure team)Right#2016-12-0814:58thegeezI went with:#2016-12-0814:59thegeez
(defmacro match-seq [c]
  (let [ks (vec (mapv keyword (repeatedly (count c) gensym)))
        cat-kvs (mapcat vector ks (map hash-set c))]
    `(s/with-gen
       (s/&
        (s/cat 
#2016-12-0818:23bbloomjust remembered an old happening from the .net world that I thought the folks here may be interested in: https://blogs.msdn.microsoft.com/kathykam/2007/03/29/bye-bye-system-timezone2-hello-system-timezoneinfo/#2016-12-0818:23bbloomin short, System.TimeZone was broken in a bunch of ways, so they made System.TimeZone2, but people freaked out about that name, so they caved and called it TimeZoneInfo.... leading to substantial confusion#2016-12-0818:44gfredericks> using numeric modifier is not scalable in the long term#2016-12-0818:45gfrederickseventually the numbers just get too big#2016-12-0818:45gfredericksmathematicians tell us that there is no limit to how big numbers can get#2016-12-0818:47eraserhdI wonder what the smallest number which isn't representable with the amount of information present in the universe is.#2016-12-0818:48bbloomknowing that number would change the amount of information in the world, hence changing that number#2016-12-0818:48gfrederickshttp://www.scottaaronson.com/writings/bignumbers.html#2016-12-0818:48eraserhdWell, I obviously couldn't know it.#2016-12-0818:49bfabryI remember watching a ted talk on this, it's interesting. but yeah, what's to stop me saying "the biggest number the universe can represent, times two"#2016-12-0818:49eraserhdRight, that's a representation.#2016-12-0818:49bfabrywe could just redefine 1 to be "the biggest number the universe can represent" šŸ˜›#2016-12-0818:49bbloombfabry: the trick is not to abstract over numbers, but to abstract over the operations which generate those numbers#2016-12-0818:51gfredericksuse BB(11111), as scott aaronson suggests#2016-12-0818:51eraserhd@bfabry If I could find the number by an algorithm, the algorithm would have to require more information to represent than present in the universe to represent.#2016-12-0818:55gfredericksBB(n) exposes an ambiguity in the question, I suppose -- if a number is platonically well-defined but can't be computed, does that count?#2016-12-0818:56eraserhdInteresting#2016-12-0818:56eraserhdI just got to that part#2016-12-0818:56gfredericksthe fun part is I don't even know what I mean by "platonically well-defined"; it's just an emotion#2016-12-0818:57bbloomas fun as i find numbers, i was hoping somebody had some insights in to the hardest problem in computer science: the paucity of names šŸ™‚#2016-12-0819:01gfredericks@bbloom what do you think of the renaming you linked to?#2016-12-0819:01bbloomhonestly, i liked the name TimeZone2 - it means i donā€™t need to look at the docs for TimeZone or TimeZoneInfo to know which does what#2016-12-0819:01bbloombasically i just ignore TimeZone#2016-12-0819:02bbloomi can understand the desire to not have to write TimeZone2 in places#2016-12-0819:02gfredericksbecause it's ugly?#2016-12-0819:02gfredericksyou feel like you have to memorize a version number for every class/API you learn to use?#2016-12-0819:02bbloomyeah, but specifically itā€™s ugly for an ugly reason: it showcases a mistake#2016-12-0819:02bbloomitā€™s not my mistake, but thereā€™s still a human error as an accident of history in my code#2016-12-0819:03bbloomnow, thatā€™s fine, b/c everything is human errors and accidents of history#2016-12-0819:03bbloommy next thought is: ok, so letā€™s use aliases, which is what rich suggested too#2016-12-0819:03bbloombut iā€™m not so sure thatā€™s a good solution either#2016-12-0819:03gfredericksns aliases in particular? :rename too I guess#2016-12-0819:03bbloomyeah, those combined#2016-12-0819:03bbloomor even just a simple def#2016-12-0819:04bbloomthat helps for migration, which is key, but it still kinda sucks b/c of the nature of context#2016-12-0819:04bbloomif you and i have a conversation about TimeZone, how do you know which version i mean?#2016-12-0819:04gfredericksyeah there'll always be a version number at one of the levels#2016-12-0819:04bbloomby utilizing rename/alias, you wind up in the same place you started with if you had used semver or whatever (shudder)#2016-12-0819:05gfredericksfunction name, ns name, artifact name, ecosystem name...#2016-12-0819:05bbloomyeah, well ecosystem name is the easy one#2016-12-0819:05bbloomyou just invent a brand name and youā€™re good#2016-12-0819:05gfredericksevery 5-10 years we declare bankruptcy on all this and start a new programming language#2016-12-0819:05bbloomlike instead of Clojure 2.0 you just fucking call it Twojure or something and you can change whateve ryou want, but at the cost of bifurcating the community#2016-12-0819:05bbloomyup lol#2016-12-0819:06gfredericksand then everybody rushes in to write the first http library#2016-12-0819:06bbloomhereā€™s where iā€™d use a /giphy lion king circle of life#2016-12-0819:07gfredericksI don't think there's any Super Happy approach to this :(#2016-12-0819:07bbloomiā€™d settle for a reasonable principal by which i can rationalize my decisions and remain mostly happy that i didnā€™t LOSE a battle with the laws of nature#2016-12-0819:08bbloomalias & rename give us some powerful tools#2016-12-0819:08bbloombut i feel like something is missing#2016-12-0819:08bbloomone problem iā€™ve run in to a bunch is the impl vs interface namespace problem#2016-12-0819:08bbloomspec sorta has this problem in that it has some public macros that itā€™s like ā€œdonā€™t use me directly please"#2016-12-0819:09bbloomand then thereā€™s core.async that has the impl namespaces and then redefs an indirection to export all the functionss#2016-12-0819:09bbloomi wonder if there was some way to abstract over aliases and renames that you could more easily address this#2016-12-0819:09bbloomnot sure what that would look like#2016-12-0819:10bbloombut if weā€™re going to use names as a tool to achieve accretion (and i see no other way, since everything in symbolic reasoning involves names), then i think we need more tools for names#2016-12-0819:10bbloomthatā€™s as far as iā€™ve managed to think here </rant>#2016-12-0819:11gfredericksrich wanted a global mapping from namespaces to maven artifacts#2016-12-0819:11gfredericksor seemed to at least#2016-12-0819:11gfredericksthis is sort of a different topic#2016-12-0819:12bbloomhe seemed to be making two points: 1) weā€™ve got bad abstractions: project files, version numbers, etc and 2) accrete, donā€™t break, use names to do that#2016-12-0819:14bbloomif it were up to me, weā€™d abolish artifact ids etc as much as possible - namespaces have much nicer properties#2016-12-0819:14gfredericksjust publish individual namespaces? with versions?#2016-12-0819:14bbloomyou could probably define some information-preserving algebra of them. ie merge two sets of names, aliases, etc#2016-12-0819:15bbloomyeah, i think so#2016-12-0819:16bbloommaybe published dates instead of versions, since weā€™re turning version trees in to sequences#2016-12-0819:17gfrederickshe said the useful thing about artifacts is that they intentionally tie together particular versions of particular namespaces#2016-12-0819:17bbloomi mean do they tho? only if you bundle your deps#2016-12-0819:20nicolaHi all. I need spec for map such as: if key - some pred, then value - spec1, else if key - other pred, then value - spec2?#2016-12-0819:21gfredericks@bbloom I mean if your artifact contains multiple namespaces, which it probably does, they are free to assume things about each other because you know that you'll get exactly the same version of all of them#2016-12-0819:25bbloom@gfredericks ah, i understand what youā€™re saying. yeah, in that sense those are actually tied together and tested together. i think thatā€™s not incompatible with the idea of ā€œeliminatingā€ artifacts or projects thoā€¦ you'd just use the context of a project in resolving versions of namespaces, ideally in some content addressable way maybe?#2016-12-0819:26bbloomso if i have namespace foo and i require foo.bar, then i have a project that says ā€œhey, iā€™m going to publish namespaces #ā€foo.*ā€, then when those get published they get tagged with some artifact bundle in some way, but i donā€™t actually think about that, i think about the version of the nodes in that dependency graph i point to directly#2016-12-0819:26bbloomso you resolve internal dependencies within a project using immutable references, like hashes, and then you resolve external dependencies using lamport logic#2016-12-0819:27bbloomeg by intersecting date ranges#2016-12-0819:28gfredericksIf backwards compatibility is reliable presumably your date range can be open on the right side and intersections are trivial#2016-12-0819:29bbloomright, but of course people make mistakes, so youā€™d probably need two features:#2016-12-0819:29bbloom1) right bound to say ā€œi no longer trust this maintainer"#2016-12-0819:29bbloomand 2) a black list of bad versions#2016-12-0819:29bbloomso your timelines could have right bounds or holes#2016-12-0819:30gfredericksIf intersections are empty then the build tool reports a failure? #2016-12-0819:30bbloomyup#2016-12-0819:30bbloomand you always select the most recent version from the intersection#2016-12-0819:31bbloomthen layer on top the standard dependency pinning behavior for prod
#2016-12-0819:32bbloomi want this. somebody make this.#2016-12-0819:32bbloomšŸ™‚#2016-12-0819:32gfredericksIt can be a first-class feature of twojure#2016-12-0819:32bbloomalex probably wouldnā€™t tell us if rich was working on a new tool here šŸ˜‰#2016-12-0819:32bbloomTwojure has a nice ring to it#2016-12-0819:34dspiteselfIt seemed their primary goal was running the minimal set of tests and generative tests for a given change.#2016-12-0819:36dspiteselfhere is to hoping they will continue the https://github.com/Datomic/codeq thought#2016-12-0821:00nicolaI was able to spec my map with
(s/coll-of (s/or :option1 (s/tuple key-spec-1 val-spec-1) ....))
, but path in error messages is not informative: [0 1 0 1]
#2016-12-0821:50nwjsmithšŸ¤” Iā€™m having trouble supplying generator overrides
(ns scratch
  (:require [clojure.spec :as s]
            [clojure.spec.gen :as gen]))

(s/def ::id (s/or :tempid int? :uuid uuid?))
(s/def :thing/id ::id)
(s/def :thing/entity (s/keys :req [:thing/id]))

(gen/sample (s/gen :thing/entity
                   {:thing/id #(gen/int)}))
#2016-12-0821:51nwjsmithI would expect that call to gen/sample to give me :thing/entitys with int values for the :thing/id key.#2016-12-0821:52nwjsmithInstead, itā€™s as if I hadnā€™t supplied the override at all. I get a mix of int and uuids.#2016-12-0821:56hiredman::id makes it work#2016-12-0821:57hiredman(dunno what the expected behavior is, if that is a bug, etc)#2016-12-0821:57hiredman::id in the override map#2016-12-0821:57nwjsmithYeah, it does. This is a minimized example, but Iā€™d like to keep from overriding all ::ids, and just override thing/entityā€˜s#2016-12-0822:07nwjsmith@hiredman thanks, I think your suggestion help me identify my problem. Iā€™m pretty sure itā€™s a bug#2016-12-0822:07nwjsmith
(ns scratch
  (:require [clojure.spec :as s]
            [clojure.spec.gen :as gen]))

(s/def ::number number?)
(s/def ::numeric ::number)
(gen/sample (s/gen ::numeric
                   {::numeric gen/int}))
#2016-12-0822:08nwjsmithI would expect that gen/sample call to only return ints, but it returns ints and floats instead#2016-12-0822:11nwjsmithI think that this means that you canā€™t override ā€œaliasedā€ specs, which is a bug?#2016-12-0822:11gfredericksI hope it's a bug#2016-12-0822:22bfabryprobably raise a jira, that behaviour is really surprising
boot.user=> (s/def ::numeric ::number)
:boot.user/numeric
boot.user=> (s/def ::numeric-coll (s/coll-of ::number))
:boot.user/numeric-coll
boot.user=> (gen/sample (s/gen ::numeric {::number gen/int}))
(-1 1.0 -1 NaN -1 2.5625 -1.5 -47 -13 24)
boot.user=> (gen/sample (s/gen ::numeric-coll {::number gen/int}))
([0 0 0 0 0 0 0 0 0] [-1 1 0 -1 -1 1 0] [-1 -2 1 -1 2 -2] [-2 -1 -2 -2 3 -2 1 -3] [-1 1 4 3 -4 4 -1 2 -4 -2 -4 4 -2 -4 0 -1] [1 -4 0 -1 0 -2 -1 -4 -2 -3 5 0 3 -1 -1 -3 -3 -1 -4] [3 -2 4 -3 -6 4 -3 3 2] [-7 -7 -1 -1 3 6 0 -7 0 -3 1] [] [9])
#2016-12-0822:24nwjsmithFiling now, thanks!#2016-12-0822:39nwjsmithhttp://dev.clojure.org/jira/browse/CLJ-2079#2016-12-0912:12yendaHi, could someone help me understand why in the above code generating ::engine-variable takes a split of a second and test/check runs forever until hitting a GC overhead ?#2016-12-0912:16yendaI even tried
(binding [s/*recursion-limit* 1]
    (stest/check `convert {:num-tests 1 :max-size 1}))
#2016-12-0912:16morrifeldmanI'd like a constraint that only allows one branch of an or in a derived spec. Is this possible? Here is an example of what I'm trying to do.
(s/def ::an-int-or-string (s/or :int int? :string string?))
 
  (s/def ::my-map (s/keys :req [::an-int-or-string]))

  (s/def ::my-map-string (s/and ::my-map
                               #(string? (::an-int-or-string %))))

#2016-12-0914:06mpenetam I awful thinking about writing an (with-ns ...) macro that does create-ns + in-ns + restore proper ns once out of scope ?#2016-12-0914:07mpenetgetting tired of writing super verbose specs preceeded by (alias baz (create-ns 'foo.bar.baz))#2016-12-0914:08mpenet
(with-ns 'foo.bar.baz
  (s/def ::a string?) ;; ::foo.bar.baz/a
  )
#2016-12-0914:09mpenetit could be a 2 arg version of in-ns otherwise#2016-12-0914:12bronsa@mpenet that won't work#2016-12-0914:12bronsathe reader will still resolve ::a in the current ns#2016-12-0914:13mpenetšŸ˜ž#2016-12-0914:13bronsayou can't change ns context & use it in the same expr#2016-12-0914:13bronsaneed to be 2 separate exprs#2016-12-0914:13mpenetI see#2016-12-0914:13mpenetthat's too bad#2016-12-0914:13bronsa(not even do would work)#2016-12-0914:14mpenetIt'd require reader hackery then.#2016-12-0914:15mpenetthe syntax is quite appealing, it would really save a ton of boilerplate#2016-12-0914:15bronsayou could get e.g. (do (in-ns 'foo) ::foo) to work with reader hackery but no way to get (with-ns 'foo ::foo) to work w/o introducing placeholder objects & resolve aliases at compile time#2016-12-0914:16mpenetI see#2016-12-0914:16bronsarealistically, neither is doable#2016-12-0914:17mpenetWell I might give it a try once I am done with the giant schema -> spec port I am into atm. Sounds fun nonetheless#2016-12-0916:01lvh@alexmiller Are you a good person to report suggested stdlib macro specs to? If so, I submit http://dev.clojure.org/jira/browse/CLJ-207 for consideration ā€” I just got bitten by that; was expecting :let in the first position to work (kinda feel it should anyway, but OK ā€” if it doesnā€™t the error message could be a lot more descriptive, and that seems like the kind of thing spec can help with here, plus youā€™re probably already speccing what for looks like)#2016-12-0916:01Alex Miller (Clojure team)jira is a good person to report suggestions to :)#2016-12-0916:02lvhDoes that go on the ticket?#2016-12-0916:02lvhOr do I re-tag it spec?#2016-12-0916:02lvhor is that a new ticket#2016-12-0916:02Alex Miller (Clojure team)Iā€™d file a new enhancement ticket#2016-12-0916:02Alex Miller (Clojure team)and point to this one#2016-12-0916:02lvhcool, thanks#2016-12-0916:02Alex Miller (Clojure team)requesting a spec for for#2016-12-0916:08yendaI am trying to validate data after edn/parse-string, the field above doesn't validate with the predicate #{true false} even though (s/valid? ::boolean (java.lang.Boolean. "true")) does :edit actually it does not for "false", only for "true"#2016-12-0916:12minimaluse boolean?#2016-12-0916:13yendapoor me I'm still on clojure 1.8 šŸ˜ž#2016-12-0916:15yendabut yeah I can just add the boolean? predicate source to my spec#2016-12-0916:15minimalsee https://github.com/tonsky/clojure-future-spec/blob/master/src/clojure/future.clj#2016-12-0916:33yendanice thanks ! didn't think it was also handling this#2016-12-0916:50yendathat is so wierd I add a test/check working before and now it throws an exception cider.nrepl.middleware.test$report cannot be cast to clojure.lang.MultiFn. I tried :monkeypatch-clojure-test false in lein profile but it doesn't change anything#2016-12-0917:14yendaI am trying to figure out what is causing this, it happens both in clojure 1.8 and 1.9#2016-12-0917:17yendaif I try to run the test right after jacking-in I get the error, if I recompile only the test one or more time and run it it works, if I recompile the whole namespace I get the error even if I make it work once before#2016-12-0917:30yendaOk I found the problem I had to explicitely require [clojure.test.check] in my test namespace#2016-12-0918:36Alex Miller (Clojure team)@yenda note that in Clojure we use only the canonical Boolean/TRUE and Boolean/FALSE for true/false values so you should never call the Boolean constructor. This is in particular a problem with false as the non-canonical false value will be treated as true (although youā€™ll see subtle weirdnesses with the non-canonical true value too).
(false? (Boolean. ā€œfalseā€)) ;; false
 (false? (Boolean/FALSE)) ;; true
#2016-12-0918:38yendaThanks for the tips#2016-12-0918:45Alex Miller (Clojure team)itā€™s pretty subtle and can definitely be a gotcha in certain Java serialization / interop cases#2016-12-0921:45dm3is unform supposed to handle nested values?#2016-12-0921:47dm3
boot.user=> (s/def :a/test-map (s/map-of :a/key :a/val))
:a/test-map
boot.user=> (s/def :a/key keyword?)
:a/key
boot.user=> (s/def :a/val (s/or :str string? :int int?))
:a/val
boot.user=> (s/unform :a/test-map (s/conform :a/test-map {:x "a", :y 1}))
{:x [:str "a"], :y [:int 1]}
#2016-12-0922:13Alex Miller (Clojure team)that should work, but it is buggy#2016-12-0922:13Alex Miller (Clojure team)I actually wrote a patch for it this morning#2016-12-0922:14Alex Miller (Clojure team)http://dev.clojure.org/jira/browse/CLJ-2076#2016-12-0922:24dm3ok, thx#2016-12-0922:34lvhIs there a way to introspect the generated samples for fspecs? Iā€™m trying to see what sort of stuff it comes up with now to see if itā€™s worth coming up with a more specific (and ideally: faster to generate) spec#2016-12-0922:35lvhspecifically right now I have a HOF that takes forever to test (for good reason)#2016-12-0922:35lvh
(s/def ::variant
  (s/fspec
   :args (s/cat :input string?)
   :ret (s/coll-of string?)))

(s/fdef
 comp*
 :args (s/cat :vs (s/+ ::variant))
 :ret ::variant)
#2016-12-0922:36lvhSo Iā€™m guessing coming up with lots of ::variants, and then checking if a bunch of combinations of them is still a variantā€¦ well, thatā€™s going to be expensive šŸ™‚ Iā€™m not sure if I should just run the generative tests extremely sparingly, or work on producing faster ones (e.g. by helping the generator generate realistic variants faster ā€” I have a bunch laying around anyway...)#2016-12-0922:41Alex Miller (Clojure team)you might try changing the :ret spec to (s/coll-of string? :gen-max 3) to see if that helps#2016-12-0922:44lvh (s/and string? #(<= 8 (count %) 11))#2016-12-0922:44lvhyeah thatā€™s not gonna be fast.#2016-12-0922:44lvhthere arenā€™t any fancy string specs yet, right? for printable or something?#2016-12-0922:46Alex Miller (Clojure team)no, but lots of ways to create custom gens#2016-12-0922:46lvhyeah. (fn [] (gen'/string-from-regex #ā€[a-z0-9]{8,11}")))) right now#2016-12-0922:46lvhbecause I got lazy, mostly šŸ˜‰#2016-12-0922:46lvh(`genā€™` is test.chuck)#2016-12-0922:47Alex Miller (Clojure team)I guessed :)#2016-12-0922:47lvhI figured; mostly for the benefit of other people šŸ™‚#2016-12-0922:57nwjsmithIā€™m having some modelling trouble with spec. Letā€™s say Iā€™ve spec-ed a map :child/entity:
(s/def :child/entity (s/keys :req [:child/name]))
I want to have functions that only operate on stored versions of this entity, so I create a stricter version of the spec:
(s/def :child/stored-entity (s/merge :child/entity (s/keys :req [:child/id])))
This is working pretty good! I can write specs for stored and unstored entities (or both!). The trouble starts when I add a parent entity:
(s/def :parent/entity (s/keys :req [:parent/children]))
(s/def :parent/stored-entity (s/merge :parent/entity (s/keys :req [:parent/id])))
The :parent/stored-entity spec is incomplete. I canā€™t specify that its children are required to be :child/stored-entitys.
#2016-12-0923:01nwjsmithIf I use a different key for a :parent/stored-entities children, it means that functions written to work with :parent/entitys wonā€™t work with the :parent/stored-entity, which is a Bad Thingā„¢.#2016-12-0923:03Alex Miller (Clojure team)well in this case, youā€™d probably need stored-entity to be built from scratch rather than by restricting entity#2016-12-0923:05Alex Miller (Clojure team)sometimes I feel like the right answer to these kinds of questions is to re-think the modeling approach though#2016-12-0923:07nwjsmithIā€™m open to remodelling šŸ™‚. Iā€™m not sure how I could re-write :parent/stored-entity to solve this.#2016-12-0923:07Alex Miller (Clojure team)I guess I would ask whether you really need to differentiate stored/unstored#2016-12-0923:11nwjsmithI think I do. There are some functions that would need to ensure the entity is already on disk. An update-parent-in-place function would need it for example.#2016-12-0923:12nwjsmith(that example probably counts as vulgarity on Clojurians)#2016-12-0923:14bbloomspec lacks parameterized abstractions / generics / whatever - trying to encode that invariant is unlikely to work out - i recall some discussion of alternatives at some point in this room#2016-12-0923:14bbloomone suggestion was just use two different keys#2016-12-0923:15bbloomfor example ::parent/stored-children and ::parent/children#2016-12-0923:16bbloomwhat extra data do stored children have?#2016-12-0923:16nwjsmithsame thing, just a :child/id#2016-12-0923:16bbloomcould you have ::parent/children and ::parent/stored where the stored key contains a map of children ids to a stored boolean?#2016-12-0923:16bbloomor timestamp?#2016-12-0923:16bbloomsee if you can come up with a representation that doesnā€™t change over time#2016-12-0923:16bbloomwhere stored state ~= time#2016-12-0923:17nwjsmithIā€™ve considered using something like Om/Datomicā€™s tempids#2016-12-0923:17bbloomyeah, or symbols#2016-12-0923:17bbloomyou can always unfold a graph with symbols, ids, whatever#2016-12-0923:18nwjsmithThat way a parent always has an id, but it a stored one has a ā€˜permanentā€™ id.#2016-12-0923:18nwjsmithK, Iā€™m going to explore this a bit more and see what comes of it#2016-12-0923:19bbloomyeah, try to model it openly where you can accrete data by adding more keys w/o ā€œchangingā€ the thing#2016-12-0923:19bbloomthis way some key always means the same thing#2016-12-0923:19bbloomit will be easier to reason about and youā€™ll gain new features you didnā€™t even expect to have#2016-12-0923:21bbloomfor what itā€™s worth, i think ā€œparameterized specsā€ may be useful, but iā€™m tempted to say weā€™re better off without them for as long as we can get away with it - if not forever#2016-12-0923:48Alex Miller (Clojure team)Alternately, consider a separate predicate that checks whether all things are storeable#2016-12-0923:48Alex Miller (Clojure team)That can be combined with the information #2016-12-1000:39gfredericks@bbloom I can't really spec test.check generators meaningfully without parameterized specs#2016-12-1000:39bbloom@gfredericks 1) i assume that thereā€™s existing code out there for which thatā€™s true 2) iā€™d love to hear why and 3) iā€™m curious if youā€™d do things differently now to avoid that#2016-12-1000:40gfredericks@bbloom what's a good generator for the function (defn vector "Returns a generator of vectors of items from g" [g] ...)?#2016-12-1000:41bbloomi havenā€™t a clue#2016-12-1000:41gfredericksright now I just have the args as (s/cat :g generator?) and the ret as generator?, which doesn't remotely capture the relationship between the two#2016-12-1000:41gfredericksor even the fact that the return value generates a vector#2016-12-1000:41gfredericksso there's two aspects to this#2016-12-1000:41bbloomwell at least now we can join the Go community in complaining that our ā€œtype systemā€ doesnā€™t have generics šŸ™‚#2016-12-1000:42bbloomthatā€™s interesting#2016-12-1000:42gfredericksI can't imagine what would be done differently to avoid this#2016-12-1000:42bbloomiā€™d be really curious to hear richā€™s thoughts on this#2016-12-1000:42bbloomb/c i canā€™t imagine he hadnā€™t considered it#2016-12-1000:43gfredericksany comment I've seen about higher-order-ish things is usually along the lines of "specs are for data"#2016-12-1000:43bbloomsure, the problem comes in once you introduce contextual specs#2016-12-1000:44gfredericksa corollary of which I have to imagine is "specs aren't useful for things like generators", but I couldn't defend that and haven't heard it explicitly taken that farb#2016-12-1000:44bbloomwhich happens instantly once you have functions#2016-12-1000:44gfredericksyeah you can't spec map completely for the same reason#2016-12-1000:44gfredericksso I guess you're right that they must have thought about it :)#2016-12-1000:44bbloomyeah, i think itā€™s perfectly OK to only spec data ā€œat restā€ so to speak#2016-12-1000:45bbloomin theory it could be extended to spec more things, but youā€™re not going to serialize a generator to edn#2016-12-1000:45gfredericksyou ultimately can't, because of bind#2016-12-1000:45bbloomat least not from opaque closures#2016-12-1000:45bbloomyeah#2016-12-1000:45gfrederickseven fmap I guess#2016-12-1000:46bbloomi guess ā€œspec doesnā€™t cover that use caseā€ is fine - iā€™m just curious if thatā€™s now & more stuff could be specā€™d later with more powerful primitives, or if itā€™s like ā€œeh, this approach only goes so far & you can spec some other way later if you need to"#2016-12-1000:47bbloomlike in theory you could reuse specs in another context, which i guess is what Typed Clojure will do: generate types from specs#2016-12-1000:47bbloomor otherwise treat specs as types#2016-12-1000:47bbloomfor me, iā€™ve basically only bothered to spec things that i intend to pprint#2016-12-1000:47bbloomšŸ™‚#2016-12-1000:48bbloombasically: ā€œhey spec, save me the trouble of looking at this thing and checking to make sure it looks right"#2016-12-1000:49gfredericksso a parameterized spec is a function that takes a spec and returns a spec#2016-12-1000:50gfrederickspresumably you could make a spec for that function#2016-12-1000:50gfredericksand test it by generating specs?#2016-12-1000:50gfredericksthis is really hazy#2016-12-1000:50gfrederickshaskell uses => to talk about this stuff doesn't it?#2016-12-1000:50bbloomsoooorta#2016-12-1000:51bbloom=> is used to constrain this stuff#2016-12-1000:51bbloomthe => basically means to unify things on the left with the environment#2016-12-1000:51gfredericksoh I guess the parameterization is more basic#2016-12-1000:52gfrederickslistOfThings :: [a]#2016-12-1000:52gfredericksis already parameterized#2016-12-1000:52bbloomso like if you have some type a => b -> c that says you have some function of type b, returning c, and there exists some aā€¦ and you can constrain it like: Num a -> b c there exists some a such that a is tagged with the class Num#2016-12-1000:52bbloomyeah, you donā€™t need to talk about constraints to get to parameters#2016-12-1000:52gfredericksdoes that mean anything interesting if a doesn't appear on the right side of the =>?#2016-12-1000:52bbloomyou could have listof Things :: Num a => [a] if my syntax is correct#2016-12-1000:53gfrederickslistOfNumberishThings#2016-12-1000:53bbloomi believe that the itā€™s required to be on the right side at least in haskell 87 w/o extensions#2016-12-1000:53bbloomi used unique letters for the sake of clarity talking about parts of it#2016-12-1000:53bbloomalso, i suck at haskell, so donā€™t ask me šŸ™‚#2016-12-1000:54gfrederickshey @bbloom I think I got my lens stuck in a monad transformer can you help.#2016-12-1000:54bbloomyouā€™re going to have to amputate.#2016-12-1000:55bbloombut yeah, spec obviously has specs parameterized by specs, like every#2016-12-1000:55gfredericksare those all special?#2016-12-1000:55bbloomor cat, or alt#2016-12-1000:55gfredericksor fspec#2016-12-1000:55bbloomyeah#2016-12-1000:56bbloomthe issue is that named specs canā€™t have parameters#2016-12-1000:56gfredericksI've wondered if (generator integer?) could be a thing#2016-12-1000:56bbloomthereā€™s no s/fn#2016-12-1000:56gfrederickspresumably it could be implemented with the lower-level machinery#2016-12-1000:56gfrederickseven then you couldn't fully describe gen/vector#2016-12-1000:57bbloomlike in theory you could have (s/fn [val] (s/every-kv integer? val))#2016-12-1000:57gfredericksthis is the "function from specs to specs" idea#2016-12-1000:57bbloomyeah#2016-12-1000:57bbloomhowever, this gets in to an area where i feel the current tradition of type systems is real bad#2016-12-1000:58gfredericksuh oh#2016-12-1000:58bbloompositional parameters force you to provide all the values when you call it#2016-12-1000:58bbloomtypes are curried in haskell (i think), but still, you donā€™t actually get a type until you give it all the args#2016-12-1000:59gfrederickswhere does that get awkward?#2016-12-1000:59bbloomb/c once you add a parameterized type, itā€™s like a virus#2016-12-1000:59bbloomeverything above it needs to be parameterized until you know a concrete type#2016-12-1000:59bbloomsame problem with monads šŸ˜‰#2016-12-1001:00bbloomor checked exceptions#2016-12-1001:00bbloom"oh shit, this function has an effect. ooooh well, letā€™s change the signature of every function everywhere"#2016-12-1001:01bbloomif i were designing a type system with generics, iā€™d make all generic parameters named and optional#2016-12-1001:01bbloomso youā€™d have (List {}) and (List {:element String}) - in the former case, the default for :element would be Any#2016-12-1001:01gfredericksso "the type system knows about effects" seems like a positive thing, it's just the "and I have to type it in all sorts of unrelated places" that's the bad part, right?#2016-12-1001:02gfrederickswould a more advanced inference system fix that?#2016-12-1001:02bbloomor whatever List declared it to be#2016-12-1001:02bbloomyeah, thatā€™s the bad part#2016-12-1001:02bbloommore advanced inference would help, but the badness comes from the syntax#2016-12-1001:02bbloomhaskell avoids this problem a little bit by using currying#2016-12-1001:03bbloomif you have some function that has an effect, you just leave the monad as the last parameter & then if you change the monad, thatā€™s fine b/c youā€™re using curring / point-free to avoid mentioning the monad type as much as possible#2016-12-1001:05gfredericksthe monad is the last constraint mentioned before the =>? or it's actually a physical argument?#2016-12-1001:06bbloomthe order ofthings left of the => donā€™t matter, as far as i know#2016-12-1001:06bbloomthe syntax is super confusing#2016-12-1001:06bbloomthe monad would be an actual argument#2016-12-1001:07bbloombut you can leave it out of the signature or just say ā€œhey, i return an a"#2016-12-1001:07gfredericksis that a general thing you deal with with monad transformers?#2016-12-1001:07bbloommonad transformers are a whole ā€˜nother bag of goofy ideas šŸ™‚#2016-12-1001:08bblooma lot of these problems stem from the fact that they donā€™t have associative or open structures in their type systems#2016-12-1001:09bbloomif you want to combine more than 1 monad without manually creating a custom monad, you can layer them and the order in which you layer them matters b/c the type of a monad stack is essentially a cons list of monad types#2016-12-1001:09bbloomthe "extensible effects" stuff comes up with a clever way to encode a set without order in to the type#2016-12-1001:10bbloomGHC now has some kind of union types with some extension, but at some people they were like sorting types to represent a multi-set or some craziness to achieve ā€œorder independenceā€#2016-12-1001:10bbloombut that still failed to address openness properly#2016-12-1001:11gfredericksokay so static types aren't all good yet.#2016-12-1001:11bbloomlol nope#2016-12-1001:12bbloomeverybody in the ML/Haskell camp AND the Lisp camps were afraid to take a step back and say ā€œhey, what happens if we have maps and setsā€ šŸ™‚#2016-12-1014:32cemerickwhat is the current best idea for how to spec a map with (un-namespaced) symbol keys? The best I've figured is to use every to spec map entries, not very satisfying.#2016-12-1016:14gfredericks@cemerick I don't know of anything besides an opaque function#2016-12-1016:14gfredericksI haven't investigated how to use the lower-level facilities to build something more first-class#2016-12-1016:23cemerick@gfredericks having something equivalent to s/keys (s/syms? :-P) doesn't look hard, but way more work than manually :syms destructuring and conforming each part#2016-12-1016:31gfredericksI'm more likely to need strings, personally#2016-12-1016:38cemerickand bob needs numbers, and poor jane is dealing with maps that have namespaced and un-namespaced keywords with the same name carrying different stuff#2016-12-1016:39gfredericksmake a new system and call it spec2#2016-12-1016:39cemerickstahp#2016-12-1016:43cemerickyour convo with @bbloom above is serendipitous. I was really trying to get spec to do way more than it's meant to last night, notwithstanding my cat problems.#2016-12-1016:44gfrederickssomeone should build some sort of adapter to plumatic/schema for fallback needs#2016-12-1016:44gfredericksteach spec that schemas are specs and teach schema that namespaced keywords are schemas#2016-12-1016:45cemerickwell, part of my frustration is in wanting to get useful generators out of specs, so, things like schema aren't super useful#2016-12-1016:46gfredericksschema has generator support too#2016-12-1016:46gfredericksit's just more wiring to do#2016-12-1016:47cemerickwell#2016-12-1016:47cemerickinsofar as I need to do extra work to get corresponding generators (esp. if schema/spec is a separate thing from the generator), it's not super useful#2016-12-1016:48cemerickI mean, beyond doing the work to have a validating/destructuring spec and corresponding generator#2016-12-1016:49gfredericksI'm not sure what extra work you're thinking of; I think the overhead to get to generators is comparable in the two systems#2016-12-1016:50cemerickI'm not sure either, I guess. It's been a while since I touched schema.#2016-12-1016:50cemerickmy recollection was that I ended up bailing and hand-writing my own generators at a certain level of complexity#2016-12-1016:52gfredericksthe big unfortunality is that (every version I've used) has bad errors when there's a missing generator you need to fill in#2016-12-1016:52gfredericksit just says "Missing leaf generator" or something of that sort and doesn't attempt to tell you which schema caused the problem#2016-12-1016:52gfredericksbut once you build up your stockholm syndrome around that issue then it goes pretty well#2016-12-1016:53cemerickthis callous is my own, there is no other like it#2016-12-1017:47bbloom@cemerick i too suffer from instantly running in to edge cases and limits when attempting to use new things šŸ™‚#2016-12-1017:53bbloomgfredricks as well - weā€™re just those guys who instantly find the dark corners#2016-12-1018:27cemerickI don't know that I find dark corners any more frequently or quickly than others. Probably just louder than most about it, maybe#2016-12-1018:30cemerick@bbloom the filesystem expression problem tweet was deeply triggering#2016-12-1018:30bbloomĀÆ\(惄)/ĀÆ#2016-12-1018:30cemerickmmm, guess that's OT here#2016-12-1019:20Alex Miller (Clojure team)@cemerick Re your question above, did you look at s/keys with :req-un ? #2016-12-1019:21Alex Miller (Clojure team)@gfredricks for string keys, you can use a leading conformer that converts to keywords as a workaround#2016-12-1019:23Alex Miller (Clojure team)Assuming you have an information map with keys that are strings ala json#2016-12-1020:19gfredericks@alexmiller was the s/keys with :req-un the same as your other two comments, or about something different?#2016-12-1020:19gfredericksI didn't know about the leading conformer idea, so that's useful#2016-12-1020:57Alex Miller (Clojure team)It was about the original question of maps with unqualified keys#2016-12-1021:06Alex Miller (Clojure team)you can do lots of tricks with leading conformers, like for example calling select-keys to narrow the key set youā€™re validating#2016-12-1021:06Alex Miller (Clojure team)yada yada beware of conformers yada yada#2016-12-1021:21gfredericks@alexmiller unqualified keys are pretty basic, right? chas had been asking about symbols as keys, which I assume is the same approach as strings#2016-12-1021:30Alex Miller (Clojure team)oh, right I heard unqualified keys in my head#2016-12-1021:31Alex Miller (Clojure team)but you could use aĀ leading conformer for symbol->keys too#2016-12-1021:32gfredericksyeah#2016-12-1021:51bbloommay i ask: why are you using strings or symbols heterogeneously?#2016-12-1021:51bbloombesides maybe strings ~= keys when doing interop w/ java maps or javascript objects#2016-12-1022:05gfredericksI like using strings when talking about javascript in clojure#2016-12-1022:05gfrederickss/javascript/json/#2016-12-1022:06gfredericksmy rule of thumb is not using keywords that don't appear in the source code anywhere#2016-12-1022:06gfredericksin the past I've used schemas to describe both the internal clojure-fluent version of data but also the external JSON-flavored version#2016-12-1022:06gfredericksand I found that terribly useful#2016-12-1022:07gfredericksspec does not fit that pattern nearly as well as p**matic/schema#2016-12-1022:10seancorfieldIā€™m curious why you donā€™t use keywordize at the boundary? I find that approach far more natural.#2016-12-1022:11gfredericksI essentially do, but I prefer it in a less blind way#2016-12-1022:13gfredericksto be clear, these are mostly just thoughts. I've only tried this in one codebase, but couldn't figure out how to get a nice API for it. most of my thoughts are here: https://github.com/gfredericks/schema-bijections#rationale#2016-12-1022:13gfredericksso what I want is something like ā†‘ that library, but less confusing/verbose, and maybe involving spec if that ends up making sense#2016-12-1022:16bbloomyeah, i definitely support using strings for json rather than keywordize everywhere#2016-12-1022:17bbloomdoes raise some questions about how spec should work for json tho#2016-12-1022:17gfredericksI suppose that readme doesn't talk about trying to separate bijections and surjections, which I never figured out how to do#2016-12-1022:18gfredericksI wish bijections and surjections were first-class concepts#2016-12-1022:19gfredericksthey compose well; surjections are interesting for generators#2016-12-1022:20gfredericksif I can ever work this into a better library it probably ought to be named jections#2016-12-1023:26pedroiagois it intentional that s/? unforms a s/cat into a list? 'e.g. (s/unform (s/? (s/cat :a #{1})) {:a 1}) ;=> [(1)]'#2016-12-1100:45Alex Miller (Clojure team)That shouldn't be nested like that - there's one bug filed about that and a few other cases we are looking at#2016-12-1102:43cemerick@alexmiller the challenge is unqualified symbols for keys#2016-12-1102:44cemerickah, saw @gfredericks already corrected you on that#2016-12-1102:47cemerick@alexmiller yeah, I'm familiar with conformers; what I was wanting to do is retain the automagic generator generation, so conformers aren't that. Only the top level is currently symbol-keyed (everything else is sequences of various sorts), so I can easily destructure out for conforming, and manually assoc in the results of generators when doing that.#2016-12-1102:51cemerick@bbloom I've been angry about JS[ON] not having symbol literals every single day for ~6 weeks.#2016-12-1102:55cemerick@gfredericks looking forward to jections. I actually want two different destructurings coming out of conform, depending on context: JSON -> basically symbolic representation that makes sense for the particular structure I've spec'd, or (2) a completely desugared representation that does things like tag the matching spec on s/or's, flows through the custom destructurings I've put in place via conformers, etc.#2016-12-1102:55cemerickwouldn't be surprised if this requires the aforementioned higher-order / spec generics though#2016-12-1105:17gowderI'm just gonna be proud of myself that I wrote my first actual specs today, while y'all are talking the fancy stuff :-)#2016-12-1105:47cemerick@gowder I'm proud of you, too šŸ™‚#2016-12-1117:39thomasHi, I am trying to spec my function... and I have spec'ed the :args and the :ret... but now trying to spec the :fn...#2016-12-1117:40thomasand I have added the example spec (which I know is wrong... but now I don't get any errors ...#2016-12-1118:10gfredericks:ret and :fn only get checked by doing clojure.spec.test/check or something of that sort#2016-12-1120:55gowderOr instrumenting#2016-12-1122:09Alex Miller (Clojure team)No, not instrumenting#2016-12-1201:10gowderOh whoops. Instrument just for args, duh. Lazy speed reading of slack posts is never a good idea :-(#2016-12-1208:15thomascheers... I'll investigate it some more.#2016-12-1210:16alexisvincentAny patterns to avoid things like this#2016-12-1210:16alexisvincent
(s/def ::dimensions/seq (s/cat ::dimensions/x ::dimensions/x ::dimensions/y ::dimensions/y ::dimensions/z ::dimensions/z))
#2016-12-1210:18dergutemoritz@alexisvincent You could use s/tuple instead - or are you interested in the conforming behavior of s/cat?#2016-12-1210:19dergutemoritzThen I think your only option to avoid the redundancy is to write a macro around s/cat#2016-12-1210:19alexisvincent->
(defn make-dimensions [& d]
  (s/conform ::dimensions/seq d))
#2016-12-1210:20alexisvincenthmm. was wondering if there was something like the destructureing {:keys []}#2016-12-1210:21dergutemoritzSo you want to conform your sequential input into a map, right?#2016-12-1210:21alexisvincentyep šŸ™‚#2016-12-1210:21dergutemoritzAh, how about s/keys*#2016-12-1210:21dergutemoritzI think that's what you're looking for#2016-12-1210:21dergutemoritzWell no, it's not, sorry šŸ˜„
#2016-12-1210:22dergutemoritzThat would require the keys themselves to be present in the input sequence, too#2016-12-1210:22dergutemoritzSo yeah, you'll have to roll your own wrapper macro#2016-12-1210:24alexisvincent@dergutemoritz Thanks šŸ™‚ I wonder though if tuple is not what I want actually#2016-12-1210:25dergutemoritz@alexisvincent YW! Yeah, s/tuple would get rid of the redundancy but it wouldn't conform into a map#2016-12-1210:25dergutemoritzBut perhaps you can live with that#2016-12-1210:25alexisvincentOh right.#2016-12-1210:25alexisvincentI wonder if s/keys maintains order?#2016-12-1210:26alexisvincentBecause then I could write a function that first accepts req keys and then optional keys#2016-12-1210:26alexisvincentin order#2016-12-1210:28dergutemoritzFWIW, here's a wrapper macro of the kind I'm talking about:
(defmacro keys-cat [& keys] `(s/cat 
#2016-12-1210:28dergutemoritzOr maybe keys-tuple would be a more fitting name#2016-12-1210:33alexisvincent@dergutemoritz Thanks šŸ˜„#2016-12-1210:33alexisvincentmaybe cat-keys#2016-12-1210:34dergutemoritzhttps://s-media-cache-ak0.pinimg.com/originals/c1/19/8f/c1198f0990a37c100437b8c31309c4e8.jpg#2016-12-1210:49alexisvincentxD#2016-12-1211:35alexisvincentis the general consensus to prefer namespaced keys in maps?#2016-12-1212:50dm3it seems so, for best synergy with spec#2016-12-1213:07alexisvincentAre people generally using :: as a helper#2016-12-1213:08alexisvincentIts a pain when wanting to use the spec in another nsā€¦ You have to specify the full mylonglongname.namespace.thing/name#2016-12-1213:09alexisvincentbecomes so long windedā€¦#2016-12-1213:09alexisvincentOr are people just registering thing/name#2016-12-1213:14alexisvincentOH šŸ™‚ :: can also be used for aliasing#2016-12-1213:15dm3also (require [some.ns :as n]) (is (= ::n/kw :some.ns/kw))#2016-12-1213:15dm3I guess that's what you meant#2016-12-1213:54alqvistWhat is the best way of namespacing an existing map? Using a function, not the reader#2016-12-1214:26alexisvincent@dm3 yep šŸ™‚#2016-12-1215:15Alex Miller (Clojure team)@alqvist what do you mean? adding a namespace to the keys of a map?#2016-12-1215:16Alex Miller (Clojure team)there is no function to do that right now#2016-12-1215:34alqvist@alexmiller exactly that - I made a small function for it. Posted it in #clojure#2016-12-1215:34Alex Miller (Clojure team)cool#2016-12-1217:34sparkofreasonIs there some preferred way of writing simulation-testable specs for functions with constraints between parameters? I have a function that takes a map and a key. The key must exist in the map for both the input and output. I can make it work by writing a custom generator for :args using bind, was wondering if there was a preferred method.#2016-12-1218:28Alex Miller (Clojure team)that is the preferred method#2016-12-1218:29Alex Miller (Clojure team)https://www.youtube.com/watch?v=WoFkhE92fqc#2016-12-1218:29Alex Miller (Clojure team)^^ is about that#2016-12-1218:44sparkofreasonThanks, that's basically how I handled it.#2016-12-1218:46sparkofreasonI notice in the video that the generator override was specified directly in the test, rather than in the spec. Don't know that it makes much difference, though giving it in the spec would allow you to reuse the override if that spec was composed with others.#2016-12-1218:48bbloomis there a spec or predicate for objects that have strict value equality? looking for something like a clojure.core/edn? or something like that#2016-12-1218:49bbloomedn? isnā€™t quite right, since thatā€™s about serializability, but would be close enough for my needs#2016-12-1218:52hiredmanthat would be nice#2016-12-1218:52hiredmanany? is so close, but fails with NaNs#2016-12-1218:53bbloomno, any? isnā€™t close - it includes functions#2016-12-1218:53bbloomi have a procedure that will go in to an infinite loop if the function isnā€™t pure & iā€™ve been bitten a few times by including a function in it#2016-12-1218:54bbloomso iā€™m doing like add-watch! does, which is force key values#2016-12-1218:54hiredmanI am pretty sure the generator for any? doesn't generate functions#2016-12-1218:54bbloomgeneration aside, the spec is wrong if i allow functions šŸ™‚#2016-12-1218:55bfabrythe generator for any? was way simpler than preferred last time I checked, there was a bug on test.check about it#2016-12-1218:55bbloomi need a predicate like value-equality?#2016-12-1218:56bbloomwhich i guess i could make, but it seemed like it might have been something that existed or should exist#2016-12-1218:56bbloomi didnā€™t feel like exhaustively listing whatā€™s in edn in a spec#2016-12-1218:56Alex Miller (Clojure team)I think there are some things in test-check with "printable" in their name?#2016-12-1218:57Alex Miller (Clojure team)I know we spent a lot of time building up stuff like that in the old data.generative#2016-12-1218:57Alex Miller (Clojure team)Certainly open to ideas#2016-12-1218:57bbloomhmm ā€œprintableā€ is a good hint, iā€™ll look - thanks#2016-12-1218:58hiredmanthe problem with the printable stuff is it generates NaNs#2016-12-1218:59hiredmanand those are generators, not predicates#2016-12-1218:59bbloomalexmiller: this looks more or less like what i need. would be cool to have a predicate for it#2016-12-1219:00Alex Miller (Clojure team)Jira it up#2016-12-1219:00Alex Miller (Clojure team)Seems like we could also fix test.check re NaN (which is not currently printable)#2016-12-1219:05bbloomhttp://dev.clojure.org/jira/browse/CLJ-2083#2016-12-1219:05bbloomi think clojure.core/edn? would be the least objectionable formulation of this request#2016-12-1219:06bbloomwell, actually - may not#2016-12-1219:06bbloomb/c that would be a ā€œdeepā€ predicate#2016-12-1219:06bbloomi donā€™t think any such deep predicates exist - it would have O(N) runtime šŸ˜•#2016-12-1219:39Alex Miller (Clojure team)I think it's probably unlikely we would make that predicate#2016-12-1219:40bbloomyeah - after realizing it was O(N), i realized that - but a spec makes more sense, right?#2016-12-1219:41Alex Miller (Clojure team)Well, definitely a generator, maybe a spec#2016-12-1219:41bbloombesides the stuff in clojure.core.specs, are there any ā€œstandardā€ specs anywhere? i donā€™t think so, right? just predicates...#2016-12-1221:06jeremyrainesIn this example from the spec guide, is the ability to key into the arguments with :start and :end created by the s/cat above the anon function? Does the destructuring happen such that the first expression given to s/and has an effect on whatā€™s passed to the other?
(s/fdef ranged-rand
  :args (s/and (s/cat :start int? :end int?)
               #(< (:start %) (:end %)))
ā€¦
#2016-12-1221:07bfabry@jeremyraines yes, the s/cat conforms the data. s/and threads the conformed data#2016-12-1221:07jeremyrainesok, thanks#2016-12-1221:07zaneSuper useful for conformers that coerce between types.#2016-12-1304:39j-poI'm trying to spec something like s-expressions, and my current flailing in that direction involves a recursive s/tuple (something like
(s/def ::sexp
  (s/alt :arg integer?
            :expression (s/tuple fn?
                                              ::sexp
                                              (s/* ::sexp))))
), but I'd like for the input not to have to be a vector of vectors. Am I better off just turning the input into such a vector as part of the checking process, or is there another way?
#2016-12-1305:29Alex Miller (Clojure team)s/cat?#2016-12-1305:32Alex Miller (Clojure team)then you can just replace ::sexp (s/* ::sexp) with (s/+ ::sexp) too#2016-12-1305:34Alex Miller (Clojure team)
(s/def ::sexp (s/alt :arg integer? :expression (s/cat :f fn? :args (s/+ ::sexp))))
#2016-12-1309:22j-po@alexmiller Thanks! When I gen/generate from an s/cat approach, though, I don't get a sequence of sequences, though, but just one flat one.#2016-12-1309:25vandr0iyI was asking stuff in the beginners channel, but probably this is a better place for doing that. I posted this snippet of code, claiming that it doesn't work:
(let [arst {:a [1 2 3]
            :b {:c #{::z ::x ::c}}}]
  (s/keys :req-un (get-in arst [:b :c])))
And I was told that :req-un wants a literal vector of namespaced keywords, not something that evaluates to one, because of it being a macro (obviously) - which makes sense. So I went a step further, and macro'd my stuff like this:
(def my-bunch-of-stuff {:a {:b [:x :c :d]}
                      :z {:b vector?}}

(defmacro arst [type]
  (let [sp# (get-in my-bunch-of-stuff [type :b])]
    (if (fn? sp#)
      `(s/spec ~sp#)
      `(s/spec (s/keys :req-un ~(vec (map #(->> % str keyword) sp#)))))))
and it still doesn't work OOB; while, if I copy-paste the result of macroexpand '(arst :a) in the REPL - it works flawlessly. What's going on here? I'm afraid that there's some rookie mistake here somewhere...
#2016-12-1309:28mpenetyou need to use either a macro or eval:#2016-12-1309:28mpenet
(let [arst {:a [1 2 3]
                :b {:c #{::z ::x ::c}}}]
  (eval `(s/keys :req-un ~(get-in arst [:b :c]))))
#2016-12-1309:28mpenet(s/keys is a macro)#2016-12-1309:30mpenetand you might actually need to call seq or vec on the result of the get-in, not sure s/keys will happily take a set as :req-un value#2016-12-1309:33mpenetyour macro should work (kinda, you need namespaced keys in your bunch-of-stuff and there's a paren missing there too)#2016-12-1309:40vandr0iyI'm sorry for adding the macro after the request, I accidentally pressed enter before completing my post. My keys aren't namespaced - I "namespace" them with the (map #(->% str keyword) sp#) thing. What puzzles me is: why does this macro work if I try to do (arst :z) - which takes a function from the bunch-of-stuff and spits a spec out of it - and doesn't if I get a vector of un-namespaced keywords, "namespace" them with the dirty hack described above??? I (macroexpand '(arst :a)), and copy-paste what I get in REPL - and it works, but it doesn't if I just try to do (arst :a). I'm pretty sure that ~(vec (map #(->> % str keyword) sp#)) yields a vector of namespaced keywords...#2016-12-1309:43mpenetit doesnt I think, it creates keys with ":foo" content with a leading :, so yeah ::foo but not namespaced per say#2016-12-1309:43mpenettry running your spec, it will complain about this. Try changing your keys in bunch-of-stuff with ns keys you ll see#2016-12-1309:52vandr0iyOh... I see. Was able to make it work using this other horrible hack:
(defmacro arst [type]
  (let [sp# (get-in settings [type :b])]
    (if (fn? sp#)
      `(s/spec ~sp#)
      `(s/spec (s/keys :req-un ~(vec (map #(keyword (str *ns* "/" (name %))) sp#)))))))
#2016-12-1309:57trisshow do I write a spec for an instance of Comparable?#2016-12-1310:01vandr0iy(s/valid? (s/spec #(instance? Comparable %)) "arst")#2016-12-1310:02trissthanks @vandr0iy . Iā€™ll give that a try#2016-12-1310:43trissthat worked. Brilliant thanks.#2016-12-1310:44trissIf I want a generator that produces a wider range of vaues than the ones provided for double-in and int-in how would I go about getting one of those?#2016-12-1310:44trissall the random numbers Iā€™m getting are pretty small and I need to test against some big ones.#2016-12-1310:50mpenettest.check has generators that allow you to specify a range#2016-12-1310:50mpenetlarge-integer* and double* I think#2016-12-1310:51trisscheers @mpenet Iā€™ll have alook#2016-12-1310:53triss(gen/sample (gen/double* {:min 1 :max 200})) still produces really small values#2016-12-1310:53trissIā€™d like to see some values closer to 200. Is this possible?#2016-12-1311:00mpenetTry sampling more values#2016-12-1311:13trissah ok.. looks generators probably arenā€™t going to be efficient enough for the way I was abusing them.#2016-12-1311:50trissIf I wanted a more evenly distributed generator would it be possible to write one?#2016-12-1312:13luxbock@triss yes but you need to use the generators in test.check#2016-12-1312:14gfredericks@triss what's a uniform distribution over doubles?#2016-12-1312:14gfredericks@triss oh I think this is a sizing issue, sorry I didn't read back far enough#2016-12-1312:15triss@gfredericks ah I can see why that might be awkward. rand seems more even than this though#2016-12-1312:15gfredericks@triss you should get lots of values close to 200, but gen/sample is deliberately only showing you small examples#2016-12-1312:15gfredericks@triss try (gen/sample ... 200)#2016-12-1312:15luxbockis there any reason for why s/spec couldn't accept a third optional argument for defining a custom generator? I thin it would make it easier to statically analyze specs and might end up with some cool tooling use cases#2016-12-1312:16gfredericks@triss test.check has some subtleties with sizing that need to be documented better, and I have 45 minutes free right now so I think I will start on that#2016-12-1312:17triss@gfredericks I still see a similar distribution. biased to low numbers#2016-12-1312:17trissah many thanks @gfredericks will be much appreciated#2016-12-1312:17gfredericks@triss it's not going to be uniform, but you should definitely get some larger numbers#2016-12-1312:18gfredericks@triss one reason I asked my original question about what a uniform distribution is, is because doubles themselves aren't uniform -- there are a lot more numbers between 1 and 50 than 50 and 200#2016-12-1312:18gfredericksso uniform might mean different things in different contexts#2016-12-1312:18trissoh thatā€™s interesting. because of the way they are represented in the machine?#2016-12-1312:18gfredericksyou could say that#2016-12-1312:19gfredericksit's kind of inherent in the idea of floating point#2016-12-1312:19trissI guess Iā€™d like a distribution of Real numbers? does that make more sense?#2016-12-1312:19triss^a more even distribution of real numbers#2016-12-1312:19gfredericksnot really, but I think I know what you're trying to get at#2016-12-1312:19gfredericks@triss you could use large-integer to get this pretty easily#2016-12-1312:20trissIā€™d love to see a flatter histogram!#2016-12-1312:20gfredericksor at least something close#2016-12-1312:20gfredericksyes that's a good way of putting it :)#2016-12-1312:20trissah brilliant will look at large-integer#2016-12-1312:20gfredericks(gen/let [x (gen/large-integer {:min 0 :max 200000000})] (/ x 1000000.0))#2016-12-1312:21gfredericks@triss ā‡‘ something like that#2016-12-1312:21gfredericksit won't get you every double in that range, but it might be okay for what you're doing#2016-12-1312:21gfredericksand it wouldn't be hard to make it fancier so that it hits most things#2016-12-1312:22trissthanks. this will probably do nicely.#2016-12-1312:22gfredericksnp#2016-12-1312:27gfredericksoh ha yes I forgot that#2016-12-1312:27gfrederickssorry#2016-12-1312:28trisslooks like I wonā€™t be generating my population of genotypes with spec or generators.#2016-12-1312:28trissor hang on I can write a custom one?#2016-12-1312:28gfredericks@triss try (gen/let [x (gen/choose 0 200000000)] (/ x 1000000.0))#2016-12-1312:29gfredericks@triss ā‡‘ these essentially are custom generators#2016-12-1312:29gfredericksit's all about composing lower-level generators to get what you want#2016-12-1312:29gfredericksgen/choose is one of the lowest-level ones, and gives you a uniform distribution over a range of integers#2016-12-1312:29trissgen/choose is perfect for what Iā€™m doing I think....#2016-12-1312:29gfredericks@triss now that I think about it, I might recommend the large-integer approach anyhow, but it's tricky to explain why#2016-12-1312:30mpenetworst case you can cheat the thing with (gen/fmap #(do whatever you want) (gen/return nil))#2016-12-1312:30trissok - I need to scratch my head about composing these things for a while....#2016-12-1312:31gfredericks@triss this might be useful: https://github.com/clojure/test.check/blob/master/doc/generator-examples.md#2016-12-1312:32trisswonderful thanks.#2016-12-1312:42mpenetI have this one bookmarked šŸ™‚ https://github.com/clojure/test.check/blob/master/doc/cheatsheet.md#2016-12-1312:44gfredericks@triss w.r.t. choose vs large-integer, the short explanation is that test.check has a strategy where it tries small things first and slowly tries larger and larger things; by using choose you're overriding that strategy, and that strategy is the reason you saw an uneven distribution with large-integer#2016-12-1312:45gfrederickswhich is best depends on your goals, but I'd say large-integer would be a good default#2016-12-1312:46trisswould you consider using generators outside of testing code foolish?#2016-12-1312:47trissit seems very much tailored for testing.... (as the namespace indicates I suppose)#2016-12-1312:49gfredericks@triss clojure.spec uses them for a bit more than test.check does, but still for testing purposes; if you're using them for not-testing-at-all, it will be a little weird but not terrible; understanding the sizing subtleties could be more important though#2016-12-1312:50gfredericksI suppose based on that jungle music talk at the conj I should expect people will be using generators for all sorts of things#2016-12-1312:53gfredericks@triss the gen/generate function will probably be useful for you, since it accepts a size parameter#2016-12-1312:53gfrederickse.g., (repeatedly 200 #(gen/generate (gen/let ...) 200)) should give you the distribution you wanted even using large-integer#2016-12-1322:53agGuys, can someone help me with this: I have a long string of lorem ipsums, something like this "pellentesq dapib suscip liguldon posue augquaeti v tort ..." and now I need a spec with a generator that would generate a name of 3 to 5 words long by randomly pulling those words from that lorem-ipsum string. How do I do that?#2016-12-1323:21adambros
(def s ā€œpellentesq dapib suscip liguldon posue augquaeti v tortā€)
(clojure.string/join " " (take 3 (shuffle (clojure.string/split s " ā€œ))))
;=> "augquaeti posue vā€
@ag does this look right?
#2016-12-1323:21adambrosoh you said generatorā€¦ hmm#2016-12-1323:26j-poYou could make a generator for sequences fitting that format. Composing in string/join is the unknown bit, but it seems doable.#2016-12-1323:28j-poYou could do that in a custom generator, for instance, but then you'd have a custom generator šŸ˜•#2016-12-1323:29adambros
(gen/sample (gen/bind (gen/shuffle (str/split "foo bar baz qux asd wet ab" #" ")) #(gen/frequency [[5 (gen
/return (take 3 %))] [5 (gen/return (take 5 %))]])))
;=> (("foo" "baz" "bar" "ab" "wet") ("foo" "ab" "baz") ("wet" "bar" "baz") ("baz" "bar" "ab") ("bar" "ab" "baz") ("foo" "
ab" "baz") ("foo" "wet" "asd" "ab" "baz") ("asd" "wet" "ab") ("qux" "wet" "foo" "bar" "baz") ("ab" "bar" "asd" "foo"
"baz"))
#2016-12-1323:29agyeah but I canā€™t make this work, I got this far:
(gen/generate
 (s/with-gen string? #(gen/generate (gen/shuffle (clojure.string/split lorem-ipsum #ā€\s")))))
#2016-12-1323:30adambrosgot it to alternate between take 3 and take 5#2016-12-1323:30adambrosi think you see the pattern#2016-12-1323:30agoh, let me try your snippet#2016-12-1323:30adambrosjust add an entry for (take 4 %) with w/e freq you want#2016-12-1323:32adambrosi think the only reason i came up with that so fast was the time i spent in haskell šŸ˜…#2016-12-1323:32adambrosbind man...#2016-12-1323:36ag@adambros oh this is awesome, now I think how could be gen/frequency part can be generalized#2016-12-1323:38gfredericksum#2016-12-1323:38gfrederickshow about#2016-12-1323:38gfredericks(gen/vector (gen/elements the-words) 3 5)#2016-12-1323:38gfredericks@ag ā‡‘#2016-12-1323:39gfredericksshould be more efficient than the shuffler too since it doesn't do lots of shuffling work just to throw it away#2016-12-1323:39gfredericksonly difference is it won't give you distinct elements#2016-12-1323:42ag@gfredericks oh, yeah, makes sense#2016-12-1323:42agthanks#2016-12-1323:42adambrosthats interesting, so i guess it depends on if you want/need distinct elements#2016-12-1323:45gfredericksif you need distinct, wrapping it in such-that might be efficient, as long as the collection is large enough#2016-12-1323:47agerrrhā€¦ now how I actually emit strings? str/joined? I am trying to wrap it in gen/fmap - doesnā€™t work ;(#2016-12-1323:54agnevermind got it!#2016-12-1323:54ag
(gen/fmap
    #(clojure.string/join " " %)
    (gen/bind (gen/shuffle (clojure.string/split lorem-ipsum #" "))
              #(gen/vector (gen/elements %) 1 5)))
Thanks everyone!
#2016-12-1400:02gfredericks@ag that does a bunch of unnecessary shuffling on each generation#2016-12-1400:03gfredericksI'd do (gmap/fmap #(clojure.string/join " " %) (gen/vector (gen/elements (clojure.string/split lorem-ipsum #" ")) 3 5))#2016-12-1400:03gfredericksno need for a shuffle at all#2016-12-1400:18agso gen/elements guarantees that it would pull elements randomly? then yeah, makes sense#2016-12-1400:18agthanks!#2016-12-1400:21agdamn it.. now Iā€™m struggling with turning it into a spec
(def ^:private l-ipsum-gen
  (gen/fmap #(clojure.string/join " " %)
             (gen/vector (gen/elements (clojure.string/split lorem-ipsum #" ")) 3 5)))

(s/gen (s/with-gen (s/and string?) l-ipsum-gen))
#2016-12-1400:21agdoesnā€™t work#2016-12-1400:23hiredmanif gen/ is clojure.spec.gen then you need to wrap things in no argument functions#2016-12-1400:23hiredmanor, I should say, you need to wrap generators in#2016-12-1400:26aghmmm, ya, canā€™t figure out right syntax#2016-12-1400:27agokā€¦ I think Iā€™ve figured it:
(def ^:private l-ipsum-gen
  (gen/fmap #(clojure.string/join " " %)
             (gen/vector (gen/elements (clojure.string/split lorem-ipsum #" ")) 3 5)))

(gen/sample (s/gen (s/with-gen (s/and string?) (fn [] l-ipsum-gen))))
#2016-12-1409:16artemyarulinHello. Where I should put specs: next to functions, next to tests, one global file with all specs for my project. Once Iā€™ve seen [module-name]-spec.clj separate file for each module.#2016-12-1410:14pyrHI! How do you go about generating related values?#2016-12-1410:14pyrA simple case would be an indexed map where keys need to reappear in values#2016-12-1412:18gfredericks@pyr are you familiar with gen/let and/or gen/fmap?#2016-12-1413:37bronsaI was toying earlier with implementing split-with using spec
user=> (require '[clojure.spec :as s])
nil
user=> (defn s-split-with [pred?] (s/cat :left (s/* pred?) :right (s/* any?)))
#'user/s-split-with
user=> (s/conform (s-split-with int?) (range 1e5))
StackOverflowError   clojure.lang.RT.get (RT.java:751)
#2016-12-1413:38bronsaanything obviously "wrong" or is this stack overflow a real issue?#2016-12-1413:39bronsaby contrast, if the pred fails early and the right branch is used, it doesnt overflow#2016-12-1413:39Alex Miller (Clojure team)It's greedy so I'd expect all 1e5 to go left#2016-12-1413:39bronsae.g.
(s/conform (s-split-with ident?) (range 1e5))
returns fine
#2016-12-1413:39bronsa@alexmiller yes, I'd expect that to happen, not at stack overflow#2016-12-1413:40Alex Miller (Clojure team)Yeah just thinking through#2016-12-1413:41bronsahttp://sprunge.us/SPDT#2016-12-1413:41bronsahere's the overflow stack trace if that's helpful#2016-12-1413:41Alex Miller (Clojure team)What if you don't have right?#2016-12-1413:42bronsait runs fine#2016-12-1413:42bronsa(this is ok (s/conform (s/cat :left (s/* int?)) (range 1e5)))#2016-12-1413:43bronsauhm, so is (s/conform (s/cat :left (s/* int?) :right (s/* ident?)) (range 1e5))#2016-12-1413:44bronsabut (s/conform (s/cat :left (s/* int?) :right (s/* int?)) (range 1e5)) overflows#2016-12-1414:53Alex Miller (Clojure team)the regex derivative is keeping track of all possible solutions - in the last case the split could occur anywhere so there are a large number of those (the others do not have that ambiguity)#2016-12-1414:54Alex Miller (Clojure team)so these are (algorithmically) very different specs#2016-12-1415:00Yehonathan Sharvitwhat is ident?#2016-12-1416:48Alex Miller (Clojure team)Keyword or symbol#2016-12-1417:13ddellacostafolks Iā€™m a bit confused by something right now, and apologies if this is documented somewhere that I missed: If I have a spec keyword in namespace x (`(def ::foo ā€¦)`) and then I refer to it via a ref in a separate namespace (`[x :as y]`), I canā€™t seem to use that refā€™ed keyword in a defmethod (`(defmethod foo :y/foo ā€¦)`)#2016-12-1417:13ddellacostadoes that make sense? Am I doing something obviously wrong?#2016-12-1417:28hiredman::y/foo#2016-12-1417:39josesanchThis is not working because s/keys is a http://macro.Is there any way to do this?#2016-12-1418:16bbloomjosesanch: i donā€™t really think spec wants to be used in that way dynamically. i think itā€™s somewhat designed to ensure some things are static at useful times.... can you do get-mandatory-fields in a more static way and then validate with a dynamic spec NAME rather than a dynamic spec?#2016-12-1418:17bbloomsomething like (s/explain-data (get-spec-name llambda) llambda))#2016-12-1418:17bbloomotherwise, your choices are eval (probably not a good idea) or macros (which are going to be static anyway)#2016-12-1418:18josesanchThank you @bbloom. I was thinking about that. In this case I'm validating each field individually#2016-12-1418:18bbloomin that case, i suggest using the per-field specs (which presumably already exist?) and a simple loop or map or reduce on top#2016-12-1418:19bbloomie do custom validation in addition to the spec validation#2016-12-1418:19bbloombut others may have other suggestions ĀÆ\(惄)/ĀÆ#2016-12-1418:21josesanchYes. This way. Thank you very much.#2016-12-1418:22bbloomnice. this is one of the great features of spec vs a traditional type system: you can use it as a building block for other dynamic validations#2016-12-1418:28Alex Miller (Clojure team)also, (s/keys) will validate all keys#2016-12-1418:28Alex Miller (Clojure team)you could combine that with a check for whether a map contains all the required keys#2016-12-1418:29Alex Miller (Clojure team)that could be a custom predicate that built from a set of keys#2016-12-1418:30bbloomhm, are you saying like youā€™d make a wrapper object {:required #{::foo ::bar}, :object {::foo ā€¦ ::bar ā€¦}} and then validate that with a custom predicate?#2016-12-1418:31Alex Miller (Clojure team)no#2016-12-1418:32Alex Miller (Clojure team)Iā€™m saying you could (s/and (s/keys) #(every? req-keys-set (keys %)))#2016-12-1418:32Alex Miller (Clojure team)I probably typoed that, but you get the idea#2016-12-1418:34bbloomah ok#2016-12-1418:56ddellacosta@hiredman yeah I figured it out after a little whileā€¦PBKAC. Thanks!#2016-12-1520:01sveriI wonder if there is a way to make sense of such a spec message: https://pastebin.com/QrU49Thr#2016-12-1520:02sveriits hard to figure out if I am just missing a key somewhere or if the structure in general (coll-of (coll-of ...)) is broken#2016-12-1520:21Alex Miller (Clojure team)I think there might be more than one thing wrong too#2016-12-1520:22Alex Miller (Clojure team)this: val: 16 fails spec: :de.sveri.getless.service.off/id at: [:args :foods :id] predicate: string? seems pretty straightforward#2016-12-1520:22Alex Miller (Clojure team)the :id key is an int not a string#2016-12-1520:24Alex Miller (Clojure team)the foods arg also seems to be missing some keys
:de.sveri.getless.service.off/product at: [:args :foods] predicate: (contains? % :image_small_url)
:de.sveri.getless.service.off/product at: [:args :foods] predicate: (contains? % :image_thumb_url)
:de.sveri.getless.service.off/product at: [:args :foods] predicate: (contains? % :lang)
:de.sveri.getless.service.off/product at: [:args :foods] predicate: (contains? % :code)
:de.sveri.getless.service.off/product at: [:args :foods] predicate: (contains? % :rev)
#2016-12-1520:24sveri@alexmiller Yea, thats right, uhm, ok, I am, surprised#2016-12-1520:24sveriCan you tell me how you extracted the important information so quick?#2016-12-1520:25Alex Miller (Clojure team)looking at the line breaks#2016-12-1520:25sveriIt took me a lot of time figuring out the first error, then reload and look for the second one. My problem is, its just a wall of text.#2016-12-1520:25Alex Miller (Clojure team)each line is a problem - those are the ends of the lines#2016-12-1520:25sveriI see, turning off soft wraps would have helped here.#2016-12-1520:25Alex Miller (Clojure team)the (large) data value is the distracting part in each line#2016-12-1520:26Alex Miller (Clojure team)you can install a custom explain printer too if you want (could actually hide the val, or limit itā€™s size)#2016-12-1520:27sveriThat sounds good, my application makes use of an external service, that returns a lot of data. Is there an example somewhere on how to make a custom printer?#2016-12-1520:27Alex Miller (Clojure team)
(set! s/*explain-out*
  (fn [explain-data]
    (binding [*print-length* 3] 
      (s/explain-printer explain-data))))
#2016-12-1520:27Alex Miller (Clojure team)*explain-out* is a dynamic variable holding the function that prints explain data#2016-12-1520:28Alex Miller (Clojure team)^^ that will work at the repl, but you might need to use with-bindings to make it work in your code#2016-12-1520:28Alex Miller (Clojure team)s/explain-printer is the default print function#2016-12-1520:29Alex Miller (Clojure team)this example limits *print-length* to 3 when printing big collections#2016-12-1520:29Alex Miller (Clojure team)(thatā€™s the most common source of large printed values)#2016-12-1520:29Alex Miller (Clojure team)for example (s/explain empty? (range 100)) with the above will not print the full seq#2016-12-1520:34sveriAwesome šŸ™‚ First question, may I put that on a gist with your comments and second: when I try your code it throws the following exception during runtime: java.lang.IllegalStateException: Can't change/establish root binding of: *explain-out* with set#2016-12-1520:49Alex Miller (Clojure team)Yes and that's what I mentioned #2016-12-1520:50Alex Miller (Clojure team)It works at the repl because the repl binds explain-out#2016-12-1520:50Alex Miller (Clojure team)But in your code you need to set it in a dynamic scope with something like binding#2016-12-1520:51Alex Miller (Clojure team)Or alter-var-root it#2016-12-1520:51Alex Miller (Clojure team)I should probably just write a quick blog on it#2016-12-1520:56sveri@alexmiller Great, I got it now šŸ™‚ thank you very much#2016-12-1600:49noprompti think thereā€™s a bug with :clojure.core.specs/binding-form. itā€™s defined as
(spec/or :sym :clojure.core.specs/local-name
         :seq :clojure.core.specs/seq-binding-form
         :map :clojure.core.specs/map-binding-form)
but this is a problem if you actually want to use the data from conforming against it because
(spec/conform
 :clojure.core.specs/binding-form
 '{})
;; => [:seq {}]
which is the smallest example that demonstrates the problem (`{:as m}` conforms to a [:seq ,,,] value). in order to fix this problem the spec needs to be reorganized to place :map spec above the :seqspec.
(spec/or :sym :clojure.core.specs/local-name
         :map :clojure.core.specs/map-binding-form
         :seq :clojure.core.specs/seq-binding-form)
hereā€™s an example of the updated behavior
(spec/conform
 (spec/or :sym :clojure.core.specs/local-name
          :map :clojure.core.specs/map-binding-form
          :seq :clojure.core.specs/seq-binding-form)
 '{})
;; => [:map {}]

(spec/conform
 (spec/or :sym :clojure.core.specs/local-name
          :map :clojure.core.specs/map-binding-form
          :seq :clojure.core.specs/seq-binding-form)
 '[])
;; => [:seq {}]
#2016-12-1600:53noprompti guess this is another good case for specs for specs. šŸ˜‰#2016-12-1600:53nopromptcc @alexmiller#2016-12-1600:54Alex Miller (Clojure team)There's already a ticket and a patch for this I think#2016-12-1600:56Alex Miller (Clojure team)http://dev.clojure.org/jira/browse/CLJ-2055#2016-12-1600:56Alex Miller (Clojure team)Same thing I think?#2016-12-1601:03nopromptah nice! yep, thatā€™ll do it too! šŸ™‚#2016-12-1601:03noprompt(although my patch would have been smaller) šŸ˜›#2016-12-1601:05noprompt(brandonā€™s is better though) šŸ™‚#2016-12-1601:11Alex Miller (Clojure team)We really need a vcat which would be better than either#2016-12-1601:14Alex Miller (Clojure team)In particular to c#2016-12-1601:14Alex Miller (Clojure team)Conform to a vector#2016-12-1615:59devthwhere does spec store registered specs? can i list / inspect them?#2016-12-1616:14bronsa@devth (s/registry)#2016-12-1616:15devth@bronsa thanks!#2016-12-1616:15Alex Miller (Clojure team)you might also find stest/instrumentable-syms or stest/checkable-syms of interest re functions#2016-12-1616:16devthah, very cool#2016-12-1700:52bbloomiā€™m trying to get a feel for when it makes sense to use multispec vs or/and/merge/etc.... at first i thought it might be the same tradeoff as with multimethods vs if/case/etc. ie: open vs closed primarilyā€¦ however iā€™m not quite sure what the generator behavior will be for and/or/merge w/o the retag function#2016-12-1700:53bbloomwhatā€™s folks experiences with small/closed alternatives? like say 3 cases where all of them share some ā€œbaseā€ fields and each has a couple unique fields#2016-12-1701:06bbloomi think in this case iā€™m looking at now, i can just ignore the ā€œbaseā€ notion, but not quite sure#2016-12-1705:00tjtoltonare videos from the clojure-spec workshop at the recent conj up online anywhere?#2016-12-1705:01tjtoltonthe datomic workshop videos are incredible, and depending on how my meeting on monday goes, might mean a new enterprise datomic customer!#2016-12-1713:07luxbockI started working on a convenience macro for defining a function spec and its definition within the same defn like macro#2016-12-1713:09luxbockand I'm trying to decide it it would make more sense to infer the spec of the arguments from the argument destructuring syntax, or to allow you to destructure the output of conform on the input parameters#2016-12-1713:11luxbockso an example of the first would be something like [{:keys-req [foo ::a-foo]}] (the argument vector of the function) expanding to (s/cat :thing-with-foo (s/keys :req [::a-foo])) in the :args part of the fdef form#2016-12-1713:15luxbockand the second one would look something like [({:keys [xxx yyy]} ::a-foo)] which binds xxx and yyy to be things pulled from (s/conform thing-which-is-a-foo) by wrapping the body of the function definition in a let form that does the destructuring for you#2016-12-1713:15luxbockI guess these things are not mutually exclusive, so I could just write two separate macros#2016-12-1713:17luxbockboth macros expand to (do (fdef ...) (defn ...)) at the top level#2016-12-1713:18luxbockmaybe I should postpone this until I have more experience with using spec the usual way#2016-12-1713:28luxbockI suppose you could even combine them by always grouping bindings and specs in parentheses, so you could do [{:keys-req [({:keys [xxx yyy]} ::a-foo)]}] which defines the :args as (s/cat :arg-a (s/keys :req [::a-foo])) and assuming ::foo is something like (s/cat :xxx int? :yyy int?) then you can refer to xxx and yyy as such inside the function body#2016-12-1713:30luxbockif anyone can think of reasons for why this is a bad idea that'd be great so that I won't have to discover this by trying šŸ™‚#2016-12-1715:58paytonrules@luxbock I would love to see this macro on github when itā€™s ready#2016-12-1718:50bbloombeen using update-in, get-in, etc happily for quite some time, but seeing such heavy usage of paths in spec makes me wish Rich would someday give a talk including a rant about xpath/xquery šŸ™‚#2016-12-1718:51Alex Miller (Clojure team)Why would he rant about that?#2016-12-1718:52bbloomi donā€™t know. i would just like to hear his insights/thoughts about it#2016-12-1718:52Alex Miller (Clojure team)I've done a bunch of xquery and I think it's a pretty reasonable functional query language with some good stuff in it
#2016-12-1718:52Alex Miller (Clojure team)Not all good of course, but nothing is#2016-12-1718:53bbloomyeah, itā€™s just sorta this forgotten/unloved old good idea to be able to talk about slices of hierarchical data#2016-12-1718:54Alex Miller (Clojure team)So you mean good rants :)#2016-12-1718:54bbloomthereā€™s stuff like lens in the haskell world and nathan marzā€™s specter thing (was that what itā€™s called?) but richā€™s take on these sorts of things is always so lucid#2016-12-1718:54robert-stuttafordRichā€™s recent NYC talk mentions a ā€˜path systemā€™ in spec. is there some place i can read about this?#2016-12-1718:54bbloomi enjoy a solid rant, so i donā€™t assign the negative connotation by default šŸ˜‰#2016-12-1718:54Alex Miller (Clojure team)It's just the idea that alternatives or components are tagged#2016-12-1718:54Alex Miller (Clojure team)So you can talk about them#2016-12-1718:55robert-stuttafordok, so one could form a path into a conformed structure using those names, got it#2016-12-1718:55robert-stuttafordas with get-in and friends#2016-12-1718:55bbloom@robert-stuttaford itā€™s not really a ā€œsystemā€ so much as it is two things: 1) a vector of keys a la get-in or update-in and 2) makign sure spec gives you good paths whenever youā€™d need it#2016-12-1718:55Alex Miller (Clojure team)Globally qualified spec name + path together let you talk about any piece#2016-12-1718:55robert-stuttafordyep, got it ā€” just wanted to check it wasnā€™t moreā€™n that#2016-12-1718:56robert-stuttafordhow stable is the API now? i took a look in the early alphas, and not since#2016-12-1718:56Alex Miller (Clojure team)Yeah it's used more in the second half of spec#2016-12-1718:56robert-stuttafordwondering if the waterā€™s warm enough now šŸ™‚#2016-12-1718:56Alex Miller (Clojure team)(That was a joke btw :)#2016-12-1718:56Alex Miller (Clojure team)I'd say the API is pretty stable#2016-12-1718:57Alex Miller (Clojure team)We are unhappy with the details of gen override and instrument options and those might change slightly #2016-12-1718:57Alex Miller (Clojure team)Unhappy with their inconsistency across fns#2016-12-1718:58robert-stuttafordand i believe you may also be providing exercise with gen overrides#2016-12-1718:58robert-stuttafordoh, wait, there it is#2016-12-1719:00bbloomiā€™ve got richā€™s lisp nyc talk on in the background, thatā€™s why i thought of the xpath thing#2016-12-1719:00bbloomi havenā€™t giving the generator side of spec a serious workout yet - probably should make a concerted effort to try#2016-12-1719:41bbloomIā€™m surprised to see that *print-namespace-maps* is a boolean value and not a number specifying how big of maps to scan in the private lift-ns function#2016-12-1719:42bbloomafter 1.9 ships iā€™ll want to add lifted namespace map syntax to fipp, but i want to preserve the linear time and constant space overhead promises#2016-12-1719:42bbloomsurprised to see no such guarantee in the default clojure repl - but then i guess printing large values is sorta bad interactively anyway#2016-12-1720:01dergutemoritzIs it intentional that there are no bigint? and biginteger? predicates + corresponding generators?#2016-12-1720:09gfredericksmight have to do with cross-platform issues#2016-12-1720:13dergutemoritz@gfredericks Hmm possibly... there is bigdec?, though#2016-12-1720:14gfredericksYeah I dunno. Can write your own though. #2016-12-1721:01Alex Miller (Clojure team)I think it was just an oversight and I might even have a ticket for it#2016-12-1723:06dergutemoritzOh yeah: http://dev.clojure.org/jira/browse/CLJ-1951#2016-12-1723:26Alex Miller (Clojure team)right! has some issues. Feel free to work on a patch! :)#2016-12-1723:29gfredericksI could probably do it#2016-12-1723:39gfredericks@alexmiller adding the generator in test.check could work too, right?#2016-12-1723:39gfredericksI guess it could start in clojure.spec and get pulled out if the timing works; users wouldn't need to care#2016-12-1723:48Alex Miller (Clojure team)Yup#2016-12-1812:52gfredericksI wrote a little essay about growth & shrinking of test.check generators: https://github.com/clojure/test.check/blob/master/doc/growth-and-shrinking.md#2016-12-1817:33mfikesIs it considered bad practice to employ custom generators to produce values that lead to decent perf of the functions being checked? Motivation: You have a function which, for certain valid inputs, would slow down checking considerably. Would it be considered an abuse of s/with-gen for this purpose, or is there a better way, or is this still an open question?#2016-12-1818:15mfikesA concrete hypothetical example: Testing the 2-arg arity of repeat. You want the spec to allow any number for n, but you want to limit the size of the generated values of n, just for testing purposes.#2016-12-1818:27dergutemoritz@mfikes Seems like the essay @gfredericks just posted here is relevant to your question: https://github.com/clojure/test.check/blob/master/doc/growth-and-shrinking.md -- it only hints at sizing being relevant for performance (memory usage) but in general I think it makes sense to apply custom generators for this purpose. I don't have authority on blessing it as a good practice, though šŸ™‚#2016-12-1819:58Alex Miller (Clojure team)Seems fine to me#2016-12-1819:59Alex Miller (Clojure team)You can also supply the generators at test time rather than in the registered spec if you want smaller scope#2016-12-1819:59Alex Miller (Clojure team)With gen overrides#2016-12-1820:09gfredericks:+1: to all that#2016-12-1918:26naomarikI recall reading advising caution on using conform awhile back. Is this still the case?#2016-12-1918:29bbloomare you sure it wasnā€™t caution on using conformer?#2016-12-1918:30naomariki canā€™t be sure#2016-12-1918:30naomarikwhatā€™s the downside in any case?#2016-12-1918:31naomarikfor conformer, if thatā€™s what it is#2016-12-1918:32bbloomconformer enables your specs to do a little bit of data coercion#2016-12-1918:33bbloomsometimes thatā€™s useful to make a spec work, but itā€™s usually better to do that work in a function rather than a spec#2016-12-1918:33bbloomie just translate the data and spec the data that exists before and/or after that translation#2016-12-1918:33bbloomthatā€™s my best attempt as a short explanation, but others here can probably elaborate#2016-12-1918:34naomarikokay cool, so thereā€™s no problem with conform then in both the latest cljs and clojure right?#2016-12-1918:34bbloomyeah, definitely not - conform is a non-trivial part of the utility of spec šŸ˜‰#2016-12-1918:35naomarikperfect, gonna save me a non-trivial amount of time writing destructuring branches on my multispecs šŸ˜‰#2016-12-1918:41seancorfieldParaphrasing Alex: if your spec also coerces data, bear in mind that this coercion will apply to all clients of that spec.#2016-12-1918:41seancorfieldAt World Singles, we have a lot of specs that coerce string data to typed data. Weā€™re OK with that šŸ™‚#2016-12-1918:42seancorfieldSpecifically, we have specs for our API calls that accepts strings as input and the result of conforming them is domain model values (so string -> long in some cases and string -> keyword in others).#2016-12-1918:44seancorfieldWe generally write the specs for the domain model values first, and then write the API specs in terms of those (so an API spec is (s/and string? (s/conformer some-coercion) ::domain/spec-name) and we have a generator that fmaps the reverse coercion over the ::domain/spec-name (either str or name usually).#2016-12-1918:45seancorfieldWe have some shorthand coercion specs thoā€™ (e.g., ->long which does (try (Long/parseLong s) (catch Exception _ ::s/invalid))).#2016-12-1918:53naomarikaha#2016-12-1918:53naomariki guess itā€™s extremely helpful when dealing with 3rd party APIs#2016-12-1918:55bbloomi think what seancofield is saying, and by extension alex, is that conformer couples some interpretation of the data to the specification - however somebody else may want a different interpretation. so if you choose to use conformer, you prevent yourself and others from being able to apply a different interpretation#2016-12-1918:55naomarikis this the reason to use caution and not that itā€™s in alpha?#2016-12-1918:55bbloomif you donā€™t expect to ever need another interpretation, then go right ahead, but just know youā€™re making that trade off#2016-12-1918:56bbloomfor conformer specifically, yes#2016-12-1918:56naomarikperfect thanks#2016-12-1918:56bbloomseancorfield: the other option would be to not use conformer and then to do a secondary pass where you apply coercions, right?#2016-12-1918:57bbloomiā€™ve been hacking on this thing: https://github.com/brandonbloom/ambiparse and i made a decision up front to include semantic rules as a primary feature - itā€™s basically the same idea as conformers#2016-12-1918:57bbloombut parsing is different than validation - in parsing, youā€™re pretty certain to have a lot of ā€œtriviaā€ like whitespace etc that you want to discard#2016-12-1919:02seancorfield@bbloom My objection to that is that you end up doing the work twice: once to determine whether a value is valid and then again to actually convert it ā€” since you always want to convert it if it is valid.#2016-12-1919:03seancorfieldI mean, you could use a regex I suppose to say ā€œItā€™s valid if itā€™s just digits and not longer than Nā€ but thatā€™s not the same as actually parsing a string to get a number ā€” youā€™ll either allow too many digits or not enough.#2016-12-1919:03seancorfieldAnd if you decide to use a regex, now you have two problems šŸ™‚#2016-12-1919:03bbloomsure, the guidance is essentially ā€œconsider if you might want to NOT convert it in the futureā€ - and if the answer is ā€œnah, iā€™ll never need not do thatā€ then go right ahead#2016-12-1919:03bbloomthereā€™s sorta a 3rd approach#2016-12-1919:04bbloomwhich is what instaparse does: allow you to specify rewrite rules independently#2016-12-1919:04seancorfieldAnd itā€™s easy to write a non-coercing version of these specs: you just call str or name on the conformed result šŸ™‚ (s/and ::api-spec/spec-name (s/conformer str)) šŸ™‚#2016-12-1919:04bbloomso like youā€™d have a map of specs to post-conformers#2016-12-1919:32naomarikthe amazement of discovering exercise is like using the repl for the first time#2016-12-1919:36naomarikis there a way to pprint the output though?#2016-12-1919:47bbloomyou can just call clojure.pprint/pprint#2016-12-1919:47bbloomor you can use something like https://github.com/greglook/whidbey#2016-12-1919:49jrhttps://clojure.github.io/clojure/clojure.pprint-api.html#clojure.pprint/pp#2016-12-1919:50naomarikyeah i think itā€™s fine, itā€™s just pulling out the common namespace part of the keywords but I guess this is normal behavior for pprint#2016-12-1919:51naomariklike [#:firstpart {:next ā€œblah"}] for things that are :firstpart/next#2016-12-1919:55richiardiandrea@naomarik yes that's new syntax for namespaced keys in maps (namespaced maps?), I think it has been added in the first 1.9-alphas. They will all print that way#2016-12-1919:57richiardiandreaah ok the bot is not on 1.9 yet#2016-12-1919:57richiardiandrea
(get #:test{:foo 1 :bar 2} :test/foo)
1
#2016-12-1920:00naomarikahh okay#2016-12-1921:24kyle_schmidtDo specs belong with my application code or should I put them in their own namespace? Also, should I assert within application code or use the :pre and :post keywords to check my function i/o?#2016-12-1921:24bbloomi asked the same question about where to put specs - it seems like both options occasionally make sense#2016-12-1921:25bbloomit seems like if you have a single-namespace that wants to export some specs, go right ahead and put them inline#2016-12-1921:25bbloomif you have a bunch of namespaces collaborating around some common stuff, maybe put the specs in their own namespace#2016-12-1921:25bbloomstart single-file and go multi-file if you need to#2016-12-1921:26bbloomthe only decision you really need to make up front is what namespace you want them to land in in terms of usage externally#2016-12-1921:26bbloomthereā€™s also a risk of a circular dependency between the spec namespace and the other namespaces#2016-12-1921:36seancorfield@kyle_schmidt As for assert / :pre / :post ā€” ā€œit dependsā€.#2016-12-1921:38seancorfieldI would normally turn on instrumentation as part of my tests, and have certain tests that run check on key functions (ones that can be generatively tested). I generally do not have assert in any production code and only very rarely have :pre or :post.#2016-12-1921:39kyle_schmidtThank you for the replies. And just to make sure, is clojure.spec a replacement for clojure.test? Or would I still want to write both?#2016-12-1921:42zaneIn my experience in practice you wind up writing both.#2016-12-1921:43bbloomspec is definitely not a replacement for test#2016-12-1921:43bbloomyou use your specs from tests#2016-12-1921:44bbloomat least thatā€™s one use of spec#2016-12-1921:44kyle_schmidtoh interesting. so use specs to aid tests#2016-12-1921:45zaneAh, I interpreted that question as being whether clojure.test.check replaces clojure.test.#2016-12-1921:45bbloom@kyle_schmidt: thereā€™s a bunch of spec videos by stu halloway on youtube that are a good intro of actually doing some testing w/ spec#2016-12-1921:46kyle_schmidtcool thanks Iā€™ll check them out!#2016-12-1922:45lambdahandsIs there a way to spec out a function as part of a collection, such as a map? For example, if I define a spec like:
(s/def ::callback (s/fspec :args (s/cat :x number?) :ret number?))

(s/def ::foo (s/keys :req-un [::callback]))
Calling (s/valid? ::foo {:callback inc}) returns false
#2016-12-1922:47lambdahandsI think in practice, I would typically just spec out the function I was using as the callback itself, and spec the key as just fn?. It seems simpler, but I wasn't sure if I was thinking about this correctly#2016-12-1922:49bfabry@lambdahands I get true for that valid? call#2016-12-1922:50Alex Miller (Clojure team)yeah, I would expect that to work#2016-12-1922:50lambdahandsHmm. I am using clojure-future-spec ā€“ let me try with the latest alpha as well.#2016-12-1922:50Alex Miller (Clojure team)(also ifn? is usually better than fn?)#2016-12-1922:50lambdahandsAh, thanks for the tip @alexmiller!#2016-12-1922:51Alex Miller (Clojure team)@naomarik you can also turn off namespace map syntax for maps if you want (set! *print-namespace-maps* false)#2016-12-1922:52Alex Miller (Clojure team)it will default to false in Clojure but true in the repl#2016-12-1922:56lambdahandsChecked out that spec again, and it does work as you said. Strange, I must have been doing something wrong earlier as I was getting an unexpected result.#2016-12-2003:01gdeer81this spec (s/def ::string-num (s/and string? #(not= nil (re-find #"^[0-9]+$" %)))) is unsatisfiable to the generator. I wanted to make sure there wasn't a better way to spec out a string of numbers that could be satisfied before I wrote a custom generator#2016-12-2008:24artemyarulinHello. Is there a way to create connected spec generators? I have
(s/def ::user #{ā€œuser1ā€ ā€œuser2ā€})
(s/def ::pwd #{ā€œpwd1ā€ ā€œpwd2ā€})
(s/def ::acc (s/keys :req [::user ::pwd]))
I want generator that will returns me accounts like {::user ā€œuser1ā€ ::pwd ā€œpwd1ā€}, but with current setup it will return random user and pwd. I wonder - is there any way to somehow connect specs to generate valid accs for my context?
#2016-12-2012:20gfredericks@artemyarulin is this not what you're seeing, or else what do you mean by "valid accs"? https://www.refheap.com/124315#2016-12-2012:23artemyarulin@gfredericks Well I meant that valid accounts are pairs of user1 pwd1 and user2 pwd2 and not user1 pwd2 with such data. It was solved like this:
(def credentials {"user1" "pwd1" "user2" "pwd2"})
(s/def ::acc (-> credentials seq set))
(s/def ::user (-> credentials keys set)})
(s/def ::pwd (-> credentials vals set)})
#2016-12-2012:25gfredericksoh; this seems like a strange way to make specs for user data#2016-12-2012:25gfredericksbut okay#2016-12-2012:30robert-stuttafordcool that itā€™s possible šŸ™‚#2016-12-2013:22Alex Miller (Clojure team)@artemyarulin: there is a screencast about "modeling" your data in creating generators#2016-12-2013:23Alex Miller (Clojure team)https://m.youtube.com/watch?v=WoFkhE92fqc#2016-12-2013:24artemyarulin@alexmiller thanks, Iā€™ll have a look#2016-12-2014:12yogidevbearHi everyone. I bought myself Living Clojure for a Christmas present (5 more sleeps to go!). Can anyone recommend a good "beginners" intro course to spec (as this isn't included in the book)? (Other than the standard docs)#2016-12-2014:13yogidevbearI'm hoping to find something very beginner-friendly#2016-12-2014:14yogidevbearThanks in advance šŸ˜„ #2016-12-2014:49tjg@yogidevbear I havenā€™t watched this myself, but I like Lambda Island in general, and their intro to spec is a freebie: https://lambdaisland.com/episodes/clojure-spec#2016-12-2014:57mpenet@artemyarulin seems like a s/and with the part containing the s/keys you're interested in with it's own gen might fit better imho#2016-12-2014:58mpenetI had to do something like this for user profile data that had to return valid/realistic data (gender matching firstname, profile pic matching "fake" user etc)#2016-12-2015:00mpenetin short, check out :gen in s/keys#2016-12-2015:01mpenet./back to xmas stuff#2016-12-2016:49joshjones@yogidevbear This is not a guide, but itā€™s been the most helpful for me to think of it this way ā€” the question that helps get things going is: What am I trying to spec? So, I wrote this out in a table on my board, and itā€™s helped me immensely. First of all, remember that for all intents and purposes, every predicate is a spec. It answers yes or no when given something. If itā€™s an associative structure, like a map, youā€™ll use keys, map-of, and merge. If itā€™s ā€œdataā€, you will use a set as a predicate (`#{}`), your own functions, built-in predicates like string?, pos-int?, and so on, or for ranges, int-of, double-of, and inst-of (for dates) If itā€™s a sequential structure (lists, vectors), youā€™ll use coll-of (where every element is validated by the same predicate), tuple (where each element may have a different predicate), or the regex operations which can be used to combine patterns of specs within the sequence: cat, *, +, ?, alt, keys*, &, and spec (for nesting structures) You will want to fdef some function specs as well. So, for me the hard part initially was taking this huge mass of functionality and finding ā€œwhere do I start?ā€ The above is a very rough, incomplete guide to what you should be using in the spec library to get started, based on what you specifically are trying to spec. Testing, instrumenting, generating data is another thing, but first learn the above and I think it will not be hard to transition from there.#2016-12-2017:41mpenetto expand on the related key/val before, this ends up something like this:
(s/def ::user string?)
(s/def ::pwd string?)
(s/def ::acc (s/keys :req-un [::user ::pwd] :gen
                     #(gen/fmap first (gen/shuffle [{:user "user1" :pwd "pwd1"}
                                                    {:user "user2" :pwd "pwd2"}
                                                    {:user "user3" :pwd "pwd3"}]))))
at least the gen and spec parts are separated this way
#2016-12-2017:47mpenetyou can do the same with gen/let or gen/one-of + 1 gen/return per pair#2016-12-2021:04yogidevbearThanks @tjg and @joshjones :+1: @joshjones I think I'm going to have to save your response somewhere so I can digest it further down the line once I'm a bit more familiar with Clojure and some of the terms you're using šŸ˜‰ #2016-12-2022:24jiangtsiā€™m a bit of a newcomer to spec, but when defining specs can we use other specs instead of predicates? Iā€™m testing in CLJS and it seems that itā€™s allowed, but then exercise on coll-of seems to break.#2016-12-2022:25jiangtsThe following is throwing an error in ClojureScript 1.9.293
(s/def ::a nat-int?)
(s/def ::many-as (s/coll-of ::a :kind set?))
(s/exercise ::many-as)
#2016-12-2022:28jiangtsah, checking a bit closer it seems that specifying :kind set? does not work#2016-12-2022:30gdeer81āœ… I spec'd out a function that takes a set and makes a power set, wanted to know if I'm doing this right https://www.refheap.com/124325#2016-12-2022:40gdeer81the funny thing is when you write an :fn spec for a function, now you have two functions to debug.#2016-12-2022:43bsimahm, spec can't capture side-effects, can it?#2016-12-2022:46gdeer81like if you had a function that fired rockets into space and you wanted test.check to be able to fail a case where there weren't rockets in the air?#2016-12-2022:47bsimayeah, like there's no way to capture rockets-fired = true in the spec definition#2016-12-2022:48bsimawhich makes sense, b/c spec isn't trying to be a type system like Haskell's or anything.#2016-12-2022:50gdeer81unless your function returned a map like {:num-rockets 5 :target "moon" :fired true} but then its actually your function that captured the side effect#2016-12-2022:50bsimayeah good point#2016-12-2022:55joshjones@bsima you cannot capture effects outside a function (AFAIK), but you can capture the return value and its relationship with the args in the :fn portion of the fdef of course#2016-12-2022:57joshjonesI suppose you could use a :postcondition in the definition of the function, or check some outside state inside the function, since you have access to the state anyway.#2016-12-2023:10gdeer81@bsima I spec'd out a function that updates a global atom https://www.refheap.com/124326 doesn't smell right#2016-12-2023:17joshjonesSince state is involved, there's no (good) way to avoid race conditions when verifying that the updated value is correct. Code elsewhere could have modified the atom.#2016-12-2023:20gdeer81also, the generator is pretty abusive and will throw large numbers at it until it breaks šŸ˜†#2016-12-2023:29gdeer81@joshjones well the scary thing is that it has passed test.check over 3k times now. I guess I'd have to use pmap to see if I can ferret out race conditions#2016-12-2023:35joshjonesi am guessing check runs the function sequentially, so no danger of seeing race conditions in this case#2016-12-2100:00bsimagdeer81 that's cool#2016-12-2100:01gdeer81I also used pmap to run it over a vector of 99,999 numbers, still no failures#2016-12-2100:02bsimamaybe you could use the funcool "cats" library to handle side effects, and check for right/`left` return types in the spec? idk, haven't used it much#2016-12-2107:24jiangtsusing conformer, wondering how to conform a given input efficiently without conforming then unforming#2016-12-2108:57dergutemoritz@jiangts Can you give an example or elaborate your problem some more? I don't see why you'd need to conform and unform using conformer#2016-12-2109:10jiangts@dergutemoritz when you coerce with just conform you get a parsed version of the data, including the paths on ors or keys for example. Iā€™d like a coerced version of the data without those key paths, which can be removed with unform#2016-12-2109:10dergutemoritzAh, I see#2016-12-2109:12dergutemoritzThere are multiple ways around that. For example, in case of s/or, something like this will do: (s/and (s/or ...) (s/conformer val))#2016-12-2109:12dergutemoritzFor s/cat, often if you don't care about the path labels, you are probably looking for s/tuple instead#2016-12-2109:14dergutemoritzBut perhaps you should think about whether keeping the path labels is actually the better idea!#2016-12-2119:47ag#2016-12-2119:58gdeer81does it make sense to have a separate file for general specs and then inline specs that are specific to that namespace?#2016-12-2120:05gfredericks@ag (gen/let [gen'd-accounts (-> :account/spec s/gen (gen/vector (count accounts)))] (map merge gen'd-accounts accounts))#2016-12-2120:13ag@gfredericks hmmm, mkay#2016-12-2121:46Oliver GeorgeHas anyone written a destructuring to spec translator? Idle thought really. I was wondering about how to spec namespaces with less overhead... a convention based around arglist destructuring and using variable names which match registered specs could work. e.g.
(s/def ::base-url string?)
(s/def ::request (s/keys :req-un [::url ::params]))
(s/def ::page-limit pos-int?)
(s/def ::page-count pos-int?)

(x/defn load-options
  [{:keys [base-url]} [_ request page-limit page-count]]
  ...
#2016-12-2121:47Oliver GeorgeAlso, merry xmas everyone.#2016-12-2122:15Oliver GeorgeI guess the same could be done after the fact by referring to the fn :arglists metadata.#2016-12-2122:46Oliver GeorgeNoted here for posterity: https://gist.github.com/olivergeorge/85c2b263227676324275ce13408c0fb8#2016-12-2122:47Oliver GeorgeFeel free to comment there if it's a terrible idea or I'm missing key points.#2016-12-2122:53Alex Miller (Clojure team)for perf reasons, we would not do that in core#2016-12-2123:13bsimais there a way to write an fdef that says "this function should take :my/spec as it's argument but only after the arg has conformed with (conform :my/spec arg)" ??#2016-12-2123:17Oliver George@alexmiller that sounds sensible. hard to imagine it being unopinionated enough to be suitable as a core feature too. Do you think the idea has merit? I guess the real benefit is tidier code which is superficial.#2016-12-2123:18Oliver George@alexmiller there was talk of a "dev version" of clojure with more spec integration... less performant but more errors for the developer. Is that still on the cards?#2016-12-2123:36Alex Miller (Clojure team)@olivergeorge: seems like most cases that's not what I would want #2016-12-2123:37Alex Miller (Clojure team)On the dev version, possibly although just not front burner right now #2016-12-2200:10Oliver GeorgeThanks Alex. Enjoy the break.#2016-12-2200:17naomarikWas wondering, is it possible given two different representations of data to conform to the same thing?#2016-12-2200:20naomarikUse case would be for instance taking {:key/id {:some :data}} and turning that into {:key/id id :some :data} given either format.#2016-12-2200:27naomarikthis seems like a silly question, doesnā€™t make sense other than having to transform the data after the fact.#2016-12-2201:20naomarikIā€™m unable to figure this out: given example in Richā€™s talk if you have a map of email keys and people, how do you spec that? example {ā€#2016-12-2201:21gfredericks(s/map-of ::email ::person) I believe#2016-12-2201:22naomarikhehe i just found that after failing with s/cat and s/tuple#2016-12-2201:22naomarikthanks šŸ˜‰#2016-12-2203:35Alex Miller (Clojure team)Well you can do it with s/every and s/tuple (which is how map-of is implemented)#2016-12-2211:02luxbockwas there any reason to break the symmetry of symbols always introducing themselves in destructuring syntax by using {:keys [:foo/bar]} instead of {:keys [foo/bar]}?#2016-12-2211:38luxbock@olivergeorge I added a comment to your gist with some ideas I've been working on#2016-12-2211:38luxbockhttps://gist.github.com/olivergeorge/85c2b263227676324275ce13408c0fb8 in case anyone else wants to have a look as well#2016-12-2213:13joost-diepenmaat@luxbock isnā€™t {:keys [:foo/bar]} for aliased namespaces and {:keys [foo/bar]} for unaliased ns? So foo is an alias in the first expression and a full namespace in the second?#2016-12-2213:40Alex Miller (Clojure team)No, those mean the same thing#2016-12-2213:41Alex Miller (Clojure team)You can use the first with ::foo/bar though for aliased#2016-12-2214:04luxbockoh I did misunderstand how it works, I thought {:keys [foo/bar]} would give me foo/bar but just bar works, makes sense#2016-12-2215:12sveriAfter using spec for a while now and especially heavily the fdefs I just noticed something important for me. What I did up to now, was to spec out the parameter maps for every function extensively. While now I see that it is more practical and enough for me to just spec out endpoints (like database / rest) and spec my functions in a different way for maps. What I do now, is to just add the keys to the spec that I require in that function instead of speccing the whole map. So (s/cat :product ::product-ns/product becomes (s/cat :product (s/keys ::req-un [::product-ns/date-key] Just wanted to share this šŸ™‚#2016-12-2215:59Alex Miller (Clojure team):+1:#2016-12-2218:27bbloomwhatā€™s the guidance for spec-ing pseudo-namespaced keywords? iā€™m thinking like datomic attributes when you do :customer/name or something like that#2016-12-2218:30Alex Miller (Clojure team)do you mean ā€œqualified but not referring to an actual namespaceā€ keywords?#2016-12-2218:30bbloomyeah#2016-12-2218:30Alex Miller (Clojure team)go for it#2016-12-2218:30bbloomeven when the qualifier is not at all unique?#2016-12-2218:30Alex Miller (Clojure team)but if you plan to ever share that code with other people, use adequately unique qualifiers#2016-12-2218:30bbloompresumably specā€™ing something like :event/type may conflict#2016-12-2218:30Alex Miller (Clojure team)yes#2016-12-2218:30bbloomyeah - ok so itā€™s the app vs lib thing#2016-12-2218:31Alex Miller (Clojure team)yeah#2016-12-2218:31bbloomok - fair enough#2016-12-2218:31bbloomthanks#2016-12-2218:31Alex Miller (Clojure team)I just wrote up a side bar for this in Programming Clojure :)#2016-12-2218:31bbloomyeah, i could sense that - iā€™m awesome like that#2016-12-2218:31bbloomjust helping you test out your advice#2016-12-2218:32Alex Miller (Clojure team)well usually people ask about the things I havenā€™t yet written :)#2016-12-2218:32bbloomuntil you write too much and then everybody decides it is easier to ask you than it is to read the mountain of writings! heh#2016-12-2218:34Alex Miller (Clojure team)itā€™s always simultaneously not enough and too much#2016-12-2218:35bbloomdamn tradeoffs strike again#2016-12-2219:54nopromptiā€™ve grown tired of writing common :pre and :post conditions and calling s/assert everywhere so i wrote a macro to help me with that šŸ™‚ https://gist.github.com/noprompt/ae3cf5f174a11a61a41e1bdfa96bde28#2016-12-2219:54noprompt(example down at the bottom)#2016-12-2219:55nopromptit needs just a little more polish but iā€™ve been happy with it.#2016-12-2219:58noprompti put it to use while implementing the CEK, CESK, and CESK* abstract machines for interpreting the pure lambda calculus and itā€™s been helpful.#2016-12-2219:58nopromptspared my sanity a bit and help catch a few bugs where i was wiring things up incorrectly.#2016-12-2219:59nopromptiā€™m not sure if itā€™s worth a full blown library or not but i thought iā€™d share this to show folks something interesting.#2016-12-2219:59Yehonathan Sharvitwhere is the code of the pure lambda calculus interpreter?#2016-12-2220:00nopromptiā€™ll gist CEK#2016-12-2220:01bbloomside note: implementing /CES?K/ machines is crazy fun#2016-12-2220:01bbloomhighly recommended to all#2016-12-2220:02noprompt@viebel i added it as a comment to the gist.#2016-12-2220:03noprompt@bbloom it really is. a co-worker and i have been taking a deep dive in to this stuff. itā€™s a ton of fun.#2016-12-2220:04noprompti havenā€™t gotten up to the timestamped or garbage collected implementations yet though.#2016-12-2220:04nopromptitā€™s mostly a matter of reading and converting the math into code.#2016-12-2220:06Yehonathan Sharvitthx @noprompt. You should write an interactive blog post about CEK stuff:clap:#2016-12-2220:07nopromptanyway, the basic premise of the macro is that we often use symbols in function parameters to stand for something and we tend to reuse them in several functions.#2016-12-2220:08nopromptthe idea here is to specify what those symbols mean and have a macro which handles the business of creating an fdef, instrumenting, etc.#2016-12-2220:09noprompttake @bbloomā€™s GLL parser code for example; heā€™s got a glossary at the top of one of the namespaces which explains what each of the symbols mean (a good practice btw).#2016-12-2220:13noprompt@viebel i might consider doing that during my vacation.#2016-12-2220:15nopromptugh, i just realized i made a mistake specifying ::lam!#2016-12-2220:26bbloom@noprompt fun to know somebody is reading that code šŸ™‚#2016-12-2220:27bbloomi have some ideas to dramatically simplify and speed it up too - but have to find a few days to work on it#2016-12-2220:33noprompt@bbloom i plan to look at it, and the papers, more seriously soon. i implemented a parser combinator library not to long ago and i had to ā€œcheatā€ in several places to squeeze out more perf.#2016-12-2220:45bbloommsg me directly if you wanna chat about it - coding this thing gave me a ton of insights#2016-12-2220:57manutter51@bbloom I never heard of CES?K machines, do you have a link I could start with?#2016-12-2220:59bbloommatt mightā€™s blog is where to go#2016-12-2220:59bbloomheā€™s got various articles on them#2016-12-2221:00manutter51Thanks#2016-12-2221:32dottedmagI wrote a stateful transducer (I know, I know, it's a variation on a partition theme, so it has to have some state). How do I spec it? I can do it for every arity, but the resulting specs aren't that useful -- I'd better have a spec for the transducing process, not for a single invocation.#2016-12-2221:54noprompt@manutter51 https://arxiv.org/pdf/1007.4446.pdf#2016-12-2302:40sophiagohi. i took about a month off working on this library i was specing out and now figured it's a good time to come around and finish it since i'm going to be refactoring a bunch of things and would rather not have to end up manually debugging them. the last thing i was stuck on was defining specs for custom types from defrecords. i was trying something like this:
(s/spec ::rational? #(= Rational (type % )))
but that's obviously not correct...
#2016-12-2303:33seancorfieldinstance?#2016-12-2303:35seancorfieldI.e., (s/spec ::rational #(instance? % Rational))#2016-12-2303:36seancorfieldI don't think it's idiomatic to use ? on a spec name (just on a predicate).#2016-12-2303:37seancorfield? usually implies a Boolean result. #2016-12-2303:37seancorfield(sorry for briefness - on my phone) ^ @sophiago #2016-12-2303:43sophiagothanks @seancorfield! i'll give it a try#2016-12-2303:55sophiago@seancorfield that's not compiling:
No value supplied for key: (fn* [p1__302#] (instance? p1__302# Rational))
#2016-12-2303:57sophiagoit seems you had the arguments to instance reversed, but i get the same stack trace regardless#2016-12-2304:08sophiagoto provide some background on what i'm trying to test here: i have five binary functions defined in protocols over four types and want to enumerate tests on all possible combinations of types as parameters. i figure i'll just focus on getting that right for now, but the the tricky part is i then need to include certain subtyping relations between them (shouldn't be super complicated) and also make sure i have tests where the results are of each possible type to check the coercion function is working (a bit trickier). i think i should be able to build up the subtypes manually once i can figure out how to define basic ones based on defrecords, seeing as they obey certain numerical laws. and then i think i should just be able to write fdefs for each of the five functions where i provide them as a list of inputs to each parameter and then maybe use a cond to determine what the possible return values are given each pairing. the final part i suppose is making sure exercise generates a certain number of tests for each result, which i haven't really thought about yet. hope that makes sense#2016-12-2304:10joshjonesfwiw regarding the previous part of the conversation, there's already a predicate function called rational? so no need to spec it, it's already there#2016-12-2304:11sophiagobut does that pertain to Java ratios? i'm referring to a record called Rational i defined myself#2016-12-2304:12joshjonesah, i see .. nvmd šŸ™‚#2016-12-2304:13joshjonesi am guessing you've explored multi-spec and that it's not applicable for your use case?#2016-12-2304:13sophiagothanks, tho...i didn't know they supported that. last i heard there was just int? and float? so seems odd to have rational? given the level of specificity and little use that type gets#2016-12-2304:14sophiagono, i haven't looked at multi-spec#2016-12-2304:14joshjonesthere are several new predicates in 1.9, such as nat-int?, pos-int? .. all very good set of built-in "specs"#2016-12-2304:15sophiagoah cool#2016-12-2304:15joshjonesthe great thing about spec is that any predicate is a spec, so clojure already has tons of them built right in#2016-12-2304:16joshjonesI have not had a need for multi-spec yet but it is related to spec-ing based on types; I suggest looking at that section in the spec guide and perhaps it may be of some use for your use case.#2016-12-2304:16sophiagoyeah actually i am familiar with muli-spec and it may make sense when i get to enumerating the pairings rather than using fdefs, but don't see how it would help with just testing for types#2016-12-2304:17joshjonesoh well, it was worth a shot šŸ™‚#2016-12-2304:19sophiagothe guide says it's for defining things like types within spec, which makes sense, but as far as just speccing out an existing type the suggestion to use instance? seems like it should work...#2016-12-2304:21joshjones
(s/def ::mystring #(instance? String %))
(s/valid? ::mystring "asdf")
#2016-12-2304:21joshjonesworks for me?#2016-12-2304:21sophiagoyes, it should be exactly like that#2016-12-2304:22sophiagoexcept that String is a built-in Clojure type#2016-12-2304:22seancorfield
boot.user=> (defrecord Rational [a b])
boot.user.Rational
boot.user=> (require '[clojure.spec :as s])
nil
boot.user=> (s/def ::rational #(instance? Rational %))
:boot.user/rational
boot.user=> (s/conform ::rational (->Rational 1 2))
#2016-12-2304:23seancorfieldAnd
boot.user=> (s/explain ::rational (/ 1 2))
val: 1/2 fails spec: :boot.user/rational predicate: (instance? boot.user.Rational %)
nil
boot.user=>
as expected.
#2016-12-2304:23sophiagoah! so sorry. i just realized i was using spec instead of def#2016-12-2304:24sophiagoas mentioned...it's been a month#2016-12-2304:28sophiagook, hoping this gives me enough to work with until i come back tomorrow with another level of broken šŸ™‚#2016-12-2304:29sophiagoi.e. from here it should be trivial to define subtypes using selectors already have. not sure about matching return values in my fdefs tho...#2016-12-2305:22weiis there a way to use spec that works with both maps and datomic's entitymaps?#2016-12-2305:26bbloomthere may be something better, but you can use associative?#2016-12-2305:26bbloomnote that will also find vectors#2016-12-2305:27bbloomso maybe #(and (not (vector? %)) (associative? %)) is passable, but hacky#2016-12-2308:41seancorfieldMy initial reaction is you're way-overspecifying things... maybe...#2016-12-2308:44seancorfieldOne simplification would be to have a predicate for int or Rational and use that for the numer/denom and real-part/imag-part -- that would cut a lot out of it.#2016-12-2308:56sophiago@seancorfield yeah, that's one way to think about it. like maybe spec just isn't meant to test for this level of correctness? like whether my coercion functions are reducing values correctly based on the composition of their inputs and outputs. i just wasn't sure if there was a way to automate some of the pairing? otherwise i'm afraid i could be left with something that's not testing that much...#2016-12-2308:59sophiagofor example, i currently have known bugs with subtyping. hence why i want tests for those. that's easy on its own. but the other issue is correctness of coercion up and down the numerical tower and i'm not sure there's any way to test that (that actually makes sense...that is) since it depends on the actual values and calls to gcd#2016-12-2309:24sophiago@seancorfield i guess on the one hand, i'm not quite sure what you mean by how i could simply it by using predicates for int and Rational? and otoh, defining the possible types was rather easy even if it does seem like overkill... where i'm stuck is if there's a way to at least partially automate testing type coercion by matching input values to return types. here's the last version of the full code if it helps explain what i'm talking about: https://github.com/Sophia-Gold/Symbolic-Algebra.clj/blob/master/src/symbolic_algebra/core.clj#2016-12-2309:28sophiagooh wait, Sean...you're the one who wrote that old numeric tower library! funny i'm asking you about this then. i've looked at that code a bunch. this started out as intending to be a Scheme-style numeric tower, but obviously with the emphasis on types rather than functions#2016-12-2313:41Yehonathan SharvitWhat is the idiomatic way to define a spec for a collection of numbers not including Double/NaN?#2016-12-2313:43Alex Miller (Clojure team)double-in can be used to spec doubles not including NaN - that can be combined with others#2016-12-2313:44Alex Miller (Clojure team)Or spec something generic and add a predicate to exclude NaN (but use Double.isNaN, not anything = based)#2016-12-2313:45Yehonathan Sharvitcurrently I have (s/def ::coll-of-numbers (s/coll-of number?)#2016-12-2313:46Yehonathan SharvitSo your suggestion is to create a predicate (defn not-nan-number? [x] (and (number? x) (not (Double.isNaN x)))?
#2016-12-2313:47Yehonathan SharvitBTW the function I want to spec is a function that receives a collection of numbers and calculates their average#2016-12-2313:58Yehonathan SharvitYou probably meant this: (s/def ::seq-of-numbers (s/coll-of ::not-nan-number))#2016-12-2313:59Yehonathan Sharvit(s/def ::not-nan-number (s/and number? #(not (Double/isNaN %)))) #2016-12-2313:59Alex Miller (Clojure team)Actually I would use s/and to make a spec, not a pred#2016-12-2313:59Alex Miller (Clojure team)Well, you could do it either way#2016-12-2313:59Alex Miller (Clojure team)There are tradeoffs#2016-12-2314:00Alex Miller (Clojure team)Depends whether you care about gen too#2016-12-2314:02Yehonathan SharvitYeah. I need gen#2016-12-2314:03Yehonathan Sharvitdo you know if there is a isNaN that is cljs compatible?#2016-12-2314:04Yehonathan Sharvitothrewise I would need to make my own #?(:clj Double/isNaN :cljs js/isNaN)#2016-12-2315:12bhagany@viebel in cljs, double-in takes a :NaN? parameterā€¦ Iā€™m kind of surprised this wasnā€™t in clojure first? (Iā€™ve only done spec in cljs)#2016-12-2315:13bhaganyah, it is in clojure, too: https://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec/double-in#2016-12-2315:14bhaganyso, looks like you can (s/double-in :NaN? false) and get correct check and gen behavior in both clj and cljs#2016-12-2316:12bhaganyDidnā€™t have time to complete this thought earlier - Iā€™d probably end up with something like (s/or :int int? :double (s/double-in :NaN? false)). This way also has the advantage of easily including ratios, if you wanted to do that. (`number?` wonā€™t gen ratios)#2016-12-2318:31sophiagogoing to try this again during the daylight...and hopefully before too many people have left for the weekend, if not already. i'm looking for some advice speccing out this project:#2016-12-2318:34sophiagoessentially i'm just wondering if there's any way to automate pairing return types based on input values like this or whether i'm just being far too ambitious about what spec can do#2016-12-2318:36sophiagoit was also suggested i might be able to simplify how i've specified all the combinations of types, but i'm not quite sure how to go about that#2016-12-2318:42sophiagoi suppose one way would be to use gen with the type constructors instead of using the selectors to define all these types? then feeding those into the input and of the functions i'm using fdef with? i suppose when it comes to checking cases where it should reduce i may have to settle for just enumerating a large number of tests i can go over visually rather than having it flag the return values as invalid for me, although that would be ideal#2016-12-2319:37assoc-inDoes anyone know if when I instrument a function with a spec that defines :ret as int? and the function I am testing returns a string if the spec error should show up in the repl? I was able to get the error of the return not being an int? when I used spec/check but expected it with instrument as well. Thanks!#2016-12-2319:38assoc-inAlso I am able to get the :arg spec error to show up with invalid inputs so I believe I am instrumenting the function correctly.#2016-12-2319:38seancorfieldinstrument only verifies that functions are passed valid arguments, i.e., that the calling code ā€œworksā€. If you want to verify a function ā€œworksā€ ā€” testing the return and invariants ā€” you need to look at clojure.spec.test/check.#2016-12-2319:39seancorfieldTwo different types of testing.#2016-12-2319:40assoc-inInteresting I figured it would be a nice addition to the development workflow being able to check the return types were correct on the fly like the arguments are checked#2016-12-2320:04sophiago@seancorfield i was confused by your comment on my snippet last night. were you suggesting i use s/valid with the type constructors rather than specifying so many s/defs using the selectors and instance?#2016-12-2320:06seancorfieldI was suggesting reducing the number of specs so that you didnā€™t have separate cases for int + int / complex + int / int + complex / etc ā€” I think youā€™re over-specifying things and making life harder for yourself.#2016-12-2320:07seancorfield@assoc-in Probably a good idea to watch some of the videos online about clojure.spec so you understand the drivers behind its design ā€” since expecting instrument to run :ret and :fn checks is common when folks first start using spec unless theyā€™ve read / watched a bunch of the design justifications.#2016-12-2320:09seancorfield@assoc-in Not sure if youā€™re in the #beginners channel? Thereā€™s some discussion right now about spec and someone just linked to a great talk by Stu Halloway on it.#2016-12-2320:12sophiago@seancorfield what i'm really concerned with testing are subtyping + coercion. so i do need those cases specified for it to be useful and at least it's something i can do with relative ease. the latter...i'm less sure about. it may come down to just visually checking a large search space to see if any aren't reduced or if they cause the coercion functions themselves to throw errors. but at least for that i need a variety of subtype relationships#2016-12-2320:14seancorfieldAh, I seeā€¦ Then I donā€™t really have any suggestions...#2016-12-2320:15sophiagoah ok. thank anyway. i'm just going to start by having it enumerate a certain number of each type per each function i'm testing with no :fn key or more particular restrictions on :ret#2016-12-2320:16sophiagofor the future...if i do want to use a helper function for :fn do you know the syntax for passing the arg values to that? as opposed to what i tried in that snippet?#2016-12-2320:17sophiagoor ret rather#2016-12-2320:18sophiagoi was trying to do something like this : :ret (coerce-types #(:args :a %) #(:args :b %)))#2016-12-2323:46hlshipIs it allowed to instrument protocol methods? That is, I define a protocol, and want to instrument the created dispatch function.#2016-12-2323:46hlshipSo far, this is not working; Iā€™d imagine that it would instrument the function to verify arguments & etc., then delegate to the original function to actually dispatch on type to an instance of the protocol.#2016-12-2323:47hlshipNever mind, it works. Problem w/ a test.#2016-12-2323:51sophiago@hlship i was going to say...that's exactly what i'm working on right now!#2016-12-2323:54hlshipItā€™s odd; I get the expected failures from the REPL, but inside tests, I can pass non-confirming values without failure.#2016-12-2323:54hlshipI suspect this would work if it was a normal function, but Protocol methods are their own beasts.#2016-12-2323:55sophiagohmm...that's odd#2016-12-2323:55sophiagothis is my first go at fully testing a project and i'm having trouble getting the syntax correct it seems#2016-12-2323:56hlshipI can imagine any number of load-order or bytecode generating things that could be getting in the way.#2016-12-2323:56hlshipIā€™d hate to have to add a function in front of the method, just to get the desired test-time spec help.#2016-12-2323:57sophiagoyeah, i'm not planning on doing that either#2016-12-2400:18sophiagoi still can't figure out how to specify a list of possible s/defs to pass to :args and :ret šŸ˜• i keep getting a "No value supplied for key" error#2016-12-2402:28seancorfield@hlship see http://dev.clojure.org/jira/browse/CLJ-1941 -- known places where instrument doesn't work (includes primitive returning fn and protocols)#2016-12-2402:33sophiagoah, that says only primitives are affected tho?#2016-12-2402:34seancorfieldProtocols are mentioned in the comments.#2016-12-2402:34seancorfieldCan you share the problematic code re: :args :ret?#2016-12-2402:35sophiagoi'm just trying to pass a list of specs to each:#2016-12-2402:35sophiagocan't imagine this is the correct way to do it?#2016-12-2402:36seancorfieldremove the [a b] s-exp#2016-12-2402:37seancorfieldThen it should work.#2016-12-2402:37seancorfieldAs for ::sym -- it can't be all those things at once -- shouldn't that be s/or?#2016-12-2402:37seancorfieldand int? to make it a predicate#2016-12-2402:38sophiagooh yeah. i was playing around with several expressions trying to get it to work#2016-12-2402:38seancorfieldhttps://clojure.org/guides/spec#_spec_ing_functions#2016-12-2402:39sophiagowow. i can't believe i even had the guide open and misread speccing functions as taking an argument list#2016-12-2402:39sophiagothat means it's time to go to sleep soon šŸ˜›#2016-12-2402:39seancorfieldHeh, I've read that thing over and over and I still write nonsense for specs at times šŸ™‚#2016-12-2402:40sophiagoso i should just be able to call (s/exercise add)`?#2016-12-2402:40sophiagooh that doesn't place nicely with slack#2016-12-2402:40sophiago
(s/exercise `add)
#2016-12-2402:41seancorfieldIf your specs are generatable... which yours won't be since you can't generate from instance? I believe...#2016-12-2402:41sophiagoah no wait#2016-12-2402:41sophiagos/exercise-fn right?#2016-12-2402:41seancorfieldHmm, actually I'd have to try s/exercise to check whether it only uses the :args spec#2016-12-2402:43sophiagofor functions defined in protocols, but dispatched on defrecords not primitives so hopefully it should work#2016-12-2402:43seancorfieldAh, yes, it must generate data to even pass stuff into :args...#2016-12-2402:44sophiagoah ok. so would i use s/gen for the list of types?#2016-12-2402:48seancorfieldYou'll need to write generators for your rational and complex types I suspect...#2016-12-2402:49seancorfieldWhat happens if you try (s/exercise ::rational)?#2016-12-2402:52sophiagoi get the same error: "Unable to construct gen..."#2016-12-2402:58seancorfieldRight... you need something like this:
(defrecord Foo [a])
(require '[clojure.spec :as s] '[clojure.spec.gen :as gen])
(s/def ::foo (s/with-gen #(instance? Foo %)
               (fn [] (gen/fmap ->Foo (s/gen int?)))))
(s/exercise ::foo)
#2016-12-2402:59seancorfieldBasically you need to specify a generator that maps your record constructor over a generator for the values it accepts#2016-12-2402:59sophiagoah i see#2016-12-2402:59sophiagoi truly did make this much more difficult by trying to go that granular#2016-12-2403:00sophiagoit wasn't something obvious in the guide i missed going over this all day#2016-12-2403:00sophiagook, thanks. going to give it a try#2016-12-2403:01seancorfieldYeah, that was my first thought... a very ambitious spec! https://clojurians.slack.com/archives/clojure-spec/p1482548398003597#2016-12-2403:02seancorfieldMaking specs generative can be quite the process... we try to do that with all of ours but sometimes it takes... ingenuity...#2016-12-2403:06sophiagowell i was trying to get it to somehow enumerate tests on the correct output types automatically...#2016-12-2403:06sophiagothis is actually much simpler than i imagined#2016-12-2403:06sophiagobut still not covered in the guide#2016-12-2403:07seancorfieldGary Fredericks has created a good companion library for spec that helps generate all sorts of things BTW (we use it, specifically, for regex generation)#2016-12-2403:07sophiagooh i think i saw that#2016-12-2403:08sophiagotest.chuck#2016-12-2403:08sophiagoi just glanced over the functions and don't think it would help in this case though#2016-12-2403:09sophiagotbh this is not so complicated. protocols are generally really easy to debug#2016-12-2403:09sophiagoi actually thought since it's so systematic it would be good to learn how to spec a whole project so i could apply it to code where i really can't figure out what's going on#2016-12-2403:14seancorfieldFYI, s/exercise-fn doesn't check :ret -- only :args#2016-12-2403:15seancorfield
(defn foo [a b] (+ a b))
(s/fdef foo :args (s/cat :a int? :b int?) :ret string?)
(s/exercise-fn `foo)
doesn't complain that foo is supposed to return string? values
#2016-12-2403:28sophiagowell i passed it the whole list anyway šŸ™‚#2016-12-2403:28sophiagoi'll have to check it manually#2016-12-2403:29sophiagobut it will tell me if any subtyping combinations are throwing errors, which i believe was the case before#2016-12-2403:34sophiagohuh. i'm getting the same error...#2016-12-2410:25sveriI get this error: RuntimeException Var clojure.test.check.generators/keyword-ns is not on the classpath clojure.spec.gen/dynaload (gen.clj:21)when running this code: (gen/generate (s/gen number?))in the REPL. Any ideas what I am missing here?#2016-12-2411:59gfredericks@sveri do you have a test.check dependency declared?#2016-12-2411:59sveri@gfredericks No, I dont think so#2016-12-2411:59gfredericksyou need one to use generators#2016-12-2411:59sveriSo this is needed whenever I want to use generators#2016-12-2411:59sveriok šŸ™‚#2016-12-2411:59gfredericks[org.clojure/test.check "0.9.0"]#2016-12-2411:59sveriThank you very much#2016-12-2411:59gfredericksnp#2016-12-2418:30sophiagowhat is clojure.spec/Specize? i'm getting the following error: IllegalArgumentException No implementation of method: :specize* of protocol: #'clojure.spec/Specize found for class: nil#2016-12-2418:34donaldballIIRC itā€™s the protocol by which values are coerced into specs, notably by which keywords resolve themselves in the spec registry#2016-12-2418:48sophiagoah, so it does have to do with trying to exercise protocol functions? because i think i may be experiencing this issue, but am having trouble isolating it to be sure: http://dev.clojure.org/jira/browse/CLJ-1941#2016-12-2418:49sophiagoto be fair, that's the not the error reported in that ticket#2016-12-2418:53sophiagomy case is complicated because the protocol functions i'd like to test all dispatch on records so theoretically should be unaffected. however, i went so granular as to define arguments to their type constructors using s/with-gen and gen/fmap. so i can't tell if some of the errors are actually what i'm looking for, known subtyping bugs, because i'm not sure if the generators that don't do that are affected and am currently getting three different error codes dependent on whether the arguments to the type constuctors are primitives or other records and also (the really odd part) the function itself, which shouldn't make a difference as far as i can tell#2016-12-2509:18naomarik@bbloom > whatā€™s the guidance for spec-ing pseudo-namespaced keywords? iā€™m thinking like datomic attributes when you do :customer/name or something like that Iā€™ve actually been using it like this pretty much exclusively and I feel reading and referring to the spec becomes pretty trivial especially when you have multiple types that have the same name but the namespace disambiguates it (like multiple name fields for instance :customer/name :business/name)#2016-12-2510:03dvcrnHey guys! Using clojure-spec for the first time to describe a db schema. Does this look okay or am I doing something obvious wrong?
(s/def ::first-name string?)
(s/def ::last-name string?)
(s/def ::username string?)
(s/def ::picture (s/nilable string?))
(s/def ::relationship-type int?)

(s/def ::friend (s/keys :req-un [::username ::last-name ::first-name ::picture ::id ::relationship-type]))
(s/def ::friends (s/coll-of ::friend :kind vector?))
#2016-12-2510:03dvcrn(no error, just want to check in)#2016-12-2510:04dvcrnI think I don't completely understand why :req-un is necessary#2016-12-2510:29naomarik@dvcrn :req-un so when you pass in the data to be validated you can do
{:username "",
   :last-name "",
   :first-name "",
   :picture "",
   :relationship-type -1}
otherwise with :req youā€™d have to qualify the keywords.
#2016-12-2510:31naomarikalso try out (s/exercise ::friend) to test out your specs šŸ˜‰#2016-12-2510:34dvcrn@naomarik I can't follow. You mean with req-un I can leave out keys?#2016-12-2510:35naomarikwithout :req-un you do something like this:
{:friend/username "",
 :friend/last-name "",
 :friend/first-name "",
 :friend/picture "",
 :friend/relationship-type -1}
#2016-12-2510:35dvcrnah, I see!#2016-12-2510:35naomarikso these are fully qualified keywords#2016-12-2510:35naomarikwhich is very useful imo when dealing with multiple entities in the same map#2016-12-2510:36naomarikhelps disambiguate whose ID or name youā€™re talking about#2016-12-2510:36dvcrnright šŸ™‚#2016-12-2510:37dvcrnso I guess fully qualified / namespaced keywords are better#2016-12-2510:38naomarikiā€™m not qualified to give an answer on that but i can say at least in my extremely limited usage on an immature project it has helped me discern what data iā€™m looking at#2016-12-2510:38dvcrnyeah completely makes sense#2016-12-2511:24noogaunqualified keywords would usually come from/go on wires#2016-12-2616:59roelofI have this error message :
(clojure.spec.gen/generate (clojure.spec/gen :paintings2.routes.home/page))
RuntimeException Var clojure.test.check.generators/large-integer is not on the classpath  clojure.spec.gen/dynaload (gen.clj:21)
 
#2016-12-2617:01roelofon this code :
(ns paintings2.routes.home
  (:require [paintings2.layout :as layout]
            [compojure.core :refer [defroutes GET]]
            [ring.util.http-response :as response]
            [ :as io]
            [compojure.route :refer [resources]]
            [environ.core :refer [env]]
            [paintings2.api-get :as api]
            [clj-http.client :as client]
            [clojure.spec :as s]))

(defn ->long [s] (try (Long/parseLong s) (catch Exception _ ::s/invalid)))

(s/def ::page (s/and (s/conformer ->long) (s/int-in 1 471)))

(s/def ::optional-page (s/nilable ::page))

(defn page-check [page] (let [page page page-num (or (s/conform ::optional-page page) 1)] page-num))

(s/fdef page-check
        :args (string? (::page :page))
        :ret  (number? (::page :page))
        :fn  (s/valid? true?  (::page :page)  )
       ) 
#2016-12-2617:01joshjonesand did you put [org.clojure/test.check "0.9.0"] in your project dependencies?#2016-12-2617:01roelofand I have this in my projects.clj :
:profiles {:dev {:dependencies [[org.clojure/test.check "0.9.0"]
                                   ]}} 
#2016-12-2617:02joshjonesWhy not put it in your main :dependencies? Why in a profile?#2016-12-2617:02roelofsee my last response, @joshjones#2016-12-2617:02roelofo, then I misunderstood you#2016-12-2617:02roelofmoment, I will change it#2016-12-2617:03joshjones
(defproject your-project "1.0"
  :description "spec playground"
  :url ""
  :license {:name "Eclipse Public License"
            :url ""}
  :dependencies [[org.clojure/clojure "1.9.0-alpha14"]
                 [org.clojure/test.check "0.9.0"]])
#2016-12-2617:04roelofoke, now I see another error message :
ExceptionInfo Unable to construct gen at: [] for: (conformer ->long)  clojure.core/ex-info (core.clj:4725) 
#2016-12-2617:06joshjonesnow you're getting somewhere -- i get why you want to use a conformer, but i've not used a conformer to generate data before, nor have i seen any info on doing so -- only seen info on conforming/validating using a conformer#2016-12-2617:08joshjonesso you just want to generate numbers in a certain range?#2016-12-2617:08roelofyes, I want to check if the url parameter is between the 1 and 471 or is not there#2016-12-2617:08roelofis the last case the pagenumber schould be set to 1#2016-12-2617:09joshjonesand the input can be either a string or a number, right?#2016-12-2617:11joshjonesthe way you've written the spec for ::page, I do not think there's enough info there for it to do any data generation#2016-12-2617:12roelofoke, what I try to do it this : The user can use this url : http://localhost:3000/#2016-12-2617:12roelofand then my code schould take that the pagenumber is 1#2016-12-2617:13roelofor the user can use this url : http://localhost/?page=n#2016-12-2617:13roelofn will be a integer where my code makes a long of it#2016-12-2617:13roelof@joshjones so far, clear ?#2016-12-2617:14roelofNow I want to make a generator which makes several cases for n#2016-12-2617:15roelofso n could be "a" then the outcome is invalid or "124" then the outcome is 124 or "500" then the outcome is also invalid#2016-12-2617:15roelofso I want testcases that test this#2016-12-2617:16joshjonesok -- what you want to actually generate is the entire URL?#2016-12-2617:18joshjonesyou are using something already for routing, and for parsing the query string?#2016-12-2617:44joshjones
(defn gen-page-num
  []
  (gen/fmap #(str %) (s/gen (s/int-in 1 471))))

(s/def ::page
  (s/spec (s/and (s/conformer ->long) (s/int-in 1 471))
          :gen gen-page-num))
#2016-12-2617:45seancorfieldYou might want to keep this level of back and forth in the #beginners channel#2016-12-2617:57roelof@joshjones my code for the url parsing with the pagenumber could be find here : https://github.com/rwobben/paintings/blob/master/src/clj/paintings2/routes/home.clj#2016-12-2617:58roelof@seancorfield The last question is about spec because josh thinks that my fdef for testing and generating test-data is not good#2016-12-2617:59joshjones@roelof Use the above generator and revised def for ::page and you'll be able to generate data using ::page#2016-12-2618:01roelofand I can use that data for checking the page-check function#2016-12-2618:03roelof@joshjones Am I right that I cannot check what happens if a user enters false data like this : ' /?pag="a"#2016-12-2618:07seancorfield@roelof Although this is about clojure.spec, there are still a lot of beginner-level issues at play here so Iā€™d suggest keeping to the #beginners channel where folks have opted into helping folks with this level of discussion, rather than swamping the other channels with long back and forth about syntax and basic errors.#2016-12-2709:37curlyfryHi, I have the following scenario:
(s/def ::name string?)
(s/def ::person (s/keys :req-un [::name]))

(s/def ::name (s/and string? #(str/starts-with? % "Jean")))
(s/def ::french-person (s/keys :req-un [::name]))
I want to keep these specs in the same file, but I also want to disambiguate the different ::name specs, without giving them different names. Is there any way to do this? Something like ::french/name vs just ::name, but that isn't working.
#2016-12-2710:13mpenet@curlyfry you ca use create-ns and alias for that#2016-12-2710:22curlyfry@mpenet Cool, thanks! Can I create a concatenation of the "current" ns with a symbol using create-ns?#2016-12-2713:31roelofI hope this is not a beginner question.#2016-12-2713:31roelofI want to test a function that has as input something like this : [{ :name "Roelof", :place "internet"} {:name "Jane Doe" :place "unknown" }] and as output {"Roelof", "Jane Doe"} . If I want to test this , can I use the same idea as this :
(def email-regex #"^[a-zA-Z0-9._%+-]
#2016-12-2713:33roelofor what must I fill in if a use a fdef with :args and :ret and :fn#2016-12-2716:47joshjonesPerhaps your question is constructed in a way that makes answering it a bit difficult. For example, ā€œinput something like thisā€ is ambiguous ā€” are there always two maps in the input vector, or might there be more? And, do you really mean to have a single k/v pair map as the output, where both the key and value are strings? Or did you mean to make that a vector of the :names from the input? Itā€™s clearing up this confusion that results in the things the admins donā€™t want in the channel, I think. But hereā€™s some info that might be valuable to anyone new to spec, just a sample of how to construct what youā€™re asking for. What you want to do is ask, ā€œwhat am I trying to specā€? In your case, itā€™s a function which takes a vector of maps, and returns a vector of strings. Those are your :args and :ret, and a decent sanity check for :fn is that the number of names you return should equal the number of maps you were given. So, hereā€™s a spec based on that:#2016-12-2716:47joshjones
(ns clj.core
  (:require [clojure.spec :as s]
            [clojure.spec.gen :as gen]
            [clojure.spec.test :as stest]))

(defn get-names [v]
  (reduce #(conj % (:name %2)) [] v))

(s/def ::name string?)
(s/def ::place string?)

(s/def ::get-names-args (s/coll-of
                          (s/keys :req-un [::name ::place])
                          :kind vector?
                          :gen-max 5))

(s/fdef get-names
        :args (s/cat :v ::get-names-args)
        :ret (s/coll-of string? :kind vector?)
        :fn (fn [{{input-vector :v} :args, return-vector :ret}]
              (= (count input-vector) (count return-vector))))
#2016-12-2716:48joshjonesYou can now exercise and check the function with:
(s/exercise-fn `get-names)
(stest/check `get-names {:clojure.spec.test.check/opts {:num-tests 100}})
#2016-12-2820:53joshjonesIā€™m a little confused on fmap vs bind ā€” according to the docs for clojure.test.check.generators/bind: Create a new generator that passes the result of gen into function k. k should return a new generator. fmap also takes a generator, and applies some function to it. The only difference I can see is that in bind, the function is responsible for returning a generator, and in fmap, the function just returns data. Is there some other difference Iā€™m not seeing?#2016-12-2821:07gfredericksThat's exactly the difference#2016-12-2821:08gfredericksbind is more powerful than fmap#2016-12-2821:20sophiagoso the idea is this is somewhat like Haskell with generators analogous to monads? that comparison doesn't make the most sense to me tho#2016-12-2821:56gfredericksGenerators are a monad, yes#2016-12-2821:56gfredericksI expect that's literally true in quickcheck, which test.check is largely based on#2016-12-2822:15sophiagoi suppose in the sense that they're bound to a certain type? but i don't see how the monad laws could even possibly apply to generators#2016-12-2822:34gfredericksThe first three make sense to me. I'll translate them to generators when I get to a keyboard#2016-12-2823:02gfredericksā†‘ I haven't grokked the fourth one but it probably works ĀÆ\(惄)/ĀÆ#2016-12-2823:02gfredericks@sophiago#2016-12-2823:05sophiagoi don't think that SO post is the best source. i've always only known three laws: left identity, right identity, and associativity. https://wiki.haskell.org/Monad_Laws#2016-12-2823:06gfrederickslooks like it just skips the 3rd one#2016-12-2823:06sophiagothe associativity one in your example is unclear to me#2016-12-2823:06gfredericksso the first two are equivalent#2016-12-2823:06sophiagothe first two make sense. i didn't know there was a gen/return either!#2016-12-2823:07gfrederickswe could experiment to see if the two seem to do the same thing šŸ™‚#2016-12-2823:08sophiagoi'm confused about the purpose of gen/return from reading the api docs#2016-12-2823:08gfrederickswhat it does, or when it's useful?#2016-12-2823:08sophiagojust what it does#2016-12-2823:09sophiagoi suppose returns a generator? lol#2016-12-2823:09sophiagoso maybe i should have asked the second question...#2016-12-2823:09sophiagolike in what case would you bind a generator and then return it?#2016-12-2823:12gfredericksbind+return is just fmap#2016-12-2823:12gfredericksso it's easier to use fmap#2016-12-2823:12gfredericksreturn is more often useful on its own#2016-12-2823:12gfredericksI'll go grep my source code to see when I use it#2016-12-2823:14gfrederickshere's an example that uses it as the base case in a recursive generator: https://github.com/gfredericks/chess-clj/blob/89b2e5543534f82915a1aab67e026a326a0ed0a5/test/com/gfredericks/chess/generators.clj#L194#2016-12-2823:16gfredericksthe associativity law looks right to me#2016-12-2823:16gfredericksit's just double-binding in two different ways; looks suspiciously trivial actually#2016-12-2823:16sophiagowoah, that's a really cool application of generators!#2016-12-2823:17gfredericksšŸ™‚#2016-12-2823:17sophiagoyes...because >>= is the bind operator šŸ™‚#2016-12-2823:18sophiagoso if you can associate that then you've met the third monad law...really the important one in that it lets you compose monads that encapsulate different typeclasses#2016-12-2823:19sophiagothis would make a good blog post or something to that effect#2016-12-2823:19gfredericksI would need an example of such a composition to see what that means#2016-12-2823:20sophiagoit doesn't have a corollary in dynamic typing. that's why i always thought algo.monad was quite odd...#2016-12-2823:20sophiagobut it makes more sense with generators because they do in a sense have type signatures#2016-12-2823:20gfredericksyeah I've never seen the point of a monad library in clojure#2016-12-2823:21sophiagoit's really just a library of macros that needlessly replicate the functionality of a number of common haskell monads#2016-12-2823:21gfrederickstoo much you can't do and not enough help from the compiler#2016-12-2823:22sophiagobut this makes sense. you need bind in order to compose generators with different types#2016-12-2823:24sophiagowow, now i'm really interested in uses of spec beyond things like quickcheck. i haven't seen other examples actually using generators to write code, although i think rich touched on it in his talk i saw#2016-12-2823:24gfredericksI'm not even sure what that means#2016-12-2823:24gfredericksgenerating code?#2016-12-2823:25sophiagowell, i only glossed over your chess example, but it seems you're using them to actually generate moves rather than to just test functions you've written?#2016-12-2823:27gfredericksno it's still for tests#2016-12-2823:27gfredericksit is generating moves, but only in service of generating legal positions, which is for testing#2016-12-2823:28sophiagoyeah i looked at it again. at first i thought you were using it to play chess#2016-12-2823:28gfredericksI don't think test.check generators are necessarily the best fit for non-testing purposes#2016-12-2823:28gfredericksgenerally they'll give you distributions that are a bit weird outside of testing#2016-12-2823:29sophiagoyeah, i've only noticed that now that i've started getting them working for whole projects...are they purposefully non-random or that's just a consequence of their implementation?#2016-12-2823:30gfredericksthey are random#2016-12-2823:30gfredericksjust skewed in their distribution, as a heuristic for catching bugs#2016-12-2823:32sophiagoi mean, for example, if i call (gen/sample (s/gen int?) n) the size of the ints increase with n. initially they're quite small...which, as you said, makes sense for testing#2016-12-2823:32gfredericksyeah, that's half of the weirdness#2016-12-2823:33gfredericksthat's demonstrating that gen/sample uses an increasing size parameter#2016-12-2823:33sophiagooh#2016-12-2823:33gfredericksbut even if you fix size, which you can do with gen/generate, there are still elements of the "smaller things are worth trying more often" heuristic#2016-12-2823:34gfredericksmore about sizing: https://github.com/clojure/test.check/blob/master/doc/growth-and-shrinking.md#2016-12-2823:36sophiagoah, these are the docs i should have been looking at this whole time! i've been using this: https://clojure.github.io/clojure/branch-master/clojure.spec-api.html which just says everything is a wrapper of test.check macros#2016-12-2823:37gfredericksI'm trying to do a big documentation overhaul, so I'll try to look for ways to make discovery better too#2016-12-2823:41sophiagoyeah, your test.check docs are most useful for what i personally needed than most of what's in alex's guide. like i could have used gen/nat earlier to avoid divide by zero errors and instead rolled my own#2016-12-2823:47sophiagoanyway, i think it'd be interesting to have something about the generator combinators and category theory. it's an interesting consequence of how they're typed that we're not used to in a dynamic language#2016-12-2823:48sophiagoi'd consider writing something like that for a wiki if i thought i could both make it accurate and substantial enough for anyone to care about šŸ˜›#2016-12-2823:50gfredericksthat would be fun to read#2016-12-2823:50sophiagomaybe...#2016-12-2907:00seako@sophiago carin meier wrote a blog post about using spec to generate code that you might find interesting http://gigasquidsoftware.com/blog/2016/07/18/genetic-programming-with-clojure-dot-spec/#2016-12-2907:08seakoalso, i would like chime in that i also think it would be very interesting to read about generator combinators and category theory#2016-12-2914:43stephenmhopperI have a spec that I've written with clojure.spec and I can use it to generate valid Clojure data structures. Can I use it to generate invalid structures?#2016-12-2914:45gfredericksgenerating things in a negative way is hard to do in general#2016-12-2914:45gfredericksI expect clojure.spec makes no attempt to support that, since it's not obvious what strategy it would use#2016-12-2914:46gfredericksyou can use unit testing for that, or custom generators to cover more specific cases#2016-12-2914:46gfredericksfor a lot of cases you might do fine by using gen/any and filtering out things that match the spec#2016-12-2914:50stephenmhopperI did not know about gen/any, but that should do the trick. I was considering just creating a version of my schema where all required keys are instead marked as optional while also changing the data type / generators for some of the keys, but gen/any is simpler#2016-12-2914:50stephenmhopper@gfredericks thank you#2016-12-2914:51gfredericksnp#2016-12-2915:06naomarikare you guys converting database records field names to namespaced keywords?#2016-12-2915:07naomarikor just specing database stuff with :req-un#2016-12-2915:13gfredericksI think converting would be cooler, but it takes more work and I haven't worked on something like that yet anyhow#2016-12-2915:14gfrederickse.g., a namespace per table#2016-12-2915:14naomarikyes this is what iā€™d like#2016-12-2915:14naomariki spent some time specing all my stuff like that and while i can just use req-un i would rather see it qualified#2016-12-2915:15naomarikwould be great if i didnā€™t have to rename joined tables either, ie multiple id fields#2016-12-2915:24gfredericksright#2016-12-2916:16naomarikthereā€™s anyway to conform a spec given keys with :req-un into the matched qualified keywords?#2016-12-2916:55joshjonesI do not think so; do you have a particular use case for this?#2016-12-2916:57seancorfieldFWIW clojure.java.jdbc supports producing qualified names via the :qualifier option. Doesn't handle joins tho'. #2016-12-2916:59joshjonesYou could use s/describe on the spec to get the vector of :req-uns, and then pull the key with the name of the key you want to qualify, grab its namespace, and assoc a ā€œnewly createdā€ fully qualified key onto the conformed data#2016-12-2917:00joshjonesIf you really wanted to. šŸ™‚#2016-12-2917:13naomarik@seancorfield ya saw that, will probably do that for non joined data and use an identifier that will handle select AS statements#2016-12-2917:41joshjonesjust as an exercise @naomarik :
(defn qualify-keys
  [spec conformed]
  (let [unq-to-qual (as-> (s/describe spec) $
                          (second (drop-while (partial not= :req-un) $))
                          (zipmap (map name $) $)
                          (reduce-kv #(assoc %1 (keyword %2) %3) {} $))]
    (reduce-kv #(assoc (dissoc %1 %2) %3 (get %1 %2))
               conformed unq-to-qual)))

(s/def :ns.one/requnkey string?)
(s/def :ns.two/reqkey string?)
(s/def ::mymap (s/keys :req-un [:ns.one/requnkey] :req [:ns.two/reqkey]))

(s/conform ::mymap {:ns.two/reqkey "required!" :requnkey "not required!"})
=> {:ns.two/reqkey "required!", :requnkey "not required!"}
(qualify-keys ::mymap *1)
=> {:ns.two/reqkey "required!", :ns.one/requnkey "not required!"}
#2016-12-2918:08joshjones@seancorfield you just mentioned in the #clojure channel about using with-redefs to stub functions for testing; whatā€™s your take on taking a specā€™d function and now being able to do this, versus the with-redefs approach?:
(stest/instrument `my-func {:stub #{`my-func}})
#2016-12-2918:09seancorfield@joshjones I havenā€™t experimented with stubs in clojure.spec yet so I donā€™t have an opinion.#2016-12-2918:10seancorfieldMostly weā€™re specā€™ing data structures and using s/conform etc ā€” weā€™re not specā€™ing functions much.#2016-12-2918:14joshjonesok ā€” I have not tried it for production-level testing either, but it looks useful:
(defn get-data-from-db [] nil)

(s/fdef get-data-from-db
        :args empty?
        :ret (s/int-in 1 10))

(stest/instrument `get-data-from-db {:stub #{`get-data-from-db}})

(get-data-from-db)
=> 6
(get-data-from-db)
=> 7
#2016-12-2918:17seancorfieldSo it stubs the named function to return generated data per spec rather than actually calling it? Interesting. Wouldnā€™t work for anything that actually required mock state, rather than just stubbed data, but I can see that being useful sometimes.#2016-12-2918:18seancorfieldI havenā€™t generally found stubs to be as useful as mocks so I donā€™t know how much Iā€™d use it.#2016-12-2918:18joshjonesright, thatā€™s a good point#2016-12-2918:19seancorfieldI can see it being useful as youā€™re developing top-down where you stub functions as placeholders until you actually write them.#2016-12-2918:20seancorfieldRight now, Iā€™d actually write the physical stubā€¦ with the spec approach you could just get instrument to ā€œwrite the stubā€ for you, but then youā€™d have to re-instrument or un-instrument as you actually wrote each function.#2016-12-2918:21joshjonesis there something you typically use for mocking state? or it just depends on your particular application?#2016-12-2918:21roelofHow can i check in a spec for a key in a nested map ? so this is valid (s/explain ::objectNumber-list [{:body{:artObjects{:objectNumber "sk-c-5"}}}]) #2016-12-2918:21seancorfieldSince mocks tend to be pretty tied to your implementation, I generally just hand-roll them.#2016-12-2918:22seancorfield@roelof Please keep working in the #beginners channel on that.#2016-12-2918:22seancorfieldYou need to think carefully about the elements of that list, is all Iā€™ll say.#2016-12-2918:24seancorfield@joshjones When we mock things, we tend to need to coordinate data across a series of calls ā€” or even just record data across a series of calls ā€” so custom functions that use atoms or refs to track that is our ā€œgo toā€ approach.#2016-12-2918:25seancorfieldFor example, we mock a payment provider to track calls and amounts, and pass or fail the transaction based on state.#2016-12-2918:25joshjonesI guess the amount of data you mock is pretty small though right? i.e., the purpose of the mocking at this stage is not for load-testing, which is done in a completely different way.#2016-12-2918:27joshjonesfor example, we load test our elasticsearch code using dedicated aws servers, so itā€™s purely a load test ā€” for testing correctness, etc., small amounts of mock data is used locally to be sure everything works as expected.#2016-12-2918:29seancorfieldMocking is to verify that a function calls some other function(s) in a particular way and/or that a series of calls to the mocked functions returns data in a particular way.#2016-12-2920:23henriklundahl@roelof, you spec each value which is associated with a specific key and then specify which keys should be included in each map. Start with the innermost map(s) and work your way out.#2016-12-3009:00sophiago@seako thanks! i've been busy so just getting around to reading this, but it looks really interesting!#2016-12-3009:03sophiagoand i'll think a little more about writing the post. it's not so complicated from the category theory side, so i don't think i need to worry about my level of knowledge with that...more so with test.check. maybe if i've worked with it a bunch more by the time @gfredericks revamps his docs i can add it to the wiki there#2016-12-3100:11sophiagowow, so i finally got around to going through this: https://github.com/gigasquid/genetic-programming-spec and now i'm wondering how it could be used for actual testing#2016-12-3100:14sophiagofor example, in the program i just specced out i originally wanted to actually match the correctness of args and return values, which would have meant a quadratic number of sfdefs. i had a list of i think 11 types and five functions...so if i'm doing the math right that would have meant 605 different definitions? now i'm thinking it could actually be possible to enumerate that entire testing space if i learned to have the specs generate themselves... something to think about#2016-12-3122:09bbloomis there some way to get more info from this error: "Couldn't satisfy such-that predicateā€ ? the ex-info map is empty#2016-12-3122:09bbloomiā€™m curious what predicate is failing#2016-12-3122:09gfredericksit's um#2016-12-3122:10gfredericksprobably from an s/and somewhere#2016-12-3122:10gfrederickstest.check master has a mechanism for spec to add more info in this case, but spec would have to be updated as well#2016-12-3122:10gfredericksit's something rich asked for#2016-12-3122:11bbloomok cool - will just bin-search it like i do practically all my debugging šŸ˜‰#2016-12-3122:11gfredericksyep šŸ˜¢#2017-01-0111:31tianshudoes clojure.spec.test/instrument support check :ret and :fn for common function call?#2017-01-0111:31tianshucurrently it seems only work for test.check.#2017-01-0116:22settingheadI noticed spec tolerates ambiguity (say (s/cat :first (s/* number?) :second (s/* number?)) can interpret [1 2 3] in 4 different ways, but it returns the first way it could find). is there any way to disallow ambiguity?#2017-01-0116:54english@doglooksgood from the spec guide: "Note that the :ret and :fn specs are not checked with instrumentation as validating the implementation should occur at testing time."#2017-01-0118:45mingpThose of you who use a lot of spec, what would you say to concerns that spec makes you code a certain way to be able to work with it, e.g. namespaced keywords for map keys? This is something that has worried me and made me hesitant to start using spec, but of course I say this as someone who hasn't used it, so what would those of you who have say?#2017-01-0118:46mingpSomething else I've been wondering. To what extent are you all using function specs vs explicit conform? Do you leave both/either on in production?#2017-01-0118:47notanon@mingp what is the concern with namespaced keywords?
#2017-01-0118:49mingpMy concern is, more broadly speaking, that spec expects me to code a certain way to be able to work with it. In the specific case of map keys, I'm normally in the habit of using regular keywords, but spec would work best if I switch to namespaced keywords instead.#2017-01-0118:49mingpAgain, I say this as not an actual user, so I'm trying to ask actual users if they believe these concerns are valid and if I should worry about them.#2017-01-0118:49notanonwell you can still use unqualified keys, it's fully supported#2017-01-0118:50notanonthe reason for namespaced keywords is just because specs are global, and so must have unique names#2017-01-0118:50sveriI am not sure if thats a good approach, but, what I do is to use namespaced keywords for my definitions and unqualified names for the maps like (s/def ::foo-map (s/keys :req-un [::bar-key ::baz-key]))#2017-01-0118:50notanonotherwise you wouldnt be able to reuse specs so easily#2017-01-0118:51mingpMore generally speaking, would you say spec has/hasn't constrained how you code Clojure?#2017-01-0118:51notanonsveri's example is a good example of a compromise, ::foo-map can be re-used all across the app#2017-01-0118:52notanonthe only downside is that ::bar-key and ::bas-key will not be self-describing but... that's probably not going to be an issue for most use cases#2017-01-0118:54notanon@mingp i don't see it constraining me personally. it's just a name#2017-01-0118:54mingpThat's good to hear. Thanks for that.#2017-01-0119:46seancorfieldNote that you can use namespace qualifiers on your :req-un keys to distinguish between different uses of the "same name".#2017-01-0119:48seancorfield(s/keys :req-un [:foo/id]) is still the unqualified :id in the map but it uses the (qualified) :foo/id spec. #2017-01-0119:50seancorfieldSo while ::id would be the same spec for all :id fields, :foo/id and :bar/id can be distinct :id specs. #2017-01-0120:03joshjones@mingp As sean just said, just to be clear -- spec totally works with unqualified map keys. What's required to be namespace qualified are the keywords that name specs themselves.#2017-01-0120:10notanona subtle, but important difference. said another way (because it's subtle) only the specs themselves need to be namespaced qualified. the data they're spec'ing does not need to be qualified.#2017-01-0120:14mingpThanks @seancorfield @joshjones and @notanon.#2017-01-0122:25Alex Miller (Clojure team)spec would like to encourage you to work with maps with qualified keywords, but will support you to a limited degree if you canā€™t/donā€™t want to.#2017-01-0122:26Alex Miller (Clojure team)from my own work with using specs while developing new code, I have found it has made me more likely to break out predicates and helper functions about my data (which has made the code better)#2017-01-0122:29notanonI think I've heard or read that the fully qualified keys will make your data 'self-describing' but I really don't understand what that is buying me#2017-01-0122:50potetm@notanon You get validation on keys you might not even know exist, because they're in a global registry.#2017-01-0122:54potetmThat's only possible because they're in a global registry. And to form a global registry, you need keys that are globally unique.#2017-01-0122:58notanonhm. i don't think i get that, if I conform or validate based on a spec i've defined, i wouldnt have told it the 'keys i dont know about' are of this spec. and of course i might not want it conform/validate keys i didn't ask it to#2017-01-0123:04Alex Miller (Clojure team)s/keys will validate all qualified keys in the map based on the matching spec in the registry#2017-01-0123:05Alex Miller (Clojure team)weā€™re suggesting that you should want this :)#2017-01-0123:07Alex Miller (Clojure team)this may seem weird, but itā€™s a key enabler for the kind of evolution Rich talked about in his Conj keynote https://youtu.be/oyLBGkS5ICk where you are ā€œgrowingā€ a system over time#2017-01-0123:08notanonyeah that makes sense. if i add a key later, it will automatically be validated without me updating the spec#2017-01-0123:23notanoni guess if i just wanted to validate the presence of keys (only presence) i wouldn't use s/keys. not to mention that i don't think that would be very useful#2017-01-0123:41Alex Miller (Clojure team)yes, something like #(every? #{:foo :bar} (keys %))#2017-01-0209:22roelofI have these specs :
; specs for validatimg input to find rhe ids
(s/def ::artObject (s/keys :req-un [::objectNumber]))
(s/def ::artObjects (s/coll-of ::artObject))
(s/def ::body (s/keys :req-un [::artObjects]))
(s/def ::response (s/keys :req-un [::body]))
 
#2017-01-0209:23roelofnow I have to extend them for another call with some things#2017-01-0209:24roelofCan I just make a copy with it and extend it and named all the keywords then ::data-response#2017-01-0209:24roelofor is there a better way ?#2017-01-0211:11luxbockso I'm working on my idea of writing a defn-like macro that infers specs from the (extended) destructuring syntax of the argument vector#2017-01-0211:12luxbockand I was thinking that in the case of a simple case of {:keys [(foo int?)]}, which of course means that the passed map contains an optional unqualified key of :foo which is an int#2017-01-0211:14luxbockI need to define the spec for :foo. would it be a terrible idea to just use a randomly generated ns for that such as :foo1235/foo?#2017-01-0211:15luxbocki.e. the argument vector [{:keys [(foo int?)]}] expands to (s/def :foo1235/foo int?) + (s/cat :m (s/keys :opt-un [:foo1235/foo]))#2017-01-0211:17luxbockI don't want to end up in a situation where my defn-macro ends up overwriting specs for situations where multiple functions accept maps that contain simple-keywords that can contain different types of values#2017-01-0216:08roelofnobody who can help me with this one#2017-01-0217:18sveri@roelof It is hard to tell what exactly you want to do. At least for me. I am pretty sure you can do what you want, but if you need help I guess you need to be more specific and provide some examples.#2017-01-0217:32seancorfieldAlso, it seems to be the same question youā€™ve asked several times now ā€” and it has been answered in the #beginners channel...#2017-01-0217:33seancorfieldI have answered it at least twice now in the #beginners channel and youā€™ve had input from other people there.#2017-01-0217:47fadrianI'm doing my best to understand spec. I'm working through a cards example, using my own representation of a card. I've defined my specs as follows:#2017-01-0217:49fadrianAccording to the spec spec, my definition for cards should be checking for correct conformance for ::suit and ::rank. But (s/conform ::card "1m") returns "1m", not an invalid as I expect. What's happening here?#2017-01-0218:31gfredericks@luxbock I'd use the current ns plus an extra random segment as the namespace#2017-01-0218:33seancorfield@fadrian suits should be a set, not a map.#2017-01-0218:33seancorfield(line 1)#2017-01-0313:46curlyfryWhat is the best way to spec a string that contains a floating point number? Using a regex or some other method?#2017-01-0313:52mpenetFloat/parseFloat comes to mind#2017-01-0313:53curlyfrySo what would the spec look like? In particular, would I have to write a custom generator?#2017-01-0313:59donaldballIt might look like: (s/spec #(Double/parseDouble %) :gen (gen/fmap str (gen/double)))#2017-01-0314:00donaldballPossibly using gen/double* with some constraints#2017-01-0314:19curlyfry@donaldball Cool, thanks! Unfortunately I'm in cljs-land, so I think I might have to go the regex path anyway#2017-01-0314:20dialelomaybe js/parseFloat can help then @curlyfry#2017-01-0314:22curlyfryRight!#2017-01-0314:24curlyfryJust one more thing: When you make a spec like (s/def ::double-string #(Double/parseDouble %)) we rely on parseDouble throwing an exception rather than matching a certain predicate, right?#2017-01-0314:45curlyfryjs/parseFloat is a bit too forgiving: (js/parseFloat "1.23abcdef") => 1.23#2017-01-0314:46curlyfryEnded up with:
(def decimal-regex #"-?\d+\.\d+")
(s/def ::double-string (s/spec (s/and string? #(re-matches decimal-regex %)) :gen #(gen/fmap str (gen/double))))
Anything that could be written in a nicer way?
#2017-01-0314:47mpenetif you end up doing that a lot (specing via regex) you might want to check test.chuck for the gen part#2017-01-0314:48mpenetespecially test.chuck.generators/string-from-regex#2017-01-0314:49mpenetI've thrown crazy regexes at it and it never failed me (so far)#2017-01-0315:08joshjones@curlyfry not saying a nicer way doesnā€™t exist, but that way is straightforward and works well, and itā€™s the way iā€™d do it#2017-01-0315:09curlyfry@mpenet @joshjones Sweet, thanks guys!#2017-01-0315:17roelof@seancorfield I know that you answered it with I think a namespace and a function ( artObjects-id:response ) but when I used it in a sdef I did not work#2017-01-0418:30zmarilis there a way of hiding a tag in spec? In instaparse, there is <a> = "a" and the hide-tag combinator#2017-01-0418:33seancorfieldDo you mean from an s/or clause?#2017-01-0418:36zmarilyes or a s/cat clause#2017-01-0418:38joshjonesā€œhidingā€ it in what way?#2017-01-0418:38joshjonesin the conformed value, the tag will be the key in the map identifying the value#2017-01-0418:46zmarilhmmm okay nvm#2017-01-0504:11bbloomso when weā€™re not using generative tests, what do people use for utilizing specs from tests?#2017-01-0504:13bbloomi find s/assert annoying b/c of the check-asserts compile time flag - need to forcibly reload everything that might have asserts when working interactively#2017-01-0504:15bbloominstrument is nicely dynamic, but i understand why the asserts are static#2017-01-0504:16bbloomi still donā€™t really understand why :ret and :fn are not checked when instrumented#2017-01-0504:17bbloomThe guide now says "Note that the :ret and :fn specs are not checked with instrumentation as validating the implementation should occur at testing time.ā€ - but instrument are in the stest namespace, soo presumably you only instrument at test time anyway?#2017-01-0504:18bbloomI get the desire to not instrument ret and fn for other peoples code in some cases, but honestly, i also want to validate that the libraries iā€™m calling respect their specs#2017-01-0504:18bbloomi donā€™t trust them#2017-01-0504:19bbloomiā€™m just going to call s/assert* for manual (non-generative) tests and deal with it if it breaks at some point šŸ˜‰#2017-01-0512:53dacopareHello, I've just started using Spec! I'm trying to use it in a particular situation that I can't see covered in the documentation but I may be wrong. I have two datastructures:
(def created
 {:event-type "TxnCreated"
  :data {:id "foo"
         :amount 6}})

(def deleted
 {:event-type "TxnDeleted"
  :data {:id "foo"
         :reason "bar"}})
#2017-01-0512:54dacopareI'd like to have a different spec for each of the different :data items, but from what I can see s/keys will use one spec for one key...#2017-01-0512:57dacopareSo that a deleted item would have different required keys to a created item. Is the problem that my keys are not namespaced? Or is this going against: > "the map spec never specifies the value spec for the attributes, only what attributes are required or optional"#2017-01-0512:57dacopareThank you for any help simple_smile#2017-01-0513:00manutter51Yeah, this is exactly the use case for namespaces. You want to re-use the same keyword to mean 2 different things, put them in different namespaces and youā€™ll be fine#2017-01-0513:27dacopare@manutter51 the only problem I have with this is that I currently have functions that accept either of these structures, and rely on the presence of the data key.#2017-01-0513:56thomasHi, just as an idea/question I had... is it possible the do clever things with date/times in spec. I assume you can say that a certain value should be an date/time instance... but wouldn't it be nice if you could express in a spec that a certain date/time should come before/after a different date/time ?#2017-01-0513:57thomasfor instance {:start #inst 2017-01-01 :end #inst 2017-01-5} and if the end is before the start it fails to validate?#2017-01-0514:00mpenetI guess you can do that with s/and#2017-01-0514:01mpenet(s/and (s/keys ....) #(start<end? %)) where the latter function checks the values#2017-01-0514:03thomasok cool @mpenet that looks like a good solution.#2017-01-0514:03mpenet
(s/def ::fancy (s/and (s/keys :req-un [::start ::end]) (fn [{:keys [start end]}] (> end start))))
:user/fancy
user> (s/def ::start int?)
:user/start
user> (s/def ::end int?)
:user/end
user> (s/valid? ::fancy {:start 0 :end 1})
true
user> (s/valid? ::fancy {:start 0 :end -1})
false
user> (s/valid? ::fancy {:start 1 :end 0})
false
#2017-01-0514:06thomasaah cool... I hadn't realised you can attach the function to the spec. that is very handy indeed#2017-01-0514:06mpenetspecs are just predicates really#2017-01-0514:07mpenetwell the building blocks at least#2017-01-0514:22Alex Miller (Clojure team)http://blog.cognitect.com/blog/2017/1/3/spec-destructuring#2017-01-0514:31Yehonathan Sharvit@alexmiller your article is related to this issue right? http://dev.clojure.org/jira/browse/CLJ-2003#2017-01-0514:32Alex Miller (Clojure team)in what way?#2017-01-0514:33Alex Miller (Clojure team)that is, no not in any way obvious to me :)#2017-01-0514:34Yehonathan SharvitOne reproduction of the issue is shown by @bbloom
(s/unform :clojure.core.specs/defn-args (s/conform :clojure.core.specs/defn-args '(f [& xs])))
;;; (f ((& xs)))
#2017-01-0514:35Alex Miller (Clojure team)oh, just another case. the bug is in spec itself, not in whatā€™s in the post.#2017-01-0514:36Yehonathan SharvitI never meant that there was a bug in the post#2017-01-0514:36Yehonathan SharvitšŸ˜Š#2017-01-0514:36Alex Miller (Clojure team)well, not sure why youā€™re asking then#2017-01-0514:37Yehonathan SharvitIā€™m asking because after reading your post, people might try to do a ā€œconform unformā€ loop and they might be surprised by the result...#2017-01-0515:18martinklepschIā€™ve been using collection-check https://github.com/ztellman/collection-check a while ago ā€” is something like that becoming easier with spec? Maybe there are generators etc for Ā·built-in data structures?#2017-01-0515:28joshjones@dacopare Given your data, you can still use unqualified keys in your existing data. The spec keys must be qualified, but not the keys in your map. They would look something like this:
(s/def :generic/event-type string?)
(s/def :generic/id string?)
(s/def :created/amount pos-int?)
(s/def :deleted/reason string?)

(s/def :created/data (s/keys :req-un [:generic/id :created/amount]))
(s/def :deleted/data (s/keys :req-un [:generic/id :deleted/reason]))

(s/def :created/event (s/keys :req-un [:generic/event-type :created/data]))
(s/def :deleted/event (s/keys :req-un [:generic/event-type :deleted/data]))

(s/conform :created/event created)
(s/conform :deleted/event deleted)
#2017-01-0515:45Alex Miller (Clojure team)@martinklepsch Iā€™m not sure spec adds much over collection-check, which has a pretty specific purpose. spec does allow you to gen from collection specs, but not sure if thatā€™s easier. depends what youā€™re trying to do.#2017-01-0515:45thegeez@alexmiller in the blogpost the result for the 3rd "WORK IN PROGRESS" block should be: (s/conform ::seq-binding-form '[a b & r :as s]) {:elems [a b], :rest {:amp &, :form r}, :as {:as :as, :sym s}}#2017-01-0515:46Alex Miller (Clojure team)ah, thx#2017-01-0515:47martinklepsch@alexmiller I basically just want to check if a thing behaves like a set, a vector, a list. Guess collection-check it still is then šŸ™‚ Thought maybe through specs for core it would have become easier to generate operations#2017-01-0515:50Alex Miller (Clojure team)yeah, nothing at the level of collection-check#2017-01-0518:20bbloomone thing i learned from your blog post was that simple-symbol? existed šŸ™‚ i had defined my own unqualified-symbol?#2017-01-0518:51manderson500 fake internet points to the person that spots the problem (took me all morning to debug this and thought it'd be more fun to make it a challenge simple_smile )
(spec/fdef run-thread!
           :args (spec/cat :fn (spec/fspec :args empty? :ret nil?)
                           :name string?)
           :ret #(instance? Thread %))

(defn run-thread!
  [f n]
  (let [t (Thread. ^Runnable f ^String n)]
    (.start t)
    t))

(defn my-runnable
  []
  (try
    (while true
      (Thread/sleep 1000)
      (println "Hello, World!"))
    (catch Exception e
      (println "Got an exception!" (.getMessage e) "Exiting..."))))

(clojure.spec.test/instrument)

(def t (run-thread! my-runnable "my-thread"))
#2017-01-0518:54bfabryname is optional in your fdef but not in your function?#2017-01-0518:55mandersonAh, good catch. That's an artifact of copy/paste (my original run-thread! had 2 arities where name was optional). But, that's not the issue that got me...#2017-01-0518:55mandersonfixed it šŸ™‚#2017-01-0518:56mandersonI think that's worth 50 fake internet points, though šŸ˜‰#2017-01-0518:57mandersonhint: it is related to spec...#2017-01-0518:57bfabryother than that seems to work for me. except of course there's not really any reason why either sleep or println would throw an exception#2017-01-0518:57mandersonwell, the catch exception is for killing the thread with (.interrupt t)#2017-01-0518:58mandersontry pasting the function definitions in the repl first and then hit enter. after that paste the instrument and def separately...#2017-01-0518:59manderson(just to make sure you are instrumenting on the newly added run-thread! spec)#2017-01-0518:59bfabrydoesn't that need to be `run-thread! for that?#2017-01-0519:00bfabryah, nope#2017-01-0519:01bfabryiunno, nothing particularly interesting is happening for me#2017-01-0519:01mandersonSome more detail: What this does is runs the my-runnable in the current REPL thread and never returns from run-thread! when I would expect the run-thread! to return immediately with the Thread object and the printlns happen in the background.#2017-01-0519:02mandersonReason: spec/instrument attempts to validate the my-runnable lambda per my fspec, before calling the function, which starts a continuous loop and the run-thread! fn is never called!#2017-01-0519:02mandersonLesson learned: don't instrument a "runnable" fn that doesn't terminate...#2017-01-0519:03bfabryah. I see. yes don't instrument functions with side-effects#2017-01-0519:03mandersonYep. Makes perfect sense now, but took me a while to figure it out šŸ™‚#2017-01-0519:04bfabryyeah I totally forgot, but it has been mentioned before#2017-01-0519:04mandersonThought I could save someone a potential headache by mentioning here...#2017-01-0519:04bfabrythere was talk of more coming for dealing with spec+side-effects maybe, but I'm not sure if it still is#2017-01-0519:05mandersonYeah, it's one of those things you know in the back of your mind, but forget when in the thick of debugging a weird error#2017-01-0520:53bbloomiā€™m not sure i understand what you say is happening here...#2017-01-0520:54bbloomi did this:#2017-01-0520:54bbloom(s/fdef f :args (s/and #(do (.printStackTrace (Exception.)) %) any?))#2017-01-0520:54bbloominstrument#2017-01-0520:55bbloomthen call f or (.start (Thread. f))#2017-01-0520:56bbloomfrom what i can tell, the spec-checking-fn is called by Thread.run#2017-01-0520:56bbloomwhich seems right to meā€¦ sooo whatā€™s the problem?#2017-01-0520:59joshjoneswhat are you trying to accomplish?#2017-01-0520:59joshjonesWhen I do this I get a stack trace, is this not what you want?#2017-01-0521:00bbloomI was trying to understand @manderson and bfabryā€™s discussion above#2017-01-0521:03manderson@bbloom problem is with the fspec. It's not intended for fn's with side effects, which a while true looping runnable most certainly is. When instrument is turned on, it attempts to validate the fspec by testing it out with different data to ensure it conforms. This ends up running the loop in the current thread and never actually executing the originally intended function.#2017-01-0521:03mandersonThrew me for a while, because it looked like it was running as intended, but was running in the current thread.#2017-01-0521:03mandersonAs part of instrumentation#2017-01-0521:04bbloomhuh? fspec should work just fine for functions with side effects outside of generative testing#2017-01-0521:04bbloomwhat ā€œdifferent dataā€ is it using?#2017-01-0521:06bbloominstrumented functions are only called once - with the same args they are given, and presumably on the same thread they would normally have been called on#2017-01-0521:07gfredericks@bbloom instrumenting uses random data to validate functions passed as an arg to the instrumented fn#2017-01-0521:07bbloomwait ā€¦ what?#2017-01-0521:08gfrederickse.g., if you spec clojure.core/map#2017-01-0521:08gfrederickswhose first arg is a function#2017-01-0521:08gfredericksthen you instrument map#2017-01-0521:08gfredericksthen you call it with some function#2017-01-0521:08gfredericksthat function will be test.checked as part of the arg-validation step#2017-01-0521:08bbloomwhoaaaa OK now it finally makes some sense why :fn and :ret arenā€™t checked by instrument#2017-01-0521:09gfredericksyeah? I haven't connected these two things#2017-01-0521:09bbloomi was under the impression the purpose of instrument was to make sure i wasn't calling other peopleā€™s stuff wrong#2017-01-0521:09bbloomand that it was also used during generative testing to validate my stuff#2017-01-0521:10bbloomi had no idea generative data was happening if i did instrument normally#2017-01-0521:10gfredericksif somebody wrote a clickbaity listicle of facts about clojure.spec this would be the "number seven will shock you"#2017-01-0521:10gfredericksit's not generatively tested the instrumented function#2017-01-0521:11gfredericksit's generatively testing fspec args to the instrumented function; so for non-HOFs this doesn't apply at all#2017-01-0521:11joshjonesā€œnumber three: ALL specā€™d keys in a map are validated, not only those explicitly stated as :reqā€˜d ā€¦ā€ šŸ™‚#2017-01-0521:11mandersonyep, i discovered this ^^^ when i had some println's for debugging and saw a bunch junk printed out i wasn't expecting. just didn't connect it with my issue above, but it hits home the point as to why fspec isn't for side effects#2017-01-0521:11mandersonthe generative testing of fspec, that is#2017-01-0521:11bbloomiā€™m pretty confused now.#2017-01-0521:12gfredericksis that the advice? don't use fspec at all for side-effecting functions?#2017-01-0521:12bbloomignore what i said - i still donā€™t understand what is happening here#2017-01-0521:12bbloomif i use stest/instrument#2017-01-0521:12bbloomand then call some instrumented functions from the repl#2017-01-0521:12bbloomare generators involved at all?#2017-01-0521:12bbloommy impression was no#2017-01-0521:12gfrederickspossibly#2017-01-0521:13bbloomā€¦ when? why?#2017-01-0521:13gfredericksthink about the spec for clojure.core/map#2017-01-0521:13gfredericks(s/cat :f (s/fspec :args (s/cat :x any?) :ret any?) (s/coll-of any?))#2017-01-0521:13gfrederickssomething like that#2017-01-0521:14gfredericksimagining it only took 2 args#2017-01-0521:14gfredericksif you instrument map, that means you're asking for its args to be validated#2017-01-0521:14gfredericksso you instrument and then from the repl you call (map inc [1 2 3])#2017-01-0521:14gfredericksso you must be expecting spec to validate that [1 2 3] matches (s/coll-of any?), but what do you expect it to do with inc?#2017-01-0521:15gfredericksit could merely check that inc is an IFn#2017-01-0521:15bbloomholy hell - thatā€™s what i would have expected: to just check ifn#2017-01-0521:15bbloomi expected only instrumentation of var calls#2017-01-0521:15gfredericksnumber seven will shock you#2017-01-0521:15bbloomi would not expect instrumentation of higher order functions - thatā€™s crazy#2017-01-0521:15bbloomclojure is just not prepared for that#2017-01-0521:15bbloomheh#2017-01-0521:16bbloomin the absence of chaperones, proxies, and all that other crazy platform stuff that racket contracts have#2017-01-0521:16bbloomin the presence of pervasive effects#2017-01-0521:16bbloomthatā€™s just nuts#2017-01-0521:17bbloomnnnow i understand - here i am trying to recreate the problem discussed above with thread/start etc, but it was only a problem with higher order functions in there#2017-01-0521:18joshjoneswell, thatā€™s what fspec is mostly concerned with ā€” hof#2017-01-0521:19mandersonAs example:
(spec/fdef test-fn 
           :args (spec/cat :str string? 
                           :fn (spec/fspec :args (spec/cat :x string?) :ret nil?)) :ret nil?)

(clojure.spec.test/instrument)

(test-fn "hi" #(println %))

U
O

6K1Y
sqD
4PsKAL4
1wbc5
1kq9bvc
wqgAN0gb
kdKAI1aE24
22r
2
6TUvuD2x8c
95NKQmTX6c7kdTO
98O0dGrT9vr65
PE5712237F7
6
Rwc93xcj510gn7
MOvNSAJl
Received x: hi
hi
=> nil
#2017-01-0521:19mandersonSo, if #(println %) was instead a fn with side effects (looping, hitting a db, etc), then you can see the problem...#2017-01-0521:19potetmInteresting. So, long story short: HOFs are "instrumented" via generative testing?#2017-01-0521:20bbloomyeah#2017-01-0521:20bbloomseems really weird to me#2017-01-0521:20potetmI'd never considered it directly. I think deep down I was thinking they would be proxied.#2017-01-0521:20mandersonso, in my case, i was seeing my runnable fn executing, and it looked like it was functioning as normal, except it was running in the current thread instead of in a new thread.#2017-01-0521:20bbloomyeah proxied makes more sense#2017-01-0521:21potetmBut I understand the argument for non-proxied verification.#2017-01-0521:21bbloomie you check itā€™s ifn? or fn? and then wrap it with clojure.spec.test/spec-checking-fn (private function that does the proxying)#2017-01-0521:21bbloomi don't#2017-01-0521:21bbloomthis makes instrument much less useful for repl use#2017-01-0521:22mandersonyep, i went back and changed some of my specs to #(instance? IFn %)#2017-01-0521:22gfredericksrich talked about the proxying approach in a ML thread somewhere#2017-01-0521:22potetmTBH, I'm not sure what utility there is in specing a side-effecting function#2017-01-0521:22bbloombefore i look for that thread - the reason i wouldnā€™t expect proxying is b/c it breaks equality potentially#2017-01-0521:22gfredericksI think he thought it was difficult to design well because of having to track the proxy as it flies around the program#2017-01-0521:23potetmYeah that's^ what I was thinking.#2017-01-0521:24gfredericksonce you use proxying, the validation of your function can happen anywhere at any time, not just at the moment of calling it#2017-01-0521:24potetm"We do verification here and only here."#2017-01-0521:24gfrederickse.g., with map, it would immediately return a lazy seq#2017-01-0521:24gfredericksand then blow up later when somebody consumed it? or not blow up because the function call already ended?#2017-01-0521:24potetmCreates separation in execution.#2017-01-0521:24gfredericksif you unstrument before consuming the lazy seq what should it do?#2017-01-0521:24bbloomthis is all why my assumption was that instrument only instrumented vars#2017-01-0521:25potetmright#2017-01-0521:25bbloomie it was a better than nothing interactive tool#2017-01-0521:25bbloomcover the 90% case, rather than attempt the 100% case with fallout such as ā€œcanā€™t use instrument if you fspec any side effecting functinos"#2017-01-0521:25potetmYeah the fact that it does that by default and "transparently" is... unexpected.#2017-01-0521:26bfabryinstrument does only instrument vars#2017-01-0521:26bbloomwell, i mean only calls to vars and not use the generators#2017-01-0521:27bfabrybut, instrumentation means validation of arguments to a function, if one of your argument is an fspec, validation necessarily means generative testing#2017-01-0521:27bbloomfound http://dev.clojure.org/jira/browse/CLJ-1936#2017-01-0521:27potetmcode is data, data is code bruh#2017-01-0521:27bfabryso, don't write an fspec for functions that could potentially be side-effecting šŸ™‚#2017-01-0521:28gfredericks@bbloom I think I'm remembering a long thread on the clojure ml (not -dev) from 3-9 months ago#2017-01-0521:32bfabryI suppose the other thing you could do is stub out specs for functions that are potentially side-effecting#2017-01-0521:32potetmThat seems to be the intended path^#2017-01-0521:33bfabryhopefully the next screencast/blog on spec is advice on impure functions šŸ™‚#2017-01-0521:37potetmYeah.... it's still interesting to me that they would commit to verifying HOF args. I think see the argument (pun definitely intended), but "if 'x is on classpath' do 'y' else 'z'" seems complex.#2017-01-0521:38potetmI mean, I think it brings back up the question: "What's the relative utility of specing side-effecting fns?"#2017-01-0521:39potetmIf the answer is "so low it's not worth specing them," then committing to verifying HOFs makes a lot more sense to me.#2017-01-0521:40bbloomitā€™s only low value if you view spec-ing as a testing mechanism and not as developer guard rails#2017-01-0521:41potetmYeah perhaps. OTOH, if you view side-effects as poison, no guard rail will save you.#2017-01-0521:41bbloomi donā€™t view side effects that way tho#2017-01-0521:42potetmYeah, Rich has said exactly that I think (in Simple Made Easy)#2017-01-0521:42bbloomyou can say that the function passed to reduce should be effect-free, but itā€™s frequently useful to gather side data or whatever#2017-01-0521:42potetm(Not saying it's right or wrong. But the decision makes sense when viewed with that mindset I think)#2017-01-0521:43bbloomi think the decision makes sense when you consider that conform is trying to say ā€œyup, this thing is valid"#2017-01-0521:43bbloomvs ā€œas far as i can tell, this thing isnā€™t invalid"#2017-01-0521:43bbloomthe former requires checking everything, the later doesn't#2017-01-0521:43bbloomif you want to validate some data off the wire for security reasons, you need the former#2017-01-0521:44bbloombut that doesnā€™t work in the presence of higher order functions or mutation - as the racket contracts folks learned, hence proxies, chaperones, etc - and still, they have problems with side effects b/c they donā€™t have an effect handlers system#2017-01-0521:45bbloom@gfredericks I realize Iā€™m a heretic in clojure-land, but i abuse side effects quite frequently šŸ˜›#2017-01-0521:46potetmI've never used it that way either šŸ™‚ I've use reductions when I'm curious what happens during reducing, but not side-accumulation.#2017-01-0521:46joshjonesI can also say Iā€™ve only ever used the reducing function to .. well, reduce the collection at hand#2017-01-0521:46gfredericks@bbloom you're a monster#2017-01-0521:47joshjones@bbloom can you give an example of a side-effecting reducing function? (like a common use case for what youā€™re describing)#2017-01-0521:48gfredericks@bbloom do you sprinkle local atoms all over the place and swap them a bunch and then deref them to figure out what happened#2017-01-0521:48bbloomiā€™ve done stuff like that - yeah#2017-01-0521:48gfredericksokay.#2017-01-0521:48bbloomif itā€™s local to a function or a concrete process, itā€™s totally fine#2017-01-0521:49bbloomā€œtree falls in a forrestā€ and all that#2017-01-0521:49potetmAnyways, not sure I understand what @bbloom was trying to say about conformers with regard to the decision to validate higher order fns, but it makes some amount of sense to me having thought about it a minute.#2017-01-0521:49bbloomespecially when perf is involved#2017-01-0521:49bbloomusing map/filter/reduce etc in 9 passes over a long sequence is just not OK for some use cases#2017-01-0521:50bbloomiā€™d love to be able to tell the compiler ā€œhey, these three things traverse the same data structure, do loop fusionā€ but thatā€™s just not realistic#2017-01-0521:50joshjonestransducers in the 9 pass case! šŸ™‚ (if applicable)#2017-01-0521:50potetmYeah, I'm curious how that's more useful than loop/recur or doseq+accumulator.#2017-01-0521:50bbloomi do the loop/recur or doseq thing plenty too#2017-01-0521:50potetmYeah me too#2017-01-0521:50bbloombut sometimes yo ujust already have a function, so you just call reduce šŸ™‚#2017-01-0521:51potetmlol truestory#2017-01-0521:51bbloomanyway - this is waaaay besides the point#2017-01-0521:52bbloompoint is that valid? wants to return true, but really the best it can do in the presence of fspec is say ā€œmaybe true"#2017-01-0521:52bbloomalso, i gotta run#2017-01-0521:52bbloomthanks all for the discussion#2017-01-0600:36uwohow would you recommend handling this? The map Iā€™m validating has an attribute whose value is an entity. Depending on the context, at times that entity will have only one required attribute, and at other times it will have many required attributes. So, given two forms:
(s/def ::form1 (s/keys :req [::attr1]))
(s/def ::form2 (s/keys :req [::attr1]))

;; what we want in form 1                       
(s/def ::attr1 (s/keys :req [::attr-a]))
;; what we want in form 2
(s/def ::attr1 (s/keys :req [::attr-a ::attr-b]))
It seems to me, weā€™ll have to create another version of ::attr1, but that seems problematic because we use the ::attr1 key to mean the same entity across the project.
#2017-01-0601:00bbloomthere are a few things you can do that depends on your situation#2017-01-0601:00bbloomthe simplest is just make the ā€œsometimes requiredā€ fields optional and call that good enough, if it meets your validation needs#2017-01-0601:01bbloomanother thing you can do is to have decorated and undecorated versions of the data, and just put them at two differently named keys#2017-01-0601:02bbloomso instead of (update m :foo assoc :bar 123), you do (assoc m :foo-bared (assoc (:foo m) :bar 123)#2017-01-0601:02bbloomthen use foo vs foo-bared as appropriate#2017-01-0601:03bbloomiā€™ll leave more complex solutions for others to discuss šŸ™‚#2017-01-0604:30joshjones@uwo The part that interests me is the "depending on the context" -- you might be tempted to do something like this:#2017-01-0604:30joshjones
(s/def ::attr-a any?)
(s/def ::attr-b any?)

(s/def ::entity (s/or :multiple (s/keys :req [::attr-a ::attr-b])
                      :single (s/keys :req [::attr-a])))

(s/def ::form (s/keys :req [::entity]))
#2017-01-0604:31joshjoneshowever, this does not do you any good, as in a context where you need multiple the single will still match ... can you be more specific on "context"?#2017-01-0604:31uwo@bbloom thanks. Iā€™ve got to be strict about required versus optional because Iā€™ve got to bark at the user if they donā€™t provide a field in a particular context. Iā€™ll have to mull over what the decorated versus undecorated means#2017-01-0604:35uwo@joshjones thanks. the ā€œcontext" is two separate forms, one of the forms has a subset of the fields in the other. Both forms describe the same entities. Itā€™s just one form requires more than the other. Iā€™m loath to change to name (or ns) of the attribute, because itā€™s really the same entity#2017-01-0604:35bbloom@uwo is your form necessarily hierarchical?#2017-01-0604:35uwoyeah, it gets very nested#2017-01-0604:36bbloomhm, yeah, so this is something was talking about with @gfredericks i believe last week or so: parameterized specs#2017-01-0604:37bbloomor maybe it was @seancorfield#2017-01-0604:37uwoI havenā€™t played it out fully yet, but I think the s/or may work @joshjones#2017-01-0604:37bbloomthis article is relevant: http://blog.ezyang.com/2013/05/the-ast-typing-problem/#2017-01-0604:37uwo@bbloom yeah, I was wondering if there was some avenue to parameterization#2017-01-0604:37joshjonesso, what is the criteria, in plain english, for choosing map spec one, versus map spec two? It sounds like you are saying "map spec one has these keys" and "map spec two has these keys" ... but the presence or absence of keys is not really an appropriate way to spec it. If you have a key itself, something like {:data {...} :type "A"} .. then you can identify which "version" of the map you have on hand#2017-01-0604:38joshjonessaid another way, you have to bark at the user if they don't provide a field, in WHAT context? how do you know?#2017-01-0604:39joshjonesthe s/or won't work, for the reason i described ... the single will always match, regardless of your context#2017-01-0604:40uwo@joshjones there are two separate forms on different pages. I was simply going to validate against a different aggregate spec on each. Do I understand your question?#2017-01-0604:40bbloomas a total NON SUGGESTION, but just for funsies: i bet you could hack a (very bad) solution with s/conformer and s/andā€¦ idea is you assoc in a parameter and then use arbitrary code to look for it ā€” oh man, thatā€™s an evil hack, i kinda wanna try it.... for science....#2017-01-0604:40joshjones@bbloom we've established you're a heretic, please cease your evil ways#2017-01-0604:41bbloomnever.#2017-01-0604:41uwo@joshjones oh oh. lol reading comprehension fail. I just looked at your example the first time around. yeah, youā€™re right thatā€™s no good šŸ˜Š#2017-01-0604:41uwofor science...
#2017-01-0604:41joshjonesare the keys in the maps namespace-qualified? or no?#2017-01-0604:42uwoyes#2017-01-0604:42bbloomif you use un-qualified keys, you can build up a library of specs for non-recursive parts and then manually recreate all the recursive bits#2017-01-0604:42bbloombut really, thereā€™s probably a better representation#2017-01-0604:42bbloomespecially a non-recursive one#2017-01-0604:42uwohehe. using qualified keys. Theyā€™re the same names we use in datomic#2017-01-0604:43joshjonesi suggest two specs then .. it's the most straightforward, and models your use case#2017-01-0604:43joshjonesare you checking the spec as a function arg, or what?#2017-01-0604:44bbloomiā€™d like to know more about your use case, maybe we can propose a simplification#2017-01-0604:44uwoDo I misunderstand spec? Can I have two specs for the same (namespaced) key?#2017-01-0604:44bbloomno, you canā€™t, which is an intentional design decision#2017-01-0604:44bbloomand a pretty good one at that šŸ˜‰#2017-01-0604:44uwocool. thatā€™s what I assumed#2017-01-0604:44joshjonesare you checking the spec as a function arg?#2017-01-0604:44bbloomthe challenge is that specs arenā€™t really context sensitive in any useful way#2017-01-0604:45bbloomalso probably on purpose, but questionable if you actually do care about context#2017-01-0604:45uwo@joshjones sorry not sure I follow.#2017-01-0604:45bbloomif i were you, iā€™d probably use spec in a context-insensitive way#2017-01-0604:45joshjoneshow do you plan to use this spec?#2017-01-0604:45bbloomie make everything optional#2017-01-0604:45bbloomand then check required independently#2017-01-0604:45bbloomie donā€™t use spec for requiredā€™s take#2017-01-0604:45bbloomif your required fields are context sensitive, then the fields are optional from the perspective of spec#2017-01-0604:46bbloomyou can conform and then ALSO traverse yourself to check required#2017-01-0604:46uwoah. grab the underlying data structure from the form and then verify it against the spec#2017-01-0604:46joshjonesand something in the data structure identifies it as one version, or the other?#2017-01-0604:46bbloomsure#2017-01-0604:46uwo@bbloom hmm. yeah, thatā€™s a possibility#2017-01-0604:47bbloommy apologies for my goofy hacky mood .... this is real advice: donā€™t try to force spec to do 100% of the work. use it for whatever % you can get away with, and write your own code to do the rest#2017-01-0604:47bbloomclojure is still really good at recursive functions šŸ˜‰#2017-01-0604:48uwo@joshjones not really no. one form is a proper subset of the other. And, as forms go, validations run while theyā€™re partially filled out, so thereā€™s really no way to tell a difference from the specs perspective. Of course, I know what form Iā€™m on, and so could call with an appropriate spec#2017-01-0604:48joshjoneshow do you know what form you're on?#2017-01-0604:48uwo@bbloom heh. thanks. we already have an implementation that doesnā€™t leverage spec very much. I was just wondering if I could improve what we have#2017-01-0604:50uwo@joshjones I think we may be passing around a form key that further identifies it, now that I think about that šŸ˜Š#2017-01-0604:50joshjonesone of two things should happen: 1) since you know at this point in your code which type of data you're dealing with, then you should have a different spec for each, and validate against that spec. 2) your map itself should contain a key which specifies which type of map this is. in this case, use a multi spec#2017-01-0604:50bbloomyou can use conform at the leaves of your own recursive validation function#2017-01-0604:52joshjonesis the form key a clojure keyword, or something else?#2017-01-0604:54uwoyes#2017-01-0604:56joshjonesdoing the spec, give me a sec#2017-01-0605:00joshjones
(s/def ::attr-a any?)
(s/def ::attr-b any?)

(s/def ::id-key keyword?)
(defmulti map-type ::id-key)

(defmethod map-type ::single-key-version [_]
  (s/keys :req [::attr-a]))
(defmethod map-type ::multi-key-version [_]
  (s/keys :req [::attr-a ::attr-b]))

(s/def ::some-map (s/multi-spec map-type ::id-key))

(s/conform ::some-map {::id-key ::single-key-version ::attr-a 42 ::attr-b "abc"}) ; valid
(s/conform ::some-map {::id-key ::single-key-version ::attr-b "abc"})             ; invalid
(s/conform ::some-map {::id-key ::multi-key-version ::attr-a 42 ::attr-b "abc"})  ; valid
(s/conform ::some-map {::id-key ::multi-key-version ::attr-a 42})                 ; invalid
#2017-01-0605:03uwoI can see the multispec approach working. So, Iā€™d need an additional key in the model, which Iā€™m guessing would be transitory (I wouldnā€™t persist it to the db)#2017-01-0605:04uwothanks, by the way. I really appreciate your time!#2017-01-0605:04uwo@joshjones @bbloom ^#2017-01-0605:05bbloomyeah, so multi-spec works, but you need to eliminate context for nested stuff#2017-01-0605:05joshjonesyou are welcome @uwo#2017-01-0605:05bbloomone way to do that is to walk the tree and add a key to each node in the tree with the type#2017-01-0605:06bbloombasically pre-load all the context, so that the spec can be context-free#2017-01-0605:11bbloomiā€™m working up an example of that for my own sake - almost ready#2017-01-0605:16uwotoo bad I canā€™t key a multispec off of a type key in a parent map. Because there are so many (nested) entities on some of the forms, Iā€™d have to annotate each one of them with a dispatch key for the multispec.#2017-01-0605:16bbloomyeah - thatā€™s what i was saying about spec being context-insensitive#2017-01-0605:16bbloomitā€™s also what i was saying about the hack with conformers šŸ˜‰#2017-01-0605:16uwosince itā€™ll be the same key though, I could, like you said, just walk the tree and toss the context around#2017-01-0605:17uwo(yeah that hack went over my head, sorry šŸ˜„ )#2017-01-0605:18bbloombasically using conformer to automate the annotation#2017-01-0605:18bbloomitā€™s a dirty dirty hack šŸ˜›#2017-01-0605:18bbloomconformer lets you change the value that flows through spec#2017-01-0605:18uwocrazy#2017-01-0605:19bbloomso you could do something like (s/and (s/conformer (fn [x] (assoc x :context foo))) ::node)#2017-01-0605:19bbloomnow ::node can see :context#2017-01-0605:21bbloomhttps://gist.github.com/brandonbloom/59d3e0d002f34b67f3ee2e99224745fa#2017-01-0605:21bbloomthere, that seems to work#2017-01-0605:21bbloomi quite like that#2017-01-0605:21bbloom@uwo & @joshjones ^^#2017-01-0605:22bbloommake-fancy basically adds the context to each node, such that the specs no longer need be context sensitive#2017-01-0605:27uwothanks, Iā€™ll have to mull that over. The form Iā€™m working with isnā€™t recursively structured, but thatā€™s still useful#2017-01-0605:28bbloomrecursive/nested/whatever#2017-01-0605:28bbloomsame idea#2017-01-0605:28uwotots#2017-01-0605:28bbloomsending data down the tree#2017-01-0605:28uwoworks with pre/post walk šŸ˜„#2017-01-0605:28bbloomindeed#2017-01-0605:28joshjonescool#2017-01-0605:28uwowell thanks again, both! Iā€™m out for the night.#2017-01-0605:29joshjonesone last thought @uwo -- take a step back, and ensure that any added complexity is helpful rather than harmful. nite šŸ™‚#2017-01-0605:29joshjonesas complexity is the enemy of good software, and of clojure itself šŸ˜‰#2017-01-0605:29bbloomlol yes, that#2017-01-0605:29uwoexactly @joshjones#2017-01-0605:30uwošŸ‘‹#2017-01-0615:08carocad@kenny are you sure that your defspec-test macro works? I tried it with my specs and it always succeeds which is weird because if I use s/check some tests fail šŸ˜•#2017-01-0615:48lmergencould someone elaborate what exactly the difference between :ret and :fn are with fdef ?#2017-01-0615:48lmergen:fn feels more like a higher level qualifier, while :ret describes a type#2017-01-0615:48lmergenis that correct ?#2017-01-0615:50lmergenwhen i look at the spec documentation, it uses a ranged-rand example, and i see this: > The :ret spec indicates the return is also an integer. Finally, the :fn spec checks that the return value is >= start and < end. sounds like the :fn spec passing implies the :ret spec passing ?#2017-01-0615:50lmergenso then why define :ret as well ?#2017-01-0615:50Alex Miller (Clojure team):ret describes the return value#2017-01-0615:50Alex Miller (Clojure team):fn receives the conformed values of both args and return and can thus validate more complicated relationships between inputs and output#2017-01-0615:51lmergenaha!#2017-01-0615:51lmergenahhh, i see now#2017-01-0615:51Alex Miller (Clojure team)In that example, :ret can only say that the return value is an integer, but :fn can say how it relates to the args#2017-01-0615:51lmergenyes, okay#2017-01-0615:51lmergenthis makes sense#2017-01-0615:52lmergenand feels extremely powerful#2017-01-0615:52lmergenthanks alex#2017-01-0617:06carocadhas anyone tested the defspec-test macro that is pinned on this channel? For some reason it always succeeds even if I give it garbage specs šŸ˜ž#2017-01-0618:47kenny@carocad It works. Can you give an example?#2017-01-0619:22carocad@kenny the following should break but it doesnt:
(s/def ::lat (s/and number? #(<= -90 % 90)))
(s/def ::lon (s/and number? #(<= -180 % 180)))

(def RADIOUS 6372800); radious of the Earth in meters
(defn haversine
  [^double lon-1 ^double lat-1 ^double lon-2 ^double lat-2]
  (let [h  (+ (Math/pow (Math/sin (/ (- lat-2 lat-1) 2)) 2)
              (* (Math/pow (Math/sin (/ (- lon-2 lon-1) 2)) 2)
                 (Math/cos lat-2)
                 (Math/cos lat-1)))]
    (* RADIOUS 2 (Math/asin (Math/sqrt h)))))

(s/fdef haversine
  :args (s/cat :lon-1 ::lon :lat-1 string?
               :lon-2 ::lon :lat-2 ::lat)
  :ret ::dist)

(defspec-test test-haversine      [haversine] {:clojure.spec.test.check/opts {:num-tests 50}})
#2017-01-0619:24carocadwhen I run lein test it says run 4 test containing 4 assertions. 0 failures, 0 errors#2017-01-0619:24carocadam I doing something wrong?#2017-01-0619:24carocadI omited the namespaces to make it short#2017-01-0619:27carocadin case you want to further test, you can find the original specs here: https://github.com/carocad/hypobus/blob/master/src/hypobus/conjectures/specs.clj and the tests here: https://github.com/carocad/hypobus/blob/master/test/hypobus/basic_test.clj#2017-01-0619:32kenny@carocad The parameters passed to it should be the same as the parameters passed to clojure.spec.test/check -- you need to pass a fully qualified symbol.#2017-01-0619:58carocad@kenny as I mentioned, I ommited the ns in this case only for brevity. The real test have the fully qualified symbol. see https://github.com/carocad/hypobus/blob/master/test/hypobus/basic_test.clj#L54#2017-01-0619:58kennyI just tested your code and it fails with a ClassCastException šŸ™‚#2017-01-0620:00kenny@carocad You need to quote your symbols#2017-01-0620:00kenny
(defspec-test test-haversine      [`hypobus.basics.geometry/haversine] {:clojure.spec.test.check/opts {:num-tests 50}})
#2017-01-0620:01kennyParams are exactly the same params you'd pass to clojure.spec.test/check šŸ™‚#2017-01-0620:04carocadoh I see @kenny. Thanks a lot šŸ™‚ I took the sample code from https://stackoverflow.com/questions/40697841/howto-include-clojure-specd-functions-in-a-test-suite, and since it was the same code I assumed that I was correct. My mistake#2017-01-0620:05kennyNo link to the gist on SO. The gist is here: https://gist.github.com/kennyjwilli/8bf30478b8a2762d2d09baabc17e2f10#2017-01-0620:10carocadI didnt know there was a gist as well. I added a comment on how to use it for those with not much macro understanding like me šŸ™‚#2017-01-0620:16schmeewhat does the retag argument do in multi-spec?#2017-01-0620:16schmee> retag is used during generation to retag generated values with matching tags. retag can either be a keyword, at which key the dispatch-tag will be assoc'ed, or a fn of generated value and dispatch-tag that should return an appropriately retagged value.#2017-01-0620:16schmeesays the docs, but I think I need an example to understand what that means#2017-01-0620:18schmeeahh, okay, now I see it šŸ™‚#2017-01-0621:00carocad@kenny if you dont mind ... could you help me a bit further. I get a weird java.lang.ClassCastException when running lein test. It seems to be a compilation error though šŸ˜ž#2017-01-0621:07kennyTry changing failure# to (throw failure#) (line 20 in the gist)#2017-01-0621:15carocadI still get java.util.concurrent.ExecutionException: java.lang.ClassCastException: clojure.lang.AFunction$1 cannot be cast to clojure.lang.MultiFn, compiling:(clojure/test/check/clojure_test.cljc:95:1)#2017-01-0621:35Alex Miller (Clojure team)maybe youā€™re running into the lein monkeytest problem#2017-01-0621:36Alex Miller (Clojure team)yeah, itā€™s this: https://github.com/technomancy/leiningen/issues/2173#2017-01-0621:37Alex Miller (Clojure team)just add this to your lein project.clj: :monkeypatch-clojure-test false#2017-01-0621:37Alex Miller (Clojure team)and you should be good @carocad#2017-01-0621:37Alex Miller (Clojure team)itā€™s a conflict with how test.check and lein are both modifying clojure.test#2017-01-0621:38carocad@alexmiller I just found that bug report in github. Indeed I just tried that and it worked šŸ™‚#2017-01-0621:38Alex Miller (Clojure team)btw, it is fixed in test.check too for next release of that lib#2017-01-0621:39carocad@alexmiller , @kenny You saved a very frustrated man, thanks for the help and the heads up#2017-01-0621:42Alex Miller (Clojure team)comes up here about once a month :)#2017-01-0622:08schmeecan I declare ā€œdependenciesā€ between keys in a map?#2017-01-0622:09schmeefor example,
(def m {:url ""
        :top-level "com"})

(= (:top-level m) (-> (str/split (:url m) #"\.") last))
can I have this in the key specs somehow?
#2017-01-0622:09schmeeor is this the business of function specs?#2017-01-0622:10schmeethe reason Iā€™m asking that I want to generate data with this property#2017-01-0622:10bfabry@schmee I think that's exactly what s/and is for#2017-01-0622:11schmeeI donā€™t see how s/and helps here, since keys have to be speced separately#2017-01-0622:12bfabry(defn top-level-matches-url? [m] (= (:top-level m) (-> (str/split (:url m) #"\.") last))) (s/def ::my-map (s/and (s/keys :req-un [::url ::top=level]) top-level-matches-url?))#2017-01-0622:12bfabryor maybe I'm misunderstanding what you're saying#2017-01-0622:14schmeeooooooohhhhh#2017-01-0622:14schmeeI never though about using s/and with s/keys that way šŸ˜„#2017-01-0622:15schmeeIā€™m gonna try that out, thanks!#2017-01-0622:15bfabrynp#2017-01-0622:16schmeeI guess it will be tricky to write generators for stuff like this#2017-01-0622:16schmeeitā€™s fine if you have one relation like this in the map, but if you have 10 or more...#2017-01-0622:20bfabryoh totally as soon as you spec something like that you're going to need to do custom generators#2017-01-0622:21schmeeI guess I can write separate generators for the special relations, then generate a sample from the spec itself and merge all the special cases onto that#2017-01-0622:22schmeethat way I can still use most of the default generator and just "add onā€ the exceptions#2017-01-0622:22bfabryI don't know much about it sorry. I'm holding out for beta before we switch to 1.9 and start using it in anger. there was a screencast about custom generators though ĀÆ\(惄)/ĀÆ#2017-01-0622:22schmeeonce you get in the mindset theyā€™re quite fun to write actually#2017-01-0622:23schmeeit always feels like magic when you get to see it in action šŸ˜„#2017-01-0622:24gfredericksDependencies in a map can likely be generated by wrapping the map generator in gen/fmap#2017-01-0623:40crimeministerI was looking for a way to serialize a clojure.spec to return from an API and stumbled across this nifty-looking library: https://github.com/uswitch/speculate#2017-01-0623:41crimeministerAny idea if similar functionality is likely to ship with spec itself?#2017-01-0623:49schmeecrimeminister if I interpret this correctly, that seems unlikely: http://clojure.org/about/spec#_code_is_data_not_vice_versa#2017-01-0623:53crimeministerThanks for the link @schmee, it was instructive#2017-01-0700:37Alex Miller (Clojure team)@crimeminister: the intention is that s/form will return you something serializable#2017-01-0700:38Alex Miller (Clojure team)(As Clojure data)#2017-01-0700:38Alex Miller (Clojure team)There are some known bugs with that that are in process#2017-01-0700:39Alex Miller (Clojure team)Also there will be specs for spec forms#2017-01-0700:39Alex Miller (Clojure team)Which you can conform to retrieve data#2017-01-0701:06crimeministerSounds great, thank you @alexmiller #2017-01-0701:07crimeministerSounds like that will serve admirably for what I need#2017-01-0710:21schmeeis there a way to access the default gen when overriding a gen in s/spec?#2017-01-0710:43schmeeI guess my question above is still reasonable, but I solved my problem by reading the docs and realising that the retag arg for a multi-spec can be a function#2017-01-0714:32Alex Miller (Clojure team)The only way to do it atm is to separate the spec into a var (or different registry spec) so you can refer to it #2017-01-0715:05stathissiderisIā€™m trying to use :clojure.core.specs/arg-list to parse an arg list, but the clojure.spec/every of :clojure.core.specs/map-bindings means that I canā€™t fully destructure the arg-list with conform#2017-01-0715:05stathissiderisis there a way to force the every to conform?#2017-01-0716:50zaneNot that I know of, @stathissideris. If you want that behavior you'll probably have to roll your own with clojure.spec/coll-of.#2017-01-0716:51stathissideris@zane thanks! I was hoping to use the ā€œofficialā€ clojure spec to parse the args, but it looks like Iā€™ll have to copy and modify it#2017-01-0716:52zaneThat's what it looks like to me.#2017-01-0716:52zaneYou could always just validate with clojure.spec/valid? and :clojure.core.specs/arg-list and then write your own transformation function.#2017-01-0716:55stathissiderismy main use case is not validation, Iā€™d like to extract the names of arguments (and be able to handle all cases of destructuring etc)#2017-01-0812:30luxbockI'm a bit surprised that :clojure.core.specs/seq-binding-form doesn't restrict itself to only acceting vectors, i.e. you can attempt to destructure {(foo) :something} and it fails on because of an assert in clojure.core/destructuring rather than doing so at the spec level#2017-01-0813:26stathissideris@alexmiller in your recent article on the destructuring spec you mention > Rather than recursively parsing the binding form, we could simply conform it to receive a more regular structure described in terms of the parts we've defined in the spec. ā€¦but thatā€™s not entirely true because the use of every would prevent conform to recur to deeper levels to find the ::binding-form used within ::map-binding for example. Right?#2017-01-0814:16mpenetanyone toyed with https://github.com/uswitch/speculate ?#2017-01-0814:17mpenetSeems great on paper.#2017-01-0815:11lmergenso, i was under the impression that fdef instruments a function and ensures that its args and return values conform to the spec on all invocations, but i think iā€™m wrong#2017-01-0815:11lmergeni think i still need to manually ensure conform somewhere, right ?#2017-01-0815:12lmergenfor what itā€™s worth, iā€™m porting a schema-based codebase to spec, and want something similar to schemaā€™s ^:always-validate#2017-01-0815:13lmergenof course i could always use :pre and :post#2017-01-0815:14mpenetinstrument is a dev time thing really (it can trigger gen). To get ret validation you need to call check#2017-01-0815:15mpenetand yeah, pre/post assertions is the closest to what you want i think#2017-01-0815:15lmergenyeah but i am one of those people who runs its assertions in production šŸ™‚#2017-01-0815:16lmergenhttps://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec.test/instrument#2017-01-0815:16lmergenthatā€™s probably it, eh ?#2017-01-0815:18mpenetNo, it s not a production usable thing. It ll trigger gen and be super slow/heavy. It doesnt just wrap args with simple validators#2017-01-0815:18lmergenok#2017-01-0815:18lmergenthanks, saves me some headaches šŸ™‚#2017-01-0815:19mpenetI wish we had another instrument for that personally, but now you have to roll your own basically#2017-01-0815:20lmergenwell, something like schemaā€™s defn overload would be great#2017-01-0815:20mpenetYup#2017-01-0815:30lmergenis anyone aware of some utility that does this ? basically all i want is to call s/explain whenever some pre or post condition is not matched#2017-01-0815:30lmergeni might make it myself#2017-01-0815:31lmergenit would make sense to me to have a function like this that has the same signature as fdef#2017-01-0816:19brabster@lmergen something to make :pre/:post call s/explain sounds useful#2017-01-0908:12pyrHi#2017-01-0909:29pyrdoes anyone have an example showing how they wired generative tests in their test suite?#2017-01-0909:29pyrI'm using exercise-fn and instrument at the repl but wanted to know if there was a standard way of wiring that in tests#2017-01-0909:32mpenetI guess there are a few ways to do this: one would be just to call instrument on "normal" tests, then another could be replacing your test values with generated versions of it, which might or might not be difficult depending on what it is.#2017-01-0909:33mpenetI haven't done any of it so far personally. Well actually just the instrumentation part on 1 project.#2017-01-0909:33mpenetclj jdbc and alia both have instrumentation on in the test suite for instance#2017-01-0909:35pyrcool, will look at that, thanks!#2017-01-0909:44lmergenso, as an alternative of calling s/explain on all pre or post conditions, what about calling s/assert rather than s/valid ?#2017-01-0909:44lmergene.g.
(defn new-web-server [config]
  {:pre [(s/assert ::config config)]}
  (-> (map->WebServer config)
      (using [:routes])))
#2017-01-0909:45lmergeni know the semantics aren't very clean (:pre is supposed to return true/false rather than an exception), but it does contain a lot more information#2017-01-0910:36pyr@mpenet Integrating with the output of check works well#2017-01-0910:36pyrHow do you go about generating fixed size arrays quickly, usually?#2017-01-0910:37mpenet@pyr gen bible: https://github.com/clojure/test.check/blob/master/doc/cheatsheet.md#2017-01-0910:38mpenetdepends if you mean primitive array or vector/list#2017-01-0910:39pyrprimitive in this case, but the cheatsheet will help, thanks again#2017-01-0910:40mpenetyou can generate a list of ints then feed it in byte-array in a gen/fmap for instance#2017-01-0910:40mpenetthere are probably easier ways#2017-01-0910:42pyr(tgen/resize 64 (gen/bytes)) does the trick#2017-01-0910:43mpenetnice#2017-01-0911:09pyr@mpenet (actually not at all, so I'm left with a gen/bind on top of a vector)#2017-01-0911:14mpenetI had something like this in mind: (gen/fmap byte-array (gen/vector gen/byte 10))#2017-01-0911:14mpenet@pyr ping#2017-01-0911:15mpenetprobably very similar to what you have#2017-01-0911:16mpeneta gen/bytes* with a size argument would be nice tho#2017-01-0911:56pyr@mpenet this is what I have, yes#2017-01-0911:59pyr(except there's no gen/byte)#2017-01-0912:00mpenetthere is in clojure.test.check/gen#2017-01-0912:00pyrah#2017-01-0912:00mpenethttps://clojure.github.io/test.check/clojure.test.check.generators.html#var-byte#2017-01-0912:01pyrindeed, that's much better#2017-01-0912:02pyrthen what you have works out of the box and is much shorter than what I had#2017-01-0912:02mpenetclojure.spec aliases some of test.check but not all of it#2017-01-0912:03mpenetI guess the idea is to (maybe) someday promote most of it in spec itself, but I could be wrong#2017-01-0912:04mpenettest.chuck is also awesome for more gen utils#2017-01-0912:04mpenetstring-from-regex for instance was quite the time saver#2017-01-0912:19pyrA last question for now, is it possible to specify custom generators only for clojure.spec.test/check#2017-01-0912:20pyrIt's only a matter of visual preference, to be able to have simple specs in my main namespaces and specs with generators for tests#2017-01-0912:23mpenetYup, I think so, test/check takes a number of options for this#2017-01-0912:25gfredericksI think that means you want to use overrides#2017-01-0912:26pyr@gfredericks yes, but I only see support for those in exercise and exercise-fn#2017-01-0912:26pyri'd like c.s.t/check to use them#2017-01-0912:26pyrwait#2017-01-0912:26pyri misread the doc#2017-01-0912:26pyrsorry#2017-01-0912:27pyrit seems as though :gen does this for check#2017-01-0912:54pyrAh, so the map given to :gen in c.s.t/check is shallow#2017-01-0912:55pyri.e: the generator overrides are only taken into account for the called fn. If the fn depends on other functions, they won't use the provided generators#2017-01-0912:55gfrederickslike for stubbing in particular?#2017-01-0913:03pyr@gfredericks not sure I understand#2017-01-0913:04gfredericksI just don't know why else, when using c.s.t/check, functions other than the one being tested would use generators#2017-01-0913:04pyryou're right i'm not articulating my pb properly#2017-01-0913:05pyrit's when generators rely on other overrided generators that i run into the issue#2017-01-0913:05pyron of my generators does:#2017-01-0913:07pyr(def map-gen (gen/bind (s/gen (s/keys :req [::a ::b ::c]) mangle-and-return))#2017-01-0913:07pyrif i do (c.s.t/check map-test :gen {::map mag-gen ::a a-gen ::b b-gen})#2017-01-0913:08pyrthe generation of ::a and ::b in map-gen will not use overrides#2017-01-0913:08pyrso I have to provide overrides in the s/gen call in map-gen too#2017-01-0913:16gfredericksspec uses functions that return generators in a lot of places#2017-01-0913:17gfredericksWhich can defer resolution, I assume. But maybe that can't help here.#2017-01-0913:18pyrit's not a big thing#2017-01-0913:18pyri just need to specify my overrides in a couple of places instead of once.#2017-01-0913:36gfredericksthat seems unfortunate#2017-01-0914:06pyr@gfredericks @mpenet thanks for your help today#2017-01-0914:07pyrI described my approach here: http://spootnik.org/entries/2017/01/09_an-adventure-with-clocks-component-and.html#2017-01-0914:11schmeewhoa, great write-up!#2017-01-0914:11pyrI struggled to decouple the generators from the spec because otherwise it makes things harder to follow. My first iteration had with-gen on the specs themselves and that made it much harder to explain what was going on#2017-01-0914:27lmergenso, if i have a defrecord for which I want to fdef and instrument certain functions defined within that record, how am I supposed to be referring those functions ?#2017-01-0914:28lmergen(i think this is more related to clojure in general, and what the symbol names for functions inside records are)#2017-01-0914:29schmeeAFAIK all record functions must be specified with protocols, so you can spec the protocol methods#2017-01-0915:09pyr@lmergen behavior is implemented on top of records through protocols, so as @schmee mentioned, you can specify protocol signatures as you would do functions#2017-01-0915:10lmergenhmmm ok#2017-01-0915:10pyr(defprotocol Encoder (encode [this])) (defrecord A [name] Encoder (encode [this] (str name)))#2017-01-0915:11pyrif you were to have the above protocol and record#2017-01-0915:11pyryou could specify encode with (spec/fdef encode :args ... :ret ... :fn ...)#2017-01-0915:12lmergenbut that would do it for all instances of Encoder, right ? not just of A ?#2017-01-0915:12pyr@lmergen in this case, (spec/fdef encode :args (s/cat :encoder #(satisfies? Encoder %)) :ret string?)#2017-01-0915:14pyrindeed, this is for Encoder in general#2017-01-0915:14pyrbut that's how protocols work#2017-01-0915:14lmergenok#2017-01-0915:15lmergenso my concrete problem at the moment is Component#2017-01-0915:15lmergenand ensuring that the start function of certain components return a specific type#2017-01-0915:15lmergenso if i have 5 different components (all using defrecord and the Lifecycle protocol), i want 5 different fdefs#2017-01-0915:16lmergenfor those 5 different start function implementations#2017-01-0915:16pyrthe expected behavior of start in Lifecycle is that you return the same record possibly augmented with additional fields#2017-01-0915:17pyrso you could write a generic spec#2017-01-0915:18pyr
(def component? #(satisfies Lifecycle %))
(spec/fdef start :args (s/cat :component component?) :ret component? :fn (= (.getClass (:component %)) (.getClass (:ret %))))
#2017-01-0915:18pyror something to that effect#2017-01-0915:18pyrunless you're doing something funny with component#2017-01-0915:20pyrIf you are really intent on spec'ing per-type behavior for your protocol, there is another much kludgier way. extend (https://clojure.github.io/clojure/clojure.core-api.html#clojure.core/extend) takes a map of protocol implementation for types, which takes fns as args. You could implement Lifecycle this way and then spec the individual functions you have in the map#2017-01-0915:21pyrBut again, it seems as though this should not be necessary for Lifecycle#2017-01-0915:22lmergenok#2017-01-0915:22lmergeni understand#2017-01-0915:22lmergeni'm probably solving this problem at the wrong level#2017-01-0915:23lmergenand should probably verify whether my entire system satisfies a certain spec, after it has been initialized#2017-01-0917:05Alex Miller (Clojure team)I donā€™t think it was mentioned above, but you cannot spec protocol functions#2017-01-0917:25schmeewhy?#2017-01-0917:36lmergenI suspect that it's an implementation issue, since protocol functions have very different function signatures #2017-01-0917:37hiredmanhttp://dev.clojure.org/jira/browse/CLJ-1941?focusedCommentId=43084&amp;page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-43084#2017-01-0917:38hiredmanand previous comments have some discussion of why that is#2017-01-0917:39hiredmanbasically certain kinds of "function" calls(including protocols) are compiled differently and spec just handles the generic case#2017-01-0917:39schmeewhat the recommended workaround?#2017-01-0917:39schmeewrap it in a regular function and spec that instead?#2017-01-0917:40lmergenthat's what I ended up doing yes#2017-01-0921:14Oliver GeorgeAm I missing a cleaner way to write this. Goal is to check that the options list has the required keys which are also declared in the data)
(s/def ::Select.props
  (s/and
    (s/keys :req-un [::value ::options ::value-key ::display-key ::on-change]
            :opt-un [::placeholder ::disabled ::loading])
    #(every? (fn [option] (contains? option (:value-key %))) (:options %))
    #(every? (fn [option] (contains? option (:display-key %))) (:options %))))
#2017-01-0921:15Oliver GeorgeThis won't produce very specific errors#2017-01-0921:18Yehonathan SharvitFrom today, you can easilly share your clojure.spec related code snippets on klipse - it loads pretty fast#2017-01-0921:18Yehonathan Sharvithttp://app.klipse.tech/?cljs_in.gist=viebel/cd7b4c26cf5a9fcc2d53e21021a25df0&amp;eval_only=1#2017-01-0921:40joshjones@olivergeorge so ::Select.props describes a map, one of whose keys is :options, which is a list of maps, each of which must contain values referred to by keys named :value-key or :display-key, which are in ::Select.props -- is that right?#2017-01-0922:22Oliver GeorgeCorrect #2017-01-0922:23Oliver GeorgeSo I really want to confirm the options based on the map data
#2017-01-0922:23Oliver GeorgeLike this movie inception #2017-01-0922:24carocad@kenny I gave it a try at simplifying your macro and came up with this:
(defmacro defspec-test
  ([name sym-or-syms] `(defspec-test ~name ~sym-or-syms nil))
  ([name sym-or-syms opts]
   `(t/deftest ~name
      (let [check-results#  (clojure.spec.test/check ~sym-or-syms ~opts)]
        (doseq [result# check-results#]
          (t/is (nil? (:failure result#)) (clojure.spec.test/abbrev-result result#))
          (when (nil? (:failure result#))
            (println "[OK] - " (clojure.spec.test/abbrev-result result#))))))))
It can be simplified even more but I like the result so far. Hopefully it can be useful to somebody šŸ™‚
#2017-01-0922:27kennyYou want to use do-report, not println otherwise you'll break integrations with clojure.test#2017-01-0922:29carocadwhy? I dont know much about clojure.test actually šŸ˜•. I figured that the call to t/is already took care of the integration. doesnt it?#2017-01-0922:29kennyThe other important difference is that this is counting every single generative test instead of grouping them as one#2017-01-0922:30kennyis will handle it but you will get confusing results for when tests fail#2017-01-0922:41carocadah I see what you mean. true the output is not as clean as with a normal test since you get the generated symbol but I still find the output better since in the previos macro I was only getting the stacktrace of an error. No idea which function failed nor other info. I dont know if it was a problem in my project though#2017-01-0922:56joshjones@olivergeorge unfortunately, I think what you have is about as straightforward as you can get with what you want. While spec can nest sequential and associative structures easily, there is not another way (that I'm aware of) to relate various levels other than another predicate which is what you've done. If anything, I might combine the two additional predicates into one function, though you may find it gives you an undesirable level of detail:
(fn [{:keys [value-key display-key options]}]
      (every? #(and (contains? % value-key)
                    (contains? % display-key))
              options))
#2017-01-1006:22luxbockif I want to use spec for runtime validation, in a situation where my functions accept a large map but only care about a few keys, I should probably create my own validation function rather than using s/valid? because it validates all of the keys in the map even if I'm looking to only validate a subset of them, right?#2017-01-1010:11mpenet@luxbock I don't think so, it will just check what's in (s/keys ...) since map specs are "open" (can contain extra keys) there's no need to check all the entries.#2017-01-1010:12mpenetAt least that's what I assume given how it's defined#2017-01-1010:14mpenetif you use map-of, it's another story#2017-01-1010:20luxbock@mpenet https://gist.github.com/luxbock/261a40e83816d8a428cd991e50c7f2a7#2017-01-1010:21mpenetwell that sucks, I wonder why it does this#2017-01-1010:25mpenet@alexmiller any reason why this should happen?#2017-01-1010:26mpenet
In addition, the values of *all* namespace-qualified keys will be validated
(and possibly destructured) by any registered specs. Note: there is
no support for inline value specification, by design.
from the doc of s/keys
#2017-01-1010:26mpenetanyway, that's kind of odd ...#2017-01-1010:32mpenetso using non ns keys will not trigger this#2017-01-1012:41gfrederickscurrently test.check has a bunch of assertions sprinkled around for checking that the args to particular functions are generators#2017-01-1012:42gfrederickswhich is something that could be somewhat accomplished with spec#2017-01-1012:43gfredericksso I'm wondering if it's worthwhile to remove those assertions and just have specs; the downside being that error messages are worse when you haven't instrumented clojure.test.check.generators, which might not be something that users would naturally do#2017-01-1012:43gfredericksshould library authors expect that users will turn on instrumentation everywhere to debug a problem, and therefore do less runtime validation than they would otherwise do?#2017-01-1013:46Alex Miller (Clojure team)@mpenet the whole idea behind s/keys is to register your attributes with known semantics and have them checked throughout your system. By having it validate all keys, your spec grows automatically with your system. If you want to narrow the scope, you can select-keys on the map before validating #2017-01-1013:47Alex Miller (Clojure team)@gfredericks: I'm a little worried about the circularity issues of having specs in test.check but maybe it's fine with the dynamic loading gap#2017-01-1013:49Alex Miller (Clojure team)I'd say you shouldn't rely on instrumentation in cases where you always want those checks (and this might be one of those places)#2017-01-1013:49mpenetit's a bit of implicit vs explicit. So if you use ns keys you can (not that it's a good idea) just use (s/keys :req []) for your map validation#2017-01-1013:50mpenetI personally find that odd, but I can live with it#2017-01-1013:50Alex Miller (Clojure team)You don't even need the :req there, just (s/keys) is sufficient#2017-01-1013:50mpenetright#2017-01-1014:28gfredericks@alexmiller I'm pretty sure putting specs in test.check is as feasible as putting them in clojure.core šŸ˜‰ #2017-01-1014:29gfredericksI can at least run the test.check test suite with instrumentation#2017-01-1015:26tjtoltonThis channel needs a "quicklinks" section for newbs like me. I'm looking for materials on integrating core.spec test.check... can anyone offer me any leads? Is the core.spec workshop from the conj online anywhere?#2017-01-1015:30donaldballSomeone posted a nice writeup yesterday that might be useful: http://spootnik.org/entries/2017/01/09_an-adventure-with-clocks-component-and.html#2017-01-1015:37tjtoltonthanks, @donaldball I will take a look at that!#2017-01-1015:39donaldballThe second pinned item in this channel might also be useful#2017-01-1016:13cgrandHi, is there a handy way of writing a s/fdef for fns with multiple arities (and differing behaviorsā€¦)?#2017-01-1016:16joshjones
(s/alt ::arity-1 (s/cat :a int?)
       ::arity-2 (s/cat :a int? :b string?))
something like that?
#2017-01-1016:17joshjonesthat would be the :args of the fdef#2017-01-1016:19cgrandI have something like
(s/fdef my-function
  :args (s/alt :1 (s/tuple ::x) :2 (s/tuple ::x ::y))
  :ret (s/alt :1 ::something :2 ::other)
  :fn #(= (key (:ret %)) (key (:args %)))
and Iā€™d like to disentangle the two arities
#2017-01-1016:21cgrandI could wipe a macro to do
(fdef+ my-function
  (:args (s/tuple ::x) :ret ::something)
  (:args (s/tuple ::x ::y) :ret ::other))
#2017-01-1016:23joshjonesfor the level of separation you seem to want, that might be useful#2017-01-1017:59crimeministerHi folks, this is my current conundrum: https://www.refheap.com/124571#2017-01-1018:01crimeministerTrying to figure out what the idiomatic way to validate something while "transforming" it, i.e. validate a string using a regex, and then checking that the capture groups have the required values using (spec/cat)#2017-01-1018:06joshjonesare you able to work with actual instant objects, or must it be strings?#2017-01-1018:07crimeministerStrings would be ideal in that I could avoid a transformation step, but if that would make life easier I'd try it out#2017-01-1018:17joshjonesis your goal to just get validation of dates/times, or are you looking to leverage spec to generate data, etc.?#2017-01-1018:20crimeministerPrimary goal is to get validation, but if I could get the latter as well I'd be pretty happy#2017-01-1018:24crimeministerIf need be I could write a little transformer fn that turned my string data into, say, a map for easy validation, and another transform the map value into a string to assist with data generation#2017-01-1018:24crimeministerI just worry that it introduces an extra step each time I make a (spec/ā€¦) call#2017-01-1018:28crimeministerAnother (bad?) option might be some macro magic to allow spec to take a :pre-transform hook#2017-01-1018:28joshjonesI think what you have now is headed in the right direction. With things like date/time, I always long to use existing libraries. The clojure.instant library (specifically the read-instant-timestamp might be of use) can be used to parse a string into a date/time. for data generation, spec provides inst-in to generate dates within a range.#2017-01-1018:30crimeministerThanks for the suggestion @joshjones. I think I need to take that idea away and hammock on it for a few šŸ™‚#2017-01-1018:38seancorfieldWhat are folks doing for atoms in terms of spec? I thought I read about an atom-of function but canā€™t find any context for that now.#2017-01-1018:39seancorfield(I recall the ā€œdonā€™t spec atoms, only spec their dereferenced valuesā€ advice but that doesnā€™t help when I have a map that contains two atoms šŸ™‚ )#2017-01-1018:44Alex Miller (Clojure team)It's possible that we might add something like atom-of#2017-01-1018:45gfredericksagent-of ref-of var-of future-of promise-of generator-of#2017-01-1018:45gfrederickschannel-of#2017-01-1018:49seancorfield@alexmiller Cool. Do you have any interim advice until that happens? šŸ™‚#2017-01-1018:50Alex Miller (Clojure team)No :)#2017-01-1022:27tjtoltonanyone who has taken a look at this -- http://spootnik.org/entries/2017/01/09_an-adventure-with-clocks-component-and.html could someone explain why authorized-request has a separate, specialized test using a different function spec?#2017-01-1022:35tjtoltonerr, okay, it seems kind of weird#2017-01-1022:36tjtoltonits specifying that the return value is not just a boolean, but specifically that the return is true#2017-01-1022:36tjtoltonbut how can that be guaranteed since the input is generated randomly based on the spec?#2017-01-1022:40schmeetjtolton the spec has a bunch of custom generators supplied through gen-overrides#2017-01-1022:40schmeewhich ensures that only ā€œvalidā€ domain values get generated#2017-01-1022:44tjtoltonhuh. Isn't that beyond the purview of a schema or function spec? Ensuring that data is not only in a given form, but correct according to application logic?#2017-01-1022:45tjtoltonMaybe I just need to read that a bit more carefully#2017-01-1022:45gfredericksif it's just for testing, then that's not really part of the spec#2017-01-1022:45gfredericksyou might want your generators to be rather more specific/fancy than the specs#2017-01-1022:46gfredericksmore or less because of what you just said#2017-01-1023:29bbloomheh, i just found a pretty interesting spec bug....#2017-01-1023:29bbloomapparently if you define a spec for ::keys, it interferes with destructuringā€™s spec#2017-01-1023:29bbloomā€¦ filing a ticket#2017-01-1023:31bbloomah, apparently iā€™m not the first: http://dev.clojure.org/jira/browse/CLJ-2074#2017-01-1023:53michaeldrogalisIs there a preferred way to expression mutual exclusion of the presence of two keys in a map?#2017-01-1023:54bbloommy guess would be no#2017-01-1023:54michaeldrogalisMissed this piece on the docs:#2017-01-1023:54bbloomxor inherently involves a ā€œnot"#2017-01-1023:54michaeldrogalis> The :req key vector supports 'and' and 'or' for key groups: > > (s/keys :req [::x ::y (or ::secret (and ::user ::pwd))] :opt [::z])#2017-01-1023:54bbloomoh, thatā€™s interesting#2017-01-1023:54bbloomitā€™s still not an xor tho šŸ˜‰#2017-01-1023:54michaeldrogalisItā€™s not quite mutual exclusion in that both can be absent, but thatā€™s closer.#2017-01-1023:55michaeldrogalisHeh. Stay out of my brain Brandon!#2017-01-1023:55bbloomp xor q = (p or q) and not(p and q)#2017-01-1023:55bbloomthat not is sorta against the whole open ethos of spec#2017-01-1023:55michaeldrogalisYeah. šŸ˜•#2017-01-1023:55bbloombut of course you can do that with a predicate#2017-01-1023:56michaeldrogalisThanks for the thoughts šŸ™‚ See you this week, probably.#2017-01-1100:05cap10morgan@jmglov did you ever get around to doing this: https://clojurians-log.clojureverse.org/clojure-spec/2016-09-12.html#inst-2016-09-12T18:28:43.000617Z#2017-01-1100:05cap10morganor is there an updated way to integrate clojure.spec generative tests into your lein test runs?#2017-01-1100:13uwo@alexmiller has there been any consideration put toward separating required semantics from s/keys, and creating another function s/required?#2017-01-1100:58Alex Miller (Clojure team)No#2017-01-1109:32tapI have a spec which uses s/fspec (`:args` specified). As you know, this spec requires test.check. Is there a way to temporary skip validating s/fspec so that I can execute s/valid? on this spec at runtime?#2017-01-1113:12Alex Miller (Clojure team)You can instrument and supply a spec override to ifn?#2017-01-1113:42tapMy code is in this shape
(def fn1 [body])
(def fn2 [body])
; ... fnN

(s/def ::a-fn (s/fspec :args (s/cat :body string?)))
(s/def ::a-map (s/keys :opt-un [::a-fn]))

(s/valid? ::a-map {:a-fn fn1})
(s/valid? ::a-map {:a-fn fn2})
I'll need to stub fn1,fn2,...,fnN, right? Maybe I should structure my code differently then.
#2017-01-1113:51pyrhola spec'ers#2017-01-1113:52pyrare there precedent to people using specs with channels?#2017-01-1113:52pyr(as in c.c.async channels)#2017-01-1113:53mpenetif you mean specing what's inside of them no, specing channels as "values" yes#2017-01-1113:53mpenetas for instrumentation I bet it's hairy territory but I never tried tbh#2017-01-1113:54pyrthe former is what i meant#2017-01-1113:54pyrmeh, no worries#2017-01-1114:00cgrand@pyr a/chan + conforming|validating transducer + ex-handler?#2017-01-1114:02mpenetalmost suggested the xform thing, but I am not sure it will work with the advanced stuff (instrumentation)#2017-01-1114:04cgrandwhich interaction do you see between a channel and an instrumented defn?#2017-01-1114:09tjtoltonso, perhaps an odd question, but is there a clojure 1.9 build that is production ready? I want to use spec, but being the guy that insists we put an incomplete version of clojure in prod is a bit nerve racking#2017-01-1114:09mpenetarg gen triggered by fdef for instance#2017-01-1114:09mpenetso far it has been one of the thing I have found quite confusing/surprising#2017-01-1114:14mpenetI guess as long as the fn is pure it's ok#2017-01-1114:14cgrand@mpenet: I see: not instrumentation but validation of a fspec#2017-01-1114:15mpenetif it's only validation yes, I was referering to this kind of issue: http://dev.clojure.org/jira/browse/CLJ-1936#2017-01-1114:16mpenetI mixed fdef and fspec earlier tho#2017-01-1114:16schmeetjtolton clojure 1.9 is still alpha, no word on when the official release will be out yet#2017-01-1114:18tjtoltongotcha. so, officially, no guarantees on spec in production#2017-01-1114:18tjtoltonbut#2017-01-1114:18tjtoltonunofficially#2017-01-1114:19tjtoltonis it a good idea for me to try to get spec into our code now? or should I wait, and just keep using spec in toy projects until the official release#2017-01-1114:59Alex Miller (Clojure team)Thatā€™s a judgement call only you can make. spec is still alpha and subject to change. We are aware of the desire from many directions to have a 1.9 (non-alpha) with spec available.#2017-01-1115:02pyr@cgrand yes, it's an option. I was interested in specing promise-chans which should return a specific type of value.#2017-01-1116:24lmergen@tjtolton we're running spec in production. haven't found any problems, but YMMV#2017-01-1116:24lmergen(we're also running aleph and manifold alpha's in production, so there's that)#2017-01-1116:25tjtolton@lmergen cool. I'm going to basically hook up 1.9-alpha and spec to the service that I'm the owner of.#2017-01-1116:26tjtoltonIf it breaks, it will be my job to fix it, which I'm cool with.#2017-01-1116:34lmergeni hope you're not in charge of the authorization service then šŸ˜‰#2017-01-1116:44seancorfield@tjtolton: we've been running Clojure 1.9 in production for months with no problems. We've always run alpha and beta builds in production as they've appeared in order to leverage new features early. We first went to production on Clojure 1.3 alpha 7 or 8 back in 2011. #2017-01-1116:46tjtoltongood thing clojure is stable as hell#2017-01-1116:46tjtoltonGod I love this language culture.#2017-01-1118:10seancorfieldI think in nearly seven years weā€™ve seen just two builds with production-affecting bugs. We skipped one of those (just a matter of timing, due to when our production builds fell, either side of that release), and the other one was fixed quickly enough that we just pushed a new production build with the updated Clojure JAR.#2017-01-1118:25joshjonesLike every single thing in life, it comes down to a simple ratio of risk:reward. If the code wonā€™t run on 1.8, and if the consequences of delaying a production release are worse than the consequences of a production product exploding (i.e., first to get to market on something may be most important than it actually working well), then it might make sense to use a 1.9 alpha in production. Also, if the user impact is low (i.e., this isnā€™t a mission critical application and several hours/days of downtime wonā€™t kill anyone) then it might also make sense. However, as a general process, it just doesnā€™t make sense to run alpha in production. The liability is all on you. I prefer to shift liability elsewhere when itā€™s pragmatic to do so, and when the clojure core code is considered ā€œproduction readyā€ and it fails, the liability has at least partially shifted away from you as the developer, and onto the clojure core team. For products with relatively low use, Iā€™d run an early alpha in production since the track record seems to be good. But for a service which receives several thousands of requests per second, around the clock, and upon which many millions of dollars hang ā€” for us personally, being able to run spec isnā€™t worth that risk, i.e., we can wait a few months. Each developer and team has to make the call and accept the consequences šŸ™‚#2017-01-1119:52bbrinckIs it a design goal of clojure.spec (and the core specs in particular) that conform/unform should always round-trip? For instance, the built-in spec for defn-args doesnā€™t round-trip in at least one case (more details http://blog.klipse.tech/clojure/2016/10/10/defn-args.html) - is that considered a bug?#2017-01-1119:54bbrinckThe reason I ask is that weā€™re experimenting with using the conform+modify+unform pattern for modifying data, and just wondering if the core specs will be written to support that#2017-01-1120:37seancorfieldThere are currently some known bugs in unform based on discussions Iā€™ve seen here.#2017-01-1120:57mikerodit seems like having unform be able to roundtrip successfully in all cases is a tricky situation#2017-01-1120:59bbrinck@mikerod can you talk a little more about that? ā€œtrickyā€ as in thereā€™s going to be a long-tail of bugs for this? or ā€œtrickyā€ as in ā€œitā€™s not realistic to treat every case as a bugā€? or something else?#2017-01-1121:00mikerod@bbrinck oh, Iā€™m just hypothesizing and considering how Iā€™ve seen the topic come up quite a few times.#2017-01-1121:00mikerodI actually donā€™t know. I do hope that it is meant to be able to do the type of roundtripping you are going for in you defn-args article though#2017-01-1121:00bbrinck@mikerod ah, i see. for now, we just ended up writing our own spec for this case#2017-01-1121:01mikerodIā€™m definitely not going to give any definitive say on it though šŸ˜›#2017-01-1121:01bbrinckšŸ™‚#2017-01-1121:01mikerodYeah, I saw you mentioned adding a conformer#2017-01-1121:01bbrinckOh, to be clear, I didnā€™t write that post - we just used it as a starting point for writing our macro#2017-01-1121:01bbrinckbut itā€™s a good explanation of the issues we ran into#2017-01-1121:02mikerodah, I misread that part#2017-01-1121:04Alex Miller (Clojure team)yes, conform -> unform should roundtrip (with a few caveats)#2017-01-1121:08Alex Miller (Clojure team)there are some known issues http://dev.clojure.org/jira/browse/CLJ-2076 (coll-of, map-of) http://dev.clojure.org/jira/browse/CLJ-2003 (s/?)#2017-01-1121:10Alex Miller (Clojure team)and then I'd say one area that could benefit from something additional is specifically for specs that are using regexes in a vector - some of the core macro specs suffer from this and unform to a list rather than a vector. thatā€™s a common and important case and I think we need something specific for it (whether thatā€™s creating vcat or something else is TBD)#2017-01-1121:11Alex Miller (Clojure team)thatā€™s the case mentioned at the top of this conversation#2017-01-1121:14bbrinckOK, thatā€™s very helpful. Thanks!#2017-01-1121:15Alex Miller (Clojure team)if you find something else, feel free to file a bug#2017-01-1121:16mikerodthanks!#2017-01-1121:17mikerodfor the details#2017-01-1123:51rickmoynihan@tjtolton: Another option is to run clojure 1.9alphas in dev/test profile's only and have a policy to avoid including 1.9 features in the production environment.... you can set this up with leiningen profiles pretty easily. This way you could e.g. write specs in your unit tests, but not put them in the production code itself, keeping it on 1.8 ā€¦ when spec is finally released you could then move some of the specs out of the tests into the production code. You could also switch to 1.9 whenever you want and know you can switch back if itā€™s not good. Obviously it requires some discipline, and might mean you canā€™t test code in the production env; so thereā€™s still some significant risk - and you might also want to separate the 1.9 tests into a separate profile. Anyway itā€™s just one possibility Iā€™ve been considering#2017-01-1123:53rickmoynihanThe core team are incredible at keeping clojure stable though - but itā€™s alpha for a reason, so we should respect that they may want to change their minds and make significant changes to 1.9 code at any point.#2017-01-1123:59seancorfieldIā€™d add the caution that there are quite a lot of oh-so-convenient new predicate functions in 1.9 that youā€™ll find hard to resist using in your code and then youā€™ll be unable to run on 1.8. Youā€™ll also be forced into keeping all the spec stuff in separate namespaces so your non-test code will run on 1.8.#2017-01-1200:00seancorfieldThat said, clojure.java.jdbc currently takes that approach (specs in separate namespaces) so it can run on 1.4 thru 1.8, and it optionally uses spec for the tests if they are run on 1.9.#2017-01-1200:00seancorfieldAnother option would be to look at the backport of spec to 1.8 that someone in the community maintains (future spec, I think?).#2017-01-1200:01seancorfieldhttps://github.com/tonsky/clojure-future-spec#2017-01-1200:56tjtoltonThat's a cool idea, thanks!#2017-01-1216:35actsasgeekI have a philosophical (?) question about how far spec should be pushed in terms of semantic types, type checking and testing. For example, is it a reasonable use of spec to specify that the list argument to a binary search is sorted? Can you do it?#2017-01-1216:37williewillusit's certainly possible#2017-01-1216:39williewillusthat said I'm also interested in exactly where it should be applied, and if "overuse" of it is possible, and what that would look like#2017-01-1216:46actsasgeekwell, itā€™s funny you say that because that was my leading question...#2017-01-1216:46actsasgeekone of the things that has been mentioned is the separation of specifying the keys in a map and the values those keys may take.#2017-01-1216:49actsasgeekbut it is very often the case in complex data structures that those values are interdependent. For example, is it too far to require a ::street to actually be in the ::city in the ::state with the right ::zip-code? Or, perhaps midway between the two, if Iā€™m modeling a ā€œblocks worldā€ using blocks with the keys ::id ::pile ::above ::below, itā€™s certainly true that a block cannot be above itself, below itself or both above and below the same block. And so I find myself writing block? instead of (s/def ::block (s/map ā€¦).#2017-01-1217:04joshjonesIt is very true that certain ā€œschemaā€ is expressed via the relationship of one value to the other in the entity map. Spec still allows us to express this relationship via a custom predicate, but the nice part is that it does not force us to define this structure. Instead, its intention is for us to specify reusable pieces where possible. How far we take it is largely up to us. In the example of city/zip, personally this feels as if itā€™s overstepping the intended bounds of spec, but I cannot quantify that yet with anything sensical so itā€™s not worth much.#2017-01-1217:10actsasgeekyeah, Iā€™m just trying to think out loud a bit. One of the points stressed in spec is the generation of examplesā€¦but automatically generated samples of addresses that are just random strings seemsā€¦sad to me.#2017-01-1217:11schmeewhy?#2017-01-1217:12actsasgeekbecause if I want to use this as documentation and to specify what would be acceptable input, a ::street of ā€œV+q.29b7gqao9$ā€ seems strange.#2017-01-1217:12joshjonesthatā€™s what custom generators are for...#2017-01-1217:13schmeethere are a lot of street names out there šŸ˜›#2017-01-1217:13joshjonessee this for building data based on your own data model, also: http://blog.cognitect.com/blog/2016/8/10/clojure-spec-screencast-customizing-generators#2017-01-1217:14actsasgeekyeah, I watched thatā€¦I would like to see more on that topicā€¦it seems like a buried but important part of the overall usefulness of spec.#2017-01-1217:14actsasgeekafter watching that, Iā€™m still not entirely clear what a model is.#2017-01-1217:15schmeeI donā€™t see what is wrong with random data unless you depend on a certain structure for the string in your code#2017-01-1217:15joshjonesI think itā€™s only buried because itā€™s still in alpha and most people donā€™t ā€œget specā€ yet even for basic cases, and need to build a level of understanding that can support that level of detail#2017-01-1217:15actsasgeekand thatā€™s fine @schmee. I do.#2017-01-1217:15schmeeahh, then custom generators are the way to go šŸ™‚#2017-01-1217:16actsasgeekI think that video above and the one before it illustrate the point to some degree. I was watching the testing video, and kept going, ā€œbut you arenā€™t actually testing the functionā€¦the odds of generating a random string that is a substring of another random string are very lowā€.#2017-01-1217:17actsasgeekbut even at the end, if my-index-of was an actual function, thereā€™s absolutely nothing in there that tests if the index returned is the correct one.#2017-01-1217:19joshjonesgotta lunch but would love to discuss after i eat šŸ˜‰ you have a great topic and is at the heart of spec and how it can be useful to everyone#2017-01-1217:19actsasgeekif think if I replaced the use of string/index with 1, I think all the tests would pass.#2017-01-1217:21schmeeI donā€™t see how you could test that in a generative fashion without re-implementing the function itself#2017-01-1217:21schmeeI suppose this is where example-based testing as a complement comes in#2017-01-1217:22schmeeor generative testing as a complement to example-based testing, if you prefer#2017-01-1217:43actsasgeek@schmee I think I agree. My concern would be that in the video (the 2nd one of 3 in the series on Testing), Stu specifically contrasts the generative approach with the assert approach where you have to do everything yourself.#2017-01-1217:44actsasgeekNow, one interesting thing is that the custom generator contains the information (implicitly) that you need. The generator takes a prefix, match and suffix to generate items that will definitely have a match. If you could save the length of the prefix, you could verify that my-index-of was working correctly.#2017-01-1217:46schmeeyou could return a tuple of [(count prefix), rest of the stuff] and use that in your tests#2017-01-1217:53actsasgeekHmm, doesnā€™t what is generated have to be arguments to the function under test?#2017-01-1217:55schmeeIā€™m not sure, but either way you can still write tests that use generators without spec and do whatever you want šŸ™‚#2017-01-1217:55actsasgeekI think a subtle related problem is that the :ret is specified as <= the length of the string but it is < length of the string because of zero indexing.#2017-01-1217:56actsasgeekwell, you can always do what you want šŸ˜›#2017-01-1217:57schmeehaha, yeah#2017-01-1217:58schmeebut I mean, itā€™s cool that spec can auto-generate tests and all, but if you have a very specific (generative) test you want to make it might be better to just write that the old-fashioned way and not try too hard to cram it into a spec-based approach#2017-01-1218:01joshjones@actsasgeek which video are you talking about, with my-index-of?#2017-01-1218:02actsasgeekyou posted a video on custom generators. It was 3rd in a series of 3.#2017-01-1218:04joshjonesI have not watched it, Iā€™m looking at it now#2017-01-1218:06actsasgeekhttps://www.youtube.com/watch?v=nqY4nUMfus8&amp;list=PLZdCLR02grLrju9ntDh3RGPpWSWBvjwXg#2017-01-1218:06actsasgeekyou can safely watch it a 1.25 speed šŸ™‚#2017-01-1218:14actsasgeeksorry, I have to pair for a bit so Iā€™ll be away from this conversation but I am interested in continuing it.#2017-01-1219:30tjtoltonSo, the docs for spec.test/instrument seem kind of circular: > Instruments the vars named by sym-or-syms, a symbol or collection of symbols, or all instrumentable vars if sym-or-syms is not specified. Is there a decent instrument lesson of some sort out there?#2017-01-1219:31schmeetjtolton https://clojure.org/guides/spec#_instrumentation_and_testing#2017-01-1219:32schmeeread the whole thing like 10 times if you havenā€™t already šŸ˜„#2017-01-1219:32tjtoltonjackpot, don't know how I missed that!#2017-01-1219:32tjtoltonI will!#2017-01-1219:34williewillusthe series of videos mentioned above are great introductions as well#2017-01-1219:34williewillushttps://www.youtube.com/playlist?list=PLZdCLR02grLrju9ntDh3RGPpWSWBvjwXg#2017-01-1220:08joshjoneshttps://clojurians.slack.com/archives/clojure-spec/p1484243750006346 @actsasgeek Regarding this, the index-of ā€œā€ within ā€œā€ is 0, so in this one case the length of the return is equal to the length of the source. edge case though, and you could account for this in the :fn portion of the spec So to continue the earlier discussion: in order to test something, we have to have something that is known, correct, reliable, etc., to test against. In the case in the videos for index-of, it was the ā€œeye testā€ ā€” make some sample data, manually count the characters, see if the function spits out what we determine as the ā€œcorrectā€ value. The :fn spec can be made more robust (Iā€™ll post an example later), but imagine weā€™re just specā€™ing substring, determining whether a string is actually a substring of another. Well, this is not so easily specā€™d, because thatā€™s essentially what this function does! So, if you had another ā€œsource of truthā€ that could definitively tell you that, well, youā€™d just use that function. Even when you write your tests manually, youā€™re still placing your trust in your ability to accurately type strings correctly. There is always room for error in the testing phase, and you canā€™t get around it. Perhaps the answer is to hand write your own tests as usual, and it may make sense to do that. But one takeaway is that going through this process is likely to find edge cases that your own manual testing would NOT have found, even if it does not have the ability to determine with 100% certainty that your function is correct. (In the case of the video though, I think it can be made more rigorous by altering the :fn)#2017-01-1220:33actsasgeekTo the specific point, I was a little surprised that (index-of ā€œabcdā€ ā€œā€) even works. It feels like an overly mathematical definition of ā€œindex ofā€ because (index-of ā€œabcdā€ ā€œaā€) returns the same resultā€¦but thatā€™s neither here nor there.#2017-01-1220:36actsasgeekTo the general point, I think I agree. My point was that in the 2nd video, it seemed to me at any rate, that generative testing was presented as an alternative to manual testing. Just in the way Stu describes how ā€œdifficultā€ that coming up with the manual test was. I completely agree on the point that generative testing is much, much better at finding the edge cases. In a way, Iā€™ve always thought of manual testing as a way of preserving REPL development of the shape of the function and documentation.#2017-01-1220:37actsasgeekBut documentation, examples, etc., are being touted as leverage points of spec and it seems to me that these are going to require more custom generators than the relative abundance of documentation for making customer generators would suggest. I could be wrong, tho.#2017-01-1220:41actsasgeekbut then I think that the opposite of conform should be deform and not unform. šŸ˜‰#2017-01-1309:48pyrHi spec'ers#2017-01-1309:48pyrI'm up against a plumbing issue it seems#2017-01-1309:49pyrI wrote a simple function to run generated tests in a test namespace:#2017-01-1309:50pyr
(defn run-generated-tests
  []
  (vec
   (for [out  (st/check (st/enumerate-namespace 'my.namespace)
                        {:gen overrides})
         :let [ret (:clojure.spec.test.check/ret out)]]
     {:success? (:result ret)
      :output   ret})))
#2017-01-1309:52pyrThis works as expected#2017-01-1309:52pyrIf I integrate this in tests like this:#2017-01-1309:53pyr
(deftest generated-test
  (let [output (run-generated-tests)]
    (is (every? :success? output))))
#2017-01-1309:54pyrI can do (run-tests 'my.namespace) at the repl, which also works as expected#2017-01-1309:54pyrbut if I run lein test my.namespace#2017-01-1309:55pyrit fails with: Caused by: java.lang.ClassCastException: clojure.lang.AFunction$1 cannot be cast to clojure.lang.MultiFn#2017-01-1309:55pyrI fail to see what might cause this, since I can't reproduce at the repl.#2017-01-1315:02Alex Miller (Clojure team)thatā€™s the lein monkey patch / test.check bug#2017-01-1315:02Alex Miller (Clojure team)https://github.com/technomancy/leiningen/issues/2173#2017-01-1315:02Alex Miller (Clojure team)add :monkeypatch-clojure-test false to your project.clj to bypass#2017-01-1315:03Alex Miller (Clojure team)the consequence is losing the ability to run lein retest (which most people donā€™t do)#2017-01-1315:04Alex Miller (Clojure team)@pyr ^^#2017-01-1315:20pyr@alexmiller ah, sorry, I missed that#2017-01-1315:20pyrthanks for the heads up#2017-01-1315:25pyr\o/#2017-01-1315:31trisshey all#2017-01-1315:32trissiā€™m speccing some functions that take javascript objects of particular types as arguments and return them#2017-01-1315:33trissi want to conform some data that includes functions of these fspecs#2017-01-1315:36trissbut i get error stating there is no generator available for my specs of the following form:#2017-01-1315:37triss#(instance? js/AudioNode %)#2017-01-1315:47gfredericksProbably conforming an fspec involves generativelh testing it?#2017-01-1315:48trissok, think i might swap the fspecs out for fn?for what Iā€™m doing#2017-01-1315:49trisssetting up generators for js objects of specific types seems fiddily#2017-01-1316:23gfredericksit's not hard if they can be constructed easily from data#2017-01-1316:23gfrederickse.g., with gen/fmap#2017-01-1321:11eggsyntaxI want to make a map spec, but with a custom generator that generates it with an extra key (so eg the spec might be for a map like {::a 1 ::b 2}, but when I generate one I want to generate it like {::a 1 ::b 2 ::c 3}). Right now I'm mucking around with creating a spec for the fuller map including ::c, pulling the generator from that, and using that generator in a with-gen (and I haven't even quite got that working yet). It just seems excessively ugly -- anyone see a better way?#2017-01-1321:13eggsyntaxI could create the generator essentially manually, generating {::a (gen/generator (s/gen ::a)) ...} but that seems ugly too, especially because I'm creating these specs dynamically on the fly, so it'd result in awfully complicated code.#2017-01-1321:14Alex Miller (Clojure team)Iā€™d probably do your first idea with grabbing the gen from the fuller spec - why is it looking ugly?#2017-01-1321:15Alex Miller (Clojure team)if itā€™s repetition, you might try s/merge of the original spec with an additional map spec, then grabbing the gen from that#2017-01-1321:16eggsyntaxOh, that's exactly the elegant sort of solution I was looking for šŸ™‚. I think this'll be my first time actually using s/merge. Thanks, Alex!#2017-01-1321:16bbloommerge is kinda awesome#2017-01-1321:16eggsyntaxThe ugliness is just yeah, the repetition & all the lines of code, and creating a spec only to throw it away.#2017-01-1321:16bbloomway nicer than typical inheritance schemes#2017-01-1321:21Alex Miller (Clojure team)to get in the spec mindsetā€¦ maps are sets of attributes. merge is the union.#2017-01-1321:22bbloom@alexmiller i found union and intersection to be very confusing terms when talking about ā€œsemantic subtypingā€ and open maps - the problem is that there is a subtle distinction between the union of keys and the union of the sets of values that match the specification#2017-01-1321:22bbloomand they have opposite implications#2017-01-1321:22bbloomif you union the keys, you shrink the set of possible values#2017-01-1321:22Alex Miller (Clojure team)yeah, donā€™t think about it that hard :)#2017-01-1321:23bbloomi only share this information with you b/c iā€™ve seen the same confusion about the words ā€œunionā€ and ā€œintersectionā€ affect people working with TypeScript and Flow#2017-01-1321:23bbloomthatā€™s why iā€™m glad the name chosen was ā€œmerge"#2017-01-1321:30eggsyntaxHmm. Now that I try to think it through further, I'm less sure I see how to use s/merge to make it better. Are you envisioning something like this?
(s/def ::full-spec (s/with-gen
                     ::smaller-spec
                     (s/gen (s/merge ::smaller-spec ::extra-key-spec))))
That seems like it would require creating three specs in total instead of two...
#2017-01-1321:30eggsyntaxBut it's very possible I've just gotten muddled trying to think it through šŸ˜‰#2017-01-1321:31eggsyntax(where ::extra-key-spec is a spec for a map containing only the extra key)#2017-01-1321:37eggsyntaxOh, wait, I can use s/spec to make most of that implicit, I think.#2017-01-1321:37eggsyntaxDuh šŸ˜‰#2017-01-1321:41rickmoynihanI have a function in some legacy code that I want to write an example based clojure.test test forā€¦ it returns a vector of various types and I thought I could write a spec for it. In an is definition the granularity of failure is just a booleanā€¦ so I could extend clojure.test to display the explain data (Iā€™ve experimented with this once before). I did think I could instrument it with an fdef but obviously that only works for input values. Anyway just wondering if people are currently using spec for example based testing#2017-01-1321:41rickmoynihan(I realise itā€™s for generative testing)#2017-01-1322:09eggsyntaxDamn, I'm getting close, but I keep tripping over java.lang.IllegalArgumentException: No matching ctor found for class clojure.test.check.generators$such_that$fn__32821 when I try to eval the s/def (which I have to do because this is all getting defined at runtime)#2017-01-1322:12eggsyntaxI can just do the s/def in the REPL and it works properly. And the eval was working until I started trying to add this generator, so I'm screwing something up...#2017-01-1322:23tjtoltonis there some kind of configuration I need to set up to make spec.test/check work?
(->> (st/check 'lookup-requests-for-day) st/summarize-results)
=> {:total 0}
#2017-01-1322:40eggsyntaxGot it šŸ˜„#2017-01-1322:41eggsyntaxHaving just gotten some help myself, I wish I could help either of the people who just asked questions, but unfortunately I don't have an answer to either of those šŸ˜•#2017-01-1322:49rickmoynihantjtolton: did you try with a back-quote?
(->> (st/check `lookup-requests-for-day) st/summarize-results))
#2017-01-1400:21richiardiandreaso let's say I am enriching a data structure with chained transformations (`map`-ing or specter-ing)...and I want to spec every substep. For instance the first step has {:family {:name "bla"}} the second step will have {:family {:name "bla" :id :sda}}.#2017-01-1400:22richiardiandreahow can I spec the second step after having already spec-ed the first as:
(s/def :family/entity (s/keys :req-un [:family/name]))
(s/def :family/family :family/entity)
(s/def :root/entity (s/keys :req-un  [:family/family]))
#2017-01-1400:22richiardiandreaups#2017-01-1400:23richiardiandreanow it should be ok#2017-01-1400:24richiardiandreaso basically I'd need to add another spec adding :family/id but I can't see how to do it without overriding the :family/family spec#2017-01-1400:26richiardiandreaok I could change the namespace probably#2017-01-1400:26eggsyntaxCan you s/merge the :family/family spec with a new one which only specifies the new element?#2017-01-1400:26richiardiandreamaybe šŸ˜„ let me try that#2017-01-1400:27richiardiandreabut then I still need to change the namespace in the register#2017-01-1400:29eggsyntaxYou could eval the s/def with a modified nameā€¦although having been doing a lot of it, Iā€™ll warn you that thatā€™s a route that can get reeeaaally sticky pretty fast. & there seem to be issues with evaling s/merge (thatā€™s where I stopped today, had to call a halt. In fact, crap, what am I doing here? Going AFK immediately šŸ˜‰ )#2017-01-1400:35richiardiandreait works, but now for each transformation I need:
(s/def :entity-with-id/family (s/merge :family/entity (s/keys :req-un [:family/id])))
(s/def :entity-with-id/definitions (s/keys :req-un [:entity-with-id/family]))
#2017-01-1400:35richiardiandreaso the namespace becomes the name of the transformation I want to target#2017-01-1413:25trisshey all. is there a handy spec i can use for things that can be passed to s/valid? for testing?#2017-01-1413:26trissi.e. name-spaced keywords, single-arity fnā€™s and specs I guess#2017-01-1418:06gfredericksaren't specs just either of the first two?#2017-01-1421:31bbloomis there something like ā€œmerge, but make all the keys optionalā€?#2017-01-1421:31bbloomor i guess optionally-merge?#2017-01-1421:31bbloomnot sure what i mean / want šŸ˜›#2017-01-1422:03bbloomalso thinking aloud: i kinda want a predicate that is basically any? but means ā€œnot yet specā€™d"#2017-01-1423:13rickmoynihan@bbloom (def not-specd-yet? any?)#2017-01-1518:11bbloom@rickmoynihan thatā€™s a decent hint to programmers, but itā€™s not quite good enough semantics#2017-01-1518:12bbloomfor example, a generator will produce arbitrary values there, which is clearly wrong#2017-01-1518:12bbloomoh well actually maybe spec isnā€™t aware of aliasing and youā€™ll get an error#2017-01-1518:12bbloomlet me see...#2017-01-1518:14bbloomthis works: (def unspeced (fn [x] true))#2017-01-1518:14bbloombreaks the generator, which is what iā€™d expect#2017-01-1521:27dottedmagIs there a way to drill into specs from REPL? Say, an output from (doc ns) gives args: (cat :name simple-symbol? :docstring (? string?) :attr-map (? map?) :clauses :clojure.core.specs/ns-clauses), how do I see the definition of :clojure.core.specs/ns-clauses spec?#2017-01-1521:28dottedmagOh, (doc :clojure.core.specs/ns-clauses) works šŸ˜„#2017-01-1521:36joshjonesthis is effectively the same as (s/describe :clojure.core.specs/ns-clauses) fyi#2017-01-1522:47bbloomsome specs seem to resolve eagerly, some lazily - is there some way to force laziness for the sake of forward references?#2017-01-1522:48bbloomie if i do (s/def ::foo ::bar) that fails#2017-01-1522:48bbloombut (s/def ::foo (s/and ::bar) works
#2017-01-1522:48bbloomwhich seems like a quirky way to accomplish this....#2017-01-1522:49bbloomi was hoping s/spec would do it, but sadly it resolves eagerly#2017-01-1523:35rickmoynihan> thatā€™s a decent hint to programmers, but itā€™s not quite good enough semantics for example, a generator will produce arbitrary values there, @bbloom: What else could a generator do though? As far as I can see it can only explicitly fail or generate a likely incorrect value#2017-01-1523:36bbloomiā€™d want it to fail#2017-01-1523:37bbloomi kinda also want to be able to test things w/ incomplete specs - like if i have (s/or :foo string? :bar some-crazy-thing?) i should at least be able to test the :foo path, even if :bar isnā€™t tested#2017-01-1523:37bbloomalthough iā€™d expect a warning#2017-01-1523:37rickmoynihanbbloom: then maybe just use the generator (fn [] (throw (ex-info ā€œUnspeced functionā€))#2017-01-1523:39bbloomthat solves the conform/instrument case (modulo fspec) but not quite the partial generator case - which i guess would require some changes to the generator monad thing? i dunno if it supports filtering#2017-01-1523:39bbloomgenerator would be like (fn [] (warn ā€œunspeced functionā€) (skip))#2017-01-1523:40rickmoynihanyeah - was just thinking you want them to flow#2017-01-1523:41rickmoynihanit seems you want something a little like a prolog cut#2017-01-1523:42bblooma little šŸ˜‰#2017-01-1523:42rickmoynihanyeah might be the opposite of that šŸ™‚#2017-01-1523:43bbloomi havenā€™t looked too deeply at writing generators yet, so i donā€™t know what the monad thing is about - i imagine itā€™s like mfilter or something#2017-01-1523:43rickmoynihanits not something Iā€™ve looked into#2017-01-1523:44bbloomah yeah, itā€™s ā€œMonadPlusā€ - basically you just lift map in to mapcat#2017-01-1523:44bbloomie you had map x -> y, now you have map x -> [y] and then you can filter by doing map x -> []#2017-01-1523:44bbloomi dunno if the generators support that or not šŸ™‚#2017-01-1523:44bbloomi assume they must#2017-01-1523:45bbloomsince they are generating multiple possible things#2017-01-1523:45bbloombut they might expect exaclty one item at a time, not zero-or-one item#2017-01-1523:46bbloomMonadPlus provides the ā€œzeroā€ value, ie [] for the list monad or just an empty stream of generated data#2017-01-1523:50rickmoynihanIā€™m still trying to get my head around exactly what being a partial generator meansā€¦ presumably itā€™s just something like (s/or :walk gen-me :don-walk unspecced-gen-ignore-me)#2017-01-1523:51bbloomwell, youā€™d walk it, but emit a warning and then not actually generate anything#2017-01-1523:51rickmoynihan(semantically - rather than in practice)#2017-01-1523:51bbloomyeah#2017-01-1523:51rickmoynihanwhy walk it? rather than ignore it?#2017-01-1523:52bbloomi guess thatā€™s the same thing#2017-01-1523:52gfredericksgen/such-that is the monad-plus part#2017-01-1523:53bbloomis the warn & skip idea doable?#2017-01-1523:53gfredericksbut it's not really monad-plus since it will just give up if it can't get the pred to pass#2017-01-1523:54gfredericks@bbloom warn and skip just applies to s/or specs?#2017-01-1523:54bbloomthatā€™s where it would be most useful, but i donā€™t see why it wouldnā€™t apply to any specā€¦ for example:#2017-01-1523:54bbloom(s/exercise unspeced)#2017-01-1523:55bbloomthatā€™d just print something like ā€œskipping unspeced at []"#2017-01-1523:55bbloomsomething like that at least#2017-01-1523:55bbloomfor s/or specs it would be much more useful b/c i could run partial tests#2017-01-1523:56gfrederickssounds possible, sure#2017-01-1523:57rickmoynihancanā€™t a spec be included anywhere though? And anywhere you include an unspeced spec wouldnā€™t it imply the specs had to effectively be rewritten to use an s/or over it?#2017-01-1523:58bbloomyeah - specs are implicitly a big and/or tree#2017-01-1523:59rickmoynihanprecisely#2017-01-1523:59rickmoynihanbut just thinking the s/and with an unspec in implicitly also becomes an unspec#2017-01-1523:59bbloomhhmm#2017-01-1523:59rickmoynihani.e. its contagious like a type#2017-01-1600:00bbloomnah, i think it would only pause that level of the tree#2017-01-1600:00bbloomlike if you did (s/and a? b? unspeced c? d?)#2017-01-1600:00bbloomthen you would check a and b, then issue a warning, but not even bother with c or d#2017-01-1600:00bbloomin theory it shouldnā€™t affect the context at all tho#2017-01-1600:01rickmoynihanfor checkingā€¦ but for gen?#2017-01-1600:01bbloomah, hmm#2017-01-1600:02bbloomyeah, i think it would annihilate specs until an or higher up in the context#2017-01-1600:02rickmoynihanyeah#2017-01-1600:02bbloomwhich matches my intuition for cut#2017-01-1600:02bbloomor continuations in general#2017-01-1600:02bbloomie skip is ā€œthrowā€ and every or has a catch#2017-01-1600:03rickmoynihanthats what I was getting at before#2017-01-1600:03bbloomgotcha#2017-01-1600:03bbloomanyway - this is a neat idea that i have no time to investigate implementing šŸ˜‰#2017-01-1600:03bbloomthanks for the discussion tho!#2017-01-1600:03rickmoynihanalso wonder if itā€™s kinda like choose#2017-01-1600:04bbloomchoose?#2017-01-1600:04rickmoynihanpretty sure it was called chooseā€¦ will need to dig out my ansi common lisp#2017-01-1600:10rickmoynihanok just dug it out - itā€™s in ā€œOn Lisp - Advanced Techniques for Common Lisp"#2017-01-1600:11rickmoynihanitā€™s basically just an operator in common lisp for doing this kinda thing - choose and fail#2017-01-1600:12rickmoynihananyway itā€™s not really helpful#2017-01-1709:05lmergenis anyone else struggling with a sudden ā€˜spec creepā€™ in their code ? how would one effectively determine what does and what does not belong in a spec ? e.g. if youā€™re accepting a file as a parameter, should you spec that this file exists, or should it only spec that it looks like a file ?#2017-01-1709:06lmergeniā€™m errā€™ing towards making spec only define what things should look like#2017-01-1709:21joost-diepenmaat@lmergen I would assume that checking for file existence is not very useful since you need to check that actually opening the file succeeded anyway (and do something relevant when that failed). a file existing during precondition does not guarentee that itā€™s still there when you try to access it#2017-01-1709:21lmergenthat makes sense#2017-01-1709:22joost-diepenmaatdepending on your code it may make sense to generate files though...#2017-01-1709:22joost-diepenmaatfor checking#2017-01-1709:23joost-diepenmaatpersonally Iā€™d try to avoid messing with the filesystem in tests and write as much as possible in terms of sequences or whatever makes sense to your problem#2017-01-1709:26joost-diepenmaatThe idea with speccing function arguments as I understand it is to describe whatever input data that is supposed to be handled. In this case, handling non-existing files is sort of necessary given the mutable nature of the file system.#2017-01-1709:26dottedmag@lmergen My rule of thumb: spec predicates should be pure functions.#2017-01-1709:28dottedmagAnother one: behaviour of the system should not change if spec checking is disabled (except for the boundaries where spec is explicitly used for parsing/destructuring external data). So all runtime error handling (file not found etc) ought to be in the function itself, and spec should only prevent programming errors.#2017-01-1709:29lmergenwhat about s/conform ?#2017-01-1709:31joost-diepenmaatif youā€™re using that explicitly in your function body then youā€™ve sort of guaranteed the specs canā€™t be disabled šŸ™‚#2017-01-1709:31joost-diepenmaatsame with macros#2017-01-1709:32joost-diepenmaatthatā€™s the parsing case that @dottedmag is talking about I think#2017-01-1709:35dbushenkoare there any libs wich already have specs? just wanted to see some good examples of using specs#2017-01-1709:37dottedmagThere are some specs for clojure.core already. Mostly macros.#2017-01-1709:37dbushenkonice!#2017-01-1712:38joshjones@lmergen I found myself asking this question recently, and reading the rationale/motivation for spec made things a little clearer. while I like @dottedmag ā€˜s ā€œonly pure functionsā€ rule of thumb, the spec guide itself has a section called ā€œUsing spec for validationā€ that includes several runtime validation examples which are not at the edge of the system at all, but which are deeply integrated. That level of integration makes me a bit nervous, but itā€™s clear that spec is integrated enough that this is a valid enough use case, at least for some people.#2017-01-1716:57tetriscodesIā€™m struggling with how much spec right now#2017-01-1716:57tetriscodesIā€™m thinking of putting it only where developers use the system (http endpoints, components apis,ā€¦)#2017-01-1716:58tetriscodeshttps://github.com/boundlessgeo/spatialconnect-server/tree/master/server/src/spacon/specs#2017-01-1716:59tetriscodesIā€™ve seen the most benefit in my system from using the generators after specing. Its found more bugs in there than our devs have seen using the functions that are specā€™ed.#2017-01-1718:20angusiguessHas anyone had to spec something that might be used by older versions of clojure?#2017-01-1718:20angusiguessWhat's a good way of providing spec that's only read for 1.9+?#2017-01-1719:53gfredericksseperate namespace for the specs#2017-01-1802:22bbloomthis room might appreciate this: https://gist.github.com/brandonbloom/7a76d55ffeaf4ecc46832060d659dfc1#2017-01-1802:22bbloomi could imagine such a thing being integrated with spec, but not exactly sure how#2017-01-1802:27bbloomiā€™m using it to do something akin to constraint propagation#2017-01-1802:30bbloomlots of other details to work out with it tho - like what to do if a contradiction occurs#2017-01-1807:25ikitommiAre there any news on the next steps & roadmap for the spec? Trying to develop the runtime dynamic conforming stuff on top of spec. Would love to see Specs as types/records instead of reified protocols so that one could extend the specs easier. I'm toying now with stripping out extra keys from map specs & optionally giving errors for extra keys. Both at runtime, which is cool at api borders.#2017-01-1807:28ikitommie.g. clojure.spec/KeysSpec. could make it implement protocols to extract itā€™s conforming type & to list itā€™s keys. and to generate docs etc.#2017-01-1807:32ikitommithis works now, but requires explicit conforming type hints (to tag specs):
(require '[clojure.spec :as s])
(require '[spec-tools.core :as st])

(s/def ::a int?)
(s/def ::b keyword?)
(s/def ::c string?)
(s/def ::ab (st/spec ::map (s/keys :req [::a ::b])))
(s/def ::cab (st/spec ::map (s/keys :req [::c] :opt [::ab])))

(st/conform
  ::cab
  {::c "c"
   ::ab {::a 1
         ::b :b
         ::EXTRA "SHOULD BE REMOVED"}
   ::EVIL "SHOULD BE REMOVED"}
  {::map st/remove-extra-keys})
; => #:user{:c "c", :ab #:user{:a 1, :b :b}}
#2017-01-1814:07souenzzoHi! https://clojure.org/guides/spec#_using_spec_for_validation https://clojure.org/guides/spec#_spec_ing_functions There is a way to "integrate" the two uses?? Or I will need to specify the function s/fdef,,,,, and valid the input :pre s/valid?,,,, separately?#2017-01-1814:39mpenet@ikitommi did you notice https://github.com/uswitch/speculate#2017-01-1814:39mpenetit's on my "kick tires" short list#2017-01-1814:40mpenetbut yes, an update would be nice indeed#2017-01-1815:14ikitommithanks @mpenet for the link. Spectrum, spec-tools and speculate are all trying to convert specs into something that is easier to mold - with custom records, walkers & ast. The upcoming(?) specs of specs will make them easier to interpret, but records would give a one way to extend them. #2017-01-1815:19mpenethard to know how to do something like that until spec stabilizes really#2017-01-1816:03souenzzoIs someone getting pull-patterns from specs?#2017-01-1816:21kspearDoes spec enable transformation from one 'spec' to another? i.e. could I take a map {:name "Bruce" :email " and transform it to conform to another spec {:name-first "Bruce" :contact {:email "? I feel like that might be possible with conform/unform or some other spec function, but I have yet to figure it out. Any help or direction would be appreciated.#2017-01-1817:31seancorfieldYes, butā€¦ bear in mind that all users of your spec would then get the transformed result, which is why that sort of thing is generally frowned on.#2017-01-1817:32seancorfieldFWIW, we use specs that conform strings to non-strings as part of API input handling, but we try to avoid changing the shape of the data, unless thereā€™s a very specific reason for that (and we want all spec client code to experience that change).#2017-01-1817:32luxbock@seancorfield I took his question to mean if there might be an automated way of translating data that has the same components but different structure by specying two specs and maybe some mapping between sub-components#2017-01-1817:35seancorfieldAh, I see what @kspear means now. TL;DR: ā€œnoā€ šŸ™‚ Thatā€™s a general data transform issue thoā€™ ā€” even with specs, youā€™d have to be able to specify exactly what parts of one spec are transformed to which parts of another.#2017-01-1817:36luxbockif you have specs-of-specs perhaps you could automatically map certain ident-kw's to each other#2017-01-1817:47lmergenI think then you're starting to build a type system, where you define rules how types can be casted #2017-01-1817:47lmergenI think this is something Clojure wants to avoid, and wants to make these transformations explicit #2017-01-1817:49kspearYeah-- it seems like the sort of thing I could do 'easily' enough without spec at all; the thought occurred when I was playing around with conform/`unform`. I can, for example, use s/cat to say that each value in a list corresponds to some key and spec and see it in the 'specced' form with (conform ::my-first-spec my-list); then, i can unform it with a different spec to re-arrange the values into a different format (conform ::my-second-spec (conform ::my-first-spec my-list)). I thought if such a thing were possible on a list, there might be a way to do it with a map.#2017-01-1817:55kspearI figure a bit of context might help. In this specific case I'm taking reports generated (XML and json files, typically) by third party tools (vulnerability scanners) and I want to be able transform each 'issue' in each report into a common internal format within my program. Then, I want to be able to transform issues in my internal generic format to a JSON object that I can pass to our issue tracker API (i.e. JIRA). I'm just wondering if there is a way to specify what all of those formats look like (the data I get from my various third party tools, my internal generic issue format, JIRAs create issue format) and transform between them taking advantage of spec.#2017-01-1818:04lmergenyeah, I think that just mapping them might be better, you want to make these types of transformations explicit #2017-01-1818:32Alex Miller (Clojure team)Iā€™d say the purpose of spec is not a generic data transformation library#2017-01-1818:34Alex Miller (Clojure team)and Clojure itself has plenty of tools for doing that (outside of spec)#2017-01-1818:38Alex Miller (Clojure team)conform/unform can be used to move back and forth between the original and conformed version of data, but pushing this use into transformation leads to treating spec as a ā€œmeat grinderā€ (Richā€™s words) rather than something that states a specification of your data. in the long run, I think that path will cut you off from some uses of spec as you are no longer using it as a specification but as a process.#2017-01-1818:42mpenetWe use spec a lot. Never to coerce data tho. We actually use specter for this #2017-01-1818:43mpenetNow that I think of it l dont think we use coercers at all #2017-01-1818:43Alex Miller (Clojure team)+1 :)#2017-01-1818:43mpenetI mean directly#2017-01-1818:44Alex Miller (Clojure team)@ikitommi Iā€™d say it is still unresolved what the final implementation form will be for specs (protocols, records, or something else) but regardless we would not encourage direct access to those internals (and in fact the likelihood of people abusing that access is considered to be one downside to records as impls). The encouraged path to move from a spec (which is data, but in a form) to more map-like data is to conform the specā€™s form.#2017-01-1818:45mpenetWe did go through a phase where we did use coercers for transforms, but imho it gets messy fast#2017-01-1818:46Alex Miller (Clojure team)yes#2017-01-1818:46Alex Miller (Clojure team)spec works best when you primarily use it to speak truthful, assertive statements about what your data is#2017-01-1818:47Alex Miller (Clojure team)and not to transform your data or make statements about what your data is not#2017-01-1818:48mpenetOne of the thing I like is togglable asserts in pre/post#2017-01-1819:00hiredmanonce you've created data transformation functions, and spec'ed them, you can examine the spec registry to find transforms with matching inputs and outputs to derive larger transforms#2017-01-1819:17kspearThanks everyone so far for the responses. Definitely helps. In my specific case, I'm getting this third party data that, in essence, is the same data, just in proprietary representations. My assumption was that, using spec, I could author specs that define what the common data is in these different representations and leverage those to extract the common data into a generic representation. If spec isn't the answer to this problem, I have to assume there is another idiomatic approach to this type of problem in clojure already?#2017-01-1819:19joshjonesyour first example seemed to imply that youā€™re just talking about different key names for the same basic map structure ā€” is that accurate, or was your example oversimplified?#2017-01-1819:20Alex Miller (Clojure team)well I think all that is fine if youā€™re talking about specifying the data in either the different representations or the common representation and where it is less so is when itā€™s about transformation#2017-01-1819:22kspear@alexmiller Aye, ok. So realistically I should be writing transformation functions like (defn external-format-1->common-format [data] ...), and then using spec to validate the input (external format spec) and output (my common format spec).#2017-01-1819:25kspear@joshjones Simplifying a bit. I need to rename keys, but also apply some transformation to the corresponding values (i.e. one external format might represent a date as a string with a particular ISO format that I then want to transform it to something compatible with the clj-time library). The structure, also, is going to be different between sources.#2017-01-1819:26Alex Miller (Clojure team)yeah, thatā€™s not specā€™s job (imo)#2017-01-1819:26Alex Miller (Clojure team)even though you could coax it do so with conformers etc#2017-01-1819:27Alex Miller (Clojure team)but you could for example write a spec for both sides and then write code that read both specs and built the transformation between them for you#2017-01-1819:29joshjonesso in this case @alexmiller , youā€™re not using spec to do the actual transformation, but to describe the two pieces, and then you write something to bridge the two specs ā€¦ is that kind of the idea? kind of like keeping spec ā€œcleanā€ and letting something else do the dirty work.#2017-01-1819:30Alex Miller (Clojure team)right (although I object to clean/dirty connotations :)#2017-01-1819:30joshjoneslol#2017-01-1819:30Alex Miller (Clojure team)in that I think spec is good at specifying and that clojure transformation functions are good at transforming#2017-01-1819:31Alex Miller (Clojure team)and you should use the right tool for the job#2017-01-1819:32gfredericksman I seriously need to go take a year off in the woods and figure out how to do bijections right#2017-01-1819:32Alex Miller (Clojure team)the first 80% is easy#2017-01-1819:32Alex Miller (Clojure team)the last 10% is impossible :)#2017-01-1819:33Alex Miller (Clojure team)in other words, the usual :)#2017-01-1819:33Alex Miller (Clojure team)seriously though, bijections over spec would be a fantastic lib#2017-01-1819:34gfredericks@alexmiller did you ever see this https://github.com/gfredericks/schema-bijections#2017-01-1819:34Alex Miller (Clojure team)nope#2017-01-1819:35Alex Miller (Clojure team)actually even narrowing the problem from json <-> clojure would be interesting in that itā€™s clearly the most common need for this and might be able to capture a bigger piece of the pie with default behavior#2017-01-1819:36gfredericksevery time I think about that I get grumpy that spec doesn't have a s/string-keys#2017-01-1819:37gfredericksbut maybe that can be done in userland#2017-01-1819:38gfredericksthe two things I hate about that ā‡‘ library are A) I think surjections/coercions should be first-class, hooking in to generators, and B) the composition API is terrible and I don't know how to make it better#2017-01-1819:40trisshey all. has any one looked at generating WSDL from specs yet?#2017-01-1819:44Alex Miller (Clojure team)I am professionally obligated to question your use of WSDL in the first place :) moving past that, I have not seen anyone mention it#2017-01-1819:45trisshah indeed! does seem like a messy bit of techā€¦ not sure it solves what it claims to or not. Seems to me a lot of it could be generated though#2017-01-1819:45trissIā€™m just looking at it for academic/comparitive reasons#2017-01-1819:46Alex Miller (Clojure team)from my experience, wsdl is almost always generated (which is why itā€™s so bad)#2017-01-1819:46Alex Miller (Clojure team)people have primarily used it to write giant xml docs that describe an rpc call#2017-01-1819:47trissyup. loose coupling. did that ever come really? I suppose weā€™ve abstracted over the lower level comms protocols at least#2017-01-1819:49Alex Miller (Clojure team)Rich has a lot of ideas about this area, many of which are in harmony with spec he teased those in the last conj keynote, but they are worthy of a lot more time#2017-01-1819:52donaldballMany folk are using swagger to do the job WSDL used to pretend to do, though I think it may be called openapi now#2017-01-1819:54trissthanks @donaldball Iā€™ll take a look.#2017-01-1821:01ikitommithanks @alexmiller. looking forward to the new alphas. #2017-01-1821:33hiredmanI've been using spec internally, and generating json schemas from specs for external interfaces (and I think someone else had a github repo or a pastebin doing it too), and I've been thinking about using it to generate graphql types (but I am not sure if I am going to use graphql) and the same for protobufs. Because spec is clojure data it is great for using the same spec to generate lots of other kinds of schemas. You take the same spec data and interpret it different ways.#2017-01-1823:25qqqI have a three line spec that looks like:
(s/def ::node any?)
(s/def ::children (s/coll-of ::tree :kind vector?))
(s/def ::tree (s/keys :req-un [::node ::children])))
;; then I call generate on it via
(gen/sample (s/gen ::tree))
this is getting me stack overflow, probably due to generating infinite depth tree how cna I make generating trees work?
#2017-01-1823:30gfredericksI thought spec handled that via some depth limit#2017-01-1823:30qqqGiven how well thought out it is, it probably did, and I, asa a spec noob, simply am not using it correctly.#2017-01-1823:30qqqHow do I use depth limits?#2017-01-1823:31gfredericksIf that doesn't help, you can build a generator manually in test.check via the recursive-gen function#2017-01-1823:31gfredericksI don't know, I'd search the two namespaces for anything with the word depth in the name#2017-01-1823:32jrhttps://github.com/clojure/clojure/blob/master/src/clj/clojure/spec.clj#L19#2017-01-1823:35qqq(def ^:dynamic recursion-limit "A soft limit on how many times a branching spec (or/alt/*/opt-keys/multi-spec) ^^ I'm sorry, can you please explain how this helps with my ::children / ::tree recursion via s/coll-of ?#2017-01-1823:35qqqI don't seem to be using any of the functions they 'limit'#2017-01-1823:36gfredericksThat's probably the problem#2017-01-1823:37qqqHow do I rewrite my spec to use those functions? šŸ™‚#2017-01-1823:37gfredericksSo you want node values on internal nodes too?#2017-01-1823:37gfredericksNot just leaves? #2017-01-1823:37qqqYes, both leaves and internal nodes can store values.#2017-01-1823:37qqqAre you suggesting separate classes and and (s/or ::internal ::leaf) ?#2017-01-1823:38gfredericksThat might fix it, yeah#2017-01-1823:38gfredericksOnly other idea I have is the manual generator idea from earlier#2017-01-1823:39gfredericksSeparate classes is probably easier#2017-01-1823:40qqqSpec is important enough that. I want to learn both.#2017-01-1823:40qqqDo you know of a good tutorial for manual generators? seems like I'm going to run into it sooner or later#2017-01-1823:41gfredericksThere are some docs in the test.check repo#2017-01-1823:45bfabrythere's a screencast on manual generators#2017-01-1823:46qqq@gfredericks : I can't get the two class approach to work; would you mind showing me sample code?#2017-01-1823:46qqqthis is what I have so far:#2017-01-1823:46bfabryhttp://blog.cognitect.com/blog/2016/8/10/clojure-spec-screencast-customizing-generators#2017-01-1823:47seancorfieldThis feels like a great use case for the new threaded messages feature in Slack...#2017-01-1823:47bfabrythat's actually the last in a series of 3 casts, they're very good#2017-01-1823:47qqq` ::nt->node any? ::nt->children (s/coll-of ::nt :kind vector?) ::nt-leaf (s/keys :req-un [::nt->node] ::nt-internal (s/keys :req-un [::nt->node ::nt->chidlren] ::nt (s/or ::nt-leaf ::nt-internal)#2017-01-1823:47qqqbut when I try to gen on ::nt, I get some error about nt-leaf nt->children nt-leaf nt->chlidren ...#2017-01-1823:47bfabrywhen does threading drop? this week right? I've been missing threading since flowdock died#2017-01-1823:47seancorfieldRefresh now -- it's enabled for this Slack already.#2017-01-1823:49qqqthis is the new ->> ?#2017-01-1823:49bfabryseancorfield: oh sweet. aaaaand I'm disappointed. this is not low friction. oh well#2017-01-1823:50seancorfieldYes, so you and Gary can chat in a thread about your specific issue at length and then just report a summary back to the channel -- and folks can read all the details if they want, or be spared them if they don't.#2017-01-1823:50qqqinteresting#2017-01-1823:51qqqI'd still like an ignore button though šŸ™‚#2017-01-1823:52bfabryyeah... and no one will notice a conversation they can help out with casually and chip in, like I just did with the screencast link for qqq. meh. feels like if people use it low traffic slack channels will be even less useful than they are now. oh well#2017-01-1823:55qqqresolved it: I was mis using (s/or .... )#2017-01-1900:11seancorfield(You could / should have marked that last reply as ā€œAlso send to #clojure-specā€ so it would appear back in the main channel)#2017-01-1823:52bbloomi fear this is one of those ā€œgood ideas on paperā€ sort of things...#2017-01-1823:52bbloomthe slack threads, that is#2017-01-1823:56qqqI don't like threads either; what I want is a way to "mute" people taht seem way too chatty; but the problem is, those ppl are probably not goign to voluntarily start a thread#2017-01-1900:12seancorfieldqqq: We (admins ā€” and the community at large) can educate them to do soā€¦ Threads will just take some getting used to, like all changes šŸ™‚#2017-01-1823:56qqqs/"mute"/ignore#2017-01-1823:56qqqtrees via 2 classes works now#2017-01-1823:57qqq@bfabry : going to study your video on manual generators now#2017-01-1823:58bfabryqqq: not mine! stuart holloway's work#2017-01-1902:54qqqjust worked through taht video ; am now convinced that stuart holloway is a genius#2017-01-1900:00qqqsomethign else tht would be nice in spec would be to gnerrate recursive structures breadth first, and when we hit a certain node count, only take non recursive branches#2017-01-1900:18qqqis there a spec option to limit the size of coll-of ?#2017-01-1900:28qqqwhere is the code which shows how coll-of are generated?#2017-01-1900:29qqqI'm looking at github/clojure/clojure, which sees to show checking of coll-of, but not generating of coll-of#2017-01-1900:36Alex Miller (Clojure team):gen-max is the option to limit coll-of#2017-01-1902:57qqqdoes clojure spec support things like parameterized spec? I want to be able ot say things like "this is a ::tree-of ::widget" and "this is a ::tree-of ::gadget" instead of havint to define ::tree-of-widget and ::tree-of-gadget (or one for each 'type')#2017-01-1903:01seancorfieldNo.#2017-01-1903:12Alex Miller (Clojure team)you can however use s/multi-spec to create an ā€œopenā€ thing in the middle#2017-01-1903:14seancorfieldAh yes, I need to into multi-spec some day šŸ™‚#2017-01-1903:29qqq@alexmiller: I have no idea what this multi-spec technique is. Can you point me at a tutorial?#2017-01-1903:37Alex Miller (Clojure team)http://clojure.org/guides/spec has an example#2017-01-1903:38Alex Miller (Clojure team)but basically you use a multimethod to choose which spec to use based on the data#2017-01-1912:26roman01laThis question probably was raised already. Iā€™ve played with spec in ClojureScript a little, created a spec for application state for validation purpose. But I found it annoying to use fully-qualified keys with maps, because it requires the namespace whenever I need to get or destruct a map. This is what stopping me from using spec in production code. Am I missing something? I feel like there should be a reasonable explanation for this.#2017-01-1912:27gfredericks@roman01la there are some new syntaxes for namespaced keywords#2017-01-1912:29roman01la@gfredericks do you mean namespace alias? Iā€™m using them.#2017-01-1912:30gfredericksfor constructing maps whose keys all have the same namespace#2017-01-1912:30gfredericksalso {::keys [foo bar baz]} I think#2017-01-1912:30gfrederickswhen destructuring#2017-01-1912:31gfredericksthe map syntax is #:foo{:bar 12} and #::baz{:bang 12} I think#2017-01-1912:31gfredericksyou should see it at the repl when it prints a map with namespaced keys#2017-01-1912:32rickmoynihan@roman01la: Not sure if this is what you mean by "require" you donā€™t need to :require and load a namespace to use keys that are fully-qualified#2017-01-1912:33gfredericksbut you do to alias#2017-01-1912:33rickmoynihanyes#2017-01-1912:35rickmoynihanroman01la: also not sure if you know but s/keys has :req-un and :opt-un to let you bridge between unnamespaced keys in maps and namespaced keys in specsā€¦#2017-01-1912:36roman01la@gfredericks thatā€™s great, thanks! #::baz{:bang 12} doesnā€™t work in ClojureScript, probably not implemented yet#2017-01-1912:36roman01la@rickmoynihan yes, thanks#2017-01-1912:37gfredericks@roman01la: if the clojurescript compiler runs in clojure 1.9 does it work?#2017-01-1912:40roman01la@gfredericks didnā€™t try it on 1.9 yet#2017-01-1912:42gfredericksif cljs still uses clojure's reader then that'd be a prereq for sure#2017-01-1912:48roman01la@gfredericks tried with 1.9.0-alpha14, still doesnā€™t work. I guess ClojureScript has itā€™s own reader. Anyway, thanks for sharing the new syntax, this will save me some time!#2017-01-1912:48pyr@mpenet can you expand on what you mean by "One of the thing I like is togglable asserts in pre/post" ?#2017-01-1913:20mpenet@pyr using :pre/:post in fn definitions, mostly s/valid? calls. and having the abitily to switch it on/off depending on assert value at compile time#2017-01-1913:20mpenet*assert*#2017-01-1913:21mpenetex with a profile with :global-vars {assert false} for prod#2017-01-1913:21mpenetyou can do the same with s/assert, inline, but I find pre/post kind of nice for this stuff#2017-01-1913:21mpenets/assert has the advantage that you can toggle in on/off at runtime#2017-01-1913:24pyryup ok#2017-01-1913:24mpenetall of this is handy for the cases when instrumentation is either too intrusive or when you want to actually run this stuff in prod#2017-01-1913:56pyr@mpenet thanks, i'll play with that later today to see how I fare#2017-01-1913:57pyrI didn't know :pre/:post validation asserts could be en/disabled at compile time#2017-01-1914:03mpenetI found that old post talking about it, old stuff šŸ™‚ http://blog.fogus.me/2009/12/21/clojures-pre-and-post/#2017-01-1914:04pyrI was reading that yes, still first google link to show up šŸ™‚#2017-01-1914:05mpenetI guess it could make it relatively easy to write some sugar on top of it to have Schema like defn with annotations.#2017-01-1914:05mpenetBut personally I am good with what we have#2017-01-1914:06pyryes, this is also enough for my needs#2017-01-1915:18owenis there a good way to test that an fdef with a generator will throw an exception somewhere? I'm not having success#2017-01-1915:59Alex Miller (Clojure team)can you expand that question? not understanding it#2017-01-1916:04owenSure thanks Alex. I'm testing a function with stest/check, an fdef and a generator that is guaranteed to cause an exception. I can't figure out what :ret I can use in the fdef to check for the exception#2017-01-1916:06Alex Miller (Clojure team)you canā€™t#2017-01-1916:06Alex Miller (Clojure team):ret is for non-exceptional returns#2017-01-1916:07Alex Miller (Clojure team)there is no way to spec exceptions#2017-01-1916:08owenAh ok makes sense then#2017-01-1916:16ddellacostawhat is the rationale behind clojure.spec/or returning map pairs? I was hoping to be able to use it to validate what is essentially a sum type, but seems like I will have to do more processing to handle that. I guess my other question is, is there anything suitable for validating a sum type (i.e, simply has a bunch of predicates, and is valid if one of the predicates returns true)?#2017-01-1916:16ddellacostahappy to get pointed to docs; apologies if this has been asked many times here#2017-01-1916:18ddellacostaI do see the doc-string says > Returns a destructuring spec that returns a map entry containing the key of the first matching pred and the corresponding value. Thus the 'key' and 'val' functions can be used to refer generically to the components of the tagged return.#2017-01-1916:19ddellacostaI sense the rationale is lurking in there, but I suppose this speaks to my larger ignorance of how to use spec idiomatically#2017-01-1916:19ddellacostasomething something use maps more something something#2017-01-1916:27joshjonesMaybe Iā€™m oversimplifying your question @ddellacosta , but when you conform using s/or, how would you know which predicate matched without some way to identify it?#2017-01-1916:28Alex Miller (Clojure team)the idea is that the tag is used to identify how spec parsed the value (and also used to identify problems in explain errors)#2017-01-1916:28ddellacosta@joshjones why do I care if itā€™s a valid value of one of the types Iā€™m expecting?#2017-01-1916:28ddellacosta@alexmiller I guess I donā€™t see how to easily use that when composing it with another spec#2017-01-1916:29Alex Miller (Clojure team)many specs conform to a structure that describes how they were parsed#2017-01-1916:29Alex Miller (Clojure team)all of the regex specs, etc#2017-01-1916:29ddellacostayeah, there seems to be something basic Iā€™m missing here#2017-01-1916:29Alex Miller (Clojure team)two options for not having this issue specifically with s/or...#2017-01-1916:30Alex Miller (Clojure team)1. write a function that accepts all the options, so instead of (s/or :e even? :p pos?), do #(or (even? %) (pos? %))#2017-01-1916:31Alex Miller (Clojure team)that makes the choice opaque (which may be good or bad depending on your needs)#2017-01-1916:32Alex Miller (Clojure team)2. use the currently undocumented and possibly to-be-removed spec s/nonconforming: (s/nonconforming (s/or :e even? :p pos?))#2017-01-1916:33Alex Miller (Clojure team)if you just want to check whether a value is valid, use s/valid?. if you want to know why itā€™s valid, use s/conform. If you want to know why itā€™s invalid, use s/explain.#2017-01-1916:34ddellacosta@alexmiller thanks for all of that#2017-01-1916:34ddellacostaI guess part of the problem is that Iā€™m using s/conform and expecting it to more or less simply return the value Iā€™m testing, except in specific cases (changing vector to set, in this case I think)#2017-01-1916:34ddellacostaand part of that is maybe that I donā€™t yet understand the use-case for s/conform#2017-01-1916:35ddellacostaand how one would use something like s/or effectively#2017-01-1916:35ddellacostawithin that context I mean#2017-01-1916:35Alex Miller (Clojure team)yeah, s/conform returns a different value than the original value specifically to tell you why/how itā€™s valid#2017-01-1916:35Alex Miller (Clojure team)if you donā€™t care, donā€™t use conform#2017-01-1916:35ddellacostaright, okayā€¦that is a helpful way to think about it#2017-01-1916:36Alex Miller (Clojure team)note also that you can use s/unform to retrieve the original value from the conformed value too#2017-01-1916:37ddellacostaI guess thinking about it a bit more, I see two distinct possible values that could be returned from conform: one is the value coerced to a new value, and the other has added spec metadata ā€”it seems to me right now that s/conform conflates these two somewhat#2017-01-1916:37ddellacostafor example, converting a vector to a set is the former#2017-01-1916:37ddellacostawhereas the output of s/or is the latter#2017-01-1916:38ddellacostaam I off here? I still feel like I donā€™t get the broader implications of spec, or how we might be able to use it#2017-01-1917:16seancorfield(if (s/valid? ::my-spec some-value) some-value (handle-error (s/explain-data ::my-spec some-value))) seems to be what youā€™re looking for @ddellacosta ?#2017-01-1917:16seancorfield(or some similarly structured setup)#2017-01-1917:18seancorfieldWe use both s/valid? and s/conform ā€” we use the latter when we want a result that ā€œconformsā€ to a spec but may start off slightly differently (which we use for our API inputs, to produce domain model values). And as Alex said, when we donā€™t care about any potential change in the data, we use the former.#2017-01-1917:29ddellacosta@seancorfield thanks, will see if that does it. Per your second point, ideally Iā€™d like to use s/conform in the same wayā€”but if s/conform returns data that has been changed to represent how a value conforms to a spec, doesnā€™t that defeat the purpose?#2017-01-1917:30seancorfieldWell, it will only do so under the rules of the spec itself.
#2017-01-1917:30lmergen@seancorfield: i would be very interested to learn how you use spec in real world projects #2017-01-1917:30lmergenwe're introducing it right now, running alpha 14#2017-01-1917:31lmergenand it's a bit tough to find the right "pattern" that makes sense to us#2017-01-1917:31seancorfieldFor example, we have API input specs that accept either a date, or a string that can be parsed to a date using two allowed formats. s/valid? on such a string will just return true (and we still have a string; or false for an illegal string). s/conform on such a string will return a date (or ::s/invalid for an illegal string).#2017-01-1917:33joshjonesso in this case is it accurate to say that you are using spec to actually transform the data? (string -> date)#2017-01-1917:34seancorfieldAnother use case: we allow members to search within specific distances (dating site). If they are a subscriber, they can specify distances of 20, 30, 40, 50. All members can specify distances of 150, 500, 1000, and -1 (ā€anyā€). We use a spec with s/or to get back either [:basic distance] or [:platinum distance] so we can validate both the actual value and which category it belongs to.#2017-01-1917:35seancorfield@joshjones Yes, some coercions. As Alex often cautions, that conforming / coercing means all clients of those specs get the coercion if they use s/conform but in this case thatā€™s what we want.#2017-01-1917:35ddellacostato me that seems like a spec is somewhere in between a value and a statement about a value#2017-01-1917:35ddellacostaI find this pretty confusing, fundamentally#2017-01-1917:35ddellacostaseems like it makes s/conform of limited use#2017-01-1917:35ddellacostaor at least, limited in the case of using s/or#2017-01-1917:36joshjonesyes yesterday @alexmiller said: ā€œspec works best when you primarily use it to speak truthful, assertive statements about what your data is and not to transform your data or make statements about what your data is notā€ https://clojurians.slack.com/archives/clojure-spec/p1484765212007116 https://clojurians.slack.com/archives/clojure-spec/p1484765279007117#2017-01-1917:36ddellacostaagain, Iā€™m talking about s/conform here @joshjones#2017-01-1917:37ddellacostaand it seems to violate exactly what those comments are saying, in fact#2017-01-1917:37seancorfieldIf you need to know which ā€œpathā€ of conformance your value takes, s/or is very useful. Since otherwise youā€™d need to code up both the validation logic and the parsing logic in order to determine that.#2017-01-1917:37joshjonesIā€™m not referring to anything youā€™re talking about @ddellacosta , only to what @seancorfield said about his use of spec#2017-01-1917:37ddellacosta@joshjones ah, apologiesā€”misunderstood#2017-01-1917:37joshjonesno problem, I am also seeking a better understanding of best practice šŸ‘:skin-tone-2:#2017-01-1917:40ddellacostayeah, along those lines this is all really helpfulā€”thanks for linking to those comments @joshjones#2017-01-1917:42seancorfieldItā€™s taken us a while to settle on this approach ā€” given that Alex has repeatedly cautioned against specs that coerce data šŸ™‚ ā€” and we also have the added complication that for a given API, some of our input parameters must be looked up and ā€œconformedā€ dynamically (i.e., we have a database table listing possible inputs and each row identifies a spec that should be applied). So we canā€™t just write static specs for input arguments, we have to have somewhat generic specs and conform each input argument based on a lookup.#2017-01-1917:43seancorfield(and weā€™re still wading through cleaning that up)#2017-01-1917:43seancorfieldWe could probably simplify this by dynamically generating specs directly from our metadata at this point ā€” and we may well do so.#2017-01-1917:44ddellacostaso question for you, @seancorfield ā€”why not simply write something explicit that does not rely on spec to add the metadata about your distance (platinum vs. basic)? It seems to me that this is a value, not a spec.#2017-01-1917:44ddellacostaIā€™m asking selfishly to clarify my understanding of spec, to see how you are thinking about this, as I still donā€™t fully get it#2017-01-1917:45seancorfieldBecause using spec allows us to remove a lot of logic that we had before.#2017-01-1917:47seancorfield
(s/def ::platinum-distance #{20 30 40 50})
(s/def ::basic-distance    #{150 500 1000 -1})
(s/def ::distance (api-spec ->long (s/or :basic ::basic-distance
                                         :platinum ::platinum-distance)))
(and the api-spec macro) replaces a bunch of explicit code that attempted to coerce string to number and validate it and categorize it as basic vs platinum. Now we can just have (let [[level distance] (s/conform ::distance input)] ā€¦) and weā€™re done.
#2017-01-1917:48seancorfieldHaving the declarative specs means we can show those to business and they can ā€œread the codeā€ without having to actually read real code full of conditionals and transformations.#2017-01-1917:49seancorfieldWe can also have the business rules in one place (for the most part) and they drive not only our actual business logic but can also be used to generate test data (and test functions extensively).#2017-01-1918:46ddellacostaokay, Iā€™ll have to think about that a bit. Thanks a lot @seancorfield , this has been very helpful#2017-01-1920:50scriptorhow would one spec varargs?#2017-01-1920:51scriptorcould I just use the vararg's name and use coll-of?#2017-01-1920:55schmeescriptor I think the idea is to spec the args as a sequence with spec/cat and use spec/? for the optional args#2017-01-1920:56joshjones@scriptor yes, compose the var args the same way youā€™d compose any sequence.. only it doesnā€™t have to be coll-of ā€¦ can be k/v pairs, any sequential spec. for example:
(s/def ::someopt nat-int?)
(s/def ::vargs (s/cat :optargs (s/keys* :opt-un [::someopt])))

(defn myfunc [& optargs] nil)
(s/fdef myfunc
        :args ::vargs)
(stest/instrument `myfunc)
(myfunc :someopt 55 :otheropt 99)
#2017-01-1920:56joshjonescan be a coll-of, use spec regexes, anything ā€¦ the above just shows a common use case for var args: k/v pairs in a sequence (as opposed to a map)#2017-01-1920:59scriptorso :rest indicates the following spec is for the optargs parameter?#2017-01-1920:59scriptorwell, not indicates, you're just using the name :rest#2017-01-1921:00joshjonesyes, should probably have called that optargs .. will change it for clarity#2017-01-1921:00scriptorperfect, thanks @schmee and @joshjones#2017-01-1921:24joshjones@scriptor I need to clarify something after doing another example ā€” my example above works but I wanted to use another common case because I was wrong about just using coll-of for cases where you have one required and then an optional argument. for example:
(s/def ::vargs (s/cat :req string? 
                      :optargs (s/* int?)))
#2017-01-1921:24joshjonesif you use coll-of for :optargs it will not work because youā€™re giving it multiple items, not an actual collection#2017-01-1921:25scriptorah, interesting, I guess it's because of the regexy nature of s/cat#2017-01-1921:26joshjoneswell, s/cat gives the actual argument vector spec. but inside that, you then have several elements, and one of those is not a collection if itā€™s for example: [ā€abcā€ 42 43 44]#2017-01-1921:26scriptorthe s/* greedily consumes the rest of the arglist, which as you said is multiple separate items and not a single collection#2017-01-1921:26joshjoneswhat you have there is a string, an int, an int, and an int .. no collections#2017-01-1921:27joshjonesyes thatā€™s correct. i feel my answer above was misleading so i wanted to clarify#2017-01-1921:43joshjonesanother example that will match (fn [s & opts]):
[ā€abcā€]
[ā€abcā€ 3/4]
[ā€abcā€ 3/4 42.9]
[ā€abcā€ 3/4 33.3 10.5 99.9]

(s/def ::vargs
  (s/cat :req string? 
         :optargs (s/? (s/alt :ratioonly ratio? 
                              :ratio-double (s/cat :s ratio?
                                                   :i (s/+ double?))))))
#2017-01-1921:44joshjoneswanted to show that the optional argument portion of the sequence can be more than just a simple homogeneous concat of elements#2017-01-1921:54Alex Miller (Clojure team)you should really use s/keys* for this - that is itā€™s reason for existence#2017-01-1921:55Alex Miller (Clojure team)(s/def ::varargs (s/cat :req string? (s/keys* :opt-un [::ratioonly ::ratio-double]))) etc#2017-01-1922:06joshjones@alexmiller that was in my first example above ā€” but doesnā€™t that require k / v pairs?#2017-01-1922:09Alex Miller (Clojure team)yes, isnā€™t that what youā€™re specing?#2017-01-1922:10Alex Miller (Clojure team)oh, I see you have a :ratio-double with multiple vals#2017-01-1922:10joshjonesno, in this case i was showing a spec that would match all of the following:
(myfunc ā€œrequired stringā€)
(myfunc ā€œrequired stringā€ 4/3)
(myfunc ā€œrequired stringā€ 4/3 33.2 99.9)
#2017-01-1922:11Alex Miller (Clojure team)Iā€™m going to go out on a limb and say thatā€™s weird :)#2017-01-1922:11Alex Miller (Clojure team)and you shouldnā€™t write a function signature that way in the first place :)#2017-01-1922:11joshjonesyes it is ā€” my first example showed what you said ā€” i was just showing how the optional arguments portion of a function need not be a homogeneous group of items .. not endorsing it, just demonstrating the versatility šŸ™‚#2017-01-1922:12Alex Miller (Clojure team)ok#2017-01-1922:12joshjonessaying ā€œhey, if it can do this, it can definitely do something much simplerā€ šŸ‘:skin-tone-2:#2017-01-1922:32gdeer81is there a way to do multiple specs at once like you have a bunch of things that should be a string so instead of doing (s/def ::foo string?) (s/def ::bar string?) you can just say [::foo ::bar :baz ::fname ::lname ::email] string?#2017-01-1922:54bbloomhas somebody done a richer explain function yet? i find that when you have some ors or alts w/ larger maps involved, the output becomes unwieldy#2017-01-1923:01Alex Miller (Clojure team)@gdeer81: no#2017-01-1923:02Alex Miller (Clojure team)@bbloom Drogalis started porting some of his onyx stuff to work over spec explain-data (and using fipp) - really more of a poc stage#2017-01-1923:02bbloomcool#2017-01-1923:02bbloomšŸ™‚#2017-01-1923:02Alex Miller (Clojure team)I actually picked that up and hacked on it a bit but found that explain-printers are missing some crucial data#2017-01-1923:02Alex Miller (Clojure team)namely, the original root value and root spec#2017-01-1923:02Alex Miller (Clojure team)thatā€™s missing from the explain-data map#2017-01-1923:03Alex Miller (Clojure team)I have a pending enhancement + patch to add that#2017-01-1923:03bbloomah yes, those things and the immediate parent or some chain of anscestors should be passed down#2017-01-1923:03bbloomthis way explainers can be context sensitive#2017-01-1923:03Alex Miller (Clojure team)everything else is discoverable within the explain-data path#2017-01-1923:03bbloomah right, can just pop on the path#2017-01-1923:03bbloomthat makes sense#2017-01-1923:03Alex Miller (Clojure team)but you donā€™t have the root value#2017-01-1923:03bbloom(get-in root (pop path))#2017-01-1923:04bbloomgotcha. well looking forward to that šŸ™‚#2017-01-1923:04bbloomthanks#2017-01-1923:04Alex Miller (Clojure team)https://github.com/onyx-platform/empathy is Mikeā€™s stuff#2017-01-1923:04bbloomlol @michaeldrogalis love the name#2017-01-1923:04Alex Miller (Clojure team):)#2017-01-1923:04bbloomthis brings up one other issue that iā€™m running in to now#2017-01-1923:04michaeldrogalisHeh. Never did get around to finishing that.#2017-01-1923:04bbloomin fact, itā€™s one that i ran in to in the past#2017-01-1923:04Alex Miller (Clojure team)Iā€™ve done some more work on it#2017-01-1923:05Alex Miller (Clojure team)but not in a public repo#2017-01-1923:05bbloomhttps://github.com/plumatic/schema/pull/134#2017-01-1923:05Alex Miller (Clojure team)Iā€™ve shelved it for the moment until the patch above gets through as then it will be easier to demo#2017-01-1923:05bbloomiā€™m running in to a situation now where i have a recursive spec that is giving me useless explain output b/c itā€™s producing LOTS of output#2017-01-1923:06bbloomb/c the recursive spec fails to match#2017-01-1923:06bbloomone thing you could do with explain-data is find the CLOSEST match#2017-01-1923:06michaeldrogalis@alexmiller Curious as to how you attempted it. Took me ages to do it with Schema, not really looking forward to trying it again with Spec.#2017-01-1923:06Alex Miller (Clojure team)I seem to recall Colin made something similar in the macro grammar stuff he did#2017-01-1923:07Alex Miller (Clojure team)^^ @bbloom#2017-01-1923:07bbloomyeah, itā€™s kinda critical i think#2017-01-1923:07bbloomotherwise spec falls down on ASTs and other recursive structures after data gets too big#2017-01-1923:07Alex Miller (Clojure team)@michaeldrogalis well, I mostly brought it up to more recent spec and extended it just slightly further#2017-01-1923:07bbloomat least explain does#2017-01-1923:08michaeldrogalisAh, got it.#2017-01-1923:08bbloomunfortunately the literature on error detection and recovery in parsers seems to be kinda ad-hoc#2017-01-1923:08Alex Miller (Clojure team)@bbloom I also have a patch that sorts the longest errors first, which generally is a good strategy#2017-01-1923:08bbloomyeah, thatā€™s a good start#2017-01-1923:08Alex Miller (Clojure team)except with recursion when you probably want the reverse :)#2017-01-1923:08bbloomheh#2017-01-1923:09bbloomis that longest path to an error?#2017-01-1923:09Alex Miller (Clojure team)yeah#2017-01-1923:09bbloomor just BIGGEST error?#2017-01-1923:09bbloomok#2017-01-1923:09bbloomi asked about this on twitter a while ago in the context of text parsing#2017-01-1923:10bbloomhttps://twitter.com/jordwalke/status/805921251004821504#2017-01-1923:10bbloomreference manual is here: http://gallium.inria.fr/~fpottier/menhir/manual.pdf#2017-01-1923:10bbloomchapter 10 is about error handling, which may be worth studying#2017-01-1923:10bbloomthere seems to be some attempt at a sensible system for producing good error messages#2017-01-1923:11Alex Miller (Clojure team)I think cleaning up fanout is another big area for improvement #2017-01-1923:11Alex Miller (Clojure team)For removing repetition#2017-01-1923:11bbloomyou mean like representing the output as a tree?#2017-01-1923:12Alex Miller (Clojure team)Nah, just reporting all errors at common path with single context#2017-01-1923:12bbloomah ok#2017-01-1923:12Alex Miller (Clojure team)An ide might do a tree though#2017-01-1923:12bbloomanother idea is a sort of two column view#2017-01-1923:12bbloomie pretty print (or just indent) on the left#2017-01-1923:12bbloomand messages on the right#2017-01-1923:13Alex Miller (Clojure team)I suspect recursion will just suck regardless :)#2017-01-1923:13bbloomyeah, well recursion always sucks when it comes to printing stuff šŸ™‚#2017-01-1923:13bbloomas a man who does a lot of recursion and a lot of printing, believe me, i know#2017-01-1923:13bbloomheh#2017-01-1923:13bbloomwhatever happened to that data structure visualizer? is that still built in? lol#2017-01-1923:13Alex Miller (Clojure team)it is :)#2017-01-1923:14Alex Miller (Clojure team)it still doesnā€™t work very well :)#2017-01-1923:14bbloomyeah, thatā€™s the thing about text based UIs#2017-01-1923:14bbloomwhen they are bad, they are kinda bad#2017-01-1923:14bbloomvs guis#2017-01-1923:14bbloomwhen they are bad, THEY ARE VERY BAD#2017-01-1923:14Alex Miller (Clojure team)gotta run, later#2017-01-1923:14bbloomcya#2017-01-2001:23bbloomjust curious: why does s/keys check the map? predicate? seems like ILookup is the minimum requirement, or am i missing something?#2017-01-2001:26seancorfieldI believe there's a JIRA issue open for that...#2017-01-2001:27bbloomfound it: http://dev.clojure.org/jira/browse/CLJ-2041 thx#2017-01-2003:10Alex Miller (Clojure team)It has to iterate through the entries so ILookup is not sufficient#2017-01-2003:31bbloomMakes sense. I forgot about how it handles present keys even if not specified as optional.#2017-01-2003:33bbloomOne solution would be to only provide that functionality if iterable or seqable. Not sure if that's a great idea though.#2017-01-2009:26joost-diepenmaatmaybe this is a FAQ, but is there a way to automatically re-run clojure.spec/instrument after defining new functions or redefining already specced ones? Iā€™d be fine if it would work in CIDER only.#2017-01-2009:52souenzzoCan I use it in "production"? Or call s/conform at beginning of function? https://clojure.org/guides/spec#_instrumentation#2017-01-2009:56mpenetinstrumentation is not meant to be used in production#2017-01-2009:56mpenetit's actually spelled out explicitly on that page#2017-01-2009:57mpenetas for conform, yes you can#2017-01-2009:59mpenetIn production what I do is leverage :pre/:post + s/valid? in functions. and you can always turn it off with a compile time toggle. It works quite well#2017-01-2014:49timgilbertHey, I have a kind of polymorphic map where some keys should be present depending on the value of one of the other keys, so it's like either {:x/type :t/foo :foo/val 32} or {:x/type :t/bar :bar/baz "47"}#2017-01-2014:49timgilbertIt seems like this is difficult to describe with spec since the map specs only look at keys, not values. Am I missing something?#2017-01-2014:50potetm@timgilbert https://clojure.org/guides/spec#_multi_spec#2017-01-2014:51potetmIf I understand right, that's what you're looking for.#2017-01-2014:51timgilbertAha! Exactly what I was looking for. Thanks @potetm.#2017-01-2014:52potetmnp šŸ™‚#2017-01-2021:14viestiPosed a question in the general Clojure channel, but realised that I need to think about predicates that reference their environment a little more. This was the snippet that I was pondering:
user> (let [limit 100] (s/form (s/spec #(< % limit))))
(clojure.core/fn [%] (clojure.core/< % limit))
#2017-01-2021:14viestispecifically the limit in there#2017-01-2021:15bbloomsadly, clojureā€™s quotations donā€™t capture environments#2017-01-2021:15bbloomthereā€™s really nothing you can do other than eager substitution#2017-01-2021:16bbloomie a macro#2017-01-2021:16viestiwas using s/form for unit tests on spec that I generate from data#2017-01-2021:17Alex Miller (Clojure team)I just answered in #clojure#2017-01-2021:17viestithinking that the serialization thing is probably still on itā€™s way to clojure.spec#2017-01-2021:17Alex Miller (Clojure team)s/form is there for that purpose (with some known bugs)#2017-01-2021:18viestithanks for the answer @alexmiller and sorry for posting on wrong channel šŸ™‚#2017-01-2021:18tbaldridgeserialization with captured environments can get a bit tricky anyways.#2017-01-2021:19bbloomindeed#2017-01-2021:19bbloomitā€™s among the crazy ideas Iā€™ve been studying for a number of years šŸ˜‰#2017-01-2021:20viestišŸ˜„#2017-01-2021:45viestiwas actually using s/form for unit testing specs that I generate from sql ddl statements, just toying around for now#2017-01-2208:32andrewzhurov#2017-01-2208:32andrewzhurov#2017-01-2208:32andrewzhurovfirst#2017-01-2208:33andrewzhurovI more than sure that those questions appear often here, so, some links ?#2017-01-2214:45settingheadfor those interested in seeing another javascript port of clojure.spec (in additon to js.spec), here is one I've made: https://github.com/clausejs/clausejs. Regex specs and fspecs are currently working. Data generation comes next. I've also made a spec documentation generator called clausejs-docgen (still WIP), which I have been using to generate Clause.js's own documentation. Lastly, there is a syntax function which takes in an fspec, enumerates over all of its arguments' possibilities, and outputs a concise list of lambda-ish syntax references (you can find examples in the doc site here: https://clause.js.org)#2017-01-2214:46Yehonathan Sharvitmetalclj#2017-01-2223:24mrgDoes anyone have any examples on how clojure.spec/conformer works? I am trying to coerce a json map (including UUIDs and instants, and seqs of keywords) and I cannot find any good examples anywhere#2017-01-2223:46Oliver GeorgeI don't think it's really intended to use conform in that way#2017-01-2223:49Oliver Georgeas best I understand it, spec is intended to describe static data rather than the coerce data#2017-01-2223:50Oliver GeorgeTechnically that looks a bit like coercion in that the conformed data is a different shape. #2017-01-2223:50Oliver GeorgeHope that helps. #2017-01-2223:55seancorfield@mrg I answered in #clojure but I'll respond to @olivergeorge's comment: yes, you generally want to be very wary of specs that coerce data because then every "client" of that spec has to accept the coercion. So it's only good for a spec where you always want all "clients" to see the same coercion too.#2017-01-2300:12mrgThanks @olivergeorge and @seancorfield . I was basing that off a comment of Timothy Baldridge on reddit that was suggesting that conformer was the way to go if one wanted to use a spec to coerce data#2017-01-2300:12mrgWhat would be the right way to do something like that then though?#2017-01-2300:13mrgSay I have a spec (s/def ::myspec #{:foo :bar})#2017-01-2300:13mrgand then i have a deeply nested json map that at multiple places uses ::myspec#2017-01-2300:14seancorfieldWell, yes, s/conformer is the way to go -- but it's a "concern" to have a spec do data transformation.#2017-01-2300:14mrgsorry, there would also be (s/def ::myspecs (s/coll-of ::myspec :min-elements 1 :max-elements 5)) or something like that#2017-01-2300:14mrgI don't really like it either#2017-01-2300:15mrgBut I can't think of a better way that doesn't have me re-implement half of spec for every branch of the tree#2017-01-2300:16seancorfieldI'm not sure what you're asking -- and I think you're misunderstanding what we're saying...?#2017-01-2300:16mrgQuite likely#2017-01-2300:17mrgMy actual use case is this: I have a service that 1) generates random data that conforms to a spec and spits it out as json 2) is supposed to take such a json map and check its conformance to the spec#2017-01-2300:19mrg1) works mostly, using generators and gen/generate#2017-01-2300:19mrgMy problem is in step #2 when I try to read the generated json back in#2017-01-2300:20mrgthe uuids have become strings, the enums (like the ::myspec) above as well, and so have the #inst datetimes#2017-01-2300:20mrgAnd I can't figure out how to coerce them back into the format that my spec expects#2017-01-2300:32seancorfieldHere's an example of some specs etc we use that accept strings and coerce them to dates:
(defn coerce->date
  "Given a string or date, produce a date, or throw an exception.
  Low level utility used by spec predicates to accept either a
  date or a string that can be converted to a date."
  [s]
  (if (instance? java.util.Date s)
    s
    (-> (tf/formatter t/utc "yyyy/MM/dd" "MM/dd/yyyy"
                      "EEE MMM dd HH:mm:ss zzz yyyy")
        (tf/parse s)
        (tc/to-date))))

(defn ->date
  "Spec predicate: conform to Date else invalid."
  [s]
  (try (coerce->date s)
       (catch Exception _ ::s/invalid)))

(defmacro api-spec
  "Given a coercion function and a predicate / spec, produce a
  spec that accepts strings that can be coerced to a value that
  satisfies the predicate / spec, and will also generate strings
  that conform to the given spec."
  [coerce str-or-spec & [spec]]
  (let [[to-str spec] (if spec [str-or-spec spec] [str str-or-spec])]
    `(s/with-gen (s/and (s/conformer ~coerce) ~spec)
       (fn [] (g/fmap ~to-str (s/gen ~spec))))))

(s/def ::dateofbirth (api-spec ->date #(dt/format-date % "MM/dd/yyyy") inst?))
#2017-01-2300:33seancorfieldThis accepts an instant or a string that will coerce to an instant. It auto-generates as well based on inst? and converts it to a MM/dd/yyyy string.#2017-01-2300:34mrghey, thank you @seancorfield , that is super useful!#2017-01-2300:34mrgAnd I appreciate you helping me out on a Sunday evening šŸ™‚#2017-01-2313:31rickmoynihan@brownmoose3q: Interestingā€¦ Iā€™ve asked similar questions before, but never had any responsesā€¦ Iā€™m not sure if itā€™s because there isnā€™t an answer for it yetā€¦ Iā€™d certainly like to know how to integrate clojure.spec with a clojure.test runner.#2017-01-2313:32rickmoynihannot sure what others doā€¦ but might be nice to have some clojure.test wrappers for tools like exercise etc...#2017-01-2313:33rickmoynihanobviously you can also use instrument#2017-01-2313:58andrewzhurovI just find out that lein test is giving that wierdo error and refind-out for myself this issue: https://github.com/technomancy/leiningen/issues/2173 turned off monkeydispatch and whoop - all is good.#2017-01-2313:59andrewzhurovSo... seems to be a way of testing šŸ™‚#2017-01-2317:20lsnapeCan anyone suggest how to simplify the following:
(s/def ::created inst?)
(s/def ::updated inst?)
(s/def ::foo* (s/keys :req-un [::created ::updated]))

(s/def ::foo
  (s/with-gen ::foo*
    #(gen/such-that (fn [{:keys [created updated]}]
                      (>= (.getTime updated) (.getTime created)))
                    (s/gen ::foo*))))

(s/exercise ::foo)
I would like to ditch ::foo*, but if I reference the ::foo generator from within s/def ::foo, then understandably I get a SOE
#2017-01-2320:22souenzzoThere is option in s/keys that fails/remove on unwanted keys?? (s/valid? (s/keys :req [:a/b]) {:a/b 0 :c/d 1}) ;=> false or something like (s/"filter????" (s/keys :req [:a/b]) {:a/b 0 :c/d 1}) ;=> {:a/b 0} #2017-01-2320:32seancorfield@souenzzo spec is designed to be open for extension ā€” read the guide for more details.#2017-01-2320:33seancorfieldIf you want a spec to fail for unwanted keys, you need to explicitly s/and a predicate that checks the set of keys in the map.#2017-01-2320:33seancorfield(but you probably shouldnā€™t do that)#2017-01-2404:32bastiHi, Iā€™m pretty new to Clojure and I am playing around with Spec. Iā€™ve got following error message
[{:type java.io.FileNotFoundException,
                                                      :message "Could not locate clojure/test/check/generators__init.class or clojure/test/check/generators.clj on classpath.",
while evaluating
(ns foo-test
  (:require [clojure.spec :as s]
            [clojure.spec.test :as stest]
            [clojure.spec.gen]
            [clojure.pprint :as pprint]
            [clojure.test :as t]))

;; Sample function and a function spec
(defn fooo [i] (- i 20))

(s/fdef fooo
        :args (s/cat :i integer?)
        :ret integer?
        :fn #(> (:ret %) (-> % :args :i)))


(stest/check `fooo)
Whatā€™s going on here? ^^
#2017-01-2404:33Alex Miller (Clojure team)generators use the test.check library. this is optional so itā€™s not a required compile-time dependency#2017-01-2404:34Alex Miller (Clojure team)but if you want to use it at test time, youā€™ll need to include test.check as a test dependency#2017-01-2404:36Alex Miller (Clojure team)[org.clojure/test.check ā€œ0.9.0ā€]#2017-01-2404:37Alex Miller (Clojure team)youā€™d want that in the :dev profile for it to be test-scoped only (not a dependency that downstream consumers would get)#2017-01-2404:39bastiokay I c, sweet now it works, thank you for explaining it to me so clearly @alexmiller - really appreciated šŸ™‚#2017-01-2404:42Alex Miller (Clojure team)np - this is in the spec guide too - https://clojure.org/guides/spec#2017-01-2404:42Alex Miller (Clojure team)if you havenā€™t seen that yet#2017-01-2404:46bastino I havenā€™t, thanks for pointing me to it!#2017-01-2409:26lsnapeHi, I am having trouble writing a spec that has the following constraints:
{:id 123
 :foo [{:id 456 :parent-id 123}
       {:id 789 :parent-id 123}]}
See that :id and :parent-id are required to have the same value.
#2017-01-2409:34lsnapeAha, a simple predicate s/anded with s/keys would do nicely :duck:
(s/and (s/keys :req-un [::id ::foo])
       (fn [{:keys [id foo]}]
         (every? #{id} (map :parent-id foo))))
#2017-01-2413:29luxbockI'm specced a function which performs a deep-merge via addition on some maps with longs at the leafs, and I wanted to create a generative test for it#2017-01-2413:30luxbockhowever my generative test actually fails because I run into integer overflow, which is fair enough, that's good to know in theory#2017-01-2413:30luxbockbut in practice none of the values I use are ever going to get that large as to cause an overflow#2017-01-2413:31luxbockso in this case, should I be handling the bounding of these values via instrument by using :gen overrides?#2017-01-2413:32luxbockor should I handle it at the site of the spec definition by adding some arbitrary bound as a predicate to go with pos-int?#2017-01-2414:44luxbockI tried working around this issue by using stest/instrument and overriding the gen that is causing the overflow via :gen, and then running stest/check on the function again but this did not seem to have any effect#2017-01-2414:45donaldballMaybe you want something like a ::smallish-long that describes a domain consistent with your expected values in production?#2017-01-2414:49luxbock@donaldball yeah I tried that approach as well, but it has the following problem: the function I'm testing is essentially taking a collection of values of ::map-with-smallish-int and summing all of the ::smallish-int together, so I specced the function to use the same return value#2017-01-2414:50luxbockand this of course fails because the largest values of what ::smallish-int generates summed together are going to be larger than the bound I'm using#2017-01-2414:50luxbockso of course I could use use a new spec for the return value of the function under test, which uses a higher bound#2017-01-2414:50luxbockbut that feels less than ideal#2017-01-2414:51donaldballAnnoying, albeit correct šŸ˜›#2017-01-2414:51donaldballYouā€¦ could merge with +ā€™#2017-01-2414:52luxbockyeah that's another option but then I'm changing the definition of my function to use a slower function just to appease my tests#2017-01-2414:52luxbockbut I'm guessing that I don't understand how instrument with :gen overrides is supposed to be used#2017-01-2414:54luxbockit would be nice if during tests I could just override the generators of my specs to match to the domain values I'm expecting during normal use, and then I write the test to perform inside this context#2017-01-2415:18luxbockusing +' doesn't really help either as once the values get high enough to coerce to BigInt they will fail nat-int?#2017-01-2415:53Alex Miller (Clojure team)@luxbock the approach of using instrument with a :gen override is what I would recommend and should work#2017-01-2415:53Alex Miller (Clojure team)if you can a repro, would be interested#2017-01-2416:34luposlipIs there any chance that you guys could make the clojure.spec guide (PDF) printable? šŸ™‚ https://clojure.org/guides/spec#2017-01-2416:45qqqDoes Chrome -> Print -> Save as PDF work? Or do you need something else?#2017-01-2416:49luxbock@alexmiller this is what I'm doing, is this supposed to work? https://gist.github.com/luxbock/ecb263750f61cc6fab7a736e5bd96008#2017-01-2417:11luposlip@qqq The result is just a lot of empty pages (I'm using Chromium BTW)#2017-01-2417:14qqq@luposlip: my bad, I also get only empty pages in chrome / safari#2017-01-2417:14qqqtime to edit some HTML#2017-01-2417:15qqqhmm, I get something printable#2017-01-2417:15qqqbut it loses all the formatting#2017-01-2417:16luposlipSounds just great @qqq, because printable pages would make http://clojure.org so much more useful (for users that want to read the guides etc. on ereaders etc.)#2017-01-2417:16qqqI don't know how to fix this, sorry -- it seems like the CSS I delete to make it printable also deletes the nice CSS formatting that makes it pretty and has nice code layout.#2017-01-2417:17luposlipWould be really nice with a epub export button for the long pages šŸ™‚#2017-01-2417:17qqqis the main clojure site on github ?#2017-01-2417:17qqqI wonder if you can just print the github rendering of the page#2017-01-2417:17luposlipI don't think so [UPDATE: I stand corrected]#2017-01-2417:17qqqhttps://github.com/clojure/clojure-site#2017-01-2417:17qqqhttps://github.com/clojure/clojure-site/blob/master/content/guides/spec.adoc#2017-01-2417:18qqqcan yo print that page?#2017-01-2417:18qqqhttps://github.com/clojure/clojure-site/blob/master/content/guides/spec.adoc <-- is printable for me, think this is probably the best for now#2017-01-2417:18luposlipgreat idea, didn't know it was on github - thanks for the input @qqq!!#2017-01-2422:35mac@luposlip Also Firefox does a decent job with the page if you print to pdf.#2017-01-2422:50Alex Miller (Clojure team)there is an issue for adding print support at https://github.com/clojure/clojure-site/issues/99#2017-01-2422:52Alex Miller (Clojure team)Making that work is probably not something Iā€™m going to work on, but if someone else wanted to help, that would be cool#2017-01-2503:31bbloom@alexmiller i think you could get 90% of the way there with a smaller change than a custom stylesheet - Safari etcā€™s reader mode is broken on the current site b/c div#preamble doesnā€™t match div.sect1#2017-01-2503:32bbloomi think that if you just make all the seconds have the same class, then reader modes will treat it all as body text and thatā€™s a de facto print mode#2017-01-2503:32Alex Miller (Clojure team)most of that did not actually mean anything to me :)#2017-01-2503:33bbloomeach page in the reference has multiple divs, but only the first div shows up if you use an article reading filter#2017-01-2503:33Alex Miller (Clojure team)Iā€™m like a monkey banging rocks together in web programming#2017-01-2503:33bbloomlol#2017-01-2503:33bbloomyou on a mac? go to any reference page in safari and youā€™ll see a little paragraph icon in the top left corner of the url bar#2017-01-2503:34bbloomthereā€™s various plugins for all the other browsers that do that too#2017-01-2503:34bbloombut since the ā€œpreambleā€ in the generated pages have different structure and style classes applied, only that first section gets treated as body text#2017-01-2503:35Alex Miller (Clojure team)so just some tweaks might fix that#2017-01-2503:36bbloomyup, and then people can print that view just fine#2017-01-2503:36Alex Miller (Clojure team)ok, if I get a chance Iā€™ll see if I can muggle my way through it#2017-01-2503:36bblooman aria hint might do the trick super quick too#2017-01-2503:36Alex Miller (Clojure team)is that a font?#2017-01-2503:37bbloomšŸ™‚ ARIA = accessible rich internet applications#2017-01-2503:37bbloomitā€™s about accessibility#2017-01-2503:37bbloomscreen readers etc#2017-01-2503:37Alex Miller (Clojure team)šŸ’#2017-01-2503:38Alex Miller (Clojure team)sadly, no rock emoji#2017-01-2503:38bbloomthe simplest thing to try:#2017-01-2503:38bbloomwrap the entire content of the page in a <article> tag#2017-01-2503:39bbloomspecifically, change the div with class clj-content-container from div to article - and see if that does it#2017-01-2503:39bbloomi bet that it does, no impact to styles at all#2017-01-2503:40Alex Miller (Clojure team)cool#2017-01-2503:41Alex Miller (Clojure team)thanks#2017-01-2503:41bbloomno problem. hopefully that helps šŸ™‚#2017-01-2503:41bbloomi abuse reader mode and use huge fonts for most stuff i read šŸ™‚ i have good eye sight and hope to keep it that way!#2017-01-2503:46Alex Miller (Clojure team)I match my font size to my age#2017-01-2503:47bbloomheh, i like it#2017-01-2504:34luxbock@alexmiller could you take a look at this and confirm if I'm using instrument correctly or if there might actually be an underlying issue with :gen https://gist.github.com/luxbock/ecb263750f61cc6fab7a736e5bd96008#2017-01-2504:38Alex Miller (Clojure team)I will take a look in the morning#2017-01-2505:59nickMaybe this is silly but say I have a spec like this (s/def ::dog (s/keys ::req [::name ::type ::sound])) but I also want to include a key that's always something specific like :feet 4 - so if I generate that I'll get like {:name "red" :type "golden retriever" :sound "woofwoof" :feet 4}#2017-01-2506:01nickI want to do something like this:
(defmulti dog-type :dog/type)
(defmethod dog-type :golden-retriever [_]
  (s/keys :req [::name ::sound]))
But also have in the keys a type key that is dog-type and the actual key.
#2017-01-2506:56tengIs it possible to use spec to validate maps that has keys that are not namespaced, like :name instead of :user/name?#2017-01-2507:54seancorfield@teng Yes, with :req-un and :opt-un. Although they require namespaced keys for the specs, the actual map keys are unqualified.#2017-01-2507:55seancorfield(see the guide for more details)#2017-01-2507:55tengok, thx.#2017-01-2507:55seancorfield@nick Do you mean you'd want (s/def ::feet #{4}) as the spec for the :feet key?#2017-01-2512:04odinodin
(clojure.spec/explain string? 123)
val: 123 fails predicate: :clojure.spec/unknown
How can I get the predicate to show string?
#2017-01-2512:12dominicm@odinodin I think you need to do (s/def ::string? string) and then you can (s/explain ::string? 123)
#2017-01-2512:12odinodin@dominicm thanks, makes sense šŸ™‚#2017-01-2513:20Alex Miller (Clojure team)This unknown is a bug that has a pending patch btw #2017-01-2513:23Alex Miller (Clojure team)http://dev.clojure.org/jira/browse/CLJ-2068#2017-01-2513:23Alex Miller (Clojure team)So that will work eventually#2017-01-2513:41odinodin@alexmiller that is awesome šŸ™‚#2017-01-2515:13Alex Miller (Clojure team)@luxbock I agree that Iā€™d expect something like this to work. I did just notice this line in the instrument docs though: ":gen overrides are used only for :stub generation.ā€ which honestly surprises me. it seems like they should be used also for instrumented :args gen too.#2017-01-2515:14Alex Miller (Clojure team)although maybe you should really be passing those :gen overrides in your call to check#2017-01-2515:24Alex Miller (Clojure team)yeah, you should move your gen overrides map from instrument into the check options map. also, in the :gen map, the values should be no-arg functions that return the generator, not the actual generator, so just prefix all of them with # to turn them into anonymous no-arg functions#2017-01-2515:28Alex Miller (Clojure team)once I did that, it was running, but I started getting OOMEs#2017-01-2515:29Alex Miller (Clojure team)to address that, I narrowed the collection generators which default to max size 20. I added :gen-max 3 to both the fdef :args spec and the ::operations override gen.#2017-01-2515:29luxbock@alexmiller ah yeah I think I tried :gen with check but I didn't realize that I need to wrap them in thunks#2017-01-2515:29Alex Miller (Clojure team)Then it worked#2017-01-2515:29luxbockthanks, I will play around with it a bit more later today#2017-01-2515:30Alex Miller (Clojure team)these days I generally preemptively add :gen-max 3 to all collection specs reachable by fdef :args or used in recursion#2017-01-2515:30Alex Miller (Clojure team)I think it would be a good idea to change the default there from 20 to 3#2017-01-2515:32luxbockyeah that's good to know as well#2017-01-2515:36Alex Miller (Clojure team)http://dev.clojure.org/jira/browse/CLJ-2102#2017-01-2515:36Alex Miller (Clojure team)if youā€™d like to vote :)#2017-01-2515:36luxbockit feels that all the spec related testing functionality one needs to write these types of tests is quite spread out, both in terms of namespaces and where the different options for customizing the generators go#2017-01-2515:37luxbocknothing that a good utility library won't fix, but I have a hard time keeping in my head where everything is supposed to come from and how it ties together#2017-01-2515:49Alex Miller (Clojure team)well the stuff in spec vs spec.test is intentionally split to emphasize the parts that are designed specifically for testing. and spec.gen is the same - split out the part that dynamically loads and depends on test.check generators.#2017-01-2515:49Alex Miller (Clojure team)those are at this point pretty well factored in my mind#2017-01-2515:50Alex Miller (Clojure team)the gen overrides stuff is a bit messier and not consistent enough between s/exercise, stest/instrument, stest/check etc and we may still make some changes in those#2017-01-2516:02luxbockI'm writing code which I can't fully test in use until it hits the production server so I'm trying to cover as many edge cases as possible with spec and generative tests to avoid the long feedback loop of deploy + test, and it's definitely giving me a lot more confidence in that I got rid of most of the silly bugs before trying it in actual use#2017-01-2516:05Alex Miller (Clojure team)cool, thatā€™s good to hear#2017-01-2520:52agquick noob question how do I limit the size of a number in spec like: (s/and int? pos?)#2017-01-2521:12qqq@ag: you can pass in arbitrary function#2017-01-2521:12qqqif you also need to generate, you need to write you rown generator#2017-01-2521:37Alex Miller (Clojure team)@ag (s/and int? pos-int? #(< % 100)#2017-01-2521:37Alex Miller (Clojure team)but once you start going there, consider integer ranges#2017-01-2521:37Alex Miller (Clojure team)(s/int-in 1 100)#2017-01-2521:38Alex Miller (Clojure team)the range specs come with a better generator by default#2017-01-2521:41agah, rightā€¦ thanks @alexmiller#2017-01-2523:03aguhā€¦ so how do I make a spec for a key like this
:options {:on-click `~(fn [r] ,,,,)}
it contains an fn but in quoted form?
#2017-01-2523:16agI guess I need s/fdef, how do I make a spec for something like {:on-click (fn [e],,,} - function that takes one argument of anything and returns whatever?#2017-01-2600:00agoh, another total noob question.. I need a spec that generates vector of 1 to 5 elements, each element is of another spec (map), something like this:
(s/def ::col (s/keys :req-un [::id ::title]))

(gen/generate (s/gen (s/+ ::col)))
but it has to be 1) a vector 2) of 1 to 5 elements
#2017-01-2600:02agoh, nevermind I think I got itā€¦#2017-01-2600:02agdammit specs and test.check makes me feel stupid#2017-01-2600:05agmeh, it seems I only got the generator part#2017-01-2600:06ag(s/with-gen vector? #(gen/vector (s/gen ::col) 2 5)) ainā€™t good, I need spec to conform to vector of ::col#2017-01-2601:42joshjones@ag this?
(s/def ::id (s/int-in 1 100))
(s/def ::title string?)
(s/def ::col (s/keys :req-un [::id ::title]))
(s/def ::cols (s/coll-of ::col :kind vector?
                               :min-count 1
                               :max-count 5))
(gen/generate (s/gen ::cols))
#2017-01-2602:02Alex Miller (Clojure team)@ag for your prior question, see s/fspec#2017-01-2616:51mrcncis it preferable to use s/assert over s/valid? so you can disable it at runtime?#2017-01-2616:53mrcncthey seem pretty similar#2017-01-2617:17seancorfieldTo me they have very different uses: we use spec for input validation (and some coercion) so we have error handling that says (if (s/valid? ::spec value) (do-good-stuff) (handle-error)) ā€” thatā€™s not the same as (s/assert ā€¦)#2017-01-2617:18seancorfieldI personally donā€™t much like asserts ā€” Iā€™ve never liked them in any language ā€” and part of that is because folks tend to turn them off in production, which is when itā€™s even more critical that your code doesnā€™t attempt to operate on invalid data. Seems backwards to me...#2017-01-2617:44mrcncgood points @seancorfieldā€¦thx!#2017-01-2618:48spiedensame ā€” usually check valid? then throw an ex-info with explain-data#2017-01-2620:37moxajspecs cannot be created at runtime, right? since the api is mostly macros#2017-01-2620:58schmeeI think you can get it done with quoting and eval#2017-01-2621:01moxajwell, maybe as a last resort .. eval is evil#2017-01-2621:03bfabryspecs being macros is what gives them reasonable reporting on failures, a spec created at runtime would have to either do away with a lot of the error description, or have a very clunky api. there was some talk of some non-macro versions of some of the spec api maybe in the future though#2017-01-2621:06spiedenseems to guide the intended usage as something static, too#2017-01-2621:06spieden.. in a world of dynamism =)#2017-01-2621:15tbaldridge@moxaj why is eval evil?#2017-01-2621:18moxaj@tbaldridge well, most of the time when you use it, there's a better solution (there are exceptions of course)#2017-01-2621:19tbaldridgeThat doesn't make it evil though šŸ™‚#2017-01-2621:19moxaji'm pretty sure there's a workaround for my issue which does not include eval#2017-01-2621:19moxajwell yeah, it's perfectly fine, but sometimes abused#2017-01-2621:20tbaldridgeI say this because the "eval is evil" trope is mis-applied in Lisps. Eval is evil in a language like Javascript (where the phrase comes from) where eval means concat-ing strings together. In Clojure eval is akin to something like stored procs in SQL. It makes code injection really hard#2017-01-2621:21tbaldridgeSo all that being said, macros + eval works pretty well for dynamic specs, and done correctly you could even get pretty good source code mappings#2017-01-2621:26moxajalso, i'm targeting cljc, and eval only works in bootstrapped cljs afaik#2017-01-2621:27tbaldridgefair enough#2017-01-2703:58weicould someone point me to some real world examples of conform? I have a hunch that it would be very useful for me, especially with specs like s/or, but I havenā€™t found the output to be especially easy to adapt, for example in a cond statement.#2017-01-2704:08weihereā€™s an example that I think is overly verbose: https://gist.github.com/yayitswei/fc60da9271452716ae2a197c945a4b3e, would appreciate any tips on trimming this down#2017-01-2707:30luxbockis there some existing function that gives me the same value that the predicates/specs in :fn receive?#2017-01-2707:30luxbockI am assuming no, but it's probalby not too difficult to create#2017-01-2708:28luxbockthis seems to work#2017-01-2712:35nblumoeHey, what are uses for the :opts key spec on maps? As additional keys are allowed anyways and all present keys are being validated even if they are not in the explicit spec there does not seem to be any relevance for validation. Is it for generation? Something else Iā€™m missing?#2017-01-2713:54schmeenblumoe AFAIK itā€™s for generation and as documentation#2017-01-2713:57nblumoeThe guide says this: ā€œ When conformance is checked on a map, it does two things - checking that the required attributes are included, and checking that every registered key has a conforming value. Weā€™ll see later where optional attributes can be useful. Also note that ALL attributes are checked via keys, not just those listed in the :req and :opt keys. Thus a bare (s/keys) is valid and will check all attributes of a map without checking which keys are required or optional.ā€ I do not know what ā€œWeā€™ll see later where optional attributes can be usefulā€ is referring to#2017-01-2713:58nblumoehttps://clojure.org/guides/spec#_entity_maps#2017-01-2714:45Alex Miller (Clojure team)They are used in generators (as well as serving as doc)#2017-01-2714:51Alex Miller (Clojure team)@luxbock the :fn spec receives a map {:args ā€¦, :ret ā€¦} where the values are the conformed values of the args and ret specs. you donā€™t need that get-fspec-attr - you can call (s/get-spec sym) to get the fspec and that supports ILookup, so you can (:args (s/get-spec sym)) to get (for example) the args spec.#2017-01-2714:52luxbock@alexmiller: thanks, yeah figured there might have been a better way#2017-01-2800:12adambros@alexmiller getting weird behavior when trying to conform this:
(s/def ::thing (s/cat
                 :a (s/? string?)
                 :b (s/+ number?)))
(s/def ::seq-of (s/+ ::thing))

(s/conform ::seq-of '(ā€fooā€ 1 ā€œbarā€ 2 3 ā€œquxā€ 4))
;=> [{:a "foo", :b [1]} [{:a "bar", :b [2 3]} {:a "qux", :b [4]}]]

;expected
;=> [{:a "foo", :b [1]} {:a "bar", :b [2 3]} {:a "qux", :b [4]}]

;; ONLY 2nd thing matters?
(s/conform ::seq-of '(ā€fooā€ 1 2 ā€œ bar 3))
;=> [{:a "foo", :b [1 2]} {:a "bar", :b [2]}]

;; NO OPTIONAL
(s/def ::thing (s/cat
                 :a string?
                 :b (s/+ number?)))
(s/def ::seq-of (s/+ ::thing))

(s/conform ::seq-of '(ā€fooā€ 1 ā€œbarā€ 2 3 ā€œquxā€ 4))
;=> [{:a "foo", :b [1]} {:a "bar", :b [2 3]} {:a "qux", :b [4]}]
This also only shows up if there are 2+ numbers in the 2nd or later ::thing AND the problem goes away if I make :a not optionalā€¦ is this at all related to http://dev.clojure.org/jira/browse/CLJ-2003 ?
#2017-01-2800:33agis it possible to ā€œredefā€ the spec? if I have a spec that depends on other spec and that one uses a function as a predicate and I want to predicate to be something else in the test#2017-01-2800:34hiredman@adambros that looks like what phil brown mentions in the comment on that, but alex says it isn't actually related to that issue and he should open a new issue for it#2017-01-2800:38hiredmana cons somewhere there should be a concat#2017-01-2800:42Alex Miller (Clojure team)@adambros not sure if thatā€™s the same or not, feel free to log#2017-01-2803:05uwoIā€™d be grateful for feedback šŸ™‚ https://groups.google.com/forum/#!topic/clojure/wwJVJKtxps8#2017-01-2809:39nblumoeI ran into some issues with spec.test/check not terminating when using a coll-of spec with :kind:
(s/def ::bar (s/coll-of number? :kind vector?))

(defn foo [a]
  (reduce + 0 a))

(s/fdef foo
  :args (s/cat :a ::bar)
  :ret number?)

(first (stest/check `foo))
The last call does not terminate, just keeps heating my CPU. When removing the :kind vector? constraint from the spec ::bar everything works fine. With :kind list? I also get the erroneous behaviour.
#2017-01-2809:42nblumoeWondering If I am doing something wrong, if this is a bug and if it was filed on Jira already#2017-01-2809:44schmeenblumoe try adding :gen-max 3 to the ::bar definition#2017-01-2809:44schmeeit could be that it is generating some crazy big collections#2017-01-2809:45nblumoethanks, but still the same issue#2017-01-2823:22leongrapenthin@nblumoe If you don't provide :into [] to s/coll-of spec will generate a vector every time just to call empty on it to then generate the stuff into it. Apparently these "generated constructor vectors" explode and you can't control their sizing. So you have to provide :into. @alexmiller Is this intended or ticketworthy?#2017-01-2823:27gfrederickssounds like a ticket to me#2017-01-2823:29Alex Miller (Clojure team)Not sure I understand all that without looking at the code but doesn't sound right#2017-01-2823:51leongrapenthin@alexmiller @nblumoe @gfredericks created http://dev.clojure.org/jira/browse/CLJ-2103#2017-01-2915:48nblumoeGreat, thanks @leongrapenthin #2017-01-3020:32mrgIs there a good way to generate an increasing number as id in a variable-length collection?#2017-01-3020:32mrgi.e. i have this:#2017-01-3020:33mrgnow if i generate subsegments, how would I get the first one to have the ::id 1, the second one (if there is one) the ::id 2, and the third one (if there is one) the ::id 3 ?#2017-01-3020:40Alex Miller (Clojure team)you canā€™t make that happen automatically, but you could build a generator that did that#2017-01-3020:41Alex Miller (Clojure team)I would create a custom generator for ::subsegments using gen/fmap - use the default generator to create, then in the fmap function replace the ::id values to have the index#2017-01-3021:29mrgI'll give that a shot @alexmiller .#2017-01-3021:30mrgHow would you keep the index state? There doesn't seem to be a fmap-with-index, nor a multi-arity version where I could pass in a range or something and zip them together#2017-01-3022:03Alex Miller (Clojure team)Yeah, you can just map over the generated value and range#2017-01-3022:25mrgDo you have an example or a pointer to somewhere that would explain that?#2017-01-3022:25mrgI don't quite grok generators yet#2017-01-3022:45jrIs there a way to spec a mapargs fn? for example:
(defn foo [& {:keys [bar]}] bar)
(foo :bar "value of bar")
#2017-01-3022:50joshjones
(s/def ::bar string?)
(s/def ::baz int?)

(s/def ::mapargs (s/keys* :req-un [::bar ::baz]))

(defn foo [& {:keys [bar baz]}] bar)
(s/fdef foo :args ::mapargs)

(stest/instrument `foo)
(foo :bar "value of bar" :baz 42)
@jr
#2017-01-3022:54jr@joshjones wow that's great. thanks!#2017-01-3022:56jrI didn't realize that keys* is different in that it transforms a sequence of values into a map#2017-01-3023:02joshjonesjust modified the above example to remove cat which was not necessary fyi#2017-01-3023:03jrno worries I got the gist of it. I was specing a macro with the following structure
(defui foo []
  :tracks [name [:path :to :value]]
  [:h1 name])
#2017-01-3023:04jrwhere the options like :tracks are optional#2017-01-3102:00rmuslimovHello, I have newbie question for spec. Letā€™s suppose I have map {"first-name" ā€œnameā€} , can you please show the spec which check key (= ā€first-nameā€) and if the is string. Thanks#2017-01-3102:08rmuslimovThing I cannot understand here, that if I had keyword instead of ā€œfirst-nameā€ everything would be trivial:
(s/def :first-name string?)
(s/valid? (s/keys :req-un [:first-name]) {:first-name ā€œnameā€}) ;=> true
But if ā€œfirst-nameā€ I cannot attach spec to it
#2017-01-3102:45joshjonesif you are doing this just for experimentation -- then
(s/def ::str-str-map #(string? (get % "first-name")))
(s/valid? ::str-str-map {"first-name" "josh"})
but in practice, this only makes life difficult. there's not a way i know of to do this cleanly, as spec is not trying to promote this type of behavior
#2017-01-3103:40rmuslimov@joshjones got it, thanks!#2017-01-3109:09luxbockit's quite difficult to figure out what other options stest/check accepts in the :clojure.spec.test.check/opts key#2017-01-3109:09luxbockthe doc string says: "::stc/opts opts to flow through test.check/quick-check" but it's not really clear which function this refers to#2017-01-3109:12luxbockI can go to the source and see it calls stest/check-1, which calls a private function stest/quick-check which calls clojure.spec.gen/quick-check which calls clojure.test.check/quick-check#2017-01-3109:13luxbockwhich tells me the keys are :seed and :num-tests, so as far as I can tell :seed is the only other option it accepts#2017-01-3109:14luxbockI wish the doc-string for stest/check just told me this right away because I might not have clojure.test.check required and so I need to dig quite deep to find the information#2017-01-3109:15luxbockthe same goes for s/coll-of which tells me it accepts the same options as every, rather than just telling what those options are#2017-01-3111:16thhellerso I have a hard time composing specs due to its macro nature#2017-01-3111:16thhellerie. anything that creates a spec must be a macro#2017-01-3111:17thhellerI can't have a function (validate-a-thing TypeOfThing) that returns a spec#2017-01-3111:18thhellerit must be a macro that generates (s/spec #(instance? TypeOfThing %))#2017-01-3111:18thhelleror am I missing something obvious? using instance? as a basic example my spec is actually a bit more complex than that#2017-01-3111:29thheller(defn validate-a-thing [type] (s/spec #(instance? type %)) works well enough but the error suffers as it is always fails predicate: (instance? type %)#2017-01-3111:31thhellercan't find an obvious way to provide custom explain logic, don't care about the gen part in this case#2017-01-3111:34nblumoeShouldnā€™t my generator be good enough to be used to check the corresponding function against itā€™s spec?
(frequencies (map (partial s/valid? (s/and ::matrix
                              ::non-zero-row-sums)) (gen/sample (matrix-gen) 100)))
;; => {true 27, false 73}
I end up with ExceptionInfo Couldn't satisfy such-that predicate after 100 tries. clojure.core/ex-info (core.clj:4725) again and again.
#2017-01-3115:00dergutemoritz@thheller Yeah you are right, validate-a-thing needs to be a macro to achieve what you want#2017-01-3115:00dergutemoritz@thheller I don't think you're missing anything#2017-01-3115:03dergutemoritz@thheller See this paragraph for the rationale: https://clojure.org/about/spec#_code_is_data_not_vice_versa#2017-01-3119:13uwoIā€™d really appreciate some feedback on this: https://groups.google.com/forum/#!topic/clojure/wwJVJKtxps8#2017-01-3120:05joshjonesthis exact situation has come up several times here @uwo ā€” unfortunately, thereā€™s not much you can do other than what youā€™ve already identified, at least, none that Iā€™m aware of. The reality is that you do not have one set of billed-charges; you have two, which means you will need to spec two, whether thatā€™s through a multi-spec or explicit naming#2017-01-3120:07joshjonesdo you have more levels of nesting than what youā€™ve shown?#2017-01-3121:39uwo@joshjones thanks, and yes, many many levels of nesting#2017-02-0107:32Yehonathan SharvitWhat are the conceptual and practical differences between s/or and s/alt?#2017-02-0107:40mike_ananevhi! if i need to transfer spec via network, how to get "source" of spec by its name?#2017-02-0107:40Yehonathan Sharvitsomething like s/describe?#2017-02-0107:42mike_ananev@viebel yeah, but any anonymous fn in spec is not readable. i need to get source of spec, like I see in IDE#2017-02-0108:29dergutemoritz@viebel s/alt is a regex op, s/or is not#2017-02-0108:30Yehonathan Sharvit@dergutemoritz But if I put s/alt inside an s/cat expression it seems to work as expected.#2017-02-0108:30dergutemoritz@viebel You mean s/or?#2017-02-0108:32Yehonathan Sharvityeah sorry for the typo: But if I put s/or inside an s/cat expression it seems to work as expected.#2017-02-0108:34dergutemoritz@viebel The difference is that a s/cat within an s/alt will be spliced whereas within an s/or it won't#2017-02-0108:34dergutemoritzLet me cook up an example#2017-02-0108:37dergutemoritz@viebel Here you go - hope it's clear, couldn't come up with a more concise example šŸ˜ž#2017-02-0108:57Yehonathan SharvitThanks a lot @dergutemoritz !#2017-02-0108:58Yehonathan SharvitNow itā€™s 100% clear!#2017-02-0108:58dergutemoritzYou're welcome @viebel, glad it helped!#2017-02-0109:08Yehonathan SharvitAnd to make it 200% clear here is your code snippet @dergutemoritz inside klipse for a live demo http://app.klipse.tech/?eval_only=1&amp;cljs_in.gist=viebel/6bdefe58f4a38591399f0628fb775418#2017-02-0114:27souenzzoOh...
(spec/valid?  :my.spec/base  lista)
=> false
(spec/explain  :my.spec/base  lista)
Success!
=> nil
#2017-02-0114:38souenzzoThere is some debug Tool? lista is a deep nested map My Spec has 10+ referentes/ns and some recursion#2017-02-0114:44souenzzoI'm using 1.8 backport#2017-02-0114:47joshjoneswell that changes things ā€” does alpha14 exhibit the error?#2017-02-0116:29souenzzoNo bugs on 1.9.0-alpha14 šŸ™ šŸŽ‰#2017-02-0116:33pesterhazyanyone have a reading/watching/listening list for spec? I'm just getting started#2017-02-0116:33pesterhazyother than https://clojure.org/about/spec of course which I'll start with#2017-02-0116:34mpenet@pesterhazy for gen related stuff I really like https://github.com/clojure/test.check/blob/master/doc/cheatsheet.md#2017-02-0116:59gdeer81does anyone fdef their conformer functions?#2017-02-0119:41Alex Miller (Clojure team)@pesterhazy well https://clojure.org/guides/spec of course#2017-02-0119:41Alex Miller (Clojure team)and then there are some vids Stu has done at https://www.youtube.com/watch?v=nqY4nUMfus8&amp;list=PLZdCLR02grLrju9ntDh3RGPpWSWBvjwXg#2017-02-0119:42Alex Miller (Clojure team)and some at http://blog.cognitect.com/?tag=clojure.spec#2017-02-0121:11pesterhazyThanks!#2017-02-0221:14villesvHi, I am wondering if maybe I have stumbled across an issue in clojure.spec. I have perused the issue list but have found nothing that I think describes the issue at hand for me.#2017-02-0221:15villesvA small example:
(s/def :some.ns.1/k string?)
(s/def :some.ns.1/m (s/keys :req [:some.ns.1/k]))

(s/def :some.ns.2/k string?)
(s/def :some.ns.2/m (s/keys :req-un [:some.ns.2/k]))

(s/explain-str :some.ns.2/m {:k "1"
                             :some.ns.1/m {}})
;; => "In: [:some.ns.1/m] val: {} fails spec: :some.ns.1/m at: [:some.ns.1/m] predicate: (contains? % :some.ns.1/k)\n"
#2017-02-0221:17villesvMy point being, although I am checking (doesn't matter if I use s/explain-str, s/assert etc) for :some.ns.2/m - clojure.spec ends up also validating another spec.#2017-02-0221:18villesvThat is, although :some.ns.1/m is in there and not conforming to spec - I did not expect it to be checked for. I also noticed that this happens both when I use :req-un and :req for :some.ns.2/m#2017-02-0221:18villesvI am thinking this is a bug#2017-02-0221:37villesvHmm, no seems I am mistaken, I found this in the spec source docs for clojure.spec/keys: > In addition, the values of all namespace-qualified keys will be validated > (and possibly destructured) by any registered specs. Note: there is > no support for inline value specification, by design. How have I not stumbled across this before?#2017-02-0223:30ghadidunno -- it's in the guide too , I'm pretty sure. Best way to understand is that namespaced keys have program-global meaning#2017-02-0302:38joshjones@fnil, it's always been this way...#2017-02-0307:32luxbockfinally starting to get a hang of how to use instrument effectively while doing interactive development via the REPL, :replace is very handy in isolating parts of a long pipeline of functions that build up the computation step by step#2017-02-0307:38luxbockhaving a version of enumerate-namespace that recursively enumerates all the internal functions called by some top-level function would be quite handy#2017-02-0307:39luxbocknot sure if that already exists in some library, if anyone knows let me know#2017-02-0309:08villesv@ghadi @joshjones You are correct and I am embarrassed šŸ˜„ I think the case at hand may have contributed to the confusion. I was applying specs to a deep, previously unspec:ed map where both spec:ed and unspec:ed keys were present.#2017-02-0309:50chrisblomi'm converting the json schema for AWS cloudformation to clojure.spec#2017-02-0309:51chrisblomthis schema has a few reflexive references#2017-02-0309:52chrisblomi'm running into problems when converting it to spec#2017-02-0309:53chrisblomi have a spec of form (s/def :foo/bar ...) where :foo/bar is used somewhere in the ...#2017-02-0309:54chrisblomthis gives me an Unable to resolve spec error, what's the recommended way to do such circular definitions in spec?#2017-02-0310:38villesvOh, I would love to have some sanity brought on to cloudformations json. Love the service but I stumble on the JSON details all the time#2017-02-0311:09villesvHow is :foo/bar used recursively? Doesn't something like:
(s/def :foo/bar (s/keys :opt [:foo/bar]))
(s/valid? :foo/bar {:foo/bar {:foo/bar {}}})
work? Or am I being naive?
#2017-02-0314:55frankthat seems to work in the repl#2017-02-0314:58joshjones@chrisblom What @fnil has suggested works for a map. A more fleshed out example:
(s/def ::k string?)
(s/def ::m (s/keys :req [::k] :opt [::m]))
(s/valid? ::m {::k "abc" ::z 42
               ::m {::k "def"
                    ::m {::k "ghi"}}})
#2017-02-0314:59joshjonesbut I don't know if you're spec-ing a map, a collection, or what ? you can also do recursive definitions for collections, etc., but if you can be more specific about the structure you're trying to spec..#2017-02-0315:49chrisblomhmm thanks guys, it's working now, not sure what i was doing wrong#2017-02-0315:55preporHello. I think that fully namespaced keywords without full application's namespace should be considered as very bad practice (:message/state for example) at least in library code. But why the official guide is full of such usages?
(s/def :event/type keyword?)
(s/def :event/timestamp int?)
(s/def :search/url string?)
(s/def :error/message string?)
(s/def :error/code int?)
#2017-02-0315:59villesv@prepor Why do you think so?#2017-02-0316:00joost-diepenmaatI think I agree @prepor. since specs are global I would say that libraries should never use keys outside of the namespaces already in the lib#2017-02-0316:01joost-diepenmaatto prevent clashes with application code and other libs#2017-02-0316:01frankI think it makes it a tad harder to find the namespace where the s/def lives. Also, if you're referring to the specs in other namespaces, it's harder to figure out which namespace you need to require in order to have those specs available#2017-02-0316:02prepor@fnil because of @joost-diepenmaat explanation, yes#2017-02-0316:08villesvAll true, although I find the flexibility great.#2017-02-0316:09joost-diepenmaatIā€™m using single-keyword namespaces in an application right now and itā€™s very neat. Writing libraries generally means sacrificing some easyness for interoperability#2017-02-0316:09villesvI definitely agree that it can be hard to locate where a certain keyword was defined (or sometimes redefined)#2017-02-0316:21prepor@joost-diepenmaat a boundary between application and library is often not clear. today it's application code, and tomorrow you can decide to split it into libraries to use inside your microservices.#2017-02-0316:22joost-diepenmaatTrue #2017-02-0316:23preporso, personally I decided to not use "short namespaces" at all. and for me it's strange that official guide forces this usage. maybe I don't understand something...#2017-02-0316:35seancorfieldThe guide uses ::stuff in most places tho` which is qualified by the full namespace. I think the other places in the guide are just examples -- certainly not "forcing" any particular usage on you. #2017-02-0316:36joshjonesThe spec guide seems to be aimed at learning how spec works, and as such, brevity and simplicity are key -- adding too much info on namespaces makes it more difficult to learn. ::mykey is short, and simple ... at any rate, if you want to do something like this, I think it's a good way to shorten the code while still being namespace-safe:
(create-ns 'my.app.events)
(alias 'event 'my.app.events)
(s/def ::event/type keyword?)
#2017-02-0316:39preporWhen something is used in guide without red warnings ā€” it's "forcing" )#2017-02-0316:42prepor@joshjones yes, I know about create-ns / alias#2017-02-0316:42seancorfieldIt's a guide not a set of rules. You're reading too much into it. #2017-02-0316:43seancorfieldIf anything I'd say the core team don't provide enough guidance. They really don't force their views on anyone. #2017-02-0316:47prepor@seancorfield will you think the same when you catch a bug because two of your dependencies have name conflicts without any warnings because the official guide guided their authors to not always use fully qualified keywords? šŸ™‚#2017-02-0316:50preporIt's near the only way how you can broke others code in clojure ecosystem in non tricky way, I think#2017-02-0316:56seancorfieldWhen the core team talk about spec, they're pretty clear about using namespaces to prevent exactly that bug. Again, you're treating a learning guide as a rule book. There doesn't seem much point in this discussion if you're set on blaming the core team for your mistakes šŸ˜ø #2017-02-0317:05preporhm. wrong tools force me to make my mistakes. is there no chance that core team did something wrong? don't think so. I'm sure that its ok, then somebody asks questions about core team decisions.#2017-02-0317:06preporI'm trying to find some big real world applications on github which uses core.spec but can't šŸ™‚#2017-02-0317:50sattvikOne option I have found that can work is using the :full.ns.no.alias/kw form can help work around circular dependencies.#2017-02-0318:18seancorfield@prepor We use spec in production fairly heavily ā€” and I know there are other companies already doing so ā€” but of course you wonā€™t find commercial application code on GitHub šŸ™‚#2017-02-0318:19seancorfield@prepor And remember the saying: ā€œItā€™s a poor craftsman that blames his tools.ā€ (not sure whether that saying originates but I remember hearing it a lot growing up)#2017-02-0318:54fentoncan you spec a variadic function?#2017-02-0318:56sattvikSure. You use the regex specs to do so.#2017-02-0319:00sattvikYou can do something like:
(s/fdef hello
  :args (s/cat (s/? string?))
  :ret string?)

(defn hello
  ([] (hello ā€œworldā€))
  ([name] (str ā€œHello, ā€œ name \!)))
#2017-02-0319:04fentonok... I haven't looked into this much but wondering about doing pattern matching with core.match and spec to be like a guard or something...#2017-02-0319:16Alex Miller (Clojure team)@prepor the guide uses shorter names and :: kws for readability. it would be reasonable to add a side bar explaining this.#2017-02-0319:35Alex Miller (Clojure team)@prepor sidebar added https://github.com/clojure/clojure-site/commit/cf0a744c2e59fcd0c6459d55dba429577399e6c0#2017-02-0319:50prepor@alexmiller cool, thank you! the only thing which I imagine to catch (and debug) such errors automatically is "production mode" for clojure.spec which forbids redefinition of specs in explicit way#2017-02-0319:59spieden@fnil we generate cloudformation templates as EDN and serialize to JSON, works great#2017-02-0320:00spieden@chrisblom that sounds like a great project. huge surface area, though#2017-02-0320:08villesv@spieden very cool! I have had such a thought but have not fully grasped cloudformation and its template language yet#2017-02-0320:09spiedenitā€™s a bit of a curve but has been a great workhorse for us#2017-02-0320:09spiedencanā€™t imagine maintaining a template by hand, though#2017-02-0320:10spiedenwe have lots of functions that build up common structures, etc.#2017-02-0320:10spiedenwithout DRYing things out like this itā€™d be crazy town#2017-02-0320:10villesvSpec should be great for it#2017-02-0320:13spiedenyeah for sure. they have their own (hosted) validation operation, but it misses things and works against the JSON representation#2017-02-0320:25villesvSo I noticed šŸ˜„#2017-02-0322:38kassapoCould you help a Newbie: I am going through http://blog.cognitect.com/blog/2016/10/5/interactive-development-with-clojurespec and cannot get past
codebreaker=> (s/exercise (:args (s/get-spec `score)))

FileNotFoundException Could not locate clojure/test/check/generators__init.class or clojure/test/check/generators.clj on classpath.  clojure.lang.RT.load (RT.java:458) 
I am using Clojure 1.9.0-alpha14 in lein repl
#2017-02-0322:41jradd [org.clojure/test.check "0.9.0"] to your (dev) dependencies#2017-02-0517:32nicolaHello, i need some verification from experts! Iā€™m thinking about refactor my json-schema library to clojure.spec generator from json-schema definitions. The problems i see: this will be generation based on macros (or there is another way?) and some json-schema could come from users - but specs are not garbage collected (some sandbox?) ? Does anybody have spec generation experience?#2017-02-0517:34nicolaOr may be this is a bad idea at all?#2017-02-0606:22Yehonathan SharvitIā€™m trying to understand the difference between s/coll-of and s/every. > Note that 'every' does not do exhaustive checking, rather it samples coll-check-limit elements. Nor (as a result) does it do any conforming of elements. 'explain' will report at most coll-error-limit problems. Thus 'every' should be suitable for potentially large collections.#2017-02-0606:23Yehonathan SharvitWhat does it mean that every doesnā€™t do any conforming of elements?#2017-02-0606:23Yehonathan Sharvit(s/conform (s/every number?) (range 1000)) seems to do the conforming pretty well!#2017-02-0608:52dergutemoritz@viebel Try with a spec that conforms to a different value#2017-02-0608:52dergutemoritzLike s/or#2017-02-0608:54Yehonathan SharvitLike this:
(s/conform (s/every (s/or :n number?
                          :s string?))
           (concat (range 1000)))
#2017-02-0608:55dergutemoritzYeah that should work#2017-02-0608:55Yehonathan SharvitIf I understand correctly s/every leaves the data as-is#2017-02-0608:55dergutemoritzYup!#2017-02-0608:55Yehonathan Sharvitwhile s/coll-of conforms it#2017-02-0608:55dergutemoritzIt will only check whether the values match the spec#2017-02-0608:56dergutemoritzRight#2017-02-0608:56Yehonathan SharvitThx @dergutemoritz#2017-02-0608:56dergutemoritzYW!#2017-02-0609:41kassapoI cannot find the correct way to call clojure.spec.test/check on my function that might read a file `#2017-02-0609:41kassapo
(s/def :read/file #(instance? File %))
(s/def :read/content string?)
(s/def :read/fail nil?)

(defn read-file [^File f] (when (.isFile f) (slurp f)))

(s/fdef read-file
  :args (s/cat :f :read/file)
  :ret (s/or :read/content :read/fail)
  :fn (fn [{{f :f} :args ret :ret}]
        (if (.isFile f)
          (s/valid? :read/content ret)
          (s/valid? :read/fail ret))))

(defn gen-file []
  (gen/elements (map (fn [^String f] (File. f))
                     ["./project.clj" "foo"])))

(s/exercise :read/file 4 {:read/file gen-file}) 
#2017-02-0610:47Yehonathan SharvitIā€™m reading (with enthusiasm) "CREATING A SPEC FOR DESTRUCTURINGā€ http://blog.cognitect.com/blog/2017/1/3/spec-destructuring. Everything is pretty clear except one point related to s/merge at the end of the article#2017-02-0610:47Yehonathan Sharvit
;; collection of tuple forms
(s/def ::map-bindings
  (s/every (s/or :mb ::map-binding
                 :nsk ::ns-keys
                 :msb (s/tuple #{:as :or :keys :syms :strs} any?)) :into {}))

(s/def ::map-binding-form (s/merge ::map-bindings ::map-special-binding))
#2017-02-0610:48Yehonathan SharvitI donā€™t understand why inside ::map-bindings, we have the :msb part#2017-02-0610:48Yehonathan SharvitAFAIU, the :msb part relates only to ::map-special-binding#2017-02-0612:16Yehonathan Sharvitany idea @dergutemoritz or @alexmiller ?#2017-02-0612:24dergutemoritz@viebel It makes sure that no other keywords than the ones in the set are present#2017-02-0612:24dergutemoritz::map-special-bindings is defined in terms of s/keys which is an open key set by design#2017-02-0612:25dergutemoritzAt least that's what I surmise is the motivation. Being the author of that post, @alexmiller might have more insight šŸ™‚#2017-02-0612:25Yehonathan SharvitThat makes sense. But I think it would be clearer if (s/tuple #{:as :or :keys :syms :strs} any?) was part of ::map-special-binding. What do u think @dergutemoritz ?#2017-02-0612:31dergutemoritzNot sure I would find it clearer that way#2017-02-0612:32dergutemoritzThe way it is now makes the ::map-special-bindings spec a tad more widely usable I guess#2017-02-0612:35Yehonathan Sharvitok. makes sense#2017-02-0618:23Alex Miller (Clojure team)making ::make-special-bindings reusable is not really a goal#2017-02-0618:23Alex Miller (Clojure team)the tuple check canā€™t be part of ::map-special-binding b/c that spec is a map spec which is merged with the rest#2017-02-0618:28Alex Miller (Clojure team)I think itā€™s probably a reasonable criticism that this spec is too restrictive (from a future-looking point of view) in narrowing to just this set of options as we know it now.#2017-02-0618:30Alex Miller (Clojure team)the intention is really to segment the space of possible tuple forms to bindings (which are simple symbols, vectors, or maps), qualified keywords whose names are ā€œkeysā€ or ā€œsymsā€, or special options. Here Iā€™ve listed a concrete set of options but lost the open-ness of something like keys. That could be addressed by replacing the #{:as ā€¦} with something more generic like simple-keyword?.#2017-02-0618:31Alex Miller (Clojure team)^^ @viebel @dergutemoritz#2017-02-0619:11bbloomthat makes some sense to me ^^ another option would be to use conformers to rewrite the :as in to a ::as node, so that itā€™s value can be properly checked to be a simple-symbol, for example#2017-02-0619:12bbloomthat seems like the the way to recover the openness with all the benefits of the global namespaced keys validation#2017-02-0619:21Yehonathan Sharvit@alexmiller I donā€™t understand why in this case, the open-ness is desirable. I mean if someone passes :wrong-keys (instead of :keys) to a let statement, it should be invalidated...#2017-02-0619:22Alex Miller (Clojure team)@bbloom no desire to use conformers for something like this#2017-02-0619:22Alex Miller (Clojure team)@viebel the same reason openness is desirable elsewhere (see Richā€™s keynote from the conj)#2017-02-0619:23Alex Miller (Clojure team)that is, if you specify a constraint, then removing the constraint later is a breaking change, not an additive change#2017-02-0619:23bbloom@viebel just to make up some straw man extension, consider if you wanted to add some metadata to the match#2017-02-0619:23bbloom{:keys [x y] :types {x int}} ;-)#2017-02-0619:24bbloomyou donā€™t want the spec to reject that types#2017-02-0619:26bbloom@alexmiller sorry, i was thinking about :as in vectors - i think conformers make sense there#2017-02-0619:26bbloomfor maps, itā€™s already a map šŸ˜›#2017-02-0619:27bbloomthe :msb form is interesting tho - why not just use a :req-un for the :as key?#2017-02-0619:28bbloomi must be missing some subtlety#2017-02-0619:29bbloomoooh, you also do that, this is about accumulating extra information in the parse - ok, i think that makes sense.... sorry for thinking aloud here#2017-02-0619:29Alex Miller (Clojure team)yes#2017-02-0619:30Alex Miller (Clojure team)this is a particularly tricky instance of what has been called a hybrid map#2017-02-0619:30Alex Miller (Clojure team)which has both a k/v spec + a keys-type spec#2017-02-0619:31Alex Miller (Clojure team)the tricky part being the :<ns>/keys and :<ns>/syms which are like keys but only semi-fixed#2017-02-0619:31bbloomyeah, my experience w/ hybrid maps has been that they are more trouble than they are worth except for very limited syntax use cases#2017-02-0619:31Alex Miller (Clojure team)nonetheless, they exist :)#2017-02-0619:31bbloomindeed#2017-02-0619:31bbloomiā€™ve run in to the ::keys bug#2017-02-0619:31Alex Miller (Clojure team)yeah#2017-02-0619:31bbloomit is not obvious what to do about that šŸ™‚ so i just renamed my thing, heh#2017-02-0619:32Alex Miller (Clojure team)well, I have some ideas about that but havenā€™t had a chance to put a patch on there#2017-02-0619:32Alex Miller (Clojure team)but basically it needs to not use s/keys#2017-02-0619:32Alex Miller (Clojure team)as that has behavior that we donā€™t want in this particular case#2017-02-0619:33Alex Miller (Clojure team)namely, matching all keys as specs#2017-02-0619:33bbloomyeah, i think the patch i saw there was like a flag to disable the nice open validation of keys - which seems like the wrong way to go about it. easier to factor out the parts you want and then call the underlying part directly#2017-02-0619:33bbloomie composition over parameterization#2017-02-0619:33Alex Miller (Clojure team)that patch is dead, not doing that#2017-02-0619:33bbloomi assumed that šŸ™‚#2017-02-0619:33Alex Miller (Clojure team)and I think I rewrote the ticket to remove that as an option#2017-02-0619:34bbloomcool - i have only been following at a distance#2017-02-0621:47hiredmanit seems like you can't use spec to spec a map like {::results ... ::type ...} where the value associated with the key ::type determines what spec is used against the value associated with the ::results key#2017-02-0621:49hiredmanit seems like the best you could do would be something with multi-spec, with a different ::results spec (registered under a different key) for each possible value of ::type
#2017-02-0621:57hiredmanI am trying to spec what I have been thinking of as a result set, a map with a structure like {::results [...] ::next ...} where ::next is used for pagination, and the items under ::results will vary with the query that was run. my first thought was to slap a type tag on the result set, and use a multi-spec, but that doesn't work.#2017-02-0622:15bbloomhiredman: weā€™ve talked about context sensitivity a bunch of times here in the past - you have a number of options#2017-02-0622:15bbloomnone of them particularly great#2017-02-0622:16bbloomthe simplest one is to just call the spec you want yourself#2017-02-0622:16hiredmantoo opaque#2017-02-0622:16bbloomwell it depends on your use case#2017-02-0622:16hiredmanyeah, for mine it is too opaque šŸ™‚#2017-02-0622:16bbloomheh, ok#2017-02-0622:17bbloomi mean, itā€™s just an s/conform call#2017-02-0622:17hiredmanI think I will just have a bunch of slightly different specs, remove any kind of polymorphism#2017-02-0622:17bbloomyeah, so thatā€™s the next simplest solution#2017-02-0622:17hiredmanfor now, or until something better comes up#2017-02-0622:17bbloomone key for each possible type and then use multi-spec#2017-02-0622:18bbloomso you have like ::foo-results and ::bar-results#2017-02-0622:18hiredmanI am generating documentation and json schema from these specs#2017-02-0622:18hiredmanyeah#2017-02-0622:18hiredmanwell, easiest, since consumers consume json, is to differ based on namespace, which just disappears in the json encoding#2017-02-0622:18bbloomheh, well i guess that works šŸ˜›#2017-02-0622:32bronsawhat about something like
(require '[clojure.spec :as s])

(s/def ::type keyword?)
(s/def ::value any?)

(defmulti t-val identity)
(defmethod t-val :a [_] int?)
(defmethod t-val :b [_] string?)

(s/def ::tagged (s/and (s/keys ::type ::value)
                       #((t-val (::type %)) (::value %))))

(def foo {::type :a ::value 1})
(def bar {::type :b ::value "2"})
(def baz {::type :b ::value 1})

(s/valid? ::tagged foo) ;-> true
(s/valid? ::tagged bar) ;-> true
(s/valid? ::tagged baz) ;-> false
#2017-02-0622:34hiredmanonce you start building using functions instead of the combinators (like s/keys) explain is less useful, and any kind of treatment of specs as data becomes more of pain (like if you are walking them to generate something else)#2017-02-0710:01p-himikIs it possible, given some data, a keyword for its spec and a keyword for a simple spec that is a part of the data spec, extract all values conforming to the simple spec?#2017-02-0710:02p-himikE.g. I have a map id->val and three specs - for id, for val and for the map itself. And I'd like to extract all vals from the map.#2017-02-0714:42frankso not all of the map entries in this map conform to the map spec, but you want to find the ones that do?#2017-02-0720:42ddellacostahow to specify a map that may be recursive?#2017-02-0721:07joshjonesactually, any map spec is recursive, since a map spec allows keys not specified in the spec#2017-02-0721:08joshjonesbut for clarity's sake, you may want to put the spec name in the :opt portion of the map spec#2017-02-0721:08ddellacostaitā€™s recursive perhaps, but not useful if it doesnā€™t validate anything about the map, is it?#2017-02-0721:09joshjonesit will validate anything given in :req etc#2017-02-0721:09hiredman
user=> (s/def ::foo (s/or :a number? :b ::bar))
:user/foo
user=> (s/def ::bar :req [::foo])
ArityException Wrong number of args (3) passed to: spec/def  clojure.lang.Compiler.macroexpand1 (Compiler.java:6832)
user=> (s/def ::bar (s/keys :req [::foo]))
:user/bar
user=> (s/valid? ::bar {::foo 1})
true
user=> (s/valid? ::bar {::foo {::foo 1}})
true
user=> (s/valid? ::bar {::foo {::foo :a}})
false
user=> 
#2017-02-0721:09ddellacostaas it is, I have
(s/or (s/map-of string? keyword?) (s/map-of string? map?))
but that is only recursing one level
#2017-02-0721:10ddellacosta@hiredman thanks, that looks like it#2017-02-0721:12ddellacostaoh hrm maybe not with the map I have here#2017-02-0721:12hiredman
user=> (require '[clojure.spec :as s])
nil
user=> (s/def ::foo (s/map-of string? (s/or :value keyword? :rec ::foo)))
:user/foo
user=> (s/valid? ::foo {"a" :a})
true
user=> (s/valid? ::foo {"a" 1})
false
user=> (s/valid? ::foo {"a" {"b" :b}})
true
user=> 
#2017-02-0721:12ddellacostahaha, on cue#2017-02-0721:12joshjonesyou don't even need to do that#2017-02-0721:13joshjones
(s/def ::a string?)
(s/def ::mymap (s/keys :req [::a]))
(s/valid? ::mymap {::a "abc" ::mymap {::a "def" ::mymap {::a "xyz"}}})
#2017-02-0721:13ddellacostathat works if your keys are namespaced keywords#2017-02-0721:13ddellacostabut I have string keys here#2017-02-0721:14joshjonesi see#2017-02-0721:14ddellacostabut, Iā€™m filing all of this away for the time I do need such a constructionā€¦thanks @joshjones and @hiredman , very helpful#2017-02-0723:48agHow do I generate map with keys and values that correspond to a given (s/keys) spec? let's say I have (s/keys :req-un [::id ::name] :opt-un [::money]) based on that I should generate vector like:
[{:id 1 :name "Greg" :money 0.02} {:id 2 :name ā€œAliceā€}]
upd: I actually want the keys to be randomly selected out of that spec, not precisely the same keys
#2017-02-0723:50bfabry@ag https://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec/exercise#2017-02-0723:50hiredmanhaving the same key in req and opt doesn't make sense#2017-02-0723:50bfabry^ that is true#2017-02-0723:51ag@hiredman sorry my mistake. fixed it#2017-02-0723:52hiredmanmaybe close the paren too while you are at it#2017-02-0723:53hiredmanit depends sort of on what purpose you are generating them for, but if you generally just want to poke at them, exercise is a way to generate data for a give spec, s/keys created or otherwise#2017-02-0723:55agoh, I guess I got it all wrongā€¦ hold on..#2017-02-0723:57agI need to generate map where the keys are ā€œrandomly selectedā€ out of the s/keys spec and the corresponding values also generated according to the spec#2017-02-0723:58bfabrysounds like you want another s/keys spec with all the keys optional#2017-02-0723:59agwell, can I read the keys out of the previously defined s/keys spec?#2017-02-0800:00agor somehow tell test.check.generator/map or hash-map that I want keys to be out of the spec?#2017-02-0800:02bfabryyou should be able to read the keys out using s/form, assuming it's a simple spec#2017-02-0800:03bfabry
cljs.user=> (require '[clojure.spec :as s])
nil
cljs.user=> (s/form (s/keys :req-un [::foo] :opt-un [::bar]))
(cljs.spec/keys :req-un [:cljs.user/foo] :opt-un [:cljs.user/bar])
#2017-02-0800:04bfabryofc if it's a more complicated spec you'll need to walk the form finding the key specs
cljs.user=> (s/form (s/and (s/keys :req-un [::foo] :opt-un [::bar]) identity))
(cljs.spec/and (s/keys :req-un [:cljs.user/foo] :opt-un [:cljs.user/bar]) cljs.core/identity)
#2017-02-0800:05bfabry(being able to drop into a 1.9 repl in sub-seconds rules, thank you planck)#2017-02-0800:09aghmmm, interesting. thanks!#2017-02-0800:24ag@bfabry I did not know about s/form, just played with it - itā€™s nice. thanks again!#2017-02-0800:25qqqwhat is the standard way of memoizing spec checks? I have a recursive data structure (say a tree), and I don't want it to have to verify the property holds on subtrees every time -- I want to somehow 'cache' "this node passed spec xyz" somehow#2017-02-0800:26qqqwhen memoizing, can we somehow do it with "weak references" so that instead of having gc leaks, the object is thrown away when it's no longer needed ?#2017-02-0800:28qqqI suspect one nice thing about meta data over memoization is that with meta data, with the object is no longer needed, and the gc gcs it, it takes away the metadata too#2017-02-0800:28qqqwith memoization, I fear it'll prevent the gc from doing its work because the spec function will keep th eobject around#2017-02-0800:32hiredmanwhy are you running spec checks over and over on a large structure? like, just don't do that#2017-02-0800:36qqqif I have input argument spec validation on, wouldn't it run the spec every time the function is called?#2017-02-0800:37qqqnow, if this structure is recursive (say: this is a tree, where the sum of every subtree is a multiple of 3), then it ends up checking the entire tree every time#2017-02-0800:38hiredmanmy first concern with that question is it sounds like you are planning to turn on spec instrumentation outside of your test suite#2017-02-0800:38qqqthat is true; is it bad to have spec instrumentation turned on in production code? if it's a costant time hit, I would not mind#2017-02-0800:38hiredmanit is bad, spec is not designed for that#2017-02-0800:39hiredmanI should say#2017-02-0800:39hiredmanthe instrument stuff in spec is not designed for that#2017-02-0800:40hiredmanthe intended use, as far as I understand, is to turn instrumentation on when running your test suite#2017-02-0800:40hiredmanand that is it#2017-02-0800:40qqqI think assertions are good in dev code (not just in test quite). I'm trying to use spec as "ultra powerful assert" in dev code. You're saying this is bad, and I should not use spec as an assert ?#2017-02-0800:43qqqFor some functions, I want to assert that their input satisfies certain pre-conditions. These pre-conditions can be expensive (checking all subtrees); therefore, I'd like a way to cache this. Now, given that I want to do the above, can I do this with spec, or would spec be a bad match? I'm hoping that spec can do this, as I really like spec's language.#2017-02-0800:43bfabryimo many many people will use spec the way you describe, but there are some things to be wary of: it will be slow. if you spec any functions that you pass as arguments those functions will be called multiple times to check that they pass the spec as part of instrumentation. instrumentation does not check return values#2017-02-0800:44qqq@bfabry: we can assume I don't need to "run specs on functions passed as inputs" -- and my only conern at the moment being simple data structures and recurisve data sturcutres // functions are way too hard#2017-02-0800:45hiredmanif you don't have any automatic checking turned on, checking will only happen when you ask for it, so just don't ask for expensive checks more than once#2017-02-0800:46bfabryfwiw there's no reason you can't use spec's data description language to validate data without using instrumentation. just call s/valid or s/conform directly. you could even wrap your calls to s/valid and s/conform in something that memoizes based on the object's reference id or whatever if you really want to tweak performance of that validation#2017-02-0800:48qqq(defn my-valid [spec data] (or (contains? (meta data) spec) (s/valid spec data))#2017-02-0800:48qqqor something of that nature, this is clever#2017-02-0800:50bfabryyeah exactly, then you can stick that in :pre if that's your preferred mechanism#2017-02-0800:50joshjonesif the function is in clojure.spec.test, don't use in prod#2017-02-0800:50bfabryyou could even still associate the spec with the function, so you get the documentation, generation stubbing when wanted etc, just don't call s/instrument#2017-02-0800:51qqqhttp://blog.fogus.me/2009/12/21/clojures-pre-and-post/ <-- this existed 7 years ago, finally starting to use it now#2017-02-0800:52joshjonesthe spec guide under "Using Spec for Validation" gives the :pre and :post example for reference#2017-02-0800:55qqq@joshjones: found it, thanks!#2017-02-0804:20onetom@ag ^^^#2017-02-0804:27onetomIs it expected behaviour to get a clojure.spec/unknown explanation if a non-conforming spec has a custom generator, like this:
(s/def ::a (s/with-gen string? identity))
  (s/def ::some-map (s/keys :req [::a]))
  (s/explain ::some-map {::a nil})

In: [:app.deals/a] val: nil fails spec: :app.deals/a at: [:app.deals/a] predicate: :clojure.spec/unknown
#2017-02-0804:28onetomInsead of
(s/def ::a string?)
...
In: [:app.deals/a] val: nil fails spec: :app.deals/a at: [:app.deals/a] predicate: string?
#2017-02-0804:46seancorfieldWhat sort of generator is identity?#2017-02-0804:46seancorfield(hmm, and doesn't with-gen take a nilary function that returns a generator?)#2017-02-0804:47seancorfieldYup, "Takes a spec and a no-arg, generator-returning fn and returns a version of that spec that uses that generator" -- identity does not satisfy that.#2017-02-0804:48seancorfieldSo (s/with-gen anything identity) doesn't make sense... you're not going to get a valid generator from that?#2017-02-0804:52seancorfieldYeah, if you (s/exercise ::a) you'll get
boot.user=> (s/exercise ::a)

clojure.lang.ArityException: Wrong number of args (0) passed to: core/identity
which is what I'd expect.
#2017-02-0804:52seancorfieldSo the error from s/explain isn't surprising but it is perhaps a bit misleading @onetom#2017-02-0804:53onetomah, sorry, i just threw identity in there because it's an irrelevant detail#2017-02-0804:54onetomi observed the very same behaviour in our actual app with this real generator:
(s/def :deal/name (-> string?
                      (s/with-gen #(gen/fmap
                                     (fn [s] (str "<DEAL-NAME-" s ">"))
                                     (gen/string-alphanumeric)))))
#2017-02-0804:56onetom
(s/def ::a (s/with-gen string? gen/string-alphanumeric))
(s/def ::some-map (s/keys :req [::a]))
(-> ::some-map s/gen gen/generate)
(s/explain ::some-map {::a nil})

=> #:boot.user{:a "lybK4CU4teXKCnErk9h5ajdGBu"}
In: [:boot.user/a] val: nil fails spec: :boot.user/a at: [:boot.user/a] predicate: :clojure.spec/unknown
#2017-02-0804:56seancorfieldOK, that does s/exercise...#2017-02-0804:57seancorfieldYeah, that sounds like it's worth a JIRA issue...#2017-02-0804:57seancorfieldI'd expect a better message, at least.#2017-02-0804:59onetomand this is the very first time i wrote a custom generator for an actual real-world use case... thats my generic experience with software... sometimes im wondering im just unlucky. what comforts me slightly is that i have a friend who is an order of magnitude "unluckier" šŸ™‚#2017-02-0804:59onetomok, i will make a JIRA issue. (this will be my first JIRA issue... im already worried what will happen ;)#2017-02-0805:03seancorfieldFunctionally-linked people are very useful in QA'ing software šŸ™‚#2017-02-0805:35onetomI've created http://dev.clojure.org/jira/browse/CLJ-2107 but it seems I can't edit it to correct the markdown syntax in it šŸ˜•#2017-02-0805:48seancorfieldHow's that? (edited)#2017-02-0805:49seancorfield(uses {code} around code not three backticks)#2017-02-0805:50seancorfieldand {{ }} instead of backticks for inline code.#2017-02-0807:20hiredmanwith-gen is a function, so it's arguments are evaluated, so the string? argument to with-gen is a function object, when spec is trying to find the name to report it does some stuff, which for symbols and keywords reports a good name, but for other Objects (including function objects) you get :clojure.spec/unknown#2017-02-0807:21hiredman
user=> (s/def ::a (s/with-gen (s/spec string?) identity))
:user/a
 (s/def ::some-map (s/keys :req [::a]))
:user/some-map
user=>   (s/explain ::some-map {::a nil})
In: [:user/a] val: nil fails spec: :user/a at: [:user/a] predicate: string?
nil
user=> 
#2017-02-0807:21hiredmanwrapping with s/spec allows the spec macro to capture the meaningful, the symbol before evaluation#2017-02-0808:41mpenets/spec also takes :gen so you can avoid calling with-gen separately, I find it nicer personally#2017-02-0808:41mpenet(s/def ::a (s/spec string? :gen identity))#2017-02-0808:42mpenetI almost never use with-gen because of this now that I think of it#2017-02-0810:41luxbockI have some data which I can neatly define a spec for using the regexp combinators of spec, which is great because I can generate example data for free, but for the functions I'm testing it's quite important that the returned sequences are vectors rather than list#2017-02-0810:43luxbockcoll-of and every allow you to define the type of the sequence, but they don't know about the structure of the sequence#2017-02-0810:44luxbockis there any easier way to force the generators to produce vectors than using a custom generator that calls vec on them?#2017-02-0810:46linussHey guys, I'm trying to use the simple-type generator in one of my specs, but I can't seem to get it to work. I currently have (s/def ::any (s/spec (fn[] (true)) :gen #(gen/simple-type))) but it seems that whatever permutation of the :gen value I use it returns an error#2017-02-0811:53dergutemoritz@linuss Can you paste the error you get? At any rate, the parens around (true) look wrong, that means you are calling true as a function#2017-02-0811:56linuss@dergutemoritz Ah, thanks! Yeah, that doesn't help. However, I'm still getting the following error: java.lang.IllegalArgumentException: No value supplied for key: (fn* [] (gen/simple-type))#2017-02-0811:57linussoh, hold up#2017-02-0811:57linusstypo on my end#2017-02-0811:57linussjava.util.concurrent.ExecutionException: clojure.lang.ArityException: Wrong number of args (1) passed to: specs/fn--10638 That's the error I keep getting#2017-02-0812:03dergutemoritz@linuss Ah, right, your spec predicate function needs to accept a single argument.#2017-02-0812:04dergutemoritzSo (fn [x] true) would do the trick. You can use any? instead, too, which is the core function with the same behavior.#2017-02-0812:05dergutemoritzPlus that clojure.spec has a default generator for it#2017-02-0812:05linussOh wow, thanks!#2017-02-0812:06dergutemoritzYou're welcome!#2017-02-0812:07linussOh, hm, I can't seem to find any?. Could you point me to the right location?#2017-02-0812:08dergutemoritz@linuss It's in clojure.core since 1.9#2017-02-0812:08linussah!#2017-02-0812:33trisshey all, Iā€™m trying to run stest/check against some functions#2017-02-0812:33trissbut i get the following error:#2017-02-0812:33triss
...
{:result #error {
 :cause "Could not locate clojure/test/check/generators__init.class or clojure/test/check/generators.clj on classpath."
 :via
 [{:type java.io.FileNotFoundException
   :message "Could not locate clojure/test/check/generators__init.class or clojure/test/check/generators.clj on classpath."
   :at [clojure.lang.RT load "RT.java" 458]}]
 :trace ...
#2017-02-0812:35trissdo i need a particular dependency?#2017-02-0812:36not-raspberryYes. [org.clojure/test.check "0.9.0"]#2017-02-0812:37not-raspberryAs in the docs https://clojure.org/guides/spec#_project_setup#2017-02-0812:56pbailleHi, i'm curious about how/should spec can be used as dispatching system?#2017-02-0813:45pbailledoes this question even make sense? šŸ™‚#2017-02-0813:59dergutemoritz@pbaille Depends on what you mean by dispatching sytem šŸ˜„#2017-02-0813:59pbaillesomething like multimethods#2017-02-0814:00dergutemoritzNot really.. you can certainly use s/conform in a multimethod's dispatch function, though#2017-02-0814:00dergutemoritzWhich seems like it could be a useful thing#2017-02-0814:01pbaillei've done a little gist about this, doesn't look really nice... https://gist.github.com/pbaille/b1bc0d05c2ec428e220fa28d28c8354f#2017-02-0814:03pbaillelooks like performance is an issue here, and the try catch stuff is ugly...#2017-02-0814:04pbaillebut that illustrate what i am trying to acheive#2017-02-0814:06pbaillethat's probably a terrible idea šŸ™‚#2017-02-0814:30linussWould it be possible to write a spec for a function with side-effects, like slurp?#2017-02-0814:38Timmaybe if it takes input?#2017-02-0816:08cryptoratI am trying to write a definition for ā€œweeks in a yearā€ and having difficulty checking that the integer is < 53. (s/def ::valid-weeks-in-year (s/and ::non-negative-integer #(< % 53))) It is failing with
java.lang.ClassCastException: clojure.lang.MapEntry cannot be cast to java.lang.Number
. What am I missing?
#2017-02-0816:09cryptoratIn a general case, how do I check that a number falls in a range?#2017-02-0816:12linussI think your error is somewhere else, this works without issue on my end#2017-02-0816:13linuss(s/def ::non-negative-integer (s/and int? #(>= % 0))) (s/def ::valid-weeks-in-year (s/and ::non-negative-integer #(< % 53))) (s/valid? ::valid-weeks-in-year 10) => true#2017-02-0816:14ghadi@cryptorat int-in ?#2017-02-0816:14ghadihttps://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec/int-in#2017-02-0816:15cryptoratoh, int-in. That looks like it will work.#2017-02-0816:15cryptoratI wonder if my non-negative-integer is the problem. Let me try with yours.#2017-02-0816:16cryptoratYup, that is it.#2017-02-0816:16cryptoratWell I learned two things. Thank you.#2017-02-0816:23cryptoratThe problem seems to lay in using `(s/def ::non-negative (s/or :positive pos? :zero zero?))` instead of #(>= % 0)#2017-02-0816:28cryptoratAnd since or returns a map entry....#2017-02-0816:28cryptoratWell that all makes sense now.#2017-02-0816:54joshjones@cryptorat a non-negative integer is also known as a natural integer. 1.9 has a predicate for this, so you can just use nat-int? as your predicate, although for your case, as @ghadi said, int-in is probably more appropriate since you need an upper bound#2017-02-0816:57cryptoratMy understanding is that whether or not zero is included in the set of natural numbers can be debated. I figured best to avoid that in case someone changes their mind. Too cautious perhaps.#2017-02-0816:58joshjonesthis is a good point ā€” i doubt the definition will change in the clojure universe but good catch nonetheless. but as you have already seen, (s/int-in 0 53) is much better anyway for your case#2017-02-0817:18akielIf you have two maps where map :a has :a/id and map :b likes to reference a particular map :a. How would you call the key in :b? Would you just use :a/id or would you create a new key called something like :b/a-ref?#2017-02-0817:33joshjones@akiel an example of the two maps would help clarify#2017-02-0817:36akieleither {:a/id 1 :b/name ā€œfooā€} references {:a/id 1 :a/name ā€œbarā€} or {:b/a-ref 1 :b/name ā€œfooā€} references {:a/id 1 :a/name ā€œbarā€}#2017-02-0817:37joshjoneswhat do you mean ā€œreferencesā€ ?#2017-02-0817:38akiela big disadvantage of using :a/id also for such kind of references is, that not every map containing an :a/id can be considered to be an :a.#2017-02-0817:38akielby references I mean the same what in relational databases forein keys do#2017-02-0817:39akielI canā€™t embed the referenced map directly, because that would blow up the data#2017-02-0817:44akielTo make it more spec relevant - I ask because in spec keywords are used as names for something like types and that keywords are also used in maps to name something like attributes.#2017-02-0817:47dergutemoritz@akiel I'd go with a different name, i.e. the :b/a-ref version. Because a reference to a thing is not the same as the thing itself after all.#2017-02-0817:51dergutemoritzThen again, embedding the data directly shouldn't really have that much of an impact if you have the thing that is pointed to in memory anyhow#2017-02-0817:55akiel@dergutemoritz I lean also towards using :b/a-ref. Regarding embedding directly you are right, it wonā€™t cost memory. There I was wrong.#2017-02-0817:56akiel@dergutemoritz But it would cost on wire. I need to transport that data over wire.#2017-02-0817:56dergutemoritzI see#2017-02-0817:59dergutemoritzAnother reason to name it :b/something is that you can then use a name that describes the relationship. E.g. if :a is :person and :b is :book then you could have (s/def :book/author :person/id) which I'd say would even justify leaving off the -ref suffix.#2017-02-0817:59dergutemoritz@akiel ^#2017-02-0818:00akiel@dergutemoritz Yes you are right - role names.#2017-02-0818:03akiel@dergutemoritz A related thing: Would you go for all keys in a person to start with the namespace :person or would you also use other common keys in a person? Like :person/name, :book/name vs. just :common/name.#2017-02-0818:09dergutemoritz@akiel I don't know, I think that's subject to an ontological debate you have to have with your domain experts šŸ™‚#2017-02-0818:11dergutemoritzNote that you could have both in a way: (s/def :common/name string?) (s/def :person/name :common/name)#2017-02-0818:11dergutemoritzThat way you could attach additional meaning to :common/name and also have it influence :person/name#2017-02-0818:12akiel@dergutemoritz The domain is given in my case. But I just think about map keys in relation to specs. With 20 different names, you end up with 20 specs for names which are all the same. Iā€™m not sure if thats a bit of an antipattern now regarding to spec.#2017-02-0818:13dergutemoritzIt really depends on your domain#2017-02-0818:13akiel@dergutemoritz Than you have specs like :common/name which are never used as keys in a map. That doesnā€™t have to be a problem - just thinking about it.#2017-02-0818:14dergutemoritzYeah, that would be an abstract spec probably#2017-02-0818:15dergutemoritzI guess the question is whether the concept of a name is universal in your domain or specific to each entity. Or maybe a mixture of those.#2017-02-0818:15dergutemoritzBut I don't feel like I've fully figured this out, yet, either#2017-02-0818:19akielFor example I have a transaction type which can have values like :insert or :update. It has the same meaning and the same values for each entity. Should I have a transaction type for every entity or just one?#2017-02-0818:26dergutemoritzIf it has the same meaning in all contexts, then I'd go with a single one#2017-02-0818:27akiel@dergutemoritz Thanks, that sounds reasonable. Same name for the same thing. šŸ™‚#2017-02-0818:27dergutemoritzšŸ‘#2017-02-0818:28dergutemoritzAt least that's my current understanding of things šŸ™‚ If anyone else has another interpretation, I'm all ears#2017-02-0818:28akielThat's why I liked to discuss a bit on that matter.#2017-02-0823:00seancorfieldWas that meant for a different channel @zane ?#2017-02-0823:10zaneSure was!#2017-02-0911:17mike_ananevHello! If i have regular fn which returns some number, how to wrap it into generator for spec?#2017-02-0911:37not-raspberryLike, random number? This is pretty much not the point of generators. To my understanding, you're supposed to extend some existing generator by using it as a seed for your generator.#2017-02-0911:39not-raspberrylook at clojure.spec.gen/fmap#2017-02-0911:44mike_ananev@not-raspberry i know about fmap. Can you provide some example how to use it in my question? I have fn (e.g. block-number-fn) which can produce block numbers. I have a spec for block. How to express block-number-fn as gen for block spec?#2017-02-0911:45not-raspberrywhat's a block number?#2017-02-0911:45not-raspberryan id?#2017-02-0911:45mike_ananevit is objects#2017-02-0911:45mike_ananevmore complex structure#2017-02-0911:46not-raspberryand without clojure.spec, how would you generate it? {:www (random) :zzz (random)}?#2017-02-0911:47mike_ananevi have a Java constructor#2017-02-0911:47mike_ananev(Block. "bla" "bla")#2017-02-0911:48mike_ananevmore simple example#2017-02-0911:48not-raspberrycan you generate it based on 1 random integer/double?#2017-02-0911:49mike_ananevforget blocks#2017-02-0911:49mike_ananevanother example#2017-02-0911:50mike_ananevi need gen for uuid#2017-02-0911:50mike_ananevi know that gen for uuid already in spec#2017-02-0911:50mike_ananevbut if there no one?#2017-02-0911:50mike_ananev(UUID/randomUUID) fn return uuid#2017-02-0911:51mike_ananevhow to wrap fn with (UUID/randomUUID) into gen?#2017-02-0911:51not-raspberryLet's limit ourselves to textual uuids.#2017-02-0911:52not-raspberryyou write a spec for a character allowed in uuid, e.g. a set of those, you get a gen for free#2017-02-0911:52mike_ananevi understand it, this sounds like workaround#2017-02-0911:52not-raspberrythen you write a spec for a coll of those chars with rexactly 24 chars (let's say uuids have 24 chars, i dunno)#2017-02-0911:53mike_ananevhow to wrap fn above into gen?#2017-02-0911:53not-raspberrythen you write your own genetrator with fmap that maps the previous generator with a function that inserts hyphens blocks with#2017-02-0911:54not-raspberryyou miss the point - don't use null-ary functions in specs. That makes them impossible to reproduce.#2017-02-0911:54mike_ananevok#2017-02-0911:55mike_ananevbe reproducable is mandatory?#2017-02-0911:58not-raspberry
(defn gen-string-thats-somewhat-funny []                                                                            
  (sgen/fmap
    do-something-funny-with-string  
    (sgen/string-alphanumeric)))
where do-something-funny-with-string is not funny itself but the results of it should be. Also it's pure.
#2017-02-0911:59not-raspberryIs reproducible mandatory? Nothing is pretty much. But it's inconvenient to stand out.#2017-02-0912:00not-raspberryI haven't found an easy way to write "non-pure" generators.#2017-02-0912:01not-raspberryyou can always discard the argument the function passed to fmap gets.#2017-02-0912:01not-raspberryBut the API overall seems to have been designed to discourage that.#2017-02-0912:09gfredericksreproducibility and shrinking are the two big features you get by constructing generators using the combinators#2017-02-0912:24mike_ananev@not-raspberry thanks. went to "hammock"#2017-02-0912:25gfredericks@mike1452 to address your specific example, I would generate a UUID by generating a pair of longs and calling the UUID constructor#2017-02-0912:26gfrederickswhich is essentially what the builtin UUID generator does#2017-02-0912:27mike_ananev@gfredericks can you provide source of builtin gen?#2017-02-0912:27mike_ananevlink i mean#2017-02-0912:27mike_ananevi saw macros name in sources but can't find particular code for this#2017-02-0912:27gfredericks@mike1452 https://github.com/clojure/test.check/blob/test.check-0.9.0/src/main/clojure/clojure/test/check/generators.cljc#L1256#2017-02-0912:28mike_ananevthanks#2017-02-0912:29gfredericksas the comments say, the jvm code there is lower level because it's faster; that wouldn't be a normal way to build generators#2017-02-0912:40mike_ananev@gfredericks thank you, I will try it#2017-02-0915:04ericnormandHello!#2017-02-0915:04ericnormandI have a question about the clojure.spec guide#2017-02-0915:04ericnormandit says: > The map spec never specifies the value spec for the attributes, only what attributes are required or optional.#2017-02-0915:05ericnormandbut then it does look like the values are being specified#2017-02-0915:05ericnormandthe next sentence is: > When conformance is checked on a map, it does two things - checking that the required attributes are included, and checking that every registered key has a conforming value.#2017-02-0915:06not-raspberryyou mean s/keys?#2017-02-0915:06not-raspberryit specifies keys that specify values#2017-02-0915:07joshjones@ericnormand so what is the question eric?#2017-02-0915:08ericnormandthey seem to contradict#2017-02-0915:08ericnormanddoes the s/keys specify values or not?#2017-02-0915:09ericnormandit seems that it does#2017-02-0915:10joshjonesno, it does not, only keys. however, when a map itself is validated or checked, the check is done to ensure that the keys specs are present, and that the values that those key specs specify are correct#2017-02-0915:10ericnormandok, that's what I'm saying#2017-02-0915:10ericnormandthe first sentence says it doesn't specify the value#2017-02-0915:10joshjonesit doesn't#2017-02-0915:10ericnormandit's a direct contradiction#2017-02-0915:11ericnormandit does#2017-02-0915:11ericnormandyou just said it does#2017-02-0915:11joshjonesiā€™ll give an example, just a min#2017-02-0915:11ericnormandhere's my example:#2017-02-0915:11ericnormand
(s/def ::name string?)

(s/def ::person (s/keys :req-un [::name]))

(s/explain ::person {:name 3})
#2017-02-0915:11ericnormandthis fails with this message: In: [:name] val: 3 fails spec: :foo.core/name at: [:name] predicate: string?#2017-02-0915:12ericnormandthe value is clearly expected to be a string#2017-02-0915:12ericnormandperhaps it's just a problem with the wording of that sentence#2017-02-0915:13ericnormand> The map spec never specifies the value spec for the attributes, only what attributes are required or optional.#2017-02-0915:13ericnormandto me, that means that s/keys only deals with existence of keys, not their values#2017-02-0915:15joshjonesyou quoted the guide which says: ā€œThe map spec never specifies the value spec for the attributesā€ ā€” is this the part that you say is confusing?#2017-02-0915:15tbaldridge@ericnormand it means that s/keys never contains specs for the values, aside from the values spec'd by the keys themselves#2017-02-0915:16ericnormand@joshjones yes#2017-02-0915:16ericnormand@tbaldridge I see, I totally misunderstood#2017-02-0915:16joshjones
(s/def ::mymap (s/keys :req [::mykey]))
(s/def ::mykey number?)
::mymap does not specify the value spec for the attribute above. It only specifies which key is required.
#2017-02-0915:16tbaldridgeMy favorite analogy: A "Car" is (s/keys [::wheels ::seats ::engine]), doesn't say what a ::wheel is, it just says "A car contains these things"#2017-02-0915:16ericnormandbut it implicitly says the value has to be a number#2017-02-0915:16joshjones::mykey can change all day long, and the spec for ::mymap doesnā€™t change#2017-02-0915:17joshjonesthe spec for the KEY says the value has to be a number, NOT the spec for the map#2017-02-0915:17ericnormandok, thanks#2017-02-0915:17ericnormandbut I still think it's unclear enough to warrant a rephrasing#2017-02-0915:17tbaldridgeBut if you say "Is this a car?", it has to check to make sure that what you say is a ::wheel is actually a ::wheel#2017-02-0915:17ericnormandif I had trouble with it, others probably will too#2017-02-0915:19tbaldridge@ericnormand isn't that all covered in the spec guide? From the section on entity maps:#2017-02-0915:19tbaldridge"Clojure programs rely heavily on passing around maps of data. A common approach in other libraries is to describe each entity type, combining both the keys it contains and the structure of their values. Rather than define attribute (key+value) specifications in the scope of the entity (the map), specs assign meaning to individual attributes, then collect them into maps using set semantics (on the keys). This approach allows us to start assigning (and sharing) semantics at the attribute level across our libraries and applications."#2017-02-0915:20ericnormandthat combined with the sentence I quoted allows for the interpretation I made#2017-02-0915:20ericnormandthat keys are what matter#2017-02-0915:20ericnormandand that if you want to conform the value yourself, you can#2017-02-0915:21ericnormandplease take my suggestion for what it is#2017-02-0915:21ericnormandI think it's unclear and would help others if it were rephrased#2017-02-0915:21ericnormandI don't want to argue about it šŸ™‚#2017-02-0915:21joshjonesI can understand why itā€™s confusing ā€” (s/valid ::somemapspec {ā€¦}) could only check that the keys are there. then another (s/validextra ::somemapspec {ā€¦}) could validate both that the keys are present, and that the keys themselves are valid. i get it ā€¦ but thatā€™s not how it works#2017-02-0915:22tbaldridgeI don't understand: https://clojure.org/guides/spec already explains this about 5 different ways in the section labeled "Entity Maps". I'm not trying to argue, I'm just not sure how it could be any more explicit.#2017-02-0915:22ericnormandit explains it five different ways, but the first way it is explained can be read the wrong way#2017-02-0915:22ericnormandand it can color how the rest are interpreted#2017-02-0915:23joshjonesiā€™ve not heard anyone ask this question before, but Iā€™m sure some people have wondered. more people are confused about how keys that are in the map are checked, even if theyā€™re not given in :req and :req-un#2017-02-0915:23ericnormandthanks for your help!#2017-02-0915:24tbaldridgeBut I mean....that's the 3rd paragraph in the section. And your initial question about "the map spec never specifies" is tied to a code example as context.#2017-02-0915:29tbaldridgeTo be frank, I think "read from top to bottom taking context into account" is assumed as part of the guide. At least that's my opinion.#2017-02-0915:30mpenetyeah I got surprised by this one once (s/def ::foo string?) (s/valid? (s/keys) {::foo 1}) -> false
#2017-02-0915:31mpenetkinda makes sense after all, but I guess doesn't come to mind at first#2017-02-0915:38tbaldridgeYeah, that one is a bit surprising#2017-02-0915:38tbaldridge(didn't know about it till I read the guide just now)#2017-02-0915:39mpenetit makes namespaced kw very specific to specs#2017-02-0915:39mpenetmaybe too much, but that's a matter of opinion I guess#2017-02-0915:41mpenetone could argue it goes against the "specify what you want in there and ignore the extras" direction spec took#2017-02-0915:43not-raspberryspecifying against extra keys makes it impossible to write backwards-compatible specs#2017-02-0915:44mpenetyeah the argument you'll hear is "create new keys"#2017-02-0915:44mpenetwhich is again, arguably, a bit odd#2017-02-0915:45not-raspberry(actually, future-proof, not backwards-compatible - it forces you to synchronize 2 services when adding an extra key)#2017-02-0915:50mpenet@not-raspberry not sure we're talking about the same thing tho. I was referring to the example I mentioned, not to the fact that s/keys allows extra (unespecified) keys to be valid#2017-02-0915:51not-raspberryI was just guessing the rationale for s/keys behaviour.#2017-02-0915:51mpenetthat is actually a good thing as you mention. the example I mentioned, is, odd.#2017-02-0915:52tbaldridgeIt is an interesting topic. One one hand I do like that any value of my map will be valid after I've run it through conform.#2017-02-0915:52joshjones@mpenet why is your example surprising?#2017-02-0915:52tbaldridgeOn the other hand it is silent behavior (code that is executed not based on any specs I wrote in this context).#2017-02-0915:53mpenetit ran valid? on a key that's not present in (s/keys) (the spec)#2017-02-0915:53mpenetand failed#2017-02-0915:53tbaldridge@joshjones because the overall language around spec has been "specify what you want to check, and allow the rest to be whatever".#2017-02-0915:53mpenetin my book, this is a bit too surprising. Unless you read the doc very carefully you wont know about this behavior#2017-02-0915:54joshjonesbut the spec guide clearly states that all keys in a map will be checked for conformance#2017-02-0915:54mpenetAs I said, it's arguable.#2017-02-0915:54joshjonesitā€™s definitely a ā€œgotchaā€ but itā€™s something that is pretty quickly found, if youā€™re writing specs, IME#2017-02-0915:55mpenetwell the fact that @tbaldridge didn't know about it might be revealing šŸ™‚#2017-02-0915:55tbaldridgeFunny enough I've been writing specs for months, and although I've encountered this, I've never understood it until now.#2017-02-0915:55tbaldridgethat's really my fault though for never reading the guide šŸ™‚#2017-02-0915:56joshjonesthe usual case is, write a key spec, write a map spec that doesnā€™t include that key, and put an invalid value for that key ā€” thatā€™ll uncover the ā€œgotcha"#2017-02-0915:56tbaldridgeFor better or worse, I tend to skim guides. So I like my APIs to be explicit. I can handle (s/keys :req [...] :also-check true), since I can go look up what :also-check does. It's silent behavior that can become a "gotcha"#2017-02-0915:57mpenetI never encountered it personally, we have probably thousand of lines of spec and it's 99% un-* keys#2017-02-0915:57joshjonesi do agree that the behavior is not the most intuitive ā€” but because iā€™m dense, i usually read the guides very thoroughly šŸ˜‰#2017-02-0915:57mpenetI think I saw it mentioned here#2017-02-0915:57tbaldridgealthough all of this is also in the doc string, so double bad on me for not reading that either šŸ™‚#2017-02-0915:57joshjoneswell, normally when you write specs for regular situations, youā€™re not going to put a key in the map that is not specā€™d in the map spec. but when youā€™re learning and ā€œtinkering aroundā€, trying to figure things out, itā€™s pretty normal thing to do imo#2017-02-0915:59mpenetit makes kinda useless (other than for doc) :opt for instance#2017-02-0915:59mpenetI think that's actually what the doc says#2017-02-0915:59joshjonesyes šŸ™‚#2017-02-0915:59mpenetbut odd nonetheless#2017-02-0915:59mpenetah, and gen#2017-02-0915:59mpenetso not 100% useless#2017-02-0916:03mpenetIt has one big drawback tho, if you have a "giant map" and need to only validate 1 key it will still try to do its thing on all the others#2017-02-0916:03mpenetthis won't show on synthetic benchmarks like the one that's on github, but i am not sure the other frameworks do this#2017-02-0916:04mpenet(pretty sure they don't actually)#2017-02-0916:19tbaldridge:opt is also useful for gen, as I don't think s/keys gens other keys, only :req and :opt#2017-02-0916:21mpenetyep#2017-02-0919:21bbloomJust a random note on the open-maps discussion: I discovered that TypeScript allows open js objects but has an interesting feature called ā€œexcess property checksā€ - essentially, it only disallows extra keys at compile-time for literal values. This way it catches typos etc, but doesnā€™t prevent future growth. A pretty neat solution, I think.#2017-02-0921:13onetomWe just had an awesome introduction to spec at the Clojure Remote 2017 conference! Adoption will increase for sure šŸ™‚#2017-02-0921:54Alex Miller (Clojure team)@ericnormand just skimming the backchat, is there something that needs to be clarified in the guide? if so, let me know or file an issue at https://github.com/clojure/clojure-site/issues#2017-02-0922:03Alex Miller (Clojure team)issue welcome even if not sure#2017-02-0922:21ericnormandthanks, alex!#2017-02-1000:13bsimacan I use a multispec on a vector of tags, instead of just a single keyword?#2017-02-1000:13bsimahow would I write such a defmulti?#2017-02-1000:28bsima
(defmulti event-type
  (fn [m] (->> m :event/type (into #{}))))
#2017-02-1000:28bsimahere's what I came up with#2017-02-1000:29bsimaturns the vector of tags into a set, so the multispec can use the set as a predicate function for the multimethod dispatch#2017-02-1000:31bsimathis way I can combine multiple defmethods. If we take the example of :event/type from the official clojure spec guide, then I could do:
{:event/type [:event/search :event/error]
 :event/timestamp 1463970123000
 :error/message "Invalid host"
 :error/code 500
 :search/url ""}
#2017-02-1000:32bsimaand so both :event/search and :event/error would be valid specs#2017-02-1001:28Alex Miller (Clojure team)Yeah, there are no constraints on multi-spec as long as you follow the pattern#2017-02-1002:05dragoncubeIn docs for s/keys there is ā€œNote: there is no support for inline value specification, by design.ā€. Does some expanded explanation of this exist somethere?#2017-02-1002:41dragoncubeis it possible to specify spec for value referred by specific key in the map? I.e I want to say that {:id ā€œmy-idā€} ā€œmy-idā€ should be validated by spec ::resource-id#2017-02-1002:43dragoncubecurrently if you use s/keys there is only implicit assignment of the spec in case the keys are same#2017-02-1002:53Alex Miller (Clojure team)@dragoncube there is some additional info about that at http://clojure.org/about/spec#2017-02-1002:54Alex Miller (Clojure team)re rationale that is#2017-02-1002:54Alex Miller (Clojure team)re the second question, the key must match the spec name#2017-02-1002:54Alex Miller (Clojure team)the idea here (also in the rationale) is that we are assigning enduring semantics to an attribute that can be used throughout the system#2017-02-1002:55Alex Miller (Clojure team)it is admittedly a different philosophy than something like Schema#2017-02-1002:57dragoncubeyeah, but it is quite hard to work with the external data which shape you donā€™t control#2017-02-1002:58dragoncubefor example, {:id ā€œres-1ā€ :name ā€œMy resource 1ā€ :sub {:id ā€œsub-res-1ā€ ā€œSub-resource 1"}}#2017-02-1003:00dragoncubeshould I use synthesized keys for registering specs or I need to mimic data structure with packages structures?#2017-02-1003:01dragoncubeIt is so tempting to use :: for specs registration#2017-02-1003:04dragoncubefor example above I would like to register two specs: ::resource-id and ::sub-resource-id in the same namespace#2017-02-1003:04Alex Miller (Clojure team)well you can certainly do that along with s/keys and :req-un which will only match the name part, not the namespace part#2017-02-1003:08dragoncubeyes, it is possible but in this case you need either give it synthesized (not existing) namespace part like :rest-resources/id and :rest-resources.subresource/id or literally create ā€˜subresourceā€™ package and define ::id spec there#2017-02-1003:10dragoncubeboth of these are far from ideal#2017-02-1013:27luxbockhaving s/vcat would definitely be quite helpful#2017-02-1013:29tcouplandespecially if you could unform back to a vector#2017-02-1013:47Alex Miller (Clojure team)Agreed :)#2017-02-1013:47Alex Miller (Clojure team)Rich is as well I think but hasn't decided how it should be implemented yet #2017-02-1013:57tcouplandhow do people feel about something like this:
(defn coerce
  [spec data]
  (let [conformed (s/conform spec data)]
     (if (= :clojure.spec/invalid conformed)
       conformed
       (s/unform spec conformed))))
I've been avoiding conform == coerce, after getting bitten, but this has got me thinking of dabbling again.
#2017-02-1015:19luxbockI wish the :gen overrides of stest/instrument behaved the same way as the :gen of stest/check#2017-02-1015:20luxbockright now you can only use it for functions which have been stubbed#2017-02-1017:03msshey all, quick q about how instrument and check work in cljs. Iā€™m running a figwheel repl in cursive. the repl is choking and timing out when I eval an instrument or check call for an fdefā€˜ed function . further evaluations ā€“ even of simple forms (e.g. (+ 1 1) ā€“ then result in an eval timeout. posted in the figwheel channel already but feel like I might just be misunderstanding a core spec concept and how Iā€™m supposed to use it#2017-02-1018:19Alex Miller (Clojure team)that sounds weird#2017-02-1018:20Alex Miller (Clojure team)instrument should not take a long time to execute#2017-02-1018:20Alex Miller (Clojure team)check certainly can take a while as it runs 1000 tests by default - you can change that by passing it additional options. although most often adjusting your generators is what really needs to happen.#2017-02-1018:33mssyeah it feels like I must be doing something wrong. fwiw my spec is just a set of a few dozen string values (i.e. (cljs.spec/def ::my-spec #{ā€somethingā€ ā€œsomething-elseā€ ā€¦}). my fdef looks something like
(cljs.spec/fdef my-bool-returning-func
  :args (s/cat :my-spec-val ::my-spec)
  :ret boolean?)
simply calling "cljs.spec.test/instrument `my-bool-returning-funcā€ or ā€œcljs.spec.test/check `my-bool-returning-funcā€ causes the same choking behavior
#2017-02-1018:33mssdonā€™t know if thatā€™s idiomatic/correct or not#2017-02-1021:42assoc-inI am looking at using spec for validation for a SPA app. I am hoping that I can use the same specs for both the front and backend. I am guessing that I would place them in a cljc file, but I was wondering how the s/def would work with s referring to clojure.spec and cljs.spec. Any ideas?#2017-02-1021:53bbloomwhatā€™s the preferred idiom to ā€œoptionalizeā€ some map with :req keys? specifically, something like (s/merge ::foo (s/or ::bar (s/keys)))#2017-02-1021:54bbloomthat is, defining a new spec that is a ::foo and optionally also a ::bar#2017-02-1022:05Alex Miller (Clojure team)@assoc-in the new cljs feature for automatic aliasing means that I think you can just use the clojure namespace in the cljc file and it will work for both#2017-02-1022:06Alex Miller (Clojure team)but you should check with David in #clojurescript#2017-02-1022:07Alex Miller (Clojure team)@bbloom not sure I understand what youā€™re asking?#2017-02-1022:09bbloom@alexmiller to make a simplification of what iā€™m doing concrete: imagine i have some spec (s/def ::syntax (s/keys :req [::line ::column])) and i have some other AST nodes that might or might not have line/column information on it: (s/def ::ast (s/keys :req [::type ::op ::whatever] :opt [::line ::column]))#2017-02-1022:09bbloomhowever, i have itā€™s not just two fields, itā€™s quite a few and i want to basically say ā€œan ast node is these keys and optionally a ::syntax"#2017-02-1022:09bbloomis that clearer?#2017-02-1022:10Alex Miller (Clojure team)you could define ::syntax with :opt instead and just s/merge?#2017-02-1022:10bbloomideally not, since the parser requires everything be a ::syntax with proper line and column info#2017-02-1022:10bbloomright now iā€™m doing s/or with an empty (s/keys)#2017-02-1022:10Alex Miller (Clojure team)then why did you say optionally above?#2017-02-1022:11Alex Miller (Clojure team)in that case, just merge ::syntax and ::ast ?#2017-02-1022:11bbloomthereā€™s two parts of the system#2017-02-1022:11assoc-in@alexmiller okay thanks I'll ask#2017-02-1022:11bbloomthe parser requires all the output have ::syntax, and the analyzer allows it#2017-02-1022:11bbloomie not every node that comes out of the analyzer has line/col info on it - but in the parser, it is required#2017-02-1022:12bbloomi donā€™t really want :opt tho, since if there is a ::line, i expect there to also be a ::column#2017-02-1022:12bbloomthey arenā€™t individually optional#2017-02-1022:12bbloomhence (s/def ::ast (s/merge (s/keys :req [.....]) (s/or ::syntax (s/keys)))#2017-02-1022:12Alex Miller (Clojure team)why do you need to do anything in the analyzer? the s/keys you already have in ::ast will validate those#2017-02-1022:13bbloomb/c it only analyzes them independently#2017-02-1022:13bbloomso if i do (assoc foo ::line 1) that should be a problem b/c thereā€™s no column#2017-02-1022:13bbloomthe s/or doesnā€™t really solve that problem tho#2017-02-1022:14bbloomthe s/or does help generation tho#2017-02-1022:15Alex Miller (Clojure team)I donā€™t think there is a way to do it in keys, so youā€™d need an extra constraint#2017-02-1022:16bbloomi figured as much šŸ™‚ was wondering if there was a particular pattern that has been successful for this sort of thing#2017-02-1022:16Alex Miller (Clojure team)donā€™t think Iā€™ve seen it#2017-02-1022:17Alex Miller (Clojure team)I think I would leave the :opt in ::ast to get gen and and that with a custom predicate#2017-02-1022:18Alex Miller (Clojure team)the pred should filter any generated maps that donā€™t satisfy the pred#2017-02-1022:18Alex Miller (Clojure team)in gen that is#2017-02-1022:18bbloomhm ok#2017-02-1022:19bbloomit feels like a symmetric operation to s/?#2017-02-1022:19bbloomor maybe s/nilable#2017-02-1022:19bbloomoooh#2017-02-1022:19bbloomwould that work? (s/merge (s/keys ā€¦) (s/nilable ::syntax))#2017-02-1022:21bbloomah, sadly not#2017-02-1022:21bbloomwell waitā€¦ hmmm maybe it sorta does? just doesnā€™t seem to do what iā€™d want in gen#2017-02-1022:23assoc-in@alexmiller Thanks I asked David and he said I can just use clojure.spec everywhere in the cljc file and I'll be good to go. I am really happy to see that spec's will be usable across both clojurescript and clojure so easily#2017-02-1022:26bbloom@alexmiller hereā€™s what i want to work šŸ™‚ https://gist.github.com/brandonbloom/5d89fe87cb6fe34844423f9e88939085#2017-02-1022:27bbloomthatā€™s a pretty minimal repo of a shortcoming of generate on s/merge with s/nilable#2017-02-1112:41trissok this is a long shot I think but just double checking...#2017-02-1112:41trissso iā€™ve got a vector of vectors#2017-02-1112:42trissand i want to ensure that the integers within it are no greater than the number of vectors in the outer one#2017-02-1112:43trisscan i say this with spec?#2017-02-1112:44triss
(s/coll-of (s/coll-of (s/and int? #(>= % 0))
                        :kind vector?
                        :count 8)
             :kind vector?
             :min-count 2)
#2017-02-1112:46trissis there a way of referencing the size of the outer vector here?#2017-02-1113:05dergutemoritz@triss Only by wrapping the check around the outer s/coll-of with s/and AFAICT#2017-02-1113:05triss@dergutemoritz thanks man. just realized this!#2017-02-1113:06dergutemoritzYW!#2017-02-1118:07Yehonathan SharvitWhat is the proper way to integrate spec.test/check with clojure.test?#2017-02-1205:23zaneSee the pinned items in this channel.#2017-02-1208:23Yehonathan SharvitItā€™s not clear from the guide how to use spec inside deftest#2017-02-1208:22nblumoeDoes anyone have experience with using clojure.spec in a scenario where numerical precision becomes an issue? I would like to have generative testings for a function whose output would match the input (except for some numerical imprecision due to matrix calculations happening). I struggle to come up with a good :fn spec which does not require crippling the generator. A simple :fn #(m/equals (:ret %) (-> % :args :matrix) 1e-6) does not work, because the generator ends up using ā€œlargeā€ doubles which break the max. diff of 1e-6. I tried a spec using ratios but this ends up with ā€œCouldnā€™t satify such-thatā€ issues:
#(-> (m/div (:ret %) (-> % :args :matrix))
       (m/sub 1.0)
       m/abs
       m/maximum
       (< 1e-8))
#2017-02-1208:23nblumoeI am pretty sure I could constrain the generator in a way to make the simple check work but of course I would like to avoid doing that.#2017-02-1208:23nblumoeideas?#2017-02-1213:58gfredericks@nblumoe what does the function you're trying to spec do?#2017-02-1215:25nblumoe@gfredericks a simplified answer would be: Principal component analysis and then reconstruction of the original data from scores and loadings. Some special twists due to the specific domain of the problem though (chemistry).#2017-02-1215:35gfredericks@nblumoe so your problem is that the algorithm breaks down at extreme values, but it would be arbitrary and artificial to impose constraints on the values in the spec?#2017-02-1215:36nblumoeno, the algorithm works fine. the only issue is specing it reliably without making the generator too specific (with ā€œspecingā€ I mean including generative testing)#2017-02-1215:36nblumoe(well, if by ā€œalgorithmā€ you mean the :fn spec predicate then YES šŸ˜‰ )#2017-02-1215:37gfredericksokay, so it's not that the algorithm doesn't work at extreme values, just that the arbitrary equality threshold of 1e-6 breaks?#2017-02-1215:38nblumoeyes exactly. because this criterion is easy to break for example when using doubles in the range of 10e100 or whatever large enough#2017-02-1215:38gfrederickscan you make the equality threshold a function of the input data?#2017-02-1215:38gfredericksthat might have the advantage of giving you a tighter threshold at the other end as well#2017-02-1215:39nblumoeI was trying that too but ended up with unsatisfied ā€œsuch-thatā€ issues. would also need to set the threshold element wise (e.g. if a matrix contains 1e100 as well as 1e1 those should have different thresholds then)#2017-02-1215:40gfrederickswhat does such-that have to do with it?#2017-02-1215:40nblumoeusing ratios of the inputs and returns was how I tried to circumvent the issues....#2017-02-1215:41nblumoequote from above: ā€œI tried a spec using ratios but this ends up with ā€œCouldnā€™t satify such-thatā€ issues"#2017-02-1215:41gfredericksasserting the ratio should be close to 1?#2017-02-1215:41gfredericksare you using such-that directly, or is it generated by spec because of a spec/and?#2017-02-1215:42nblumoethe such-that issue basically means that the generator was not able to find valid data within 100 tries, which can happen when the spec is quite specific and the search space for data is rather large#2017-02-1215:42nblumoe(lol pun not intended)#2017-02-1215:43nblumoeindeed I am using s/and#2017-02-1215:43gfredericksI'm just not seeing why the "using ratios" idea entails modifying the input spec; I'd imagine it just means modifying the :fn on the spec#2017-02-1215:43gfredericksā‡‘ edited#2017-02-1215:44nblumoeyeah I was also surprised that changes to :fn could result in those issues#2017-02-1215:44gfredericksI'm 80% sure changes to :fn should be independent of the probability of such-that errors; what does your s/and look like?#2017-02-1215:45nblumoeI would have that that only :args could have such effect. is anything generated for the :fn itself maybe?!?#2017-02-1215:45nblumoe:args is a bit complicated tbh:
:args (s/and (s/cat :matrix (s/and ::matrix
                                     ::range-transformable
                                     ::non-zero-row-sums
                                     ::non-zero-length-rows
                                     #(s/valid? ::non-zero-length-rows (range-transformation %)))
                      :const-row-sum (s/int-in 1 100)

                      :num-eigenvectors (s/with-gen pos-int?
                                          #(s/gen (set (range 0 20)))))
                                        ; upper limit of eigenvectors
                                        ; should not be hardcoded
               #(= (:num-eigenvectors %)
                   (min (count (first (:matrix %)))
                        (count (:matrix %))))
               #(s/valid? ::non-zero-row-sums (-> (:matrix %)
                                                  range-transformation
                                                  (initial-loadings (:num-eigenvectors %))))) 
#2017-02-1215:47gfredericksthe such-that errors might be nondeterministic -- are you sure you don't get them with the non-ratio code?#2017-02-1215:47nblumoeusually that does not cause any such-that issues on generation, works pretty flawlessly accept for the aforementioned changes to :fn#2017-02-1215:47gfredericksmaybe your test fails before it's likely to encounter them?#2017-02-1215:47nblumoecan test againā€¦ how many tries should I give it? šŸ˜‰#2017-02-1215:47gfredericksI'm not sure what you're asking#2017-02-1215:48nblumoesry, edited#2017-02-1215:48gfredericksI think just try s/exercise on the args spec#2017-02-1215:50gfredericks(s/exercise (s/and (s/cat ...) ...) 10000)#2017-02-1215:50gfredericksprobably wrap that in (count ...) šŸ™‚#2017-02-1215:50gfredericksto avoid printing big things#2017-02-1215:56nblumoeindeed! it occasionally fails, so the issue is rather the :args spec and the generator....#2017-02-1216:02gfredericksit could be either of the s/ands, or both#2017-02-1216:03gfredericksthe way s/and works as a generator is to generate something from the first spec and then filter on the rest of them; so the generator of the first spec needs to be highly likely to generate something that passes all of the predicates, or you get the such-that error#2017-02-1216:04gfrederickssometimes you can simply restructure the s/and to tailor to that, and it works out; other times the only way to get a good generator is to write a custom one for the s/and#2017-02-1216:04gfrederickssince you have nested s/ands it's not obvious which one is the problem#2017-02-1216:04gfredericksI'd try exercising the inner one and see what you get#2017-02-1216:06nblumoeok thanks, that is valuable information! I already have a custom generator for ::matrix but that is not really tailored in any way to satisfy the following predicates. will do some exercises šŸ™‚#2017-02-1216:20nblumoegreat! the outer s/and was the issue: the second predicate for :num-eigenvectors was apparently failing too often#2017-02-1216:21nblumoethanks a lot @gfredericks !#2017-02-1216:25gfredericksnp#2017-02-1216:25nblumoethat specific predicate also does only cover one case where the number of eigenvector matches either the number of columns or number of rows. that is only one specific case though (in which the input data will be reproduced). I need to think about how to cover the usage with less eigenvectors than thatā€¦ the invariant for the generative testing seems to be quite tricky though then#2017-02-1308:28dbushenkohi all!#2017-02-1308:28dbushenkohow to test side-effected functions with clojure.spec?#2017-02-1308:55Oliver GeorgeThat complicates things but there are some features you might like to check out. Pretty sure you can stub out functions. That might let you isolate you're logic for testing. #2017-02-1308:55Oliver GeorgeI haven't used that feature myself. #2017-02-1310:52not-raspberry@dbushenko Attach the spec as a validator to the var/ref/agent/atom, maybe... https://clojuredocs.org/clojure.core/set-validator!#2017-02-1311:22dbushenkothanks!#2017-02-1312:44joshjones@dbushenko @olivergeorge regarding stubbing a function ā€” after fdefing the function, use
(stest/instrument `your-func {:stub #{`your-func}})
to use the :ret in the fdef instead of calling the real function, fyi
#2017-02-1313:25cgrandevery doesnā€™t use res on its element predicate argument:
=> (s/form (s/every string?))
(clojure.spec/every
 string?
 :clojure.spec/kind-form
 nil
 :clojure.spec/cpred
 #object[user$eval8666$fn__8668 0x5ba5f0bd ā€œ
Iā€™ve found http://dev.clojure.org/jira/browse/CLJ-2035 which covers that but with a larger scope. @alexmiller Would it be sensible to open a ticket just for the resolution of the predicate?
#2017-02-1313:33Alex Miller (Clojure team)No, please add to 2035 if it's not already covered by the patch there#2017-02-1313:34Alex Miller (Clojure team)I think there may actually be another ticket for this already though#2017-02-1313:36Alex Miller (Clojure team)Nah, I was thinking of 2059 which is in explain-data#2017-02-1313:38cgrandok thanks#2017-02-1314:50mssare js objects spec-able without converting to cljs data structures?#2017-02-1315:24gfredericksyou can always write arbitrary predicates that check whatever you want#2017-02-1315:30sveriHi, I have a list of maps and each map has an :idx key which is an int. Now what I want is that given one list every :idx is a unique int. Is there a way to enforce that?#2017-02-1315:36frank(= (count my-list) (-> (map :idx my-list) distinct count))#2017-02-1315:37sveriCool, even exercise respects that function, thank you @frank#2017-02-1315:39franknp!#2017-02-1316:46Alex Miller (Clojure team) @sveri the collection spec have a :distinct option#2017-02-1316:46Alex Miller (Clojure team)Although I guess that goes farther than you want#2017-02-1316:56piotr-yuxuanHi here! Letā€™s say Iā€™ve got a map of user. The key ::password must be present if and only if a user is new (the key ::new? is bound to a truthy value). Is it possible to express this with clojure.spec?#2017-02-1317:00sveri@alexmiller I have seen that, but I suspect it means distinct for the whole map and not only for the :idx key#2017-02-1317:04Alex Miller (Clojure team)Yep#2017-02-1317:04Alex Miller (Clojure team)@piotr2b: sure, use a custom predicate#2017-02-1319:41Oliver George@mss if you can convert to clojure data structures first then you'll get access to more spec features. In particular s/keys & all the standard generators #2017-02-1323:42gfredericks@sveri test.check has distinct-by generators for that sort of thing; I don't know if spec exposes it in the collection specs or not#2017-02-1409:42dbushenko@joshjones: thanks for your answer!#2017-02-1410:06kirill.salykinHi guys, I am new to clojure spec, It looks like it can be used for data validation and coercion. eg checking if if username and password provided, or if phone number in correct format. But I donā€™t know how I can show proper error message for failed spec? I was not able to attach any meta data to specs. Any advice on this? Maybe someone got similar ideas? Thanks!#2017-02-1410:16tcoupland@kirill.salykin try explain, explain-str or explain-data#2017-02-1410:50kirill.salykin@tcoupland contains failed predicate, but what I want is to have a custom message in explain-data something like, (s/def ::year integer?, :message ā€œwrong formatā€) I tried to attach it as meta-data - but no luck#2017-02-1410:57Oliver GeorgeI don't think that's possible. #2017-02-1410:58Oliver GeorgeWould a lib focused on user input validation be more appropriate?#2017-02-1410:58Oliver George(Vs data structure validation)#2017-02-1411:00kirill.salykinBut clojure.spec looked so good to describe form objectsā€¦ I guess Iā€™ll find some validation library, thanks#2017-02-1411:02Oliver GeorgeI agree it's great for that. #2017-02-1411:04sveri@kirill.salykin I was at a local clojure meetup lately and @akiel presented this library: https://github.com/alexanderkiel/material-comp I totally liked the approach and think this is the direction to go. It basically normalizes a spec and returns a generic error message for it. Everything is based on explain-data.#2017-02-1411:05sveriThere is not much in it right now, but it should get you started.#2017-02-1411:05kirill.salykinthanks a lot!#2017-02-1411:33akiel@kirill.salykin I would be happy to hear what you think about it. Iā€™ll certainly separate the validation aspect from the material-ui lib. There are also some slides from the meetup: http://www.slideshare.net/alexanderkiel/form-validation-with-clojure-spec#2017-02-1414:32witekWhat is the right way to provide a predicate which is always true? I want a spec which is a tuple of a keyword and anything. How to spec the anything?#2017-02-1414:33thhellerany? was added for this purpose#2017-02-1414:58witekAnd how to spec a tuple with an optional element?#2017-02-1414:59witek[:a] or [:b :c] should be valid. Is this possible with a simple tuple? Or do I need an or?#2017-02-1415:09joshjones@witek one way is:
(s/def ::a-or-bc (s/or :a (s/tuple #{:a})
                       :bc (s/tuple #{:b} #{:c})))
#2017-02-1415:09joshjonesdepending on whether it's nested, you can also use cat instead of tuple#2017-02-1417:54pesterhazya wiki of "how do I spec X?" snippets would be useful#2017-02-1418:40tbaldridgeoooh...something like that talk at the last Clojure/Conj where you write example data and the computer starts generating specs until it finds one that matches all your example input?#2017-02-1419:07richiardiandreaIf I have:
(s/fdef resource->relationship
  :args (s/cat :from-fn (s/fspec :args (s/cat :resource :resource/entity)
                                 :ret qualified-keyword?))
  :ret :relationship/entity)
and my from-fn, which is :name runs fine: (:name {:name "item-definition-components", :description "The list of item definition components.", :uri "{item-definition}/components", :list-of "item-definition-component", :family-id :itemdefinitions, :id :itemdefinitions/item-definition-components}) Why is that if I instrument I get:
Call to #'rest-resources-viz.extractor/resource->relationship did not conform to spec:
                            In: [0] val: "" fails at: [:args :from-fn :ret] predicate: qualified-keyword?
                            :clojure.spec/args  (:name {:name "item-definition-components", :description "The list of item definition components.", :uri "{item-definition}/components", :list-of "item-definition-component", :family-id :itemdefinitions, :id :itemdefinitions/item-definition-components})
                            :clojure.spec/failure  :instrument
#2017-02-1419:08richiardiandreait looks like it runs it in a different way when instrumented#2017-02-1419:10richiardiandreaI am now doubting, is instrument good for use at the repl?#2017-02-1419:38Alex Miller (Clojure team)instrumentation of an fdef with an fspec arg will check that arg by invoking the function you pass it with args generated from the fspec :args#2017-02-1419:39Alex Miller (Clojure team)here itā€™s invoking the function you passed to resource->relationship with a generated :resource/entity and getting back empty string rather than a qualified keyword#2017-02-1419:40Alex Miller (Clojure team)instrument is primarily designed for use at the repl (or during tests), so it should be good for that#2017-02-1419:42Alex Miller (Clojure team)if from-fn is a keyword, then it will return a string based on the example you have above, not a keyword#2017-02-1419:42Alex Miller (Clojure team)looks like a bug in either your code or your spec to me#2017-02-1419:45richiardiandreano well, I forgot to include the code of the function, but it "keywordizes" the result#2017-02-1419:45richiardiandreain fact, when I launch it with instrument the result is the above, but without it it runs fine, same example...of course I need a repro case šŸ˜„#2017-02-1419:45richiardiandreathanks for confirming though#2017-02-1419:46hiredmanwell, (qualified-keyword? (keyword "")) is going to be false#2017-02-1419:50hiredmanthe input you pass in doesn't matter#2017-02-1419:50hiredmanthat error is from the generated data being used to test the spec#2017-02-1419:50richiardiandreaoh really#2017-02-1419:50richiardiandreaok so I was completely missing that point#2017-02-1419:50hiredmanyep#2017-02-1419:50richiardiandreaoh wow that explains everything#2017-02-1419:51hiredmanyep#2017-02-1419:51richiardiandrealol thanks...weirdly enough, printlns in resource->relationship where not printing as well#2017-02-1419:52hiredmanI would have to look at the source for instrument, because if a spec for a function argument fails, it never actually runs the function#2017-02-1419:52richiardiandreayeah it exactly looks like it#2017-02-1419:53richiardiandreathanks (hired) man! You saved my chickens šŸ˜ø#2017-02-1501:34seantempestaWhatā€™s the best practice for validating data submitted to a server with spec (for an API)? Should I use just check with s/valid? and use code to return an error? Or maybe use an assert and throw an exception?#2017-02-1501:52Oliver GeorgeYour first suggestion sounds sensible to me. #2017-02-1501:54Oliver George(In general people try to avoid relying on exceptions in pure functional code since it breaks the input->output contract. )#2017-02-1502:12seantempestaRight. That makes sense. Thanks. #2017-02-1502:15agI need to have either bool or string, why this generates single value wrapped in a vector? (gen/generate (s/gen (s/alt :s string? :b boolean?)))#2017-02-1502:16ageh, nevermind. I forgot that thereā€™s s/or#2017-02-1511:09smoggHowdy. Assuming I'm writing a spec looking like this:
(s/def ::labels (s/coll-of string?))
(s/def ::values (s/coll-of integer?))
(s/def ::chart (s/keys :req-un [::labels ::values]))
how would I go about validating that both :labels and :values are of equal length in :chart?
#2017-02-1511:11dergutemoritz@smogg You could do it like this (s/def ::chart (s/and (s/keys :req-un [::labels ::values]) #(= (count (::labels %)) (count (::values %)))))#2017-02-1511:14smoggok, so a custom fn#2017-02-1511:14smoggthank you @dergutemoritz#2017-02-1514:10cryptoratSo today I am trying to generate a list. (s/def ::team (list? (s/+ ::team-member))) I have verified that ::team-members can be generated. What obvious thing am I missing this morning?#2017-02-1514:17cryptorathmm.. Looking at what smogg wrote (s/def ::team (s/coll-of ::team-member))#2017-02-1514:17cryptoratSeems to work for generating.#2017-02-1514:17smogg@cryptorat yeah, if you want a coll of something you'd use coll-of#2017-02-1514:19cryptoratDoes that mean there is no way to determine if it is a list vs a map?#2017-02-1514:20cryptoratOr is a map not considered a collection? hmmā€¦let me read up on coll-of#2017-02-1514:26smoggif you want a map you can use s/keys to specify the keys#2017-02-1514:26smoggif you want to specify what collection you want to get with s/coll-of you'd pass a :kind#2017-02-1514:27smoggfor example: (s/coll-of integer? :kind vector?)#2017-02-1514:27cryptorat:kind yeah. I just found that.#2017-02-1514:28cryptoratFantastic. It is all coming together nicely. Thank you for the help.#2017-02-1514:28smoggyw#2017-02-1516:01lsnapeInteresting semantic differences between s/merge and clojure core merge:
(s/def :foo/bar string?)
(s/def :baz/bar nat-int?)

(s/def ::bar-map (s/merge (s/keys :req-un [:foo/bar])
                          (s/keys :req-un [:baz/bar])))
I would expect :baz/bar to override :foo/bar, but AFAICT it ands the specs together for that key.
#2017-02-1516:01Yehonathan Sharvits/merge behaves like s/and#2017-02-1516:02Yehonathan SharvitThe term merge might be misleading#2017-02-1516:02Yehonathan Sharvit
merge
macro
Usage: (merge & pred-forms)
Takes map-validating specs (e.g. 'keys' specs) and
returns a spec that returns a conformed map satisfying all of the
specs.  Unlike 'and', merge can generate maps satisfying the
union of the predicates.
#2017-02-1516:04lsnapeThanks! What exactly is meant by union of the predicates here?#2017-02-1516:04Yehonathan SharvitIt means: union of required keys, union of optional keys#2017-02-1516:05Yehonathan SharvitAnd I guess that there is some rule for handling cases where a key appears in one spec as required and in the other as optional#2017-02-1516:05lsnapeGotcha#2017-02-1516:05Yehonathan SharvitCould you test it?#2017-02-1516:05lsnapeyup I reckon#2017-02-1516:06Yehonathan SharvitYou can create an interactive snippet with klipse e.g : http://app.klipse.tech/?cljs_in=(require%20%27%5Bclojure.spec%20%3Aas%20s%5D)%0A%0A(s%2Fdef%20%3Afoo%2Fbar%20string%3F)%0A(s%2Fdef%20%3Abaz%2Fbar%20nat-int%3F)%0A%0A(s%2Fdef%20%3A%3Abar-map%20(s%2Fmerge%20(s%2Fkeys%20%3Areq-un%20%5B%3Afoo%2Fbar%5D)%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20(s%2Fkeys%20%3Areq-un%20%5B%3Abaz%2Fbar%5D)))%0A%0A(s%2Fvalid%3F%20%3A%3Abar-map%20%7B%3Abaz%2Fbar%201%7D)#2017-02-1516:07Yehonathan Sharvit(Sorry for the long url)#2017-02-1516:08joshjonesinteresting, i did not think merge behaved this way because i havenā€™t used it much#2017-02-1516:09lsnapeIt looks like the :req-un wins#2017-02-1516:10joshjones
(s/def :foo/bar #(< 1 % 10))
(s/def :baz/bar #(< 5 % 20))

(s/def ::bar-map (s/merge (s/keys :req-un [:foo/bar])
                          (s/keys :req-un [:baz/bar])))

(s/valid? ::bar-map {:bar 2})
=> false
(s/valid? ::bar-map {:bar 11})
=> false
(s/valid? ::bar-map {:bar 7})
=> true
#2017-02-1516:10joshjoneshad no idea#2017-02-1516:10lsnape
(require '[clojure.spec :as s])

(s/def :quux/flib int?)

(s/def ::bar-map (s/merge (s/keys :req-un [:quux/flib])
                          (s/keys :opt-un [:quux/flib])))

(s/valid? ::bar-map {}) ;; false
#2017-02-1516:10Yehonathan SharvitNo matter whatā€™s the order inside the merge?#2017-02-1516:11lsnapeOrder doesnā€™t matter. I guess this behaviour is consistent with the anding of the value specs.#2017-02-1516:11Yehonathan Sharvitmakes sense
#2017-02-1516:12lsnapeYeah, good to know šŸ™‚#2017-02-1516:14Alex Miller (Clojure team)Merge means: the value must pass each merged spec#2017-02-1516:15Alex Miller (Clojure team)The specs are not combined or unions though#2017-02-1516:18lsnapeWhat I was hoping for was someway to relax an existing spec that Iā€™ve defined. For example, a new map that has blank strings initially ready for a user to fill out.#2017-02-1516:20lsnapeThe spec on the server requiring that these strings be filled out in the map.#2017-02-1517:43vikeriIs there a spec that specifies a sorted-map?#2017-02-1518:37spieden@vikeri not built in i donā€™t think, but you could just have a type checking predicate i suppose#2017-02-1518:39vikeri@spieden Alright thanks, in the end I decided to use a vector instead, better when serializing#2017-02-1518:47spiedenyeah that may make your life easier overall#2017-02-1519:56Alex Miller (Clojure team)@vikeri you can do something like (s/map-of int? string? :kind sorted?)#2017-02-1519:59vikeri@alexmiller That sure looks like it, didnā€™t think of that you could use :kind for broader things than vector?, list? etc.#2017-02-1520:06Alex Miller (Clojure team)itā€™s an arbitrary function#2017-02-1520:06Alex Miller (Clojure team)however, it does affect gen - if you use, either it should gen or you should also use :into#2017-02-1520:09Alex Miller (Clojure team)Iā€™m not sure there is some way to get the sorted map constraint in there and make map-of gen (without supplying a custom gen for the overall coll)#2017-02-1520:12spiedeni seem to be hitting an infinite recursion trying to gen for a part of my spec#2017-02-1520:12spiedenitā€™s not the only recursive definition i have ā€” other one works fine#2017-02-1520:15viestihum, trying to figure out this:
(s/exercise (s/keys :req #{:lol/id} :opt #{:lol/name}))
ExceptionInfo Couldn't satisfy such-that predicate after 100 tries.  clojure.core/ex-info (core.clj:4725)
#2017-02-1520:16hiredmansomewhere in the specs for id and name or in a spec they rely on you are using s/and#2017-02-1520:16hiredmanand somewhere in that s/and is a predicate that is very picky#2017-02-1520:17viestiwhere :lol/id is int? and :lol/name is
(s/and string? #(<= (.length %) column_size))
#2017-02-1520:17hiredmanthe way generates work for s/and is it uses the generator for the first predicate and then filters the output using such-that#2017-02-1520:17hiredmangenerators#2017-02-1520:17hiredmanright, so you should hook up a custom generator#2017-02-1520:18viestihum, s/exercise works individually for both specs but not for the key spec#2017-02-1520:18hiredmansure#2017-02-1520:19viestishould I write a custom generator for the key spec#2017-02-1520:19hiredmanthat is going in to the weeds a little bit on what happens as you compose test.check generators#2017-02-1520:19hiredmanfor :lol/name#2017-02-1520:20viestifirst thought that the problem was strings limited by length, but itā€™s probably combination of :req and :opt, since having both key specs as :opt passes s/exercise#2017-02-1520:20hiredmanbecause you can generate random strings for a long time before you get one that is less than or equal to column_size#2017-02-1520:21viestisince this seems to work:
(s/exercise (s/keys :req #{} :opt #{:lol/name :lol/id}))
([{} {}] [{} {}] [{} {}] ...)
#2017-02-1520:22hiredmanthis is the nature of randomness, sometimes things work that don't always work#2017-02-1520:22viestišŸ˜„#2017-02-1520:22viestihave to take a shower now, introducing some randomness šŸ™‚#2017-02-1520:23hiredmanI don't recall the details of how the generator for s/keys works, but it may ignore :opt most of the time, or some large enough amount of the time for it to mostly work when you move the keys to opt#2017-02-1520:30spiedenany thoughts on infinite recursion during spec generated gen? only difference i can see is the ā€œcall depthā€ where the recursion happens ā€” itā€™s deeper in the case thatā€™s going into the infinite loop#2017-02-1521:26tbaldridge@spieden you have to always give the generator a "way out"#2017-02-1521:26spiedenah like a terminal node in the recursion?#2017-02-1521:27tbaldridgeso one of the s/or or s/keys needs to have a nil, or fully optional params, etc.#2017-02-1521:27tbaldridgeyeah#2017-02-1521:27spiedeni thought maybe thereā€™d be a way to control depth#2017-02-1521:27tbaldridgethere is, but the generator needs something to bottom out at, if you say "go 2 levels deep", what's it going to gen at the bottom level? Has to be a empty map, or a nil, or something.#2017-02-1521:28tbaldridgeor basically a scalar value.#2017-02-1521:28spiedenok thanks, iā€™ll take a look at my specs again#2017-02-1521:28spiedenseems like there ought to be, actually#2017-02-1521:30spiedenyeah thereā€™s an s/or on the path that has both a recursive and non recursive case#2017-02-1521:31tbaldridgeany hint from the stacktrace where the problem is?#2017-02-1521:32spiedenthereā€™s no stack trace, it just spins endlessly and eats CPU when i try to sample one from it#2017-02-1521:33spiedenhereā€™s the code in case you feel like taking a look: https://gist.github.com/spieden/49b6df8c2dae9f8e659edac3b974bb94#2017-02-1521:33spiedenitā€™s ::sgr/parallel thatā€™s the problem while ::sgr/condition is fine#2017-02-1521:36spieden::sgr/parallel -> ::sgr/branches -> ::sgr/states -> ::sgr/state -> ::sgr/parallel is the only cycle i can find#2017-02-1521:37tbaldridgeyeah, I don't think that's the problem#2017-02-1521:37tbaldridgeif you have a loop like taht you'd get a stack overflow#2017-02-1521:37spiedenoh hereā€™s the other spec ns https://gist.github.com/spieden/58f0fac97f725522860f2b68e25f3769#2017-02-1521:37tbaldridgethe problem here is that spec creates the generators very eagerly.#2017-02-1521:37spiedenah ok, SO#2017-02-1521:38tbaldridgeand so I've seen really long times to create generators.#2017-02-1521:39spiedenhmm iā€™ll try giving it longer to see if it ever completes#2017-02-1521:44spiedennope just cooks my cpus#2017-02-1521:46Alex Miller (Clojure team)one thing worth trying is to add :gen-max 3 to all of your coll-of generators#2017-02-1521:46Alex Miller (Clojure team)thatā€™s frequently a problem for me in composite gens as the default is 20 and it doesnā€™t take much nesting for that to be big#2017-02-1521:49spiedenok thanks, iā€™ll try that!#2017-02-1521:50spiedenboom, results#2017-02-1521:51spiedenyeah i guess the branching was just going out of control#2017-02-1521:51spiedenthanks @alexmiller @tbaldridge !#2017-02-1521:51Alex Miller (Clojure team)I have a pending enhancement to change the default to 3 :)#2017-02-1521:54spiedensounds like a sensible default =)#2017-02-1521:58qqqI'm wondering if it's possible to statically analyze, for each non-trminal, the smallest structure it can generate -- then, after gen hits a certain size, it tells everything to use it's smallest choice#2017-02-1521:59qqqthus, instead of controlling depth, gen merrily does its thing until it hits say 1000 nodes, then it goes "oh -%&#$&" and it tries to finish as quickly as possible#2017-02-1522:06aghey guys, I forgot how do you select random multiple elements out of a vector? something like gen/elements but for a random number elements instead of just one#2017-02-1522:07agehm, nevermind, I guess I could combine (gen/sample) and (gen/elements)#2017-02-1522:22agā“ what a spec would look like for a map where values depend on each other? e.g. date range with keys :from and :to where :to always should be bigger that :from?#2017-02-1522:29agis it possible at all?#2017-02-1522:31spieden@ag using a predicate anything is possible!#2017-02-1522:32spiedeni.e. (s/and (s/keys ā€¦) (fn my-pred [the-map] ā€¦))#2017-02-1522:33agoh, rightā€¦ let me try that#2017-02-1522:53agwowā€¦ Spec truly is amazing!#2017-02-1522:57spiedenso satisfying when spec finds allthebugs =)#2017-02-1522:57spiedenfrom hundreds of lines of test code to three#2017-02-1523:06gfredericks@ag don't use sample to build new generators#2017-02-1523:06gfredericks(gen/vector (gen/elements ...)) should do what you were asking#2017-02-1600:45ag@gfredericks Iā€™m using test.chuck, but canā€™t get my head around this. How do I generate two dates - one always smaller than the second?#2017-02-1600:46agIā€™m using test.chuck.generators/datetime#2017-02-1600:49gfredericks@ag fmap+sort, such-that+not=#2017-02-1600:50agare you saying I should generate many, sort and then grab first two?#2017-02-1600:50agmakes sense#2017-02-1600:50gfredericksNo#2017-02-1600:51gfredericksGenerate exactly two using tuple or vector#2017-02-1600:51gfredericksThen fmap and filter that#2017-02-1600:51agfilter? what filter for?#2017-02-1600:53gfredericksThe filter is to make sure they're not equal#2017-02-1600:54gfredericks("filter" meaning such-that)#2017-02-1600:54agoh, rightā€¦ thanks!#2017-02-1603:18Alex Miller (Clojure team)or generate one date and a quantity and add the quantity to the date#2017-02-1607:38viestiFor the record, problem that I had yesterday was actually due to using set (#{}) in :req/:opt for s/keys:
user> (s/exercise (s/keys :req #{:lol/id2} :opt #{:lol/id}))
ExceptionInfo Couldn't satisfy such-that predicate after 100 tries.  clojure.core/ex-info (core.clj:4725)
user> (s/exercise (s/keys :req [:lol/id2] :opt [:lol/id]))
([#:lol{:id2 0} #:lol{:id2 0}] [#:lol{:id2 0} #:lol{:id2 0}]ā€¦
#2017-02-1607:40viestithe s/keys generator also obeys :req and :opt, so generates maps that have only :req keys, which is neat šŸ™‚#2017-02-1607:49viestianother thing, Iā€™m tinkering with generating specs from a source outside code (database schema), and doing something like this:
user> (let [k :lol/id]
        (s/def k int?))
user/k
#2017-02-1607:50viestiwhich is not actually what I wanted, so Iā€™m working around it by doing this:
user> (let [k :lol/id]
        (eval `(s/def ~k int?)))
:lol/id
#2017-02-1607:53viestiWhich seems a bit odd. Am I missing something or is the idea that specs are defined firstly in Clojure code and not derived from outside sources?#2017-02-1611:29Oliver GeorgeI faced similar problem when I wanted to generate specs based on an sql db schema by inspection. In that case I generated clojure code. #2017-02-1611:29Oliver GeorgeThe macro based API complicates things. #2017-02-1611:31Oliver GeorgeIt must be possible to do. I'd love to see examples. #2017-02-1615:34Alex Miller (Clojure team)you can do that, or use eval#2017-02-1615:34Alex Miller (Clojure team)in the future there may be a lower-level function-based api, however it will not be able to automatically capture forms for explain reporting#2017-02-1615:58mpenetlovely idea -> http://dev.clojure.org/jira/browse/CLJ-2112#2017-02-1615:58mpenetthat'd make the life of all the spec to <whatever-format> projects a lot easier#2017-02-1616:05ikitommithumbed that one up, thanks @mpenet. Hacked around that just yesterday to get dynamic conformers ā€œdrop extra keysā€ & ā€œfail on extra keysā€ working for map specs.#2017-02-1616:20Alex Miller (Clojure team)@mpenet has been in the plan from the beginning, but has been blocked behind the various form errors#2017-02-1617:22eraserhdIt would be cool to use s/exercise to generate a sample structure for documentation, but I would need to tweak a few things: Make all keys required, ensure all collections have >= 1 element.#2017-02-1617:22eraserhdI realize this would probably be hacky, but any thoughts on how to do this?#2017-02-1619:28gfredericksI'd probably write a function doc-sample that has the special cases you described, calls itself recursively, and falls back on gen/generate when it doesn't know what to do#2017-02-1623:01calvisIā€™m writing a macro that is essentially a wrapper around defn, so I thought it would be cute to use defnā€˜s spec to conform for easy access to the args & body. it works on some, but not when the body of the macro is a map, because..
user> (s/conform :clojure.core.specs/defn-args
                 '(foo [bar] {:baz 42}))
{:name foo, :bs [:arity-1 {:args {:args [[:sym bar]]}, :prepost {:baz 42}}]}
the body conforms as :prepost rather than :body. is there any way to make it backtrack and choose the path where :prepost is empty and :body is correct?
#2017-02-1623:08gfredericksoh man; does that mean defn is essentially unspecable?#2017-02-1623:08gfredericksI mean I guess you could rewrite it with an or#2017-02-1623:09calviswell I donā€™t think itā€™s un-specable, just that the conform result does not align with the semantics#2017-02-1623:10gfredericksthat's presumably because the spec is ambiguous#2017-02-1623:10gfredericksand I don't think it should be#2017-02-1623:11calviswell yeah, I think the spec as written is a bug / oversight#2017-02-1623:11bronsayeah#2017-02-1623:11bronsai'd open a ticket about that#2017-02-1623:11calvisjust hoping for a way around it#2017-02-1623:11gfrederickschange the spec yourself šŸ™‚#2017-02-1623:11calvismore work than I was hoping to do#2017-02-1623:13gfredericksI think it'd be a pretty simple tweaking of this bit here https://github.com/clojure/clojure/blob/c0326d2386dd1227f35f46f1c75a8f87e2e93076/src/clj/clojure/core/specs.clj#L76#2017-02-1623:14gfredericksyou'd wrap the :prepost and :body in an or or alt or whatever, where the body uses + instead * in the branch with the prepost#2017-02-1623:14calvisyeah I agree#2017-02-1623:18bronsa
(s/def ::args+body
  (s/cat :args ::arg-list
         :rest (s/alt :body+attr (s/cat :prepost (s/? map?)
                                        :body (s/+ any?))
                      :body (s/* any?))))
#2017-02-1623:18bronsathis works#2017-02-1623:18bronsa
user=> (s/conform :clojure.core.specs/defn-args '(foo [bar] {:baz 42}))
{:name foo, :bs [:arity-1 {:args {:args [[:sym bar]]}, :rest [:body+attr {:body [{:baz 42}]}]}]}
user=> (s/conform :clojure.core.specs/defn-args '(foo [bar] {:baz 42} 1))
{:name foo, :bs [:arity-1 {:args {:args [[:sym bar]]}, :rest [:body+attr {:prepost {:baz 42}, :body [1]}]}]}
#2017-02-1623:19gfredericks@bronsa no need for the s/? at that point, right?#2017-02-1623:20bronsawhat about (foo [bar])#2017-02-1623:20gfredericksthat would go through the second branch#2017-02-1623:21bronsaah sorry, I read s/? and thought s/*#2017-02-1623:21bronsayeah no need for that s/? indeed#2017-02-1623:22bronsabetter name for :rest?#2017-02-1623:23bronsa:body-or-body+prepost doesn't look too good :)#2017-02-1623:24gfredericks:body+ :body* :body' dunno too many names#2017-02-1623:24calvisseems like the spec should be named ::args+prepost+body and the :rest is ::prepost+body#2017-02-1623:25calvisfurther broken down into :prepost+body or :body#2017-02-1623:27bronsa:?prepost+body -> :prepost+body/:body#2017-02-1623:27bronsa:P#2017-02-1623:28bronsaI guess an argument could be made that prepost is part of the body#2017-02-1623:28calvisgiven the original name of the spec that seems to be the direction they went#2017-02-1623:29bronsaaight, I'll make a ticket + patch and just keep it as
(s/def ::args+body
  (s/cat :args ::arg-list
         :body (s/alt :prepost+body (s/cat :prepost map?
                                          :body (s/+ any?))
                     :body (s/* any?))))
and rich/alex can figure out some better naming
#2017-02-1623:33calvisthanks!#2017-02-1623:36bronsahttp://dev.clojure.org/jira/browse/CLJ-2114#2017-02-1623:38bronsahmm, maybe prepost should be stricter than just map?#2017-02-1623:39bronsathe actual grammar for defn only accepts a map with :pre [fn*] :post fn#2017-02-1623:40bronsashrug#2017-02-1623:41calvisstill an improvement ĀÆ\(惄)/ĀÆ#2017-02-1623:43gfredericksit does?#2017-02-1623:44bronsawell#2017-02-1623:44bronsait ignores any other key#2017-02-1623:44gfredericksit looks to me like the only requirement is :pre's value is a collection#2017-02-1623:44bronsaan throw at runtime if you don't provide [fn*] for :pre or fn for :post#2017-02-1623:45gfredericksI thought they were arbitrary expressions#2017-02-1623:45bronsaarbitray predicates#2017-02-1623:45bronsai'm using fn to mean IFn instance#2017-02-1623:46bronsawait#2017-02-1623:46bronsano you're right I'm stupid#2017-02-1623:46gfrederickshave you been using pre-post wrong for years and didn't notice because the functions are always truthy šŸ˜„#2017-02-1623:47bronsayes, absolutely#2017-02-1623:47gfredericksdynamic languages are a magic surprise#2017-02-1623:52bronsa@gfredericks the few slack users that use the slack->irc bridge will know what we've been up to#2017-02-1623:52bronsawe need to find them and silence them#2017-02-1623:52gfredericksglad I stopped using that when the threads feature was released#2017-02-1623:52bronsaoh god I don't want to know how that looked#2017-02-1623:53gfredericksI think it just looked like regular messages#2017-02-1623:54gfredericksso I would've ended up unknowingly replying to thread messages in the main channel#2017-02-1623:54bronsalol#2017-02-1623:55gfredericksI can put up with all sorts of nonsense for the sake of technological idealism, but unknowingly looking like a crazy person is not one of them#2017-02-1623:55bronsaone thing I really miss about the #clojure irc: slack makes me feel so much more guilty when I derail a conversation/channel into offtopic banter#2017-02-1623:56gfredericksonly because threads exist? or because topics are more specific?#2017-02-1623:56bronsathe latter I think#2017-02-1700:04gfredericksI miss the jovial bots#2017-02-1700:05bronsaand the internet points#2017-02-1700:05gfredericksoh yeah I had a lot of those I think#2017-02-1700:05gfredericksshould've sold'em before the market crashed#2017-02-1700:06bronsano point in crying over what could've been now#2017-02-1700:06bronsathat was a terrible pun i apologize#2017-02-1700:07gfrederickscan we start threads based on thread messages#2017-02-1700:07bronsaI've frequently complained to my coworkers about the lack of sub threads#2017-02-1700:07gfredericksseems not#2017-02-1700:08gfredericksI miss trying to get the bots to talk to each other#2017-02-1700:09bronsawas it you who came up with that brilliant double quoting thing to bypass their "security" measures?#2017-02-1700:10bronsai remember being very amused#2017-02-1700:10gfredericksI don't think I ever achieved an infinite loop personally#2017-02-1700:12bronsaI might spend the next 3 hours going over old irc logs to find the one I'm thinking about#2017-02-1700:13gfredericksthat would be fun to see; I don't recall anything like that#2017-02-1700:14gfredericksthis feels deceptively like a private conversation#2017-02-1700:16hiredmandeceptive indeed#2017-02-1700:16bronsasomebody's gonna chime in real soo-#2017-02-1700:17gfredericksit lists names at the top, but I doubt that includes lurkers#2017-02-1711:27borkdudeClojure IRC still exists, right? Or is it effectively dead now#2017-02-1711:28bronsait exists but i don't think it's nowhere as active as it used to be#2017-02-1711:29bronsaalso i don't join it anymore so from my perspective it's dead and I'm mourning#2017-02-1713:56shemit's there and still useful and enjoyable#2017-02-1715:27Alex Miller (Clojure team)still in regular use#2017-02-1716:59ggaillardHi everybody ! I'm having a hard time with cljs.spec (1.9.473). I want to instrument my spec-ed functions, but I can't find the instrument and instrument-all macros it seems they have been removedā€¦ do they still exist somewhere ? I can't find any blogpost about it ā€¦#2017-02-1717:03gfredericksdid you check under the .test namespace @ggaillard#2017-02-1717:05ggaillardI'm checking right now#2017-02-1717:08ggaillardGreat ! I had a caching problem with figwheel preventing me to :require cljs.spec.test. Cider was showing me an other file for some reason ā€¦ Thank you @gfredericks !#2017-02-1723:03bnoguchiAnyone here figure out how to constrain a multi-spec generator to generate maps constrained to always generate maps with only one of the tagged values (i.e., to always follow only one multimethod branch during generation)?#2017-02-1723:04bnoguchie.g., in the guide example, only generating events of type :event/search#2017-02-1723:27Alex Miller (Clojure team)You can just invoke the generator for the result of invoking one of the method specs#2017-02-1723:31bnoguchi@alexmiller You mean if mm is the multimethod, then (s/gen (mm {:the/tag tag-val})?#2017-02-1723:32Alex Miller (Clojure team) Yeah, or just invoke the spec directly if you van#2017-02-1723:32bnoguchiWon't that not constrain the tag to the given value?#2017-02-1723:33bnoguchii.e., :the/tag is open ended as keyword?, so will generate infinite sample of possibly non-matching keywords#2017-02-1723:35bnoguchie.g., for :event/type example in the guide, this would effectively be for :event/search:
clojure
(s/gen (s/keys :req [:event/type :event/timestamp :search/url]))
#2017-02-1723:35bnoguchiAh, I guess, I can pass gen overrides for [:event/type] to (s/gen spec overrides)#2017-02-1723:41bnoguchiIt would be cool if multi-spec generators supported overrides, to force a particular generated type, e.g.,:
(s/gen :event/event {[:event/type] #(gen/return :event/search)})
#2017-02-1801:12weiwhat happened to spectrum? is it still being developed?#2017-02-1908:08Yehonathan SharvitIs there a way to instrument by default while in dev environment?#2017-02-1914:17gfredericks@viebel I think you could pull that off with leiningen profiles; not sure how that would interact with code reloading though#2017-02-1915:02nblumoeIs there an established standard to run spec tests (`stest/check`) from a clojure.test suite (e.g. on lein test)?#2017-02-1915:21tbaldridge@nblumoe not a standard, but I would probably reach for test.check's defspec: https://github.com/clojure/test.check#clojuretest-integration#2017-02-1915:21tbaldridgeCall that and use (s/gen ::my-spec) to get the gen to us in prop/for-all#2017-02-1915:25gfredericksIf it's just the builtin spec tests, then just using deftest and wrapping the stest/check call in an assertion might make more sense#2017-02-1915:25gfredericksdefspec would be more useful for custom assertions/tests#2017-02-1915:32nblumoe@gfredericks something like (deftest specs (is (stest/check))) would not be sufficient as it seems, what would you check for?#2017-02-1915:33gfredericksExamine the return value. Is there a result key?#2017-02-1915:34Yehonathan SharvitThis might help#2017-02-1915:35Yehonathan SharvitItā€™s for cljs but not too hard to adapt to clojure#2017-02-1915:38nblumoethx. If I understand correctly all three solutions require looking at the :failure key manually / with a helper. Okay then I will do that too. Was hoping for a generic integration into clojure.test#2017-02-1916:03nblumoeok works from the REPL and via Cider, but not with lein test. Any ideas? Here is the error (I donā€™t get it so far tbh):
ERROR in (foo) (FutureTask.java:122)
foo
expected: (nil? (-> spec-check first :failure))
  actual: java.util.concurrent.ExecutionException: java.lang.ClassCastException: clojure.lang.AFunction$1 cannot be cast to clojure.lang.MultiFn, compiling:(clojure/test/check/clojure_test.cljc:95:1)
 at java.util.concurrent.FutureTask.report (FutureTask.java:122)
    java.util.concurrent.FutureTask.get (FutureTask.java:192)
    clojure.core$deref_future.invokeStatic (core.clj:2290)
    clojure.core$future_call$reify__9377.deref (core.clj:6849)
    clojure.core$deref.invokeStatic (core.clj:2310)
    clojure.core$deref.invoke (core.clj:2296)
    clojure.core$map$fn__6881.invoke (core.clj:2728)
    clojure.lang.LazySeq.sval (LazySeq.java:40)
    clojure.lang.LazySeq.seq (LazySeq.java:56)
    clojure.lang.LazySeq.first (LazySeq.java:71)
    clojure.lang.RT.first (RT.java:682)
    clojure.core$first__6404.invokeStatic (core.clj:55)
    clojure.core/first (core.clj:55)
    geomix.calc.pva_test$check_SINGLEQUOTE_.invokeStatic (pva_test.clj:244)
#2017-02-1916:14nblumoethis happens whenever I call some function with (stest/check my-fn)` as an argument within a deftest body....#2017-02-1917:48nblumoehad to require [clojure.test.check.clojure-test] #2017-02-1918:50Alex Miller (Clojure team)You're running into the lein test monkey patch problem#2017-02-1918:51Alex Miller (Clojure team)There are tickets in both lein and test.check about it, but you can work around it by turning it off in project.clj#2017-02-1918:52Alex Miller (Clojure team)@nblumoe ^^#2017-02-1918:54Yehonathan SharvitWhat is the flag that needs to be set in order to instrument all the namespaces (while in dev environment)?#2017-02-1918:54Alex Miller (Clojure team)There is no such flag#2017-02-1918:54nblumoe@alexmiller I guess that would be monkeypatch-clojure-test https://github.com/technomancy/leiningen/blob/master/sample.project.clj#L351 ?#2017-02-1918:54Alex Miller (Clojure team)@nblumoe yep#2017-02-1918:55nblumoethanks a lot, good to know! :+1:#2017-02-1918:55Alex Miller (Clojure team)@viebel you can call instrument with no args though#2017-02-1918:56Yehonathan SharvitWhere would I put this call?#2017-02-1918:57Yehonathan SharvitIs there a dynamic var that I could set from leiningen like *assert*?#2017-02-1921:03richiardiandrea@viebel I do a classic when at the top of my files to instrument everything, I haven't seen anything fancier #2017-02-1922:00frankyou can set up a :once fixture that calls (s/instrument) before running tests and (s/unstrument) after running tests#2017-02-1922:01frankhttps://clojuredocs.org/clojure.test/use-fixtures#2017-02-2010:06akielI have a question regarding required keysets in combination with om.next or pull queries, where required attributes might be skipped. I have formulated my questions in the following gist: https://gist.github.com/alexanderkiel/81daedb133f7939d0c88ff28dd457442#2017-02-2014:33mike_ananevHi! I have ns with spec's. how to load all spec names from another ns?#2017-02-2014:34mike_ananevI want to get list of def's from particular ns#2017-02-2014:35akiel@mike1452 you can just require the ns with specs#2017-02-2014:35mike_ananevI want to print on screen available specs#2017-02-2014:35mike_ananevnames#2017-02-2014:35mike_ananevin order to do it I need to get list of spec names#2017-02-2014:36akielthere is (s/registry)#2017-02-2014:37mike_ananev@akiel wow! yeah, sure! thanks!!!#2017-02-2014:39akiel@mike1452 no problem šŸ™‚#2017-02-2015:22dacopareI found that some of my spec check were running slowly and causing timeouts. If you find the same thing happening, you can use the options to specify the number of tests you want to run.
(s.t/check `my-ns/my-func {:clojure.spec.test.check/opts {:num-tests 10}})
I hope this helps one of you.
#2017-02-2015:26gfredericksprobably had more to do with the volume of data generated than the number of tests#2017-02-2015:44akiel@dacopare As @gfredericks said already: keep an eye on your generators. You can also overwrite them in tests or control them globally with something like :max-gen in coll-of.#2017-02-2015:51dacopare@gfredericks: yes, the datastructures that are input and output are very large. Limiting the number of tests is the only way I can see to mitigate this.#2017-02-2015:56gfredericks@dacopare there are better ways. The :max-gen option mentioned above might do it, I'm not sure. Otherwise there is gen/scale to manipulate the size parameter that a generator sees#2017-02-2016:48lwhortonheya guys ā€¦ iā€™m curious if thereā€™s an idiom for nil-able keys in (s/keys ā€¦) wrt other namespaces? what I want to do is something like this:
(ns entity-a)
(s/def ::entity-a (s/keys :req [::b]))
(s/def ::b (s/nilable :entity-b/entity-b)) <ā€” probably wont work

ā€¦
(ns entity-b)
(s/def ::entity-b (s/keys .... (more stuff)))
1) seems redundant to have to define :req [::b] , where ::b is defined locally but just points to another ns 2) ::b should be nilable only from aā€™s point of view ā€¦ so I donā€™t want to put nilable down inside the spec of entity-b ā€¦ so should this really just be
(s/def ::entity-a (s/keys :req [::b]))
(s/def ::b (s/or :nil nil :val :entity-b/entity-b)
?
#2017-02-2016:52gfredericks@lwhorton did you consider making the key optional instead of making the value nilable?#2017-02-2016:54lwhortoni did, but in this particular case the domain is really shoddy. i donā€™t have time to go refactoring (yet), so entity-a can have the key, but the value might be nil. I suppose that leads to another question - is it more common to just ignore such cases with an opt key, or to capture all possible nil states?#2017-02-2016:56gfredericksI don't know if there's any sense of what's more common with spec yet#2017-02-2016:56gfredericksone of spec's core ideas is that the meaning of a key is the same everywhere, which is why you're having to define a different key#2017-02-2016:57gfrederickswhich I imagine implies that somewhere you'll have code that translates something to that new key. and if that's the case, then it shouldn't be much more difficult to instead remove the key when the value is nil#2017-02-2016:58lwhortongood point. or we could just have a proper domain model thought out without monkey patching and quick fixes šŸ˜‰. thanks @gfredericks#2017-02-2020:09lvhIf I have a fn with 2 arguments that are related for correct operation (itā€™s a fn-var and an argument name), can I use clojure.spec.test/check to check it, or should I write a more traditional clojure.test.check prop?#2017-02-2020:10gfredericksdepends on if you want to make that relationship part of the arg spec or not#2017-02-2020:10gfredericksvia s/and probably#2017-02-2020:14lvhIā€™m not sure I understand the suggestion; if I do something like (s/and ... is-a-valid-arg-name?) I still need to know the value of the method (and random strings are extremely unlikely to be arg names) ā€” is there something else Iā€™m missing?#2017-02-2020:19gfredericksin that case you'd need to add a custom generator too#2017-02-2020:21zaneThere's no way to use clojure.spec/or or clojure.spec/multi-spec to do what lvh wants here?#2017-02-2020:22lvh@gfredericks just one that applies to the entire arg spec. Makes sense. Thanks šŸ™‚#2017-02-2110:47mike_ananevHi! If I need (gen/sample ::somespec) about 1M or more, how to parallelize this operation? is there build-in tools? or just pmap or something?#2017-02-2113:14gfredericks@mike1452 what do you need them for? #2017-02-2113:43mike_ananev@gfredericks for data sample generation. I want to give samples to other team, which needs shape of our data for dev process.#2017-02-2113:56mpenet@alexmiller what's the extra form parameter in s/valid? ?#2017-02-2113:56mpenetdoesn't seem to be documented#2017-02-2113:57mpenetwell, it's not used actually#2017-02-2114:06Alex Miller (Clojure team)It's an alternate value to return if not valid#2017-02-2114:07Alex Miller (Clojure team)Or maybe I'm reading that wrong#2017-02-2114:07mpenetmaybe I misunderstand something then: https://github.com/clojure/clojure/blob/d920ada9fab7e9b8342d28d8295a600a814c1d8a/src/clj/clojure/spec.clj#L672-L683#2017-02-2114:07Alex Miller (Clojure team)Can't say I've ever used it#2017-02-2114:08mpenetdoesn't seem used at all but in the exception#2017-02-2114:13Alex Miller (Clojure team)thatā€™s not the current version of that code#2017-02-2114:13gfredericks@mike1452 you can parallelize it however you like, that's kind of independent#2017-02-2114:14mpenet@alexmiller it's the one linked from the api docs I think#2017-02-2114:14Alex Miller (Clojure team)@mpenet looks like its used internally via pvalid?#2017-02-2114:14mpenetI ll have a look#2017-02-2114:14Alex Miller (Clojure team)yeah, the autodoc updating was broken by an issue in spec, so they are not the latest#2017-02-2114:14Alex Miller (Clojure team)I have a patch pending to fix the code and get that working again#2017-02-2114:15Alex Miller (Clojure team)that valid? form is used when you have an alternate form to supply for the error reporting#2017-02-2114:16mpenetthat's exactly what I wanted it to be šŸ˜„#2017-02-2114:16mpenetI ll try it out#2017-02-2114:17Alex Miller (Clojure team)I think maybe the only place itā€™s used is from within keys#2017-02-2114:17Alex Miller (Clojure team)used with that extra form that is#2017-02-2114:18mpenetso it's not supposed to be "public" ?#2017-02-2114:18mpenetI mean subject to changes/removal#2017-02-2114:26mpenetnot sure it helps actually#2017-02-2114:26mpenetI am wishing for a explain-info I think. Sometimes I need to add some context to explain data#2017-02-2114:27mpenetI know I can wrap the whole spec and add this at a higher level, but it's a bit gross#2017-02-2114:59Alex Miller (Clojure team)the valid? function should be considered public and I donā€™t think there are any expectation that api will change
#2017-02-2115:00Alex Miller (Clojure team)my impression is that Rich doesnā€™t want to provide extensions to modify explain (other than by writing your own spec and taking care of it there)#2017-02-2115:02mpenetšŸ˜ž care about more detail of my use case for a potential suggestion?#2017-02-2115:06mpenetin short: that spec predicate is quite resource heavy, and upon failure the function wrapped by that predicate provides a lot of metadata about the failure (think a query parser output, with line/col num, explain traces etc). Right now not only I loose all this data but I need to re-trigger a parse to get back the metadata (or throw upstream, bypassing spec, which is ugly). In theory, one could just wrap and report the error manually, but this values are "deep" in a (spec validated) map.#2017-02-2115:07mpenetopen to suggestions on how to better handle this#2017-02-2115:11Alex Miller (Clojure team)sounds like a good use case for writing a custom spec#2017-02-2115:12mpenetyou mean implementing the internal protocol?#2017-02-2115:12Alex Miller (Clojure team)yeah#2017-02-2115:12mpenetis that "safe"?#2017-02-2115:12mpenetI mean allowed#2017-02-2115:13Alex Miller (Clojure team)itā€™s available to do but is subject to change while in alpha#2017-02-2115:14mpenetok, I guess I might just do this. That's how it used to work with Schema btw, seems quite an elegant way to do it#2017-02-2115:14Alex Miller (Clojure team)itā€™s something I would avoid doing as much as possible#2017-02-2115:14mpenetyes, same feeling.#2017-02-2115:56mpenetworks like a charm#2017-02-2116:22carocad@mpenet I was under the impresion that explain-data was meant to provide that functionality: https://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec/explain-data#2017-02-2116:23mpenetno explain-data just returns the data behind a failing spec, doesn't allow you to "customize" said data#2017-02-2116:23mpenetsame diff as ex-info vs ex-data (kinda)#2017-02-2116:27carocadbut isnt the data behind a failing spec the one that your parser already returned? I mean if your parser would be a speced fn then the data that you get already contains that custom information that you want. Or am I missing something?#2017-02-2116:28mpenetno it's the spec form : something like #(try (parse q) (catch ExceptionInfo e ...))#2017-02-2116:29Alex Miller (Clojure team)well, youā€™ll also get the failing value#2017-02-2116:29Alex Miller (Clojure team)but not all the work that went into evaluating the failing value#2017-02-2116:29mpenethmm I don't think so#2017-02-2116:30mpenetvalid will return false, conform ::invalid#2017-02-2116:30mpenetex-data the form#2017-02-2116:30Alex Miller (Clojure team)each problem in explain-data has the value that failed the spec#2017-02-2116:30mpenetyes, the initial value true, but not the "detail" of the failed predicate (in my case an ex-info instance)#2017-02-2116:31Alex Miller (Clojure team)right#2017-02-2116:31Alex Miller (Clojure team)maybe the exception case in particular is interesting#2017-02-2116:32mpeneteven then, that wouldn't make it as easy as it is not with the custom spec, it just feeds that extra data to the explain map#2017-02-2116:32mpenetyes, imho it's something that might make sense in spec itself.#2017-02-2116:32Alex Miller (Clojure team)well, what if when conforms fails with an exception, the ex-info was conveyed#2017-02-2116:32Alex Miller (Clojure team)normally it just returns a truthy/falsey value (and nil/false canā€™t convey additional info)#2017-02-2116:33Alex Miller (Clojure team)but exceptions can#2017-02-2116:33mpenetthat would be nice, that's what I meant with explain-info , it could return that#2017-02-2116:33mpeneteither accept a explain-info return, or the kw#2017-02-2116:34mpenetbut the reify Spec option is actually nicer in my case (for other reasons)#2017-02-2116:34carocadhow about (or (s/explain-data spec x) ::ok )#2017-02-2116:35carocadreturns ::ok when everything when well or the ex-info from your spec if it failed#2017-02-2116:41mpenet@alexmiller using explain-info you would basically write your own conformer( try/catch potential exception in my case) and return the (explain-info {...}) on failure. I am not sure doing by default for any exception (or ex-info instance) is desirable#2017-02-2116:42Alex Miller (Clojure team)perhaps not#2017-02-2119:22tjtoltonIs core.spec able to use something else as keyword qualifiers besides the current ns? As I say that out loud, I realize how silly it would be if that were not the case, but I see a lot of the example code using the double colon keywords, which auto expand to current ns#2017-02-2119:23Alex Miller (Clojure team)any ns is fine#2017-02-2119:23tjtoltonI don't suppose there's some kind of dynamic binding that would change the way the double colon autoexpands?#2017-02-2119:24tjtolton::keyword => :com.mycompany/keyword#2017-02-2119:24tjtoltonetc#2017-02-2119:25Alex Miller (Clojure team)you can use aliases#2017-02-2119:26Alex Miller (Clojure team)
(alias ā€˜a ā€˜com.mycompany)
::a/keyword
#2017-02-2119:26Alex Miller (Clojure team)the tricky part is that right now, com.mycompany must be an actual namespace#2017-02-2119:26tjtoltonšŸ˜®#2017-02-2119:26tjtoltonthat's great!#2017-02-2119:27Alex Miller (Clojure team)you can work around that by doing (create-ns ā€˜com.mycompany)#2017-02-2119:27Alex Miller (Clojure team)that is something weā€™ve talked about changing though#2017-02-2119:28Alex Miller (Clojure team)https://www.deepbluelambda.org/programming/clojure/know-your-keywords covers this#2017-02-2120:38agguys, Iā€™m kinda stuck again. Iā€™ve done it before (I think) now I canā€™t remember the right way of doing that. How do I properly generate things based on vector? So If I have generated vector of maps, and then for each item I need to generate something else#2017-02-2120:40agpassing vector generator with fmap into a function with for kinda works, but I think this isnā€™t right approach, because I need to call gen/generate inside for#2017-02-2120:41agsince it canā€™t return generator#2017-02-2120:42zaneLike this, @ag? https://github.com/clojure/test.check/blob/master/doc/intro.md#bind#2017-02-2120:43zane> For example, say we wanted to generate a vector of keywords, and then choose a random element from it, and return both the vector and the random element.#2017-02-2120:43ag@zane here in fn in just grabs random element from generated vector and thenā€¦#2017-02-2120:44agactually that may work for me#2017-02-2121:38gfredericksRandom element? #2017-02-2121:39gfredericksWhy did you generate a whole vector of them just to pick one out? #2017-02-2122:19Alex Miller (Clojure team)well one reason you might do that is if you need a vector AND an element from that vector#2017-02-2123:27agactually @gfredericks comment in my case make sense.#2017-02-2123:48agok, thereā€™s still something I canā€™t figure out for the second case, where I have a predefined vector of maps and for each item I need to generate (something based of that map) and attach in a key to that map and finally return that modified vector#2017-02-2123:50agmy solution requires to call gen/generate within for, which I donā€™t like.#2017-02-2123:50agI though test.chuckā€™s for would work for me, but I canā€™t get it right#2017-02-2123:54gfredericksCalling gen/generate in general is not a proper way to build a generator#2017-02-2123:55gfredericksAnd it shouldn't ever be necessary#2017-02-2123:56gfredericksIf you want to describe what you're doing I'm happy to help work it out#2017-02-2123:59agso I have list-of-accounts (predefined list) which is a vector of maps, e.g. [{:id 1 :type :customer} {:id 2 :type :investor},,,]. Now, for each item in that vector I need to generate a balance-update data and add it in :bal key and return that modified vector#2017-02-2200:01agletā€™s say balance-update-generator for the sake of exercise generates vector of ints#2017-02-2200:06aghold on. gen-balance-update generates data based on the information of each individual item in the original vector i.e.: (gen-balance-update m)#2017-02-2200:07gfredericksk, modifying...#2017-02-2200:07agit returns a vector, that vector should go to m under the :bal key#2017-02-2200:09agaha!#2017-02-2200:10agI think this is what I need. Iā€™m not sure yet#2017-02-2200:10aglet me try#2017-02-2200:10gfredericksan equivalent approach would be to generate the whole map from gen-balance-update, and then avoid doing the map and assoc on line 15#2017-02-2200:10agGary, you are my hero!#2017-02-2200:25agbtw, has anyone encountered that - very often when I run (gen/generate) in clojurescript it just kills the websocket repl#2017-02-2200:26gfredericksis it possible it's generating something Very Big?#2017-02-2200:31agnot too big thoughā€¦#2017-02-2200:31agstill must be doing something wrong#2017-02-2200:34gfredericksthere are certain combinations of generators that have unfortunate distributions where they are Not Too Big most of the time but occasionally Very Big#2017-02-2200:34gfredericksbut I don't know anything about the cljs repl so I don't know if that's even a reasonable guess#2017-02-2200:40agitā€™s ok, not critical, at least right now for me. I have specs in .cljc files and thatā€™s pretty nice#2017-02-2200:40agI think it should work fine with tests running in headless browser#2017-02-2200:41agbut in general I try to avoid generating large items, clojurescript struggles#2017-02-2200:47gfredericksI would like it to be difficult to accidentally generate Very Large things, but unfortunately I don't think there's a good general solution for the existing pitfalls#2017-02-2207:56mpenet@alexmiller Should I bring up my case earlier in a jira ticket? or is it something that should be discussed with Rich first. Happy to provide a patch if there's interest.#2017-02-2213:33ikitommiStarting to be quite happy with my custom 3-arity conformer: easy to selectively conform on runtime (json, string, fail-on-extra-keys, drop-extra-keys etc). Is this something that could go into clojure.spec itself? ping @alexmiller. If so, should I just write an issue/proposal of this into Jira? And the idea is allow separating ā€œwhatā€ (the spec) from ā€œhowā€ (the conformer). e.g.
(s/def ::name string?)
(s/def ::gender keyword?)

(conform 
  (s/keys :req-un [::name ::gender]) 
  {:name ā€œTommiā€, :gender ā€œmaleā€, :weight 92} 
  (merge json-conformers strip-extra-keys-conformer))
; => {:name ā€œTommiā€, :gender :male}
Requires some spec inspection, but the http://dev.clojure.org/jira/browse/CLJ-2112 should make it hack-free. Spec as Records would help even more here (could just add a custom new Protocol for this).
#2017-02-2213:36mpenetclj-2112 could exist as a library until it's finalized#2017-02-2213:36mpenetas long as the final thing doesn't end up too far from what's in the ticket it'd be a decent solution#2017-02-2213:36mpenetparsing clj forms is a bit gross otherwise imho#2017-02-2213:37mpenet(there's also https://github.com/uswitch/speculate doing that, monkey patching spec along the way to reach its goals)#2017-02-2213:37mpeneta bit too convoluted to my taste#2017-02-2213:38ikitommiwe have similar patching around.#2017-02-2213:38ikitommiwhile waiting for things to resolve in the clojure.spec side.#2017-02-2213:41ikitommiand 2112 could be a separate lib, but I would still need the 3-arity confrom for the specs. currently need to wrap all specs to make it workā€¦ https://github.com/metosin/spec-tools#2017-02-2213:42ikitommi(and the dynamic binding hack)#2017-02-2213:43mpenetthat's the one thing that I don't like so far with spec-tools#2017-02-2213:44ikitommiwell that was my question about getting help from the spec, would like to dissolve the hacks on our side.#2017-02-2213:44mpenetI'd rather have the conformers as an arg at another level (spec maybe).#2017-02-2213:45ikitommime too, the 15:33 comment šŸ™‚#2017-02-2213:45mpenetunderstood#2017-02-2213:47mpenetspeculate goes further in the monkeypatch'iness -> https://github.com/uswitch/speculate/blob/master/src/clojure/spec/override.clj#2017-02-2213:47mpenetšŸ™‚#2017-02-2213:49mpenetthat said I am dying for swagger/spec in compojure-api#2017-02-2213:49Alex Miller (Clojure team)@mpenet ticket would be fine - important thing is a good problem statement#2017-02-2213:50mpenet@alexmiller I'll try#2017-02-2213:50Alex Miller (Clojure team)@ikitommi I don't understand what you're proposing #2017-02-2213:54ikitommia 3-arity conform that would a third argument, a callback which the Specā€™s conform would call with the sepc and the value as arguments, the callback would return either a new conformer function of nil. If it returned, it would be used instead of the installed normal conformer.#2017-02-2213:54Alex Miller (Clojure team)It seems unlikely Rich would want to add that#2017-02-2213:55Alex Miller (Clojure team)But starting with a good problem is the way to win his heart :)#2017-02-2213:55Alex Miller (Clojure team)This is a solution, not a problem#2017-02-2213:56ikitommiIf I just write the problem statement into Jira?#2017-02-2213:58Alex Miller (Clojure team)Yes, that's ok, although I will close it if I don't think it's a problem we are trying to solve#2017-02-2214:03ikitommiok, will do. thanks.#2017-02-2214:23mpenetI hope it's not too opaque: http://dev.clojure.org/jira/browse/CLJ-2115#2017-02-2214:41Alex Miller (Clojure team)Iā€™ll take a look, thx#2017-02-2214:46mpenetthe proposal "only" covers the need to have more info for failures the rest is probably too specific to our stuff (even tho the example mentions it)#2017-02-2214:47tbaldridge@mpenet it's unclear from the ticket, are you parsing strings with spec?#2017-02-2214:48mpenetI want conform/valid to tell me if a spec value is a valid string representation of a "Query"#2017-02-2214:49mpenetso it does parse under the hood, and right now (with our patch) conform returns the parsed AST#2017-02-2214:49mpenetbut this is a detail of our use case I think#2017-02-2214:51mpenetthe important bit is about the need to preserve the metadata from the failure to extend what's returned by explain* later#2017-02-2214:51tbaldridgeYeah, that'd be nice, imo. But it also seems that passing a parse tree into spec would get your around the problem.#2017-02-2214:52tbaldridgetreating Spec like a parser, is a good way to get yourself into trouble, in my experience.#2017-02-2214:52mpenetI am not using spec as a parser at all#2017-02-2214:53mpenetthe "parse" function returns either a valid ast or an ex-info, the ast is not consumed by spec in any way ( it's basically a (conformer #(try (parse s) (catch ExceptionInfo :clojure.spec/invalid)))#2017-02-2214:53mpenetthe goal ultimately is to have our schemas (that contains these query string representations + tons of other stuff), validated by spec#2017-02-2214:54mpenetand have error reporting/handling at a single level#2017-02-2214:54mpenetotherwise yes, we could do another pass just for query validation#2017-02-2214:54mpenet(outside of spec)#2017-02-2216:12tjtoltonOkay, finally managing to get my company to use spec! great stuff! Here's one thing we're stuggling to internalize - our public API uses json that isn't namespaced, and our internal spec's, obviously, use namespaced keys. Is there some way to say "this incoming data is of the same structure as the spec, but the keys are unqualified, so handle all the keys as though they are qualified to this namespace"?#2017-02-2220:08zaneDid you ever get an answer to this question, @U37NPE2H0?#2017-02-2220:10tjtolton:req-un is actually the answer, yeah#2017-02-2220:10tjtoltonthanks for following up#2017-02-2216:12tjtoltonor does one have to generate a spec-specific view of the data before passing it into a spec'd function -- essentially
(clojure.core.walk/prewalk #(keyword *desired-ns* %) data)
#2017-02-2216:16tjtoltonfurthermore, I'm struggling to get my company to accept namespaced keys. They essentially view namespaced keys as an implementation detail of spec that should not be complected with our data model or public API. I think the big idea with spec is that namespacing keys is actually just a different way to represent data which spec has chosen to support and encourage. But without having enough experience using namespaces and qualified data, I'm pretty sympathetic to the "don't mix implementation details with data" argument.#2017-02-2216:21not-raspberry:req-un?#2017-02-2216:23not-raspberryIf I understand correctly. I only tried to answer the first part of the question.#2017-02-2216:23tjtoltonyeah, I think that's outstanding.#2017-02-2218:29freromHi, I have a small problem with spec that I would like to get some thoughts on. So the problem is spec name conflicts. Since s/keys couples key and spec names together conflicts can arise:
;; Pet
(s/def ::name string?) ;; CONFLICT
(s/def ::pet (s/keys :req-un [::name]))

;; Person
(s/def ::last string?)
(s/def ::first string?)                 
(s/def ::name (s/keys :req-un [::first ::last]))  ;; CONFLICT
(s/def ::person (s/keys :req-un [::name ::animal]))

(s/valid? ::person {:name {:first "John"
                           :last  "Smith"}
                    :pet  {:name "Buddy"}})
How would your handle this without: - Separating these into different files - Combining the specs, e.g. with s/or
#2017-02-2218:41not-raspberry(s/def : string?)#2017-02-2218:41joshjonesWell, you can either use two different files, each having its own namespace, in which case ::name will resolve to the namespace it's in, or, you can skip the auto-resolve :: and just name them differently, in the same file:
(s/def :some.namespace/name string?)
(s/def :other.namespace/name number?)
@frerom
#2017-02-2218:44freromI would love to do the above. But s/def only accepts qualified keywords no?#2017-02-2218:47tbaldridge@frerom sure, but you can do: (s/def :person/name ...) (s/def :pet/name ...)#2017-02-2218:52freromBut s/keys only accept qualified keywords#2017-02-2218:53joshjones
(s/def :person/name string?)
(s/def ::yourmap (s/keys :req-un [:person/name]))
#2017-02-2218:54joshjones
(s/valid? ::yourmap {:name "Fred"}) ;; true
#2017-02-2218:56freromHuh. So this is exactly what I want, but it doesn't seem to work in ClojureScript? All I get is
Assert failed: all keys must be namespace-qualified keywords
#2017-02-2218:57tbaldridgecan you copy your code?#2017-02-2218:57tbaldridge:person/name is a qualified keyword, I think you might have a typeo#2017-02-2219:00freromSorry about that, it was a typo. It works perfectly fine. Thanks a lot for the help šŸ™‚#2017-02-2219:05freromI guess the risk with this solution is that because the spec doesn't have the namespace in the qualified keyword you might actually get spec name conflicts for different reasons? If :person/name is defined in difference files. So I should probably keep that in mind and not be too generic with the naming.#2017-02-2219:07not-raspberry@joshjones Doing (s/def :person/name string?) is a good idea until someone does the same.#2017-02-2219:12freromYes, what I would want to do (If I can make up syntax) is something like ::person/name which expands to :my.namespace.person/name#2017-02-2219:13joshjones
(create-ns 'my.namespace.person)
(alias 'person 'my.namespace.person)
(s/def ::person/name string?)
(s/valid? ::person/name "fred") ;;true
::person/name
=> :my.namespace.person/name
#2017-02-2219:22joshjones@not-raspberry I agree, although nothing prevents anyone from naming any spec anything they want, including any namespace, so it's just making a potential problem less likely, not eliminating it, and assumes good behavior#2017-02-2219:24jasonjcknis there a way to create an s/keys spec from data? e.g. (s/keys :req-un value)#2017-02-2219:25tbaldridge@frerom right, the names of specs should be bounded to the problem at hand. So if I'm writing a compiler I may use :expr.if/condition while if I was writing a public facing api for a company I may use :com.mycomp.department.person/name#2017-02-2219:25tbaldridge@jasonjckn macros or eval are your options, in that order normally.#2017-02-2219:26jasonjcknalright, i guess macros it is#2017-02-2219:26jasonjckni have never used eval in 7 years of clojure#2017-02-2219:31frerom@tbaldridge I agree#2017-02-2223:12ddellacostawhatā€™s the idiomatic way to write something that is essentially an enum of keywords?#2017-02-2223:13ddellacostaI was looking for something like (s/enum :foo :bar) but nothing like that exists; should I simply use (s/or :foo #(= :foo %) :bar #(= :bar %)) ?#2017-02-2223:13ddellacostaseems overly verbose, to say the least#2017-02-2223:14zane@ddellacosta: #{:foo :bar :baz ā€¦}#2017-02-2223:15ddellacosta@zane šŸ˜„ I feel dumb#2017-02-2223:15ddellacostathanks!#2017-02-2223:15zaneDon't!#2017-02-2223:15zaneNo worries!#2017-02-2304:36not-raspberry@ddellacosta watch out - set-based specs dont work for false and nil#2017-02-2304:38Oliver GeorgeIs anyone using caching/memoize type techniques to avoid instrumentation cost? Say keeping a cache of hash values for data which has been checked so that future checks are cheap.#2017-02-2304:51seancorfield@olivergeorge Given that instrument should be a dev/test-only activity, does performance matter?#2017-02-2304:53Oliver George@seancorfield in this case it causes odd performance#2017-02-2304:54Oliver GeorgeLaggy data input etc. In my case: checking the full definition of a complex form at each function/api call.#2017-02-2304:55seancorfieldWell, yes, instrument has an overhead. Of course.#2017-02-2304:55Oliver GeorgeSomething you keep an eye out for when developing. Nice not to have false positives.#2017-02-2304:55not-raspberry@seancorfield not if performance doesnt matter.#2017-02-2304:56Oliver GeorgeYeah. I appreciate that. Love spec. Caching would work. Just wondering if anyone has looked into using it.#2017-02-2304:57seancorfieldIf you have an implementation function and then a wrapper -- and the wrapper is memoized -- then instrumenting the implementation should only apply the overhead for each new set of arguments (and this seems the right way to deal with it).#2017-02-2304:58seancorfieldI'd consider very slow/erratic behavior of an instrumented function to be a good indicator that I might need to cache/memoize the function anyway...?#2017-02-2304:59seancorfield(happy to be proved wrong but that's my initial intuition)#2017-02-2305:00Oliver GeorgeThanks. I see what you mean about wrapping a specific fn. That would do the trick for specific cases.#2017-02-2305:08Oliver George@seancorfield any chance you know of a fifo memoize implementation for cljs?#2017-02-2305:09Oliver George(worried that a long running SPA would gobble memory with memoize in the wrong place)#2017-02-2305:10seancorfieldWe can hope that core.cache and core.memoize get converted to .cljc perhaps?#2017-02-2305:11Oliver GeorgeThat does sound ideal.#2017-02-2305:11seancorfield(and we just got that functionality in the CI support for contrib projects so...)#2017-02-2305:12seancorfieldAs to your original Q, no, sorry, I've no idea about memoization for cls otherwise, sorry šŸ˜ž#2017-02-2307:41ikitommito support selective conforming (or protocol extensions) for the spec: http://dev.clojure.org/jira/browse/CLJ-2116#2017-02-2315:03ddellacosta@not-raspberry I was looking for the equivalent of an enum of keywords, so think Iā€™m safe here...#2017-02-2321:39Oliver George@alexmiller I'm trying to understand if memoizing s/conform is likely to cause problems.
(set! s/conform (memoize (fn [spec x] (s/conform* (s/specize spec) x))))
Biggest risk I see is a spec changing causing the cached results to be invalid. Seems like the spec arg may vary in nature (keywords vs implementations). Perhaps it's better to memoize after calling (specize spec) ?
#2017-02-2322:47Alex Miller (Clojure team)same problems as usual with memoize:#2017-02-2322:47Alex Miller (Clojure team)1) itā€™s an unbounded cache, which only fills up at 3 am in production#2017-02-2322:48bbloomaw cā€™mon now, sometimes it fills up at 4am šŸ˜›#2017-02-2322:48Alex Miller (Clojure team)2) you wonā€™t see effects of any stateful changes (if in your repl, probably from changing specs). but if you made calls to anything stateful, you wonā€™t see changes.#2017-02-2322:48Alex Miller (Clojure team)@bbloom only on daylight savings change#2017-02-2322:48bbloomgood point.#2017-02-2322:49Alex Miller (Clojure team)but specs are built with a delay anyways, which is forced on first use#2017-02-2322:49Alex Miller (Clojure team)so they effectively already cache the spec conform implementation and wonā€™t see changes if you change specs after use anyways#2017-02-2322:49Alex Miller (Clojure team)so thatā€™s maybe only a small downside#2017-02-2323:28agso I have a s/keys spec with :opt fields. When I use (gen/generate (s/gen ::my-spec) it generates them with :req fields and skips :opt fields. Is there a way to force :opt fields to be added?#2017-02-2323:29schmeeif you generate a bunch of them it will add in optionals#2017-02-2323:29schmeegenerate starts out with the simplest possible values and gets more complicated as it goes#2017-02-2323:32agoh, wait, it actually does not skip :opt fields all the time, but now if I have (gen/vector (s/gen ::my-spec)) some items will have some of :opt fields and some would not. Is there a quick and easy way to fix that?#2017-02-2323:32schmeeag thatā€™s what I meant above#2017-02-2323:33schmeeif you want the field all the time, why are they optional?#2017-02-2323:34agbecause, in the runtime they are.#2017-02-2323:34agmeh, I guess itā€™s actually fine.#2017-02-2323:34agI thought for a moment itā€™s just not generating :opt fields at all#2017-02-2323:34agI was wrong#2017-02-2323:37schmeeyou can use a custom generator if you want to add them all the time#2017-02-2400:12Oliver George@alexmiller Thanks Alex, that's helpful. Sounds like some of those issues could be addressed with a LRU/TTL version of memoize. Shame CLJS doesn't have one packed up at this point.#2017-02-2400:13Oliver George(or flushing the memo cache when the spec registry changes)(#2017-02-2400:13Alex Miller (Clojure team)Yeah, core.cache and core.memoize have those, but not cljs right now#2017-02-2400:14Alex Miller (Clojure team)@ag there is a pending patch to fix keys gen for opts#2017-02-2400:19Alex Miller (Clojure team)http://dev.clojure.org/jira/browse/CLJ-2046#2017-02-2401:09agif my vector contains maps with a key thatā€™s value is a function, how do I check for it? is there a predicate like function?. Actually what I have there is a clojurescript function. (type) returns something like this: #object[Function "function Function() { [native code] }ā€] how can I get a predicate that checks for that?#2017-02-2401:16bnoguchiIf we don't want specs' conformed values to thread in an (s/and spec-1 spec-2 from spec-1 to spec-2 to etc , can we substitute s/merge for s/and? And can spec-* be a non-map spec or just a predicate when called with merge?#2017-02-2401:22danielcompton@ag: cljs.core/ifn?#2017-02-2401:28ag@danielcompton something tells me that this is right, but for some reason not really working when placed into .cljc file (s/map-of keyword? ifn?)#2017-02-2401:29agoh, I think my problem not that, spec works, but needs a generator. hold on#2017-02-2401:31ag
Unable to construct gen at: [:options 1] for: ifn?]
#2017-02-2401:32agthis worked:
(s/with-gen (s/map-of keyword? ifn?)
                         #(gen/return {:on-click (fn [e] nil)}))
#2017-02-2401:40danielcomptonOh, maybe cljs.core/fn? would be more appropriate, as maps, keywords, e.t.c. are also ifn?#2017-02-2402:49agan hour ago I didnā€™t know that either of them existed. ĀÆ\(惄)/ĀÆ#2017-02-2402:51agyup, youā€™re right! Thank you @danielcompton#2017-02-2421:15j-poShould (require '[my-ns :as n]) allow you to then address :my-ns/kw as :n/kw? I'd been functioning under the impression that yes, but now I'm trying to address a spec defined in another file in the repl and I'm getting "unable to resolve spec".#2017-02-2421:16Alex Miller (Clojure team)No, would be ::n/kw#2017-02-2421:16j-poAha!#2017-02-2421:16j-poThanks#2017-02-2421:17Alex Miller (Clojure team)The :: means "auto-resolve" which will use aliases#2017-02-2502:44thedavidmeisteris there a way in spec to define relationships between attributes, like #(> ::foo :bar) for example?#2017-02-2504:01joshjones@thedavidmeister
(s/def ::foo int?)
(s/def ::bar int?)
(s/def ::keyspec (s/and (s/keys :req [::foo ::bar])
                        #(> (::foo %) (::bar %))))
#2017-02-2504:03thedavidmeister@joshjones so... is % in that context going to be the thing being validated?#2017-02-2504:08joshjonesyes -- for example (s/valid? ::keyspec {::foo 2 ::bar 3}) the map is the argument to the function#2017-02-2504:13thedavidmeister@joshjones oh, cool#2017-02-2504:14thedavidmeisterso then my next q is#2017-02-2504:15thedavidmeisterif i have :req, how do i get a list of the keys that are missing?#2017-02-2504:25joshjonesthis is not meant to be a 'best practices', just to show how you can get the data:
(->> (s/explain-data ::keyspec {::foo 2})
     :clojure.spec/problems
     (map :pred)
     (filter #(= 'contains? (first %)))
     (map last))
#2017-02-2504:46thedavidmeister@joshjones oh ok#2017-02-2504:46thedavidmeisterlooks a bit weird but i guess it does the job#2017-02-2504:46thedavidmeistercheers šŸ™‚#2017-02-2505:23Alex Miller (Clojure team)Call explain-data to get a list of all problems#2017-02-2505:40joshjones@thedavidmeister hmm, "weird" is about all I got late Friday after evening activities šŸ™‚ but here you go, same as above but rewritten in a function. cheers to you too
(defn keys-missing
  [spec data]
  (let [explanation (s/explain-data spec data)
        problems (:clojure.spec/problems explanation)
        xf (comp (map :pred)
                 (filter #(= 'contains? (first %)))
                 (map last))]
    (into [] xf problems)))
#2017-02-2505:41joshjonesit relies on things you possibly shouldn't rely on, like contains? implying that a key is missing, works for now but it's not "public" ... but it's enough to get you going I think šŸ˜‰#2017-02-2507:04thedavidmeister@joshjones yup, looks good enough for me to start swapping out my hand-rolled validation code#2017-02-2518:45bbloomi forget - was there a collection of attempted/in-progress specs for all the core functions somewhere?#2017-02-2519:34Alex Miller (Clojure team)from core, no (beyond the one ticket out there that slightly extends whatā€™s committed)#2017-02-2519:34Alex Miller (Clojure team)there is another project someone started, but Iā€™ve intentionally not looked at it#2017-02-2601:01agHey guys, so if I have s/keys spec, that enforces a predicate on one of the keys (so it actually requires a generator, since it cannot satisfy predicate after so many tries), how the generator part would look like for the spec like that? trivial example:
(s/with-gen
  (s/and
   (s/keys :req [:foo :bar :baz :qux :zap :dap])
   (fn [m]) (= (:foo m) 42))
  (fn []
    ;; how the generator would look like?
    ))
Do I have to generate a hashmap with (s/gen) for each key and then some logic that satisfies the constraint? Is there a better way?
#2017-02-2601:05agI guess I would create a spec without the field(s) that I want to constraint and then merge it with another spec (with the constraint(s) and generator)#2017-02-2601:09seancorfieldYeah, splitting it in two would be my suggestion.#2017-02-2601:10seancorfieldSince then you can (g/fmap #(assoc % :foo 42) (s/gen ::base-spec)) or something like that...#2017-02-2622:24Oliver GeorgeIt seems like I can't unform collections. What am I missing?
(require '[clojure.spec :as s])

(s/def ::text
  (s/and string?
         (s/conformer
           (fn [s] {:type ::text :value s :errors []})
           (fn [m] (:value m)))))

; GOOD: string conforms correctly
(s/conform ::text "blah")
=> {:type :user/text, :value "blah", :errors []}

; GOOD: map unforms back to string
(s/unform ::text (s/conform ::text "blah"))
=> "blah"

; Bad: collection unform has no effect
(s/unform (s/coll-of ::text) (s/conform (s/coll-of ::text) ["blah" "blah"]))
=> [{:type :user/text, :value "blah", :errors []} {:type :user/text, :value "blah", :errors []}]
#2017-02-2622:45Oliver GeorgeAh, it's a known bug. http://dev.clojure.org/jira/browse/CLJ-2076#2017-02-2701:27bbloomhey @gfredericks et al - have you explored concurrent (ie non-monadic, maybe applicative) generation in any way?#2017-02-2701:31gfredericks@bbloom what would you use that for#2017-02-2701:31gfredericksI'm assuming you're not talking about "running my tests in parallel"#2017-02-2701:31bbloomno, iā€™m talking about conceptual concurrency#2017-02-2701:32bbloomnot necessarily parallelism#2017-02-2701:32bbloomiā€™m mostly thinking about the effort required to make generators that behave well#2017-02-2701:32bbloomthe generate & test approach is kinda cumbersome for some more interesting properties#2017-02-2701:33gfredericksOh you're talking about that awkward separation#2017-02-2701:33gfredericksFor when you wanna do some testy things and then generate some more stuff? #2017-02-2701:34bbloomtrying to come up with a good example, way to explain my thought here#2017-02-2701:34bbloom1 sec#2017-02-2701:38bbloomok so if i do something like:#2017-02-2701:38bbloom(s/exercise (s/and int? #(> % 50000000000)))#2017-02-2701:38bbloomthis still works#2017-02-2701:38bbloombut only b/c the int generator is producing lots of values and then the s/and generator is filtering them#2017-02-2701:39bbloomiā€™m wondering if thereā€™s any experimentation done where the filter can feed information back to the generator#2017-02-2701:39bbloomrather than being a strictly forward pipeline#2017-02-2701:41gfredericksOhright. I feel like rich might have wanted something like that. But I have no idea what direction you would go for a general solution to that. Did you have something specific in mind when you said concurrent? #2017-02-2701:41bbloomsomething earlier had reminded me of the art of the propagators stuff#2017-02-2701:42bbloomand i re-read the section in that report about the truth maintenance systems and the search problem#2017-02-2701:42bbloomonce you add predicates, generation changes from an enumeration problem in to a search problem#2017-02-2701:42bbloomnumbers with inequalities are a pretty easy search space - but can get rough if you did something like ā€œis primeā€ or whatever#2017-02-2701:43bbloomso you wind up having to override the generator for the whole thing#2017-02-2701:43bbloomand the more interesting the predicates, the more generator work you need to do#2017-02-2701:43gfredericksYeah, it smells like declarative programming, where you can do much more than you might think, but way less than you need in real life#2017-02-2701:44bbloomheh#2017-02-2701:45bbloomi was just wondering if you guys had thought about this sort of thing much at all - since i keep trying to use generative testing, and it keeps being a lot of work. so far iā€™ve only really been truly successful with it in the form of ā€œsimulation testingā€ on a service#2017-02-2701:46bbloombut i had to do a lot of work to build a good model / generator / validators / etc - made sense for a whole system, but s/exercise etc is fun and rapidly loses utility as i enhance my specs#2017-02-2701:46gfredericksYeah I agree about all of that#2017-02-2701:48gfredericksI haven't thought about it too much because I haven't had any reason to be hopeful that it would be fruitful. I'm definitely open to being persuaded otherwise. #2017-02-2701:49bbloomi guess thatā€™s why i specifically mentioned ā€œapplicativeā€ - as it seems like there might be some way to improve generators in a compositional way using binary combinators#2017-02-2701:49bbloomthe two interesting ones being intersect and union#2017-02-2701:49bbloomso like maybe rather than defining a generator override for a spec by name#2017-02-2701:49bbloomyou can specify an override for a tuple of union/intersect and a pair of generators#2017-02-2701:50bbloomwhich is already sorta what s/and and s/or do, but the do it left to right#2017-02-2701:51bbloomalthough itā€™s not quite clear to me how to go about this without immediately winding up in the land of SAT/SMT šŸ™‚#2017-02-2701:54gfredericksyeah; and inevitably it would only work for a few lucky combinations of things and otherwise you'd be on your own#2017-02-2701:55gfredericksif there are common combinations of specs that are easy to generate automatically, that can, I assume, be automated by writing special spec macros for them#2017-02-2701:57bbloomok - well i may noodle on this some more at some point, but for now iā€™ll just resign myself to abandoning generative testing as soon as it gets tough and then re-adopting it as soon as my system gets big enough to justify the simulant sort of thing šŸ™‚
#2017-02-2701:57bbloomthanks for the discussion#2017-02-2701:58gfrederickshaha#2017-02-2701:59gfredericksnp; good luck#2017-02-2711:21jrychterWhile writing specs for some existing data structures I encountered a case where a structure has two fields with the same unqualified keyword (nested on different levels, obviously). One field is a collection, the other is a string. Is there a way to make this work with spec? I tried searching online, but couldn't find anything and it seems like generally the answer would be "your data structure is wrong, fix it". Other than splitting the structure into two namespaces (so that each keyword and its associated specs is in a different ns), is there anyway to address this?#2017-02-2711:24mpeneteither you need 2 different ns'ed keywords as spec ids, or use a predicate instead of s/keys#2017-02-2711:50Yehonathan SharvitThe interesting question is: why s/keys has been designed this way?#2017-02-2711:51Yehonathan SharvitI mean: probably the case raised by @jrychter is not idiomatic#2017-02-2711:51Yehonathan SharvitMaybe, it is not idiomatic to have different keys with the same name?#2017-02-2711:52Yehonathan SharvitIā€™m asking because I have also encountered the same issue#2017-02-2711:52jrychterWell, I can see why "your data structure is wrong, fix it" makes sense ā€” I was just hoping for a reasonable workaround. In the meantime, I am writing db migration code to rename one of the keys.#2017-02-2711:53Yehonathan SharvitActually, I donā€™t understand why it is wrong - but Iā€™ve got the feeling that itā€™s not the clojure.spec way to shape a map#2017-02-2711:53Yehonathan SharvitBut I donā€™t understand why s/keys doesnā€™t support passing pairs to it#2017-02-2711:54Yehonathan SharvitSomething like: (s/keys :req-un [[::aa integer?] [::bb string?]])#2017-02-2711:54Yehonathan SharvitThen the name of the key would be decoupled from its meaning#2017-02-2711:59jrychterWell, spec by being tied to namespaces assumes that a spec key is "global" within the data structure, as long as everything is within a single ns. I can see (and like) the advantages of that, but there are cases where this becomes problematic.#2017-02-2712:04gfredericksthe workaround is two keywords with different namespaces#2017-02-2712:10gfredericksyep ā˜ļø that#2017-02-2715:00Alex Miller (Clojure team)@viebel the whole point of spec is that the key of a map is a semantically enduring thing with meaning and that maps are nothing but sets of those attributes#2017-02-2715:33jrychterThis makes perfect sense, except sometimes you run into maps where there is a :name and a :seller :name, and you don't necessarily want to introduce a whole new seller namespace. Spec pretty much forces you to. Note, I'm not saying it's a bad thing, just noting that this is one of the few places where Clojure steps in and says: this is the one right way to do it. I got used to Clojure being very lenient (want objects? fine. want pure functions? fine. want state? fine. want to mutate it? fine.)#2017-02-2715:34gfredericksa lot of stuff about spec would have to change to accommodate that, I think#2017-02-2715:34Alex Miller (Clojure team)spec has aspirational and opinionated goals#2017-02-2715:35jrychter@alexmiller: Yes, that's what I'm noticing šŸ™‚ I've already learned to appreciate some of those. In particular, spec encourages reuse, and I'm already making use of it.#2017-02-2715:35gfredericksI guess it could just add a new kind of map spec; that'd be pretty independent#2017-02-2715:36Alex Miller (Clojure team)weā€™re not going to do that#2017-02-2715:36gfredericksyes I do realize that šŸ™‚#2017-02-2715:36gfredericksbut I think a library could provide that without undermining anything#2017-02-2715:39mpenet@alexmiller do you know if Rich has an opinion about http://dev.clojure.org/jira/browse/CLJ-2112 and the form it's taking currently?#2017-02-2715:39mpenetI am tempted to rewrite a bunch of macros for generating specs+json-schemas on that base, and would like to know if it's viable#2017-02-2715:40Alex Miller (Clojure team)@mpenet I would not base anything off of that patch - everything there has changed multiple times and is likely to change again#2017-02-2715:40Alex Miller (Clojure team)it has been through one round of review from both Rich and Stu but there are still a number of open questions about it#2017-02-2715:41mpenetAlright, thanks for the info. Any sort of ETA for news on this proposal?#2017-02-2715:41Alex Miller (Clojure team)no#2017-02-2722:23piotr-yuxuanHello here šŸ™‚ Just a question for my information: has anyone ever tried (or plan to try) to use clojure.spec to validate form answers or models?#2017-02-2722:32seancorfieldWe use spec to validate REST API arguments ā€” which is pretty similar to form answers (in that they are both user-provided over HTTP).#2017-02-2803:35luxbockwhen writing variations of an existing spec, such as "order" -> "open order", "closed order", should there be a convention for how to name the namespace part of a spec#2017-02-2803:37luxbockright now I tend to define order as ::order and then for the variations I add the variation name after a dot to the end of the namespace, so :my.ns.closed/order#2017-02-2803:38luxbockI do this without creating a new namespace, i.e. I just write the full keyword out for the variations#2017-02-2809:07dergutemoritz@luxbock Another option would be ::order.closed. That way you can still use the real namespace and thus the auto resolving shortcut syntax.#2017-02-2809:08luxbock@dergutemoritz yeah but then I can't use use it as an unqualified key in a map#2017-02-2809:09luxbockmy use case is basically where there are a series of transformations on this value in a map, and I want to capture some properties about the transformations having done what they were supposed to#2017-02-2809:09dergutemoritzOh okay, if you are specing legacy maps, I wouldn't know what's the best way to go about that#2017-02-2809:10luxbockI think using unqualified keywords is the only way to do this type of thing, where I want to capture changes in the value between functions#2017-02-2809:10luxbockif I were to use a qualified keyword for the map then the spec has to be static#2017-02-2809:11dergutemoritzNote that you can also create and alias a namespace from within another namespace like this: (create-ns 'my.ns.closed) (alias 'closed 'my.ns.closed) which then allows you to require the my.ns.closed namespace from other namespaces and use it in the current namespace via ::closed/#2017-02-2809:11luxbockhmm yeah that could be another good option#2017-02-2809:14dergutemoritzRe capturing changes in the value between functions: One central idea of spec is to not use the same key for different kinds of values but to attach a global meaning to a key - so different things should use different keys#2017-02-2809:14dergutemoritzMaybe you can do that, too#2017-02-2809:15dergutemoritzIn other words: what's keeping you from using different keys?#2017-02-2809:16luxbockyeah I suppose that might be a better a better practice#2017-02-2809:16dergutemoritzWorth a try at least šŸ™‚#2017-02-2809:17luxbockI was thinking that since all of the values that are changed only live during one pass from a function to another, the use of unqualified keys would capture this transient nature of theirs#2017-02-2809:20dergutemoritzSounds like you have a deep chain of calls which pass a map through each other, each modifying that map? From a maintenance perspective, I think using different keys makes sense here, too. Imagine reading that code for the first time, you'd see the same keys being used in a dozen functions but in each they may have very different values, depending on from where they're called.#2017-02-2809:21luxbockyeah I think you are right#2017-02-2809:21dergutemoritzWell, it might be easier with un specs but still unnecessarily context sensitive I'd think#2017-02-2809:46ikitommi@viebel have also looked for easy way to define more local key specs. Did a simple helper on top of spec to define deeply nested map/vector/set specs. But currently pollutes the spec registry with generated keys: https://github.com/metosin/spec-tools#simple-collection-specs. There could be a custom keys in spec-tools supporting local keys..#2017-02-2815:12Alex Miller (Clojure team)@dergutemoritz re the earlier suggestion of ::order.closed - per https://clojure.org/reference/reader#_literals, keywords should not contain . in the name. This is not enforced anywhere, but I think you should steer away from it in general.#2017-02-2815:20dergutemoritz@alexmiller Ah, thanks for the hint! Yeah it's not enforced anywhere in my experience, I'd rather not rely too heavily on unspecified behavior so the headsup is definitely welcome#2017-02-2817:49vikeriI know spec does not check ā€œunspecced" keywords for a number of reasons. But sometimes it would be very handy to check big nested map (eg. re-frame app-db) and get the keys that are not specced so that I can add specs for them. Is there any way of enumerating the registered specs?#2017-02-2817:51bbloom@vikeri instrument etc does this - so you can probably look at how those work#2017-02-2817:52vikeri@bbloom Ok thanks!#2017-02-2819:15Alex Miller (Clojure team)@vikeri you can call s/get-spec to check whether a spec is in the registry#2017-02-2819:16mikerodMany Clojure features are based on some past research/books/papers. Is there a list of these for Clojure spec?#2017-02-2819:16mikerodIā€™ve seen http://matt.might.net/papers/might2011derivatives.pdf#2017-02-2819:16mikerodfrom David Nolen @ https://swannodette.github.io/2016/06/03/tools-for-thought#2017-02-2819:17Alex Miller (Clojure team)there are a couple of references in https://clojure.org/about/spec#2017-02-2819:17mikerodquoted: > clojure.spec takes "Matt Might et. al. Parsing with Derivatives" and really, really runs with it.#2017-02-2819:17mikerodRDF and Racket contracts#2017-02-2819:17mikerodIā€™ve seen those too, but I wasnā€™t sure on which aspects of those and if how it related to the parsing stuff#2017-02-2819:18Alex Miller (Clojure team)the parsing stuff is specific to the regex op specs#2017-02-2819:18mikerodbut thanks for reminding me that those were at the bottom of the about spec page - I forgot#2017-02-2819:18Alex Miller (Clojure team)contracts are at a high level similar to what spec is doing#2017-02-2819:19Alex Miller (Clojure team)the rdf reference is really about building growable information systems (Datomic steals liberally from this as well)#2017-02-2819:20Alex Miller (Clojure team)Rich is being somewhat modest imo in that heā€™s been thinking about some of the parts for as long as heā€™s been working on Clojure (like maps as a set of attributes)#2017-02-2819:21mikerodThanks for that added insight Alex.#2017-02-2819:22mikerod> the parsing stuff is specific to the regex op specs Does this mean that "Matt Might et. al. Parsing with Derivativesā€ is not really relevant to an understanding?#2017-02-2819:22Alex Miller (Clojure team)only about understanding that very specific part of the code#2017-02-2819:22mikerodI do intend to dig into the parsing of spec some out of interest in what it is doing. I was just wondering if reading that paper would have any relationship. Iā€™ve read a bit of it so far.#2017-02-2819:23mikerodI know spec has more to it than parsing of course. I can understand how the RDF is about the growable systems and Racket contracts as a similar lang feature.#2017-02-2819:23Alex Miller (Clojure team)thereā€™s very little ā€œparsingā€ - the only part doing anything youā€™d call parsing is the regex stuff#2017-02-2819:23Alex Miller (Clojure team)everything in spec works on Clojure data, not strings#2017-02-2819:23mikerodYeah, I have seen that much.#2017-02-2819:24Alex Miller (Clojure team)we looked at a lot of other existing libs in the research for this (I built a couple versions prior to Richā€™s work that were thrown away)#2017-02-2819:25Alex Miller (Clojure team)of those, I would point most at Christopheā€™s seqexp lib, which is really quite lovely#2017-02-2819:25Alex Miller (Clojure team)https://github.com/cgrand/seqexp#2017-02-2819:26Alex Miller (Clojure team)and Ghadiā€™s pex is similarly interesting https://github.com/ghadishayban/pex#2017-02-2819:27Alex Miller (Clojure team)spec does not take either of those approaches, but they were helpful as points in the design space#2017-02-2819:27Alex Miller (Clojure team)Iā€™d say the ideas of a) starting from predicates and b) automatically generative from the start are also things that Rich insisted on from the very first conversation#2017-02-2819:29mikerodThanks for all the details. Thatā€™s good background info. Itā€™s always good to get to hear some of the thought process behind these sorts of things to understand the design process some.#2017-02-2819:32ghadiIf I were to rewrite pex, I'd globally register match patterns like spec does.#2017-02-2819:32bbloom@ghadi still happy with the bytecode interpreter approach?#2017-02-2819:32bbloomiā€™ve tried to do some parsing with derivatives and have had pretty poor luck#2017-02-2819:32bbloomi find the derivative of a grammar to make sense, but the derivative of a parser makes my brain hurt#2017-02-2819:35ghadibytecode interpreter is super fast and I love it#2017-02-2819:36ghadibut -- I didn't implement a stack overflow guard#2017-02-2819:36bbloomĀÆ\(惄)/ĀÆ#2017-02-2819:36ghadiI am not 100% sold on registers vs locals though. @cgrand did registers in segexp#2017-02-2819:37ghadihttps://github.com/cgrand/seqexp/blob/master/src/net/cgrand/seqexp.clj#L178-L198 I don't fully understand it šŸ˜ƒ#2017-02-2819:39bbloomam i reading that right? only two registers?#2017-02-2819:40cgrandNo N registers of two slots#2017-02-2819:40bbloomwhat are the slots?#2017-02-2819:41cgrandEach end of a sub match. #2017-02-2819:41bbloomah - gotcha#2017-02-2819:42bbloomparsing is fun/terrible šŸ™‚#2017-02-2819:43cgrandThe goal was to avoid head retention#2017-02-2819:44bbloomlike for streaming parsing? assuming you donā€™t have unbounded backtracking in your grammar?#2017-02-2819:44cgrandYup#2017-02-2819:47cgrandI use it to query live healthcare data streams. #2017-03-0109:35cgrand@ghadi I believe itā€™s not a choice of registers vs locals. With a PEG parser you have ordered choice so I suppose than in (/ A B) you first try to parse A then backtrack and try B. In Seqexp | is non-deterministic so I have to represent concurrent ā€œthreadsā€ (and deduplicate them). Itā€™s in the deduplication part that determinism is brought back by keeping submatches of the ā€œleftmost" (according to the choice tree) thread.#2017-03-0109:39cgrandSo because threads are data, It wasnā€™t possible to use locals.#2017-03-0115:03devthnot possible to spec lambdas? i suppose you could validate it the body of the lambda via assert#2017-03-0116:22Alex Miller (Clojure team)depending what you mean, you can spec a function with fspec#2017-03-0116:23devthright, I mean specing something like f in (let [f (fn [x] ...)])#2017-03-0117:20dhruv1how can i generate a json schema from a clojure spec#2017-03-0118:48bbloomafter spec takes over the world for describing data - i hope rich, alex, et al take on session types / protocol specs šŸ™‚#2017-03-0118:58tbaldridgeI'd love to see a spec-take on "isa?", "subset of?", etc.#2017-03-0210:15thhellerso is anyone else constantly doing this: (s/keys ::req-un [::key-a ::key-b])? with all the namespaced keywords I usually don't even notice what is wrong#2017-03-0210:16thhellerand keep wondering why my spec doesn't work, any chance this could throw a warning?#2017-03-0210:58not-raspberry@thheller http://dev.clojure.org/jira/browse/CLJ-2112 should help#2017-03-0211:26thhellerdoesn't look like it would catch this case as ::req-un would just be ignored again#2017-03-0213:41Yehonathan SharvitIs there a spec for date?#2017-03-0213:42souenzzoinst?#2017-03-0213:44Yehonathan SharvitThanks @souenzzo. What about a date that is stored as a string?#2017-03-0216:37Alex Miller (Clojure team)string?#2017-03-0216:38Yehonathan SharvitIā€™d like the spec to validate ā€œ03/27/2016ā€ but not ā€œHello World"#2017-03-0216:38Alex Miller (Clojure team)There are date time parsers in the Java std lib#2017-03-0216:39Alex Miller (Clojure team)Either that or a regex pred#2017-03-0216:39Alex Miller (Clojure team)(I was just joking about string? :)#2017-03-0216:40Yehonathan SharvitOK#2017-03-0216:44Alex Miller (Clojure team)
user=> (import 'java.text.SimpleDateFormat)
java.text.SimpleDateFormat
user=> (def formatter (SimpleDateFormat. "MM/dd/yyyy"))
#'user/formatter
user=> (.parse formatter "03/27/2016")
#inst ā€œ2016-03-27T05:00:00.000-00:00ā€
#2017-03-0216:45Alex Miller (Clojure team)youā€™d want to wrap up the call to .parse to catch ParseException and return false#2017-03-0216:45Yehonathan Sharvitcool#2017-03-0216:45Yehonathan Sharvitthx a lot#2017-03-0216:46Alex Miller (Clojure team)
(defn valid-date? [s] (try (.parse formatter s) (catch ParseException e false)))
#2017-03-0216:47mpenetthis is not threadsafe tho#2017-03-0216:47Alex Miller (Clojure team)truth#2017-03-0216:47Alex Miller (Clojure team)usually you wrap that in a ThreadLocal#2017-03-0216:47mpenetI had a weird debugging session because of this a few years back#2017-03-0216:48mpenetyes or just use joda.time (or possibly new java api)#2017-03-0216:48Alex Miller (Clojure team)good point - I havenā€™t used much of the new java 8 stuff#2017-03-0216:49Alex Miller (Clojure team)youā€™d want to use java.time.format.DateTimeFormatter#2017-03-0216:56Alex Miller (Clojure team)
(import [java.time.format DateTimeFormatter DateTimeParseException] java.time.LocalDate)
(def format (DateTimeFormatter/ofPattern ā€œMM/dd/yyyy"))
(defn valid-date? [s] (try (LocalDate/parse s format) (catch DateTimeParseException _ false)))
#2017-03-0216:56Alex Miller (Clojure team)that should be thread-safe#2017-03-0217:00Alex Miller (Clojure team)please burn the prior example#2017-03-0217:28Yehonathan Sharvit@mpenet what did u mean by not thread-safe?#2017-03-0217:29gfredericksmaybe the formatter isn't thread safe?#2017-03-0217:29Yehonathan Sharvitwhat does it mean for a fomatter not to be thread-safe?#2017-03-0217:29jrmultiple threads using the same formatter is not safe#2017-03-0217:30jrsince formatter is a class instance and stores intermediate parse data in instance variables#2017-03-0217:31mpenetyes if you call .parse from multiple threads you'll get garbage back sometimes#2017-03-0217:32mpenetit gets quite obvious (date very much in the future etc etc)#2017-03-0217:32Yehonathan SharvitUnbelievable!#2017-03-0217:32gfredericksit seemed like a good idea in the 90's#2017-03-0217:50Alex Miller (Clojure team)No, it really didnt#2017-03-0220:45zerocooljshi please can you help me?? how I can make a spec case insensitive:
(s/def :vtex/channel #{"KUSHKIPAGOS"})
#2017-03-0221:00Alex Miller (Clojure team)you could replace the spec with #(= (clojure.string/upper-case %) "KUSHKIPAGOS")#2017-03-0221:04bbloomon the topic of accidental double colon on ::req-un - thatā€™s a situtation where the parameters are plainly provided statically, so MAAAAYBE warning on unrecognized keys wouldnā€™t be totally objectionable - but that would require trusting ppl to not abuse whatever flag enables that check#2017-03-0221:05bbloomi mentioned this before, see https://www.typescriptlang.org/docs/handbook/interfaces.html and search for ā€œexcess property checking"#2017-03-0221:06bbloommight be possible to accomplish with metadata or something - if the object has file/line/column info, then it might be a candidate for excess property checking#2017-03-0221:06bbloomjust a thought#2017-03-0315:07madstapIs this a known bug in spec? I'd expect both to return the same thing, but using s/and with s/keys* doesn't seem to work.
(s/def ::x int?)

  (s/conform (s/cat :a (s/and int? (constantly true))
                    :x (s/keys* :opt-un [::x]))
             [3 :x 3])
  ;=> {:a 2 :x {:x 3}}

  (s/conform (s/cat :a int?
                    :x (s/and (s/keys* :opt-un [::x]) (constantly true)))
             [3 :x 3])
  ;=> :clojure.spec/invalid
#2017-03-0315:14moxaj@madstap by wrapping your keys* spec with and, I believe it's no longer a regex spec
(s/conform (s/cat :a int?
                  :x (s/and (s/keys* :opt-un [::x]) (constantly true)))
           [3 [:x 3]])
;=> {:x {:x 3}, :a 3} 
#2017-03-0315:31Alex Miller (Clojure team)correct - use s/& instead to add a predicate but keep it as a regex op#2017-03-0315:33Alex Miller (Clojure team)the and spec above would be looking for data like [3 [:x 3]]#2017-03-0319:30bbloomThis is on HN right now. Iā€™ve read the intro & itā€™s pretty awesome: https://www.cl.cam.ac.uk/~sd601/thesis.pdf ā€” of interest to anybody doing static analysis with spec, or just generally interested in both open/extensible and types#2017-03-0323:52jrheardhiya - beginning to add spec to a preexisting project, and iā€™m running into this situation: iā€™m trying to spec some ā€œeventā€ maps, which have required keys including :type (eg one of :foo, :bar, :baz) and :data. and the thing is that when :type is :foo, :data is a map that looks like {:a 1 :b 2}; and when :type is :bar, :data is a map that looks like {:c 3 :b 4}; and when :type is :baz, :data is a number like 1234, etc. what do i want in this situation - is this the exact situation where i should reach for a multi-spec?#2017-03-0323:56jrheardiā€™m beginning to think that having :data be the name for several different categories of thing will be a problem there#2017-03-0323:57jrheardand this codebase is small enough, and there are few enough event types, that i can have :foo events have an :event/thing key, and :bar events have an :event/user-id key, etc#2017-03-0323:57jrheardso i think doing that, and using a multi-spec, is probably the right approach here. rad.#2017-03-0401:43joshjones@jrheard yes, you want a multi-spec#2017-03-0517:52michelrandahlhow do you 're-use' a random seed for re-checking or re-generating test sample?#2017-03-0518:16joshjoneslet's say you ran:
(stest/check `my-func)
which will return a map which has a :clojure.spec.test.check/ret {:result true :num-tests 100 :seed 123456789} To reuse this seed, run:
(stest/check `my-func {:clojure.spec.test.check/opts {:seed 123456789}})
this opts map also takes :num-tests which controls how many test are run, and :max-size, which constrains the size of the generated vals
#2017-03-0518:19joshjones@michelrandahl#2017-03-0611:03jrychterI used s/merge to extend a spec with additional keys, but when I s/conform, it seems that only the newly-added keys get conformed, not the original spec. In general, it seems that while s/valid validates the entire merged spec, s/conform only uses the last spec passed to s/merge. Is this something to be expected?#2017-03-0611:09jrychterOh, just found CLJ-2033 which describes this exact problem.#2017-03-0611:11jrychterAfter reading the discussion several times, I don't understand what I'm supposed to do. Is the point here that unqualified keys are second-class citizens and should be phased out? Or is there another solution to merging specs and having conform work?#2017-03-0611:11mpenets/and#2017-03-0611:12jrychterInstead of s/merge, right?#2017-03-0611:12mpenetwell it's not really merging anymore, but that can fix it depending on your use case#2017-03-0611:12mpenetyes#2017-03-0611:12mpenet(s/conform (s/and (s/keys :req-un [::value]) (s/keys)) {:value 5}) -> {:value 5.0}#2017-03-0611:12jrychterAnd more generally? Because try as I might, I find this behavior unexpected.#2017-03-0611:14jrychterIndeed, s/and works just fine. I'm making notes to never use s/merge, but I don't understand why "it does not flow conformed results". Why would one want it to behave that way?#2017-03-0611:17mpenetnot sure really, but yeah I do just like you said and just use s/and#2017-03-0611:22jrychterI'm sure there was a reason, but to explain my thinking: having read both the s/merge docstring and the clojure.spec guide, I did not expect s/merge to be a lossy operation which will drop some of my conformers. Especially since it is called just like a map merge operation and looks like one. If this is really intended, I think it deserves a warning in both places.#2017-03-0611:35mpenetwith and you also loose :gen#2017-03-0611:36mpenetso yeah, it's one of the oddities in spec. I also find this quite counter-intuitive (no matter the reason/motivation behind this)#2017-03-0616:43peejaIs there a way to ask if one spec conforms to another?#2017-03-0616:43peejaThat is, "Is every value which is valid for spec A valid for spec B?"#2017-03-0616:54not-raspberryI think in general case this is impossible, not only in spec. User-defined functions can be passed to s/def. But in a limited way, you can generate values for spec A and validate them against spec B.#2017-03-0616:57peejaI guess that makes sense. One of the major visions for spec is to confirm compatibility between, say, your code and a new version of a dependency (in a world where they both are well spec'd). Is the expected way to do that to use test.check to confirm it works for lots of random values?#2017-03-0616:58gfredericksI don't think so#2017-03-0616:58gfredericksI think the main idea was static analysis of keysets and regexes#2017-03-0616:59peejaSo you are expected to be able to do some degree of static analysis, then#2017-03-0617:10bronsaI think specs can theoretically be comparable in finite time, since AFAICT they describe regular languages#2017-03-0617:10bronsabut it might take a long time to compare them#2017-03-0617:12bronsaI might be wrong tho. if spec accepts CFGs then comparing specs is definitely not possible#2017-03-0617:13not-raspberry@bronsa What about (s/def ::a (fn some-fn [v] ...)) and (s/def ::b (fn different-fn [v] ...)) - how do you compare Clojure functions?#2017-03-0617:13bronsayou don't, you're right#2017-03-0617:14bronsaso it's a theoretical no too :)#2017-03-0617:17gfredericksunless you can get away with saying "different functions are always completely different"#2017-03-0617:17gfredericks@bronsa the sequence regexes aren't technically regular as they can self-reference#2017-03-0617:17gfredericksI'm not sure if that's an intentional feature though#2017-03-0617:18bronsaĀÆ\(惄)/ĀÆ#2017-03-0619:09souenzzoHow can I express: "One of there keys are required"???#2017-03-0619:11souenzzo(s/def ::car (s/keys :req [:document/number :license/plate])) But I just need one of these keys#2017-03-0619:12Alex Miller (Clojure team)You can use or in :req - check the doc string or the guide#2017-03-0620:59seancorfield(s/def ::car (s/keys :req [(or :document/number :license/plate)])) ā€” note this allows for cars with both document number and license plate so if you specifically want exclusive-or, you need to s/and another check onto that.#2017-03-0621:59joshjonesSimilar question / answer from 5 days ago https://stackoverflow.com/questions/42530452/clojure-spec-validating-contents-of-maps#2017-03-0708:14hkjelsthe docs say that or will return a map, but as far as I can see it's returning a vector with key/value
(s/def ::icon-params
  (s/cat :font (s/or :prefix ::font-prefix
                     :name ::font-name)
         :icon ::icon-name))
Am I using it wrong?
#2017-03-0708:16hkjelsOhh. I misread the doc. It returns a map-entry, not a map#2017-03-0710:40andrewboltachevHi. Does spec support recursive types?#2017-03-0710:57mpenetyes#2017-03-0711:05andrewboltachevcool#2017-03-0818:33nwjsmithIā€™ve really enjoyed working with spec the past few months, but there is a problem thatā€™s crept up again and again: specs are objects, and can only be manipulated w/ the closed set of operations provided by the clojure.spec ns#2017-03-0818:34nwjsmithTime and time again, Iā€™ve found myself wanting to manipulate the structure of a spec.#2017-03-0818:35nwjsmithBut the only way to do that is to parse the specā€™s form.#2017-03-0818:35nwjsmith(which is the motivation behind http://dev.clojure.org/jira/browse/CLJ-2112)#2017-03-0818:40nwjsmithIf specs were represented as data structures rather than as opaque objects, I think it would open up a lot of possibilities. Iā€™m sure there are reasons (performance and the alpha API status) that make it more desirable to have specs represented as they currently are. Hopefully somewhere down the line this kind of thing will be made, ahem, easier.#2017-03-0818:46tbaldridgeI agree, metaprogramming of specs is something I'd like to see explored. Pretty much the only missing part IMO, is the specs-for-specs thing you linked above.#2017-03-0819:08ghadiUnrelatedly, one thing that has been mentioned before was this scenario: You have a bunch of functions that are processing some map input. If you knew at the outset the specs of all the function, you'd be able to check everything before processing, Will this not repeatedly check specs for all qualified keys in the input map? Or is there some way to avoid that#2017-03-0819:19Alex Miller (Clojure team)why do you care?#2017-03-0819:20Alex Miller (Clojure team)if youā€™re doing this at dev or test time, then maybe donā€™t worry about it.#2017-03-0819:20Alex Miller (Clojure team)if youā€™re validating in production, then validate once at the entry point, and donā€™t validate in each step#2017-03-0819:20Alex Miller (Clojure team)or memoize s/valid?#2017-03-0819:22Alex Miller (Clojure team)@nwjsmith specs (as macros) capture their form for error reporting. if you are somehow manipulating the spec, the form is not going to track those changes. this seems bad?#2017-03-0819:24Alex Miller (Clojure team)is the stuff in CLJ-2112 (as a Rush fan, I canā€™t really say how happy I am this happened to be ticket 2112, plus it is trivial for me to remember it) sufficient? If not, whatā€™s missing? Note that you can start with a spec form, conform it to data, modify the data, then unform it back to a spec form#2017-03-0819:26Alex Miller (Clojure team)for that matter, you donā€™t even need the initial form - just build the conformed data, then unform#2017-03-0819:37nwjsmithYes, that would be sufficient. I hadnā€™t thought about the form capturing, which would make manipulating specs-as-data ridiculous.#2017-03-0819:38nwjsmithWhat if there was no form capturing though? And spec just provided terrible error messages?#2017-03-0819:38nwjsmithtroll#2017-03-0819:40nwjsmithI suppose I'll sit here waiting for CLJ-2112 to drop, soothed by the sounds of Geddy Lee and Neil Peart.#2017-03-0819:44ghadi@alexmiller that makes sense, thx#2017-03-0820:20Alex Miller (Clojure team)@nwjsmith well a still possible option is to provide function-level entry points for specs that did not have the ability to use forms in explain (or rather youā€™d have to supply those if you wanted them)#2017-03-0820:20Alex Miller (Clojure team)I guess in some ways the functions under the existing macros are that, although they are not designed to be public#2017-03-0820:23nwjsmithI think being able to conform, manipulate, and unform the spec forms should be enough. It would be nice to maintain the behaviour of explain through such a manipulation.#2017-03-0821:20weidoes anyone use arohner/spectrum and recommend it?#2017-03-0821:50Alex Miller (Clojure team)@nwjsmith I think you would retain that as youā€™re going back through to the form#2017-03-0821:51Alex Miller (Clojure team)@wei I saw his talk at the conj on it and was pretty impressed. it was unclear whether development on it was going to actively continue though#2017-03-0822:25arohnerIt is being developed, but Iā€™m very busy#2017-03-0822:26arohnerIā€™m working through filter, and I have a few hundred lines that havenā€™t been pushed yet#2017-03-0822:26arohnerbut yeah, itā€™s not ready for real use yet#2017-03-0822:57wei@arohner thanks for working on it, seems like a lot of people are excited about the prospect of static analysis at compile time#2017-03-0822:57weiif it works, it would be amazing!#2017-03-0910:23ilevdCan I use fdef for validation in runtime? It seems it works only for :args not :ret if use instrument#2017-03-0915:12jjttjjIs there an easy way to disable clojure.spec assertions thoughout a lein project? Right now I have
:global-vars {*assert* false
                clojure.spec.compile-asserts false
                }
in project.clj but it's not working
#2017-03-0915:29rauh@jjttjj I believe it's :jvm-opts ["-Dclojure.spec.compile-asserts=false"]#2017-03-0916:15bbrinck@ilevd Instrumentation only checks :args, not :ret or :fn. Itā€™s a common point of confusion. :ret and :fn specs are only with generative tests#2017-03-0916:19bbrinckIā€™d personally prefer if instrument checked ret and fn (at least optionally), but the authors of spec would say that the point of instrument is to check that you are calling the function correctly, whereas the point of generative testing is to ensure it is implemented correctly.#2017-03-0916:21bbrinckIn my experience, this means that I often donā€™t write ret or fn specs for functions that canā€™t be tested with generative testing, which is a shame, because I miss out on the documentation benefits of spec for these functions.#2017-03-0916:36devthany way to remove from the registry-ref atom? not seeing a way to get at it in https://github.com/clojure/clojure/blob/master/src/clj/clojure/spec.clj#2017-03-0916:41schmeedevth http://dev.clojure.org/jira/browse/CLJ-2060#2017-03-0916:41devthah, thanks#2017-03-0917:43peejaWhat's the canonical way to spec that a string matches a regex (in the #"" sense, rather than the clojure.spec sense)?#2017-03-0917:45Alex Miller (Clojure team)there is an email example in the guide https://clojure.org/guides/spec#2017-03-0917:46peejaAh, so there is! Thanks.#2017-03-0921:38bbloomfor those folks who are attempting to meta-program specs (@nwjsmith, @tbaldridge, @ghadi), iā€™m very curious to hear about your use cases#2017-03-0921:40bbloom(i love me some good metaprogramming!)#2017-03-0921:41tbaldridgeIn my case, generating JSON schema (for use with swagger) from pre-defined specs.#2017-03-0921:41tbaldridgeAlso, generating Datomic schemas from spec.#2017-03-0921:42bbloomok, so both to AND from spec use cases#2017-03-0921:42tbaldridgeAlso decomposing specs into logic/datalog, so I can say "find me all ::people in this DB"#2017-03-0921:45nwjsmith@bbloom hereā€™s an example that @peeja came up with at work yesterday: I want to have a spec for an entity map (e.g. (s/def ::widget (s/keys :req [::id ::name]))). Letā€™s say I only have a function that deals with entitiesā€™ ::names. Itā€™d be nice to take the ::widget spec and make itā€™s ::id into an :opt key. So that the name-handling fnā€™s spec could be (s/fdef munge-name :args (req-only ::widget [::name]) ā€¦#2017-03-0921:46bbloom@nwjsmith whatā€™s wrong with (s/fdef munge-name :args (s/keys :req [::name])) ?#2017-03-0922:28waffletowerI am very interested in meta-programming specs and came here to ask about spec reflection actually#2017-03-0922:30waffletowerFor my application I would like to introspect on specs to dynamically create custom generators#2017-03-0922:33waffletowerAs far as reflection I didnā€™t get past this:
user> (s/def ::museum string?)
:user/museum
user> (def as (s/keys :req [::museum]))
#'user/as
user> (type as)
clojure.spec$map_spec_impl$reify__13776
user> (pprint as)
#object[clojure.spec$map_spec_impl$reify__13776 0x10475684 "
#2017-03-0922:36tbaldridge@waffletower use s/form#2017-03-0922:37waffletowermany thanks!#2017-03-0922:38bbloomso the reflection use case makes tons of sense to me - especially for doing stuff like generating docs w/ cross reference links or whatever#2017-03-0922:41schmeecan someone enlighten me: why are specs macros instead of data?#2017-03-0922:42bbloomitā€™s mostly about error messages#2017-03-0922:42schmeecause now the accepted thing to do seems to be ā€œuse form, transform that and then conform and unformā€ which seems backwards#2017-03-0922:42bbloomfunctions donā€™t have enough reflective capability to provide good messages with predicate names, etc#2017-03-0922:42waffletowerI am interested in programmatically generating specs, and my simple example caused the s/keys macro to throw:
(def as-req [::museum])
#'user/as-req
user> (def as (s/keys :req as-req))
CompilerException java.lang.IllegalArgumentException: Don't know how to create ISeq from: clojure.lang.Symbol, compiling:(*cider-repl fulfill*:253:15) 
user> (macroexpand '(s/keys :req as-req))
IllegalArgumentException Don't know how to create ISeq from: clojure.lang.Symbol  clojure.lang.RT.seqFrom (RT.java:547)
#2017-03-0922:48waffletowerCould use another macro I guess#2017-03-0922:48waffletowerunless I am making a silly mistake I canā€™t see#2017-03-0922:48bbloomi hesitate to suggest this, but....#2017-03-0922:49bbloomeval exists on the jvm side šŸ˜›#2017-03-0922:50Alex Miller (Clojure team)@waffletower this won't work right now#2017-03-0922:50Alex Miller (Clojure team)You can eval a form or use a macro#2017-03-0922:51waffletowerThanks Alex#2017-03-0922:51Alex Miller (Clojure team)The discussion in the back channel would let you start from a map and then unform with spec form specs back to a spec#2017-03-1008:25ikitommi@tbaldridge you are doing json schema? here too. I think there are ~4 people doing the same now. Should we do join efforts somehow?#2017-03-1011:14Yehonathan SharvitIs it a good idea to use spec for validating data before inserting it into a mongodb document?#2017-03-1011:15Yehonathan SharvitThe problem I have is that spec is ā€œopenā€ and we have been told by Rich that it is a bad idea to have ā€œclosedā€ specs#2017-03-1011:15Yehonathan SharvitBut I need to make sure I catch typo errors like fristname instead of firstname#2017-03-1011:19bronsaif your firstname key is required spec will still catch it as a missing required key#2017-03-1011:20Yehonathan SharvitYes but in my case itā€™s an optional key#2017-03-1011:59yonatanel@viebel What's the behavior you're looking for?#2017-03-1012:00Yehonathan SharvitIā€™d like s/valid? ::my-spec to fails for {:fristname ā€œJoā€ :age 10} but to succeed for {:firstname ā€œJoā€ :age 10}#2017-03-1012:01Yehonathan Sharvit(s/def ::my-spec (s/keys :opt-un [::firstname ::age])) will not catch this error#2017-03-1012:03yonatanelMaybe roll your own using clojure.core/keys#2017-03-1012:04Yehonathan SharvitMy question is not technical but philosophical#2017-03-1012:05Yehonathan SharvitI mean: is it good practice to forbid keys?#2017-03-1012:05Yehonathan SharvitIn a lot of occasions Rich and @alexmiller recalls us that specs should be "open"#2017-03-1012:08yonatanelForbidding keys will inhibit forward compatibility, mixing keys e.g for metadata etc.#2017-03-1012:12yonatanelIn your case, I would at least select-keys and dissoc nils before persisting data.#2017-03-1012:38Alex Miller (Clojure team)You can catch invalid keys without writing the exclusion into the spec#2017-03-1012:52Yehonathan Sharvithow?#2017-03-1012:52Yehonathan Sharvit(In my case itā€™s a deeply nested map)
#2017-03-1013:46Alex Miller (Clojure team)You can write code, admittedly more challenging with nesting#2017-03-1014:58devth@alexmiller hey heads up, i think there's a typo in the blog post http://blog.cognitect.com/blog/2016/8/24/combining-specs-with-and. (s/conform (s/and int? even? #(> % 1000)) 1111) evals to :clojure.spec/invalid, not 1111#2017-03-1015:00Alex Miller (Clojure team)@devth indeed, although that should have just been an even number#2017-03-1015:00Alex Miller (Clojure team)will fix#2017-03-1015:03Alex Miller (Clojure team)fixed!#2017-03-1016:15yonatanel@viebel If you write a closed keys predicate for each parent key and combine with s/keys, it will drill down the nested map. (s/and (s/keys ...) your-closed-keys-pred?)#2017-03-1017:41joshkhbit of a newbie question about spec... i'm currently using gen/generator to generate some maps with UUIDs. i'd then like to generate another set of maps that will contain vectors of those UUIDs. i can generate both sets of maps and then assoc the UUIDs from one set to the other but i'm wondering if there's a better workflow for this?#2017-03-1017:49not-raspberryLike {:a #uuid "sth", :b #uuid "sth-else"} and {:sths [#uuid "sth", #uuid "sth-else"]}?#2017-03-1017:56joshkhyes, i think so. my example is a collection of "items" with UUIDs {:type :item :uuid #someuuid} and then another spec for a "package" containing random items {:type :package :contents [#someuuid #someotheruuid]}#2017-03-1017:58not-raspberryi'm no expert but I'd just write a function for that transformation#2017-03-1017:58not-raspberryand have specs for both data structures#2017-03-1017:59joshkhyes, i have specs for both data structures. after generating samples of both i can just reassoc contents from one collection to the other#2017-03-1018:00joshkhi was using this blog post as an example but i'm not sure i understand gen/fmap https://product.gomore.com/end-to-end-integration-tests-with-clojure-spec-d4a48cbf92b5#.kpvs9132v#2017-03-1018:06gfredericksgen/fmap lets you transform the value that a generator generates#2017-03-1018:07gfredericksso e.g., if you have a generator integers g, then (gen/fmap str g) is a generator of stringified integers#2017-03-1018:10joshkhoh, nifty#2017-03-1019:46Alex Miller (Clojure team)I have been remiss in not mentioning that I will be doing clojure.spec training before the Conj in Portland March 29th - you can register independently from Clojure/west (although why would you want to?) http://2017.clojurewest.org/training/#2017-03-1019:47Alex Miller (Clojure team)https://ti.to/cognitect/clojure-west-2017#2017-03-1022:10peejaIs there something like s/multi-spec that doesn't use the extra machinery of a multimethod that would be more appropriate for a situations where the cases are a closed set? I don't need to be able to extend the list of cases easily.#2017-03-1022:11peejaI could use s/or, but that doesn't dispatch by looking at the value being conformed#2017-03-1022:11peejawhich maybe is fine?#2017-03-1022:12peejaIt seems like dispatching on a tag would be faster in general when it's an option, but since it's a small, close set of variants, maybe it just doesn't matter much#2017-03-1022:23bbloomjust use multi-spec#2017-03-1022:24bbloomyou can put ^:private on your multimethod if you want#2017-03-1022:24bbloomthatā€™ll keep it closed#2017-03-1022:26yonatanel@peeja It depends on what's bothering you with the multi-spec solution. You can't do much about the style. The set of options is kinda closed and you'll get a "no method" exception on unknown dispatch values. In addition you could define a spec for the dispatch key using a set #{...} to get a nicer explanation for invalid values.#2017-03-1022:28peejaIt's not so much that I need to enforce that it's closed (I do have a spec on the key) as much as a multi-method is grammatically pretty heavy and I don't need the advantages it offers. Having written it out, though, the s/or actually works pretty well for me here.#2017-03-1022:30bbloomi do think a syntactic shortcut for the common case of single-keyword dispatch may be warranted#2017-03-1022:31bbloomitā€™s really not too bad tho#2017-03-1116:12eggsyntaxAnyone know offhand whether there's a way to set a random seed for gen/sample? I saw that you can do so in clojure.test.check/quick-check, but couldn't find mention of it re: sample and generate. I'd like to reproducibly generate some seed data.#2017-03-1116:18gfredericks@eggsyntax see this answer in #test-check: https://clojurians.slack.com/archives/test-check/p1488820803000255#2017-03-1116:18eggsyntaxCool, thanks šŸ™‚#2017-03-1116:19eggsyntaxYeah, that should do me quite nicely.#2017-03-1116:20gfredericksthe test.check API was designed for reproducibility when running tests, not for ad-hoc generator use like spec seems to encourage#2017-03-1116:21gfredericksadding an arity to gen/generate would presumably fix that#2017-03-1116:21eggsyntaxYeah, that totally makes sense -- I just happen to be generating some seed data from a spec.#2017-03-1116:23eggsyntaxIt's essentially a one-off, so I'm perfectly happy to abuse non-public API calls šŸ˜‰#2017-03-1218:36dottedmagI am trying to use spec to validate/destructure external data (JSON). This data reuses key names throughout the various entities: there are keys named N in various parts of data meaning "name", "route number" and "allowed number of things", and they should have different specifications. How do I specify that in spec?#2017-03-1218:39dottedmagI can do it via :req-un in separate namespaces, but I'm not thrilled by the necessity to create ~15 namespaces, one for every entity kind.#2017-03-1218:47yonatanel@dottedmag You don't have to actually create the namespace, just fully qualify the specs (unless you want to alias namespaces, but you don't have to)#2017-03-1218:48dottedmagd-oh!#2017-03-1218:48dottedmag@yonatanel Thanks!#2017-03-1300:14nickikWhy can I not say (spec/and (spec/keys :req-un [::id]) (spec/keys :req-un [::old-id]))?#2017-03-1300:15nickikHow do I combine two map specs?#2017-03-1300:15seancorfieldspec/merge?#2017-03-1300:16nickikThat sounds correct.#2017-03-1300:16nickikI did not know this existed#2017-03-1300:17seancorfieldThere are some examples about a third of the way thru the Guide https://clojure.org/guides/spec#2017-03-1300:17nickikI scrolled threw this multiple times, but I did not see merge#2017-03-1300:18nickikThank you.#2017-03-1300:19seancorfield(there's only three mentions of merge and it's easy to miss squished in between a couple of big code blocks)#2017-03-1300:20nickikWorks like a charm šŸ™‚#2017-03-1300:36nickikIf I have something like this: (spec.test/check `my-fn)#2017-03-1300:36nickikWhere should I put this so it will be run with other unitests?#2017-03-1300:36nickikShould I just put it like that into the test namespace?#2017-03-1300:37seancorfieldYou can run the result thru spec.test/summarize (I think) and you can write a deftest around that to assert that the summary shows success.#2017-03-1300:38seancorfieldHowever, one thing to consider is that generative tests are not "fast" like most unit tests should be -- so you might want to think about having a separate task to run generative tests, so that your unit tests stay nice and fast.#2017-03-1300:42nickikOk. So I have to write a deftest myself. Thats fine given how long it takes to run them.#2017-03-1300:45seancorfieldsummarize-results -- I had to go look it up.#2017-03-1321:43dpsuttonquestion about tooling: since spec is intextricably linked to 1.9-alphas no tooling can exist for repl's until they are separated. To include spec otherwise would necessarily require hijacking the specified clojure version to 1.9, which would possibly violate people's expectations. Does this sound true and do I have any misconceptions?#2017-03-1321:44dpsuttonI'm working on spec tooling in CIDER and I don't know how i could begin without hijacking the clojure version of the project that CIDER is interacting with#2017-03-1321:47ghadiI'm not sure I follow your base assumption. Can tooling check for the existence of spec?#2017-03-1321:48dpsuttonwell in clojure 1.8, clojure.spec doesn't exist right?#2017-03-1321:48dpsuttonah so it would have to be wrapped in lots of conditional checks?#2017-03-1321:49dpsuttoni was just wondering about
(ns nrepl-spec
   (require [clojure.spec :as s])
wouldnt' this throw an error on any clojure < 1.9?
#2017-03-1321:53dpsutton
1. Unhandled java.io.FileNotFoundException
   Could not locate clojure/spec__init.class or clojure/spec.clj on
   classpath.
#2017-03-1321:54dspiteselfi havent tried it, but https://github.com/tonsky/clojure-future-spec#2017-03-1321:54ghadiinstead check for the existence of a var#2017-03-1321:56ghadithe spec namespace is automatically loaded in clojure 1.9#2017-03-1321:56ghadithat may change, we'll see#2017-03-1321:56dpsuttonyeah that makes the problem worse, right?#2017-03-1321:56ghadi(at least when running a clojure repl in 1.9)#2017-03-1321:57ghadiI guess I don't get what the problem is#2017-03-1321:57dpsuttoni need middleware to lookup specs to provide a spec browser#2017-03-1321:57dpsuttonin cider#2017-03-1321:57ghadiyou're gonna need conditionals#2017-03-1321:57dpsuttonbut if you jack in from a clojure 1.7 project it's gonna blow up#2017-03-1321:58ghadiwell yeah the middleware needs to not be dumb#2017-03-1321:58dpsuttonwhen they decouple spec from 1.9 i can just load spec in any clojre 1.7 > which is what cider supports#2017-03-1321:58ghadisure#2017-03-1321:58dpsuttonbut until then i can't even "talk" about it, i thought#2017-03-1321:58dpsuttonlike the require statement#2017-03-1321:59ghadi
(if (spec-available?)
  (do-cider-stuff)
  :pass)
#2017-03-1321:59ghadia common thing is to try/catch around require#2017-03-1321:59dpsuttoni guess that will have to be the case anyways, as I'm sure spec versions will be mismatched#2017-03-1322:00dpsuttonthanks for the feedback ghadi#2017-03-1322:00ghadispec versions?#2017-03-1322:00dpsuttoni'm assuming when it's released, CIDER will have clojure.spec 1.8 and someone might be using 1.4? or will it be like clojure.string or set where it's built in#2017-03-1322:00dpsuttoni'm not really clear about that#2017-03-1322:02dpsuttondidn't know if it's something that will be specified in the project file with a version or if it will be more baked into the language like set and string, etc#2017-03-1322:02ghadimore conditionals#2017-03-1322:02ghadišŸ˜†#2017-03-1322:02dpsuttonhaha hooray#2017-03-1322:03ghadithere isn't a guarantee that clojure.spec will be released outside of 1.9 -- it's been mentioned before, but until you see a commit SHA, can't make assumptions#2017-03-1322:04dpsuttoni sat next to stuart at dinner at the conj and he said that's what was holding up the release of 1.9. I was going off of that but i understand it's not official#2017-03-1322:04tbaldridgeThis does bring up a interesting point in the discussion of middleware and nrepl. As long as tools insist on installing new deps into the repl there's going to be problems like this.#2017-03-1322:04dpsuttonsorry not stuart but stu#2017-03-1322:04ghaditooling should degrade gracefully#2017-03-1322:05dpsuttonnot if i'm writing it#2017-03-1322:05dpsuttonšŸ™‚#2017-03-1322:23ghadi@tbaldridge you using cider or inf-clojure?#2017-03-1322:25tbaldridgeI use Cursive, you insensitive clod šŸ˜›#2017-03-1322:26ghadihehheh#2017-03-1322:26dpsuttonhaha#2017-03-1322:26tbaldridgebut I do wish all clojure tooling could move away from having middleware in the JVM repl#2017-03-1322:27ghadiwhat are your reasons for that?#2017-03-1322:27ghadii can think of several, just curious#2017-03-1322:28tbaldridgeit just causes a lot of problems, for no real need. The dependencies used by middleware muck with the resolution order of your repl. Stuff like this spec discussion are due mostly to not being able to use spec in tooling because it might be used in a version of clojure that doesn't support it#2017-03-1322:28tbaldridgeand nrepl is too complex for something so simple, lol#2017-03-1322:29tbaldridgeall I needed was netcat + a tcp port...and now I have a RPC server inside my program#2017-03-1322:31tbaldridgeI guess it's half a rant against middleware and half about nrepl#2017-03-1323:05seancorfieldI must admit, I too find it frustrating to need different dependencies in a project based on what a particular editor or other tooling needs. CIDER is great but with the ā€œregularā€ stack of DEV middleware it takes ages to start an nREPL server. Iā€™ve recently switched to Atom/ProtoREPL and that requires different libraries in play. For non-DEV work, we use the Socker Server but thatā€™s ā€œtoo lightā€ for useful tooling (unless tooling was rewritten to assume a more neutral Clojure environment and just expected to execute ā€œregularā€ Clojure over-the-wire).#2017-03-1323:05seancorfield(at the risk of continued a somewhat off-topic discussion!)#2017-03-1323:39ghadimight be off-topic for spec but not in general - being parsimonious about dependencies is very important. gem install hairball applies to our community too#2017-03-1323:43seancorfieldIf you Bing gem install hairball it brings up this as the top result: https://rubygems.org/gems/hairball/versions/0.1.0 ā€” which in turn links to this: http://www.confreaks.com/videos/860-railsconf2012-keynote-simplicity-matters šŸ˜‚#2017-03-1323:43seancorfieldI assume you knew that was a Rich Hickey reference? šŸ™‚#2017-03-1400:10tbaldridgeThat's awesome. Although I'm disappointed that it's not a gem that downloads the most recent of every other gem...#2017-03-1401:58ghadigem install hairball is a quote from simple made easy#2017-03-1400:12cflemingFWIW Cursive doesnā€™t use any middleware at all.#2017-03-1400:13cflemingIā€™m a big fan of loading the server-side code I need over the REPL connection when itā€™s opened.#2017-03-1400:14cflemingBut tooling like CIDER that uses the REPL more than Cursive does are very prone to this sort of problem, right.#2017-03-1400:15tbaldridgeNever thought of it that way, static analysis saves you from having to do a lot of the server side stuff.#2017-03-1400:15cflemingI donā€™t think thereā€™s a solution other than saying ā€œthis functionality wonā€™t work unless youā€™re using Clojure 1.9+"#2017-03-1400:15cflemingYeah#2017-03-1400:16cflemingGenerally I try to make sure that Cursive will work independently of the version of Clojure the user is using.#2017-03-1400:16cflemingIā€™m actually looking at some REPL improvements right now, and if I want to use EDN then that means a Clojure 1.5+ project. Fortunately I know which version the user is using and can warn them, but itā€™s a little ugly.#2017-03-1400:18cfleming@tbaldridge BTW pure socket REPLs are on the way.#2017-03-1400:18cfleming(in Cursive)#2017-03-1400:19cflemingBut again, there I have to check that the user is using 1.8+#2017-03-1402:40devthis there a way to grab the :ret spec out of an fdef? trying to validate the return value of a fn in a test#2017-03-1403:33Alex Miller (Clojure team)Yep, just call get-spec on the symbol to get the function spec#2017-03-1403:34Alex Miller (Clojure team)That spec implements ILookup and can be invoked with :ret#2017-03-1403:35Alex Miller (Clojure team)So, (:ret (s/get-spec 'full/symbol))#2017-03-1513:32devthexcellent. thanks.#2017-03-1603:33englishWhat's a good way to spec a sequence with a static subsequence? For example, if I want to spec a seq that contains 1, 2, 3 somewhere in the middle, is there a better way to do it than the following?
(s/conform (s/cat :pre (s/+ any?)
                  :one-two-three (s/& (s/+ any?) #(= [1 2 3] %))
                  :post (s/+ any?))
           [:foo :bar 1 2 3 :baz :qux])
This works, but it feels like I'm abusing s/&.
#2017-03-1605:19Alex Miller (Clojure team)@english doesnā€™t seem like you need the s/& at all#2017-03-1605:21Alex Miller (Clojure team)
(s/conform (s/cat :pre (s/+ any?)
                  :one-two-three (s/cat :1 #{1} :2 #{2} :3 #{3})
                  :post (s/+ any?))
           [:foo :bar 1 2 3 :baz :qux])
#2017-03-1605:21Alex Miller (Clojure team)not very pretty, but that should work a little more directly#2017-03-1605:23Alex Miller (Clojure team)it will gen automatically too whereas the prior one will not#2017-03-1609:46thhellerI started writing a Language Server for Clojure(Script) https://github.com/thheller/shadow-devtools/issues/2#2017-03-1609:46thhellerhttps://cloud.githubusercontent.com/assets/144930/23987578/8b4bb1fa-0a2a-11e7-8bfb-7caa0b77dcc8.png#2017-03-1609:46thheller(some background on what a language server is in the gh issue above)#2017-03-1609:46thhellernaturally I want to leverage clojure.spec for this task#2017-03-1609:47thhellerie. if I have (defn x :foo) in a file or REPL I get an exception with Call to clojure.core/defn did not conform to spec and ex-data containing the usual information#2017-03-1609:48thhellerthat information however is not enough for me to provide good highlighting for the error that occurred since it contains no source information#2017-03-1609:51thhellerthe question is whether or not this is something the error should/could contain or not? I imagine all tools will require it somehow as the given exception doesn't tell you anything about the actual location of the error.#2017-03-1609:52thheller@alexmiller you said you were looking into the error presentation details a while back, any progress on that?#2017-03-1609:53thhellerfor now even just the form that caused the fdef to fail would be a start#2017-03-1610:11thhellerfound :clojure.spec/args but neither that list or the items in there contain any meta data from the reader#2017-03-1610:37lwhortonis it possible to use spec to generatively test stateful fns? for example, if I have a fn that takes an in and out core.async channel, where it performs some logic on the values in the in channel (when they show up in the future), and perhaps puts values on the out channel, can I generate any meaningful tests?#2017-03-1610:55lwhortonThere are a few attempts it seems: https://goo.gl/DLEA1K https://goo.gl/F9E4nl https://goo.gl/ESiLvz https://goo.gl/J5lKY4 and a neat resource from an Erlang implementation: https://goo.gl/lrnFB4 , but Iā€™ve been unable to dig up any recent available libs or discussions on the matter#2017-03-1612:28Alex Miller (Clojure team)@thheller yeah I made some progress on it but got blocked by not having access to the root value in the explain-data. I have a ticket and patch pending about that and had a brief conversation about it with Rich last week. I believe there is a separate ticket re spec source meta, certainly there is some overlap with the idea of spec doc meta.#2017-03-1612:33thheller@alexmiller ah thx, found the JIRA issue. will try with the patch and see if I can do what I have in mind.#2017-03-1612:34thhelleralthough it still doesn't contain the original form, only the args#2017-03-1612:45thhellermaybe this catch https://github.com/clojure/clojure/blob/2ff700ede3866f97d7b1f53342e201df94384aee/src/jvm/clojure/lang/Compiler.java#L6820#2017-03-1612:46thhellershould catch the ExceptionInfo the macroexpand-check will produce#2017-03-1614:06Alex Miller (Clojure team)hmm, it used to but I think when macroexpand-check was refactored this didnā€™t get changed along with it#2017-03-1614:06Alex Miller (Clojure team)macroexpand-check used to throw IllegalArg#2017-03-1614:06Alex Miller (Clojure team)I will take a look at that#2017-03-1614:08Alex Miller (Clojure team)https://github.com/clojure/clojure/commit/b3d3a5d6ff0a2f435bb6a5326da2b960038adad4#2017-03-1614:09Alex Miller (Clojure team)should have been changed with this commit#2017-03-1614:09Alex Miller (Clojure team)thatā€™s definitely a bug#2017-03-1614:12Alex Miller (Clojure team)so that happened in 1.9.0-alpha12#2017-03-1614:24thheller@alexmiller maybe we could add the last known form to CompilerException(String source, int line, int column, Throwable cause, Object form) as well#2017-03-1614:24Alex Miller (Clojure team)Iā€™ll look at that#2017-03-1614:24Alex Miller (Clojure team)http://dev.clojure.org/jira/browse/CLJ-2128#2017-03-1614:26thhellerthere are many places where a compiler exception is thrown and the form is known#2017-03-1614:26Alex Miller (Clojure team)oh, thatā€™s not there already. that needs to be a separate idea then.#2017-03-1614:27Alex Miller (Clojure team)that needs some thought. when youā€™re throwing arbitrary data like that you can introduce some weird problems#2017-03-1614:27Alex Miller (Clojure team)I have been burned by that a number of times over the years#2017-03-1614:28Alex Miller (Clojure team)I hate that the only exception in Clojure that can attach location info is CompilerExceptionā€¦ grr#2017-03-1614:30thhellerhehe a 2nd constructor adding the form where possible seems like the simplest non-breaking change#2017-03-1614:30thhellernot sure how the CLJ compiler/reader is implemented but if the form contains lazy things that will indead cause strange errors#2017-03-1614:33thhellerI will attempt to port the clojure.core.specs over to CLJS soon, it would be amazing if the exceptions generated on errors where somewhat similar#2017-03-1614:35Alex Miller (Clojure team)well when you have a form, itā€™s been read and should not be lazy (unless perhaps it was generated by a prior macro?)#2017-03-1614:36Alex Miller (Clojure team)do you really want the form or do you want a string representation of the form?#2017-03-1614:37thhellerthe form with metadata would be perfect#2017-03-1614:37Alex Miller (Clojure team)yeah, thatā€™s what I was wondering#2017-03-1614:37thhellerwith that I can get the string if I need it#2017-03-1614:38thhellerI have never actually worked with the Clojure analyzer, only the tools.analyzer or cljs.analyzer. so I need to figure out how the CLJ metadata looks#2017-03-1614:38thhellerbut I take everything I can get at this point#2017-03-1614:39Alex Miller (Clojure team)well, nothing in this area is going to change quickly :)#2017-03-1614:39thhellerhehe no doubt but the change to CompilerException is simple enough that I can try in a branch#2017-03-1614:39thhellerand report if thats actually as useful as I think it is#2017-03-1614:41Alex Miller (Clojure team)please add info to http://dev.clojure.org/jira/browse/CLJ-2129 if you obtain some#2017-03-1614:41thhellerwill do#2017-03-1615:44hiredmanforms can be very large, at the repl you just type (/ 1 0) and that fails, and it seems like a good idea to report the form, but I really have no need to see the macroexpansion of a 100 line go block#2017-03-1620:05timgilbertSay, is there any way to get (s/keys) to validate a map with non-namespaced keywords? Trying to add some validation around an external library I don't control#2017-03-1620:06timgilbertOh, sorry, just noticed the section in the guide around :req-un, will read that more carefully#2017-03-1621:26zaneHow does Transit intersect with Clojure Spec (if at all)?#2017-03-1621:57ghadii don't think it does directly#2017-03-1621:58ghadiHowever, Transit is a great way to pass around namespaced keys, which is the norm for things used by clojure.spec#2017-03-1714:17hkjelsIā€™d like to simplify this API, but Iā€™m not sure how to create both a ā€œvariableā€ and a function-reference from the same parameter https://gist.github.com/hkjels/1656508f7666ce6fa39699522743505f#2017-03-1714:17hkjelsAny ideas?#2017-03-1714:22hkjelsI might have to use a macro I guess#2017-03-1714:25Alex Miller (Clojure team)if you start from the symbol, itā€™s easy enough to get the meta from there#2017-03-1714:26Alex Miller (Clojure team)(meta (resolve s))#2017-03-1714:26bronsamight not be that easy from cljs#2017-03-1714:27Alex Miller (Clojure team)ah, true, didnā€™t notice that#2017-03-1714:30hkjelsyeah, unfortunately that doesnā€™t work#2017-03-1718:39waffletowerJust starting with custom generators, and I have only looked at the underlying code in clojure.test.check.generators a bit, and didnā€™t notice if there was a more straight-forward way to use custom generators without input arguments. clojure.test.check.generatorā€™s own uuid is an interesting example, but it relies on an internally passed random number (while ignoring size) and defines itself with namespace internal functions.
(s/def ::saddle (s/with-gen
                (s/and string? (partial re-matches saddle.regex))
                #(gen/fmap (fn [_] (zero-arity-gen)) (gen/return nil))))
#2017-03-1719:42gfredericks@waffletower this is generally discouraged since it sidesteps most of the test.check orchestration -- it should be possible for you to write any generator you need using the built in combinators#2017-03-1719:43gfredericksIs your specific problem about generating a string that matches a regex?#2017-03-1720:05waffletowerthe generator doesnā€™t need shrinking as with gen/uuid, and it currently uses RNG internally. I could use RNG from the generator namespace instead ā€” which seems to suggest I could continue using gen/fmap#2017-03-1720:10gfredericksyou can opt-out of shrinking by wrapping with gen/no-shrink#2017-03-1720:10gfredericksusing the randomness from test.check means that at least your generator (and any composition including it) is deterministic#2017-03-1720:11gfredericks@waffletower if you'd like to share the details of the generator you already have I could suggest how to rewrite it#2017-03-1810:42yonatanelSo I made this abomination. Anyone finds this useful or really unnecessary? https://github.com/yonatane/spec-grind/blob/master/test/spec_grind/grind_test.clj#2017-03-1815:08tbaldridge@yonatanel not a super bad idea, but add an additional function to all your conformers (`s/conform can take two args), so that we can unconform as well.#2017-03-1815:09tbaldridgehttps://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec/unform#2017-03-1816:37yonatanel@tbaldridge I don't think there's a need for unform here. I use a few predefined conformers for coercion and they yield either a coerced value or ::s/invalid, and you can chain them with s/and without using unform.#2017-03-1817:16yonatanelUnless you need the original value which wasn't a goal for this lib. Perhaps I could add it as meta of the coercion result. I never used meta so I don't know if it works for nil and primitives. Anyway it needs to go into a better thought out lib than this.#2017-03-1911:11Yehonathan SharvitWhen one uses clojure.spec to validates the body of an API request, how can we returns user-friendly errors in case the body doesnā€™t conform?#2017-03-1911:11Yehonathan SharvitI mean s/explain-str is nice for developers but not for users!!!#2017-03-1911:23yonatanel@viebel What about using s/explain-data and formatting it however you want?#2017-03-1911:30Yehonathan SharvitHow would you do that @yonatanel ?#2017-03-1911:30Yehonathan SharvitLetā€™s take a simple example of a signup request#2017-03-1911:30Yehonathan Sharvithttp://app.klipse.tech/?eval_only=1&amp;cljs_in.gist=viebel/7edd4252b6f71c9894bc979c550b359d#2017-03-1911:41yonatanelI guess it's the author responsibility to define higher level predicates that can be later examined more easily than (>= (count %) 8)#2017-03-1911:42yonatanelSomething like long-enough?#2017-03-1911:43Yehonathan SharvitNice idea. How would you manage the mapping between predicates/specs and a string that describe them?#2017-03-1911:44yonatanelNot sure right now, but I think you are right in that explain is not really meant for public data-oriented apis.#2017-03-1911:45Yehonathan SharvitIt means that I have to manually conform the data? šŸ˜ž#2017-03-1911:46yonatanelI'm not sure what you mean by that and how it's related to the error message#2017-03-1911:46Yehonathan SharvitI mean that I can write code that checks if the password is long enough and returns an appropriate error message#2017-03-1911:47Yehonathan SharvitIā€™d need to write such a piece of code in addition to the specs definitions - which is quite disappointing#2017-03-1911:48yonatanelWould you like each predicate to optionally have an explanation string or something like that?#2017-03-1911:48Yehonathan SharvitMaybe#2017-03-1911:48Yehonathan SharvitBut I am curious to know if spec is designed for that#2017-03-1912:13yonatanelYou kinda have all the building blocks for that, and it might be good to separate error message from spec itself. You could have multiple mappings from a spec to error message, one for public api, another for a kafka consumer etc, all using the same registered (s/def ::long-enough ...) spec.#2017-03-1917:16gklijsI still have to go about trying spec out for myself, but for a similar usecase I made a simple validation function, returning a list with either only true, or false and an end-user friendly error message.#2017-03-1917:16Yehonathan SharvitCould you be more specific @gklijs#2017-03-1917:17Yehonathan SharvitHow would you update this code http://app.klipse.tech/?eval_only=1&amp;cljs_in.gist=viebel/7edd4252b6f71c9894bc979c550b359d ?#2017-03-1917:20gklijsWell, I wouldnā€™t know since Iā€™m not using spec yet, I just did it like this:
(defn valid-registration-map?
  [registration-map]
  (cond
    (not (map? registration-map)) '(false "Input is not a map")
    (not (contains? registration-map :username)) '(false "Input is missing the :username key.")
    (not (contains? registration-map :password)) '(false "Input is missing the :password key.")
    (< (count (:username registration-map)) 8) '(false "Username should have a minimal of 8 characters")
    (< (count (:password registration-map)) 8) '(false "Password should have a minimal of 8 characters")
    :else '(true)))
#2017-03-1917:45yonatanelIn spec you explain separately from validating#2017-03-1918:26dominicmWith forms, there's also the issue that if you validate the whole map, you can't indicate that there's an error with a particular key in the same way s/keys does (unless that's changed?)#2017-03-1918:39seancorfieldWe use explain-data and a map of symbols or forms to messages -- and a generic function that walks the explanations and finds matching messages. That way you can choose how generic or how specific the message is by how much of the form of the spec it matches. For example, if you just want a generic bad password message, just map 'password to the message. Otherwise map more pieces of the spec forms to different messages. #2017-03-1918:44Yehonathan Sharvit@seancorfield could you share a code snippet?#2017-03-1918:47seancorfieldI'm on my phone so, no. I'll try to remember tomorrow when I'm back at work. But this is what the Clojure/core folks have been encouraging us to do with spec: walk the explanation data and map it to messages or error codes or whatever. You have a lot of useful information there. #2017-03-1918:48seancorfieldThe path, the predicate, etc. #2017-03-1918:48Yehonathan SharvitThat looks very cool!#2017-03-1918:48Yehonathan SharvitIā€™ll ping you tomorrow @seancorfield !#2017-03-1919:31Yehonathan Sharvit@alexmiller any interesting insights of how to generate user-friendly error messages from spec. See ā«#2017-03-1920:48Alex Miller (Clojure team)Advice from Sean above seems right#2017-03-1920:48Alex Miller (Clojure team)Or bind a custom explain printer#2017-03-2002:13bbloom@gfredericks recalling out discussion about generators, monads vs applicatives, annoyances in generative testing, etc - iā€™ve finally gotten around to reading ā€œConstraint Propagationā€ by Guido Tack"#2017-03-2002:13bbloomsome snippets:#2017-03-2002:13bbloom> The generate-and-test solver is too inefficient to solve real-world problems, as it requires enumeration of all assignments. We now improve on this naive method by pruning the set of enumerated assignments using propagation.#2017-03-2002:14bbloom> A propagation-based solver interleaves propagation and enumeration (search).#2017-03-2002:15bbloomseems like the way generators defined now is the output of a single manual propagator pass - ie the hand written generators are just domains#2017-03-2005:17Yehonathan Sharvit@alexmiller what do you mean by ā€œbind a custom explain printerā€?#2017-03-2008:07yonatanel@viebel Maybe like how explain-str does it with with-out-str, but it doesn't sound right.#2017-03-2008:22yonatanelOk, got it (from spec source):
(def ^:dynamic *explain-out* explain-printer)

(defn explain-out
  "Prints explanation data (per 'explain-data') to *out* using the printer in *explain-out*,
   by default explain-printer."
  [ed]
  (*explain-out* ed))

(defn explain
  "Given a spec and a value that fails to conform, prints an explanation to *out*."
  [spec x]
  (explain-out (explain-data spec x)))
#2017-03-2011:16gfredericks@bbloom how much of this 200 pg paper do I need to read to know what you're talking about#2017-03-2012:54mpenetwould it make sense for (s/conformer f) to return :clojure.spec/invalid on exception?#2017-03-2012:55mpenetseems like I wrote this a few times already#2017-03-2012:57mpenetpaired with http://dev.clojure.org/jira/browse/CLJ-2115?page=com.atlassian.streams.streams-jira-plugin:activity-stream-issue-tab it would allow to skip a lot of boilerplate code#2017-03-2016:54stathissideriswhen defining :args in fdef is there a good way to enforce some relationship between the arguments, for example (fn [a b] (+ a b)) but (> a b) (contrived example)#2017-03-2016:54stathissiderisI have a hard time expressing this as a spec#2017-03-2016:56bbloom@gfredericks just like any paper like this, you can skip all the math šŸ˜› i didnā€™t read all of it, just the intro and some prose in the major chapters#2017-03-2016:59bbloom@gfredericks or you can rewatch sussmanā€™s talk from strange loop a few years back about ā€œwe really donā€™t know how to computeā€ which may be more accessible to the propagator idea. the only really extension beyond that is that if you combine propagation and search, you get a sound/complete constraint solver. think soduku: if you have badly made puzzle (for humans) that doesn't have enough info to solve by inferences, you can take guesses, and then make more inferences. if your guesses donā€™t pan out, you can backtrack and make different guesses.#2017-03-2016:59bbloomright now spec makes some inferences from stuff like s/and etc and composes together some generators, and those things are making guesses#2017-03-2017:00bbloomthat paper mentions that the propagators that provide inferences also act as the predicates in a generate and test system, but in spec, no new inferences occur after the initial construction of the generators#2017-03-2017:02bbloomnot sure if thereā€™s any way to capitalize on this insight in a spec context tho - but the goal would be to shrink the search space to reduce how many examples need to be generated only to be thrown out by predicates#2017-03-2018:03madstap@stathissideris You can use s/& to apply additional predicates to the conformed values of a regex spec.
(s/& (s/cat :a int? :b int?)
     (fn [{:keys [a b]}] (> a b)))
#2017-03-2018:24gfredericks@bbloom thanks, I'll noodle it a bit and get back to you#2017-03-2018:25seancorfield@viebel Slackbot just reminded me about sharing details of our explain mapping code ā€” nice šŸ™‚#2017-03-2018:26seancorfieldThere are two parts to it: one is a function that reduces the explain data to a vector of relevant pieces to match on, from most specific to least specific, and then for each spec, we have a map from symbols/forms to the specific error code or message.#2017-03-2018:28seancorfieldThe key is that we have two paths through the code, one for when :path is empty and one for when :path is non-empty. The latter provides more context (a top-level key, for example) so we get better error messages that way. With the former, we can sometimes only get generic errors, unless the predicates are sufficiently specific.#2017-03-2018:35seancorfieldWe look at :path and :pred (only), and only the first element of :path (at present). We turn the predicate into a canonical form by taking the first three subforms and removing % and the path element (e.g., given a :path of [:email] and a :pred of #(re-find email-regex %) weā€™d get (ā€˜re-find ā€˜email-regex) and a path of :email) and then build a sequence of possible matches [[:email ā€˜re-find ā€˜email-regex] [:email ā€˜re-find] [:email] [ā€˜re-find]] ā€” and weā€™d look in the map of forms to errors for each of those three in turn.#2017-03-2018:35stathissideris@madstap didnā€™t know about s/& - thanks, Iā€™ll check it out!#2017-03-2018:41seancorfieldSo a given map might be
(def ^:private update-photo-fail-codes
  {:photoid           11301
   [:photoid '->long] 11302
   :caption           11401
   [:caption 'seq]    11402
   [:caption '<=]     11403})
ā€” so 11302 is for photoid failing the ->long conforming predicate and 11301 is for ā€œotherā€ photoid failures (which is just required field missing in this case). 11402 is for caption being empty, 11403 is for caption being too long, and 11401 is for caption in general (in this case just ā€œmissingā€).
#2017-03-2018:50Yehonathan SharvitThanks @seancorfield#2017-03-2018:50Yehonathan SharvitI was hoping that clojure.spec would give something out of the box#2017-03-2019:09seancorfieldI donā€™t think thatā€™s really possible since the spec forms can be both arbitrarily complex and application-specific.#2017-03-2019:11seancorfieldWe deliberately structure our specs so that our ā€œdecodingā€ function works ā€” i.e., if we find a particularly spec doesnā€™t decode well, we refactor it (usually just a matter of creating named predicates).#2017-03-2019:13seancorfieldIt also takes a bit of trial and error sometimes to get the right set of structural keys in the error code map for certain errors we want to break out specifically.#2017-03-2019:15seancorfieldAll told, our decoding logic runs about 50 lines of somewhat dense code. Then we have a map for each spec that needs specific error messages.#2017-03-2019:16seancorfieldAnd itā€™s taken two or three iterations on the decoding logic to be both general enough for all our specs and also specific enough to get the level of detail we need in error messages.#2017-03-2019:19seancorfieldThe nice thing about this approach is that specs are still clean and readable ā€” and independent of the error messages we need to produce ā€” and we can change the level of detail in how we report errors just by changing the map we pass into the decoding function (e.g., in the map I showed above, if we remove the [:photoid '->long] key, then :photoid will still be matched and we can treat it as ā€œphoto ID is required and must be numericā€ instead of two separate errors ā€” without changing the spec itself!).#2017-03-2020:24stathissideriswasnā€™t there something in spec that allowed you to define a modification of the conformed input?#2017-03-2020:27stathissiderisah, conformer!#2017-03-2020:28stathissiderisor maybe not#2017-03-2020:28stathissiderisšŸ¤”#2017-03-2020:42stathissiderismaybe it used to be there - I remember a bit of documentation saying that you have the option to do it, but you shouldnā€™t in case a consumer of your spec output needs the original info#2017-03-2020:46bbloom@stathissideris i think youā€™re thinking of conformer - why doesnā€™t it work for you / what do you need?#2017-03-2020:47stathissiderisso I have this s/or that produces [:match ā€¦] vectors, and Iā€™d like to get rid of the ā€œtagā€ around the actual data#2017-03-2020:48bbloomyou mean that calling conform produces the tagged value?#2017-03-2020:48bbloomyou canā€™t change the result of conform - only the input in to conform#2017-03-2020:48seancorfieldSo you want to add (s/conformer second)?#2017-03-2020:49seancorfieldexample:
(s/def ::boolean (s/and (s/or :b boolean?
                              :s (s/and #{"true" "false"}
                                        (s/conformer #(Boolean. %))))
                        ;; we don't care about the path, just the value
                        (s/conformer second)))
#2017-03-2020:50seancorfield(not necessarily good practice ā€” because downstream consumers of your spec are forced to take the conformed value and lose the original)#2017-03-2020:50seancorfieldFor some situations itā€™s ā€œrightā€ however.#2017-03-2020:50stathissideris@seancorfield exactly, but I was reading the documentation wrong and ended up using it incorrectly#2017-03-2020:50stathissiderismany thanks for the example#2017-03-2020:50bbloomthatā€™s a neat trick @seancorfield šŸ™‚#2017-03-2020:51seancorfieldI found the description of conformer very confusing on the first few readsā€¦ but Iā€™m not sure how to describe it any better so I havenā€™t submitted a PR.#2017-03-2020:51seancorfieldItā€™s just an odd concept at first.#2017-03-2020:52stathissideristhe reason I wanted that is that Iā€™m using s/& in my function spec to enforce constraints between args, but the conformed args donā€™t play very well with my predicates#2017-03-2020:55seancorfieldYeah, we use it to hide the ā€œimplementationā€ of certain specs.
#2017-03-2021:01seancorfieldWrong channel? šŸ™‚#2017-03-2021:02mpenetah yes šŸ™‚#2017-03-2022:10yonatanel@seancorfield I made a lib of these tricks, but I'm looking for something more generic: https://github.com/yonatane/spec-grind/blob/master/src/spec_grind/grind.clj#2017-03-2022:18seancorfieldSome thoughts: Instead of comparing against ::s/invalid, why not use the predicate s/invalid?; your conform-or-throw seems to be the built-in s/assert; Iā€™d consider a macro to reduce the boilerplate of those last set of very similar predicates.#2017-03-2022:21seancorfieldWe have a macro that takes a predicate, a coercion, and an optional reverse coercion that lets us define specs of that pattern: satisfies ā€œpredā€ or is a string that coerces to something that satisfies ā€œpredā€ ā€” and can be generated (by generating for the ā€œpredā€ and then reverse coercing back to a string).#2017-03-2022:37yonatanelWhen do you reverse-coerce?#2017-03-2022:47seancorfieldAs part of generation. The default for most of our ā€œAPI specsā€ is just str so we produce strings.#2017-03-2022:47yonatanel@seancorfield regarding s/assert I wrote my own that's independent of *compile-asserts* or check-asserts because it was part of my application logic and not just to find bugs. Anyway I hardly throw anymore and instead put the explain data in a stream. A macro combining pred and coercion is nice. I'll think about it.#2017-03-2022:47seancorfieldBut we have a few specs where we accept or coerce to a keyword so the reverse-coerce is name instead of str.#2017-03-2022:48seancorfieldAh yes, good point about *compile-asserts* etc.#2017-03-2022:49yonatanelI'm not into gen testing yet so this whole use case escapes me.#2017-03-2022:51yonatanelI used to allow strings padded by whitespace and trim them before coercing to keyword. A reverse coercion would miss that, wouldn't it?#2017-03-2022:53yonatanelBut it sounds like a regex spec would know how to generate these things instead of reverse coercion. Could you eliminate reverse-coerce in your case?#2017-03-2023:01seancorfieldWith regex as the ā€œspecā€, I use test.chuckā€™s regex generator.#2017-03-2023:03seancorfieldAnd, yes, if my spec trims whitespace before validation, I would likely not bother randomly generating strings with additional whitespace (I would instead have a single test-by-example that had whitespace to verify that validation accepted it).#2017-03-2023:05seancorfieldIt really depends on what aspects of the spec I consider important enough to be part of the generated test cases. For example, we have a spec for validating member-supplied passwords, but we use a regex-generator for it that produces only a subset of possible valid input values.#2017-03-2023:05seancorfield(itā€™s not worth the effort to accurately generate all possible edge cases in that situation since the random-but-subset-conforming password values as ā€œgood enoughā€ as test data)#2017-03-2023:12yonatanelMakes sense#2017-03-2100:36Alex Miller (Clojure team)@stathissideris @seancorfield regarding your conformer question above, use (s/conformer val) not second#2017-03-2100:37Alex Miller (Clojure team)Or if you're feeling bold you can wrap the semi hidden and possibly going away (s/nonconforming ...)#2017-03-2100:37seancorfieldAh, yes, itā€™s a MapEntry not a vector.#2017-03-2100:38Alex Miller (Clojure team)It's both! :)#2017-03-2100:38seancorfieldI would use s/nonconforming but you keep saying it might go away šŸ˜›#2017-03-2100:38seancorfieldSo why does it matter whether we use val or second?#2017-03-2100:39seancorfield(Iā€™m happy to ā€œdo the right thingā€ but want to know why my current code is ā€œwrongā€)#2017-03-2100:39Alex Miller (Clojure team)val is direct field lookup#2017-03-2100:39seancorfieldAh, so itā€™s faster.#2017-03-2100:39Alex Miller (Clojure team)second will convert to a seq and pull two values#2017-03-2100:39Alex Miller (Clojure team)Should be much faster#2017-03-2100:41seancorfieldI suspect I use second in other places where val would be better...#2017-03-2100:50Alex Miller (Clojure team)Just don't ever use second :)#2017-03-2100:51seancorfieldYeah, I have a whole bunch of (->> ā€¦ (group-by something) (map second) ā€¦) forms!#2017-03-2100:52seancorfieldWhat about destructuring where you likely have (map (fn [[k v]] ā€¦) some-map)#2017-03-2100:54seancorfieldbetter to do (map (fn [me] ā€¦ (key me) ā€¦ (val me) ā€¦) some-map)?#2017-03-2100:54Alex Miller (Clojure team)I'd use the one that you find more readable#2017-03-2100:55Alex Miller (Clojure team)I'd destructure#2017-03-2100:55Alex Miller (Clojure team)Id have to look at code to guess at any perf difference#2017-03-2100:56Alex Miller (Clojure team)If destructuring uses nth then its probably a wash#2017-03-2106:56stathissideris@alexmiller thanks, I wasn't aware that using second instead of val for map entries is less performant #2017-03-2112:51yonatanelI made a custom tagless s/oradapted from the original, but had to copy half of spec's private functions like specize, pvalid?, explain-1 etc. Am I missing some public util functions? https://github.com/yonatane/spec-grind/blob/master/src/spec_grind/impl.clj#2017-03-2113:13iGELA spec newbie question: How do I create expectation about non namespaced keyword keys of a map?#2017-03-2113:14iGEL
(do
  (s/def ::customer-id int?)
  (s/def ::category (s/keys :req [::customer-id]))

  (s/explain-data ::category {:customer-id 2}))
#2017-03-2113:17danielnealYou can use req-un and opt-un#2017-03-2113:17danielnealThere's some details on the about spec page I think https://clojure.org/about/spec#2017-03-2113:20iGELThanks. We missed that you still need to pass namespaced keywords, but the expectation is non namespaced. Now it works#2017-03-2113:23danielnealšŸ‘#2017-03-2113:48Alex Miller (Clojure team)@yonatanel no, those are internal and subject to violent change without warning :)#2017-03-2113:54iGELAnother question: I wanted to use spec in my tests, where I do want to expect most keys in my map to be exactly this, but others like the id or created_at, which I can't predict, to just conform to predicate. For example, given this map:
{:id 4
 :customer_id 5
 :section "lala"}
I want to see that customer_id is exactly 5 and section is "lala", but for id it just want to know that it's an integer. Finally, it would be nice if the spec would fail if the result has other keys, but that's not required. Is that possible and how?
#2017-03-2113:58moxaj@igel for the first part, you can use sets as predicates, like #{5} or #{"lala"}. For the second part, you can do (s/and (s/keys ...) #(every? #{:your :keys} (keys %)))#2017-03-2114:13iGELThanks šŸ™‚#2017-03-2114:00moxaj(but it's generally advised against)#2017-03-2114:03Yehonathan SharvitAnother question related to (s/keys ā€¦) Imagine I run this:#2017-03-2114:04Yehonathan Sharvit
(s/def ::my-map (s/keys))
(s/valid? {:foo/core ā€œaaā€})
#2017-03-2114:05Yehonathan SharvitIf foo.core is somehow defined in my spec registry, then its specs will run when calling (s/valid? {:foo/core ā€œaaā€})#2017-03-2114:05Yehonathan SharvitBut it is not my intent at all#2017-03-2114:05Yehonathan Sharvitfoo/core might be a namespace defined by a library that I am requiring#2017-03-2114:06Yehonathan SharvitIs there a security risk?#2017-03-2114:06Yehonathan SharvitUnexpected code can run based on user input#2017-03-2114:06Yehonathan Sharvitimagine I parse a JSON with {ā€foo/coreā€ ā€œaaā€} and decide to keywordize the keys#2017-03-2114:07yonatanel@viebel that's why it's recommended to use namespaces you own, otherwise you break others and others break you.#2017-03-2114:08yonatanel@viebel But yes, that global registry is a pain in the ass.#2017-03-2114:09Yehonathan SharvitThe problem is that once I require a library, I cannot prevent it to add specs in the registry#2017-03-2114:09Yehonathan Sharvit!!!#2017-03-2114:09Yehonathan SharvitSo when I parse the JSON, I must be careful when keywordizing the keys !!!?!?!#2017-03-2114:09yonatanel@viebel I'm not sure spec is meant for portable data validation and coercion at all.#2017-03-2114:10mpenetwhy not?#2017-03-2114:10moxaj@viebel you can still preprocess the keywordized map, before validating it#2017-03-2114:10yonatanelBecause of how it's built. I'm using it for JSON all the time but it feels wrong.#2017-03-2114:11mpenetI do the same and I don't have any issue with it. Not sure I follow#2017-03-2114:12bronsaI think viebel has a point#2017-03-2114:12yonatanel@mpenet I don't like prefixing all my keywords with a company name for example.#2017-03-2114:12bronsasuppose you have a spec whose conformer does heavy-ish work#2017-03-2114:13bronsaif you conform user input against a spec that uses keys#2017-03-2114:13bronsait will force that conformer to run#2017-03-2114:13bronsamight be an opportunity for dos attack#2017-03-2114:14bronsaforcing cpu bound work while parsing user data#2017-03-2114:14mpenetif you accept input with keywordized keys and use spec you must follow some rules yes (proper namespaces), otherwise just use the *-un variants#2017-03-2114:14mpenet(I do the later)#2017-03-2114:15mpenetthe endpoint dicts the context already, no need for namespaces usually#2017-03-2114:15bronsawell, the point is that if you're using keys and accepting user data you don't have control over what the user will pass you#2017-03-2114:15bronsaand might malignantly pass namespaced keys that force conform using an expensive conformer#2017-03-2114:15mpenetif you take edn directly yes that would matter. but arent' we talking about json ?#2017-03-2114:16bronsaare we?#2017-03-2114:16mpenetI think so#2017-03-2114:16Yehonathan Sharvitwe are talking about JSON#2017-03-2114:16bronsawell, point still stands#2017-03-2114:16bronsasuppose I was talking about edn/transit :)#2017-03-2114:16Yehonathan Sharvitthat you parse with keywrodizing the keys which is quite common practice#2017-03-2114:17mpenetyou then get un-namespaced keywords#2017-03-2114:17mpenetso no risk for "injection"#2017-03-2114:18yonatanelUnless you have data like {"event/type" "blabla"}#2017-03-2114:19bronsastill, if you do something like (comp clojure.walk/keywordize-keys cheshire.core/parse-string) over your input data#2017-03-2114:20bronsa& then conform#2017-03-2114:20mpenetah right, didn't realize that (keyword "event/type") would actually create a ns keyword, bummer#2017-03-2114:20bronsayeah#2017-03-2114:20mpenetanother argument for never keywordizing json I guess#2017-03-2114:20bronsa(doesn't solve this issue with edn/transit over the wire)#2017-03-2114:22mpenetthis kind of sucks indeed, I wonder why keyword allows this at all#2017-03-2114:22yonatanel@bronsa I guess you shouldn't blindly use (s/keys) on user input, or do anything blindly with user input.#2017-03-2114:22mpenetthere's the 2 arity version for ns stuff already#2017-03-2114:23bronsa@mpenet it's been like that since forever and it's used quite often#2017-03-2114:23mpenetstill very unfortunate#2017-03-2114:23bronsait's quite useful#2017-03-2114:23yonatanel@mpenet I read somewhere that keyword accepts anything exactly for this reason, to keywordize user input.#2017-03-2114:24bronsa@yonatanel meh, i don't buy the argument "you should be careful with what you do while parsing user input" tho -- i'm conforming user input to check for valid data#2017-03-2114:24mpenetthat said you could write a safe keywordize-keys#2017-03-2114:24mpenetusing find-keyword#2017-03-2114:24bronsai don't want to manually check for invalid data and then conform it#2017-03-2114:25bronsawhat if I do want to use namespaced keys in my user data tho#2017-03-2114:26bronsathen it's either live with the possibility of people exploiting this for possibly overload cpu while validating user input or don't use namespaced keys at all#2017-03-2114:27mpenetseems so#2017-03-2114:27bronsaI would consider this a significant limitation#2017-03-2114:28mpenetI guess this is a +1 for the "closed" key-set definition people are asking for#2017-03-2114:28mpenet"a la Schema"#2017-03-2114:28bronsaor using a private registry#2017-03-2114:28idoa01Hey, new here and new to clojure-spec, I was discussing with @viebel on this issue before he came here.#2017-03-2114:30idoa01the thing that struct me is that the (s/valid?) will call unexpected code on various input. as we saw here, it might be used to run malicious code on the server#2017-03-2114:30bronsaI wouldn't say malicious code as the user can't really inject code#2017-03-2114:31bronsaundesiderable code, tho, yes#2017-03-2114:31idoa01I think the openness is a huge problem here, (s/valid?) shouldn't run anything that I didn't ask for#2017-03-2114:31idoa01it could be malicious if the code came from a library designed to inject that (s/def)#2017-03-2114:32bronsayou're in control of what libs you're using tho#2017-03-2114:32idoa01the library has many useful code, and a backdoor#2017-03-2114:32mpenetwell that library would also write files and what not šŸ™‚#2017-03-2114:32bronsaif you're pulling in a backdoored lib spec is the last of your problems#2017-03-2114:32mpenetbuy yes, the auto validation of ns kw even when not in spec is odd#2017-03-2114:32idoa01but sure, the more common problem is a bug in the library, or just a cpu-intensive code#2017-03-2114:33bronsaI don't think that (the opennes of keys) is ever going to change#2017-03-2114:33bronsafor good reasons#2017-03-2114:33mpenetopenness is fine, auto-validation of non spec-present ns'ed key, not so much imho#2017-03-2114:34idoa01then maybe add a (s/secure-valid?) for user input#2017-03-2114:34yonatanelmore like (s/keys :only [...])#2017-03-2114:34mpenetthe famous (s/keys) that triggers validation on every single key of the map#2017-03-2114:34bronsai don't see the point of openness w/o auto conform/validation#2017-03-2114:34bronsa@yonatanel yeah that's never going to happen#2017-03-2114:35bronsait's been proposed a bunch of times and rejected with good reasons#2017-03-2114:35yonatanelCool. any links to that?#2017-03-2114:36bronsarich's last clojure/conj talk explains why closed specs make it harder to evolve systems#2017-03-2114:37yonatanelYeah, I watched it.#2017-03-2114:37yonatanelSo what's the problem really?#2017-03-2114:37bronsahttps://clojure.org/about/spec#_features#2017-03-2114:37idoa01in that case, the library has to have a way to clean the input#2017-03-2114:37bronsa
-Namespaced keyword specs can be checked even when no map spec declares those keys

This last point is vital when dynamically building up, composing, or generating maps. Creating a spec for every map subset/union/intersection is unworkable. It also facilitates fail-fast detection of bad data - when it is introduced vs when it is consumed.
#2017-03-2114:42yonatanelYou're asking spec to sanitize your inputs and conformers to be your lossy coercions. From what I gather it's not meant to do that (I do use it that way though :))#2017-03-2114:43idoa01I'm asking spec to not run code I didn't specifically intend to run.#2017-03-2114:45dergutemoritzBy that reasoning you'd also have to prohibit polymorphism across libraries#2017-03-2114:45idoa01that's taking it too far, and when I use polymorphism, I intend to use polymorphism.#2017-03-2114:45bronsadon't think that's the same at all#2017-03-2114:46dergutemoritzIt's similar in a way that you potentially run code that you didn't know you were going to run from the point of calling a function#2017-03-2114:46bronsathis issue implies that you can't run s/valid? on user-provided input before validation (which is what s/valid? is for?). Now suppose one of your libs defines a spec for some internal validation and it's bugged in a way that causes certain inputs to never terminate parsing. Now somebody discovers it, and maliciously provides those inputs to you and you have no way of avoiding that to run#2017-03-2114:46bronsaI think it's a very valid issue to raise#2017-03-2114:47bronsalet's wait to see what @alexmiller thinks about it#2017-03-2114:47dergutemoritzYeah, that spec is not good for running on untrusted inputs is a good obversation#2017-03-2114:49bronsathis to me feels similar to the problem that clojure.edn solved for parsing user input#2017-03-2114:49Alex Miller (Clojure team)but you are asking spec to run validation using s/keys, which will validate keys#2017-03-2114:49Alex Miller (Clojure team)if you donā€™t want that, donā€™t do it#2017-03-2114:51dergutemoritzGood point, hehe#2017-03-2114:52idoa01then what is the recommended way to validate a map?#2017-03-2114:53idoa01or more specifically, a JSON#2017-03-2114:54Alex Miller (Clojure team)I think youā€™re implicitly conflating multiple things into one question#2017-03-2114:54Alex Miller (Clojure team)spec has capabilities for validating data#2017-03-2114:55Alex Miller (Clojure team)one of them is to define a spec for a map as a collection of attributes#2017-03-2114:56Alex Miller (Clojure team)recognizing that many current maps have unqualified keys, s/keys provides :req-un and :opt-un options for validating unqualified keys#2017-03-2114:56Alex Miller (Clojure team)and that is one approach to validating json#2017-03-2114:57idoa01but :req-un and :opt-un still accepts fully qualified namespaces, which will cause the same issue#2017-03-2114:57Alex Miller (Clojure team)so donā€™t rely on spec to solve every problem for you#2017-03-2114:58Alex Miller (Clojure team)its your responsibility as an application developer to think about how you handle untrusted inputs from the external world#2017-03-2114:58bronsa>but you are asking spec to run validation using s/keys, which will validate keys right, but even if I'm not while validating input data, I might in some internal function that my valid input data might get threaded through#2017-03-2115:00bronsa>its your responsibility as an application developer to think about how you handle untrusted inputs from the external world which is why I'm validating it with spec :) it feels a bit odd to me that we have to be careful about passing user data to validating functions#2017-03-2115:01bronsaif "spec is not the right tool at this level" is the answer I guess I'll live with that but I can imagine a lot of people will be confused by this answer?#2017-03-2115:02Alex Miller (Clojure team)Iā€™m not saying itā€™s not the right tool, just that you should think critically about what youā€™re doing. sorry not sorry about that.#2017-03-2115:02bronsaesp because people are using schema for doing that right now and will be tempted to switch from schema to spec w/o realizing the implication#2017-03-2115:02Alex Miller (Clojure team)you control the libraries youā€™re using, the code youā€™re running, and when and how you validate data#2017-03-2115:03Alex Miller (Clojure team)all your specs call predicate functions, maybe from other libraries - how is that any different?#2017-03-2115:03bronsathat's all true, it still doesn't make me feel any less weird about having to validate my data in order to validate it through spec#2017-03-2115:03bronsait's not different, conform vs predicate functions is not the issue#2017-03-2115:04idoa01it's a bit like adding eval to the code, without telling the non-expert user that you're doing it. I was really surprised when @viebel pointed this feature to me.#2017-03-2115:04bronsathe issue is it appears it's not safe to validate/conform user-provided data with spec w/o.. validating it before passing it to spec#2017-03-2115:04idoa01so it may work especially well for anyone in the "knows" but will shot the regular developer in the leg.#2017-03-2115:08Alex Miller (Clojure team)can you give an example of a spec that would be ā€œnot safeā€ in this way?#2017-03-2115:09bronsanot safety is not an issue, I don't think. I'm thinking of some malformed spec that causes non termination with very simple inputs, which is not unlikely to happen#2017-03-2115:09Alex Miller (Clojure team)so, example?#2017-03-2115:10bronsaone sec#2017-03-2115:10Alex Miller (Clojure team)if itā€™s a bug in a predicate or spec, then you would fix it, just like you would with any bug that has that effect in your code#2017-03-2115:10Alex Miller (Clojure team)Iā€™m questioning the premise of this problem#2017-03-2115:11bronsacorrect, it would be a bug of a spec#2017-03-2115:11bronsawhich I don't have control over#2017-03-2115:11Alex Miller (Clojure team)you do - you chose to load it#2017-03-2115:11Alex Miller (Clojure team)you can choose to not load it#2017-03-2115:11Alex Miller (Clojure team)or to replace it#2017-03-2115:11bronsayes, but I might not know about the bug in this spec#2017-03-2115:11bronsanor might the library author#2017-03-2115:11Alex Miller (Clojure team)and you might not know about a bug in a function in a library you call#2017-03-2115:11bronsabut an attacker might discover it#2017-03-2115:11Alex Miller (Clojure team)same for any function#2017-03-2115:12Yehonathan SharvitThe weird thing is that even if the spec is just there - defined in the lib - without even being in use, it will be in the registry and corrupt my code#2017-03-2115:12Alex Miller (Clojure team)it will not be unless you load the code#2017-03-2115:12Alex Miller (Clojure team)you control the registry#2017-03-2115:12Yehonathan SharvitShould I check all the specs defined in all the libs I load?#2017-03-2115:12Alex Miller (Clojure team)should you check all the functions defined in all the libs you load?#2017-03-2115:12bronsaright, it seems highly more likely to be explotable in a spec tho that finding the correct data that causes an invocation of that function to trigger the bug#2017-03-2115:12Yehonathan SharvitAll the functions I use#2017-03-2115:13Yehonathan SharvitBut not the funtions I donā€™t use#2017-03-2115:13Alex Miller (Clojure team)all the functions your functions call?#2017-03-2115:13Yehonathan SharvitYes#2017-03-2115:13Alex Miller (Clojure team)then yes, you should check all the specs too#2017-03-2115:13Alex Miller (Clojure team)this is not different#2017-03-2115:13idoa01say I'm using a common library (say cheshire) and it upgraded with a buggy spec. now anyone who uses that lib, are open to that bug, without explicitly calling the spec. just by loading cheshire#2017-03-2115:14Alex Miller (Clojure team)yes, exactly like if cheshire had a function that was buggy#2017-03-2115:14bronsaI understand what you're saying, I don't agree that the two have the same severity tho#2017-03-2115:14Yehonathan SharvitI cannot discover this bug with any kind of unit tests#2017-03-2115:14Yehonathan Sharvit(Even using test.check !)#2017-03-2115:14Alex Miller (Clojure team)that does not make sense#2017-03-2115:15bronsaI have control over what functions I use and how I use them, I don't have control over what (loaded) specs some user data will invoke unless I validate that data before I spec/validate it (or don't use keys)#2017-03-2115:16rauhAt the easiest you can trigger a stackoverflow if a user has a recursive schema somewhere defined.#2017-03-2115:16bronsaanyway, it seems like the answer is "be careful", not sure it satisfies me but I can live with it#2017-03-2115:16rauhI'd love to vote on a ticket if one is created, because until 10min ago I didn't realize s/keys checked other keys implicitly.#2017-03-2115:16bronsathat's in s/keys docstring#2017-03-2115:18yonatanelusing an empty (s/keys) beats defining every object in even a medium system.#2017-03-2115:19Alex Miller (Clojure team)@rauh if the spec throws an error, then it is doing itā€™s job in telling you that itā€™s invalid data#2017-03-2115:19idoa01the implicitly of (s/keys) opens the code for trouble. an in-between solution would be that (s/keys) will invoke only locally namespaced specs, and not library defined specs.#2017-03-2115:19Alex Miller (Clojure team)those are not two differentiable things#2017-03-2115:20donaldballEvery once in a while I wish for a s/keys version that allowed a whitelist of spec keywords or namespaces that it checks#2017-03-2115:20bronsathey are if you allow using a local registry rather than a global one#2017-03-2115:21bronsathe local registry could be defined as a subset of the global one, becoming effectively a whitelist#2017-03-2115:21bronsa(not sure if it's a good idea)#2017-03-2115:21Alex Miller (Clojure team)not doing that#2017-03-2115:21idoa01used with (s/keys :ns ... )#2017-03-2115:22Alex Miller (Clojure team)s/keys combines multiple things. enhancement requests to do individual parts of that might be worth considering.#2017-03-2115:23Alex Miller (Clojure team)separating required/optional key checks from validation of specified keyed attributes from validation of all keyed attributes#2017-03-2115:24rauh@alexmiller A Stackoverflow which can easily be triggered by user input isn't something most programmers catch, and certainly valid? is expected to return true/false. Stackoverflow doesn't even get caught with (a pretty wide) (catch Exception _)#2017-03-2115:24rauhIt goes under VirtualMachineError and will likely kill the thread.#2017-03-2115:27Alex Miller (Clojure team)again, how is this different from having a bad function that throws a StackOverflowError?#2017-03-2115:27Alex Miller (Clojure team)if itā€™s a bug, fix it#2017-03-2115:27idoa01the difference is that the code only needs to be loaded, not referenced anywhere#2017-03-2115:28Alex Miller (Clojure team)which is not at all different than multimethods or protocol extensions#2017-03-2115:28Alex Miller (Clojure team)there are several Clojure constructs that create runtime state on load#2017-03-2115:29rauhBecause the attack is much easier.#2017-03-2115:29Alex Miller (Clojure team)how is it any different than a bug in the validation function you were going to hand write instead?#2017-03-2115:30Alex Miller (Clojure team)the attack (from outside) is identical#2017-03-2115:30idoa01the question is, if a programmer which isn't in this slack room, uses clojure-spec, will he think about sanitizing the json before passing it to (s/valid?) if not, then the buggy behavior will be wide spread.#2017-03-2115:31Alex Miller (Clojure team)what is the exposure?#2017-03-2115:31idoa01was this behavior of the (s/keys) been discussed in regards to this issue before? if not, I think that it qualifies as a "surprising" side effect of clojure-spec#2017-03-2115:31Alex Miller (Clojure team)itā€™s been discussed many times. itā€™s discussed in the guide, and in the spec rationale, and in the doc string.#2017-03-2115:31idoa01every one who uses clojure-spec to validate REST API calls#2017-03-2115:32Alex Miller (Clojure team)no, I mean for a particular application#2017-03-2115:32Alex Miller (Clojure team)if someone passes bad input, it could yield an error response#2017-03-2115:32Alex Miller (Clojure team)how is that different than any bad input#2017-03-2115:32Alex Miller (Clojure team)thatā€™s the whole point of validation?#2017-03-2115:32idoa01because I'm using clojure-spec to validate the input is not bad.#2017-03-2115:32Alex Miller (Clojure team)but it IS bad#2017-03-2115:33idoa01so clojure-spec should tell me it is.#2017-03-2115:33Alex Miller (Clojure team)it does?#2017-03-2115:33idoa01not run arbitrary code that might harm the machine#2017-03-2115:33Alex Miller (Clojure team)how can it harm the machine?#2017-03-2115:33Alex Miller (Clojure team)you chose to load the ā€œarbitrary codeā€#2017-03-2115:33Alex Miller (Clojure team)this is not code some attacker is supplying to you#2017-03-2115:33idoa01stackoverflow, memory consumption, cpu overload. you name it.#2017-03-2115:34Alex Miller (Clojure team)all things that can happen from an invalid input also sent to a ā€œbadā€ validation function#2017-03-2115:34idoa01it all depends on what is written in a faulty spec, that i didn't say i want to use.#2017-03-2115:35Alex Miller (Clojure team)I donā€™t see any way to actually cause memory consumption or cpu overload in dangerous ways. stackoverflow maybe, although I donā€™t have an example of that either.#2017-03-2115:35bronsai think the claim being made is that the surface area of the possible "attack" (for lack of a better word) is potentially way larger with bugged specs than other bugged constructs#2017-03-2115:35Alex Miller (Clojure team)I have no examples of ā€œfaulty specsā€ that can cause improper machine resource usage.#2017-03-2115:35Alex Miller (Clojure team)what are the alternatives?#2017-03-2115:36Alex Miller (Clojure team)1) donā€™t validate and donā€™t be aware of invalid data#2017-03-2115:36Alex Miller (Clojure team)2) validate by using functions in either your code or libs#2017-03-2115:37Alex Miller (Clojure team)1 does not seem better and 2 does not seem effectively different to me#2017-03-2115:37idoa01allow me to not run any arbitrary spec code implicitly if i don't want to šŸ˜•#2017-03-2115:37Alex Miller (Clojure team)and why is that harmful?#2017-03-2115:37mpeneta spec predicate that does something over the network (call a db?), parses string content (or whatever resource heavy operation you can think of)#2017-03-2115:37idoa01why would I want to run it?#2017-03-2115:37mpenetnot your average predicate, but not too crazy either#2017-03-2115:42rauhIMO it'd be nice to have multiple repositories, for instance, I'm using :db/id, :object/id and :file/id in my application code. Down the road when many libraries are spec'ed this will get trampled and lead to issues. Or am I missing something?#2017-03-2115:43mpenetthat's not so much of an issue with ns aliases + proper namespacing#2017-03-2115:44mpenetit's the same problem with project names/ns/package. that said multiple repos would be nice for other reasons#2017-03-2115:51Alex Miller (Clojure team)@rauh you are missing the use of proper namespacing :)#2017-03-2115:57yonatanel@alexmiller Would you say datomic attributes should be qualified with a namespace you own, or is :album/year enough?#2017-03-2115:58yonatanelI see spec and datomic go hand in hand. Correct me if I'm wrong.#2017-03-2116:04Alex Miller (Clojure team)@yonatanel same advice as spec. if youā€™re providing data for use with others, you must use a qualifier that you ā€œcontrol" (reverse domain, trademarked entity, etc). if the data will be used only in an environment that you control, it must only be ā€œsufficiently unique"#2017-03-2116:05Alex Miller (Clojure team)so in a generic open source library, use a qualifier you control. If confined in your app, do whatever you want. If in an organizational context, you might need to ensure uniqueness in your organization.#2017-03-2116:14moxaj@alexmiller care to comment on the snippet above? How should one defend against an 'attack' like this?#2017-03-2116:27Alex Miller (Clojure team)Don't call valid?, don't load this spec, don't use s/keys, or use select-keys to pre-filter what you look at#2017-03-2116:31Alex Miller (Clojure team)Check whether your input contains 10000 nested maps#2017-03-2116:34Alex Miller (Clojure team)Again, also compare this with what you would do without spec too - is that prone to the same issue?#2017-03-2116:35moxajwell, without spec, I wouldn't check for an ::x key within my input, so no#2017-03-2116:46Alex Miller (Clojure team)so is it better to notice the bad input or to pass 10000 nested maps around your system?#2017-03-2116:50dergutemoritz(provided your JSON parser didn't blow up with a StackOverflowError before that point anyway, hehe)#2017-03-2116:51dergutemoritz(or whatever your input format is)#2017-03-2116:51tbaldridgeAnd "attack" is a bit of a strong word here, considering it's an exception, not a granting of root privileges or something like that.#2017-03-2117:00moxajIt isn't a bad input though. My spec says it might have an 'a' key and that's it. I do not care about the rest, if I did, I would have specified that in my spec. #2017-03-2117:01moxaj@tbaldridge I agree about the attack part, hence the quotes (I'm pulling a trump here lol)#2017-03-2117:31Alex Miller (Clojure team)@moxaj thatā€™s not what spec says that spec means#2017-03-2117:32Alex Miller (Clojure team)that spec says ā€œa map that might have an ::a key, and where all values are valid according to the spec of their keys"#2017-03-2117:32sparkofreasonIs there any way to define coll-of specs such that the associated generator would yield a collection type such as PersistentQueue, sorted-set, etc? I assume I could do it with a custom generator, just wondering if there's a shorter path.#2017-03-2117:32Alex Miller (Clojure team)@moxaj if you want what you said, then just map? is sufficient#2017-03-2117:33Alex Miller (Clojure team)@dave.dixon look at the :into key#2017-03-2117:33Alex Miller (Clojure team)(s/def ::c (s/coll-of int? :into (sorted-set)))#2017-03-2117:34moxajThe keys spec is perfect for me, except for the implicit part. But I guess that's not subject to change, so no point in arguing :)#2017-03-2117:35Alex Miller (Clojure team)well as I said above, an enhancement ticket that separates parts of keys seems like a reasonable idea#2017-03-2117:37Alex Miller (Clojure team)I do not know how Rich would react to it, but ā€œdecomplectingā€ is usually a good thing :)
#2017-03-2117:38otfromI do a lot of ETL stuff where I start with a pile of strings and turn them into some sort of seq of data structures. The strings encode things like a map of key to string or key to set of numbers. I've had some reasonable success in using spec to check that the strings are of a format that is coercable to something "100" -> 100 or 100,101,102 -> #{ 100 101 102 } Is this a terrible, no good, pls stop use of spec?#2017-03-2117:39Alex Miller (Clojure team)are you checking that they are coercible or actually doing the coercion?#2017-03-2117:40otfromI've got a function that checks s/valid? using the spec and then s/conform using the same spec#2017-03-2117:41Alex Miller (Clojure team)so you are transforming the data via the conform#2017-03-2117:41otfromyes#2017-03-2117:41otfrom(which I think is the bad bit)#2017-03-2117:41Alex Miller (Clojure team)so the general caveat for stuff like that is not that itā€™s necessarily bad but that it has consequences that you should understand#2017-03-2117:41Alex Miller (Clojure team)namely that users of that spec cannot recover the original data (as you could via s/unform normally)#2017-03-2117:42Alex Miller (Clojure team)and if you put that spec in the registry, youā€™ve made that choice for consumers of the spec#2017-03-2117:42Alex Miller (Clojure team)if ā€œconsumers of the specā€ == you, then thatā€™s up to you :)#2017-03-2117:43Alex Miller (Clojure team)if youā€™re using conformers, and if the function is reversible, you can supply an unform function in conformer to retain the reciprocal nature of that (assuming it is worth the trouble for you, which it may not be)#2017-03-2117:45otfromI've not seen an unform example that does that yet. That would be interesting. After talking to seancorfield about it quite a bit I've been building up quite a few generators for testing things that have done a good job of driving out some bugs in my functions.#2017-03-2117:46otfromwould I be able to use an unform function like that inside a generator for testing?#2017-03-2117:46otfrom(and this is gold dust to me so thx. I'm glad it is a trade off I should think about rather than a terrible idea)#2017-03-2117:46otfrom(and I realise this spark/hadoop/ETL strings -> data is not every domain)#2017-03-2117:48sparkofreason@alexmiller Thanks, that worked. Docs make it sound like :into is limited to [], (), {}, #{}, should have just tried it.#2017-03-2117:49Alex Miller (Clojure team)
user=> (s/def ::s (s/conformer #(str "hi "%) #(subs % 3)))
:user/s
user=> (s/conform ::s "bruce")
"hi bruce"
user=> (s/unform ::s "hi bruce")
ā€œbruceā€
#2017-03-2117:49Alex Miller (Clojure team)itā€™s just functions yo#2017-03-2117:49otfromšŸ¤—#2017-03-2117:50Alex Miller (Clojure team)Rich would also caution you against treating spec as a transformation engine rather than something that validates declarative statements about your data (ie ā€œitā€™s not a meat grinderā€).#2017-03-2117:51Alex Miller (Clojure team)but that capability is there for your abuse :)#2017-03-2117:51otfromI hadn't seen that conformer done that way#2017-03-2117:51Alex Miller (Clojure team)@dave.dixon actually, I may have led you into the path of undefined behavior there#2017-03-2117:52Alex Miller (Clojure team)I think youā€™re right that the intention was to only support that fixed set in :into for the time being#2017-03-2117:53otfromalexmiller this is the "use clojure.core for this kind of thing" comment I've seen floating about#2017-03-2117:53otfromIIUC#2017-03-2117:54Alex Miller (Clojure team)yeah#2017-03-2117:54otfromI think I'm still searching for a better way of turning other peoples strings into data.#2017-03-2117:55otfrom(and dividing the invalid from the valid ones and reporting on the invalid ones well)#2017-03-2117:55otfromI quite liked the error msgs that I could get out of spec around what had failed when I tried to validate#2017-03-2117:56Alex Miller (Clojure team)@dave.dixon there is some ambiguity about whether (I think due to impl churn) about whether :kind is supposed to be a function or a spec. Only a function works now, but the doc implies a spec (which could itself have a custom generator). There is a ticket regarding this issue and I havenā€™t yet gotten Rich to respond to me about it. :)#2017-03-2117:57otfromthx alexmiller#2017-03-2117:57Alex Miller (Clojure team)np!#2017-03-2117:59sparkofreason@alexmiller Thanks, that answers my next question. Will probably just go with it for now since it seems to work, and the only other option would appear to be to write a custom generator for every "non-standard" collection spec.#2017-03-2118:46yonatanelWhat's that book/paper spec was inspired by?#2017-03-2118:51hiredmanare you talking about the parser?#2017-03-2118:55hiredmanracket has a contract system which, while I don't think I have heard anyone on the core team say was inspiration, a lot of people just assume it must have been#2017-03-2118:56hiredmanthe way spec validates sequential data structures is by "parsing" them using a parser based on parsing with derivatives (also a racket connection there)#2017-03-2119:00yonatanelyes, parsing with derivatives. Thanks!#2017-03-2119:42Yehonathan Sharvit@yonatanel Here is an interactive article that I wrote about the basics of ā€œparsing with derivatives"#2017-03-2119:42Yehonathan SharvitWith interactive clojure code snippets#2017-03-2119:42Yehonathan Sharvithttp://blog.klipse.tech/clojure/2016/10/02/parsing-with-derivatives-regular.html#2017-03-2119:52yonatanel@viebel Yeah it's one of the first google results for parsing with derivatives...#2017-03-2208:30mpenetspec made me use derive for the first time in 7 years šŸ¾#2017-03-2208:32yonatanel@mpenet What did it solve? :)#2017-03-2208:34mpenettrying a new approach to generate json-schemas from specs, I have a registry of specs/preds -> json-schema forms that's a multimethod and that allows me to have some form of inheritance#2017-03-2208:36yonatanelCan you show an example? I'm not sure what json-schema is or how you convert from spec to json-schema.#2017-03-2208:36mpenetjson-schema is the format used by swagger/open-api#2017-03-2208:36mpenetwell part of#2017-03-2208:37ikitommidoes the end result look different than the spec-tools conversion? (also uses a multimethod, but no inheritance atm)#2017-03-2208:38mpenetnot sure, I didn't look into spec-tools internals too much. What do you mean by end result?#2017-03-2208:39mpenetcode is here, but here be dragons, it's wip: https://github.com/mpenet/spex/blob/master/src/clj/qbits/spex/json_schema.clj#2017-03-2208:39ikitommimapped all the predicates that have a clojure.spec generator, with those ā€œrulesā€, here: https://github.com/metosin/spec-tools/blob/master/src/spec_tools/json_schema.cljc#2017-03-2208:40mpenetlooks very similar, but to be able to use derive you need to have ns'ed keywords#2017-03-2208:40mpenetthe same can be achieved with a macro tho, nothing too fancy#2017-03-2208:41mpenetstuff I did is also an experiement with CLJ-2112#2017-03-2208:41mpenetI am using a slightly modified/fixed version of that patch#2017-03-2208:42yonatanel@ikitommi That's a nice lib :+1:#2017-03-2208:47ikitommiderive looks nice. should we at some point merge the working stuff into XYZ? would like to copy-paste the some of the missing mappings from Spex already.#2017-03-2208:48mpenetfeel free to do that, spex is just a test-bed really#2017-03-2208:49mpenetseems like spec-tools covers a lot more already tho#2017-03-2208:51ikitommiitā€™s missing the Coercion protocol (instead of dynamic conforming), after it Iā€™m kinda happy with what tries to resolve. the ā€œdrop-extra-keys from mapsā€ things etc.#2017-03-2208:55ikitommithe 2112 would be awesome, itā€™s messy without that. Just realized yesterday that one can use and and or with s/keys...#2017-03-2208:56mpenetthese are different approaches really, spec-tools seems to wrap all specs with it's own types to add "metadata", coercion etc, I just force the user to register its specs (and provide some defaults) and don't bother with coercion at all at this level (imho these are 2 different things)#2017-03-2208:56ikitommi@yonatanel thanks!#2017-03-2208:57mpenetspec-tools is full of cool stuff, didn't see the spec maps format thing#2017-03-2208:57mpenetyeah or/and in keys is a bit of a pita, not sure it's feasible to translate in json-schema#2017-03-2208:58mpenetCLJ-2112 code conforms it happily, but it's just ignored for convertion down the road#2017-03-2208:58ikitommispec-tools doesnā€™t force to wrap into own records, JSON Schema is extracted from normal spec forms. Own records can be used to add metadata.#2017-03-2209:03mpenetI guess for or/and in keys one should/could just flatten all of it when generating a schema#2017-03-2212:48mpenetI think I like spec-tools approach better, records make all of this way nicer internally, too bad core.specs are not defined this way#2017-03-2312:15danbuneaHi, I am trying to do a simple function check using spec in ClojureScript (:require [cljs.spec :as s] [cljs.spec.test :as test]... (defn my-index-of "Returns the index at which search appears in source" [source search] (clojure.string/index-of source search)) (s/fdef my-index-of :args (s/cat :source string? :search string?) :ret nat-int? :fn #(<= (:ret %) (-> % :args :source count))) and (test/check 'my-index-of) instead of returning a problem , always returns [] meaning basically that it doesn't do anything?#2017-03-2312:16danbuneaI am basically trying this in ClojureScript https://www.youtube.com/watch?v=W6crrbF7s2s#2017-03-2312:18thhellerso this is giving me a headache, I can't figure out why this doesn't work#2017-03-2312:19thhellerI wrote a spec impl for (map-spec :req {:foo string?})#2017-03-2312:19thhellerit works just fine if I use it directly in explain or conform#2017-03-2312:19thheller
(s/def ::y (map-spec :req {:foo string?}))

(s/explain ::y {:bar 1})
#2017-03-2312:20thhellerthis however fails with CompilerException java.lang.Exception: Unable to resolve spec: :user/y, ...#2017-03-2312:23thhellerit works everywhere else but I can't def it?#2017-03-2312:39thhellerfound it, apparently you can't use deftype to create a spec impl, defrecord works#2017-03-2312:39thheller
(deftype MySpec []
  s/Spec
  (conform* [_ x]
    ::s/invalid))

(s/def ::x (MySpec.))

(s/explain ::x :foo)
#2017-03-2312:40thheller@alexmiller is that a bug? I suspect the problem is in with-name which is called in def-impl. deftype isn't IObj, defrecord is, as is reify which is used everywhere in spec#2017-03-2312:40yonatanelI've added a keyz spec that's like s/keys but with :deny :rest that will allow only keys in :req, :req-un, :opt and :opt-un#2017-03-2312:41yonatanelhttps://github.com/yonatane/spec-grind/blob/master/test/spec_grind/grind_test.clj#L67#2017-03-2312:41yonatanelIt's currently just a hack and doesn't have nice explain data. That will require implementing Spec just for the deny part.#2017-03-2312:48thheller@yonatanel was that intended for me?#2017-03-2312:52yonatanel@thheller No, but now that I saw what you wrote, are we working on the same thing?#2017-03-2312:53thhellerhehe had me confused for a second there šŸ™‚#2017-03-2312:53thhellernot exactly the same thing, but I thought about adding the do not allow unspecified keys thing#2017-03-2312:54thhellerI just have a bunch of old data and otherwise open maps that I can't easily create namespaced keys for#2017-03-2312:55thhelleryou could just use s/and#2017-03-2312:55yonatanel@thheller Isn't :req-un enough for that?#2017-03-2312:56thheller:req-un means that I add ::foo which I must s/def somewhere#2017-03-2312:56thhellerif I have :foo with 2 different meaning that gets annoying#2017-03-2312:56yonatanelRegarding just using s/and I am doing it, combining s/keys and a set predicate on the map keys, but the explain message is crap.#2017-03-2312:57thhellerah right, well if you stick to defrecord and avoid the headache that caused the impl for explain/conform is rather straightforward#2017-03-2312:57yonatanel@thheller Got you. I wanted to do the same thing, so would be nice if you release your inlined map spec in github.#2017-03-2312:58thhellerneed to figure out unform and the gen stuff#2017-03-2312:58thhellerbut conform/`explain` work just fine#2017-03-2312:59thheller@yonatanel https://gist.github.com/thheller/25d7c6c981ec902f8b2247dc7a8b88ae#2017-03-2313:01yonatanel@thheller spec-tools have something similar that you might find interesting: https://github.com/metosin/spec-tools#2017-03-2313:02thhellerah didn't know about that, thx#2017-03-2313:05yonatanelI like how you implemented explain unlike spec which creates a contains? pred for each key.#2017-03-2313:06thhelleryeah dunno map-spec-impl looked way too complicated for what I want#2017-03-2313:07thhellerstarted out by using it but made things way too complicated#2017-03-2313:12hospadarfunny that ya'll are talking about this right now#2017-03-2313:12hospadarI am implementing something very similar as we speak#2017-03-2313:19hospadarI have to validate a datastructure that already exists in some systems that's a) not using namespaced keys b) not always using keywords as the keys for maps#2017-03-2313:20hospadarand sometimes the value types of the (non-namespaced) keys are conditional on something else#2017-03-2313:21hospadari.e.
{:job-type :simple 
 :output "is a string}
{:job-type :complicated
 :output {:is "a map of something}}
#2017-03-2313:21thheller@yonatanel added the closed version#2017-03-2313:21thheller
(s/def ::y (map-spec :req {:foo string?} :closed? true))

(s/explain ::y {:foo "1" :x 1})
#2017-03-2313:21thhellerval: {:foo "1", :x 1} fails spec: :shadow.spec/y predicate: (not (contains? % :x))#2017-03-2313:22thhellermight not be the best idea but I want to use that to validate react component props, it might actually be better to use closed maps there#2017-03-2314:33yonatanel@hospadar Are you using multimethods for that?#2017-03-2314:39yonatanel@thheller Do you think the closed map explain data should show all the allowed keys instead of just the illegal one from the input?#2017-03-2314:57hospadar@yonatanel I tried using a multi spec that points and keys specs in different namespaces but that seemed to not work the way I needed#2017-03-2315:04hospadar^ that basically does what I want#2017-03-2315:04hospadarbut I didn't like the namespace hopping#2017-03-2315:04hospadarohhh, I guess I could probably s/def with namespaced keywords instead of using :: to autoresolve?#2017-03-2315:04hospadarduh#2017-03-2315:06hospadarlearning all the time šŸ™‚#2017-03-2315:14hospadarohhhh yeah that's a lot nicer#2017-03-2315:14thheller@yonatanel not sure about the explain data, good error messages are hard. could put anything in there really#2017-03-2316:46spieden@hospadar i think thatā€™s typical usage as specs are meant to augment a separate namespace that contains the keywords#2017-03-2316:46spiedenalthough i do use them inline sometimes too#2017-03-2317:35anmonteirohow does clojure.spec/instrument play with direct linking?#2017-03-2317:35anmonteirowondering if someone has tried that before I go and play with it#2017-03-2317:36Alex Miller (Clojure team)it wonā€™t do anything#2017-03-2317:37Alex Miller (Clojure team)because direct linking doesnā€™t use vars#2017-03-2317:37anmonteiro@alexmiller got me a little confused there. by ā€œwonā€™t do anythingā€ do you mean it wonā€™t instrument or it wonā€™t be impacted by direct linking?#2017-03-2317:46Alex Miller (Clojure team)I mean it wonā€™t instrument#2017-03-2317:48Alex Miller (Clojure team)or rather it will instrument, but direct linked calls donā€™t use either the original var or the instrumented var, so instrumentation will have no effect on those call sites#2017-03-2317:48Alex Miller (Clojure team)new, non direct linked invocations will use the instrumented vars#2017-03-2317:51Alex Miller (Clojure team)@thheller re your question yesterday, there is an implicit assumption in the code that spec impls also implement IObj. so in one sense, Iā€™d say your example is at fault of that. And in another sense that spec code could fall through to throwing an error in that case to better let you know that.#2017-03-2317:53anmonteirothanks#2017-03-2318:13mpenetDirect linking in dev makes little sense anyway doesnt it? #2017-03-2318:14mpenet(since instrument is also dev time)#2017-03-2318:14seancorfieldAgreed. We only enable direct linking on QA/prod. It gets in the way of REPL-driven development.#2017-03-2318:17thheller@alexmiller didn't know about the implicit assumption about IObj, took me about 2h to figure that out. an error would have helped a lot.#2017-03-2318:48Alex Miller (Clojure team)Ticket would be fine#2017-03-2318:49Alex Miller (Clojure team)@mpenet right, there are no plans to change wrt direct linking#2017-03-2318:52thhellerhttp://dev.clojure.org/jira/browse/CLJ-2135#2017-03-2319:54anmonteiro@mpenet @seancorfield we have some functions for which we want to turn instrumentation on in prod, hence my question#2017-03-2319:54anmonteirowe donā€™t have direct linking in dev#2017-03-2319:55mpenetDont do that. Instrument can/will trigger gen. Use pre/post#2017-03-2319:55anmonteirothat would be my next question#2017-03-2319:55anmonteiroweā€™re currently using Plumatic Schema#2017-03-2319:56anmonteiroand we have (s/set-fn-validation! true) for the namespace#2017-03-2319:56anmonteiroapparently instrument wonā€™t do what I expect it to#2017-03-2320:02seancorfieldI would say that if you want the checks to always be performed (i.e., in production) then explicitly code that into the functions themselves ā€” that gives you more control over the error handling anyway.#2017-03-2320:04seancorfieldAs @mpenet notes, even if you did instrument functions in production, any higher-order functions that get instrumented would trigger generative testing, and instrumentation doesnā€™t check function returns anyway, just arguments.#2017-03-2322:49Oliver George@anmonteiro would s/assert be a better choice for production code?#2017-03-2322:51anmonteirohrm I thought s/assertā€˜s point was to be turned off in Prod#2017-03-2322:51anmonteiroCurrently Iā€™m thinking about going with a combination of s/valid? and s/explain that we send to Sentry#2017-03-2323:51Oliver GeorgeSounds sensible.#2017-03-2410:37manuI've just started learning clojure-specs. My question is: what is the right place to put them? in a different namespace or right next to the production code?#2017-03-2411:05mpenetPersonally I like to put them in a separate namespace, but there's no rule about it#2017-03-2412:05Yehonathan Sharvitwhat are the pros/cons of having in same/different namespaces?#2017-03-2412:32mpeneteasier to exclude if it's in another namespace#2017-03-2412:33mpenetother than that it's a matter of taste, diff languages do it both ways (ex haskell vs ocaml)#2017-03-2412:52hospadarQ: is there any documentation of the Spec protocol? I ask because I'm building a couple implementations of it to wrap up some of my more complex predicates. Reason being I want to generate custom error messages that are more descriptive#2017-03-2412:52hospadarI think I've mostly figured it out, but I'm mainly just guessing based on my reading of the implementations already in clojure.spec#2017-03-2412:54hospadarspecific problem: I'm validating a tree-structure that's defined in an EDN file. Not only do I want to validate that the nodes are all structurally OK, I need to validate certain properties of the graph as a whole like "are all the edges pointing at valid nodes" "is it acyclic?"#2017-03-2412:54hospadarMy first approach was to just write those validators completely outside of spec, but that makes them hard to compose (like if I pack this datastructure inside of another one)#2017-03-2412:56hospadarMy second thought was "just use predicates", but an error message like {:pred (not (contains-cycles? graph)) :obj <the whole graph>} is super unhelpful to a user trying to figure out which node is invalid#2017-03-2412:57hospadarSo instead I packed the predicates into a (reify Spec ...) and it's GREAT but it took me a while to figure out a) that there was even a protocol that I could implement b) how to implement it#2017-03-2412:58mpenetI did something similar (spec that validates a dsl: parses, potentially generates error messages with line/num syntax helpers etc)#2017-03-2412:58mpenetbasically a reified Spec with a custom explain*#2017-03-2412:58mpenet+ some caching not to parse again for explain#2017-03-2412:59hospadarwhat I guess I'm really angling at is "If I wanted to contribute some helpful documentation of the spec protocol, would there be interest in receiving that, and how would I go about doing it?"#2017-03-2413:00hospadarI have some other related minor thoughts like "maybe explain-1 shouldn't be private so that Spec implementors can use it without having to #'hack/it"#2017-03-2413:00hospadaryeah @mpenet same thing for me too#2017-03-2413:00hospadarI have a little custom lispey transform language I'm using to do some basic filtering that gets packed inside other specs#2017-03-2413:02mpenetI am not sure what's the status of the Spec protocol, might be considered an (unreliable) implementation detail. Not sure but at the time @alexmiller advised me to do this so I hope not#2017-03-2413:03hospadaryeah I suspected maybe that was the case#2017-03-2413:03hospadar(or might be)#2017-03-2413:07hospadarmaybe a better approach would be extending clojure.spec/spec to take an (optional) custom explain function#2017-03-2413:07hospadarin the same way it takes an optional generator#2017-03-2413:07hospadarit would probably make my code look a little simpler šŸ˜›#2017-03-2413:13mpenetit would be nice yes#2017-03-2413:13Alex Miller (Clojure team)@hospadar the Spec protocol is subject to violent change without warning and weā€™re not interested in documenting it at the current time#2017-03-2413:14Alex Miller (Clojure team)we may not even keep it#2017-03-2413:16Alex Miller (Clojure team)I donā€™t think Rich is interested in having custom explain messages (but I could be wrong)#2017-03-2413:23Alex Miller (Clojure team)I think Richā€™s conception is that in general, people should not be writing their own spec impls, they should be composing the provided specs#2017-03-2413:24souenzzo
(def my-keys [::a ::b ::c])
=> #'my-ns/my-keys
(s/def ::my-keys (s/keys :opt my-keys))
CompilerException java.lang.IllegalArgumentException: Don't know how to create ISeq from: clojure.lang.Symbol, compiling:(/tmp/form-init5422364345568681610.clj:1:18) 
Rly? I will need to do a macro over your macro? Why not allow data on spec declaration? šŸ˜¢
#2017-03-2413:27hospadar@souenzzo you can probably unescape (so long as the list is available at compile time as in your example#2017-03-2413:27hospadar(s/def ::my-keys (s/keys :opt ~my-keys))#2017-03-2413:27hospadarUse that trick in my project.clj to parameterize versions#2017-03-2413:29dergutemoritzThat only works if the macro explicitly allows it#2017-03-2413:29hospadar@alexmiller I feel that, I'd really rather not be writing custom preds. But in some cases I have to and it's nice if I can pack them into my specs with more error detail than just "this predicate failed"#2017-03-2413:31mpenetwell you can (eval (s/def ::foo ~(...)), which is quite horrible#2017-03-2413:31souenzzo
(eval `(s/def ::my-keys (s/keys :opt ~my-keys)))
works, but (s/def ::my-keys (s/keys :opt ~my-keys)) no
#2017-03-2413:32mpenethopefully in the future we can conform the spec form, modify it, then unform it#2017-03-2413:32yonatanel@souenzzo What's your use case for defining keys separately? Are you reusing them?#2017-03-2413:32mpenethttp://dev.clojure.org/jira/browse/CLJ-2112#2017-03-2413:33mpenetwouldn't hold my breath tho#2017-03-2413:52yonatanel@hospadar Would recursive spec solve your issue of informative tree validation explain message?#2017-03-2413:54Alex Miller (Clojure team)CLJ-2112 is work that Rich asked me to do - thatā€™s something we will definitely have#2017-03-2414:33hospadar@yonatanel sometimes yes, sometimes no (at least I'm not sure how I'd implement that right now?)#2017-03-2414:33hospadarI'd love to be proven wrong though#2017-03-2414:35hospadarthat's a super-dumb simple example of the kind of problem I'm trying to validate#2017-03-2414:36hospadarit's pretty easy for me to write a validator (in the form of a predicate) that just reduces over the edges to check their validity, and I can s/and that together with the structural spec at the top level#2017-03-2414:37hospadarbut I want to get a more helpful error message that says "this edge failed validation because this node isn't valid"#2017-03-2414:38hospadarhence why I've started writing implementations of Spec that let me emit more specific error messages#2017-03-2415:13yonatanel@hospadar I wonder if there should be a line between what you should validate with spec and what should go to another validator, other than runtime dependencies such as a database. In your case the rules might be a bit complex but you don't need anything but the data, so spec sounds good, but you can't compose yourself out of every case, so if spec already supports custom predicates and generators, why not explainers?#2017-03-2415:45hospadaryeah that's my thought - if I was going out over the wire during validation ("only valid if exists in DB" situations) I'd probably not attempt to jam it in the spec#2017-03-2415:46hospadarspec is very attractive to me for this kind of problem for me though since it makes it easy to build composable validators and still get helpful robust error reporting#2017-03-2415:50hospadarI feel like spec should be able to validate (with useful errors) any data structure where the correctness of the data-structure doesn't rely on any external factors?#2017-03-2416:02hospadar(i.e. the validation is very deterministic data->spec->answer is the same every time)#2017-03-2419:10luxbockI kind of wish that a spec defined using s/and did not perform conforming while passing the value through its specs#2017-03-2419:11luxbockI find myself defining named predicates that add additional constraints on what the value should be like using s/and, and it would be more natural for those predicates to receive the unconformed value#2017-03-2419:13luxbockI know I could in theory use s/unform inside my predicates, but often times I already have an existing predicate that I use elsewhere as a part of the business logic of the program, so now I have to create a wrapper in order to use it with spec#2017-03-2419:22luxbock
user> (s/explain (s/and (s/cat :a int?) (fn [[a]] (> a 1))) [2])
UnsupportedOperationException nth not supported on this type: PersistentArrayMap  clojure.lang.RT.nthFrom (RT.java:962)
#2017-03-2419:41Alex Miller (Clojure team)@luxbock we have talked about providing both flowing and non-flowing versions of s/and#2017-03-2419:44seancorfield@alexmiller and something for s/or that doesnā€™t produce pairs? (so we donā€™t need (s/conformer val) when we donā€™t care about the label)#2017-03-2419:44Alex Miller (Clojure team)well s/nonconforming exists as a theoretical answer to that#2017-03-2419:44Alex Miller (Clojure team)but Iā€™m not sure whether Rich considers it a good idea#2017-03-2419:46Alex Miller (Clojure team)
(s/conform (s/nonconforming (s/or :a int? :b string?)) 100)
=> 100
#2017-03-2420:14seancorfieldMaybe Iā€™ll chat to him next week in Portland about it. Iā€™m loathe to rely on s/nonconforming since it may go away...#2017-03-2420:23Alex Miller (Clojure team)He wonā€™t be there#2017-03-2420:52tjtoltondoes core.spec have any kind of spec that validates emails?#2017-03-2420:53seancorfield@tjtolton Thatā€™s kind of a hard problem. You can use regexes, but it depends how ā€œcorrectā€ you want to be?#2017-03-2420:54tjtoltonright, haha, which is why I was vaguely hoping there would be some out of the box email validators and generators#2017-03-2420:54tjtoltonWe can dev one ourselves, its no problem#2017-03-2420:54seancorfieldIā€™ll share the regex we use:
(def email-regex
  "Sophisticated regex for validating an email address."
  (-> (str "(([^<>()\\[\\]\\\\.,;:\\
#2017-03-2420:54tjtoltonhahaha#2017-03-2420:55tjtoltonyeah, then the trick is writing a generator for it#2017-03-2420:56seancorfieldThen
(defn valid-email?
  "Return true if the given string looks like an email address."
  [s]
  (boolean (re-matches email-regex s)))
and
(s/def ::email (s/with-gen (s/and (bounded-string 5 128)
                                  wstr/valid-email?)
                 (wgen/fn-string-from-regex wstr/email-regex)))
where fn-string-from-regex is a wrapper for test.chuckā€˜s string-from-regex generator.
#2017-03-2420:57seancorfieldIt generates some really wild email addresses šŸ™‚#2017-03-2420:57tjtoltonooh#2017-03-2420:57tjtoltonstring from regex#2017-03-2420:57tjtoltonthat's neat#2017-03-2421:12seancorfield@tjtolton Depending on what youā€™re doing with generated emails, you might want a simpler regex for the generator portion ā€” hereā€™s just two examples from s/exercise: ā€ņ‚‹•ņ¤¹ņ±µŗš§¶Š@[858.0.376.98]" ā€\"ń³­Šš±¢†ņ˜Ŗ„󱬈ņ“–æņ¬œ†\"@5cl84.e.a2.QN-.hpr1s.H.Ikp"#2017-03-2421:12seancorfieldHmm, the unicode didnā€™t survive the copyā€™nā€™paste...#2017-03-2421:13seancorfieldAnyway, you get the idea.#2017-03-2421:19tjtoltonYeah, I do, thanks!#2017-03-2421:20tjtoltonTest.chuck is pretty neat#2017-03-2421:33don.dwoskeNewbie here. Iā€™m trying to write a multi-spec to handle a case like this:
{
            ::id "1"
            ::type "person"
            ::properties {
                ::firstName "Elon"
                ::lastName "Musk"
                ::email "
For a particular type value, I want the acceptable properties keys to be different, while every instance has an :id, :type and a few other fields. The properties are variable, but not the parent/core keys. In the examples Iā€™ve seen for multi-specs the type is always inside the variable collection itself.. e.g. Iā€™d need to move type into properties instead of leaving it at the top level.
#2017-03-2421:59kidpolloI think that you can perfectly use a multi-spec with the type where it is. Your entity specs will all have id, type and properties. No need to complicate things IMO#2017-03-2422:23don.dwoskelikely - but as I said - newbie - I donā€™t know how to do it. Iā€™m fooling around with using a nested s/spec in there to get the properties map returned for each type in the multi-spec.#2017-03-2513:01yonatanel@don.dwoske Where do you get this data from? Can you use :person/properties and :house/properties instead or do you treat ::properties as just a bag of key-value pairs? Or maybe use :properties without namespace so you can have the same keyword with different specs such as (s/def :person/properties ...) and then (s/keys :req-un [:person/properties] ...).#2017-03-2513:03yonatanelOr perhaps be strict about a flat structure such as {::id 2, ::type "house" :house/color "red" :house/style "Victorian"}#2017-03-2513:33don.dwoskeIā€™ve got a flat structure working decently now. Your idea to use different namespaces for each properties list is a good one, that should work I think. The spec is used to validate data with unqualified keys, so it fits. I only need the properties map name to be consistent across all types. We have not yet finalized our design for these payloads, but if I can get the specs working both in flat and nested properties format, that would be excellent.#2017-03-2513:38don.dwoskeBTW, for those interested, Iā€™m building out a proof-of-concept to compare using clojure.spec on an enterprise level to define and validate our entity types vs. something homegrown vs. XSD, vs. JSON Schema vs. Avro vs. whatever else. Barring any setbacks, I may propose using clojure.spec as the place-of-truth for these, and using tools to translate to any other format we may need.#2017-03-2513:41don.dwoskeIā€™d be curious also to know if there is a way to use clojure.spec to define a web API contract and then auto-generate swagger docs from it.#2017-03-2514:28yonatanel@don.dwoske There's something related here: https://github.com/metosin/spec-tools#2017-03-2514:29yonatanelAnd maybe here: https://github.com/mpenet/spex/blob/master/src/clj/qbits/spex/json_schema.clj#2017-03-2515:52mpenetThe json schema stuff in spex is far from finished. Wouldnt rely on it#2017-03-2515:52mpenetWe might just use graphql instead actually#2017-03-2612:23don.dwoske@U050SC7SV we are looking at graphql as well - any particular database or libraries in mind?#2017-03-2612:25mpenetWe use a mix of ES and cassandra, but graphql usage doesnt really depend on a particular datastore. I am experimenting with the lib from Walmartlabs, so far impressed#2017-03-2515:54devthI thought you could lookup :ret/`:args` on a spec, e.g. (:ret (s/get-spec 'full/symbol)) but this is just returning nil šŸ¤”#2017-03-2515:54devththe type of the spec is clojure.spec$or_spec_impl$reify__13856#2017-03-2516:29devthright so i wasn't trying it on an fdef spec. i was trying it on an s/keys spec. however, still can't seem to get much useful info out of it. (e.g. which keys are specified as :req vs :opt)#2017-03-2523:47hiredmanYou have to call s/form and parse the form#2017-03-2623:33hueypwarning: new to spec šŸ™‚ I discovered conformer and have a case where Iā€™m using it purely to transform the conform output. Found this about that pattern (re: hiding the spec inside conformer): "You could keep the conversion out of the spec and do that via a transformation or via a non-registered spec too.ā€. Anyone have an idea on the ā€˜via a non-registered specā€™ option? šŸ™‚#2017-03-2700:06hueyphere is an example: https://gist.github.com/eyston/c797efc5e4c07304e0c331b97decd107 ā€” was doing the first thing and its opaque, tried the second way after reading http://thegeez.net/2016/12/09/parsing_clojure_spec_advent_of_code.html#2017-03-2700:07hueypunsure if good / bad idea šŸ™‚#2017-03-2702:39seancorfield@hueyp As someone who uses s/conformer quite a bit, I'll caution against specs that transform their data (beyond the conforming that the specs themselves do). Alex Miller and other Cognitect folks also caution against mixing transformation into your specs. The problem is that once you put that in a spec, all client uses of the spec have no choice about the transformation -- they all have to deal with the conformer in the spec.#2017-03-2702:40hueypIā€™m building a parser and was kind of blown away that with conformer you could totally have it output the AST šŸ™‚#2017-03-2702:40seancorfieldWe do it for a very specific use case: where our input is always strings (from an HTTP request) and we want numbers or keywords out of the specs (and they're coded to accept numbers or keywords too, as well as strings).#2017-03-2702:41seancorfieldYeah... just because you can do something, doesn't mean you should šŸ™‚#2017-03-2702:41hueypexactly why Iā€™m here ! šŸ™‚ thx for feedback~#2017-03-2702:42seancorfieldIt's likely to be better for you down the road, to separate the transform to AST from the specs. Otherwise you're complecting validation (specs) with transformation. If you keep them separate, they'll be easier to test and potentially easier to reuse.#2017-03-2702:45seancorfieldSpec is great tho'... we've been using it in production code for quite a while (we have Clojure 1.9.0 Alpha builds in production).#2017-03-2717:20jiangtsiā€™m specing a datastructure in clojurescript#2017-03-2717:20jiangtsand the keys are coming in as strings#2017-03-2717:20jiangtsIā€™d like to write the spec with s/keys but that only supports keywords. Am I missing something simple?#2017-03-2717:26xiongtxSee every-kv and map-of: https://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec/every-kv#2017-03-2717:30jiangtsthanks! I havenā€™t used every-kv, but as far as I know using map-of I canā€™t specify that certain keys are required and others are optional#2017-03-2717:32jiangtscan that be done with every/every-kv?#2017-03-2717:40xiongtxYouā€™ll have to use some logical combinations w/ s/and and such, I think#2017-03-2717:43zanePretty sure the recommended approach is to conform the keys to keywords. Are you worried about performance?#2017-03-2804:25jiangtsyeah, worried about performance, a lot of nested maps with strings as keys#2017-03-2804:26jiangtsperhaps I can replace keys with keywords with specter or something#2017-03-2804:27jiangtsbut yeah, will be converting a couple thousand string keys to keywords, nested to ~20 levels deep, so would like to avoid if possible#2017-03-2806:19xiongtxNo need for specter, I think: https://clojure.github.io/clojure/clojure.walk-api.html#clojure.walk/stringify-keys Benchmark to see if thereā€™s actually a performance issue. A few thousand keys doesnā€™t sound like a performance bottleneck.#2017-03-2717:21jiangtsIā€™d rather not coerce the data to use a bunch of keywords cause I have a lot of data#2017-03-2721:33liamdhiya are there any good reference projects or tutorials on how to integrate spec into real world projects#2017-03-2721:33liamdmost of the examples iā€™m digging up explain how it works well but seem ā€œcontrivedā€ for lack of a better term#2017-03-2721:33liamd(e.g. modeling decks of cards)#2017-03-2721:49seancorfield@liamd I suspect most ā€œreal world projectsā€ are closed source and canā€™t easily be discussed...#2017-03-2721:49liamdi mean is spec being used in any popular open source libs?#2017-03-2721:50liamdi guess iā€™m looking for ā€œbest practicesā€. do i create a new file for my specs, do i do runtime checks with them, etc.#2017-03-2721:58seancorfield"popular open source libsā€ are nearly all remaining compatible with earlier Clojure versions...#2017-03-2721:59seancorfieldItā€™s too early for much in the way of best practices with spec to be codified.#2017-03-2721:59seancorfieldWeā€™re unusual (at World Singles) in being willing to run Alpha versions of Clojure in production.#2017-03-2722:00seancorfieldFor libraries that need to support earlier versions of Clojure, specs must be in a separate file so that folks on Clojure 1.9 can require it, but folks on earlier versions can ignore it.#2017-03-2722:00seancorfieldFor apps that run on Clojure 1.9, they can have specs inline with functions.#2017-03-2722:01seancorfieldWeā€™ve found the most value so far comes from specifying data not functions. We use spec to validate input parameters for our REST APIs and weā€™re starting to use it to validate user input in some web apps as well.#2017-03-2722:02seancorfieldSo, yes, for us it is runtime checking in production.#2017-03-2722:03seancorfieldBut itā€™s also part of testing ā€” thatā€™s where we instrument code (prior to running our test suite, for example) and some limited automation of generative tests (beware of the increased time such testing takes ā€” consider running it separately from your ā€œunit testsā€, so that it can be run ā€œas neededā€ by developers and as long form testing in CI perhaps).#2017-03-2722:04seancorfieldDoes that help @liamd ?#2017-03-2722:04liamdyes absolutely#2017-03-2722:04liamdso what iā€™m gathering is that standard practices havenā€™t been established yet necessarily because itā€™s really still in alpha#2017-03-2722:04liamdi suppose in that case i just have to play around and see what works for me#2017-03-2722:05liamdit seems like thereā€™s lots of space for libraries to be developed on top of spec to enforce certain uses and patterns, like perhaps ring middleware, or test suite integration or cleaner errors from explain#2017-03-2722:08seancorfieldOur experience so far re ā€œcleaner errorsā€ is that requires application domain knowledge, so weā€™ve created some custom code that maps from explain-data to our application domain, using some heuristics that work for the sorts of spec failures we might reasonably get.#2017-03-2722:08seancorfieldAs for the rest of it, yeah, I think itā€™s just too early for much to have solidified.
#2017-03-2722:08liamdthatā€™s actually pretty exciting, i guess iā€™ll just have to jump in#2017-03-2722:08liamdthanks!#2017-03-2722:10seancorfieldWill you be at Clojure/West later this week?#2017-03-2722:15liamdNope, Iā€™m in Chicago and donā€™t have the spare cash. Looking forward to the screen casts that come out of it though.#2017-03-2723:00sparkofreasonIs there any way to control the length of keywords generated for the keyword? predicate?#2017-03-2800:11danboykisCan anyone explain this? This works:
((fn []
   (if-let [k nil]
     k
     :other-keyword)))
=> :other-keyword
But this doesn't:
((fn []
  (if-let [k nil]
    k
    :clojure.spec/invalid)))
CompilerException clojure.lang.ExceptionInfo: Call to clojure.core/if-let did not conform to spec:
In: [2] val: :clojure.spec/invalid fails at: [:args :else] predicate: any?
...
The only difference being a keyword. I ran into this writing a conformer.
#2017-03-2800:14seancorfield:clojure.spec/invalid is a special value ā€” you have to be really careful about code that produces it#2017-03-2800:15seancorfieldI think you can do (keyword ā€œclojure.specā€ ā€œinvalidā€) as one way to get around that (i.e., generate it at runtime).#2017-03-2800:15seancorfield(I suspect thereā€™s a cleaner way)#2017-03-2800:17danboykisseancorfield, I agree that it's special, I am using it in a conformer#2017-03-2800:18danboykisthis is a stripped down example#2017-03-2800:18seancorfieldItā€™s special to spec itself. And in 1.9, spec is used to syntax-check various macros.#2017-03-2800:19danboykisin your opinion, is this expected behavior?#2017-03-2800:19seancorfieldAnother option is
(def csi :clojure.spec/invalid)
ā€¦
((fn [] (if-let [k nil] k csi)))
And, yes, this is absolutely expected behavior.
#2017-03-2800:20seancorfieldIt is certainly a known issue but I donā€™t know whether there are plans to change the behavior.#2017-03-2800:21danboykisit seems weird that there are special keywords i can't return from an if-let#2017-03-2800:21seancorfield(I was actually a bit surprised that I could just def a global alias for it like that)#2017-03-2800:22danboykiswhy not? it's just a keyword#2017-03-2800:22seancorfieldhttp://dev.clojure.org/jira/browse/CLJ-1966#2017-03-2800:23danboykisoh ok great, someone filed it#2017-03-2800:24seancorfieldKnown since last June and no indication that it really is a problem that needs to be fixed (and it has an easy workaround ā€” that Iā€™d simply forgotten!)#2017-03-2800:24seancorfield(ironic that I was the one that provided the easy workaround in that ticket but Iā€™d forgotten)#2017-03-2800:25seancorfieldso (if-let [k nil] k ':clojure.spec/invalid)#2017-03-2800:25danboykisthat's a cool workaround šŸ™‚#2017-03-2800:26seancorfieldSince macros are just functions on code-as-data, spec canā€™t tell the difference between a conforming spec that produces :clojure.spec/invalid ā€” the special value ā€” and a literal :clojure.spec/invalid in the actual code.#2017-03-2800:26danboykisi hope it get's fixed#2017-03-2800:26danboykishttps://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec/conformer#2017-03-2800:27danboykisthe docs should probably change then#2017-03-2800:27seancorfieldI suspect the answer from the Clojure/core folks will be: if youā€™re writing a spec or a conformer that needs to produce the special invalid value, you must quote it.#2017-03-2800:27danboykisit's not at all obvious that you're supposed to quote it#2017-03-2800:27seancorfieldYou can return :clojure.spec/invalid in all sorts of constructs just fine. You only need to quote it in some situations.#2017-03-2800:29seancorfieldWe have all sorts of specs and conformers that produce :clojure.spec/invalid as a literal value ā€” we just donā€™t have code around it that is a macro that is checked by spec in the compiler.#2017-03-2800:30seancorfield(if (some-check v) v :clojure.spec/invalid) works fine, as does (try (some-coercion v) (catch Exception _ :clojure.spec/invalid))#2017-03-2800:34danboykisi wonder what will happen once cond is spec'ed too#2017-03-2800:34danboykisyou can return it now#2017-03-2800:34danboykisbut not later?#2017-03-2801:19danboykiscoming back to our discussion the crux of the issue is the following#2017-03-2801:19danboykis
(s/valid? (s/* any?) [::s/invalid])
=> false
(s/explain (s/* any?) [::s/invalid])
In: [0] val: :clojure.spec/invalid fails predicate: any?
=> nil
(any? ::s/invalid)
=> true
#2017-03-2801:20danboykisthat's a contradiction#2017-03-2807:11athosSorry for breaking into the discussion. Does anyone know it's a known limitation or an unintended behavior that multi-spec in general cannot be unformed?#2017-03-2814:48tjtoltonIs there some kind of core.spec function for "generate a view of this data structure that conforms to this spec"? for instance
(s/def ::just-one-key (s/keys :req-un [::apples]))
(*mystery-function* ::just-one-key {:apples "apples" :bananas "bananas"})
;-> {:apples "apples"}
#2017-03-2814:49tjtoltonconform is supposedly for destructuring, but perhaps my understanding of destructuring is wrong, because intuitively I feel like "select this known data pattern out of this unknown data" is what destructuring is for#2017-03-2814:55tjtoltonBasically, the use case here is "web request should conform to this data, but if you pass in other data, that's perfectly fine. But when we store the data off in the database, we should only select the part that we understand and care about."#2017-03-2814:56tjtoltonso, the spec's don't disallow keys, but we select the parts we're interested in for passing along#2017-03-2814:59borkdudeRecently I sometimes see error messages like these: adzerk.boot_cljs.util.proxy$clojure.lang.ExceptionInfo$ff19274a: ERROR: Attempting to call unbound fn: #'clojure.spec/macroexpand-check at file /Users/Borkdude/.boot/cache/tmp/ā€¦/app/1x42/fg7eat/public/js/app.out/ajax/core.cljc, line 591, column 1 I think it was after upgrading clojure.future.spec to alpha15#2017-03-2814:59borkdudeNot sure where it comes from#2017-03-2815:09borkdudeNow I get: Attempting to call unbound fn: #'clojure.spec/macroexpand-check at line 2164, column 1 in file /Users/Borkdude/.boot/cache/tmp/.../app/1xx5/-x902hh/public/js/app.out/cljs/pprint.cljs#2017-03-2815:12borkdudeHmm, sorry, this was with alpha-14. Now I upgraded to clojure.future.spec alpha 15.#2017-03-2815:47tjtoltonthis is crazy, it doesn't even make any sense to me. the guide here https://clojure.org/guides/spec seems to imply that conform is used for destructuring. Here's the example it gives:
(defn configure [input]
  (let [parsed (s/conform ::config input)]
    (if (= parsed ::s/invalid)
      (throw (ex-info "Invalid input" (s/explain-data ::config input)))
      (for [{prop :prop [_ val] :val} parsed]
        (set-config (subs prop 1) val)))))
We can see that "input" is returning from "conform" as "parsed". But it doesn't actually modify the input in any way, unless the input was nonconforming, in which case it returns a special value, which it then explicitly checks for
#2017-03-2815:48tjtoltonwhy not just do (if (s/valid? ::config input)?#2017-03-2816:39seancorfieldDestructuring plucks out the parts you care about ā€” but leaves all the other parts in place (which you can see with :as).#2017-03-2816:40seancorfieldClojure takes an inherently ā€œopen for extensionā€ stance on that sort of thing: itā€™s considered good practice to be able to accept (and ignore) additions to data structures that still conform.#2017-03-2816:40seancorfieldIn order to save stuff to a database, you need to explicitly restrict the keys in a map ā€” either with select-keys (ignoring the extra keys) or something that checks youā€™ve only get the expected keys and no others.#2017-03-2816:42seancorfieldIn general, when saving to a database (with java.jdbc), we tend to build the map we need from the conformed input map, so we explicitly select the columns we need and ignore the rest. ^ @tjtolton#2017-03-2816:47tjtoltonyeah, that's a shame @seancorfield I'm totally down with the "open for extension" concept, but I'm also interested in reducing the cost of detection with spec problems.#2017-03-2816:47tjtoltonBasically, permissive specs are great, but they let you do stupid things, like mispelling an optional key#2017-03-2816:48tjtoltonif you mispell an optional key, you have no way of knowing about it. Nothing will ever fail it#2017-03-2816:53seancorfieldI would expect code that called a function incorrectly like that to be caught by tests on the calling function.#2017-03-2816:54seancorfieldi.e., if func-a calls func-b with misspelled arguments, tests for func-a should catch that.#2017-03-2817:27tjtoltonso, its interesting, because I basically just want a select-keys, but one that is recursive and spec aware#2017-03-2817:28tjtoltonIt's still very strange to me that it isn't available. Even in an "open for extension" system, this seems like an important utility.#2017-03-2817:30tjtoltonEspecially for web programming. I only want to display these values on screen, I only want to return these values in the api call, I only want to save these values to the kv store#2017-03-2817:30tjtoltonall things that spec is great at#2017-03-2817:30ikitommi@tjtolton in spec-tools there is a conformer that drops extra keys out of keys-specs. Will blog about those soon.#2017-03-2817:30tjtoltonI should be able to reuse that functionality#2017-03-2817:32tjtolton@ikitommi spec tools?#2017-03-2817:33ikitommitjtolton: yes, metosin. Just finishing up first release. Comments welcome!#2017-03-2817:34tjtoltonawesome, will take a look!#2017-03-2817:38tjtoltonin the strip-extra-keys function, what is that first argument? is that something that gets generated by core.spec?
(defn strip-extra-keys [{:keys [:keys pred]} x]
  (if (map? x)
    (s/conform pred (select-keys x keys))
    x))
#2017-03-2819:08ikitommiit's the Spec Record. Sadly, clojure.spec/Specs are not extendable and we need to wrap them to make them extendable. There is a example in the readme.#2017-03-2817:32tjtoltonmetosin?#2017-03-2820:38ddellacostais there a way to enforce that a pair of keys exists in a map, or else neither? That is, if one of them exists the other is required, however they are both optional otherwise.#2017-03-2820:41schmeeddellacosta something like
(fn [m]
  (or (and (:k1 m) (:k2 m)) 
      (or (not (:k1 m)) (not (:k2 m)))))
should do the trick
#2017-03-2820:41schmeeie. just use a regular function#2017-03-2820:42ddellacostaugly but I guess it works. Iā€™ll have to compose it with s/keys so I can enforce the presence of other keys#2017-03-2820:42ddellacostahmm actually Iā€™m not exactly sure how I would#2017-03-2820:43schmee(s/and (s/keys ā€¦) key-check-fn)#2017-03-2820:49ddellacostaI think the second term should be (not (or (:k1 m)) (or (:k2 m)))#2017-03-2820:52dergutemoritz@ddellacosta (s/keys :req [(and ::foo ::bar)])#2017-03-2820:52ddellacostahuh, that works? neat#2017-03-2820:52dergutemoritzYep, check the doc string#2017-03-2820:52ddellacostaoh but wait, that means they are both required always, doesnā€™t it?#2017-03-2820:53dergutemoritzAh yeah, maybe you have to put it in :opt#2017-03-2820:53ddellacostaoh but if I can do that then still, problem solved#2017-03-2820:53ddellacostalessee#2017-03-2820:54dergutemoritzah no, it's only supported in :req#2017-03-2820:54ddellacostawell, that is lame#2017-03-2820:55ddellacostabut, great idea dergutemoritz , thanks for trying#2017-03-2820:55dergutemoritzToo bad šŸ™‚#2017-03-2820:55ddellacostayeah#2017-03-2820:55dergutemoritzWell, good to know in other situations anyway!#2017-03-2820:55ddellacostayeah!#2017-03-2820:56ddellacostain the end this worked:
26ā”‚  (s/def ::thing
 27ā”‚    (s/and
 28ā”‚     (s/keys :req [::required1]
 29ā”‚             :opt [::optional1 ::optional2 ::start-date ::end-date])
 30ā”‚     (fn [m]
 31ā”‚       (or (and (::start-date m) (::end-date m))
 32ā”‚           (not (or (::start-date m) (::end-date m)))))))
#2017-03-2820:57ddellacostahopefully including them in the :opt enforces value checking on those fields, going to check now#2017-03-2820:58dergutemoritzYeah, s/keys checks all keys, even those not specified#2017-03-2820:58dergutemoritzHeh ok that's a bit ambiguous#2017-03-2820:58ddellacostayeah, I guess I should have assumed that#2017-03-2820:58dergutemoritzEven those not passed to s/keys if they have a spec#2017-03-2820:58ddellacostabut yeah, that worksā€”thanks @schmee and @dergutemoritz for the help#2017-03-2820:58dergutemoritzYou're welcome!#2017-03-2820:58ddellacostaoh yeah? didnā€™t realize that, thatā€™s handy#2017-03-2820:59dergutemoritzYeah, it's handy but has been the source of surprise in this channel more than once šŸ™‚#2017-03-2820:59ddellacostaI can imagine#2017-03-2820:59dergutemoritzI actually wonder why and and or are not supported in :opt now ...#2017-03-2821:00dergutemoritzYour case makes it seem like a reasonable thing to expect#2017-03-2821:26seancorfieldNote that or in :req isnā€™t mutually exclusive: (s/conform (s/keys :req [(or ::a ::b)]) {::a 1 ::b 2}) succeeds and produces {::b 2 ::a 1}#2017-03-2821:27seancorfieldIt just means "at least one of these must be presentā€. Both can be present in the normal ā€œopen for extensionā€ model that spec has.#2017-03-2821:27seancorfield^ @dergutemoritz @ddellacosta#2017-03-2821:29ddellacosta@seancorfield yeah, in this case the problem is that I want these two to either be both present, or not at all#2017-03-2821:29ddellacostabut yeah, good to note#2017-03-2821:29seancorfieldSince :opt just means ā€œand here are some more keys you may encounter that have associated specsā€ ā€” because additional keys are always accepted, they just wonā€™t be assumed to have associated specs to conform to.#2017-03-2821:29ddellacostaright, that makes sense now that Iā€™ve worked through this#2017-03-2821:31seancorfield@ddellacosta Yeah, the best you can do here is specify them both as :opt if you want them conformed, and then s/and a key check predicate around s/keys. Weā€™ve run into this a lot with mutually exclusive keys ā€” we have to have a separate predicate that checks at most one of them is present. Not sure whether I feel this should be built-in or not...#2017-03-2821:31ddellacostayeah, thatā€™s exactly what I ended up doing. Not the most elegant thing but it works well.#2017-03-2821:32ddellacostaI mean, I feel like @dergutemoritz ā€˜s suggestion would have done that very elegantly#2017-03-2821:32ddellacostabut maybe thatā€™s challenging to implement, I dunno#2017-03-2821:32ddellacostahavenā€™t looked at the actual clojure.spec code so dunno#2017-03-2823:56mattlyI'm using spec/conform to destructure & validate a largish data structure from a client, and at some point it's using coll-of. But in the process, conform seems to be reversing the contents of the list its given. Is this expected/normal?#2017-03-2900:15xiongtxSeems like this issue: http://dev.clojure.org/jira/browse/CLJ-1988#2017-03-2900:16mattlyindeed#2017-03-2913:54Alex Miller (Clojure team)Yes, sounds like that - thatā€™s been fixed (canā€™t remember which alpha it was in). You might also look at every-kv (which will not conform its input at all).#2017-03-2900:05mattlyweird, I can't reproduce it in the repl#2017-03-2900:10mattlyit's definitely happening in the call to conform though#2017-03-2900:14mattlyupgrading from alpha14 to 15 made it go away#2017-03-2915:20alex.ter.weeleWhatā€™s the easiest way to make a spec that matches {:a 1 :b 1}, {:a 1}, and {:b 1}, but not {}?#2017-03-2915:21bronsa(s/keys :req-un [(or ::a ::b)])#2017-03-2915:25alex.ter.weeleis that regular or or spec/or?#2017-03-2915:29bronsait's s/keys syntax that expands to clojure.core/or#2017-03-2915:30alex.ter.weeleok, I got it. Thanks!#2017-03-2917:28mandersonIs there a way to provide more detailed explain output for a custom conformer? In alpha15 I get a message like: val: "k" fails predicate: (conformer conform-int). I see I can supply an optional "unform", but no arity to supply an "explain"...#2017-03-2917:47bjais there a way to turn off fdef validation for defn temporarily in -alpha14? I'm seeing a failure in a third party lib that works in another project.clj so I think I'm having classpath issues and I need to load the namespace so I can interogate some objects via the classloader#2017-03-2917:53hospadar@manderson you can implement the Spec protocol to provide a custom explainer#2017-03-2917:53hospadarword on the street is that the protocol may be changed in the long run#2017-03-2917:54hospadarbut for now it works fine#2017-03-2917:55hospadarif you look at the Spec implementations included in the lib (that are used for keys and tuples, etc) you can get an idea of what kind of output they return#2017-03-2917:56hospadarif errors, return a list of errors (like what you get from explain-data), otherwise return empty list/nothing#2017-03-2917:56hospadarthat example assumes you don't want to do any special conformation#2017-03-2917:57mandersonGotcha, thanks @hospadar. Figured that might be the only way, but was hoping for a "published" approach instead of looking behind the curtain. I'll give that a try.#2017-03-2918:14hospadaryeah I feel that#2017-03-2920:25Alex Miller (Clojure team)@bja you can s/def your own spec over the top of the standard one#2017-03-2922:53bja@alexmiller thanks. that's what I eventually settled on#2017-03-2922:54bjafound out that I had a conflict with an older version of potemkin generating non-conforming functions.#2017-03-3014:59cgrandHi! Going meta: has anyone worked on generating (from a spec) the spec of itā€™s conformed data?#2017-03-3015:17zaneNo, but I've thought about writing that function and would use the hell out of it.#2017-03-3018:53alex.ter.weeleon a related note, Iā€™ve been wrestling with the "entry pointsā€ to spec. Once I learned that s/conform produces a new structure, e.g. when going through an s/or, I switched to using s/valid?. So what is the use case for s/conform? It sort of works like s/explain that produces an explanation on success too.#2017-03-3019:56adambros@alex.ter.weele my main usage of conform has been in macros to parse the body in a way that I can emit things in a much simpler way#2017-03-3020:05seancorfield@alex.ter.weele Producing a new, annotated data structure is really useful when your code needs to be able to branch on those or tags.#2017-03-3020:07seancorfieldFor us (World Singles), we use conform to process our REST API inputs (all string values) and process keywords, numbers, dates, and strings -- with annotations around ranges of values etc.#2017-03-3020:07cgrand@alex.ter.weele validate AND transform#2017-03-3101:36derwolfeUsing spec, is there a way to specify something like a future or deferred that should contain a map with a set of keys? For instance, I'm using manifold/futures to make some calls to AWS; I'd like to write specs for these, but if they only represent that the functions return instances of manifold/futures, I'm not sure how valuable that would be.#2017-03-3102:05derwolfePer the mailing list, it seems like this is similar situation to trying to spec protocol methods which seems to be discouraged in favor of wrapping protocol methods with functions and speccing those.#2017-03-3102:10seancorfieldYeah, it makes more sense to spec the functions involved that use the underlying data structures. #2017-03-3102:29derwolfeThanks šŸ˜„#2017-03-3109:46yendais there a way to remove everything that is not speced in a datastructure ? i.e the datastructure is valid but the legacy system behind it won't like extra fields to be present.#2017-04-0319:49mobileinkyenda: the way i do this with maps is, read the registry to get all deffed kws, then walk your data, and when you hit a map, iterate over the entries. for each key, if it is not in the registry, discard the entry.#2017-03-3110:15yonatanel@yenda You need to use something like select-keys yourself#2017-03-3110:16yendathe problem with this solution is that it is repeating what is already in the spec, also it is hard/not-really-readable to recursively pick the keys in the structure#2017-03-3110:19yonatanel@yenda This is often requested here. Keep an eye on http://dev.clojure.org/jira/browse/CLJ-2112 for easy extraction of the keys. Also, spec-tools and spex might have something: https://github.com/metosin/spec-tools https://github.com/mpenet/spex#2017-03-3110:22mpenetspex doesn't have this#2017-03-3110:23mpenetspec-tools does I think#2017-03-3110:23yendathanks#2017-03-3110:23yonatanel@mpenet I just like to link to your lib and see you apologize for it not being complete :)#2017-03-3110:24mpenetšŸ™‚ I should add a notice in the readme#2017-03-3119:56richardhGot a somewhat general question here ā€” I am trying to do a spike to see what it would be like to convert one of our services from prismatic schema to clojure spec. There is a clojure.test-compatible fixture in schema which runs all the functions in a test suite through a validator, and throws an error if their arguments and return values don't match their declared schemas. It strikes me that there's probably no way to do this easily with spec -- that the reason this works in schema is that the functions themselves were created using schema/defn, so they were already set up for this kind of thing. And that the only way to do something similar using spec is if you can find or write all the necessary generators to run the function through its paces in generative tests. (Which in our case seems like a daunting task as -- for better or worse -- there is a lot of hairy Java and Scala interop, futures, server responses, etc. that is all painstakingly mocked out in the tests.) Does this sound about right? I think this points to what @seancorfield was saying in a recent post, which is that the most leverage in spec comes from making schemas for data, not functions. At least not all the functions.#2017-03-3120:02seancorfieldYou can instrument (turn on spec checking) when you run tests but, if you have higher-order functions spec'd, you'll get generative testing on those. And instrument only checks arguments not return values. The idea is that instrument is for checking that functions are called correctly, not that they behave correctly. Generative testing is for behavior and is considered to be a separate type of task to "unit testing".#2017-03-3120:05seancorfieldAlex was talking last night (spec unsession at Clojure/West) about an experimental generative test runner Cognitect are working on which would run continuously and watch for source changes, and analyze which functions (and therefore which specs) need to be exercised as a result of each source change -- rather than running all tests each time.#2017-03-3120:05richardhThat sounds awesome#2017-03-3120:05seancorfieldAt World Singles, we've definitely focused on spec'ing data structures (and, for us, particularly using spec for validation and conformance of REST API parameters). Spec is very strong for that.#2017-03-3120:08seancorfieldThe generative testing is also amazing but, yeah, if you've got a lot of interop, it's really hard to generate those things. But the normal flow of running (unit) tests doesn't sit well with generative testing: unit tests run very quickly, generative tests often run slowly. We haven't yet settled on a good workflow for running those (we run a few generative tests from unit tests, where they run quickly, but we run them manually -- via check for example -- "from time to time").#2017-03-3120:11richardh@seancorfield Interesting, thanks for the responses. Yeah, Iā€™ve been playing around with spec and test.check for a while in toy projects and thinking, this is awesome, we should use them on the codebase at work! But our actual production code base is another story. Would probably be easier if we were thinking in terms of spec and generative tests when it was originally being written.#2017-03-3120:11richardhWeā€™re basically writing Scala in Clojure, is whatā€™s happening.#2017-03-3120:12seancorfieldYeah, following a TDD / BDD workflow leads to nicely testable code. Adding tests later can be much harder.#2017-03-3120:12richardhyup#2017-03-3120:14richardh@seancorfield Do you know what is the rationale behind having instrument not be able to check the return value? Thatā€™s basically the core of my original question, I think.#2017-03-3120:15richardhAlso, can you turn on instrument for all functions for which specs exist, or do you have to instrument the functions one by one?#2017-03-3120:17seancorfieldYou can (instrument) to turn on checks for all functions that are currently loaded (so it won't instrument namespaces that haven't yet been required!).#2017-03-3120:17seancorfieldOr you can (instrument sym-or-syms) for specific functions.#2017-03-3120:17richardhthat sounds good#2017-03-3120:20seancorfieldThe args vs ret thing is philosophical: Clojure/core folks believe that verifying behavior of a function is best done by generative testing based on stated properties. rather than test-by-example. Whereas instrumentation is something you want in place while you're developing / exercising code under test, to check calls are being made correctly.#2017-03-3120:21seancorfield(I hope that accurately captures what @alexmiller has been communicating about why there's this dichotomy in spec?)#2017-03-3120:21richardhYeah, that makes sense. So instrumentation should be thought of as a dev tool, not something that you run in your automated test suite.#2017-03-3120:22Alex Miller (Clojure team)Yes. And I'll note that opinions on this vary even in the core team. :)#2017-03-3120:23richardh@alexmiller ha ha. And by that do you mean that opinions and dev practices vary, or that the dev practices are constrained because of the current state of the tools, and some people are of the opinion that that state should change so that they could use different dev practices?#2017-03-3120:24Alex Miller (Clojure team)The latter#2017-03-3120:24richardhinteresting#2017-03-3120:24Alex Miller (Clojure team)I think it would be useful to have a way to instrument with ret checking#2017-03-3120:25richardhYou know this Slack record is being recorded. šŸ™‚#2017-03-3120:25Alex Miller (Clojure team)No worries :)#2017-03-3120:26seancorfieldWhen I started spec'ing clojure.java.jdbc, I set it up so when the test suite runs, all functions are instrumented. We don't do that consistently at World Single (but we don't have a huge number of functions spec'd -- see above about our focus on data structure specs).#2017-03-3120:26Alex Miller (Clojure team)I agree with the philosophy above but I do not think that stest/check is the only possible kind of test where ret checking would be useful #2017-03-3120:28seancorfield(I was vocal and upset when :ret-checking stopped being done via instrument -- but I accepted Rich's explanation of why that was changed)#2017-03-3120:28richardhIt used to be there and it was removed?#2017-03-3120:29seancorfieldYup, in an early alpha.#2017-03-3120:30richardhDo you have a link to any posts detailing the explanation youā€™re referring to?#2017-03-3120:30richardhIf not, no worries#2017-03-3120:32richardhI think one of my main issues is probably something that itā€™s not the Clojure teamā€™s job to fix, which is that I work in a place dominated by Scala, and itā€™s a lot easier to justify things that bear some resemblance to the way they do things, like being able to easily validate return types in example tests. But thatā€™s not Clojureā€™s problem!#2017-03-3120:33richardh@seancorfield and @alexmiller , thank you very much for taking the time to answer my questions.#2017-03-3120:35seancorfieldI think the most promising avenue will be new forms of test runners that are spec-aware and therefore some new workflows.#2017-03-3120:37richardh@seancorfield Yes, Iā€™ve been thinking the same thing. That there is an amazing base available now, on top of which some tools could be built.#2017-03-3120:38seancorfieldIt's something I'll probably explore with Expectations, now that I maintain that...#2017-03-3120:38richardhMaybe Iā€™ll contribute!#2017-03-3120:39seancorfieldWe have an #expectations channel and input is always welcome on both the test library itself and the supporting tooling ecosystem!#2017-03-3120:39richardhcool!#2017-04-0108:57lmergendoes anyone know whether thereā€™s some ā€œclojure.spec cookbookā€ ?#2017-04-0108:59lmergen(i.e. a collection of spec examples / community reviewed)#2017-04-0111:22ggaillardHello ! I have a set of keywords like (def ks #{::email ::password}) and IĀ would like to build a spec from it, like
(s/def ::cred 
  (s/keys :req-un (into [] ks)))
But s/keys and s/def are macros so (into [] ks) is not evaluatedā€¦ So I tried
(s/def ::cred 
  `(s/keys :req-un [
But the inner part seems to not be expanded and passed "as is" to s/def. I have a feeling telling me that I'm doing something wrong here. Is it sane ? Is there a way to expand the body before s/def is called ?
#2017-04-0111:27gfredericksYou have to write your own macro#2017-04-0111:28ggaillardA one that wrap s/def and s/keys ?#2017-04-0111:49ggaillardAh ! I get it ! Sorry for that šŸ˜• Thank you !#2017-04-0112:51mike_ananevHi! Sorry for dumb question, but how i can run my spec tests as unit tests using clojure.test? My goal is to run tests in some CI tool using lein test.#2017-04-0117:41gfredericks@mike1452 write a regular test that calls spec and asserts about the response#2017-04-0118:14mike_ananev@gfredericks is it right solution? https://middlesphere-1.blogspot.ru/2017/04/clojuretest-with-clojurespec.html#2017-04-0118:22gfredericksLooks plausible#2017-04-0121:15dviramonteshello all, having a little trouble getting up and running with clojure.spec in clojurescript have these dependencies in my project.clj
[  [org.clojure/clojure "1.8.0"]
                 [org.clojure/clojurescript "1.9.494"]
                 [org.clojure/core.async "0.2.391"]
                 ...
                 [org.clojure/test.check ā€œ0.9.0ā€] ]
and my namespace looks like
(ns foo.core
  (:require [cljs.spec :as s]
              [clojure.test.check :as tc]
              [clojure.test.check.generators :as gen]
              [clojure.test.check.properties :as prop :include-macros true]))
#2017-04-0121:16dviramonteswith all that the following doesnt work it throws an error
(s/def ::node
  (s/or
    :leaf int?
    :branch (s/coll-of ::node :min-count 1 :max-count 2)))

(js/console.log (s/exercise ::node))
#2017-04-0121:18dviramonteslet me know if you can spot the error, much appriciate#2017-04-0121:20dviramontestldr; the generator doesnt work which works in clj and does not work in cljs, not sure why or what im missing#2017-04-0211:14mike_ananevHi. if i have code (gen/sample (s/gen ::basic/string) 1000))), how i can reproduce the data generation? Where to put seed value?#2017-04-0212:07gfredericks@mike1452 what are you using this for? #2017-04-0212:45mike_ananev@gfredericks well, i want to have unit tests for my functions. I have bunch of specs for data, and i have specs for my fn's. I want to pass spec'ed data to spec'ed fns to write unit tests.#2017-04-0212:46mike_ananevi found some solution#2017-04-0212:46mike_ananevfor spec'ed data i can#2017-04-0212:47mike_ananevi know that (s/valid) here is unnecessary#2017-04-0212:48mike_ananevand for spec'ed fn example.core/average i can#2017-04-0212:51mike_ananevi wanted to have reproducible results in unit tests#2017-04-0220:32didibusQuestion: Why is cat always used for args in an fdef?#2017-04-0220:32didibusI've used tuple sometimes, and that works well, would it be incorrect to do so?#2017-04-0309:12didibusOther question: Is there an any? that means anything except nil?#2017-04-0309:29bronsasome?#2017-04-0310:37lsenjov(complement nil?)#2017-04-0315:46Alex Miller (Clojure team)@didibus regex specs are generally a good match for all of the various argument calling needs, but really any spec that can match an arg collection is fine#2017-04-0317:00didibus@bronsa Ah, should have thought of that one, thx.#2017-04-0317:02didibus@alexmiller I have to say you're an awesome community manager. I'm always impressed how active you are in all the forums and chats and conferences. I don't know how you find time to know everything about Clojure on top of that :+1: clj šŸ’Æ #2017-04-0411:14tjtoltonI said the same thing to him once. His response was "It's literally my job."#2017-04-0416:02tjtoltonAnyone know of a clean way to get the list of defined arguments for a function spec?#2017-04-0416:05tjtoltonbasically I want to pull arguments out of a request body and pass them into a function based on that function's spec#2017-04-0416:05tjtoltonI'd bet a dollar somebody's done this before me#2017-04-0416:33tjtoltonWow, this isn't as straightforward as I thought.#2017-04-0417:28minimal@tjtolton does this help? (s/form (:args (s/get-spec `my-fun)))#2017-04-0417:30minimalreturns something looking like (clojure.spec/cat :a clojure.core/string? :id clojure.core/number?)#2017-04-0417:31tjtoltonhmmm#2017-04-0417:32tjtoltonyou can just use keyword on a spec object like that?#2017-04-0417:38tjtoltonThat's actually pretty awesome#2017-04-0418:49Alex Miller (Clojure team)function specs implement ILookup for :args, :ret, and :fn#2017-04-0509:11qqqin spec, how do I say "obj" is a map, where the values are also maps ?#2017-04-0509:11qqqi.e. it's valid to write (get-in obj [k1 k2])#2017-04-0509:15bronsa
user=> (s/def ::foo (s/map-of any? map?))
:user/foo
user=> (s/valid? ::foo {:a {:b 1}})
true
#2017-04-0509:28qqqhere I could also do (s/map-of any? (s/map-of any? any?)) but the inner (s/map-of any? any?) is equiv to map? right?#2017-04-0509:28qqqthis is neat#2017-04-0509:38thheller@qqq (s/def ::map (s/map-of any? ::map) to make it recursive#2017-04-0509:41qqq@thheller: but that'd be wrong šŸ™‚#2017-04-0509:45shafeeqIs there a way to send clojure spec over the wire? Iā€™m looking to dynamically fetch specs from a clojure service via a HTTP request, into my clojurescript app that needs to use these specs for validation, etc.#2017-04-0509:46shafeeqUnfortunately, I canā€™t share these specs via cljc, because the server, and the client apps should be decoupled. So itā€™s a runtime dependency.#2017-04-0509:46thheller@shafeeq you can put your specs into a .cljc file so they can be shared, but serializing a spec will not work#2017-04-0509:46thhellerdoh šŸ˜‰#2017-04-0509:47qqq@thheller: I (perhaps incorrectly) thought specs are just data and can be passed around like data.#2017-04-0509:50ikitommithe s/form is the serialization format for specs. You need to recursively fetch all the related specs too.#2017-04-0509:50thhellerthe specs yes but the predicates they use are not#2017-04-0509:51thheller(s/def ::foo #(= % :foo)) cannot be serialized and transferred over the wire#2017-04-0509:51ikitommispiked that with a custom rmi-style ā€œspec-loaderā€ that is used to negotiate the specs over the wire.#2017-04-0509:59qqq@thheller: ah; besides functions, are there other things that need to be serailized? (and it seems like functions can be serialized if you have a giant hashmap of keyword to functions)#2017-04-0509:59qqq[I'm not saying this is a good idea; I'm now just curious whether it can be done.]#2017-04-0510:01viesti
user=> (let [x :foo ] (s/form (s/spec #(= % x))))
(clojure.core/fn [%] (clojure.core/= % x))
#2017-04-0510:02viestithere was discussion about dynamic scope somewhere in channel history but slack eats it šŸ™‚#2017-04-0510:05shafeeqWell in my case, I only use clojure.core predicates, since I also need the spec to be representable via swagger as well.#2017-04-0510:05shafeeqHereā€™s what I have so far: https://gist.github.com/shafeeq/c2ded8e71579a26e44c2191536e01c0d#2017-04-0510:07shafeeqIā€™m recursively walking down specs, and doing an s/form on them.#2017-04-0510:07shafeeqOn the clojurescript side, now I have to walk through this list of specs and make definitions.#2017-04-0510:08shafeeqWould something like this work? Am I missing some other perspectives or edge cases (apart from custom functions)?#2017-04-0510:09ikitomminice. did similar spike before the s/form bugs were fixed, now might work on most/specs already.#2017-04-0510:09ikitommiyou eval those on the client, right?#2017-04-0510:10shafeeqyeah, will need to do that. trying that now.#2017-04-0510:10ikitommithat opens up a security concernā€¦ running eval on something that has come over the wire.#2017-04-0510:11thhelleryou can't really eval these on the CLJS side#2017-04-0510:12ikitommiyou can, you shoudnā€™t? (https://github.com/metosin/spec-tools/blob/master/src/spec_tools/core.cljc#L57-L62)#2017-04-0510:12thhellerthat is reading not eval#2017-04-0510:13thhelleryou have a symbol clojure.core/string? but now what#2017-04-0510:13ikitommidoes the upcoming spec-of-specs help here?#2017-04-0510:13thhellerno#2017-04-0510:14thhelleras @qqq suggested you could make this work if turn everything into keywords and then define those keywords on both ends#2017-04-0510:14thhellerbut really ... use .cljc (just turn the specs into a library the client/server can share)#2017-04-0510:15ikitommioh, true @thheller i had test evalā€™ing the forms, but only in the clj-side.#2017-04-0510:17shafeeq@thheller from a higher level perspective: the specs on the server side will evolve with time, and we shouldnā€™t really have to compile, and build the client. The client should ideally be oblivious to changes in the specs. It should pull it down (via http), and use it.#2017-04-0510:18shafeeqthatā€™s the idea, at least.#2017-04-0510:18thhellerthen validate on the server and send the explain-data back, that is just data#2017-04-0510:20shafeeqwell, the specs are my API specification at the moment. Apart from getting a catalog of requests, Iā€™m building and coercing requests on the clojurescript side based on the spec.#2017-04-0510:21shafeeqpretty much swagger.json, but in spec#2017-04-0510:21shafeeqI already have this functionality implemented for swagger.json. Iā€™m moving to spec now.#2017-04-0510:32qqq@thheller: I'm impressed by how flexible spec is. I can't imagine even asking this question in Haskell. šŸ™‚#2017-04-0512:32odinodinIs it possible to get full spec instrumentation during ClojureScript development in your running app? (using Figwheel). Iā€™ve run cljs.spec.test/instrument which returns a list of fdefā€™ed functions, but none of the fdef functions trigger when the contract is broken.#2017-04-0512:32rovanionHi, I've been trying all day to figure out how to correctly spec a variable-length vector as described here: https://stackoverflow.com/questions/43230546/a-clojure-spec-that-matches-and-generates-an-ordered-vector-of-variable-length I can only seem to create a matching spec with spec/cat but it in turn will only generate lists šŸ˜•#2017-04-0512:37thomas@rovanion can't you use the seq regex's for that?#2017-04-0512:37thomasor is that not what you need?#2017-04-0512:38shafeeqIā€™m rewriting my prismatic-schema spec into clojure.spec. Thereā€™s a lot of it. Has anyone here heard of or used some utility to help with this?#2017-04-0512:38rovanionthomas: I'm able to match them using the seq regexes, but not generate data since they generate lists and I can't seem to be able to message them that I want vectors instead.#2017-04-0512:39thomas@rovanion just looked at our SO post... and if you do something like gen/vector ?#2017-04-0512:43rovanionthomas: I'd have to look up how that works first. The docs are very empty when it comes to explaining its use.#2017-04-0514:00rovanion@thomas Alex helped me to find a solution: https://stackoverflow.com/a/43233374/501017#2017-04-0514:34Alex Miller (Clojure team)itā€™s not uncommon to run into this when specā€™ing a macro (the need exists in a few places in the clojure.core specs) and as you can see in the example, itā€™s quite tedious to properly solve (even that example is missing what you really want for describe/form). Weā€™ve been discussing something like s/vcat for this but Rich has several competing ideas for how to implement it and Iā€™m not sure where he will end up going with it.#2017-04-0517:47ghadiAre they design alternatives or impl alternatives @alexmiller ?
#2017-04-0518:06slipsetFollowing http://dev.clojure.org/jira/browse/CLJ-2141, it seems to me that Clojure is missing a predicate namespaced? which returns true if a thing is, well, namespaced. #2017-04-0519:00ikitommiwhat kind of data does a (s/coll-of string? :into {}) expects?#2017-04-0519:00ikitommi
(def spec (s/coll-of string? :into {}))

(s/explain-data spec {"1" "2"})
; #:clojure.spec{:problems ({:path [], :pred string?, :val ["1" "2"], :via [], :in [0]})}

(s/explain-data spec ["1" "2"])
; nil

(s/conform spec ["1" "2"])
; CompilerException java.lang.ClassCastException
#2017-04-0519:31Alex Miller (Clojure team)@ikitommi if using :into {}, you should be using a spec for a map entry (like a 2 element tuple)#2017-04-0519:33Alex Miller (Clojure team)Like (s/coll-of (s/tuple string? string?) :into {})#2017-04-0519:34Alex Miller (Clojure team)@slipset I don't think that's generally useful enough#2017-04-0519:34Alex Miller (Clojure team)@ghadi design#2017-04-0519:40ikitommithanks alex#2017-04-0520:11Alex Miller (Clojure team)That's effectively how map-of is implemented#2017-04-0523:20takeoutweightIs there any (idiomatic) way to attach custom s/explain data for user specs? Use case is a predicate that supplies its own context information that I'd like to have preserved in the failure message.#2017-04-0523:38Alex Miller (Clojure team)no#2017-04-0523:38ghadi@alexmiller can you elaborate around design alternatives for vcat?#2017-04-0523:39Alex Miller (Clojure team)@ghadi no#2017-04-0523:39Alex Miller (Clojure team)@takeoutweight you can vote at http://dev.clojure.org/jira/browse/CLJ-2115#2017-04-0523:39Alex Miller (Clojure team)or weigh in there#2017-04-0523:39Alex Miller (Clojure team)@ghadi at this point I just donā€™t remember the details (and Rich did not fully elaborate on them anyways)#2017-04-0523:40Alex Miller (Clojure team)we did talk about it for a while. I mostly remember that he didnā€™t like any of my ideas. :)#2017-04-0523:40ghadisounds legit#2017-04-0710:08trisshey all. so are specā€™s searchable in anyway?#2017-04-0710:09trissletā€™s say I have (s/def ::a string?)#2017-04-0710:10trissand (s/def ::b (s/keys :req-un [::a])#2017-04-0710:11trissand I want to know if ::b contains any values that are strings?#2017-04-0710:11trissis it serious effort to find this out?#2017-04-0710:28ggaillard@triss I don't think there is a function for that purpose. But (s/form spec) give you the spec form, so in your case the list (s/def ::b (s/keys :req-un [::a]). You could walk it like a tree and recursively s/form any spec you find until you get a predicate that match you search criteria ā€¦ I do something similar to extract the set of keys used by nested s/merge specs. Maybe there is a cleaner way to do it thoughā€¦#2017-04-0710:37ggaillardDo you know if there is a way to make spec generate a minimalist sample for a particular spec ? What I mean by minimalist is "the first simplest value validating the predicate". If I have (s/nilable string?) I'd like to get nil, if I have string?, I'd like to get "", if I have (s/coll-of ::foo :kind vector?) I'd like to get []. I don't know (yet) how spec works internally but I would expect it could be able to find the shortest path to the first and simplest value validating the predicate. It would be really useful to generate kind of placeholder data waiting to be filled but still validating the spec. Especially when building forms with Re-Frame or similar frameworks.#2017-04-0711:56borkdudeHas anyone seen a similar error message like this lately? Attempting to call unbound fn: #ā€™clojure.spec/macroexpand-check We get it in our ClojureScript build. Weā€™re using clojure.future.spec alpha15.#2017-04-0712:17Alex Miller (Clojure team)@triss there is no easy way to figure that out right now#2017-04-0712:18Alex Miller (Clojure team)@ggaillard you can call gen/sample and take the first example. Samples "grow" in complexity as you use a generator so the first few are usually "simple"#2017-04-0714:18thomashi, I have a (defspec tester 100.... in my test file... but when I run either lein test or from the REPL it seems to run only once. Any idea what could cause this?#2017-04-0715:17fossifoohow do you fdef a fn without args?#2017-04-0715:24Alex Miller (Clojure team)(s/fdef foo :args (s/cat))#2017-04-0715:24Alex Miller (Clojure team)although it may not be worth defining a spec for a no-arg function#2017-04-0715:25Alex Miller (Clojure team)as you canā€™t check its args via instrumentation and generative testing is likely not too useful#2017-04-0715:27fossifoohmmm#2017-04-0715:27fossifootrue#2017-04-0715:28fossifoowell, this method will grow to support multiple parameters, so that's why i wondered in the first place#2017-04-0715:28fossifooi already found out that s/+ works for that#2017-04-0715:29fossifoojust wondered how to do no parameters#2017-04-0717:07Alex Miller (Clojure team)(s/fdef foo :args (s/* any?)) (with a suitable spec instead of any?) will support 0 or more args#2017-04-0721:24weiis there a way to make a s/keys take a var in the req list?
(let [k :ns/kw] (s/explain (s/keys :req [k]) {:ns/not-kw "hello"})) => Success!
#2017-04-0721:27Alex Miller (Clojure team)not easily - you can use macro or eval to do so#2017-04-0721:30weicurious why the above doesnā€™t work?#2017-04-0721:30Alex Miller (Clojure team)because s/keys is a macro and expects a literal vector of keywords in :req, not something that is evaluated#2017-04-0721:31Alex Miller (Clojure team)in the example above, everything thatā€™s not a keyword (`k`) is just being ignored#2017-04-0721:32Alex Miller (Clojure team)itā€™s being interpreted as (s/explain (s/keys :req []) {:ns/not-kw "hello"})#2017-04-0721:33weimakes sense, thanks#2017-04-0823:52twashingConsider this code
(s/def ::reqid number?)
(s/def ::subscription-element (s/keys :req [::reqid]))
(s/def ::subscriptions (s/coll-of ::subscription-element))
(defn next-reqid [scanner-subscriptions] :foo)

(s/fdef next-reqid
        :args ::subscriptions
        :ret number?)

(s/exercise-fn `next-reqid)
#2017-04-0823:53twashing1. Trying to run s/exercise-fn passing in N arguments, instead of a collection. How can I get s/exercise-fn to pass in a collection?#2017-04-0823:53twashing2. Also, how are people using the pinned defspec-test. Running (defspec-test test-name [sut/myfn]) always passes, even if thereā€™s bad code.#2017-04-0823:59twashingOk, Iā€™ve sorted 1.ā€¦ But 2. is still a thorn.#2017-04-0900:01twashingBasically, clojure.spec.test/check returns an empty sequence, even though :ret number? should break the function.#2017-04-0901:14twashingOk got it. The invocation needed to be (clojure.spec.test/check <backtick>myfn) , which I was messing up.#2017-04-0902:44xiongtxDoes it need to be backtick? Regular quote should do#2017-04-0903:40twashingHey Tx#2017-04-0903:41twashingFor whatever reason, the check macro needs A) (st/check `ranged-rand) B) (st/check ā€˜ranged-rand) will yield an empty result.#2017-04-0903:42twashingPresumably it has to do with how the symbol is expanded?#2017-04-0904:01seancorfieldIt needs to be a fully-qualified symbol name - which back tick does - or you could use my.ns/my-fn #2017-04-0907:03fossifoowhen i stest/check my database access functions, the real db implementation gets called. i don't want that. i currently try to with-redefs-fn those. is that the way to go?#2017-04-0907:09fossifooseems to work, i was just too dumb to get the signiture right. time for more specs i guess šŸ˜‰#2017-04-0909:40fossifoohmm. actually i'm still stuck. does sb have working examples of instrument with :stub & :gen or :spec?#2017-04-0916:51joshjones@fossifoo some sample code i have stashed for stubbing https://gist.github.com/joshjones/90f65bb11106053240baf5c6d5a4fc2b#2017-04-0917:03fossifoothanks, but only stubbing is not enough in my case#2017-04-0917:03fossifooi either need to use :gen or :spec too because i need to scope the fn spec differently#2017-04-0917:04fossifoobut in the mean time i also found an actual bug in the spec i wanted to check#2017-04-0917:04fossifooso it might've been that all along#2017-04-0917:13fossifoona, no luck. i always get Couldn't satisfy such-that predicate after 100 tries. {}#2017-04-0917:13fossifoobut the docs make it really hard to know what should be put into :gen or :spec#2017-04-0917:13fossifooand the code is not very straight forward either#2017-04-0917:34Alex Miller (Clojure team):gen or :spec where?#2017-04-1007:53srihariIs it possible to dynamically (at runtime) define a spec in clojurescript?#2017-04-1011:30rovanionThe macro clojure.spec/keys and clojure.spec/keys* expects a seq of keys as the value of its named arguments :req and :opt. I want to use the same list of keys for two different specs, one using keys and one using keys*. But the macros get passed the quoted symbol for the variable and not the value of it. Is it possible to unpack the value pointed to by a symbol at read time like the common lisp #. syntax?#2017-04-1011:30rovanionI've described the issue in longer form on SO: https://stackoverflow.com/questions/43321732/is-it-possible-to-resolve-a-symbol-to-its-value-before-passing-it-to-macro-in-cl#2017-04-1014:51frankI think that depending on how the macro was implemented, you may be able to#2017-04-1015:48donaldball
(eval `(s/def ::record (s/keys :req ~ks)))
is what Iā€™ve been using when generating specs from data
#2017-04-1020:51tjtoltonI'm sorry, I'm having trouble finding this in the docs (which leads me to believe core.spec has some kind of strong opinion on this) -- is there a canonical way of saying "a predicate that matches anything". essentially a predicate (fn [thing] true)#2017-04-1020:51tjtoltonlooking for, essentially, (s/.)#2017-04-1020:51donaldballany?#2017-04-1020:52tjtoltonis that a thing?#2017-04-1020:52tjtoltonare you sure you're not thinking of prismatic?#2017-04-1020:52schmeeit is new in clojure 1.9#2017-04-1020:53schmeehttp://clojuredocs.org/clojure.core/any_q#2017-04-1020:53tjtoltonohh, you mean its like a clojure.core/any? ?#2017-04-1020:53tjtoltonhuh#2017-04-1020:53tjtoltonhow about that#2017-04-1022:07stathissiderisare there any efforts for a spec-based condp-like macro?#2017-04-1022:07stathissiderissomething that would be a more general core.match#2017-04-1100:35madstapWoops, that is actually just
(defmacro specondp
  {:style/indent 1}
  [expr & clauses]
  `(condp s/valid? ~expr 
Probably doesn't need to be a macro...
#2017-04-1101:59athosThis might be useful. https://github.com/lambdaisland/uniontypes#2017-04-1102:51Oliver GeorgeI do look forward to instrumentation on clojure.core#2017-04-1102:52Oliver GeorgePerhaps I'm wishing but imagine it would have saved me this headache. (cljs.reader/read-string (str (keyword "3asdf")))#2017-04-1102:52Oliver GeorgeThat throws a mysterious error since we're trying to read a keyword which starts with a number.#2017-04-1102:53Oliver George(my use case was using js->clj keywordize-keys to turn json into clojure data structures and later serialising to js/localStorage)#2017-04-1102:53Oliver GeorgeA spec on the keyword function would have picked up the illegal input (I believe)#2017-04-1103:23hiredmanNah#2017-04-1103:24hiredmanThe core team has repeatedly stated that those keywords are not illegal, the keyword function is just able to create a wider range of keywords than the reader#2017-04-1104:09Oliver GeorgeIs that right. Huh. It was a bugger to find my unexpected keyword. I guess I'd be happy for the error thrown in read-string to improve.#2017-04-1104:11stathissideris@madstap yes, that would be the basic version, but I was thinking about something that actually binds local vars in the same way that core.match does #2017-04-1104:17stathissiderisWith var names derived from the keyword names in s/cat #2017-04-1104:18cfleming@olivergeorge Itā€™s worse than that - using the function you can create keywords with spaces in them, or anything else your heart desires.#2017-04-1104:19cflemingFeel like a keyword with a newline in it? No problem.#2017-04-1104:24Oliver GeorgeYeah that's a nasty surprise. Seems like a spec instrument on clojure.core/keywords would help to protect me nicely.#2017-04-1104:24Oliver GeorgeAs it is I'll log a ticket for improving read-keyword so that it's blindly obvious when the specific issue I came up with occurs.#2017-04-1104:26Oliver George@alexmiller I gather adding spec's to the core ns's is complicated by aot compilation (or similar). Does that mean user defined optional core ns specs will be difficult/troublesome?#2017-04-1104:26Oliver George@cfleming have you found any good opportunities for Cursive to integrate/leverage specs?#2017-04-1104:27Oliver George(I wondered if it could be used to improve autocomplete suggestions)#2017-04-1104:28cflemingThere are plenty, but I havenā€™t had time to investigate it yet. It could definitely be used to improve autocomplete, yes, assuming the spec machinery provides enough hooks to get that info out.#2017-04-1104:29Oliver George@cfleming good stuff#2017-04-1107:40ikitommiHmm.. s/merge seems to report problems of qualified keys twice:
(require ā€™[clojure.spec :as s])

(s/def ::a int?)
(s/def ::b string?)

(s/explain-data
  (s/merge
    (s/keys :req-un [::a])
    (s/keys :req-un [::b]))
  {:a 1 :b 2})
; #:clojure.spec{:problems ({:path [:b], :pred string?, :val 1, :via [:user/b], :in [:b]})}

(s/explain-data
  (s/merge
    (s/keys :req [::a])
    (s/keys :req [::b]))
  {::a 1 ::b 1})
;#:clojure.spec{:problems ({:path [:user/b], :pred string?, :val 1, :via [:user/b], :in [:user/b]}
;                          {:path [:user/b], :pred string?, :val 1, :via [:user/b], :in [:user/b]})}
#2017-04-1108:05akielAs I read the doc on s/& this (s/conform (s/& (s/+ char?) #(apply str %)) [\s \p \e \c]) should return "spec" but returns [\s \p \e \c]. Am I wrong?#2017-04-1108:41akiel(s/conform (s/& (s/+ char?) (s/conformer #(apply str %))) [\s \p \e \c]) returns "spec". I debugged the code and s/& calls dt with cpred? nil. So maybe it should use cpred? true here?#2017-04-1109:18wottisgreetings#2017-04-1109:21akielJuste require shepherd.spec without any alias and use :principle/type#2017-04-1109:25wottis@akiel thank you very much!#2017-04-1112:13Alex Miller (Clojure team)@akiel s/& just validates the predicate then returns the conformed value of the spec so that is the expected behavior. You could switch it to s/and though and use s/conformer for the last bit #2017-04-1112:16Alex Miller (Clojure team)@ikitommi it's reporting the same bug from each branch of the merge. Not sure if those should be deduped or not#2017-04-1112:18ikitommiI think it should, as it works with unqualified keys.#2017-04-1112:24Alex Miller (Clojure team)@olivergeorge you can add user-defined specs to core. Works fine. We have no plans to place a spec on keyword any narrower than string? though. What would be a useful enhancement though is to have either a predicate or "safe" version of keyword that accepted only print-read-safe keywords#2017-04-1112:25Alex Miller (Clojure team)keyword-strict or something#2017-04-1112:26Alex Miller (Clojure team)And then you could use that in json parsing etc#2017-04-1112:39akiel@alexmiller The doc of s/& says: takes a regex op re, and predicates. Returns a regex-op that consumes input as per re but subjects the resulting value to the conjunction of the predicates, and any conforming they might perform. especially: but subjects the resulting value and any conforming they might perform I read that as if the cvals of the preds should be returned. The same as s/and is doing, as you say.#2017-04-1113:00Alex Miller (Clojure team)Hmm, not sure. I would need to ask Rich.#2017-04-1113:14akiel@alexmiller Ok. Thanks.#2017-04-1115:50ikitommi@alexmiller wrote an jira issue of that double problems, also another of s/keys* is returning something that is not a clojure.spec.Spec#2017-04-1115:51bronsa@ikitommi no regex op return specs tho#2017-04-1115:52bronsasame for s/cat, s/+ etc#2017-04-1116:43ikitommi@bronsa oh, that bad.#2017-04-1117:07ikitommiwell, now that I look at the clojure.spec test suite, no wonder there are (form) bugs. @alexmiller, is there a way to contribute more tests? testing all forms & return values as Specs would easily reveal all things that donā€™t work right now.#2017-04-1118:03ikitommione more form bug:
(require '[clojure.spec :as s])

(s/def ::a int?)
(s/form (s/& ::a))
; (clojure.spec/& #object[clojure.spec$spec_impl$reify__13797 0x47b888f9 "
#2017-04-1118:13bronsa@ikitommi i think that's by design (regex ops not being specs)#2017-04-1119:40Alex Miller (Clojure team)correct, thatā€™s as intended (those respond to regex?#2017-04-1119:45Alex Miller (Clojure team)regarding & forms, that problem space is related to the issues with the keys* form (which is implemented with &). would be fine to log though.#2017-04-1121:02Oliver GeorgeThanks @alexmiller I'm happy with those options. #2017-04-1121:06Alex Miller (Clojure team)@olivergeorge Iā€™ve talked about these before with people but I donā€™t think it was ever actually logged#2017-04-1122:03Oliver GeorgeI wonder where that documentation should live. #2017-04-1122:04Oliver GeorgeAdding a keyfn option to js->clj would make it easier to step safely around the sharp edges#2017-04-1122:42Oliver GeorgeThere is an open ticket to improve cljs.reader/read-keyword to produce a better error here: http://dev.clojure.org/jira/browse/CLJS-1907#2017-04-1122:43Oliver GeorgePlease upvote#2017-04-1123:31waffletowerIf this is a reasonable place to ask about generatorsā€¦ I am trying to understand clojure.test.check.generators/no-shrink. I have tried to eliminate shrinking in this generator example:
(def large-flat-natural
  (gen/no-shrink (gen/fmap abs gen/large-integer)))
#2017-04-1123:31waffletowerBut it gives shrunken results:
(repeatedly 16 #(println (gen/sample large-flat-natural 24)))

(1 0 1 2 1 15 1 0 5 5 7 76 2 109 3 234 99 41987 18 9 17237 45920 1019 14841)
(0 1 1 0 2 1 5 5 46 3 177 0 6 169 23 25 25 5 367 1420 311458 36483 42628 1)
(0 0 0 0 4 2 1 0 71 1 5 35 22 208 26 11 3 8391 32493 232365 2 2 2750 5576)
(1 1 1 0 0 1 2 2 3 6 6 19 1 55 3 2 33 20263 1877 1 362177 2 6258 15641)
(1 0 1 2 2 2 6 58 0 26 66 39 1860 1 347 4 9073 6 19852 31997 25 56137 117428 23)
(0 1 1 2 0 1 1 0 31 13 1 116 3 105 1 26 8959 710 38835 4 1701 0 91091 27453)
(0 1 0 2 1 8 7 15 2 8 4 9 35 1254 0 22 20463 1 170 2 15412 1 0 23163)
(1 1 0 1 1 1 14 1 2 1 31 141 1 3 409 15 19 7946 49819 55 3831 44642 1 510806)
(0 1 1 1 1 1 8 1 1 21 3 2 32 0 504 1 4 0 1 2737 24695 2701 242 470111)
(1 1 1 0 1 1 3 5 41 37 7 5 250 515 9 0 2 13 566 44053 174 9371 401689 66970)
(1 1 1 0 2 5 4 1 36 1 21 3 106 4 813 542 116 197 19 949 187929 24 6 52)
(0 0 1 1 2 6 3 1 2 0 15 1010 50 30 13 51 15183 0 13328 1885 2 3 2787 1425)
(1 0 2 1 1 2 1 22 0 25 81 0 11 0 113 4975 177 2 15989 12 90275 186 7 2619)
(1 0 0 3 1 3 3 2 7 2 21 957 127 1055 1364 6 15462 2 40 39 16 298275 2985 762)
(0 0 1 1 7 1 2 1 1 200 256 1 1 1412 3913 0 61 1 355 0 4 25 2 292057)
(0 1 0 1 2 5 2 3 0 44 13 2 3 423 2744 381 689 233 260 1324 49733 280 14384 626)
#2017-04-1200:08gfredericks@waffletower those aren't shrunk, they're small-sized#2017-04-1200:09gfredericksBackground: https://github.com/clojure/test.check/blob/master/doc/growth-and-shrinking.md#2017-04-1200:11waffletowerWill check that out. Trying to get a uniform distribution. Am tempted to use the private (make-gen) and the random namespace as you do with the generator uuid.#2017-04-1200:11waffletowerIs there a reason that make-gen is private?#2017-04-1200:11gfredericksUniform among longs?#2017-04-1200:11waffletoweryes#2017-04-1200:11gfredericksIt's private because it's usually not necessary#2017-04-1200:12gfredericksBut we don't have a uniform long generator at the moment#2017-04-1200:12gfredericksThough choose is close#2017-04-1200:12gfredericksI'm considering adding one#2017-04-1200:12waffletowerI know I am young to test.check, and you have been really helpful, but I keep bumping into boundaries.#2017-04-1200:13gfredericksAre you using it for PBT or some other speccy thing? #2017-04-1200:13waffletowerIn this case I am making a generator for dates#2017-04-1200:13gfredericksWhy do you want a uniform long? #2017-04-1200:14gfredericks(I.e., why does that help for dates?)#2017-04-1200:14waffletowerusing clj-time and it is convenient to generate a random date from a long rather than its constiuent parts. Want them to be evenly distributed.#2017-04-1200:15waffletower
(def date-8601
  (gen/fmap
   (fn [x] (f/unparse (f/formatters :date-time) (c/from-long x)))
   gen/large-natural))
#2017-04-1200:15gfredericksYou'll get weird dates with >4 digit years that way, won't you? #2017-04-1200:16waffletowerI could use ints instead, but would be nice if they were uniform and not usually centered around 1970#2017-04-1200:17gfredericksI guess I'm trying to figure out why uniformity is important for this, especially since the domain will inevitably end up rather arbitrary#2017-04-1200:18gfredericksE.g., dates in the years -1000000 to 1000000#2017-04-1200:18gfredericksI think date & time generators end up being weird regardless :/#2017-04-1200:19waffletowerIts a question of control. For some data I feel a desire to turnoff the growth and shrinking mechanisms and control the output with localized RNG#2017-04-1200:20gfredericksgen/choose is the classic tool for turning off growth#2017-04-1200:21waffletowergreat lead, I will check it out.#2017-04-1200:21gfredericksIt just has edge cases outside of the 32 bit range#2017-04-1200:21gfredericksI was thinking of making a variant called uniform-integer that world be more robust#2017-04-1200:23waffletoweryou are supremely helpful. thanks again#2017-04-1200:24waffletowerI see gen/scale as wellā€¦#2017-04-1200:36gfredericksYep. gen/sized is a more general (ha!) version of that#2017-04-1200:36gfredericksA lot of generators that do nontrivial growth things do it by combining choose and sized#2017-04-1200:37gfredericksI guess I mean that those two things give you the power to completely customize growth#2017-04-1200:49fossifoodoes sb have working examples of instrument with :stub & :gen or :spec?#2017-04-1200:55fossifoohere's a gist of what i try to get to run: https://gist.github.com/FossiFoo/f98f2eaf920c3481d9fdf9bdeb41c420#2017-04-1200:57fossifoothe ":replace" works, but i thought i could just change the result of (db/select) so that it fits the spec when testing the function calling it#2017-04-1200:58fossifooi know that my datomic/adi db/select will return the correct types, but i'm not sure how to put that knowledge into spec#2017-04-1201:27fossifooi think an example of ":gen a map from spec names to generator overrides" in clojure.spec/instrument would help#2017-04-1201:27fossifooi got a version to run that uses ":spec", so that's a little nicer#2017-04-1201:35fossifoofdef docs says "Once registered, function specs are included in doc, checked by instrument, tested by the runner clojure.spec.test/run-tests," is there such a thing as this runner?#2017-04-1202:42wottisdoes someone know a larger project (present on github) that uses clojure.spec? I want to see how spec is really being used. All of the small, and simple repl example can't answer the questions i have.#2017-04-1203:47seancorfieldAs noted in #clojure, it's kinda hard since most real world Clojure projects have not yet moved to 1.9.0 Alpha builds.#2017-04-1208:36curlyfry@wottis I like this one (in ClojureScript): https://github.com/jrheard/voke/tree/master/src/voke It's a WIP game that uses spec for its entity system. Not super large, but makes good use of fdefs and has a few generative tests! šŸ™‚ Maybe @jrheard can tell us a bit more about it?#2017-04-1208:46rovanionwottis: Here is a list of all the projects on clojars I've found using spec: systems-toolbox, spex, oops, alia-spec, clj-edocu-users, mistakes-were-made, utangled-client, clj-edocu-help, spec, net, clj-element-type, gadjett, datomic-spec, tappit, tick, deploy-static, tick, spectrum, sails-forth, ring-spec, doh, tongue, diglett, rp-query-clj, pedestal.vase, clj-jumanpp, om-html, crucible, ctim, crucible, wheel, merlion, turbovote-admin-specs, odin, pc-message, active-status, rp-json-clj, curd, rp-util-clj, swagger-spec, invariant, functional-vaadin, untangled-client, flanders, uniontypes, dspec, schpeck, macros, replique, specific#2017-04-1208:47wottis@curlyfry @rovanion Thank you very much!#2017-04-1208:47mpenetspandex also has specs#2017-04-1208:48rovanionRight, libraries that are so "popular" that they're on http://clojure-toolbox.com are: alia, cljs-oops, closp, core.typed, graphql-clj, java.jdbc, onyx, sablono, spandex, tupelo.#2017-04-1208:48rovanionForgot that I had split them into two.#2017-04-1208:50wottishrhr that should be enough for now#2017-04-1209:26rovanionSo perhaps start with some library from the latter list.#2017-04-1209:47rovanionIs it possible to express something like (spec/keys :req-un [(spec/or ::a ::b)]) so that either :a or :b is present in the resulting map, but never the two at the same time?#2017-04-1210:00dergutemoritz@rovanion Not with s/keys alone#2017-04-1210:01dergutemoritzYou gotta s/and it with a pred of that effect#2017-04-1210:02fossifoothis seems kinda noisy#2017-04-1210:03fossifooi also wondered about that and i have some cases where a thing is either a or b or c or d (or ... some more)#2017-04-1210:03fossifoomight be bad interface design, but it's exploding fast#2017-04-1210:25dergutemoritz@fossifoo "is either a or b or c or d" sounds like s/multi-spec could be worth a look for you#2017-04-1210:28fossifooah, yeah, i glanced over that. might work out in my case#2017-04-1214:10Alex Miller (Clojure team)@fossifoo regarding your question above about the docstring for fdef - that has already been fixed - should be clojure.spec.test/check. Where are you seeing the version pointing to run-test?#2017-04-1214:12Alex Miller (Clojure team)oh, probably the online clojure doc - there is actually a known problem preventing those docs from being regenerated and they are a few alphas behind. There arenā€™t many changes to the API since then but thatā€™s one.#2017-04-1214:33fossifoookay#2017-04-1214:33fossifoodo you happen to have an example or clarification for instrument {:gen} ?#2017-04-1214:35fossifoo"a map from spec names to generator overrides" i read that as {:some/spec (gen/string)} or such , but that didn't seem to work for me#2017-04-1214:36fossifooi always got the "could'nt generate in 100 tries" error with that. don't have the actual error on hand#2017-04-1214:36gfredericks(fn [] gen/string)#2017-04-1214:37gfredericksI think#2017-04-1214:37fossifooah, worth a try#2017-04-1214:37fossifooit's not worded very clearly#2017-04-1214:38fossifooand the source is not really readable either ;D#2017-04-1222:20micahasmithany advice on spec-ing async functions?#2017-04-1222:35micahasmithalso, is this sort of redef-ing inside of a spec bad form?#2017-04-1222:36micahasmith
(s/fdef ::shopify-api!
        :args (s/cat :opts ::shopify-api-opts)
        :ret nil?
        )

(with-redefs [org.httpkit.client/get (fn [a b c] nil)]
  (s/valid? ::shopify-api! shopify-api!))
#2017-04-1300:37tbaldridgeIMO, with-redefs is always bad form#2017-04-1300:43micahasmithfeel like the more i work through this clojure.spec guide, iā€™m realizing this is more about typing + generative testing than testing#2017-04-1300:44micahasmithwhich is strange given then (s/fdef :fn #()) functionality#2017-04-1300:53micahasmithor does my mocking have to turn into custom generators#2017-04-1309:40rovanionHas any of you ever tried to s/merge two s/keys*? I can only seem to get it to work for validation and not for generation.#2017-04-1309:41rovanionWrote a complete example on SO: https://stackoverflow.com/questions/43388710/generator-for-union-of-two-sets-of-keys-for-named-arguments-with-clojure-spec#2017-04-1314:23jfntnGetting some odd validation errors on the :ret with orchestra.spec.test so Iā€™m not sure if the spec is wrong?#2017-04-1314:33Alex Miller (Clojure team)comparator fns donā€™t have to return only -1 / 0 / 1, although a particular one might#2017-04-1314:35Alex Miller (Clojure team)they can return any int - the sign is the important thing#2017-04-1314:36Alex Miller (Clojure team)comparators also typically have constraints around comparing two of the same kind of things#2017-04-1314:36jfntnoh I didnā€™t know about the sign#2017-04-1314:37jfntnI think the issue is I forget fspec is generating data from the args spec and calling the comparator with it, but I have some tricky invariants between a and b that are not encoded inside the args spec#2017-04-1314:38Alex Miller (Clojure team)yep#2017-04-1314:38jfntnthat makes sense, thank alexmiller#2017-04-1314:49jfntnTrying to resist the urge to shave that yak but Iā€™m curious what would be the right approach here: a and b only have a partial order, the state thatā€™s being closed over by the comparator is a lookup table thatā€™s used to get a total order. So if I wanted the whole fdef to work, Iā€™d have to constrain the fspec :args generator with the value generated for the fdef :args constructor šŸ¤”#2017-04-1314:51Alex Miller (Clojure team)yeah, thatā€™s not likely going to be possible to do easily#2017-04-1314:51Alex Miller (Clojure team)but what you can do is to a) build a better args spec that uses values from your lookup and b) write a fn spec that verifies something about the comparator#2017-04-1314:52Alex Miller (Clojure team)for a, I mean the generator#2017-04-1314:54Alex Miller (Clojure team)and then you also need to ask yourself whether the tests you get out of it are worth the effort youā€™re putting into the spec. itā€™s ok if the answer is no. :)#2017-04-1314:55Alex Miller (Clojure team)higher order stuff is particularly hard to gen test#2017-04-1314:59jfntnI think the answer is no in this case, but I think I understand how this would work: the fspec for the comparator would be checked in isolation so its args need to have the invariants from the lookup table but we donā€™t need to somehow ensure itā€™s lexically the same table. So the gen for the lookup table can build the relationships, and the gen for comparable would bind to it and pluck a couple of values out of the table. Is that somehow correct?#2017-04-1316:18Alex Miller (Clojure team)yeah, youā€™d have to close over some of that stuff I guess#2017-04-1414:38fossifoo@gfredericks thanks for the example, instrument {:gen (f [] somegen)} works indeed#2017-04-1414:58borkdudeBefore I post this to JIRA, could someone take a look if this is bug? https://gist.github.com/borkdude/95301951e81e3022674279d463783586#2017-04-1415:28moxaj@borkdude reloaded.repl is not a valid symbol#2017-04-1415:29borkdude@moxaj Why not - and why is this the only symbol in my dependencies causing problems?#2017-04-1415:30borkdude(symbol? ā€™reloaded.repl) ;;=> true#2017-04-1415:32borkdudeAccording to https://clojure.org/reference/reader#_symbols it is valid I think?#2017-04-1415:34moxajyeah, I was wrong about that šŸ˜® seems like a bug to me#2017-04-1415:35moxajif you move the quote inside the set it works#2017-04-1415:37borkdudeAh#2017-04-1415:38borkdudeThis also works:
user=> (def deps ā€™#{[org.clojure/core.async ā€œ0.3.442ā€]
    [prismatic/schema ā€œ1.1.4"]
    [com.stuartsierra/component ā€œ0.3.2ā€]
    [reloaded.repl ā€œ0.2.3"]})
#ā€™user/deps
user=> (s/def ::dependency deps)
#2017-04-1415:41borkdudeor the equivalent in a let, but not inside ->>. Probably because def and let are compiler level things and ->> is a macro#2017-04-1415:42borkdudeI can live with the let solution, but if itā€™s a bug, Iā€™m happy to post it to JIRA#2017-04-1417:16Alex Miller (Clojure team)itā€™s not a bug#2017-04-1417:17Alex Miller (Clojure team)the quoted form is expanded to (quote #{ ... })#2017-04-1417:17Alex Miller (Clojure team)s/def is a macro - it doesnā€™t expect to find a (quote ...) form#2017-04-1417:17Alex Miller (Clojure team)@borkdude ^^#2017-04-1417:19moxaj@alexmiller why is it trying to resolve the symbol though?#2017-04-1417:20Alex Miller (Clojure team)where do you see that?#2017-04-1417:21moxajI think this is the line throwing the exception: https://github.com/clojure/clojure/blob/master/src/clj/clojure/spec.clj#L308#2017-04-1417:21Alex Miller (Clojure team)oh, itā€™s seeing a bare symbol with a ā€œ.ā€ in it, which is interpreted as a class instance#2017-04-1417:21Alex Miller (Clojure team)so itā€™s trying to load the class#2017-04-1417:22Alex Miller (Clojure team)reloaded.repl is being interpreted like java.lang.String#2017-04-1417:26Alex Miller (Clojure team)ā€œitā€ here being the compiler, I donā€™t think itā€™s ever even getting to spec#2017-04-1417:34moxaj@alexmiller to me it seems like the resolve call at the line I linked is responsible for the exception#2017-04-1417:35moxajor what do you mean by "not getting to spec"?#2017-04-1417:35borkdudeThen whatā€™s the difference between calling
(-> '#{[org.clojure/core.async "0.3.442"]
    [prismatic/schema "1.1.4"]
    [com.stuartsierra/component "0.3.2"]
    [reloaded.repl "0.2.3"]})
which works and
(s/def ::dependencies '#{[org.clojure/core.async "0.3.442"]
    [prismatic/schema "1.1.4"]
    [com.stuartsierra/component "0.3.2"]
    [reloaded.repl "0.2.3"]})
which doesnā€™t ?
#2017-04-1417:35borkdudeboth are macros#2017-04-1417:36Alex Miller (Clojure team)well the macros do do different things :)#2017-04-1417:36Alex Miller (Clojure team)@moxaj you could be right - itā€™s basically the point at which the form passed to s/def is evaluated inside s/def#2017-04-1417:37Alex Miller (Clojure team)s/def uses the form in both an evaluated and unevaluated context#2017-04-1417:37moxajhttps://gist.github.com/moxaj/d21f8f1db3e4cf32dcb12485a248070e#2017-04-1417:37borkdudebut I quoted the form which should remain the same when evaluated?#2017-04-1417:37moxajreplaced resolve with my-resolve to confirm#2017-04-1417:39Alex Miller (Clojure team)during macro-expansion a quoted form is literally (quote ...) - it has not yet been evaluated#2017-04-1417:39borkdudehmm ok#2017-04-1417:40souenzzo+1 on clojure.spec use data > functions > macros.... I also have some problems related to this macros...#2017-04-1417:40Alex Miller (Clojure team)the quote expansion happens during reading#2017-04-1417:41Alex Miller (Clojure team)@souenzzo several parts of spec donā€™t work without macros#2017-04-1417:41Alex Miller (Clojure team)like explain#2017-04-1417:44Alex Miller (Clojure team)also, keep in mind that spec forms are data and can be conformed into a map-y form with spec specs (and unformed back)#2017-04-1417:44Alex Miller (Clojure team)they can also be transformed in the middle of that or even constructed straight from the mappy form via unform#2017-04-1417:45mpenetNot yet right?#2017-04-1417:45borkdudeok, so in the case of passing a set, itā€™s best to evaluate it before passing, instead of quoting it#2017-04-1417:45Alex Miller (Clojure team)hard to say ā€œbestā€ - just that there is an interplay here between reading, macroexpansion, and the macro that is subtle#2017-04-1417:46moxaj@borkdude in your case, the safest I believe would be to def it, and use the var#2017-04-1417:46Alex Miller (Clojure team)@mpenet patch is on CLJ-2112. itā€™s not done yet but idea is there.#2017-04-1417:46borkdude@moxaj yes, let also works (wrote that earlier already)#2017-04-1417:46mpenetYes#2017-04-1417:46Alex Miller (Clojure team)
(defn make-or [keys preds]
  (let [or-data {:s 'clojure.spec/or
                 :args (map #(hash-map :tag %1 :pred [:pred %2]) keys preds)}]
    (s/unform ::ss/or-form or-data)))
(make-or [:a :b] [`int? `keyword?])
;;=> (clojure.spec/or :a clojure.core/int? :b clojure.core/keyword?)
#2017-04-1417:46Alex Miller (Clojure team)was just playing with it this morning actually - thereā€™s a function that makes a spec from data#2017-04-1417:47mpenetSo "are" -> "will be" #2017-04-1417:47Alex Miller (Clojure team)well Iā€™d say specs ā€œareā€ data now, just that you need the specs for that data too#2017-04-1417:48Alex Miller (Clojure team)the ::ss/or-form in the example above is the only missing piece#2017-04-1423:30standIs there some way of specing a situation where a map has mutually exclusive optional keys? That is neither ::a or ::b is required but if ::a appears, ::b may not and vice versa.#2017-04-1423:43Alex Miller (Clojure team)You can and a predicate#2017-04-1423:43Alex Miller (Clojure team)Or possibly multi-spec is a better match#2017-04-1423:58standCan you explain what you mean by "and a predicate?"#2017-04-1501:10Alex Miller (Clojure team)(s/and (s/keys ...) #(xor ... %))#2017-04-1501:11Alex Miller (Clojure team)Not that xor is a thing, but you get the idea#2017-04-1501:59seancorfieldA concrete example from our codebase @stand
(s/def :criteria/general (s/and (s/keys :req-un [(or :criteria-item/value :criteria-item/values)
                                                 :criteria-item/weight])
                                #(not (and (:value %) (:values %)))))
#2017-04-1502:00seancorfieldA general criteria map must have :weight but can only have one of :value or :values.#2017-04-1502:02standI don't believe that would work in my case. I need a situation where neither :value or :values need appear but if one appears the other cannot. I don't think you can do an or in a keys :opt, can you?#2017-04-1502:06seancorfieldAh, no, you cannot.#2017-04-1502:07seancorfieldBut they would both just be :opt keys and you could still s/and the check that they are not both present#2017-04-1502:07seancorfieldlike this
(s/def :criteria/general (s/and (s/keys :opt-un [:criteria-item/value :criteria-item/values
                                                 :criteria-item/weight])
                                #(not (and (:value %) (:values %)))))
#2017-04-1502:08seancorfieldThat would work for the neither case, the either case, and prevent the both case I think?#2017-04-1613:35ikitommiWill there be a utility to turn forms back into specs (without eval)?#2017-04-1615:39mpenetI guess spec form conforming + unform would allow this
#2017-04-1702:10xiongtxIs there a plan to make an fn spec part of the fn itself, similar to :pre and :post? As a piece of metadata, maybe?#2017-04-1702:11xiongtxIt makes sense to keep contraints on an fn w/ the definition#2017-04-1703:40Alex Miller (Clojure team)No#2017-04-1714:11stathissiderisJust released spec-provider, a library that will infer specs from sample data: https://github.com/stathissideris/spec-provider#2017-04-1716:08mobileinkIs there any way to attach metadata to a spec? I want to encode json specs like the following in spec:
"n": {"type": "string",
      "readOnly": true,
      "description": "Friendly name of the resource"},
#2017-04-1716:13mobileinkthis is best iā€™ve come up with, at least it documents it:
(s/def ::n (s/and string?
                  (comment {:read-only true
                            :doc "Friendly name of the resource"})))
#2017-04-1716:16mobileinkspec works great for e.g. https://github.com/OpenInterConnect/IoTDataModels/blob/master/oic.core.json except for meta props like readOnly.#2017-04-1716:47ikitommi@mobileink there are at least two libs on top of spec which allow meta-data on specs (and produce JSON Schemas): https://github.com/metosin/spec-tools & https://github.com/uswitch/speculate. Iā€™m working on the first one, in which the meta-data is written to spec forms, so the are really persisted. Last PR before releasing, this week hopefully.#2017-04-1716:48mobileinkthank you. i did take a brief look at spec-tools and saw some spec->json schema stuff; does it go the other way ā€™round?#2017-04-1716:49mobileinkanyway, i take it that means no way to do it in plain spec?#2017-04-1716:51ikitommino, just one-way currently. @mpenet had plans of doing the other way around, would open up a whole new level of interop with js.#2017-04-1716:52mobileinkso in spec-tools, for readOnly I would do :json-schema/readOnly true?#2017-04-1716:54ikitommiyes#2017-04-1716:55ikitommiIn plain spec, I guess you could add meta-data to registered specs: read the registry value, add meta, re-register.#2017-04-1716:57mobileinkyeah, was thinking about that but I think Iā€™ll take the path of least resistance and give spec-tools a try. thanks.#2017-04-1717:24waffletowerI am trying to make generative testing of a recursive spec more feasible. Consider the following simple example:
(s/def ::throne string?)
(s/def ::monarch string?)
(s/def ::gold int?)

(def kingdom (s/spec
              (s/keys :req-un [::throne
                               ::monarch
                               ::gold]
                      :opt-un [::kingdoms])))

(s/def ::kingdoms (s/coll-of kingdom))
#2017-04-1717:26waffletowerIs there an external way to limit the size of the collection produced by s/coll-of, or is a custom generator necessary?#2017-04-1717:55nfisher@waffletower :min-count :max-count allows you to control the collection size.#2017-04-1717:56nfisher(s/def ::kingdoms (s/coll-of ::kingdom :min-count 1 :max-count 10))#2017-04-1717:58nfisherIs that what youā€™re looking for?#2017-04-1718:06waffletowerThanks! That limits the size supported in the spec though, correct? I am trying to leave the maximum unbounded but place limits to facilitate generation.#2017-04-1718:08waffletowerI have tried this route:
(s/def ::kingdoms (s/with-gen
                    (s/coll-of kingdom)
                    (gen/resize 1 (s/gen (s/coll-of kingdom)))))
this often causes an exception which I am currently tracking down:
ClassCastException clojure.test.check.generators.Generator cannot be cast to clojure.lang.IFn  clojure.spec/every-impl/reify--13992 (spec.clj:1285)
#2017-04-1718:08Alex Miller (Clojure team)Use :max-gen for that#2017-04-1718:09Alex Miller (Clojure team)Default is 20 but I find 3 to be better usually#2017-04-1718:12waffletowerThanks Alex, need to slow down to see text like: same options as 'every' šŸ™‚#2017-04-1718:42ikitommimap-of keys donā€™t get conformed?
(require '[clojure.spec :as s])

(s/conform
  (s/map-of keyword? keyword?)
  {"key" :value})
; :clojure.spec/invalid

(s/conform
  (s/map-of keyword? keyword?)
  {:key :value})
; {:key :value}

(s/conform
  (s/map-of (s/conformer keyword) (s/conformer keyword))
  {"key" "value"})
; {"key" :value}
#2017-04-1718:59Alex Miller (Clojure team)no, but you can pass :conform-keys option for that if you need it#2017-04-1805:09ikitommithanks!#2017-04-1805:15ikitommistill, I find the below bit unintuitive: without :conform-keys the first one informs of error but the second one does not:
(s/conform
  (s/map-of keyword? keyword?)
  {ā€œkeyā€ :value})
; :clojure.spec/invalid

(s/conform
  (s/map-of (s/and (s/conformer keyword) keyword?) keyword?)
  {ā€œkeyā€ :value})
; {ā€œkeyā€ :value}
#2017-04-1808:43didibus@ikitommi Why would a string conform to a keyword?#2017-04-1809:52ikitommibecause of the conformer. If I set the :conform-keys it conforms ok as Alex pointed out. But without it, the conform returns invalid data, which feels odd given the s/conform docs: ā€œGiven a spec and a value, returns :clojure.spec/invalid if value does not match spec, else the (possibly destructured) value.ā€œ.#2017-04-1812:29Alex Miller (Clojure team)Seems like you're ignoring the map-of docstring#2017-04-1812:29Alex Miller (Clojure team)There are several good reasons not to conform the keys by default#2017-04-1812:36pseudwhat's the point of clojure spec's fdef ? If I check against my specs using pre & post-conditions, I will get failures at the point of invocation, but having defined a function spec doesn't really prevent the function returning a non-compliant value (and I assume the same is true for providing wrong input)#2017-04-1812:37pseud(aside from stubbing dependent functions for tests)#2017-04-1812:37Alex Miller (Clojure team)For use with stest/instrument and stest/check#2017-04-1812:38pseudThat's it ? The spec guide hints at fdef providing something for development, I can't temporarily get more feedback during development ?#2017-04-1813:01urbank@pseud Isn't instrument meant for development - you can instrument a function so that it check its input, output when called?#2017-04-1813:03curlyfryinstrument only checks input for fdef if I remember correctly#2017-04-1813:03curlyfryBut yeah, it's meant to give more feedback during development#2017-04-1813:47luxbockthere's https://github.com/jeaye/orchestra if you want to check :ret and :fn as well#2017-04-1816:06mobileinkhaving trouble composing a few simple specs. seems to work until I use the range spec listed in the snippet.#2017-04-1816:08mobileinkalso, if I compose :oic/core with something that does not include this range spec, it works ok.#2017-04-1816:10mobileinkbackground: converting json schemata like this to spec: https://github.com/OpenInterConnect/IoTDataModels/blob/master/oic.baseResource.json#2017-04-1816:24arohneris there a way to make check stop on first failure?#2017-04-1816:27mobileinksame thing happens with (s/def ::step (s/or :i integer? :n number?))#2017-04-1816:32jfntnIs it possible during tests to have an fdef check its args but skip checking with exercised inputs?#2017-04-1816:48mobileinkis this a bug?
(s/def ::foo number?)
(s/def ::bar (s/or :i integer? :n number?))
(s/def ::baz (s/and (s/keys :opt [::foo]) (s/keys :opt [::bar])))
(s/explain ::baz {::foo 1 ::bar 9})
;; In: [:oic.r/bar] val: [:i 9] fails spec: :oic.r/bar at: [:oic.r/bar :i] predicate: integer?
;; In: [:oic.r/bar] val: [:i 9] fails spec: :oic.r/bar at: [:oic.r/bar :n] predicate: number?
#2017-04-1817:59seancorfieldNo. s/and flows the conformed value through subsequent forms.#2017-04-1817:59seancorfield(s/conform (s/keys :opt [::foo]) {::foo 1 ::bar 9}) ;=> #:boot.user{:foo 1, :bar [:i 9]} ā€” s/keys conforms all key/values if there are specs for them, not just the keys listed in :req and :opt.#2017-04-1818:00seancorfield^ @mobileink#2017-04-1818:02seancorfieldYou want s/merge here to combine key-specs:
boot.user=> (s/def ::baz2 (s/merge (s/keys :opt [::foo]) (s/keys :opt [::bar])))
:boot.user/baz2
boot.user=> (s/explain ::baz2 {::foo 1 ::bar 9})
Success!
nil
#2017-04-1818:06mobileinkdā€™oh! thanks, he mumbled sheepishly.#2017-04-1908:25mike_ananevHi! I would like to add some :doc to my spec? Is there any way to do that? I need some text comments for my spec#2017-04-1908:26mike_ananevfor data, not for fn's#2017-04-1911:35Alex Miller (Clojure team)Not currently but maybe eventually#2017-04-1914:11robert-stuttafordis there anything out there that talks about how to write custom generators for s/multi-spec?#2017-04-1914:12robert-stuttafordif we write a generator that returns something sufficient for the dispatch fn to use, will the generator code know to keep generating data for the specs returned by the defmethods?#2017-04-1914:13robert-stuttafordin our case, we dispatch in two different ways based on the value of :type, so weā€™re thinking weā€™d gen a map with :type with a valid value from one of the two sets of :type values we keep#2017-04-1915:19Alex Miller (Clojure team)Well that's where retag comes in. However I was looking at something the other day that made me question whether multi-spec was working properly in gen for non-keyword retag cases#2017-04-1915:20Alex Miller (Clojure team)I guess I would turn your question around and ask whether it's doing what you expect#2017-04-1916:48mobileinkhi. I have a map with a type key (:oic.core/rt, for resource type), whose value is a vector of type keys. how can I tell spec to form the conjunction of those types so that valid maps satisfy them all? Something like (apply s/and [::foo ::bar]), which doesnā€™t work.#2017-04-1916:52ghadiNo need. spec will automatically check all namespaced keys for which there is a spec registered#2017-04-1916:53mobileinkright, but different types will have different specs; for example, :oic.r/temperature requires ::p/temperature#2017-04-1916:54mobileinkso i need map-level validation. a map might look like
{::p/temperature 72
            :oic.scale/temperature ā€œF"
            :oic.core/rt [:oic.r/temperature :oic.r/sensor oic.r/humidity]
            :oic.core/if [:oic.if/baseline]            }
#2017-04-1916:54mobileinkthis should fail, because :oic.r/humidity requires ::p/humidity#2017-04-1916:55mobileinkitā€™s easy to use s/get-spec, but how can I then apply the spec?#2017-04-1916:59ghadiLook at s/merge, which merges map specs#2017-04-1917:01mobileinkyep, know about that, but i think thatā€™s static - here we wonā€™t know what to merge until runtime. at least, iā€™ve tried to use it but no luck so far.#2017-04-1917:20mobileinki think i need something like multi-spec with the ability to dispatch multiple times#2017-04-1917:30ghadiAre you changing the meaning of namespaced keys in different maps?#2017-04-1917:31ghadiOr are you having different sets of namespaced keys#2017-04-1917:45mobileinkthe idea is one schema for each oic.r ā€œresourceā€ at https://github.com/OpenInterConnect/IoTDataModels, but a resource ā€œinstanceā€ can have multiple types - listed in :oic.core/rt#2017-04-1917:48mobileinkI think i have a possible solution: I use multi-spec, but my multi-methods recur over the vector of keywords. so given :oic.core/rt [::foo ::bar], the dispatch method will dispatch on ::foo, and the defmethod for ::foo will recur with (rest (:oic.core/rt arg)) before calling s/keys. seems to work with a simple example at least. kinda cool imho.#2017-04-1917:48mobileinkamazing what you can do with clojure#2017-04-1917:49mobileinke.g.
(defmethod oic-rt :oic.r/temperature [m]
  (let [rts (:oic.core/rt m)
        nextm (merge m {:oic.core/rt (vec (rest rts))})]
    (s/merge
     (if (rest rts)
       (oic-rt nextm))
     (s/keys :req [::p/temperature] :opt [:oic.scale/temperature]))))
#2017-04-1917:54mobileinkbasically a runtime s/merge#2017-04-1919:34robert-stuttafordthanks @alexmiller#2017-04-2022:25donaldballI am just now using multi-spec for the first time and am finding it a little weird that it doesn't conform into a variant tuple like or does#2017-04-2221:52pseudI know there's supposed to be some way by which I can refer to a spec, bar residing in [my.example.foo :as foo] by its alias, but (s/conform :foo/bar ...) doesn't seem to be it. Anyone knows/remembers ?#2017-04-2221:57dergutemoritz@pseud You gotta use two colons like that: ::foo/bar#2017-04-2310:40urbankis https://github.com/jeaye/orchestra usable with clojurescript? It requires clojure.spec, not cljs.spec, and uses StackTraceElement->vec, which I can't find in the clojurescript api#2017-04-2310:54lsenjovLooks to be straight .clj, so probably not#2017-04-2310:55lsenjovAlthough it doesn't seem to be using any java interop, so a bit of dependency management and it should work#2017-04-2311:19urbankIt does use (.getStackTrace (Thread/currentThread) ... though I suppose that's not really a necessary function, and it would just be different in clojurescript#2017-04-2311:25lsenjovOh, I missed that one#2017-04-2311:26lsenjovHmmm, it also uses .applyTo just below it, but that shouldn't be a problem#2017-04-2314:08pseud@dergutemoritz cool, thanks šŸ™‚#2017-04-2412:57hkjelsHow do I ensure that a spec never generates the same value twice?#2017-04-2413:17tbaldridge@hkjels you mean like values in a collection are unique? And this is for the generator for a spec?#2017-04-2413:17hkjelsYes#2017-04-2413:20hkjelsI can probably make a custom generator that checks for each item, but I was hoping that something exists already#2017-04-2413:28hkjelsActually, I think I have a good solution here#2017-04-2413:36tbaldridge@hkjels I think that exists#2017-04-2413:37tbaldridges/coll-of takes an optional parameter of :distinct#2017-04-2414:25hkjels @tbaldridge Thank you!#2017-04-2418:19mobileinkany general-purpose spec libs out there? my googling didn't find any. I'm specking routine stuff like href, uri/url, name, uuid, etc. also more complicated but widely used stuff like phone number, postal address, etc. easy enuff, but a well-maintained lib would be better.#2017-04-2418:36mobileinkmore generally: lots of schema languages and schemas out there, that can be expressed in spec. i'm working on the OCF/OIC schema; another possibility is FOAF (https://en.m.wikipedia.org/wiki/FOAF_(ontology)). it would be nice to settle on conventions, e.g. spec.foo.bar for a spec implementation of schema foo.bar.#2017-04-2500:34souenzzoThere is this specs:
(s/def ::a integer?)
(s/def ::b string?)
(s/def ::m (s/keys :opt [::b]))
When I try
(s/valid? ::m {::b "b" ::a "b"})
I get :clojure.spec/invalid, explanation: val ["a"] fails... Is it a bug?
#2017-04-2500:35souenzzoIMHO it should be valid, once spec just looks for what is defined, and on ::m spec's, there is no ::a#2017-04-2500:50ghadi> When conformance is checked on a map, it does two things - checking that the required attributes are included, and checking that every registered key has a conforming value. Weā€™ll see later where optional attributes can be useful. Also note that ALL attributes are checked via keys, not just those listed in the :req and :opt keys. Thus a bare (s/keys) is valid and will check all attributes of a map without checking which keys are required or optional. https://clojure.org/guides/spec#2017-04-2500:51ghadi@souenzzo this is actually by design#2017-04-2500:53ghadiSpecs registered under namespaced keys give the keys global meaning thus are checked whenever they are encountered#2017-04-2500:54ghadiThis is different than other systems. (I like it)#2017-04-2509:00mbjarlandhave a clojure.spec question, assume I have a number of specs defined for seq:s of chars, how would I define a string spec which uses them? i.e given:
(s/def ::my-char-sequence
  (s/cat :valid-chars (s/* (s/and char? #{\a \b}))))
=> my-char-sequence

(s/explain ::my-char-sequence [\a \b])
Success!
=> nil

(s/explain ::my-char-sequence [\a \c])
In: [1] val: \c fails spec: my-char-sequence at: [:valid-chars] predicate: #{\a \b}
=> nil
how would I write a string spec using the char seq spec:
(s/def ::my-string
  (s/and string?
             (??? <something using ::myc-char-sequence and perhaps (seq %) > ???)))
#2017-04-2510:09dergutemoritz@mbjarland Inserting (s/conformer seq) after the string? predicate in the ::my-string spec should do the trick!#2017-04-2510:26mbjarland@dergutemoritz !! ahh...I knew there had to be away to make a transformation before handing it off to spec...that is great!! thanks#2017-04-2510:29dergutemoritz@mbjarland You're welcome! Note that when using that with s/conform it'll also return a seq of chars. Maybe you want to add something like (s/conform (partial apply str)) at the end of the chain.#2017-04-2510:30mbjarlandseems conformer has a second unform argument function#2017-04-2510:30mbjarlandwould that fix it#2017-04-2510:31dergutemoritzYeah but that's for use with s/unform for recovering the original value - slightly different approach with different pros and cons but works, too šŸ™‚#2017-04-2510:32dergutemoritzAnother option would be to use string regexen instead of spec regexen#2017-04-2510:33mbjarlandyeah, I was hoping not to have to do string regex#2017-04-2510:33mbjarlandI like the composability of specs#2017-04-2510:35mbjarlanderrr...I'm assuming "string regexen" refers to actual regular expression #" " matching?#2017-04-2510:45Adam MunozHi everyone. How do you spec :args of any type? i.e. takes x that can be anything and returns for example boolean.#2017-04-2510:55Adam MunozOk, any? šŸ™‚#2017-04-2510:55Adam Munozthanks anyway#2017-04-2510:58mbjarland@adammunoz, I'm a beginner with spec but perhaps something like:
(defn some-fn [x]
  true)

(s/fdef some-fn
        :args (fn [_] true)
        :ret boolean? )
->
(doc some-fn)
-------------------------
string-layout.core/some-fn
([x])
Spec
  args: (fn [_] true)
  ret: boolean?
#2017-04-2510:58Adam Munoz@mbjarland Thanks. No I meant that predicate that I want is maybe any? .#2017-04-2510:59Adam Munoz(any? 1) => true#2017-04-2510:59mbjarland: ) yes looks like it#2017-04-2510:59Adam Munoz(any? "whatever") =>true#2017-04-2510:59mbjarlandI figured any would be collection based, but you are right#2017-04-2510:59Adam MunozšŸ˜‰#2017-04-2512:10dergutemoritz@mbjarland Yeah that's what I meant by string regexen šŸ™‚#2017-04-2512:11dergutemoritzNote that there are also various libraries which provide a structured representation of regexen similar to that of spec which can be composed just as well and compiled down to java.util.regex.Pattern#2017-04-2512:20mbjarland@dergutemoritz have any clj ones to share?#2017-04-2512:33dergutemoritz@mbjarland I seem to remember having seen multiple ones actually, but I fail to find them again!#2017-04-2512:33mbjarland@dergutemoritz : ) no worries (edit: was on touch screen...they suck)#2017-04-2512:33dergutemoritzI have used Scheme Regular Expressions with great pleasure in the past, though. Perhaps I'm mixing it up.#2017-04-2512:34mbjarlandand as a side question, how do you organize specs for functions? Right after the function itself? in another namespace? somewhere else?#2017-04-2512:35mbjarlandkeeing them close feels good in a sense but also pollutes the code with a lot of "spec noise"...for good or bad#2017-04-2512:36dergutemoritzPersonally, I like to put them before the function definition (if you mean fdefs) but I don't have enough confidence in it to recommend it as a good idea or anything šŸ˜‰#2017-04-2512:36mbjarlandok#2017-04-2512:36dergutemoritzAlso, I'm not specing each and every function just for the sake of it#2017-04-2512:40mbjarlandI got another one (starting to feel like a spambot here), do you see any immediate reason why this:
(s/def ::align-specifier
  (s/and char?
         #{\L \C \R \l \c \r}))

(s/def ::bracket-expr
  (s/cat :left-bracket #{\[}
         :align ::align-specifier
         :right-bracket #{\]}))

(s/def ::between-bracket-expr
  (s/cat :between-char
         (s/* (s/and char? (complement #{\[ \]})))))

(s/def ::layout-exprs
  (s/cat :le
         (s/* (s/alt :br ::bracket-expr :be ::between-bracket-expr))))
would result in:
(s/explain ::layout-exprs [\a \b])
StackOverflowError   clojure.core/complement/fn--6675 (core.clj:1438)
#2017-04-2512:41mbjarlandimprovements to the specs also welcome, I can for example see that I could remove the (s/and char? in the first one#2017-04-2512:42mbjarlandall of ::align-specifier, ::bracket-expr and ::between-bracket-expr work in isolation#2017-04-2512:42mbjarlandas in I can run explain and valid? on them and they behave as I would expect#2017-04-2512:42mbjarlandah...complement...maybe I need to be explicit there instead...#2017-04-2512:43mbjarlandI did feel a bit woozy writing (complement #{\[ \}}) : )#2017-04-2512:51dergutemoritz@mbjarland Looks like you're running into an infinite left recursion because ::between-bracket-expr may not consume any input#2017-04-2512:58mbjarland@dergutemoritz ok, I will have to marinate on that a bit. Thanks for the pointer#2017-04-2513:00dergutemoritz@mbjarland https://en.wikipedia.org/wiki/Left_recursion#Removing_left_recursion might help#2017-04-2513:02dergutemoritzOr perhaps not šŸ˜„#2017-04-2513:03mbjarlandfood for marination#2017-04-2513:05dergutemoritzBon appĆ©tit!#2017-04-2513:06mbjarlandthanks!#2017-04-2513:17mbjarlandand yes, changing (s/* to (/s+ in :between-bracket-expr fixes the problem. It probably should have been + to begin with#2017-04-2513:55mbjarlandand so I'm lost in the woods again...and not for the first time with spec : ) If I write standalone specs I can run explain/valid?/exercise etc on them. What if I spec a function, do I have to actually instrument the function to test my spec or is there a way to get hold of the function spec and run the normal spec-exercising-functions on it?#2017-04-2513:56mbjarlandI guess I can do a s/def on say the args spec and then exercise that separately...#2017-04-2514:19Alex Miller (Clojure team)yes, you can call s/get-spec on the fully-qualified symbol to get the function spec, and the function spec supports keyword lookups for itā€™s parts (`:args`, etc).#2017-04-2514:21mbjarland@alexmiller ok, nice and data centric as usual. thank you#2017-04-2514:22Alex Miller (Clojure team)
(s/fdef user/foo :args (s/cat :a int?))
(s/valid? (:args (s/get-spec 'user/foo)) [100])
#2017-04-2514:22Alex Miller (Clojure team)or you can go the other way and define the args spec as a standalone spec and then assemble the fdef spec from it (Iā€™ve done this in some cases)#2017-04-2514:23mbjarlandthat's what I ended up with, but still good to know it's not a black hole and you have access to the function spec#2017-04-2514:26mbjarlandI find my biggest hurdle with spec so far is not understanding the language but understanding its place in the process and best practice use. Is there for example still a place for using specs as preconditions in functions? I guess it would be nice to see an example of a larger production system using spec (or a similarily reality checked code base) and what kind of usage patterns they ended up with. Any references to presentations or repos with such code base much appreciated....#2017-04-2516:33sparkofreasonIs there any way to use coll-of or every (or some other existing capability in spec short of writing custom predicates and generators) to spec non-Clojure collections, like java.util.HashSet? I feel pretty certain the answer would be "no", but just want to double-check before reinventing any wheels.#2017-04-2517:02dergutemoritz@dave.dixon Since most (all?) Java collections are seqable, you should be able to make it work by anding in a seq conformer#2017-04-2517:10sparkofreason@dergutemoritz thanks. Looks like even every works if you specify :kind to be a custom predicate, but still doesn't generate. And just verified every-kv doesn't seem to work with :kind, and keys doesn't give any hooks for custom collections, as far as I can tell. Which seems sensible - can't support everything possible in the JVM with a single API.#2017-04-2517:12dergutemoritz@dave.dixon Oh, right, generation isn't covered by my suggestion either. Note what the doc string says about a custom :kind predicate, though: > :kind - a pred/spec that the collection type must satisfy, e.g. vector? (default nil) Note that if :kind is specified and :into is not, this pred must generate in order for every to generate.#2017-04-2517:14dergutemoritzHm but I doubt that passing something like :into (HashSet.) will just work either šŸ™‚#2017-04-2517:14sparkofreason@dergutemoritz not documented, but the collection must also support the Clojure interface for supporting into. #2017-04-2517:15dergutemoritzYeah, seems reasonable#2017-04-2517:15dergutemoritzI doubt that spec is of much use for mutable collection types#2017-04-2517:36sparkofreasonActually, I want to use it for bifurcan collections. #2017-04-2518:56Alex Miller (Clojure team)covering Java colls was not really a goal of the spec coll preds#2017-04-2518:58Alex Miller (Clojure team)I canā€™t say weā€™ve really talked about it either way though. doesnā€™t seem like thereā€™s any technical reason such a thing couldnā€™t exist. not sure if there are performance concerns in growing/shrinking non-persistent colls in generators.#2017-04-2612:39mbjarlanda followup question to one I posted yesterday, if I have a conformer from string to seq of chars:
(s/def ::layout-string
  (s/and string?
         not-empty
         (s/conformer seq)
is there any way within spec to have the conformed values turned back into strings?
#2017-04-2612:41mbjarlandI take that back, turns out @dergutemoritz already answered this one#2017-04-2612:47gfredericks#2017-04-2615:33Alex Miller (Clojure team)important info re 1.9/spec - https://groups.google.com/d/msg/clojure/10dbF7w2IQo/ec37TzP5AQAJ#2017-04-2616:00bronsa@alexmiller https://github.com/clojure/core.specs.alpha/blob/master/src/main/clojure/clojure/core/specs/alpha.clj#L1 wrong ns name here#2017-04-2616:01Alex Miller (Clojure team)thanks :)#2017-04-2615:37dpsuttonspec will be versioned then?#2017-04-2615:38Alex Miller (Clojure team)yes#2017-04-2615:51Alex Miller (Clojure team)versions will be ala core.async: 0.1.<git-change-derived>#2017-04-2615:53gfredericksthe specs for clojure.core functions/macros will stay with clojure?#2017-04-2615:54bronsano, that's what core.specs.alpha is for AFAICT#2017-04-2615:54dpsuttonit'll be interesting to see how this impacts CIDER's support for spec. Now the tooling will have to make sure to match the version of spec that the application uses#2017-04-2615:54gfredericks@bronsa that's a namespace in the spec artifact?#2017-04-2615:54bronsait's a separate contrib lib#2017-04-2615:54gfredericksoh so you need three artifacts if you want to use clojure with the core specs?#2017-04-2615:55bronsaI'm assuming clojure will depend on core.spec.alpha but dunno#2017-04-2615:55gfredericksI'm just trying to figure out if/how the specs can drift from the implementations#2017-04-2615:55gfredericksif the specs stay with clojure then that's not an issue#2017-04-2615:59gfredericks> Additionally, this is a first step towards increased support for leveraging dependencies within Clojure. I don't know what that means#2017-04-2616:02Alex Miller (Clojure team)@bronsa yes, clojure will depend on the two new libs#2017-04-2616:03Alex Miller (Clojure team)but you can also specify a newer version and override the one its using#2017-04-2616:03Alex Miller (Clojure team)@gfredericks towards demonolithing clojure#2017-04-2616:03bronsaFWIW i have a similar dependency tree with t.a/t.a.jvm and I found that it's easier to just depend on the leaf dependency (t.a.jvm or in spec's case core.spec.alpha) and release concurrently when the root dependency needs a change#2017-04-2616:13mobileinkjust curious: why not put the ā€œalphaā€ bit in the version string instead of the ns?#2017-04-2616:15tbaldridge(note: I'm speaking as a Clojure user here, not as someone who has authority on the subject)#2017-04-2616:15tbaldridgeIt's answered somewhat in Rich's talk on spec: https://www.youtube.com/watch?v=oyLBGkS5ICk#2017-04-2616:16tbaldridgeBut the gist is you could have alpha, beta, v1, v2, v* code all in the same artifact if you wanted. That just because spec moves to "beta" or "RC" doesn't mean the alpha API has to go away.#2017-04-2616:22Alex Miller (Clojure team)at some point there will be namespaces that are not alpha. the alpha stuff will continue to work.#2017-04-2616:23mobileinkah, so this is just until we get to v1?#2017-04-2616:24Alex Miller (Clojure team)well, same idea may also apply to any breaking changes post v1#2017-04-2616:24mobileinkok, thanks#2017-04-2616:24Alex Miller (Clojure team)whether thatā€™s done at the fn level or ns level is a future decision point#2017-04-2616:36mikerodThe only problem I see with making libraries like spec (not part of lang) are ā€œjar hellā€ dep issues#2017-04-2616:36mikerodin JVM land libraries using deps always leads to trouble because you end up using 2 libs with 2 different version of the same dep and they are incompatible#2017-04-2616:36mikerod(eventually)#2017-04-2616:37mikerodHowever, I know Richā€™s talk at the last Clojure/conj was about never making a breaking change essentially. So maybe clojure.spec will be robust enough to avoid the problem and you can always just ā€œtake the newest versionā€ of the dep when multiple lib deps diagree on version.#2017-04-2616:38mikerodI just know that I ran into issue with say, Plumatic schema when several libraries were using it and I was using those libraries and then schema came out with a set of breaking api changes#2017-04-2616:39mikerodThis libraries having dependencies conflict problem has been one of the greatest pains of mine on the JVM it seems. I donā€™t think Iā€™m alone there though. It can get difficult to deal with. I think it often encourages libraries to not use any dependencies. Which I feel is unfortunate for code-reuse or using useful utilities for documentation like spec/schema.#2017-04-2616:40mikerodOr in Java-land sometimes people do build-time repackaging of classes to some ā€œinternalā€ package name. I think it is harder in clj to confidently add a prefix to a set of namespaces though in a way that doesnā€™t miss anything.#2017-04-2616:57seancorfieldMy experience ā€” six years of production Clojure with a code base of now 50K lines ā€” is that API version conflicts are actually pretty rare in Clojure libraries in the wild.#2017-04-2616:57seancorfieldThey are more common with Java libraries, true, but Clojure people seem to be better about API changes.#2017-04-2616:57seancorfield(says the man whoā€™s guided clojure.java.jdbc through several breaking API versions, however)#2017-04-2617:00bronsaseancorfield: c.j.j is different than most libs in that it's used much more by application code than by other libs so it's easier to avoid getting into dependency issues IMO #2017-04-2617:03seancorfieldYes, and thatā€™s in line with why I think the utility libraries tend to be the worst offenders here. Itā€™s why I have serious concerns about relying on Timbre, Carmine, Faraday, etc ā€” even thoā€™ Peter seems really careful about not breaking the API across versions.#2017-04-2616:59seancorfieldI think the danger comes mainly from Clojure libraries that themselves have a large number of transient dependencies where they rely on Swiss Army Knife utilities libraries for just one or two ā€œcool functionsā€ ā€” which is an indictment of the questionable validity of such libraries, IMO. They should be broken up into much smaller libraries so you donā€™t have to pull in the whole thing for just one or two useful functions.#2017-04-2617:00seancorfieldBut this is a case of ā€œdo one thing and do it wellā€ which such libraries inherently donā€™t follow šŸ˜ž#2017-04-2617:13tbaldridgeRight, part of the issues with "jar hell" are really problems with non-additive changes to APIs.#2017-04-2617:19mikerodYeah, thatā€™s sort of what I was thinking was the answer to this concern here. Itā€™d be great to see that workout.#2017-04-2617:19mikerodItā€™s the ā€œsecret weaponā€ šŸ˜›#2017-04-2617:46Alex Miller (Clojure team)the whole idea is to make all versions of a library compatible#2017-04-2617:47Alex Miller (Clojure team)but the important thing here is that alpha is a time before this applies and breaking stuff is ok (you as a user have to track the changes)#2017-04-2618:34seancorfield@alexmiller I asked on the list but Iā€™ll ask here too: will the new org.clojure/spec.alpha artifact be available for a few days before Clojure master SNAPSHOT actually removes the namespaces?#2017-04-2618:35Alex Miller (Clojure team)itā€™s available now#2017-04-2618:35seancorfieldOh, cool!!#2017-04-2618:35Alex Miller (Clojure team)org.clojure/spec.alpha 0.1.94#2017-04-2618:40seancorfield@alexmiller This hasnā€™t hit Maven Central yet, right?#2017-04-2618:40Alex Miller (Clojure team)it has#2017-04-2618:40Alex Miller (Clojure team)I just downloaded it from there#2017-04-2618:41Alex Miller (Clojure team)https://repo1.maven.org/maven2/org/clojure/spec.alpha/#2017-04-2618:57seancorfieldAh, guess it just hasnā€™t been indexed yet? It doesnā€™t show up on https://search.maven.org yet. OK, got it, thanks!#2017-04-2618:35Alex Miller (Clojure team)as of just a few moments ago#2017-04-2618:35seancorfieldThank you! Much appreciated!!#2017-04-2618:35Alex Miller (Clojure team)but we are moving (slowly) towards an updated alpha release of clojure that uses it today#2017-04-2618:36seancorfieldIā€™ll get our codebase switched over today thenā€¦#2017-04-2620:01dominicm@alexmiller I'm a little curious about this shift. If you depend on "1.9.0", is it part of clojure's api to expose it's transitive dependency on spec? Or should I always explicitly depend on spec if I'm using it in my lib?#2017-04-2620:03seancorfieldI would expect core 1.9 will conditionally use spec at this pointā€¦ So if you have spec loaded, youā€™ll get better macro error messages?#2017-04-2620:03Alex Miller (Clojure team)@dominicm clojure will have a compile-time dependency on spec.alpha so you can always rely on it being available. So you donā€™t need to explicitly list the new library as a dependency. You may do so though in order to specify a newer version than Clojure is depending on.#2017-04-2620:05Alex Miller (Clojure team)@seancorfield it is not currently conditional in any way#2017-04-2620:05pdlugIs there a mechanism in spec for namespacing keys as part of a conform? For example, a map parsed JSON will not have namespaced keys so I'm applying some functions that namespace them for me but this feels like a pretty common use case which must have a better solution#2017-04-2620:05Alex Miller (Clojure team)s/keys has :req-un and :opt-un options for this#2017-04-2620:06seancorfieldSo Clojure 1.9 will transitively bring in the spec library automatically? But you can bring in a different version yourself? How will that work if the spec library breaks the API?#2017-04-2620:06Alex Miller (Clojure team)@pdlug see the guide for an example https://clojure.org/guides/spec#2017-04-2620:06Alex Miller (Clojure team)@seancorfield yes. yes. then youā€™re broken (see ā€œalpha = things may changeā€).#2017-04-2620:07Alex Miller (Clojure team)sorry, have to jump in the car but can pick up later#2017-04-2620:08pdlug@alexmiller Any pointers as to where? I've been through that a few times, I see the examples on spec'ing unqualified keywords but not coercing to namespaced keywords as part of a conform#2017-04-2620:09seancorfieldAh, so for clojure.spec to break Clojure itself, youā€™d already have to have a new public release of Clojure available (even if it is 1.10 alpha perhaps at this point)?#2017-04-2620:09seancorfieldIā€™m hoping you wouldnā€™t make a new release of clojure.spec that then required users of it to rely on SNAPSHOT builds of Clojure ā€” youā€™d at least have a new alpha of Clojure available by that point?#2017-04-2620:10seancorfield(as an active user of clojure.spec in production, this is all a bit frustrating and a little worrying!)#2017-04-2620:57seancorfield@pdlug I think maybe Alex misread what you were trying to do? There's no automated way to produce qualified keys from unqualified ones. #2017-04-2620:58pdlug@seancorfield Thanks, that's exactly what I was asking, that's unfortunate because this seems like a really common use case (JSON APIs, JSONB in PostgreSQL, etc.)#2017-04-2621:00seancorfieldSince any namespace qualifier you would be adding is arbitrary, I don't see how spec plays into this: you control that arbitrary key qualification. #2017-04-2621:02seancorfieldSomewhere you're converting string data to a Clojure representation of JSON. Why not introduce the qualifier at that point -- then spec with qualified keys directly?#2017-04-2621:03seancorfieldThat's why clojure.java.jdbc provides a :qualifier option, for example. #2017-04-2621:03mobileinki do it by typing (on the keyboard).#2017-04-2621:04seancorfieldPardon @mobileink ?#2017-04-2621:04mobileinkfor a json schema i'm working with i can come up with namespaces for everything, but automating it seems like a lot more work than just typing it in.#2017-04-2621:09Alex Miller (Clojure team)@seancorfield if you were asking about spec breaking Clojure, I don't see how we would do that. The integration points are pretty small. #2017-04-2621:10seancorfield@alexmiller I was considering a situation where you made a breaking API change in clojure.spec.alpha that could affect Clojure itself (since it depends on the earlier, non-breaking API)#2017-04-2621:11mobileinke.g.json schema puts "di" in schema "foo.bar". in spec: :foo.bar/di.#2017-04-2621:13pdlugI think this would be easier if there were more functions for manipulating maps to add namespaces to keywords. I keep copying the same keywordize-keys-ns function in every project to coerce incoming JSON into keywords w/ namespaces. Would be nice to also have a rename-keys that could do unqualified to qualified mappings en masse.#2017-04-2621:13seancorfieldBut if the integration surface is small, I guess that breakage is unlikely ā€” and what I meant was that youā€™d already have to have a version of Clojure somewhere that used the new (changed) API in order to test/use it anyway. I just wanted to check that if you ever made such a breaking change to clojure.spec.*, youā€™d ensure there was a *non-SNAPSHOT* version of Clojure made available at the ā€œsame timeā€.#2017-04-2621:14bronsa@seancorfield the only thing clojure depends on from spec atm is s macroexpand-check, I find it really hard to imagine a situation where that could be broken#2017-04-2621:15Alex Miller (Clojure team)I would consider that broken :)#2017-04-2621:16seancorfieldWell, the World Singlesā€™ main codebase now uses clojure.spec.alpha with an explicit version dependency so Iā€™m ā€œhappyā€ šŸ™‚#2017-04-2621:18thheller@alexmiller any particular reason why the core.specs.alpha is a separate package?#2017-04-2621:18Alex Miller (Clojure team)Vs?#2017-04-2621:18thhellerkeeping it in clojure.spec.alpha or clojure.core I guess#2017-04-2621:19thhellerjust wondering if there are extended plans for it#2017-04-2621:20Alex Miller (Clojure team)You mean why in a different ns or why in a different project/artifact?#2017-04-2621:20thhellerdifferent artifact#2017-04-2621:21Alex Miller (Clojure team)It's logically separate and can evolve at its own rate#2017-04-2621:22thhellerok I'm still confused how the whole fdef / macroexpand-check integration is going to work, guess I'll have to wait and see#2017-04-2621:22Alex Miller (Clojure team)We made the decision long enough ago that I've forgotten the details tbh#2017-04-2621:22Alex Miller (Clojure team)@thheller it's exactly the same?#2017-04-2621:23Alex Miller (Clojure team)The code is just in a different artifact#2017-04-2621:25thhellerok, I keep thinking there is a circular dependency somewhere#2017-04-2621:26Alex Miller (Clojure team)There is#2017-04-2621:26Alex Miller (Clojure team)spec.alpha depends on Clojure#2017-04-2621:26Alex Miller (Clojure team)Clojure depends on spec.alpha#2017-04-2621:27Alex Miller (Clojure team)There is a bootstrap aspect to it#2017-04-2621:29thhellerah right, forgot the current code already does a var lookup.#2017-04-2621:30Alex Miller (Clojure team)I find it's best not to think about it too hard :)#2017-04-2621:30thhellerhehe yeah I'm thinking more about CLJS at the moment#2017-04-2621:30thhellerI was using the core.specs for CLJS but I guess that is just as easy with them in a lib#2017-04-2621:30Alex Miller (Clojure team)It's easier#2017-04-2621:37Alex Miller (Clojure team)The specs are not the same though for things like ns#2017-04-2621:37thhellerwell, everything besides ns is though#2017-04-2621:38thhellerbut still trying to come up with a good way to intergrate them with CLJS properly. might need to copy them anyways for self-hosted.#2017-04-2621:59Alex Miller (Clojure team)Open to helpful ideas#2017-04-2622:01thhellerstill trying to solve the "helpful errors" issue, currently the spec explain for something wrong in a let doesn't look much more helpful than the actual compiler error#2017-04-2622:16Alex Miller (Clojure team)Well more to come on that#2017-04-2622:17seancorfield@thheller I think the benefit will come from familiarity ā€” all spec-powered errors will have a similar output format and folks will quickly get used to them (both from let and other core constructs, as well as from general libraries that use spec). And any improvements to how explain works will improve all errors from code that uses spec.#2017-04-2622:18seancorfieldI mean, yeah, itā€™s a lot of output, but itā€™s very regular and structured.#2017-04-2622:24thhellernot too sure about that#2017-04-2622:24thheller
CLJS error in demo/errors.cljs at 3:1
In: [1] val: 1 fails spec: :clojure.core.specs/arg-list at: [:args :bs :arity-1 :args] predicate: vector?
In: [1] val: 1 fails spec: :clojure.core.specs/args+body at: [:args :bs :arity-n :bodies] predicate: (cat :args :clojure.core.specs/arg-list :body (alt :prepost+body (cat :prepost map? :body (+ any?)) :body (* any?)))
:clojure.spec/args  (x 1)
#2017-04-2622:25thhellerthe most helpful part is demo/errors.cljs at 3:1 šŸ˜‰#2017-04-2622:27thhellerthe problem in this case is that 1 is a number which doesn't have reader metadata, so you cannot narrow the cause down further#2017-04-2622:28thheller(defn x 1) is the actual source so 3:1 is the start of the (defn not the 1 which would be more accurate#2017-04-2622:28hiredmanI don't follow#2017-04-2622:29hiredmanthat seems pretty clear#2017-04-2622:29hiredman1 isn't a vector, 1 isn't that other big long thing
#2017-04-2622:30hiredmaneach case gives the value that failed, and the predicate it failed#2017-04-2622:30hiredmanand which spec the predicate came from#2017-04-2622:30thheller@hiredman the predicate can get cryptic as seen in the second message#2017-04-2622:31hiredmanthe grammar is cryptic šŸ™‚#2017-04-2622:31thhellerI think it is alright as well, a beginner might not be so inclined#2017-04-2622:32thhellerbut my issue for not is more about more accurate source locations#2017-04-2622:32thhellerfor the most part they are accurate enough though#2017-04-2622:38Alex Miller (Clojure team)The lack of reader metadata is annoying but not an endpoint necessarily#2017-04-2622:42thhelleryeah coupled with the spec errors you can usually figure out whats wrong pretty quickly#2017-04-2622:44thhellerit kinda breaks apart if the printed val gets too large though#2017-04-2622:48seancorfieldI read that first error as ā€œarg-list is not vector?ā€ which is pretty clear. Even the second one is pretty straightforward: ā€œargs+bodyā€ is not a sequence of ā€œargs followed by bodyā€ ā€” and if you (source defn) that should triangulate to the thing after the symbol being defnd is not either of those.#2017-04-2622:53thheller... please don't get hung up on my toy example. This particular example is "good enough" with spec.#2017-04-2622:53thhellerParameter declaration "1" should be a vector is the default error message without spec ...#2017-04-2622:54thhellerwhich IMHO is better than the spec error (but much less accurate)#2017-04-2623:10thheller@alexmiller will the new spec projects/artifacts get their own JIRA projects or should issues still go to CLJ?#2017-04-2623:33Alex Miller (Clojure team)CLJ #2017-04-2702:52danielcomptonI'm not even sure exactly what this would mean, but is it possible to 'render' a spec for display in for example a Swagger UI? I'm wanting to build a web UI for displaying the specs of commands that could be sent to the system, but I'm not sure if this is even possible, as Spec is so flexible#2017-04-2705:07ikitommi@danielcompton we have been working on the api-docs thing, just finished on the spec -> json schema and will do the -> openapi next. also have some tooling for the runtime conforming needed to support different wire-formats. But like Schema (and ring-swagger), spec is more powerful than the openapi spec, so some information will be lost. Might be a place to do full-clojure thing, with own ā€œspec-uiā€.#2017-04-2705:09ikitommiany schedule for cljs-port of the spec-alpha?#2017-04-2705:09danielcompton@ikitommi yeah, I don't specifically want Swagger, it was just the closest thing I could think of, a spec-ui would be what I was after#2017-04-2705:16ikitommiI think the spec-ui could be done easily, as the spec forms can be serialized. Also, something like the https://github.com/metosin/schema-viz but for spec. Both on todo-list, but with million other things.#2017-04-2706:21thheller@alexmiller I would like to add the var to the ex-data in clojure.core/macroexpand-check. so v in https://github.com/clojure/clojure/blob/master/src/clj/clojure/spec.clj#L686-L698#2017-04-2706:22thhellerso either the v itself or the (->sym v) that is currently used in the message for the ex-data call#2017-04-2706:23thhellerI'm experimenting with some different error presentations and currently the var isn't accessible so can't show the root spec that failed (other than "parsing" the (.getMessage ex))#2017-04-2706:34thhellercould either do this in the macroexpand-check or in the catch that wraps the error in Compiler.java#2017-04-2708:29jindrichmHi -- How do you make a spec for a particular value? (partial = :value) predicate doesn't seem very idiomatic.#2017-04-2709:09dergutemoritz@jindrichm #{:value}#2017-04-2709:09dergutemoritzUnless your value is nil or false šŸ™‚#2017-04-2709:09dergutemoritzBut then you have specific preds#2017-04-2709:11jindrichmThanks! This should work fine and looks more idiomatic than partial.#2017-04-2709:12vandr0iyHi clojurians! How does one make up a spec for a huge nested map that may have different values for the same key but in different places? like this:
{:a [{:foo "foo1" :bar 1}, {:foo "foo2" :bar 8}]
 :b {:baz  {:foo 3 :bar "quuz"}
     :quux {:foo [1 2 3] :baz 10}}
How do I say that :foo is usually a string, but inside a certain context (b/baz in this case) it's a number, and in certain others (b/quux) it's a vector?
#2017-04-2709:22dergutemoritz@vandr0iy If you're asking how you can make s/keys behave like that, the short answer is: it's not what it was designed for but rather it relies on namespaced keywords. However, you could define keywords of the same name in different namespaces representing your various contexts and piece the specs together accordingly with :req-un and :opt-un.#2017-04-2710:31mbjarland@vandr0iy spec-tools data specs might be worth a look? (https://github.com/metosin/spec-tools#data-specs)#2017-04-2710:34mbjarland@vandr0iy errr....ok, they might help, but don't think they solve your different values for the same key problem, @dergutemoritz answer seems to be it#2017-04-2711:33ikitommi@vandr0iy @U4VDXB2TU: data-spec can do that, but I would use the normal s/keys if you need to reuse the attribute specs elsewhere. If itā€™s just a ~one-time thing, then something like this:
(require '[clojure.spec :as s])
(require '[spec-tools.core :as st])

(def spec
  (st/data-spec
    ::foo
    {:a [{:foo string? :bar int?}]
     :b {:baz {:foo int? :bar string?}
         :quux {:foo [int?] :baz int?}}}))

(def data
  {:a [{:foo "foo1" :bar 1}, {:foo "foo2" :bar 8}]
   :b {:baz  {:foo 3 :bar "quuz"}
       :quux {:foo [1 2 3] :baz 10}}})

(s/valid? spec data)
; true
#2017-04-2711:48vandr0iyin the end I just ended up doing specs for every single element that might be recognized as a pattern in its own namespace with the same name as it's called in the data structure I want to specify - but in its own namespace. For instance:
(s/def :type1/stuff string?)
(s/def :type1/foo (s/keys :req-un [:type1/stuff]))

(s/def :type2/stuff number?)
(s/def :type2/element (s/keys :req-un [:type2/stuff]))
(s/def :type2/foo (s/coll-of :type2/element))
this way stuff is a string in a type1 element and a number in type2 one. This data would match:
(s/valid? :type1/foo {:stuff "foo"})
=> true
(s/valid? :type1/foo {:stuff 2})
=> false
(s/valid? :type2/foo [{:stuff 2} {:stuff 4}])
=> true
(s/valid? :type2/foo [{:stuff 2} {:stuff "arst"}])
=> false
#2017-04-2711:47Alex Miller (Clojure team)@ikitommi re cljs, will happen soon, after clj is done#2017-04-2711:51Alex Miller (Clojure team)@thheller does https://dev.clojure.org/jira/browse/CLJ-2085 help at all?#2017-04-2711:54thheller@alexmiller not sure, will try it. thx#2017-04-2711:55ikitommithanks @alexmiller, looking forward to it.#2017-04-2711:59thheller@alexmiller yep should do, I can get the name of the spec via the meta :clojure.spec/name#2017-04-2711:59thhellerwhich corresponds to the var#2017-04-2712:16Alex Miller (Clojure team)I guess I'm not sure why you want the var vs the name#2017-04-2713:33thhellerfor now just the name, maybe the var later. thinking that the var might contain more metadata, will see if I need that#2017-04-2713:34thhellerprobably not though#2017-04-2812:05christianromneyhmm bear with me as i try to understand this. if we have a couple of invariants like: 1. we don't break APIs in non-alpha software and 2. if we do break an API, we pick a new name and 3. clojure.core (non-alpha) will depend on clojure.spec.alpha, then doesn't this imply that for a version of clojure to depend on a clojure.spec (and NOT depend on clojure.spec.alpha), it will have to change its namespace, say to something like clojure2.core? or have i missed something obvious?#2017-04-2812:29Alex Miller (Clojure team)clojure.core doesn't depend on clojure.spec.alpha#2017-04-2813:03christianromneythanks @alexmiller although I'm more confused now than before. i misinterpreted your comment above about the circular dependency. perhaps i should just go look at the artifacts šŸ™‚#2017-04-2813:18christianromneyhmm, @alexmiller could you elaborate a bit more? I'm just not understanding how to reconcile that with your previous explanation that "clojure will have a compile-time dependency on spec.alpha so you can always rely on it being available. So you donā€™t need to explicitly list the new library as a dependency." sorry if i'm just being a dunderhead...#2017-04-2813:29Alex Miller (Clojure team)my original statements are about artifacts. yours seemed to be about namespaces, so I answered in those terms. can you clarify which level youā€™re talking about?#2017-04-2813:30Alex Miller (Clojure team)Clojure 1.9 will ship with an alpha version of spec available - use it if you like (but beware that things in spec may change)#2017-04-2813:30Alex Miller (Clojure team)thatā€™s the whole story#2017-04-2813:35mhuebertIs there a natural way to have the generator for a regular expression spec return a vector instead of a sequence? #2017-04-2813:36Alex Miller (Clojure team)currently, no#2017-04-2813:37Alex Miller (Clojure team)but it is something many people (including myself) have felt the need for and something I have discussed a couple times with Rich#2017-04-2813:37Alex Miller (Clojure team)and I think ultimately something will plug that gap#2017-04-2813:46mhuebertOk, thanks. (The two use cases I have in mind are hiccup forms and function argslists)#2017-04-2813:47Alex Miller (Clojure team)yes, Iā€™ve run into it in specā€™ing the latter#2017-04-2814:04christianromney@alexmiller oh i see so you'll declare a dep on the artifact but never require it is that right?#2017-04-2814:05Alex Miller (Clojure team)mostly - there are a couple things in Clojure that use spec#2017-04-2814:05Alex Miller (Clojure team)one is the args check for fdef macro specs during macroexpansion (so in the compiler)#2017-04-2814:06Alex Miller (Clojure team)the other is the lookup and inclusion of specs in printed docs from clojure.repl/doc#2017-04-2814:06Alex Miller (Clojure team)both of those are kind of implicit rather than being part of the API though#2017-04-2814:08christianromneyok thanks, that gives me much more to chew on. šŸ™‚ i don't see this necessarily impacting me, i just really want to grok the engineering you're doing šŸ™‚#2017-04-2814:27mhuebertSo in the meantime, something like this should be ok (to return vectors from a regular expression): https://gist.github.com/mhuebert/8fdeedae57bf797778054dcf8f33ab8b#2017-04-2814:34Alex Miller (Clojure team)gen/fmap with vec would be simpler, no need for bind here#2017-04-2814:35Alex Miller (Clojure team)#(gen/fmap vec (s/gen expr))#2017-04-2906:47fossifoo@ikitommi not sure, where to put this on slack, but i currently try to use spec-swagger from cljs/node and it seems like spec-tools depends on a lot of internals from clojure.spec and doesn't currently build on cljs. i get No such var: s/def-imp. can you (or anybody else here) confirm that?#2017-04-2906:51fossifooeh, there's a cljs.spec.cljc and a cljs.spec.cljs? how does that work?#2017-04-2911:40Alex Miller (Clojure team)Platform-specific is used first, then falls back to cljc if not found#2017-04-3005:50lincpaMake readability better spec will expressed in the form of SQL DDL?#2017-04-3011:14fossifoohmmm. since ther e is a def-impl in cljs.spec, i guess this should work then#2017-04-3011:15fossifooand actually, i had to rename the imports to cljs.spec from clojure.spec although i think the compiler should find it with either name#2017-04-3011:15fossifooso maybe something else is worng#2017-05-0103:00Oliver GeorgeJust putting it out there: I love clojure.spec#2017-05-0115:47souenzzoolivergeorge: clojure.spec.alpha šŸ˜‰#2017-05-0210:27Oliver GeorgeFair call. I had just finished watching Rich's spec-ulation talk before it happened. Makes total sense in that light.#2017-05-0109:37thhellerhttps://gist.github.com/thheller/738698dfff45280f4e004df1c46af4ba#2017-05-0109:38thhellerI think we should maybe add a #(even? (count bindings)) predicate to the clojure.core/let and other specs#2017-05-0109:39thhellerI certainly couldn't make sense of that error for a while šŸ˜›#2017-05-0113:08gfrederickstest.check and clojure.spec have made it onto neural network garbage twitter https://mobile.twitter.com/BobEbooks/status/859025622689087488#2017-05-0117:55ikitommi@fossifoo spec-tools tests are run on travis with node too, so it should work, tested with cljs-version 1.9.518. But to enable specs to be created a runtime, data-specs uses the functional internal of clojure.spec (the ^:skip-wiki fns :() => just extracted ā€˜em into separate namespace - donā€™t have to use those. Spec-swagger needs a lot of attention, next on the todo-list, will start by converting to vanilla specs.#2017-05-0117:57ikitommiIf I remember right, there will be public *functions* to create specs some time in the future.#2017-05-0118:03ikitommiis there a way to unform a conformed value of s/map-of?#2017-05-0118:05ikitommi
(s/def ::a (s/or :int int?))

(s/conform ::a 1)
; [:int 1]

(s/unform ::a (s/conform ::a 1))
; 1

(s/def ::map-of (s/map-of ::a ::a :conform-keys true))

(s/conform ::map-of {1 1})
; {[:int 1] [:int 1]}

(s/unform ::map-of (s/conform ::map-of {1 1}))
; {[:int 1] [:int 1]}
#2017-05-0118:13Alex Miller (Clojure team)known bug https://dev.clojure.org/jira/browse/CLJ-2076#2017-05-0118:29ikitommithanks. is it still a plan to create a public layer of functions to create the specs (besides the macros)?#2017-05-0118:30ikitommialso, would be interested in having a functional way to register specs. the def-impl looks private as it has the :skip-wiki meta on it.#2017-05-0118:30ikitommi(and ā€œDo not call this directlyā€ā€¦)#2017-05-0118:43fossifoo@ikitommi okay, just wanted to check the status. i want to go the other way around (use swagger json to generate testdata) and think i might go via spec. a generic spec will also be very helpful for parsing. changing it to standard spec would indeed be better for me. it's hard enough to understand one DSL#2017-05-0119:02Alex Miller (Clojure team)@ikitommi still tbd. As I've said in the past, you can get same benefits of functional API by unforming with spec specs#2017-05-0122:55danielcompton@thheller what was the form that gave that error? Here's what I got, I must be misunderstanding what you meant:
(let [a 1
      b 2
      c 3
      d]
  a)
clojure.lang.ExceptionInfo: Call to clojure.core/let did not conform to spec:
                            In: [0] val: () fails spec: :clojure.core.specs/bindings at: [:args :bindings :init-expr] predicate: any?,  Insufficient input
                            :clojure.spec/args  ([a 1 b 2 c 3 d] a)
#2017-05-0123:05mobileinkd needs a value.#2017-05-0123:17danielcomptonyep, I know, I'm trying to reproduce the error @thheller got#2017-05-0200:35jatkinsHi all! Is anyone else getting errors like Call to clojure.core/ns did not conform to spec? I'm working on a clj/s project and I want to use spec with reframe. I've tried the latest version of clojure (`1.9.0-alpha16`) and I get the aforementioned error. However, when I move the dependency to 1.9.0-alpha10 the error does not occur. Any idea for the change? BTW This is my first attempt at using a clojure alpha version, and I have many dependencies that may be incompatible (I feel that this is unlikely in this case, though correct me if I'm wrong). If needed I can provide more information.#2017-05-0200:41Alex Miller (Clojure team)The namespaces for spec just changed in alpha16 and cljs and other deps are not compatible with it#2017-05-0200:42Alex Miller (Clojure team)You might want to fall back to alpha15#2017-05-0200:43jatkinsI actually tried every alpha from 16 to 10, and 10 was the first one that worked.#2017-05-0200:47Alex Miller (Clojure team)What's the actual error you're getting?#2017-05-0200:48Alex Miller (Clojure team)Many libs had bad ns deps - most of those have been fixed and newer versions exist#2017-05-0200:52jatkinsI put a gist up: https://gist.github.com/JJ-Atkinson/a1d19e48d9440c2ca9f0bed219c8201f. How can I tell where the error is originating from? I see that it likely is from (onelog/core.clj:1:1), but I don't know where that is, (probably in timbre though, I'll see if I can bump that version up).#2017-05-0201:05jatkinsOnelog is part of ring apparently part of ring-middleware-logger. I'm currently using the latest of both. So, is the issue not with clojure itself, but with onelog? If that's the case, I'll see about a pull request/issue to them.#2017-05-0201:27jatkinsSo, after crawling through the error message more thoroughly, it appears that onelog is using the syntax (ns ... (:import (com.something SomeClass))). Spec changed something about ns with -alpha11, and it was already reported to them https://github.com/pjlegato/onelog/issues/3. Thanks for the help!#2017-05-0202:40Drew VerleeApologies as this has probably been asked a dozen times, but does clojurescript current support generators?#2017-05-0203:38tbaldridge@drewverlee not that I know of, but generators tend to encourage a impure code. Most of what you would need generators for can be done with more functionally pure constructs like map, filter and for.#2017-05-0206:34curlyfry@alexmiller So if I want to use alpha16 and have cljs in my project I should wait for a compatibility patch or something similar?#2017-05-0206:53dergutemoritz@tbaldridge I think @drewverlee is referring to the test.check kind of generators, given that he's asking in here šŸ™‚ @drewverlee test.check supports cljs at least, not sure if clojure.spec's wrapping of it works with cljs, though.#2017-05-0206:58thheller@danielcompton this was the source https://gist.github.com/thheller/738698dfff45280f4e004df1c46af4ba#file-problem-clj#2017-05-0211:07lwhortonWith the announcement of spec moving to its own lib, I see that spec.alpha is under org.clojure, but nothing like org.clojure/clojurescript.spec.alpha? Has it not moved yet?#2017-05-0212:08Alex Miller (Clojure team)Not yet#2017-05-0212:43lwhortonalrighty, thanks#2017-05-0221:55weiwhatā€™s a good example of adding validation on a macro?#2017-05-0222:07mobileinkwhat do you want to validate?#2017-05-0222:22weiactually, might need to rethink this#2017-05-0222:25weiI have a wrap-error macro that takes a map of context data and a body.
(defmacro wrap-error [args & body]
  (let [e (gensym 'e)]
    `(try 
I want to make sure that the macro user doesnā€™t forget to pass in the context map, because if so, the first line of the body wouldnā€™t get executed. but Iā€™m not sure itā€™s possible to do that at compile time
#2017-05-0222:39mobileinkwhat context map? #2017-05-0222:39mobileinkyou mean you want to validate args?#2017-05-0222:41mobileinkno way to do this without a macro?#2017-05-0222:45mobileinkyou want to use spec to validate args? add a clause to your let? #2017-05-0223:07weisorry let me try a better explanation. I want people to use this macro like this:
(wrap-error {:some :data :a 1 :b 2} ...body...)
where the first argument is some metadata to associate with the error, if there is one. the macro has been used mistakenly like this:
(wrap-error ...body...)
without the first arg, resulting in incorrect behavior because the first element of body is interpreted as the metadata and not executed. was wondering if there was a way to use spec to prevent this at read-time. less about validating the metadata itself so much as the presence of it.
#2017-05-0223:09weimight be hard to distinguish at read-time though. i wonder if this has to be a run-time check (`(s/valid? map? args)`)#2017-05-0300:01mobileinkwhy do you need a macro? you're just wrapping try/catch, no? #2017-05-0300:02mobileinki'm not an expert, but the usual advice is to avoid macros.#2017-05-0300:05mobileink"wrap-error" strikes me as misleading. you're really just wrapping a fn, which may not have an error, no?#2017-05-0301:57cflemingThe doc seems to imply that s/alt is ordered choice, i.e. the alternatives will be tested in order and the first match returned. Is that correct?#2017-05-0302:00tbaldridge@cfleming yes from the doc string: "Returns a regex op that returns a map entry containing the key of the first matching pred and the corresponding value. "#2017-05-0302:00cflemingGreat, thanks.#2017-05-0308:27mishagreetings! I am getting
(s/exercise :dsc/patch 1)
=> ExceptionInfo Couldn't satisfy such-that predicate after 100 tries.  clojure.core/ex-info (core.clj:4725)
is there a way to narrow error location down? (:dsc/patch is a compound spec, ~30 lines long)
#2017-05-0309:24slipsetI really wished spec was spec'ed. Just lost som time on why s/keys was valid on my invalid maps. Reason? I used :un-req instead of :req-un#2017-05-0311:04mishais there something built in for what I'am trying to achieve (generator for next-id, which never returns a duplicate)?#2017-05-0311:55mishahow does one combine two map specs? (union)#2017-05-0311:58mishahere I'd like to avoid custom reduce-through-hashmap predicate, and combine s/keys and s/map-of specs instead#2017-05-0312:35mishathat one fails because s/and results in empty set of valid keys. s/merge will result in an empty set too. it seems like I need a custom predicate after all:
(s/def :datascript/map-with-current-tx (s/keys :req [:db/current-tx]))
(s/def :datascript/map-with-tempids-lookup (s/map-of :datascript/temp-id :datascript/actual-id))

(def gen-tempids
  (tgen/let [ids (s/gen :datascript/map-with-tempids-lookup)
             ctx (s/gen :datascript/map-with-current-tx)]
    (merge ids ctx)))

(s/def :datascript/tempids
  (s/with-gen
    (fn [m]
      (and
        (s/valid? :datascript/map-with-current-tx m)
        (s/valid? :datascript/map-with-tempids-lookup (dissoc m :db/current-tx))))
    (constantly gen-tempids)))
#2017-05-0312:04souenzzo(s/merge ::a ::b (s/keys :req[::c]) (s/map-of ,,,))#2017-05-0312:06misha@souenzzo tried that right now. it also gives me
ExceptionInfo Couldn't satisfy such-that predicate after 100 tries.  clojure.core/ex-info (core.clj:4725)
#2017-05-0312:07souenzzoI've never used generators. =/ But I use tons of s/merge#2017-05-0312:14lwhortonhas anyone been able to use generators with cljs? Iā€™m not really sure how (I donā€™t even see gen defined in the cljs.spec.impl.gen ns, and my google foo is failing#2017-05-0312:26bhagany@lwhorton yes, but I don't remember off the top of my head what the differences with clj were...#2017-05-0312:26bhaganyhttps://github.com/bhagany/snowth/blob/master/src/snowth/satellites.cljs#2017-05-0312:26bhaganythere's an example, and there's more in that project if you'd like to click around#2017-05-0312:27bhaganytl;dr - looks like I import the clojure.test.check.generators ns, and use that in conjunction with clojure.spec/gen#2017-05-0312:29lwhortonthanks @bhagany ill take a look#2017-05-0312:29bhagany@lwhorton: sure, np. perhaps better examples here: https://github.com/bhagany/snowth/blob/master/src/snowth/astro.cljs#2017-05-0315:46chilleniousHi. Trying out spec and running into a problem right away. Simply adding [clojure.spec.alpha :as s] to a :require section in my clojure file, and then trying to load that file in the repl (Leiningen) via use gives me this error: java.lang.ClassCastException: clojure.spec.alpha$regex_spec_impl$reify__1340 cannot be cast to clojure.lang.IFn When I use require instead of use, it seems fine. Anything I'm doing wrong/ known bug?#2017-05-0316:09chilleniousactually, scrap that... I get this error with either use or require whenever I pass in the :reload-all flag#2017-05-0316:24seancorfieldYes, thereā€™s an AOT-related issue at present with the version of spec.alpha that Clojure 1.9.0 Alpha 16 pulls in. If you explicitly depend on org.clojure/spec.alpha "0.1.108" the problem should go away.#2017-05-0316:24seancorfieldIt specifically seems to affect REPL workflows and anything that reloads code.#2017-05-0316:28chilleniousthanks!#2017-05-0319:08slipsetIā€™m writing specs for my om components, which basically are vararg functions of which I normally just care about the first arg.#2017-05-0319:08slipsetSo I end up with something like#2017-05-0319:08slipset
(s/fdef content-area :args (s/cat :args ::grid :rest (s/* any?)))
#2017-05-0319:09slipsetthe :rest (s/* any?) bit seems a bit silly.#2017-05-0401:03Alex Miller (Clojure team)Why?#2017-05-0407:22slipsetSilly might be wrong, but tedious maybe?#2017-05-0407:25slipsetwhat Iā€™d like to be able to state is ā€œI only care about the first argument and it should be a grid ā€#2017-05-0407:54dergutemoritz@slipset That's what you're stating with that spec, though, isn't it? šŸ˜„ If you feel it's too noisy and/or you're going to use it in many places, you could define a macro that hides that noise.#2017-05-0407:56slipset@dergutemoritz true#2017-05-0407:57slipsetapart from that, it seems like spec is giving me exactly what I want: the possibility to sprinkle ā€œtypesā€ over my program when itā€™s ā€œdoneā€.#2017-05-0421:41joshjonesNamespace myproj/myns.clj contains a function, which requires specs defined in myproj/specs.clj. I have the function fdef'd and it currently sits next to the function itself in myns. However, I'd like to move the function spec away from the function itself. So myns depends on specs, but the fdef'd function depends on knowing about the function itself in myns, so I can't move it to specs. Circular dependency. Where are you putting your spec'd functions (or in general, any of your specs) in your project?#2017-05-0422:07ghadithe specced namespace IMHO should not require the specs#2017-05-0422:08ghadiit was explicit design of clojure.spec that you can spec things you don't own#2017-05-0422:08misha
(s/fdef myns/my-fn ...)
without explicitly importing myns in specs, @joshjones
#2017-05-0422:18mobileinkmisha: is that missing a colon?#2017-05-0422:21mishano, its a fully qualified fn name, like clojure.string/blank?#2017-05-0422:24mobileinkmisha: so you have to require that ns in your spec file? or otherwise define it there? that seems odd, but i'm a relative noob at spec.#2017-05-0422:27mishano, requireing needed. afair, spec just uses fn symbol to register spec, and does not call actual fn, so the actual require'ing is not needed#2017-05-0422:08ghadi^#2017-05-0422:12joshjonesmany thanks to you both -- @ghadi , by "the specced namespace" you mean myns or specs in my example above?#2017-05-0422:12ghadimyns#2017-05-0422:12ghadiyourproj.specs should depend on yourproj.myns, not the other way around, like @misha 's example#2017-05-0422:12mishayeah, myns might not need specs imported unless it uses one for explicit validation as a workflow step#2017-05-0422:13joshjonesit does -- (s/assert ::specs/myspec some-data)#2017-05-0422:14joshjonesMany examples of using valid?, conform, etc also#2017-05-0422:15mishathen either keep fdef's closer to functions, or use fully qualified fn names in fdefs w/o explicit imports#2017-05-0422:16mobileinkyou only need to require if you want to use an alias, is that correct?#2017-05-0422:25mishamobileink: I'd say: only if you need to call something from that ns, or import for side effects. alias works w/o explicit import (at least it worked for me just now)#2017-05-0422:28mobileinkhmm, need to experiment more. i could swear i've used :foo.bar/baz without require or import, but theres a goid chance i misunderstood what i was doing. ;)#2017-05-0422:30misha2 types of specs: for data ā€“ keyword (:foo/bar), for functions ā€“ symbol (foo/baz)#2017-05-0422:31mobileinki didn't think you could alias an ns without "making" it whether by require or sth else. i.e. just a symbol won't work. but i'm away from my machine, will try later.#2017-05-0422:31mobileinkaha, my bad - have not yet worked my way up to fn specs. sorry!#2017-05-0422:32mishaI am trying now, and I see no spec ns import required either: I can define specs for functions in foo.specs w/o importing foo.fns, and can use specs for assertions in foo.fns w/o importing foo.specs#2017-05-0422:34mobileinktime for me to take another sip from the firehose. not so easy when your lips have already been ripped off!#2017-05-0422:34mishatwo namespaces, neither imports the other. instrumentation and validation work, because specs get registered in global registry#2017-05-0422:35mishacc @joshjones#2017-05-0422:36mobileinki think i assumed reg keys are always kws. bad programmer!#2017-05-0422:37mishaI have spec guide open at all times while doing spec development, for quick reference https://clojure.org/guides/spec#2017-05-0422:39mobileinkme too - just haven't needed to deal with fns yet. it's hard enough to figure out how to do what i need with maps!#2017-05-0422:40mishathe only require use case I can think of now ā€“ is google closure compiler advanced optimization, where specs might get dead-code-eliminated. Haven't tried that yet though#2017-05-0422:41mobileinkfwiw i just got my clj map specs working in cljs, using cljc code. works like a charm - but haven't done advanced opt yet. shivers#2017-05-0422:45joshjonesthanks @misha, very helpful#2017-05-0510:27slipsetI seem to remember that the fdef :ret is not enforced/checked under instrument. Is that correct, and if so, how does one go about checking that the return value of a fn conforms to the spec?#2017-05-0512:52slipsetJust to answer myself: https://groups.google.com/forum/#!topic/clojure-dev/4W4PO5Di9WU#2017-05-0513:18slipsetThe reason I was wondering (and wanting this) is that in cljs, I run my dev-setup with (stest/instrument) which is super nice, but it would be nice to get the :ret bit as well while running in dev-mode#2017-05-0514:15joshjones@slipset The answer you found requires an additional binding in many cases, and will generally mess up the flow of the function. I'm in the same boat as you, looking for a way to automatically check the return, with the ability to disable it for production. Not having the ability to test :ret except through stest makes a function spec much less useful. something like this works at least on a small example:
(defn foo [x y]
  {:pre  [(s/assert (:args (s/get-spec `foo)) [x y])]
   :post [(s/assert (:ret (s/get-spec `foo)) %)]}
  (+ x y))
#2017-05-0514:18joshjonesit can be turned on/off with s/check-asserts, so it may be a decent workaround to this limitation (it may be by design, but it is still a limitation)#2017-05-0518:18onetomas a https://github.com/Yuppiechef/datomic-schema user, who wrote a few DSLs in Rebol, I was inspired by the data model example in this article: http://blog.cognitect.com/blog/2017/4/6/developing-the-language-of-the-domain since there was no example implementation which could parse this:
(attr :university/full-name string non-blank unique
    "Fully-expanded name of the university for public display")
I set out to implement it with clojure.spec, so I can do:
(->> '[[:person/email one unique str "Email"]
       [:person/org many ref "Orgs"]]
     (mapv (partial s/conform ::attr))
     clojure.pprint/pprint)
#2017-05-0518:19onetomhere is my implementation:
(defn with-ns [ns kw]
  (keyword ns (name kw)))

(defn conform-with-ns [ns]
  (s/conformer (partial with-ns ns)
               (comp symbol name)))

(defn type-aliases [t]
  ('{str string} t t))

(s/def ::attr
  (s/coll-of
    (s/or :db/doc string?
          :db/cardinality (s/and '#{one many}
                                 (conform-with-ns "db.cardinality"))
          :db/valueType (s/and '#{str string int ref}
                               (s/conformer type-aliases)
                               (conform-with-ns "db.valueType"))
          :db/unique (s/and #{'unique}
                            (conform-with-ns "db.unique"))
          :db/ident keyword?)
    :into {}))
#2017-05-0518:20onetomplease share your thoughts on it#2017-05-0518:58dnolenI just changed cljs.spec to cljs.spec.alpha now would be a good time to try master - would like to cut a release for this#2017-05-0602:10jimmydnolen: yay šŸ˜„#2017-05-0619:23reefersleepWhen using spec/explain on a spec which contains spec/coll-of in cljs and passing in an invalid datastructure, I don't seem to be getting any information about the offending element of the coll, only that something is wrong within the coll. Example:
(spec/explain (spec/coll-of string? []) ["hey" "yo" 9])
val: ["hey" "yo" 9] fails predicate: (coll-checker string?)
For contrast, here is the ouput for a Schema validation similar to the spec explain:
(schema/validate [schema/Str] ["hej" "yo" 9])
#error {:message "Value does not match schema: [nil nil (not (cljs$core$string? 9))]", :data {:type :schema.core/error, :schema [
Can I do something to make Spec give me more detailed output, similar to that of Schema? In my actual use case, the spec/coll-of contains more complex maps with nested specs, and if one thing goes wrong somewhere in the coll-of, I'm left in the dark, fumbling for the specifics.
#2017-05-0619:23reefersleepWhen using spec/explain on a spec which contains spec/coll-of in cljs and passing in an invalid datastructure, I don't seem to be getting any information about the offending element of the coll, only that something is wrong within the coll. Example:
(spec/explain (spec/coll-of string? []) ["hey" "yo" 9])
val: ["hey" "yo" 9] fails predicate: (coll-checker string?)
For contrast, here is the ouput for a Schema validation similar to the spec explain:
(schema/validate [schema/Str] ["hej" "yo" 9])
#error {:message "Value does not match schema: [nil nil (not (cljs$core$string? 9))]", :data {:type :schema.core/error, :schema [
Can I do something to make Spec give me more detailed output, similar to that of Schema? In my actual use case, the spec/coll-of contains more complex maps with nested specs, and if one thing goes wrong somewhere in the coll-of, I'm left in the dark, fumbling for the specifics.
#2017-05-0720:40mobileinkreefersleep: it tells you precisely that your val contains a non-string.#2017-05-0720:42mobileinkreefersleep: do you have an example of being left in the dark?#2017-05-0814:07reefersleep@U0LGCREMU: here's a more elaborate example of what I mean in regards to Schema providing more information than Spec, leaving me desiring more from Spec. Spec:
(spec/def ::id int?)

(spec/def ::name string?)

(spec/def ::person (s/keys :req-un [::id
                                    ::name]))

(spec/explain (spec/coll-of ::person []) [{:id 1 :name "john"} {:id 2 :name :heather}])
val: [{:id 1, :name "john"} {:id 2, :name :heather}] fails predicate: (coll-checker :sleepydo.db/person)

;; No information about _which_ element of the coll failed to validate, nor details about how it failed to validate
Schema:
(def Id schema/Int)

(def Name schema/Str)

(def Person {:id Id
             :name Name})

(schema/validate [Person] [{:id 1 :name "john"} {:id 2 :name :heather}])
#error {:message "Value does not match schema: [nil {:name (not (cljs$core$string? :heather))}]", :data {:type :schema.core/error, :schema [{:id 
#2017-05-0620:21xiongtxDoes s/with-gen take a thunk that returns a generator for laziness? Is that way it doesnā€™t just take a generator directly?#2017-05-0719:08reefersleepIs there an easy way to refer to a registered spec under a different name in a map spec?#2017-05-0723:09taylor@U0AQ3HP9U if I understand your question, I don't think there is and I also think that's by design#2017-05-0723:11Alex Miller (Clojure team)no, other than by using :req-un for a spec with the same name but different namespace in s/keys#2017-05-0813:43reefersleepThank you @U3DAE8HMG and @U064X3EF3 for your succinct answers!#2017-05-0722:12caiois there an easy way of defining spec for something inside a ref?#2017-05-0722:12caioI'm trying to write a spec for funcool cats' either monad, but no luck#2017-05-0722:15caioI was able to write something to conform/unform, but got stuck when trying to write a proper generator#2017-05-0723:11Alex Miller (Clojure team)@caio not yet#2017-05-0723:12Alex Miller (Clojure team)possibly will be added in the future#2017-05-0801:11caiowhat about a generator for records? can that be done?#2017-05-0801:38caiohttps://github.com/clojure/test.check/blob/master/src/main/clojure/clojure/test/check/generators.cljc#L91 one could use this to write the generator for refs, right? (gen/fmap deref (s/gen internal-spec))#2017-05-0801:42caioalso, I think my problem is a bit more generic than writing specs for refs. I was trying to write a spec for a type defined in cats. after seeing gen/fmap, looks doable#2017-05-0801:50gfredericksyes technically that will work#2017-05-0801:51gfredericksnormal test.check usage is aimed at writing generators for data, not stateful objects#2017-05-0801:51gfredericksI'm not sure what will happen if spec starts having specs for reference types#2017-05-0801:55bhagany@caio I made a generator for reified protocols, which are similar enough to records https://github.com/bhagany/snowth/blob/master/src/snowth/satellites.cljs#L215#2017-05-0801:58bhaganyI hadn't learned about gen/let when I wrote this, fyi. I'd probably use it now#2017-05-0801:59caioyeah, I did something like that (though my record was way simpler) https://github.com/funcool/cats/pull/201#2017-05-0801:59caio@gfredericks this is not stateful. IDeref was implemented as a syntatic sugar#2017-05-0802:00gfredericks@caio okay that's not bad then#2017-05-0802:00gfredericks@bhagany btw the bind+return thing there can be simplified to fmap#2017-05-0802:00bhagany@gfredericks thanks šŸ™‚#2017-05-0807:22mpenetany reason why (s/assert x) compiles to x instead of nothing (like core/assert) when s/*compile-asserts* is false?#2017-05-0808:12NiclasIs there any way of specing async channels and the data that may be sent over them?#2017-05-0808:30misha@looveh wrap >! with validating fn? you can put anything on a channel, so any limitation would need to be enforced by client code, I think.#2017-05-0808:32Niclas@misha Thatā€™s one way to do it, but it would also be useful to be able to spec channels as function arguments#2017-05-0808:32NiclasNot just the actual get/put on channels#2017-05-0808:32mishawrapping >! would bail right away. wrapping <! would bail sometime later/never. Whichever would suite you best.#2017-05-0808:34mishawrap channel in a record, spec record, spec protocols it implements?#2017-05-0808:35misha*with multispec.#2017-05-0808:35NiclasYeah that would work, feels a bit hacky though imo since channels are part of a core lib#2017-05-0808:36mishaI'd imagine you need to add something to the channel-thing itself to be able to use different specs for different channels.#2017-05-0808:37mishaso either spec entry/exit points, or wrap, and spec/pass around/expect wrapped channels.#2017-05-0808:42mishaĀÆ\(惄)/ĀÆ#2017-05-0808:44NiclasHahah thatā€™s amazing!#2017-05-0813:36tbaldridge@misha @looveh specs could be put on a filter transducer on the channel, or a call to map that throws. On an exception the exception would be handed to the channels' exception handler.#2017-05-0814:06misha@tbaldridge I think @looveh's intention was to "spec a function, so that wrong kind of channel passed as an argument would throw", not "make sure channel would not accept random stuff". I like "spec as a filter" much more than macro above, thank you.#2017-05-0814:07misha@tbaldridge can that channel's filter be used as a dispatch value once channel is constructed? is it accessible from outside?#2017-05-0814:09tbaldridge@reefersleep what does explain-data show for the spec code?#2017-05-0814:11reefersleep@tbaldridge: Nothing more.
(spec/explain-data (spec/coll-of ::person []) [{:id 1 :name "john"} {:id 2 :name :heather}])
{:cljs.spec/problems {[] {:pred (coll-checker :sleepydo.db/person), :val [{:id 1, :name "john"} {:id 2, :name :heather}], :via [], :in []}}}
#2017-05-0814:12reefersleepNote that I'm talking about cljs, not clj. šŸ™‚#2017-05-0814:16misha@reefersleep why is there [] in (spec/coll-of ::person [])?#2017-05-0814:17mishafor vector it is (spec/coll-of ::person :kind vector?), isn't it?#2017-05-0814:20mishaclojure gives me
(s/explain-data (s/coll-of string? []) ["a" "b" 1])
CompilerException java.lang.IllegalArgumentException: No value supplied for key: []
and in cljs (js) I can imagine it fails silently somewhere, but produces compromised output anyway
#2017-05-0815:46reefersleepI thought I should use :kind vector? as well, until I tried it.
(spec/explain-data (spec/coll-of ::person :kind vector?) [{:id 1 :name "john"} {:id 2 :name :heather}])
----  Could not Analyze  <cljs form>   line:1  column:20  ----

  Wrong number of args (3) passed to: spec/coll-of

  1  (spec/explain-data (spec/coll-of ::person :kind vector?) [{:id 1 :name "john"} {:id 2 :name :heather}])
                        ^--- 

----  Analysis Error  ----
Then I tried without any :kind-indication:
(spec/explain-data (spec/coll-of ::person) [{:id 1 :name "john"} {:id 2 :name :heather}])
----  Could not Analyze  <cljs form>   line:1  column:20  ----

  Wrong number of args (1) passed to: spec/coll-of

  1  (spec/explain-data (spec/coll-of ::person) [{:id 1 :name "john"} {:id 2 :name :heather}])
                        ^--- 

----  Analysis Error  ----
So I looked up the documentation:
(cljs.repl/doc spec/coll-of)
-------------------------
cljs.spec/coll-of
([pred init-coll])
Macro
  Returns a spec for a collection of items satisfying pred. The generator will fill an empty init-coll.
#2017-05-0814:26Alex Miller (Clojure team)if you remove that invalid trailing [], clojure says:
user=> (spec/explain (spec/coll-of ::person) [{:id 1 :name "john"} {:id 2 :name :heather}])
In: [1 :name] val: :heather fails spec: :user/name at: [:name] predicate: string?
#2017-05-0815:50reefersleepalexmiller: So I guess what I'm seeing is a feature of clj spec missing (as of now) from the cljs port. I kind of suspected this, as I thought I'd seen more useful output in guides/tutorials at the time spec came out.#2017-05-0814:28Alex Miller (Clojure team)which seems better than the schema message to me#2017-05-0814:50andrewboltachevHello. When I have keys #{:a :b :c} and a spec ::foo, how do I get
{:a (gen/genereate (s/gen ::foo))
 :b (gen/genereate (s/gen ::foo))
 :c (gen/genereate (s/gen ::foo))}
generated for me? (edited) Now with (s/map-of #{:a :b :c} ::foo) I've got only some keys, not all three used
#2017-05-0814:53mishas/keys#2017-05-0814:53andrewboltachevhuh#2017-05-0814:53andrewboltachevso simple šŸ˜„#2017-05-0814:55andrewboltachev@misha but I think s/keys takes both value type and key names from it's arguments#2017-05-0814:55andrewboltachev(and I have these different)#2017-05-0814:56andrewboltachevi.e. N keys and single value type#2017-05-0814:56andrewboltachev@misha thanks for answering btw šŸ™‚#2017-05-0814:56misha
(s/def :foo/bar integer?)
(s/def :my/a :foo/bar)
(s/def :my/b :foo/bar)
(s/def :my/c :foo/bar)
(s/def :my/map (s/keys :req-un [:my/a :my/b :my/c]))

=> :my/map
(s/exercise :my/map 2)
=>
([{:a -1, :b 0, :c 0} {:a -1, :b 0, :c 0}]
 [{:a -1, :b -1, :c -1} {:a -1, :b -1, :c -1}])
#2017-05-0814:57andrewboltachevsure, but I'll need to repeat for 4 or 5 different types of ::foo#2017-05-0814:58mishaafaik, you have to alias desired keys to their desired values' specs#2017-05-0814:59mishaor provide your own generator, which would make sure all the keys are present. (I'd just alias things, custom generators might screw up s/explain-data error address within the structure)#2017-05-0815:00andrewboltachevyep the custom generators are obviously "approach from the opposite side"#2017-05-0815:01andrewboltachevand well, honestly also having N keys as I do might mean semantically the sequence, not a map#2017-05-0815:02mishaplenty of options there as well: s/coll-of, s/tuple#2017-05-0815:02andrewboltachevyep, would take a look at every function#2017-05-0815:55Alex Miller (Clojure team)I think you must be using an older version of ClojureScript - thatā€™s not what I see with the latest#2017-05-0815:55Alex Miller (Clojure team)afaik, ClojureScript is basically at parity with the Clojure version wrt spec#2017-05-0815:56Alex Miller (Clojure team)
cljs.user=> (s/explain-data (s/coll-of ::person :kind vector?) '({:id 1 :name "John"}))
{:cljs.spec/problems [{:path [], :pred vector?, :val ({:name "John", :id 1}), :via [], :in []}]}
cljs.user=> (s/conform (s/coll-of ::person :kind vector?) [{:id 1 :name "John"}])
[{:name "John", :id 1}]
#2017-05-0815:57Alex Miller (Clojure team)Iā€™m using ClojureScript 1.9.293 btw#2017-05-0816:05reefersleepI thought I was up to date, but apparently, I misread the version number... 1.9.93. šŸ™‚#2017-05-0816:07reefersleepI'll try updating the version, I'm sure that'll make things better for me!#2017-05-0816:07reefersleepThanks all!#2017-05-0816:29Alex Miller (Clojure team)whatā€™s 200 commits between friends? :)#2017-05-0816:37reefersleepNow I'm having a different problem, not spec-related - I'm trying to destruct with (let [{:keys [a b] :as full} (fn-call something...)] ...#2017-05-0816:41reefersleepI've not changed the code for that destructuring or what goes on underneath between cljs version changes. However, the output of (fn-call ...), which is a map {:a "a's value" :b "b's value"}, - I can tell by printing at the end of my fn-call - turns into {[:a "a's-value] [:b "b's value"} upon destruction (I can tell by printing full), so I cannot destruct`a` or b.#2017-05-0816:41reefersleepSo weird.#2017-05-0816:51reefersleep(asking in #clojurescript , too)#2017-05-0816:53caioabout specing a deftype, this is what I came up with in the end: https://gist.github.com/caioaao/d15260076bb8ee6f48908507ae73155b#2017-05-0816:55caiowould be nice if someone could validate if this is safe. I had to go through spec's code for doing this, and some stuff were not really clear to me#2017-05-0816:59caioalso saw something in spec-tools that they do some checking on conform that made me think it may not be thread-safe. idk if that's the case#2017-05-0817:00caiohttps://github.com/metosin/spec-tools/blob/master/src/spec_tools/core.cljc#L160 this#2017-05-0817:03caiooh, nvm. looks like they do this because they call conform* from inside explain*#2017-05-0817:13Alex Miller (Clojure team)What' is either supposed to do?#2017-05-0817:14Alex Miller (Clojure team)Is this just a non conforming or of two choices?#2017-05-0817:18caioit's the either monad, from funcool cats#2017-05-0817:19caiohttps://funcool.github.io/cats/latest/#either#2017-05-0817:27ikitommi@caio about spec-tools - if the conform doesnā€™t use threads, I should be ok. But not optimal. would be nice to use just 3rd parameter in conform (http://dev.clojure.org/jira/browse/CLJ-2116)#2017-05-0817:27caioYou can say it's a non conforming or between two choices If you want to push it (and probably piss someone šŸ˜† )#2017-05-0817:29ikitommigreat article, with the ā€œSafe Dynamic Scopeā€ chapter: https://stuartsierra.com/2013/03/29/perils-of-dynamic-scope.#2017-05-0818:00mobileinkIā€™m getting a strange error that seems related to the switch to spec.alpha. In my code this works: (clojure.core/require [pagespace-sym] :reload). But if I change it to :reload-all I get the following error:
java.lang.ClassCastException: clojure.spec.alpha$regex_spec_impl$reify__1340 cannot be cast to clojure.lang.IFn
clojure.lang.Compiler$CompilerException: java.lang.ClassCastException: clojure.spec.alpha$regex_spec_impl$reify__1340 cannot be cast to clojure.lang.IFn, compiling:(clojure/tools/logging.clj:1:1)
#2017-05-0819:48Alex Miller (Clojure team)This is a known issue with spec.alpha not being aot compiled#2017-05-0819:49Alex Miller (Clojure team)It's been fixed in a new build of spec.alpha - 0.1.108. We haven't released a new Clojure alpha yet that depends on it but you can specify that to override#2017-05-0820:03mobileink@alexmiller thanks!#2017-05-0908:42mpenetWould it make sense to have a ^:fdef (or ^:spec) meta on fn to do the :pre/:post automatically from a fdef ? Before you mention instrument, it's not the same (instrument triggers gen).#2017-05-0908:45mpenetthat would avoid some boilerplate :
(s/fdef ::foo
        :args (s/cat :a ::a
                     :b ::b
                     :c ::c))
(defn foo
  [a b c]
  {:pre [(s/assert (:args (s/get-spec ::foo))
                   [a b c])]
   :post [(s/assert (:ret (s/get-spec ::foo))
                   %)]}
  42)
vs
(s/fdef ::foo
        :args (s/cat :a ::a
                     :b ::b
                     :c ::c))
(defn foo ^:fdef ::foo
  [a b c]
  42)
#2017-05-0908:51mpenetthe spec and where it's applied is still decoupled, the latter would basically expand into the former (almost, omited the potential :fn part of the spec)#2017-05-0908:55Alex Miller (Clojure team)no, not interested in that. first, this re-couples vars and function specs which are intentionally not coupled (and can be created in either order). second, we donā€™t generally want to enable pre/post checking of specs for performance reasons.#2017-05-0908:55mpenet(it doesn't have to use :pre/:post, would open doors to auto conforming via metadata hint etc)#2017-05-0908:56Alex Miller (Clojure team)I donā€™t think Rich has any interest in that#2017-05-0908:58mpenetalright, not surprised, I guess this might end up being done as part of a lib#2017-05-0908:59mpenetpre/post is toggleable (s/assert as well), but again this would/should be done without these 2 anyway#2017-05-0909:04mpenet@alexmiller about ordering I am not sure I follow, if that expands to s/assert calls it wouldn't matter/cause any issue#2017-05-0909:10Alex Miller (Clojure team)pre/post effects what ends up in the compiled code (even if toggled off) and thus has a runtime cost#2017-05-0909:10Alex Miller (Clojure team)s/assert can actually be omitted from compilation entirely using s/*compile-asserts* etc#2017-05-0909:11mpenetok, so lets say it's implemented without pre/post šŸ™‚, just with s/assert#2017-05-0909:11Alex Miller (Clojure team)re ordering, right now you create the specs before the functions or after the functions#2017-05-0909:11Alex Miller (Clojure team)by having defn rely on the spec, youā€™ve changed that#2017-05-0909:11Alex Miller (Clojure team)if youā€™re compiling#2017-05-0909:12mpenet
(s/fdef ::foo-like ...)

(defn foo [x y]
  (s/assert ::foo-like [x y])
  ...
  )
#2017-05-0909:13mpenetit would be the same as the above , you can just def ::foo-like after as well#2017-05-0909:13mpenetor I am missing something#2017-05-0909:13Alex Miller (Clojure team)I donā€™t know, maybe that would work#2017-05-0909:13Alex Miller (Clojure team)aside: the ::foo-like there would be `foo#2017-05-0909:14mpenetI don't follow#2017-05-0909:14Alex Miller (Clojure team)function specs are named by symbol, not keyword#2017-05-0909:14mpenetright!#2017-05-0909:14mpenetmissed that one#2017-05-0909:15Alex Miller (Clojure team)generally, we donā€™t think you should be doing this though#2017-05-0909:17mpenetthat would be handy to be able to reuse specs defined for instrument on live (staging/test) systems without paying the gen cost when that matters#2017-05-0909:18Alex Miller (Clojure team)there is no ā€œgen costā€?#2017-05-0909:18mpenethmm if I recall instrument does trigger gen in some cases#2017-05-0909:19Alex Miller (Clojure team)the only case like this is with fspec#2017-05-0909:20Alex Miller (Clojure team)if thatā€™s a problem for you, then donā€™t use fspec#2017-05-0909:20Alex Miller (Clojure team)and just spec as ifn?#2017-05-0909:21mpenetwell ... first class functions are quite common in clj code, but yes that's an option#2017-05-0911:23lambderhello all I got the following problem:
clojure

(s/def ::s string?)

(s/def ::g
  (s/cat :string ::s
         :g (s/* ::g)))

(s/conform ::g '["abc" "abc"]) ; --> ok;

(s/def ::g2
  (s/cat :string (s/? ::s)
         :g (s/* ::g2)))

(s/conform ::g2 '["abc" "abc"]) ; --> java.lang.StackOverflowError
any ideas?
#2017-05-0911:27dergutemoritz@lambder That recursion will never terminate because (s/? ::s) may match nothing so once the two "abc" strings are matched, it will recur forever on the remaining sequence []#2017-05-0911:32lambder@dergutemoritz how do I much the following grammar then?
S -> T R | R
T -> :tag
R -> :foo | :foo R
#2017-05-0911:41dergutemoritz@lambder That | in there is probably s/or#2017-05-0911:41dergutemoritzErr sorry, it's s/alt in regex#2017-05-0911:42dergutemoritzEverything else is s/cat#2017-05-0911:42dergutemoritzJust an educated guess, of course, not sure what grammar notation you're using exactly there#2017-05-0911:43lambderBNF#2017-05-0911:43dergutemoritzOK then yeah, that should work##2017-05-0911:43dergutemoritzHm well it's not really BNF though, that doesn't use -> at least#2017-05-0911:44lambder@dergutemoritz if you wish, please answer it on https://stackoverflow.com/questions/43868064/clojure-spec-conform-throws-stack-overflow-exception youā€™ll get my upvote#2017-05-0911:44dergutemoritzDon't have an SO account#2017-05-0911:45lambderok#2017-05-0911:46lambder@dergutemoritz many thanks#2017-05-0911:46dergutemoritzYW, hope it works out!#2017-05-0912:05vandr0iysuppose I got a spec, (s/def ::foo (s/keys :req-un [::a ::b] :opt-un [::c ::d])) and a data structure, (def bar {:a 1 :c 3 :d 4 :e 5}). How do I remove from the bar all the key-value pairs not explicitly mentioned in the spec and fill in with nils the data that's missing (so, :b nil and remove :e)?#2017-05-0912:24Alex Miller (Clojure team)spec doesn't do that#2017-05-0912:46lambderis it possible to preserve attached meta to structures which are spec conformed?#2017-05-0914:02lambderIā€™ve done this:
(in-ns 'clojure.spec)
(defn conform [spec x]
  (let [m (clojure.core/or (meta x) {})]
    (let [obj (conform* (specize spec) x)]
      (cond
        (instance? clojure.lang.MapEntry obj) (with-meta (into [] obj) m)
        (instance? clojure.lang.IObj obj) (with-meta obj m)
        :else obj))))
#2017-05-0914:03lambderitā€™s a bit hacky#2017-05-0915:49lambder@dergutemoritz I got better answer by @flowthing if you are interested. https://stackoverflow.com/a/43869208/135892 thanks to @alexmiller#2017-05-0915:51lambderIs it possible to ā€˜tapā€™ into a protocol method. Iā€™d like to have ā€˜aroundā€™ wrapper for all such method calls. e.g there is a protocol clojure.spec.Spec which has conform* method defined. Some of the conform* implementations call conform* recursively. Iā€™d like to wrap my logic around each such call.#2017-05-0915:51lambderI want to inject some code allowing to preserve meta of the parsed data-structures#2017-05-0915:52lambdermy monkey patch is :
(in-ns 'clojure.spec)
(defn conform [spec x]
  (let [m (clojure.core/or (meta x) {})]
    (let [obj (conform* (specize spec) x)]
      (cond
        (instance? clojure.lang.MapEntry obj) (with-meta (into [] obj) m)
        (instance? clojure.lang.IObj obj) (with-meta obj m)
        :else obj))))
#2017-05-0915:53lambderbut the call to conform is sometimes omitted as conform* is called instead#2017-05-0916:01Alex Miller (Clojure team)in short, no and anything touching these impl details stands a good chance of being broken later#2017-05-0916:02lambder@alexmiller yes, i think so too. Any idea why conform* implementations arenā€™t calling conform? performance?#2017-05-0916:42dergutemoritz@lambder Err I just realized that the grammar you pasted is not equivalent to what you were trying to express in spec - according to that, :tag is only allowed as the first element of the whole sequence and then only :foos are allowed (because R recurs on itself, not on S). Is that what you intended?#2017-05-0916:43lambdernot really. the grammar was simplification of what i needed#2017-05-0916:44lambderbut thanks to https://stackoverflow.com/a/43869208/135892 now I know how to express this kind of grammers in spec#2017-05-0916:48dergutemoritz@lambder AFAIUI that's more complicated than necessary. Doesn't something like (s/def ::g (s/+ (s/cat :tag (s/? ::tag) :val (s/alt :number ::n :string ::s)))) capture what you want to do much neater + give you a more convenient result structure to work with?#2017-05-0916:49lambderpossibly#2017-05-0916:49dergutemoritzMight be misinterpreting your use case, of course. Anyhow, gotta run. Cheers#2017-05-0916:59lambdercheers#2017-05-0921:18danielcomptonIs there a way to define newtype's in spec? e.g. specialise a Long to an AccountId or a ProductId. I'm pretty sure there's not, but thought I'd ask in case.#2017-05-0922:16ghadisay wha?#2017-05-0922:17ghadi@danielcompton seems like the spec name itself is that information#2017-05-0922:19danielcomptonLet's say I have a function that gets a product (get-product product-id). If the product-id and account-id are both Longs, nothing stops me from passing it (get-product account-id)#2017-05-0922:52caioI solve this by creating :account/minimal as (s/keys :req [:account/id] :opt [ all the other stuff]) when really necessary#2017-05-0922:53danielcomptonbut if you ever need to extract and pass around :account/id by itself, then you can't guarantee that that's what the functions are taking?#2017-05-0923:40caioI don't think so. specs are about contracts, not strict types#2017-05-0923:42caioif you want to be this strict, define :account/id as (s/tuple [#(= % :id/account) int?]) or something like that (maybe multi-spec?) and adapt inside the fns that use the id#2017-05-1003:50Alex Miller (Clojure team)in short, no you canā€™t do this#2017-05-1005:54Oliver George@gfredericks came across your defn+spec gist (dated 11m ago). Did it ever evolve? I know I'm not meant to want it but seems like a handy way to get type info into my code. https://gist.github.com/gfredericks/e4a7eafe5dcf1f4feb21ebbc04b6f302#2017-05-1013:32gfredericksI don't think so; but if there's nothing weird about it I could release it in schpec if that would help#2017-05-1013:40Oliver GeorgeThanks. I'm interested for use with CLJS.#2017-05-1016:32gfredericks@olivergeorge k I'll let you know when I get that pushed out#2017-05-1019:23johnIs there a good example of a project with good "test coverage" that leverages spec?#2017-05-1019:29nwjsmith@john might want to try digging through these results https://github.com/search?l=Clojure&amp;q=stest%2Fcheck&amp;type=Code#2017-05-1019:31johnThat's the search-foo I was looking for. Thank you sir.#2017-05-1019:40johnThis looks pretty thorough: https://github.com/mike706574/milo#2017-05-1019:47johnThis looks good too: https://github.com/eauc/tasks-client#2017-05-1107:31rmuslimovCan you please point me what Iā€™m doing wrong in example below:
(defn command [m] 1)
(s/fdef command
        :args (s/cat :n int?)
        :ret string?)
(command "ff") ;; => 1
It works and is not failing with exception
#2017-05-1107:33xiongtxrmuslimov: Are you instrumenting your function? https://clojure.org/guides/spec#_instrumentation_and_testing#2017-05-1107:36rmuslimovah, youā€™re right. Sorry for dumb question#2017-05-1114:18slipsetalso, remember that the return value is not checked even if you instrument it.#2017-05-1119:22Alex Miller (Clojure team)The :ret spec is checked only by stest/check#2017-05-1122:31danielcomptonYou can also use Orchestra to instrument your spec return values: https://github.com/jeaye/orchestra/#2017-05-1120:07rmuslimovOne more newbie question about spec. here as example
{:email1 "
Here is spec I wrote, anyway to make it shorter?
(s/def ::email1 string?)
(s/def ::email2 string?)
(s/def ::email3 string?)
(s/def ::email4 string?)
(s/def ::email5 string?)
(s/def ::data int?)
(s/def ::item
  (s/keys :req-un [::email1 ::email2 ::email3 ::email4 ::email5 ::data]))
#2017-05-1120:07rmuslimovOne more newbie question about spec. here as example
{:email1 "
Here is spec I wrote, anyway to make it shorter?
(s/def ::email1 string?)
(s/def ::email2 string?)
(s/def ::email3 string?)
(s/def ::email4 string?)
(s/def ::email5 string?)
(s/def ::data int?)
(s/def ::item
  (s/keys :req-un [::email1 ::email2 ::email3 ::email4 ::email5 ::data]))
#2017-05-1120:19xiongtxrmuslimov: Seems like youā€™re missing a few colons in s/keys.#2017-05-1120:21rmuslimovyep, fixed. but question remain the same. Why spec is longer than data? is it supposed to be like that? Or itā€™s my un-optimized solution?#2017-05-1121:13Alex Miller (Clojure team)itā€™s supposed to be like that.#2017-05-1121:27johnWould it be dangerous for a user to create a macro for that? like (sdef-each seq-of-keys string?)#2017-05-1121:30Alex Miller (Clojure team)no, macro as you like#2017-05-1121:32johnrgr. so, @U0AR40HJ6, it's possible, bit it's definitely not a newbie/beginner level answer, if you're new to the language in general.#2017-05-1121:33rmuslimov@U050PJ2EU I can do a macro, was just wondering if there exist any proper way to that. thanks!#2017-05-1121:34johnNo prob. Might want to consider using the long form though, from a user/documentation perspective.#2017-05-1121:36johnIf core comes out with such a function in the future, then you can rely on your downstream users to know what you're talking about (if that's a factor for you)#2017-05-1121:36Alex Miller (Clojure team)not going to add that to core#2017-05-1121:37johnright, so if you are making a library you expect to be consumed by others, remember that spec is also intended to be used for documentation.#2017-05-1121:38Alex Miller (Clojure team)if youā€™re just wrapping calls to s/def, I donā€™t see how that matters#2017-05-1121:39Alex Miller (Clojure team)there will be no difference between doing it explicitly or doing it in a macro?#2017-05-1121:39Alex Miller (Clojure team)the macro would presumably just unroll to the same set of calls#2017-05-1121:40john@U064X3EF3 so any documentation functions that operate on sources will probably operate post macro-expansion?#2017-05-1121:40john(Or whatever documentation features are expected to work with spec in the future)#2017-05-1121:40Alex Miller (Clojure team)doc functions donā€™t use the sources for specs#2017-05-1121:40johnrgr#2017-05-1121:40Alex Miller (Clojure team)docs look up the specs in the registry#2017-05-1121:40johnmacro away then šŸ™‚#2017-05-1121:47caioWhy not a vec ::emails with 5 string elements? šŸ¤” #2017-05-1121:08mobileinknot that i know of. you have an idea?#2017-05-1121:09mobileinke.g. (s/def* string? [email1 email2 ... ]),#2017-05-1200:48tbaldridge@rmuslimov perhaps structure it as a list of strings? That approach is simpler for the programmer, and easier to maintain#2017-05-1200:52bbrinckIf my code accepts JS values at the boundary of the system, whatā€™s the preferred way to use spec to validate it? AIUI, s/keys only works with keywords. Is it common practice to convert all string keys to keyword keys when converting to CLJS in order to use spec? e.g. (js->clj input :keywordize-keys true)#2017-05-1200:59rmuslimov@tbaldridge yes it is dead simple, however spec is longer that data it describes. I was wondering if Iā€™m missing some nice shortcut macro for that#2017-05-1203:04tbaldridge@rmuslimov no, I'm saying that this is the better approach:
(s/def ::email string?)
(s/def ::emails (s/coll-of ::email))

(s/def ::item (s/keys :req-un [::emails ::data]))

#2017-05-1203:06rmuslimov@tbaldridge it wasnā€™t real world example, I just created the most trivial example of the problem. Real world example would be something like:
(s/def ::original_currency ::currency-type)
(s/def ::currency ::currency-type)
(s/def ::original_total (s/and double? pos?))
(s/def ::original_taxes (s/and double? pos?))
(s/def ::taxes (s/and double? pos?))
(s/def ::total (s/and double? pos?))
(s/def ::original_total (s/and double? pos?))
(s/def ::rate (s/keys :req-un [::rate_key ::original_currency ::currency ::original_total
                               ::original_taxes ::taxes ::total ::original_total]))
#2017-05-1203:07tbaldridgeah, I see#2017-05-1203:09rmuslimovso, I thought why not something like:
(s/def
  ::rate {::taxes ::money
          ::original_taxes ::money
          ::total ::money
          ::original_total ::money
          ...etc})
#2017-05-1203:27Oliver George@gfredericks Thinking more about defn+spec made me explore why i wanted it and alternative ways to get the the same result.#2017-05-1203:28Oliver GeorgeOne key thing spec gives me is protection from "garbage in garbage out" errors while I'm hacking up new features. Colocating the spec with the defn is helpful during this phase because I'm refactoring heavily as I go - if the s/fdef specs were in another file they're more likely to get stale.#2017-05-1203:28Oliver GeorgeI can do this by adding s/assert statements in my defn.#2017-05-1203:28Oliver GeorgeLater when code stabilises I see the logic of specs living in a different namespace so they are out of the way but still available to reference.#2017-05-1203:28Oliver GeorgeAs an aside, IDE support to allow hiding of spec statements would make that less important. #cursive#2017-05-1203:29Oliver GeorgeBut I appreciate the idea that separate namespaces for specs also allow for different specs to be applied to the same code depending on need.#2017-05-1203:30Oliver George@gfredericks defn+spec would benefit from the s/assert flag so that it can have no impact on prod performance.#2017-05-1203:33tbaldridge@rmuslimov that's covered in some of the spec docs#2017-05-1203:33tbaldridge@rmuslimov https://clojure.org/guides/spec#_entity_maps#2017-05-1204:47rmuslimovI did actually, as far as I can see: only s/keys is given for building spec for maps. I cannot do something close to way I proposed. Developer should always define explicitly keys like (s/def ::total ::money) and then build spec for map with s/keys.#2017-05-1212:52Alex Miller (Clojure team)It's quite intentional that information maps are built as sets of attributes. This is a key design principle in spec - the semantics belong to the attribute, not to the map.#2017-05-1212:52Alex Miller (Clojure team)This is covered a bit in https://clojure.org/about/spec#2017-05-1214:18mishais there any context-independent benefit/downside of defining intermediate predicate fn over inline spec definition?
(defn nsless-keyword? [k]
  (and (keyword? k) (nil? (namespace k))))

(defn ::nsless-keyword nsless-keyword?)
vs:
(defn ::nsless-keyword (s/and keyword? (complement namespace)))
#2017-05-1217:22Alex Miller (Clojure team)Well it affects automatic generator capability#2017-05-1217:22Alex Miller (Clojure team)But you should really use simple-keyword? rather than either of those#2017-05-1217:22Alex Miller (Clojure team)Which has a built in generator#2017-05-1218:37creeseI would like to be able to attach a docstring to a spec. Has there been an progress on https://dev.clojure.org/jira/browse/CLJ-1965?#2017-05-1218:39Alex Miller (Clojure team)no#2017-05-1302:37danielcomptonIn https://clojure.org/about/spec#_minimize_intrusion it says "Donā€™t require that people e.g. define their functions differently."#2017-05-1302:39danielcomptonWill there also be a way to provide a spec along with a function? e.g. (s/defn ...) or (defn my-fn [args] {:args (s/cat ...)} ... or something similar?#2017-05-1302:39danielcomptonIf I do want to define my functions with my spec?#2017-05-1302:44danielcompton@rmuslimov I think spec-tools has https://clojurians.slack.com/archives/C1B1BB2Q3/p1494558543543341 as data-specs: https://github.com/metosin/spec-tools/#data-specs#2017-05-1302:47lincpaI think proactive data standardization work is better than passive data validation. Industrial assembly line recommended using standardized materials.#2017-05-1302:51lincpaStandardize input and output data at the beginning and end of data processing#2017-05-1302:57lincpaProactively standardize non-standard data, and if fail do exception handling.#2017-05-1305:00Alex Miller (Clojure team)@danielcompton no, there will not be a way to combine function definition with spec#2017-05-1309:26mishaare there any builtin predicates for queues in clj/cljs?#2017-05-1315:02john@misha What do you mean? like qeueu? Don't believe so. If you mean query for type, you could use (instance? cljs.core/PersistentQueue #queue [])#2017-05-1315:32Alex Miller (Clojure team)@misha no#2017-05-1315:32misha@john yes, sort of, with a generator included, please#2017-05-1315:34Alex Miller (Clojure team)You might be able to spec it with coll-of and some of the kind and into options, but I haven't tried#2017-05-1315:35misha@alexmiller, what is a correct way to define spec for dynamic function (like reducing function)? I end up with following, is that legal?
(s/def ::reducing-fn
  (s/fspec :args (s/cat ...))
#2017-05-1315:35Alex Miller (Clojure team)Sure?#2017-05-1315:35Alex Miller (Clojure team)What is dynamic about that?#2017-05-1315:38mishaAFAIR, s/fdef accepts qualified fn symbol, with intention to use it on a function with that exact name. There is an f(g) where g - is a function. I need to: 1) spec g as an argument, 2) re-use that spec. (s/def :foo (s/fspec ...)) seems to work, but I expect it to bite me in some way later.#2017-05-1316:18misha"how to get named spec for anonymous fn", there.#2017-05-1318:31Alex Miller (Clojure team)That should work#2017-05-1409:24thedavidmeisteranyone else seen java.lang.RuntimeException: Unable to resolve symbol: qualified-keyword? in this context, compiling:(clojure/spec/gen/alpha.clj:131:4) with clojure version 1.9.0-alpha16?#2017-05-1413:57weavejesterWhatā€™s the value of *clojure-version*, @thedavidmeister?#2017-05-1418:41Alex Miller (Clojure team)@thedavidmeister no but sounds like you are using spec.alpha with Clojure < 1.9#2017-05-1423:09thedavidmeister@alexmiller @weavejester yeah that was it, boot had its own version of clojure messing with things#2017-05-1423:09thedavidmeisterthanks#2017-05-1509:26danielnealI'm trying to take a reagent vector, conform it, make a modification and unform it back.
(s/def :reagent/vector
    (s/cat :element any? :props (s/? map?) :children (s/* any?)))
This conforms fine:
(s/conform :reagent/vector [:div {:a 3} [:div "1207"] [:div "w13"]])
;; => {:element :div, :props {:a 3}, :children [[:div "1207"] [:div "w13"]]}
but unforms back to a lazyseq - is there a spec I can use that will make unform turn it back to a vector?
#2017-05-1509:28danielnealI tried prefixing with s/and vector but I think that was wrong#2017-05-1509:35dergutemoritz@danieleneal Wrapping it in (s/and ... (s/conformer identity vec)) should do the trick#2017-05-1509:36dergutemoritzOh or maybe you have to put it before the s/cat really#2017-05-1509:36dergutemoritzInteresting ...#2017-05-1509:40danielnealyeah (s/and (s/conformer identity vec) ... works, thanks @dergutemoritz#2017-05-1509:40danielnealis that appropriate/idiomatic do you know?#2017-05-1509:41dergutemoritzI guess that's an innocent enough use of s/conformer, yeah šŸ™‚#2017-05-1509:41dergutemoritzMaybe we should call this kind of use "vegan" as opposed to the "meat grinder" pattern#2017-05-1509:44danielnealšŸ˜‚#2017-05-1509:45dergutemoritz"No animals were harmed in this use of s/conformer"#2017-05-1514:08Alex Miller (Clojure team)the need for regex ops in a vector is a known issue - itā€™s pretty challenging right now to get conforming, validation, unforming, generation, and describing to work properly together#2017-05-1521:57zaneWhat's the right way to attach a generator to a value produced by s/conformer?#2017-05-1521:58Alex Miller (Clojure team)s/with-gen#2017-05-1522:00zaneRight, but how?#2017-05-1522:00zaneDoes it need to be attached to a keyword?#2017-05-1522:00zanee.g.
(def keyword-conformer
  (s/with-gen
    (s/conformer
     (fn [x]
       (cond (keyword? x) x
             (string? x) (keyword x)
             :else :clojure.spec/invalid)))
    #(s/gen keyword?)))
#2017-05-1522:00zaneShould that work? ā˜ļø:skin-tone-2:#2017-05-1522:01zanei.e. Should I be able to (gen/sample (s/gen keyword-conformer)) on that?#2017-05-1522:03Alex Miller (Clojure team)yes#2017-05-1522:03Alex Miller (Clojure team)works for me#2017-05-1522:07zaneMight be my version of clj, then.#2017-05-1522:08zaneI have several other specs like:
(s/def ::example (s/and keyword-conformer #{:some :specific :values}))
#2017-05-1522:08zaneOf course, calling (gen/sample ::example) fails because it runs out of tries.#2017-05-1522:09zaneI've been working around that with
(s/def ::example
  (s/with-gen
    (s/and keyword-conformer
           #{:some :specific :values})
    #(s/gen #{:some :specific :values})))
but that feels gross. Is there a better way?
#2017-05-1523:56Alex Miller (Clojure team)(s/def ::example #{:some :specific :values "some" "specific" " "values"}) would work :)#2017-05-1523:57Alex Miller (Clojure team)What do you actually need to spec?#2017-05-1600:30zaneThat the incoming values will be in that set when conformed, and that strings will be conformed to keywords.#2017-05-1600:31zaneI'm trying to make my example more DRY.#2017-05-1600:45Alex Miller (Clojure team)(def vs #{:some :specific :values}) (s/def ::example (into vs (map name) vs))#2017-05-1600:48Alex Miller (Clojure team)That doesn't conform to keywords but transformation is not really the point of spec#2017-05-1606:33odinodindoes anyone know of any efforts that present clojure spec validation errors in a nice way? Iā€™m looking for ideas and inspiration for a web based error message component Iā€™m working on#2017-05-1607:05rmuslimov@odinodin just spent some time looking into the same issue, found this https://www.slideshare.net/alexanderkiel/form-validation-with-clojure-spec and can recommend this approach#2017-05-1607:07odinodin@rmuslimov thanks. I was thinking more about how to present raw clojure.spec validation errors in a nicer way during development#2017-05-1607:09rmuslimovah ok, interesting. Not sure how standard can be improved, may be you already have any ideas - how it may look like?#2017-05-1607:16odinodinJust started digging, not sure where it will lead yet#2017-05-1611:10urbankwhy has cljs.spec been renamed to cljs.spec.alpha?#2017-05-1611:25curlyfryurbank: https://groups.google.com/forum/?__s=f7szr7fg4jw7kzeciy43#!msg/clojure/10dbF7w2IQo/ec37TzP5AQAJ#2017-05-1611:26urbank@curlyfry Thanks!#2017-05-1613:14curlyfryAny news on https://dev.clojure.org/jira/browse/CLJ-2123 or something similar? Having some issues right now with name clashes on specs (that we still want in the same file). We're specing a re-frame db and want to keep the specs for a specific page in the same file. Often parts of specs for input field and similar end up having the same names as other parts of the state. The solution right now is either to create an artificial and verbose namespaced keyword (:my-page.input.domain/id) or to add a new file just for the clashing specs.#2017-05-1613:57stathissideris@curlyfry in a similar vein: when you refactor clashing keywords to have different namespaces, does the rename mean that you get rippling changes throughout your code?#2017-05-1614:23pseudI've got a map where all entries should follow some template (e.g. (s/map-of string? number?) except one optional key, let's call it :foo. I tried something like s/merge, but it seems like it was designed with multiple s/keys specs in mind. (s/merge (s/map-of string? number?) (s/keys :opt-un [::foo])). I can of course write my own predicate function to split the map in two and check against each spec with s/valid?, but I'll lose a lot of contextual information in case of failures. Is there no better way ?#2017-05-1615:43Alex Miller (Clojure team)@curlyfry no update#2017-05-1615:43Alex Miller (Clojure team)@pseud you can use the ā€œhybrid mapā€ technique that I describe at http://blog.cognitect.com/blog/2017/1/3/spec-destructuring#2017-05-1615:44Alex Miller (Clojure team)yours is a bit simpler than the one described there#2017-05-1616:59pseud@alexmiller cool, a bit too tired to digest, but bookmarked.#2017-05-1618:35griersonHey, does (s/exercise) work with CLJS?#2017-05-1618:54Alex Miller (Clojure team)afaik#2017-05-1619:06curlyfry@grierson Yup!#2017-05-1619:19griersondo I need to import anything else apart from cljs.spec?#2017-05-1619:28griersonFixed it. Need to import [clojure.test.check.generators]. I was only using Spacemacs inline eval I wasnā€™t looking at the REPL.#2017-05-1714:29chilleniousis there a way to in-line this: (s/def ::a integer?) (s/keys :req-un [::a]) so that I can state I expect ::a in the map that should be checked against integer? without having to specify ::a first?#2017-05-1714:30luxbock@chillenious no, you could write a macro to do something like (my-keys :req-un [(::a integer?)]) that expands to it#2017-05-1714:31chilleniousok, thx#2017-05-1714:31moxaj@chillenious that's by design, see https://clojure.org/about/spec - Map specs should be of keysets only#2017-05-1714:33chilleniousYeah, I was afraid you were going to say that šŸ™‚ I can't be the first one who thinks that that's a PITA though. I very often have the situation where I'd like to use a different key from the 'type'.#2017-05-1715:21ikitommichillenious: something like this? https://github.com/metosin/spec-tools/blob/master/README.md#data-specs#2017-05-1715:22chilleniousYes! Thanks!#2017-05-1715:23chilleniousThat looks great, will try it out later this week.#2017-05-1715:27wilkerlucio@chillenious I encourage you to avoid this, there are many reasons for spec to encourage doing things in the way it does, to encourage you to have value definitions and just make compositions out of it, if you start thinking on your data as records you gonna lose a lot of the benefits on the long run, I recommend watching this presentation from Rich, where he talks a lot about the grand ideas and why things are the way they are: https://vimeo.com/195711510#2017-05-1715:28chilleniousI'll look at it, thanks. I'm using this as part of test code where I want some validation and documentation as to what data structures can/ should be used. So it's as important to me that it is easy to read and terse, and doesn't have to be a perfect water tight solution for all things spec šŸ™‚#2017-05-1715:19wilkerluciohello šŸ™‚#2017-05-1715:20wilkerlucioI noticed when a spec is created with (s/with-gen) it loses track of the original form:#2017-05-1715:20wilkerlucio
(s/def ::some-spec (s/with-gen int? #(s/gen int?)))
=> :user/some-spec
(s/form ::some-spec)
=> :clojure.spec.alpha/unknown
#2017-05-1715:21wilkerluciobut doing it using the (s/def (s/spec int? :gen #(s/gen int?))) works fine#2017-05-1715:22wilkerlucio
(s/def ::other-spec (s/spec int? :gen #(s/gen int?)))
=> :user/other-spec
(s/form ::other-spec)
=> clojure.core/int?
#2017-05-1715:22wilkerluciois this expected?#2017-05-1717:37Alex Miller (Clojure team)thereā€™s a patch pending to fix this#2017-05-1717:40Alex Miller (Clojure team)https://dev.clojure.org/jira/browse/CLJ-2068#2017-05-1719:07gphilippThere was a question last year about speccing a binary tree:
(def BinaryTree 
  (maybe ;; any empty binary tree is represented by nil
   {:value long 
    :left (recursive #ā€˜BinaryTree) 
    :right (recursive #ā€˜BinaryTree)}))
#2017-05-1719:08gphilipp@alexmiller: you told Andrey Grin :ā€œYes, you can create recursive definitions by registering a spec that refers to itself via registered name (a namespaced keyword).ā€œ. I canā€™t find a way to do it, could please post a snippet of code ?#2017-05-1719:08gphilippRelevant message: https://groups.google.com/forum/#!topic/clojure/5sZdCPQgZz4%5B1-25%5D#2017-05-1719:22misha@gphilipp
(s/def :bt/value number?)
(s/def :bt/left (s/nilable :a/bt))  ;;<- recursive
(s/def :bt/right (s/nilable :a/bt))  ;;<- recursive
(s/def :a/bt
  (s/nilable
    (s/keys
      :req-un [:bt/value]
      :opt-un [:bt/left :bt/right])))

(s/explain :a/bt
  {:value 1
   :left {:value 21 :right {:value 31}}
   :right {:value 22 :left {:value 32 :right {:value 4} :left {:value 33}}}})

Success!
=> nil
#2017-05-1719:24mishas/exercise, however, stackoverflows at different points, so might need to tinker with that.#2017-05-1719:29misha
(binding [s/*recursion-limit* 2]
 (map first (s/exercise :a/bt 3)))
=>
({:value -1.0, :right {:value -1}, :left nil}
 {:value -1,
  :left {:value -3.0, :left {:value -1, :right {:value 0.75}}},
  :right {:value 0}}
 {:value 0})
#2017-05-1719:32wilkerlucio@alexmiller I was having this idea yesterday, about spec hierarchy tracking, currently when you define a spec from another (eg: (s/def ::something ::other-thing)) the resolving happens on definition time, I understand that is preferable for performance reasons, but doing that makes us lose track of the hierarchy. I was implementing a coerce engine on top of spec, and I miss having this hierarchy information, so I was thinking that we could have Clojure hierarchy for the specs, so when defining those we do a derive on those keywords, so we would have this information aside available for use, do you think this is a reasonable feature to have on spec?#2017-05-1719:36Alex Miller (Clojure team)I donā€™t know about all that. there are some open tickets about the resolution aspects that I expect will be addressed.#2017-05-1719:36Alex Miller (Clojure team)it is not the intent that you lose that tracking now#2017-05-1719:37wilkerluciofor example, given the following specs:#2017-05-1719:37wilkerlucio
(s/def ::a int?)
(s/def ::b ::a)
(s/def ::c ::b)
#2017-05-1719:37wilkerluciois there a way, starting from ::c to know that it is derived from ::b and ::a?#2017-05-1719:38Alex Miller (Clojure team)(s/get-spec ::c)#2017-05-1719:38Alex Miller (Clojure team)should tell you ::b#2017-05-1719:39Alex Miller (Clojure team)but things like https://dev.clojure.org/jira/browse/CLJ-2067 and https://dev.clojure.org/jira/browse/CLJ-2079 make me think that not everything is exactly right in this area#2017-05-1719:41wilkerluciointeresting, I didn't knew that get-spec would do that, and testing with the previous example works#2017-05-1719:42wilkerlucioglad to hear that my assumptions were wrong on this#2017-05-1719:42wilkerluciothanks Alex#2017-05-1722:16lvhhow do I make a spec refer to a spec in another ns? I did (s/fdef ::name ::storage/name) but I just get No value supplied for key: ::storage/name#2017-05-1722:23lvhnever mind, Iā€™m an idiot and meant s/def#2017-05-1722:38gphilippThx @misha. Would it be possible to have a fn to create that kind of spec with the 'number?' predicate being passed as a parameter to this fn ?#2017-05-1807:19misha@gphilipp I'd think macro is doable, if you really need to have several BTs. However I'd start with (s/or ...) with all the variants you want in it, and see if that's enough#2017-05-1814:14wilkerlucioHello, I'm trying to use the overrides argument on a generator, but I'm not sure how I supposed to use it#2017-05-1814:14wilkerlucioI'm trying like this:#2017-05-1814:15wilkerlucio
(s/exercise
    (s/keys :req [:some/data
                  :some/more-data])
    5
    {:some/data #(gen'/for [i (s/gen (s/int-in 1 1000000))]
                                 (-> i bigdec (/ 100.0M) (* -1)))})
#2017-05-1814:15wilkerluciobut that doesn't work, does someone have an example on how to use it?#2017-05-1814:25Alex Miller (Clojure team)that looks roughly correct, maybe share more of the example?#2017-05-1818:07wilkerlucio@alexmiller a more complete example here:#2017-05-1818:08wilkerluciosorry the noise now, I was looking at the wrong number, A is always even here, saying it's correct, I'll check my other example again#2017-05-1818:12wilkerlucio@alexmiller ok, I think I might have found a bug#2017-05-1818:13wilkerlucioit works fine when the spec is not defined in terms of other spec#2017-05-1818:13wilkerluciohere is an example where the override doesn't work#2017-05-1818:13wilkerlucio
(s/def ::common-etc int?)
  (s/def ::a ::common-etc)
  (s/def ::b int?)

  (s/exercise (s/keys :req [::a ::b])
              30
              {::a #(gen/fmap (fn [x] (* x 2)) (s/gen int?))})
#2017-05-1818:15wilkerlucioif ::a was defined as int? it works, but when making it reference other spec, the override version is not used#2017-05-1821:41Alex Miller (Clojure team)Oh, yes there is a ticket for this #2017-05-1900:45cflemingIā€™m playing around with the specs for spec from CLJ-2112, and Iā€™m having some problems with it.#2017-05-1900:46cfleming
(s/conform ::spec (s/form 'clojure.core/let))
IllegalArgumentException Don't know how to create ISeq from: clojure.lang.Keyword  clojure.lang.RT.seqFrom (RT.java:547)
#2017-05-1900:46cflemingWhen I try to investigate further, things get strange:#2017-05-1900:47cflemingAm I missing something obvious?#2017-05-1900:49cflemingI understand that the specs there are half-baked, but I didnā€™t expect to get exceptions back.#2017-05-1900:50cflemingThis is with alpha16, in case that matters.#2017-05-1903:01cflemingHmm, clearly I should be quoting the form Iā€™m passing in, but then I just get back to the IllegalArgumentException#2017-05-1903:12cflemingActually, this might be CLJ-2152#2017-05-1904:11Alex Miller (Clojure team)at the beginning when you do this: (s/form 'clojure.core/let) - that returns the fspec for let, not sure if you wanted that or the args spec which would be (s/form (:args (s/get-spec 'clojure.core/let)))#2017-05-1904:13Alex Miller (Clojure team)when you do (s/conform ::spec (clojure.spec.alpha/cat :name ... )), youā€™ll need to quote there as the spec is expecting a sequential thing#2017-05-1904:13cflemingRight, but when I quote then I get the exception.#2017-05-1904:14cflemingOne sec, I got sidetracked, starting my REPL againā€¦#2017-05-1904:14Alex Miller (Clojure team)Iā€™m actually heading to bed but could load it up tomorrow and look at it#2017-05-1904:15cfleming
(s/conform ::spec (s/form (:args (s/get-spec 'clojure.core/let))))
IllegalArgumentException Don't know how to create ISeq from: clojure.lang.Keyword  clojure.lang.RT.seqFrom (RT.java:547)
#2017-05-1904:15Alex Miller (Clojure team)can you (pst *e) ?#2017-05-1904:17cfleming
java.lang.IllegalArgumentException: Don't know how to create ISeq from: clojure.lang.Keyword
	at clojure.lang.RT.seqFrom(RT.java:547)
	at clojure.lang.RT.seq(RT.java:527)
	at clojure.lang.RT.first(RT.java:683)
	at clojure.core$first__6389.invokeStatic(core.clj:55)
	at clojure.core$first__6389.invoke(core.clj:55)
	at clojure.spec.alpha$multi_spec_impl$predx__912.invoke(alpha.clj:904)
	at clojure.spec.alpha$multi_spec_impl$reify__919.conform_STAR_(alpha.clj:916)
	at clojure.spec.alpha$or_spec_impl$fn__958.invoke(alpha.clj:1035)
	at clojure.spec.alpha$or_spec_impl$reify__963.conform_STAR_(alpha.clj:1057)
	at clojure.spec.alpha$conform.invokeStatic(alpha.clj:150)
	at clojure.spec.alpha$conform.invoke(alpha.clj:146)
	at clojure.spec.alpha$dt.invokeStatic(alpha.clj:748)
	at clojure.spec.alpha$dt.invoke(alpha.clj:743)
	at clojure.spec.alpha$dt.invokeStatic(alpha.clj:744)
	at clojure.spec.alpha$dt.invoke(alpha.clj:743)
	at clojure.spec.alpha$deriv.invokeStatic(alpha.clj:1475)
	at clojure.spec.alpha$deriv.invoke(alpha.clj:1469)
	at clojure.spec.alpha$deriv.invokeStatic(alpha.clj:1483)
	at clojure.spec.alpha$deriv.invoke(alpha.clj:1469)
	at clojure.spec.alpha$deriv.invokeStatic(alpha.clj:1486)
	at clojure.spec.alpha$deriv.invoke(alpha.clj:1469)
	at clojure.spec.alpha$deriv.invokeStatic(alpha.clj:1483)
	at clojure.spec.alpha$deriv.invoke(alpha.clj:1469)
	at clojure.spec.alpha$re_conform.invokeStatic(alpha.clj:1610)
	at clojure.spec.alpha$re_conform.invoke(alpha.clj:1601)
	at clojure.spec.alpha$regex_spec_impl$reify__1340.conform_STAR_(alpha.clj:1651)
	at clojure.spec.alpha$conform.invokeStatic(alpha.clj:150)
	at clojure.spec.alpha$conform.invoke(alpha.clj:146)
	at clojure.spec.alpha$dt.invokeStatic(alpha.clj:748)
	at clojure.spec.alpha$dt.invoke(alpha.clj:743)
	at clojure.spec.alpha$dt.invokeStatic(alpha.clj:744)
	at clojure.spec.alpha$dt.invoke(alpha.clj:743)
	at clojure.spec.alpha$multi_spec_impl$reify__919.conform_STAR_(alpha.clj:917)
	at clojure.spec.alpha$or_spec_impl$fn__958.invoke(alpha.clj:1035)
	at clojure.spec.alpha$or_spec_impl$reify__963.conform_STAR_(alpha.clj:1057)
	at clojure.spec.alpha$conform.invokeStatic(alpha.clj:150)
	at clojure.spec.alpha$conform.invoke(alpha.clj:146)
	at clojure.spec.specs$eval3133.invokeStatic(form-init2127301951087080660.clj:1)
	at clojure.spec.specs$eval3133.invoke(form-init2127301951087080660.clj:1)
	at clojure.lang.Compiler.eval(Compiler.java:6977)
#2017-05-1904:19cflemingI suspect this is related to CLJ-2152, Metosin in a blog post said that their spec walker/visitor didnā€™t work with s/& or s/keys* because of it.#2017-05-1904:19cflemingBut I may be off there.#2017-05-1904:19cflemingFrom your test cases in that patch, I get this:
(s/conform ::spec (s/form (s/keys* :req [::foo])))
IllegalArgumentException Don't know how to create ISeq from: clojure.lang.Keyword  clojure.lang.RT.seqFrom (RT.java:547)
#2017-05-1904:20Alex Miller (Clojure team)that code had a number of broken cases due to form - weā€™ve fixed most of them since but there are a few known problems still#2017-05-1904:20Alex Miller (Clojure team)s/&, s/keys* in particular#2017-05-1904:21Alex Miller (Clojure team)I guess some of the nested specs in let bindings could be getting tripped up#2017-05-1904:21cflemingAnd I guess s/cat uses s/keys* under the hood?#2017-05-1904:22cflemingIā€™ll check that.#2017-05-1904:22Alex Miller (Clojure team)nah#2017-05-1904:23Alex Miller (Clojure team)but the specs for s/cat could use it#2017-05-1904:23Alex Miller (Clojure team)anyhow, that ticket is incomplete because itā€™s a wip and there are many things that donā€™t work with it yet so it doesnā€™t surprise me that things donā€™t work#2017-05-1904:24Alex Miller (Clojure team)rather than just having it sit on my hard drive I figured it was better to be a wip on a ticket#2017-05-1904:24cflemingSure, is there a workaround I could use? It doesnā€™t seem like a problem with the specs themselves, but more likely with form#2017-05-1904:25Alex Miller (Clojure team)I donā€™t think it is - it looks like (s/form (s/keys* :req [::foo])) returns the right thing#2017-05-1904:25Alex Miller (Clojure team)sorry, grabbed wrong code there#2017-05-1904:25Alex Miller (Clojure team)(s/form (:args (s/get-spec 'clojure.core/let)))#2017-05-1904:25cflemingI was going to say - pretty sure an exception isnā€™t the right thing šŸ™‚#2017-05-1904:26Alex Miller (Clojure team)I get (clojure.spec.alpha/cat :bindings :clojure.core.specs.alpha/bindings :body (clojure.spec.alpha/* clojure.core/any?)) for that, which seems good
#2017-05-1904:26Alex Miller (Clojure team)the spec specs should only need to have working s/cat and s/* for that to conform#2017-05-1904:27cflemingI get the same, but the conform fails with that exception.#2017-05-1904:28Alex Miller (Clojure team)well, Iā€™m too wiped to take it any further atm#2017-05-1904:28Alex Miller (Clojure team)will take a glance tomorrow#2017-05-1904:28cflemingIt seems like s/& is the root cause - if I macroexpand (s/keys* :req [::foo]) I get:
(let* [mspec# (s/keys :req [:clojure.spec.specs/foo])]
  (s/with-gen (s/& (s/* (s/cat :clojure.spec.alpha/k keyword? :clojure.spec.alpha/v any?))
                   :clojure.spec.alpha/kvs->map
                   mspec#) (fn [] (clojure.spec.gen.alpha/fmap (fn [m#] (apply concat m#)) (s/gen mspec#)))))
#2017-05-1904:28cflemingAnd itā€™s failing on the :clojure.spec.alpha/kvs->map#2017-05-1904:28cflemingOk, no problem - thanks for the help.#2017-05-1904:29cflemingIā€™m using the code from the patch, only updated to use the new alpha ns.#2017-05-1904:29Alex Miller (Clojure team)the error message looks like the ::spec multi-spec is using the dispatch function first on a keyword#2017-05-1904:30Alex Miller (Clojure team)so maybe itā€™s just that ::spec needs to be expanded to include keywords as valid specs#2017-05-1904:31cflemingOk, Iā€™ll see if I can get my head around that.#2017-05-1904:31Alex Miller (Clojure team)
(s/def ::spec
  (s/or :set set?
        :pred symbol?
        :registered-spec keyword?
        :form (s/multi-spec spec-form (fn [val tag] val))))
#2017-05-1904:31Alex Miller (Clojure team)I probably just didnā€™t test that case#2017-05-1904:32Alex Miller (Clojure team)I was spending most of my time fixing s/form bugs :)#2017-05-1904:32Alex Miller (Clojure team)anyhow, good nightā€¦#2017-05-1904:32Alex Miller (Clojure team)for real!#2017-05-1904:32cflemingLooks good - thanks!#2017-05-1907:49seantempestaProbably a dumb question, but when I use s/describe Iā€™m getting back a list in the form of (keys :req [:person/first-name :person/last-name] :opt [:person/contact]). Is there an easy way to convert this to a map? Is it hacky to use nā€™th to access the required and optional fields?#2017-05-1912:37pseud@seantempesta I really wouldn't worry about that. If you write a single function to handle the extraction of what you want into a data structure suitable to your needs you only have a single function to refactor should the world change...#2017-05-1912:37Alex Miller (Clojure team)Well the prior conversation here is about specs for specs which allow you to conform a spec form into a map#2017-05-1912:37Alex Miller (Clojure team)But still a wip#2017-05-1919:06arohnerthis looks like a bug to me:
(s/conform (s/keys :req-un [:bogus/bogus]) {:bogus "foo"}))
#2017-05-1919:07arohner1) define an s/keys, without defining :bogus/bogus. Thereā€™s no warning/error that :bogus/bogus is undefined, and the spec conforms#2017-05-1920:41Alex Miller (Clojure team)agreed, file a jira#2017-05-1920:42Alex Miller (Clojure team)although as I try other things, this is also the behavior with :req#2017-05-1920:42mishahow is this bug, when key specs are opt in?#2017-05-1920:43mishaand s/keys only specifies keys set. why would it complain?#2017-05-1920:43Alex Miller (Clojure team)ā€œThe validator will ensure the :req keys are present.ā€#2017-05-1920:44Alex Miller (Clojure team)from the s/keys docstring#2017-05-1920:44mishabut :bogus is present#2017-05-1920:44Alex Miller (Clojure team)Iā€™m agreeing with you#2017-05-1920:45Alex Miller (Clojure team)ā€œkeys ā€¦ are required, and will be validated and generated by specs (if they exist)ā€#2017-05-1920:45Alex Miller (Clojure team)so, Iā€™ve changed my mind - I think itā€™s doing what it says will do#2017-05-1920:45misha#2017-05-1920:48mishafor a minute I was questioning 2/3 of specs I wrote#2017-05-1923:34jfntnWhat would be a good way of and-ing multiple specs without the conforming behavior of s/and?#2017-05-1923:58seantempestaGood point. Thanks!#2017-05-2019:05borkdudeIs it possible to express the following in spec: f accepts a function g from string? to int? and it is also possible to generate random instances of g?#2017-05-2019:07borkdudeI wondered about this after playing with Quickcheck in Haskell, which can do this#2017-05-2019:25luxbock@borkdude (s/fdef f :args (s/cat :g (s/fspec :args (s/cat :str string?) :ret int?)))#2017-05-2019:47borkdude@luxbock Looks good. How about the generational aspect?#2017-05-2020:40Alex Miller (Clojure team)Gen of fspec is a function that uses its ret generator#2017-05-2020:46stathissiderisHow would I register (and then use) specs that were generated dynamically? I think Iā€™m at the stage where I need to start property testing spec-provider. To do this I need to generate random data (possible), infer specs for them (thatā€™s what the lib does) and then validate that all the random data are valid for those inferred specs.#2017-05-2022:22Alex Miller (Clojure team)Well you can either call s/def and register your specs or you can just call conform on the specs directly without registering, which works for a lot of stuff#2017-05-2108:47stathissideris@alexmiller thanks. But Iā€™m generating the forms of the specs, so I guess Iā€™d have to eval them in any case, right?#2017-05-2112:49Alex Miller (Clojure team)Yes#2017-05-2118:44stathissideris@alexmiller great, makes sense, thanks#2017-05-2217:34nullptrtomorrow is the 1y anniversary of the clojure.spec announcement! https://clojure.org/news/2016/05/23/introducing-clojure-spec#2017-05-2219:07mobileinki have a bunch of map specs, each respresenting a "resource type". i want to present them on a web page. is s/form the way to go? is there a library that makes it easy to get and send a json/edn respresentation of a map spec from the registry?#2017-05-2309:24ikitommimobileink: spec visitor should help: https://github.com/metosin/spec-tools/blob/master/README.md#spec-visitors#2017-05-2221:30Alex Miller (Clojure team)s/form will be fully resolved, s/describe will be more terse#2017-05-2221:30Alex Miller (Clojure team)Stuff in there should be edn afaik?#2017-05-2304:16danielcomptonWhat is the right way to spec functions that can throw exceptions? It seems like I need to allow :ret to be nil to handle exceptions here#2017-05-2316:11Alex Miller (Clojure team)If there is an exception there is no return value#2017-05-2316:12Alex Miller (Clojure team)Exceptions are not covered by spec#2017-05-2316:12Alex Miller (Clojure team)Because they are ... exceptional cases#2017-05-2317:45deplecthai all, how do we handle ordering with spec?#2017-05-2317:57Alex Miller (Clojure team)youā€™ll need to explain more, not sure what you mean#2017-05-2318:06caio(s/and sorted? ...)?#2017-05-2318:07caio(not the clojure.core/sorted?)#2017-05-2318:26andrewmcveighIs this expected?
(let [n 2
      t int?]
  (s/conform (s/coll-of t :into [] :kind vector? :count n) [1 2]))
=> [1 2]

(let [n nil
      t int?]
  (s/conform (s/coll-of t :into [] :kind vector? :count n) [1 2]))
=> :clojure.spec.alpha/invalid

(let [t int?]
  (s/conform (s/coll-of t :into [] :kind vector? :count nil) [1 2]))
=> [1 2]
#2017-05-2318:33andrewmcveighI guess this line is responsible. https://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/alpha.clj#L541#2017-05-2318:34andrewmcveighI just wonder if this is a bug, or if the s/every macro is meant to check the 'n symbol rather than what it evals to.#2017-05-2319:19Alex Miller (Clojure team)I think Iā€™d call it a bug, given the docstring states that nil is the default which seems to imply you could pass it#2017-05-2319:19Alex Miller (Clojure team)feel free to file a jira#2017-05-2319:53andrewmcveighOk, will do. Thanks#2017-05-2415:28weavejesterShould instrument report the name of the function that failed in the ex-data?#2017-05-2415:29weavejesterIt reports the function name in the exception message, but itā€™s missing from the exceptionā€™s data.#2017-05-2415:37Alex Miller (Clojure team)most likely yes#2017-05-2415:37Alex Miller (Clojure team)example would help, but feel free to file something about itk#2017-05-2415:43Alex Miller (Clojure team)@weavejester just looked at an example - definitely yes#2017-05-2415:43weavejesterAh, I was just about to post one šŸ™‚#2017-05-2415:43weavejesterShall I add a JIRA report, @alexmiller ?#2017-05-2415:47Alex Miller (Clojure team)that would be great#2017-05-2415:47Alex Miller (Clojure team)if you patch it, I will screen it too :)#2017-05-2415:48Alex Miller (Clojure team)btw, enjoyed your defn podcast episode#2017-05-2415:49weavejesterAh thanks šŸ™‚#2017-05-2416:04weavejester@alexmiller I created https://dev.clojure.org/jira/browse/CLJ-2166#2017-05-2416:06Alex Miller (Clojure team)I assume this is obvious, but patch will be against the spec.alpha lib, not core clojure repo#2017-05-2416:18weavejesterSorry, I couldnā€™t find the spec.alpha lib in the project list when I created the issue.#2017-05-2416:39stathissiderisis there any support for string keys in s/keys? or any plan to support them later?#2017-05-2417:44ghadino @stathissideris -- s/keys needs things that are globally registered (keywords/specs)#2017-05-2417:52stathissideris@ghadi thanks#2017-05-2418:05Alex Miller (Clojure team)@weavejester you filed it in the right place - spec issues are being managed in CLJ, just meant that the patch would be on a different repo#2017-05-2418:06weavejesterAh, cool.#2017-05-2418:07jfntnWhatā€™s an idiomatic way to spec args for a multi-arity function where the longer arities only add new arguments to the shorter ones? Say the full args vector is [a b c ...] but you have aritites with defaults that only take [a], or [a b] etc?#2017-05-2418:12stathissideris@jfntn I think you either do (s/or :arity1 (s/cat ..) :arity2 (s/cat ...) :arity3 (s/cat)) or (s/cat x y (s/? w z))#2017-05-2418:12stathissiderisnot sure which is more idiomatic#2017-05-2418:12stathissiderisactually itā€™s not s/?#2017-05-2418:14jfntnAh that works: (s/valid? (s/cat :a string? :b string? :c (s/? string?)) ["a" "b"])#2017-05-2418:14stathissiderisyeah, but I think s/? is for a single predicate#2017-05-2418:14jfntnWas looking to remove the duplication from s/or so I think thatā€™s what Iā€™m after#2017-05-2418:15jfntnSeems to do what I expect: (s/explain (s/cat :a string? :b string? :c (s/? string?) :d (s/? string?)) ["a" "b" "c" :d])#2017-05-2418:18stathissideris@jfntn ok, but if for some reason you had arities 2 and 4, youā€™d say#2017-05-2418:18stathissideris(s/cat :a string? :b string? :rest (s/? (s/cat :c string? :d string?)))#2017-05-2418:19stathissideriswhich is valid for ["a" "b"] and ["a" "b" "c" "d"] but not for 3 strings#2017-05-2418:19jfntnnice#2017-05-2418:19stathissideristhis works because the nested s/cat does not imply a nested sequence#2017-05-2418:20stathissiderisif you want nesting, you need to wrap it with a s/spec:#2017-05-2418:20stathissideris(s/valid? (s/cat :a string? :b string? :rest (s/? (s/spec (s/cat :c string? :d string?)))) ["a" "b"])#2017-05-2418:20stathissideris(s/valid? (s/cat :a string? :b string? :rest (s/? (s/spec (s/cat :c string? :d string?)))) ["a" "b" ["c" "d"]])#2017-05-2418:22jfntnthanks @stathissideris#2017-05-2418:27stathissiderisnp šŸ™‚#2017-05-2419:36stephenmhopperHi everybody, whatā€™s the best way to use clojure.spec with Clojure 1.8?#2017-05-2419:36stephenmhopperIs there a clojar for that?#2017-05-2419:38seancorfieldhttps://github.com/tonsky/clojure-future-spec#2017-05-2419:38stephenmhopper@seancorfield thank you!#2017-05-2420:53mattlyis there any way to dynamically provide arguments from a collection to spec/or ? It's a macro so I can't use apply#2017-05-2420:56donaldballmattly: I have used eval for similar purposes:
(doseq [[spec-kw field-spec desc] fields]
  (eval `(s/def ~spec-kw ~(build-internal-spec field-spec))))
#2017-05-2420:57mattlyyeah, the individual specs I haven't had problems with, what I need is to validate that a value in a map conforms to one of possibly 100 specs#2017-05-2420:59donaldballWell, a way you could do that is to use eval to build the s/or form you want#2017-05-2421:00mattlyhm, thanks#2017-05-2421:11arohneris there a way to get the default generator, so I can update it?#2017-05-2421:12arohner(s/with-gen :foo #(gen/fmap update (s/gen :foo)))#2017-05-2421:23arohnerI guess I can
(s/def :foo* ā€¦) (s/def :foo (s/with-gen :foo* ...))
#2017-05-2421:27Alex Miller (Clojure team)Yeah, having a shorthand for this case is something I think would be useful#2017-05-2421:27Alex Miller (Clojure team)It comes up regularly #2017-05-2522:21jfntnAre recursive specs not supported in Clojurescript? I have (s/def ::parent ::self) (s/def ::self ...) failing at the first line.#2017-05-2523:53donaldballHas anyone written a macro to combine defn and s/fdef yet?#2017-05-2600:15Alex Miller (Clojure team)@donaldball I think maybe @gfredericks did?#2017-05-2600:15Alex Miller (Clojure team)@jfntn not sure#2017-05-2600:16jfntn@alexmiller the workaround was to move the self spec before the parent#2017-05-2600:17Alex Miller (Clojure team)hmm, fails on Clojure too - I think this is related to a couple other existing tickets where s/def chases the spec too early#2017-05-2600:43gfredericksI wrote a defn macro that dispatches on specs matching; it didn't make an fdef#2017-05-2616:04wilkerluciohello people, I wanna share with you a snippet that I wrote here to find bad generators on your project, it's helping me a lot (specially when you have a s/keys filled with items and don't know which of then is not generating)#2017-05-2616:04wilkerlucio
(let [keys (->> (s/registry)
                keys
                (filter #(and (qualified-keyword? %)
                              (not (contains? #{"clojure.core.specs.alpha"
                                                "clojure.spec.alpha"} (namespace %))))))]
  (doseq [k keys]
    (try
      (doall (s/exercise k))
      (catch Exception e
        (println "Bad generator for" (pr-str k))))))
#2017-05-2616:04wilkerluciothis will print Bad generator for :some/key for the generators that are not being able to find a value after 100 tries#2017-05-2616:05wilkerlucioI hope it may help you as it is helping me šŸ™‚#2017-05-2617:33bbsscool!#2017-05-2617:36bbssI'm having an issue generating date-ranges, getting
clojure.test.check.rose_tree.RoseTree cannot be cast to
   clojure.test.check.rose_tree.RoseTree
#2017-05-2617:36ghadilooks like an AOT compilation problem ^#2017-05-2617:36bbssit's in cljc#2017-05-2617:36bbssis that something that could be causing it?#2017-05-2617:37ghadino -- it has to do with whether you have superfluous *.class files present, and whether you've require/reloaded stuff#2017-05-2617:37bbssokay, then it might be because I hotloaded#2017-05-2617:37ghadicalling @hiredman batsignal#2017-05-2617:37bbsslet me check that, thanks#2017-05-2617:38ghadireloading things will redefine some classes. Those classes will have the same name as existing classes, but will be != from the JVM's perspective#2017-05-2617:38bbssmakes sense#2017-05-2617:39ghadithus instances of RoseTree(1) are not instances of RoseTree(2)#2017-05-2617:41hiredmanhmmm?#2017-05-2617:43hiredmanaot drowns kittens in a sack in the river (the kittens are a metaphor for your hopes and deams)#2017-05-2617:47bbsshmm, now I'm running into issues with clj-time.format not being found.#2017-05-2617:48bbssWhich the .coerce namespace depends on, which is weird because when I used the function before I restarted it worked.#2017-05-2617:49bbssIt's really the only kitten that sometimes needs to claw itself to fresh air once in a while.#2017-05-2617:49bbss(metaphor for the only thing that sometimes gets to me with cljx)#2017-05-2618:00ghadiyou're using cljx?#2017-05-2618:00ghadi@bbss#2017-05-2618:01bbss@ghadi is that not short for clojure(script) ?#2017-05-2618:01bbssOh right, it's the pre-conditional reader library#2017-05-2618:01ghadithere was an earlier library called cljx that did something similar to cljc#2017-05-2618:01bbsssorry, it's cljc#2017-05-2618:02bbssI meant to say, the only thing that I find really annoying at times is dependency issues like these. It takes by far the most of my time out of all time-taking things that aren't normal debugging/coding.#2017-05-2618:42bbsslein clean and lein deps solved it#2017-05-2618:42bbssthanks for saving my kittens šŸ™‚#2017-05-2622:22seancorfieldFor folks doing programmatic decoding of explain-data to produce nice messages, it seems that required keys now (in 0.1.123) produce a :pred of (clojure.core/fn [%] (contains? % :the-key)) whereas they used to produce (contains? % :the-key) ā€” this broke a lot of our decoding logic so I figured Iā€™d mention it here in case it helps other debug similar breakageā€¦#2017-05-2622:34arohnergiven an (s/keys :opt-un [::foo]), what is the best way to generate a map with foo always in it? I occasionally see test failures in CI Couldn't satisfy such-that predicate after 10 tries, when I do (gen/such-that :foo (s/gen ::the-map))#2017-05-2700:37athosHappy to hear a new version of spec.alpha has been released!! I've been waiting for CLJ-2059 and CLJ-2085 to be merged šŸ˜†#2017-05-2700:43athosHmm, the version in CHANGES.md looks wrong. 0.1.123 is correct, right? https://github.com/clojure/spec.alpha/blob/master/CHANGES.md#version-01109-on-may-26-2017#2017-05-2701:04seancorfieldYes, 0.1.123 is the correct version.#2017-05-2701:04seancorfieldAlex explained there was a build snafu and they had to re-release 0.1.109 under a new version number.#2017-05-2701:05seancorfieldSo itā€™s technically both 0.1.109 and 0.1.123 šŸ™‚#2017-05-2701:14athosI see šŸ¤”#2017-05-2701:27Alex Miller (Clojure team)Oh, I will update the changes file - I forgot to do that #2017-05-2701:30Alex Miller (Clojure team)Fixed!#2017-05-2702:02gfredericks@arohner what about generating it from a spec where that key is required?#2017-05-2702:22arohner@gfredericks thatā€™s not a terrible idea. I was hesitant because ā€˜in the real worldā€™, the key is optional, and I only needed it required for a specific test case#2017-05-2702:23arohnerI guess thatā€™s fine. (s/def ::foo...) (s/def ::foo-with-bar...)#2017-05-2702:23gfredericks@arohner these are unit tests? #2017-05-2702:23arohneryes#2017-05-2702:23arohner::foo is used in production, ::foo-with-bar would only be used in tests#2017-05-2814:44bbssI'm developing web stuff in cljc with spec and loving it. It's so great to start out with some specs and to generate sample data and build out your front-end from there.#2017-05-2814:45bbssthey can double as react-props with {pre [(valid? spec state)]}#2017-05-2814:48bbssone thing I'm hitting now is I'd like to generate ids so I can give those as unique react keys, which is necessary when react displays lists. Since I will eventually replace the sample data ids with :db/id from datascript, I figured I can add uuid as spec pred and have that generate unique id's. But that doesn't match what datascript generates for ids.#2017-05-2814:49bbssthose are just ints. Is there a way to generate unique ints?#2017-05-2815:14bbssI think I'll have to write my own generator for that#2017-05-2815:15bbssI've tried with s/or, which would work for the validation but it also generates random (non unique) ints. As it should of course šŸ™‚#2017-05-2912:38tianshuwhy spec become spec.alpha?#2017-05-2913:18robert-stuttaford@doglooksgood thereā€™s a post about it in the google group#2017-05-2914:13tianshubut is there any problems with current version of clojure.spec? any feature is missing or critical bug?#2017-05-2914:15mpenetit depends#2017-05-2914:17mpenetI don't think there are "critical" bugs, but there are a few bugs left to fix. missing features: certainly, but that's quite subjective, depends on what you need#2017-05-2914:17mpenetcheck https://dev.clojure.org/jira/secure/IssueNavigator.jspa?reset=true&amp;jqlQuery=project+%3D+CLJ+AND+resolution+%3D+Unresolved+AND+fixVersion+%3D+%22Release+1.9%22+ORDER+BY+priority+DESC&amp;mode=hide#2017-05-2916:37bbssI don't understand why generating this gives stack overflows:
(s/def ::thing string?)

(s/def ::test (s/+ (s/cat :thing (s/* ::thing))))
#2017-05-2916:37bbssmy use case is something like this:
(s/def ::thing string?)
(s/def ::other-thing string?)

(s/def ::test (s/+ (s/cat :thing (s/* ::thing)
                          :other (s/* ::other-thing))))
#2017-05-2916:45wilkerlucio@bbss when using a s/cat inside of another you must wrap it with s/spec#2017-05-2916:45wilkerluciotry this:#2017-05-2916:45wilkerlucio
(s/def ::thing string?)
(s/def ::other-thing string?)

(s/def ::test (s/+ (s/spec (s/cat :thing (s/* ::thing)
                                  :other (s/* ::other-thing)))))

(s/exercise ::test)
#2017-05-2917:07bbss@wilkerlucio thank you, missed that from the docs!#2017-05-3012:18gardnervickersI have a (s/cat ...) spec form that I am using for parsing a seq with s/conform. I want to make this extensible through a multi-spec. Is there a regex op that can help me here? Perhaps using the dispatch key on the multi-spec to tag conformed values?#2017-05-3013:27mishais it ok to actually rely on order of s/or sub-specs?#2017-05-3016:19don.dwoske@stuartsierra I would love to see the clojure.spec definitions used in your recent article : http://blog.cognitect.com/blog/2017/4/6/developing-the-language-of-the-domain We are going through a very similar process now. Your clojure spec and tools, web app to visually explore the models, et. al. are things I wish I had right now. The business analysts are trying to use Avro specifications to describe the domain model, and are immediately hitting limitations of what can be expressed. I am developing a clojure DSL similar to the one in your article to describe our domain, which I can propose as an alternative. Getting my hands on anything you can share would give me a great jumpstart.#2017-05-3017:34stuartsierra@don.dwoske I'm sorry, I do not have permission to share any of that work beyond the examples in the article.#2017-05-3017:36don.dwoske@stuartsierra Sorry to hear that. I enjoyed the article nonetheless, it gave me some good ideas.#2017-05-3017:38stuartsierrathanks, glad it helped#2017-05-3109:27flyboarderhttps://medium.com/degree9/data-validation-schema-spec-5547e33596bd#2017-05-3113:43mishais there a way to highjack a "bottom" component of composed spec to avoid manually writing 2 sets of specs? use case would be: a) spec for datomic entity with both temp and actual ids in refs; b) and the "same" spec, but with actual ids only?
(s/def :datomic/temp-id neg-int?)
(s/def :datomic/actual-id pos-int?)
(s/def :datomic/id (s/or
                     :actual-id :datomic/actual-id
                     :temp-id   :datomic/temp-id))

(s/def :datomic/actual-ref (s/map-of #{:db/id} :datomic/actual-id))
(s/def :datomic/ref        (s/map-of #{:db/id} :datomic/id))

(s/def :foo/bar :datomic/actual-ref)   ;; actual goal is to replace `actual-ref` with `ref` here, and still be able to use both `:entity/foo` specs in different situations
(s/def :entity/foo (s/keys :req [:foo/bar]))

;; is there a way to re-use :entity/foo, and avoid writing following:
(s/def :foo/bar* :datomic/ref)
;^^^ I don't even know how to keep attribute name, but change underlying spec  
(s/def :entity/tepm-foo (s/keys :req [:foo/bar*]))
#2017-05-3114:00danielnealMaybe you could use macros to generate both sorts of specs according to a macros convention#2017-05-3114:05misha@danieleneal the fact, that you can't have same spec-name (:foo/bar above) with a 2 different specs under it at the same time, makes me think I need to somehow hot-swap the bottom (:datomic/ref <--> :datomic/actual-ref) spec definition on-demand. Which, given global nature of registry, might not behave/scale well in e.g. treaded env.#2017-05-3114:07danielnealah so you don't want :foo/bar* to exist ever, even if it is autogenerated#2017-05-3114:10mishaI'd want at least something like
(let [actual (with :entity/temp-foo {:datomic/ref :datomic/actual-ref})] ;; swaping `ref` with `actual-ref` inside :entity/temp-foo
  (s/valid? actual {...}))
#2017-05-3114:14danielnealah ok, I don't know how to do that#2017-05-3114:15misha@danieleneal I don't know yet. But at least one of the problem with duplicating entire "tree" of :entity/tepm-foo here, is you can't have 2 different specs for the same attribute at the same time (AFAIK), so you are forced to do s/or, and then, in app code, go through conform data and see which or branch attribute conformed to. So you can't just use 2 simple (s/valid? :entity/maybe-temp-foo entity) and (s/valid? :entity/only-actual-foo entity)calls in different places. You need to conform, and search for those or branches with custom walkers.#2017-05-3114:16misha+ this is a trivial example, spec tree might be tens of levels deep (imagine nested datomic pull result, where some nodes are pulled, and some are just {:db/id 9999})#2017-05-3114:19mishabut maybe (re-)defining specs on the fly on-demand is ok. Is it, @alexmiller?#2017-05-3114:35Alex Miller (Clojure team)I don't think it's a good idea#2017-05-3114:36Alex Miller (Clojure team)If you want non-conforming s/or, then wrap s/nonconforming around it#2017-05-3114:36Alex Miller (Clojure team)Or maybe it's s/non-conforming, can't remember#2017-05-3114:37Alex Miller (Clojure team)The best way to write a spec is to make true statements about your data#2017-05-3115:16misha@alexmiller how would you go about speccing same entity for validating against these at the same time: a) being actual entity (containing no datomic temp ids), and b) being to-be-transacted entity (maybe containing some temp ids)?#2017-05-3119:31seancorfield@misha This feels to me like you might be over-specifying your data. If the difference between actual-ref and ref doesnā€™t matter in most cases, then donā€™t specify it at all. When and where it actually matters, have a separate spec that can validate you have the one you want.#2017-05-3119:31seancorfieldDeeply nested specs feel like an anti-pattern to me (based on my usage of spec so far in production).#2017-05-3119:34seancorfieldWhen we first got started with spec, I found myself trying to declare every aspect of every known key in a data structure and soon realized that got in the way of writing generic functions across variants of that data. It took a while to settle into a flow of only specifying what was important for the places where specs were being used to help drive the system.#2017-05-3119:38dpsuttonlike unit tests: don't shoot for 100% coverage, shoot for important constraints?#2017-05-3121:21misha@seancorfield @dpsutton yeah, I am sure such questions are a part of learning and calibration process, on the other hand, there might have been a "function for that" after all#2017-05-3121:25mishaMany of existing things still surprise me, and I'd like to avoid prematurely limit myself in approaches/tools ā€“ we are in lisp world after all! :)#2017-05-3121:37jjttjjif I have a set of possible usernames ["fred" "tom" "mary"] and a set of possible mail host domains ["" ""], how do I combine them into a spec generator to make strings like <mailto:/cdn-cgi/l/email-protection|/cdn-cgi/l/email-protection>#2017-05-3121:45misha@jjttjj clojure.test.check.generators/let https://clojure.github.io/test.check/clojure.test.check.generators.html#var-let#2017-05-3123:59Alex Miller (Clojure team)@jjttjj Use generators of s/tuple to combine other generators into random combinations, then use s/fmap to apply a function to the combinations#2017-06-0100:01Alex Miller (Clojure team)like
(gen/sample 
  (gen/fmap (fn [[name host]] (format "%
#2017-06-0100:03Alex Miller (Clojure team)@misha Iā€™d echo Seanā€™s advice -either donā€™t spec the ids, or spec them with an s/or (validation and gen will work better then), or combine an entity spec with an additional id spec if needed in some places, etc.#2017-06-0104:17jjttjjthanks all#2017-06-0106:28stbgzhey all I was wondering if there is a tool around that would take a swagger spec and transform it to a clojure spec#2017-06-0106:29stbgzI am interested in generating test for api specified in swagger#2017-06-0108:34misha@alexmiller is your f/map approach superior in some way to the following? Or is it just a matter of taste?
;;clojure.test.check.generators/let
(gen/sample
  (tgen/let [name (s/gen #{"fred" "tom" "mary"})
             host (s/gen #{"" ""})]
    (format "%
#2017-06-0109:44griersonHow do I define a spec with a generator? e.g. (spec/def ::letter (spec/and (gen/char-alpha) #(Character/isUpperCase %)))#2017-06-0109:46misha@grierson s/with-gen#2017-06-0109:49griersonWhen I try to exercise the def, I get an error#2017-06-0109:50griersonWhen I evaluate the def it works.#2017-06-0109:51andrewmcveighI think the spec/gen in your custom generator is the problem#2017-06-0109:51andrewmcveighIs (gen/char-alpha) not already a generator?#2017-06-0109:51grierson@andrewmcveigh https://github.com/clojure/test.check/blob/master/doc/cheatsheet.md#characters--strings--things#2017-06-0109:52griersonIt works when I (gen/sample gen/char-alpha)#2017-06-0109:52misha
(spec/def ::letter (spec/with-gen
                     (spec/and char? #(Character/isUpperCase %))
                     #(gen/char-alpha)))
(spec/exercise ::letter)
=>
([\S \S] ...)
#2017-06-0109:53grierson@misha ā¤ļø I didn't need the (spec/gen)#2017-06-0109:54griersonI was trying to generate a generator :@#2017-06-0109:54andrewmcveighšŸ™‚#2017-06-0109:54andrewmcveighwhy not?#2017-06-0113:15ikitommi@stbgz have been waiting for someone to do the json schema -> spec converter too. Would also enable things like server code-gen from swagger-spec to a clojure(.spec) web server. Might help the enterprise adoption.#2017-06-0113:16ikitommithere are spec-tools & speculate which do the spec -> json schema. and spec-swagger for spec -> swagger (only 80% done)#2017-06-0116:11danielnealis there a recursive version s/describe which returns definitions of any specs referenced in the top level spec? Kinda like a macroexpand-all?#2017-06-0118:49ikitommi@danieleneal - you could use the spec-visitor from spec-tools: https://github.com/metosin/spec-tools#spec-visitors#2017-06-0117:58royalaidSo I am experimenting with setting up specs for http://aleph.io/manifold/deferreds.html#2017-06-0117:59royalaidBecause I want to try and add specs for aleph responses#2017-06-0118:00royalaidPart of the problem is because deferred are used encapsulate async code and unrealized values the only way I can think of to check the spec is to use a wrapper function that unwraps the value and checks the spec#2017-06-0119:33wilkerlucio@grierson recommendation: use s as the alias instead of spec, I say that because it's a standard name for everybody, going away from well defined conventions is not a good ideia#2017-06-0119:37grierson@wilkerlucio I was following this guideline http://tonsky.me/blog/readable-clojure/.#2017-06-0119:38wilkerlucio@grierson yeah, this article is a bit contrived, I recommend you check this one from Stuart Sierra: https://stuartsierra.com/2016/clojure-how-to-ns.html#2017-06-0119:43grierson@wilkerlucio Thank you, I will check it out.#2017-06-0119:46wilkerlucioyou'r welcome šŸ™‚#2017-06-0119:48bfabry@grierson that article has the right idea but we make exceptions for very-widely-used libraries. in those cases there's often a shorthand convention rather than following the rule#2017-06-0119:50Alex Miller (Clojure team)@misha one benefit of my approach is that I use only generators built from clojure.spec.gen.alpha, which uses dynamic loading, which means that you donā€™t need test.check at runtime to load the namespace. other than that, personal preference.#2017-06-0119:52Alex Miller (Clojure team)@danieleneal no, there is not a deep-describe, but itā€™s something weā€™ve talked about adding. I think having spec specs (CLJ-2112) would be a big help for this to generically walk specs to find sub specs, so Iā€™ve kind of filed it behind that one. Another alternative would be to actually support this as some kind of recursive op built into the protocol.#2017-06-0207:42danielneal@alexmiller great! Yeah spec'd specs would be exactly the thing for that - would make it much easier to write a deep describe#2017-06-0222:23xiongtxIs spec supposed to treat undefined keys as ā€œanyā€?
(s/def ::foo
  (s/keys :req-un [::x ::y ::z]))    ;; x, y, z undefined

(s/valid? ::foo {:x 1 :y "abc" :z clj-uuid/+null+})    ;; => true
#2017-06-0223:31english@xiongtx for s/keys, yes. Presence of the key will always be checked when using req and req-un, but conformance is checked only if specs are registered with the same keys#2017-06-0223:50xiongtxIs that a design decision or just an implementation detail?#2017-06-0223:51xiongtxB/c that seems like it opens the possibility of forgetting to define a key spec and s/valid? still passing in tests#2017-06-0307:41misha@xiongtx if you s/exercise it will complain about "no spec for key".#2017-06-0307:43mishaIt is design decision, so you could opt in to writing detailed spec, instead of being forced to.#2017-06-0320:50Alex Miller (Clojure team)@tclamb seems like a bug, but would be curious to see the actual value causing the error#2017-06-0320:57tclamb@alexmiller s/exercise is generating maps with clojure.lang.LazySeq keys, like {(lazy-seq '(:r)) :r}#2017-06-0320:58tclambthat passes s/valid? and s/conform#2017-06-0320:58tclambI didnā€™t expect this to be true as well: (identical? (s/conform spec x) x)#2017-06-0320:59tclambbut I think thatā€™s an artifact of the map value spec being keyword? instead of e.g. s/cat#2017-06-0321:04tclambI ran into this speccing a map-of point to tile for the board in a game like scrabble#2017-06-0321:55stbgzhey all I am trying to wirite a spec that would satisfy some json data in which the keys have a shape but are not defined eg
{
        "x-identity": "2"
        "x-name": " aaa"
        "x-[some other string]": "bbb",
}
#2017-06-0321:57stbgzCan I use something like
(s/def ::x-object (s/keys ???)
where I can plugin a spec that describes the shape of the keys namely
%(string/starts-with? % "x-")
#2017-06-0321:59stbgzI though about using s/cat but that implies order of the keys#2017-06-0322:19tclambhow about (s/map-of (s/and string? #(re-matches #"x-.+" %)) string?)?#2017-06-0322:21stbgz@tclamb yeah just figured that out#2017-06-0322:21stbgzthanks#2017-06-0322:23stbgznow what if I have a json with a mix of well know keys and patterned keys like the ones above eg
{ 
         "name": .. , 
        "age":...,
        "x-internal"....
        "x-created"... 
}
#2017-06-0322:25stbgzI am modeling the well-know part of the spec with s/keys and the other with s/map-of however s/merge doesnā€™t seem to be working given that the ā€œoptional x...ā€ could be 0, does merge understand optional?#2017-06-0322:28tclambthe docs sound like s/merge only works with s/keys, not s/map-of#2017-06-0322:28tclambhttps://clojure.org/guides/spec#_entity_maps#2017-06-0322:36stbgzhmm this is harder than I though#2017-06-0322:37stbgzI think I have to drop-down to everkv#2017-06-0413:10Alex Miller (Clojure team)merge works with any map spec#2017-06-0413:11Alex Miller (Clojure team)But you do probably need to use every-kv to spec tuples if you're not keywordizing the map keys#2017-06-0413:13Alex Miller (Clojure team)@tclamb seems like a bug if you want to file a jira. The identical thing is expected - spec tries to avoid modification if possible for perf.#2017-06-0519:13wilkerlucio@alexmiller hey Alex, can you tell if there is any progress going on about the namespace aliasing for inexistent namespaces? I remember I read something around this in past, to facilitate having alias used just for namespace resolution of keywords#2017-06-0519:13wilkerluciocurrently I have a bunch of those on my code:#2017-06-0519:13wilkerlucio
(create-ns 'some.deep.nested.keyword.person)
(alias 'person 'some.deep.nested.keyword.person)
#2017-06-0519:13wilkerluciothis is ok on Clojure, but doesn't work for CLJS#2017-06-0519:14wilkerlucioI was wondering if would be a good idea to add something into the ns form, to use like this:#2017-06-0519:14wilkerlucio
(ns my.ns
  (:alias [some.deep.nested.keyword.person :as person]))
#2017-06-0519:14wilkerluciothat would work same way as require, but without actually requiring that namespace to exist#2017-06-0519:15wilkerluciogiven the reason for that namespace is just organization sake#2017-06-0520:57Alex Miller (Clojure team)No update right now, sorry#2017-06-0520:58Alex Miller (Clojure team)But I believe Rich has some idea that he has not shared with me#2017-06-0520:58Alex Miller (Clojure team)In general I think there is a strong preference not to make ns any more complicated#2017-06-0600:20mfikesI was surprised by the interaction of fspec and instrument. I was expecting instrument to only check for spec'd function arg conformance, but it seems to additionally result in "exercising" passed functions. A gist with a concrete minimal example of what caught me off guard, where, in the last two forms evaluated in the REPL session, you can see arg conformance checking, followed by the function I'm passing being exercised: https://gist.github.com/mfikes/3ac5ca668b299e104490059ad0ccfcca#2017-06-0600:24mfikesPerhaps ::number-sink should not be defined using fspec, but instead using ifn?, as in
(s/def ::number-sink ifn?)
#2017-06-0601:24Alex Miller (Clojure team)@mfikes the idea here is that if you spec a function arg, the only way to determine that the function you actually passed conforms is to exercise it. I have certainly used ifn? as an alternate when that is not appropriate.#2017-06-0601:55mfikesMakes complete sense, especially if the functions passed are pure. In my use case they were side-effecting, so ifn? it is. I did like the fact that fspec acts as good documentation, declaring the required shape of the passed functions precisely. Here is my use case and context where this came up, if anyone is interested, which might be typical: https://github.com/mfikes/planck/blob/a7116251a62b05d6ecbeeaff1da809d123ff2b4e/planck-cljs/src/planck/socket/alpha.cljs#L11-L12#2017-06-0608:46Alex Miller (Clojure team)itā€™s possible to do BOTH too by declaring the fspec with the detailed spec, then providing a simpler override spec when you call instrument#2017-06-0611:03degIt seems that (sort (gen/sample (s/gen (s/inst-in #inst "1800-01-01" #inst "2199-12-31")) 100)) has a very strong bias to generate dates within milliseconds of Jan 1, 1970.#2017-06-0613:08wilkerluciothanks for the clarifications Alex#2017-06-0613:33gfredericks@deg would you want a uniform distribution between the two dates you gave?
#2017-06-0613:48deg@gfredericks Yes. (Definitely for date instances. Probably also for numbers where I've specified a range. OTOH, for unbounded numbers, I agree that clustering around zero is probably best)#2017-06-0614:01gfredericksIt's tricky because test.check wants to be able to generate "simpler" instances when asked#2017-06-0614:02gfredericksSo you're describing a generator that doesn't attempt to do that#2017-06-0614:03gfredericksBackground: https://github.com/clojure/test.check/blob/master/doc/growth-and-shrinking.md#2017-06-0614:33degI hear ya. But, I don't believe that Jan 1, 1970 is an appropriate simplification nucleus for dates. The fact that it happens to have an internal rep of zero is not interesting for nearly all uses or tests of dates. Clustering around now is much more likely to catch interesting simple cases.#2017-06-0614:34danielnealis there something like instrument that also checks the :ret value of functions? Just for catching the bad functions I write as close as possible to the problem#2017-06-0614:35degAnd, even for numbers, if I've specified a range, than interesting simple cases are likely to be the two end-points of the range and the mid-point.#2017-06-0614:39zaneWriting your own generators that have that behavior ought to be pretty straightforward.#2017-06-0614:42degSure, but I think this should also be the default behavior.#2017-06-0615:20gfredericks@deg agreed; the unfortunate thing about "now" is it makes the generator nondeterministic#2017-06-0615:48deg@gfredericks But, aren't generators already non-deterministic? I already can't assume that I'll get the same values each time. This change just means that I can't assume the same probability distribution either.#2017-06-0616:00bronsaI might be wrong but I think test.check uses a PNRG for its generators#2017-06-0616:00bronsawhich is deterministic#2017-06-0616:01bronsathat's how you get reproducibility through the test seed#2017-06-0616:15degAh, ok. Then I see the problem. In any event, the number range case can still be improved. And, for dates, it might make sense to center around 2015 or 2020, rather than 1970. To pick one (admittedly contrived) reason: it would be crazed for a test-generator tool written today to only test dates before the y2k problem!!!#2017-06-0616:23degDifferent question.... I just realized that s/tuple needs a vector, and won't validate a list. Why? And, what is an idiomatic way to test for, say, a list of precisely two ints?#2017-06-0616:25degActually, let me rephrase: what is an idiomatic way to test for a list of one int followed by one string?#2017-06-0616:28gfredericks@bronsa correct#2017-06-0617:26Alex Miller (Clojure team)@deg a tuple is inherently positional (indexed) whereas a list is not (it can only be traversed from the beginning).#2017-06-0617:27Alex Miller (Clojure team)but you can use (s/cat :i int? :s string?)#2017-06-0617:27Alex Miller (Clojure team)regex op specs can be matched against sequential stuff (lists, vectors, seqs, etc)#2017-06-0617:30Alex Miller (Clojure team)@deg there is no set of random dates that will be sensical for anyone. if you want to influence it, you should provide a generator that does so (via s/inst-in). inst-in admittedly has a poor generator due to its non-uniform distribution. That is a bug (my fault) and should be fixed.#2017-06-0617:33Alex Miller (Clojure team)looks like I never logged that, so I just did https://dev.clojure.org/jira/browse/CLJ-2179#2017-06-0617:49deg@alexmiller Understood. That Jira case nicely sums up all my concerns about inst-in. I think s/int-in should also be generated uniformly.#2017-06-0617:49Alex Miller (Clojure team)in my head I thought it did! but it does not.#2017-06-0617:49Alex Miller (Clojure team)will fix that as well#2017-06-0617:49degthx#2017-06-0617:50Alex Miller (Clojure team)and I wrote it!#2017-06-0617:50degI'm just starting with spec (looked at it a few months ago, but only seriously these past couple of hours). So far, I like the direction a lot. Great work!#2017-06-0618:02Alex Miller (Clojure team)ok, patch attached that fixes both, hopefully will get a look at some point#2017-06-0623:14Drew Verleecan a spec be-used as a replacement for de-structuring a function? I know can spec/conform inside your function, but it would be nice if could refer to the symbols directly rather then the map. I suppose even with desctructring you need to refer to the keysā€¦#2017-06-0700:07Alex Miller (Clojure team)yes, although due to the performance, I would only recommend doing this in macros (where it happens at compile time)#2017-06-0713:22gfrederickshttps://mobile.twitter.com/gfredericks_/status/872436432459231232#2017-06-0802:03bbrinckIs it expected that maps will match cat specs?
(s/def ::kv (s/cat :k keyword? :v any?))
(s/explain ::kv {"foo" "bar"})
;; In: [0] val: ["foo" "bar"] fails spec: :radiator-react-native.error-messages-test/kv at: [:k] predicate: keyword?
#2017-06-0802:04bbrinckI find it a little confusing because my original data doesnā€™t contain ["foo" "bar"] anywhere#2017-06-0802:25Alex Miller (Clojure team)regex ops match seqable things. maps are seqable and return 2-tuples of k and v#2017-06-0802:25Alex Miller (Clojure team)
user=> (first {"foo" "bar"})
["foo" "bar"]
#2017-06-0802:25Alex Miller (Clojure team)^ your data does contain [ā€œfooā€ ā€œbarā€] when viewed as a seq#2017-06-0802:29bbrinckCorrect me if Iā€™m wrong, but maps are intended to be unordered, right? And regexp ops are about specifying an order? When might I use a regex to validate a map?#2017-06-0802:30bbrinckI suppose if I have a map with zero or one pair, it is ordered and I could use a regex sequence to describe it.#2017-06-0802:39bbrinckFor context, Iā€™m trying to write a pretty-printer that uses explain-data. One part of the pretty printer is a way to highlight where the invalid data ā€œlivesā€ in a larger tree of data. This is tricky in this case (and maybe others) because the :val and :in no longer make sense in the context of the original data, but itā€™s not clear to me how to unwind this transformation or even figure out it occurred.#2017-06-0802:39bbrinckhttps://gist.github.com/bhb/c0009583ead29e1b0fef80581481cab9#2017-06-0804:24Alex Miller (Clojure team)Iā€™m not saying you should use a regex op to spec a map, just explaining how spec interprets that :)#2017-06-0804:27Alex Miller (Clojure team)In: [0] says itā€™s the 0th element of the collection that is the problem. in a map the elements are kv pairs and its this kv pair itself that is indicating the problem (not the particular key or value). what other ā€œinā€ value would help find it better?#2017-06-0804:29Alex Miller (Clojure team)I guess you could constrain the type of the collection that works with regex op specs more to only be sequential?. Then you would get a predicate error at the map level.#2017-06-0804:30Alex Miller (Clojure team)Something like: In: [] val {ā€œfooā€ ā€œbarā€} fails spec :ā€¦ at [] predicate: sequential?#2017-06-0804:33Alex Miller (Clojure team)oh, the other thing is that in the latest version of Clojure spec (not sure if cljs is synced up but I think so), you get an extra key :value in the explain-data that is the original root value#2017-06-0804:34Alex Miller (Clojure team)^^ @bbrinck#2017-06-0809:31pithyless
(spec-tools/explain-data (data-spec/spec ::foo {:foo string?}) {:foo 42})
 Unhandled java.lang.IllegalArgumentException
Don't know how to create ISeq from: clojure.lang.Keyword
#2017-06-0809:32pithyless^ spec-tools seems really useful, but the error messages are useless. Am I missing something? Was this a regression introduced in alpha16?#2017-06-0809:35pithyless^ maybe @ikitommi can offer some advice?#2017-06-0809:48pithylessok, so it works if I use the keyword directly instead.#2017-06-0809:49pithyless
(def foo (data-spec/spec ::foo {:foo string?}))

(spec-tools/explain-data foo {:foo 42}) ;; exception

(spec-tools/explain-data :sample.core$foo/foo {:foo 42}) ;; explain-data works
#2017-06-0814:42bbrinck@alexmiller Thanks for the info! > I guess you could constrain the type of the collection that works with regex op specs more to only be sequential? Thatā€™s a promising idea. In my own specs, Iā€™ve tried your suggestion and wrapped my cat specs with (s/and sequential? ,,,) and that solves the issue.#2017-06-0814:42bbrinckI wonder what the tradeoffs would be for enforcing this at the spec level? AIUI, we already do enforce predicates in some cases namely:
(s/explain (s/keys) :foo) ; val: :foo fails predicate: map?
There may very well be downsides to requiring the value be sequential?, but a few benefits I can see are: 1. IMHO, the error about not satisfying sequential is clearer to me than an error about a key/value structure. 2. You can get into the case where two maps are equal, but one matches a spec while another does not. https://gist.github.com/bhb/b617354b22cd0508d14ccbab4e6f5585 Admittedly, this is a carefully constructed spec designed to exploit this issue. Iā€™m not sure itā€™s a practical concern. But the property of ā€œequal values are equally valid for a given specā€ would seem like a nice invariant to maintain.
#2017-06-0814:42bbrinckIā€™m happy to open a JIRA for this, if it is helpful šŸ™‚#2017-06-0814:43Alex Miller (Clojure team)@bbrinck I was suggesting we could build the sequential? check into the regex op spec itself#2017-06-0814:43bbrinckAh, agreed šŸ™‚#2017-06-0814:43Alex Miller (Clojure team)jira is fine#2017-06-0814:43bbrinckOK, will do#2017-06-0814:44Alex Miller (Clojure team)maps (and sets) are unordered and thus at odds with the value prop of the regex op specs which are about sequentially ordered collections#2017-06-0814:44Alex Miller (Clojure team)so it seems like a good idea to catch that up front rather than accidentally allow something that creates a weird error#2017-06-0814:45Alex Miller (Clojure team)if you really wanted it, you could always seq a map or set before using it#2017-06-0815:04bbrinck@alexmiller Do you find it helpful for me to link to this conversation in the JIRA ticket?#2017-06-0815:08bbrinckIā€™ve created https://dev.clojure.org/jira/browse/CLJ-2183. If Iā€™ve missed anything or created the bug improperly, please let me know#2017-06-0815:55seancorfield"Do you find it helpful for me to link to this conversation in the JIRA ticket?" -- this conversation will be gone in a few days due to the 10,000 message limit in Slack's free tier so I'm not sure you can link to it directly. #2017-06-0816:39bbrinckOh, good point.#2017-06-0816:40bbrinckOK, well hopefully I captured the relevant bits from the conversation in the JIRA ticket.#2017-06-0818:35bmabeyDoes anyone know of existing specs for Avro schemas? I haven't found any on github yet but I figured someone must have done this already.#2017-06-0818:37bmabeyI'm starting to create some now and it is pretty straight forward but I'm not sure how to make a spec conditional on other values in a map. In this case the ::default spec for an avro schema is dependent on the ::type. https://avro.apache.org/docs/1.8.1/spec.html#2017-06-0819:10dergutemoritz@bmabey Check s/multi-spec#2017-06-0911:51lambderis there a way to have :gen-max with s/+ ?#2017-06-0911:52lambdere.g. to limit the size of the collection generated by s/+ spec#2017-06-0911:59Alex Miller (Clojure team)currently no, although I think that would be a good enhancement#2017-06-0912:29lambderthanks @alexmiller#2017-06-0912:32Alex Miller (Clojure team)@linuss line 7 - should be s/and ?#2017-06-0912:33linussHahahaha, wow, thanks!#2017-06-0912:34linussthat's a waste of a morning...#2017-06-0912:34Alex Miller (Clojure team)and not a function?#2017-06-0912:35linussyeah, I must have dozed off while writing that#2017-06-0915:17stbgzHey all, I haven been having problems trying to write a spec as follows. I have a map that looks like this
{
:key1   "val1"
:key2    "
In essence the map is a mix of well know keys(:key1, :key2, :key3) and and optional set of patterned keys (:x-ā€¦). I can easily model either part: (s/keys for the first part and s/map-of for the second part) but I am having a hard time expressing both at the same time. Any ideas on how I can do it?
#2017-06-0916:37Alex Miller (Clojure team)this is sometimes called a ā€œhybrid mapā€#2017-06-0916:37Alex Miller (Clojure team)you canā€™t use s/map-of#2017-06-0916:38Alex Miller (Clojure team)but you can use an s/merge of s/keys and an s/every-kv that specs the map entry tuples#2017-06-0916:38Alex Miller (Clojure team)there is a slightly more complicated example outlined in this blog: http://blog.cognitect.com/blog/2017/1/3/spec-destructuring#2017-06-0916:39Alex Miller (Clojure team)search for ā€œhybridā€ in there to find that part#2017-06-0917:22ikitommi@pithyless hi, and thanks for the report - there was a bug in explain, should be fixed in [metosin/spec-tools "0.2.1-SNAPSHOT"] (which now uses the latest clj, cljs & spec):
(require '[spec-tools.core :as st])
(require '[spec-tools.data-spec :as ds])
(st/explain-data (ds/spec ::foo {:foo string?}) {:foo 42})
; ::s{:problems ({:path [:foo]
;                 :pred clojure.core/string?
;                 :val 42
;                 :via [:user$foo/foo]
;                 :in [:foo]})
;     :spec #Spec{:form (clojure.spec.alpha/keys :req-un [:user$foo/foo])
;                 :type :map
;                 :keys #{:foo}}
;     :value {:foo 42}}
#2017-06-0918:15pithyless@ikitommi - Perfect! I can confirm it fixes my issue. Thanks for the lightning-fast response and apologies for not going through GH issues. I figured I was just doing something wrong šŸ™‚#2017-06-0919:13don.dwoskeI'd like to pass parameters into a predicate function along with the value being checked ... a simple example might be a predicate to validate a integer with a min and max value... i.e. validate x is between 1-10. What are common design patterns for this? First thought is to write a function that returns a closure ... any real-world, slick examples of such a thing?#2017-06-0919:26joshkhis it possible to have something like (s/coll-of :some.other/spec)?#2017-06-0919:29ghadi@don.dwoske there's already some builtins like int-in-range?#2017-06-0919:30ghadierr rather int-in#2017-06-0919:44don.dwoske@ghadi - thanks for the example, that's what I was thinking - write macros or functions which return specs or predicates. https://github.com/clojure/clojure/blob/d920ada9fab7e9b8342d28d8295a600a814c1d8a/src/clj/clojure/spec.clj#L1623#2017-06-0920:03ghadi@don.dwoske yup. fyi all of clojure.spec is being tracked in a different repository now.#2017-06-0920:03ghadihttps://github.com/clojure/spec.alpha/#2017-06-0920:12don.dwoskeAh - of course. In my defense, whenever I search for docs, I end up here : https://clojure.github.io/clojure/branch-master/clojure.spec-api.html and when clicking on "source" for a function.. I end up where I linked to.#2017-06-0920:31misha@joshkh what do you mean?
(s/def :some/spec int?)
(s/def :another/spec (s/coll-of :some/spec :kind vector?))
(s/exercise :another/spec)
=>
([[0 0 -1 0] [0 0 -1 0]]
 [[-1 0 -1 -1] [-1 0 -1 -1]] ...
#2017-06-1014:21stbgzthanks for the help @alexmiller#2017-06-1016:32mikecarterif i have a person comprising of an id (int) and a name (string), how can i write a spec for a collection of people to ensure there are no duplicate IDs?#2017-06-1016:36potetm@mikecarter distinct??#2017-06-1016:38mikecarter@potetm would that catch [{:id 1 :name "Alex"} {:id 1 :name "Frank"}]?#2017-06-1016:48potetmSomething like (apply distinct? (map :id people)) would I think.#2017-06-1016:48potetm
(apply distinct? (map :id 
                      [{:id 1 :name "Alex"} 
                       {:id 1 :name "Frank"}]))
=> false
(apply distinct? (map :id 
                      [{:id 1 :name "Alex"} 
                       {:id 2 :name "Frank"}]))
=> true
#2017-06-1017:13mikecarter@potetm thanks! iā€™ll give it a go#2017-06-1213:26lambderhi All say I have a ns in which I want to define 2 specs , each for map for example:
(s/def ::a (s/keys :req-un [::value]))
(s/def ::b (s/keys :req-un [::value]))
the problem is that both maps require to have key :value but there should be different specs of ::value for each ::a and ::b how do I do it?
#2017-06-1213:56misha@lambder
(s/def :foo/value integer?)
(s/def :bar/value string?)
(s/def ::a (s/keys :req-un [:foo/value]))
(s/def ::b (s/keys :req-un [:bar/value]))
#2017-06-1213:56lambder@misha cool, thanks#2017-06-1214:01misha@lambder a and b in :a/value are not related to ::a in any implicit way, in case you might wonder#2017-06-1214:02lambderok#2017-06-1214:04mishathere, updated it to avoid any confusion#2017-06-1214:53mpenet@alexmiller should we expect a big reveal for your talk in euroclojure, given the recent mentions of "dependency" related work happening? šŸ™‚#2017-06-1214:55Alex Miller (Clojure team)no one expects the spanish inquisition#2017-06-1215:01Alex Miller (Clojure team)there are a few projects I have worked on over the last year in this area and the abstract is sufficiently vague to cover whatever I can talk about by the time I do the talk#2017-06-1215:06misha@alexmiller multispec assocs dispatch value back to the {} during s/exercise, right? is there a way to re-use same enrich {} with a dispatch value, but during s/conform?#2017-06-1215:13mishaoh, I can just call multimethod on an actual {} to get the dispatch value ... assuming defmethod's return keyword, not an inline spec-from#2017-06-1219:18souenzzoI have a query (all-enums-of :my/attr) that return's all my enums. There some how to dynamic gen (s/def :my/attr #{})?#2017-06-1219:30Alex Miller (Clojure team)not sure I understand what the last sentence means - looks like you are trying to spec the empty set, which doesnā€™t make much sense?#2017-06-1219:33souenzzoI want to dynamic gen the spec of :my/attr. In case, it's a enum so I describe it as a set of keys.... Maybe (s/def :my/attr (all-enums-of db :my/attr)) Should express better what I mean...#2017-06-1219:36souenzzoself reply - Yep. I can do a macro, It's a good way? It will be declared after connect with the db. It's a problem?``#2017-06-1219:44Alex Miller (Clojure team)or eval#2017-06-1219:53souenzzoIs it a reasonable solution? "Should I use in production"?#2017-06-1219:54Alex Miller (Clojure team)yes and yes#2017-06-1220:01Alex Miller (Clojure team)
(defn load-enum-specs [db]
  (let [enums (all-enums-of db :my/attr)]
    (eval `(s/def :my/attr ~enums))))
#2017-06-1220:10ikitommiWould be nice to have:
(defn load-enum-specs [db]
  (let [enums (all-enums-of db :my/attr)]
    (s/register :my/attr enums)))
#2017-06-1220:12Alex Miller (Clojure team)well youā€™re just a macro away from that :)#2017-06-1220:12mpenetWould be doable with unform with spec of specs too #2017-06-1220:15Alex Miller (Clojure team)
(defmacro register [name spec] 
  (let [s (eval spec)] 
    `(s/def ~name ~s)))
#2017-06-1220:16Alex Miller (Clojure team)this of course has the significant downside that it only works in a few cases like literal sets#2017-06-1220:19Alex Miller (Clojure team)or maybe wider than just those but not for things like preds, anonymous functions, etc#2017-06-1311:46carocadhey guys, quick question: what is the clojure/spec.alpha project in github? is it an old version of clojure.spec?#2017-06-1311:46gfredericksI think it's the new version#2017-06-1311:47mpenetyes, the fact that the readme mentions nothing about what it is doesn't help tbh (or how to use it; you have to dig into the google group announcement instead)#2017-06-1311:49carocadwait what! so if I update my version of Clojure then it would break šŸ˜ž ?#2017-06-1311:49carocadis it also like that for Cljs?#2017-06-1311:50mpenetyes, you need to update your require calls and potentially clj.spec/invalid kw#2017-06-1311:50mpenetdunno about cljs, but I guess so#2017-06-1311:57hkjelsThis problem had me spinning for close to an hour#2017-06-1311:58mpenetwasted quite some time on it too, even tho I knew what to do#2017-06-1314:37joshkhis it possible to use spec to ensure that one spec'ed key in a map containing a number is larger than the value another spec'ed key?#2017-06-1314:38joshkhstarting with something like this:
(s/def :my.app/start int?)
(s/def :my.app/end int?)

(s/def :my.app/timer
  (s/keys :req [:my.app/start :my.app/end]))
#2017-06-1314:42andrewmcveighYes, you can do something like
(s/def :my.app/timer
  (s/and (s/keys :req [:my.app/start :my.app/end])
         #(<= (:my.app/start %) (:my.app/end %))))
#2017-06-1314:42joshkhthanks. that makes perfect sense.#2017-06-1314:43joshkhs/def takes a keyword and a predicate (or composite of them)#2017-06-1314:46andrewmcveighYeah, the 2nd arg is a spec, which can be specs from the spec namespace, sets, or really any function that returns truthy/falsey, or any combination.#2017-06-1314:47andrewmcveighThe docs say spec, spec-name, predicate or regex-op#2017-06-1314:49joshkhi got as far as Usage: (def k spec-form) and didn't bother to read down šŸ˜‰#2017-06-1314:55joshkhthis is probably a dumb question, but when exercising a spec with a custom function (such as the <= above), is clojure generating ints at random and then only filtering out results that pass the spec? or is it truly generating valid specs on the first try?#2017-06-1314:55joshkhin other words, how hard is it working to exercise the spec?#2017-06-1315:32andrewmcveighs/and uses the first spec as a generator, and the following specs as filters#2017-06-1315:32andrewmcveighas far as I remember#2017-06-1315:33andrewmcveighIf you want something more intelligent, you'd have to provide the generator yourself#2017-06-1315:40joshkhgotcha, thanks again#2017-06-1315:40andrewmcveighnp#2017-06-1316:12mattlyis the plan to move clojure.spec.alpha back to just clojure.spec after it's released?#2017-06-1316:13mattlyI might just park at the version of clojure 1.9 I'm using until it goes back#2017-06-1316:23royalaidhttps://groups.google.com/forum/#!msg/clojure/10dbF7w2IQo/ec37TzP5AQAJ <- @mattly, seems that the plan is to move back to a non-alpha namespace after#2017-06-1316:36Alex Miller (Clojure team)It is likely that 1.9 will release before spec is finalized (that is, 1.9 final will depend on clojure.spec.alpha). Being able to do so is the main reason we did the split.#2017-06-1316:37mattlyok, thanks#2017-06-1316:41joshkhcan someone help me understand why this is producing a vector of vectors rather than a single collection of numbers?
(s/def :test/coll (s/coll-of number?))
(s/exercise :test/coll)
=> ([[-2.0 -1 0 -2.0 -2.0 1.0 0 0.5 0 0 -0.5 -2.0 0.5 -0.5] [-2.0 -1 0 -2.0 -2.0 1.0 0 0.5 0 0 -0.5 -2.0 0.5 -0.5]]
     [[0 0 -3.0 -1 0 -1 -1 -0.75 -1 -1 0 -1 -2.0]...]
#2017-06-1316:42joshkhi would have expected just a list of collections#2017-06-1316:43gfredericksme too#2017-06-1316:43joshkhi also get some NaNs#2017-06-1316:43joshkh[[0 -4.0 0 26 1 -1.25 NaN] [0 -4.0 0 26 1 -1.25 NaN]]#2017-06-1316:44gfredericksdoes NaN pass number?#2017-06-1420:35waffletoweryou probably already answered that for yourselfā€¦
(defn nan? [x]
  (if (number? x)
    (not (= (double x) (double x)))
    true))
#2017-06-1420:36waffletower
(fact "NaNs are true and numbers are false"
  (nan? 0) => false
  (nan? 1) => false
  (nan? 3.4) => false
  (nan? -7.0) => false
  (nan? "yoplait") => true
  (nan? "") => true
  (nan? Float/NaN) => true
  (nan? Double/NaN) => true
  (= (nan? Float/NaN) (number? Float/NaN)) => true          ;; this highlights that (nan?) is not a precise complement of (number?)
  (= (nan? Double/NaN) (number? Double/NaN)) => true
)
#2017-06-1422:46gfredericksI didn't; NaN is a pretty thorny case for generators, primarily because they don't equal themselves#2017-06-1422:46gfredericksotherwise I think it's appropriate for a number? generator to generate them#2017-06-1316:45joshkhi don't even know how you'd represent NaN in clj...#2017-06-1420:47waffletowerjoshkh: Float/NaN and Double/NaN are distinct floating point literals available in clojure. Sorry if I am late#2017-06-1421:01joshkhgood to know! since you seem savvy, how does one use the Infinity literal?#2017-07-2619:03waffletowerOops missed your reply sorry šŸ˜¦ Double/POSITIVE_INFINITY & Double/NEGATIVE_INFINITY#2017-07-2621:00joshkhit's all good! your guidance steered me true. i just had to adjust for javascript which... against all odds... supports positive infinity šŸ™‚#2017-06-1316:45gfredericksit's an instance of Double#2017-06-1316:45joshkhbut i'm still more confused about the deeply nested collections#2017-06-1316:46gfredericksah#2017-06-1316:46gfredericksread the docstring for s/exercise#2017-06-1316:47gfredericksit's not a coincidence that you have pairs of identical collections#2017-06-1316:47joshkhpairs of generated and conformed values for a spec ahhh#2017-06-1316:47dpsuttonit's generating ten collections of things satisfying number?, right?#2017-06-1316:48Alex Miller (Clojure team)@joshkh exercise generates values, and conforms them, then returns a collection of those pairs#2017-06-1316:48Alex Miller (Clojure team)you can also (gen/sample :test/coll) to just get the samples#2017-06-1316:48joshkhthat's the second time i've stared right past the docs today. time for a coffee. thanks guys.#2017-06-1316:48Alex Miller (Clojure team)or (map first (s/exercise :test/coll))#2017-06-1316:49Alex Miller (Clojure team)@mpenet I updated the spec.alpha readme a little#2017-06-1316:53mpenetIt's not pushed yet, right?#2017-06-1316:53Alex Miller (Clojure team)it is pushed#2017-06-1316:54Alex Miller (Clojure team)https://github.com/clojure/spec.alpha/blob/master/README.md#2017-06-1316:57mpenetI was looking at core.specs.alpha :D mixed the two. I guess it might get a readme update as well there#2017-06-1316:57Alex Miller (Clojure team)oh, yeah. will do#2017-06-1317:03mpenetA lot better, thanks#2017-06-1317:05Alex Miller (Clojure team)certainly better than nothing! I meant to come back to those, just forgot to do so after the release.#2017-06-1403:30souenzzo
(s/def ::bar (s/cat :boo string?))
(s/def ::foo (s/cat :foo symbol? :bar ::bar))
This ::foo matches with (my-sym "my-string"). But I'm trying to match with (my-sym ("my-string")). How to describe?
#2017-06-1403:54madstap@souenzzo (s/def ::foo (s/cat :foo symbol? :bar (s/spec ::bar)))#2017-06-1404:02seancorfield@souenzzo By way of explanation, s/cat -- as one of the "regex specs" -- combines with other regex specs by concatenation rather than nesting.#2017-06-1404:24souenzzoseancorfield: I was not remembering/understanding the spec usage. šŸ˜„#2017-06-1414:49odinodinHaven't pushed it yet, more of a proof of concept right now. #2017-06-1414:53richiardiandreaI would be interested in checking, the last missing piece in tooling is something like that#2017-06-1415:24odinodinSure, I will push it some time this week#2017-06-1418:25royalaid@odinodin Looks great! I would recommend looking at other kinds of error reporting that happens in other langs, I hear elm-lang has great error reporting, and I trying to get inspiration from something like that because I donā€™t think there is a real effort around making something like what you have.#2017-06-1418:37odinodinThanks for the tip, will do#2017-06-1508:48thheller@odinodin thats awesome, any chance of a non-html version? šŸ™‚#2017-06-1508:51odinodinthat would be great, but I wonā€™t make it. However, anyone can do it, itā€™s not hard šŸ™‚#2017-06-1508:53thhellerdamn šŸ˜‰ ā€¦ will that be standalone or something integrated in another library?#2017-06-1508:53thhellerprobably tied to reagent?#2017-06-1508:56odinodinWhat Iā€™m working on is tied to reagent, but it is really not that hard to make.#2017-06-1508:57thhellerI tried and failed a couple times ā€¦ Iā€™ll check out your implementation ... maybe I just missed something#2017-06-1508:59thhellerprobably shouldnā€™t have started with extreme errors like this one#2017-06-1508:59thhellerhttps://gist.github.com/thheller/738698dfff45280f4e004df1c46af4ba#file-spec-errors-need-some-work-txt#2017-06-1509:07athosI'm also working on a similar project (https://github.com/athos/Pinpointer), though it's now broken because of the recent radical rewrite.#2017-06-1509:11athosFor me, it's not so easy work, considering some corner cases.#2017-06-1509:14thheller@athos thats awesome and text as well#2017-06-1509:20athos@thheller thank you, and yeah, it's intended to be available from cider.#2017-06-1509:21thhellerIā€™ll definitely check it out#2017-06-1509:26thhellerI added pretty warnings to shadow-cljs recently which works nicely for cljs analyzer errors#2017-06-1509:26thhellerhttps://user-images.githubusercontent.com/144930/27010397-e0adacbc-4ea3-11e7-89be-7b512cf01c53.png#2017-06-1509:27thhellerbut shadow-cljs uses spec to parse ns forms and stuff#2017-06-1509:27thhellerneed something to make those errors prettier as well#2017-06-1509:27thhellerhttps://gist.github.com/thheller/c49f97183405343b93ed71749e5ccca5#2017-06-1509:27thhellercurrently the errors are basically useless#2017-06-1516:42carocad@alexmiller thanks for updating the spec alpha readme. helps a lot šŸ™‚#2017-06-1614:23tap
(defn foo [a] a)

(stest/instrument `foo)

(s/def ::m int?)
(s/def ::a (s/* (s/keys :req-un [::m])))

(s/fdef foo
  :args (s/cat :a ::a))
#2017-06-1614:24tapWhy (s/valid? ::a [{:m 1}]) returns true, but an error is raised for (foo [{:m 1}])?#2017-06-1614:25nwjsmithI think that you want the spec for ::a to be (s/every (s/keys :req-un [::m]))#2017-06-1614:28nwjsmiths/* is a regexp operator, which are used for specifying sequences of data.#2017-06-1614:29nwjsmithIn your example above, an error would be raised for (foo [{:m 1}]), but not (foo {:m 1}) or (foo {:m 1} {:m 2})#2017-06-1614:29nwjsmith(I think)#2017-06-1614:32nwjsmithAh, looks like you'll want to used coll-of instead of every#2017-06-1614:40tapAhh, ok. Thanks @nwjsmith#2017-06-1706:44colinkahnIf I have some data like this [:a :b :c :b :a] where the structure is mirrored, but those could be any values, like [:x :y :z :y :x], what would I use from spec to write that?#2017-06-1715:09weavejester@colinkahn: You could have a predicate like #(= % (reverse %))#2017-06-1715:09weavejester(s/and sequential? #(= % (reverse %)))#2017-06-1715:15colinkahn@weavejester thanks, I realized you could do something similar but my solution wasn't as concise šŸ˜„#2017-06-1715:23colinkahnis there a more declarative way to define it? I was searching around and found recursive regular expressions with this solution: (\w)(?:(?R)|\w?)\1 But that requires you to be able to reference the group, which I don't see a way to do that with clojure.spec. Maybe using tags?#2017-06-1715:27colinkahnthat's from this article: http://www.rexegg.com/regex-recursion.html I know it's for palindromes which wasn't what I originally posted, but curious if you could solve problems in that way using clojure.spec#2017-06-1717:55flyboarderHello everyone, I have a clojure macro that is in a .clj file, I use this macro within .cljs files however it seems I cannot spec this macro???#2017-06-1718:41matanCan someone kindly explain the point of what is called in spec docs "instrumentation"?#2017-06-1718:41matanI am not sure I get what it is that it actually does, or why it is called that way#2017-06-1718:42matanAren't args checked anyway, a la :pre ? if not, what is the rationale?!#2017-06-1718:42matanThanks in advance for clarifying..#2017-06-1718:57weavejester@matan Instrumentation is a little like :pre, except that you can turn it on and off selectively.#2017-06-1718:57weavejesterSo during development you might have error messages that you do without in production.#2017-06-1720:39mobileinkanybody else working on spec-based transformation? use case is representing html head/meta stuff as a clojure map. the task is to transform such a map into html <link>, <meta>, etc. my xform code mimics spec : register a transform for each specked kw, then crawl the validated structure to xform it. this seems like a natural offshoot of spec. i can't be the only person who wants it. what else is out there?#2017-06-1721:19zaneAny idea why cljs.spec.test.alpha/check would return a 0-element vector?#2017-06-1721:42zaneSeems like cljs.spec.test.alpha/check is #{}. Trying to figure out why that is.#2017-06-1722:29zaneWhat's the recommended way to integrate testing the :ret part of function specs in an automated fashion via clojure.test?#2017-06-1722:30zaneHere's one answer: https://stackoverflow.com/questions/40697841/howto-include-clojure-specd-functions-in-a-test-suite Is that the current recommended path?#2017-06-1722:32zaneThere's also this: http://spootnik.org/entries/2017/01/09/an-adventure-with-clocks-component-and-spec/index.html#2017-06-1802:02flyboarderHow can I use a spec on a macro to test for a cljs custom type?#2017-06-1802:18gfredericksprobably rewrite as a macro + function and put the spec on the function#2017-06-1802:18gfredericksmacros can't recognize runtime types#2017-06-1802:49matan@weavejester thanks, so which parts of clojure.spec are ignored when code runs uninstrumented, and which parts always execute?#2017-06-1802:50weavejester@matan All parts are ignored. If you write a spec for a function, it doesnā€™t affect how the function runs unless you instrument it.#2017-06-1803:35matanmmmm now it all makes sense. thanks a lot @weavejester šŸ™‚#2017-06-1803:37matannice stuff, spec, IMO should have been there from the beginning of clojure, as a way of taking/extending the good parts of OO rather than dismissing OO benefits so dichotomously#2017-06-1804:01Alex Miller (Clojure team)I think Clojure has always embraced some parts of OO (like interfaces and polymorphism) while letting some parts go (encapsulation, concrete inheritance)#2017-06-1804:28seancorfield@matan I'm curious: how do you think spec relates to the "good parts of OO"?#2017-06-1804:37seancorfield(and I guess that would also prompt me to ask "What do you think the good parts of OO are?" -- and, full disclosure, I was on the ANSI C++ Standards Committee from '92 to '99, and got started with Java in '97, so with that background I'm often very interested when I hear folks talk on this topic)#2017-06-1906:39matanI think the good part, as it relates to what spec is, is that structure is kept validated (in OO, only in terms of types and composite types, which is what most objects in a program are) at most points. This helps 50% of the programmer population keeping programs in check, although, recently, unit-tests have culturally evolved into another aid in that regard.#2017-06-1906:39matan@U04V70XH6 obviously my thoughts aren't that interesting, I've not been on any committee.#2017-06-1922:09seancorfieldThe problem is that with Java, and several other OO languages, you have no way to separate type and structure ā€” they are conflated. You canā€™t talk about data structures without giving them class/type names and your only form of ā€œvalidationā€ really is the type system. Itā€™s both ā€œnot enoughā€ and ā€œtoo muchā€. So I donā€™t find that to be a good part of OO ā€” I find it to be a failing. The same with encapsulation: it makes it harder to treat data generically and so you end up with a stack of design patterns to workaround the restrictions of the OO type system ā€” and Iā€™ve just gradually found the downsides outweigh the benefits. Mind you, youā€™re right about the ā€œ50% of the programmer populationā€: Java is very effective at allowing large numbers of average developers to work together to build large, complex systems. Java makes developers fungibleā€¦ But I donā€™t think developers should be fungible šŸ™‚#2017-06-2010:54matanI concur on all points made. Just not sure how encapsulation makes it hard to generically treat data#2017-06-2010:57matanIt might seem valid to encapsulate, especially when delivering a library (to make the codebase have a restricted API facade or making it explicit what are the internals of a module vs how to use it)#2017-06-2010:57matanBut as said, I probably need help seeing the negative implications. I am sincerely curious about them#2017-06-2016:13seancorfieldIf you encapsulate your data in a specific type, then you need a function on that specific type to manipulate it. If you want to use a generic function, you have to use the underlying implementation data, ā€œbreakingā€ encapsulation, and then put all the data back afterward. In other words, in Java, if you have an object that contains an array of other data, you canā€™t use generic array functions without exposing the array directly, i.e., unencapsulating it. Clojureā€™s approach is that the ā€œAPIā€ is just data ā€” immutable data ā€” so if the implementation is a vector (array), then you can use any generic sequence function on it.#2017-06-1820:20richiardiandreaNewbie question, why does s/or have keyword branches for predicates and s/and does not? I have never noticed that until now šŸ˜„ I was kind of assuming the same API but go burned there.#2017-06-1820:39gfredericksmy guess is because it's useful for s/or to know which branch matched#2017-06-1820:40gfredericksand not useful for s/and, since you already know that all of them matched (though I suppose you could argue it would be useful to know which one didn't match in the case of an error#2017-06-1820:40gfredericks)#2017-06-1820:47richiardiandreaYep that is true, it would be great to know which one didn't and also to have uniformity in the API I guess#2017-06-1820:56gfredericksthe difference is that one is used for conforming and the other isn't#2017-06-1820:56gfrederickswhether that's a good reason for the API to be different is a different question#2017-06-1901:45flyboarder@gfredericks thanks, I guess thats the difference between runtime specs for cljs#2017-06-1901:46gfredericks@flyboarder the same applies to clj-jvm#2017-06-1901:47flyboardermakes sense#2017-06-1906:40matan@alexmiller @seancorfield what's wrong with encapsulation, from the colujarians' world view? šŸ™‚#2017-06-1906:40matanI don't recall a very convincing blog post, or have just forgotten if I saw one#2017-06-1906:41matanOops should have moved it to #off-topic#2017-06-1906:42matanAnyway I am very happy about spec, it is really a "missing piece" for robust programs that teams can collaborate on#2017-06-1912:15Alex Miller (Clojure team)@matan in OO (particularly thinking of Java / C++ here), encapsulation is used to hide or protect the data inside an object and access it via methods. Because this data is mutable, the accessor/mutator methods are also a place where you can apply a locking strategy to protect that mutable data in multi-threaded usage.#2017-06-1912:16Alex Miller (Clojure team)Clojure turns this inside out and says, data is actually the most important thing here - make it visible, introspectable, and directly manipulable via a generic data interface, rather than via a custom set of API methods that is different for every class.#2017-06-1919:34mobileinkalexmiller: #off-topic an alternative perspective, just for fun: "immutable datum" is just another way of saying "pure function". we could discard the notion of "data" altogether.#2017-06-1912:17Alex Miller (Clojure team)Immutable data structures allow you to do this while also being automatically thread-safe and avoiding the need for locking around access and manipulation of mutable data completely.#2017-06-1912:24gfrederickseven more than thread-safety, you get independence of usage -- you can pass the same object to different pieces of code and not worry about what things they will each "change" that might affect the other which may or may not be what alex's last sentence meant#2017-06-2011:10matangfredericks: sorry for being late to return to the party šŸ˜… I am not sure why I would not worry about different pieces collaborating on the raw data. Or how this is different than the claim of concurrency/thread-safety. #2017-06-2011:26gfredericksthey're not collaborating, they're doing completely independent things#2017-06-1912:27leonoel@alexmiller to make data explicit and visible is undoubtedly a good thing, but there is definitely situations where encapsulation is valuable https://github.com/clojure/clojure/blob/clojure-1.9.0-alpha14/src/clj/clojure/core.clj#L4939#2017-06-1912:29leonoelcore.async channels and go blocks are another good example of encapsulation#2017-06-1912:33Alex Miller (Clojure team)Go blocks are about process, not data. Channels are about conveyance of values, not encapsulation #2017-06-1912:34leonoelsure, but they both have opaque internal state#2017-06-1912:35leonoelwhich doesn't make them bad primitives, btw#2017-06-1912:36leonoelI get your point, but "encapsulation is bad" goes a bit too far imo#2017-06-1912:37Alex Miller (Clojure team)Agreed, but I didn't say that#2017-06-1912:41Alex Miller (Clojure team)Most encapsulation in OO is incidental, not useful#2017-06-1912:44gfredericksthis is not quite the same topic but it just occurred to me that one of my biggest frustrations with legacy OO codebases that I haven't heard articulated before is that the classes I encounter have unclear lifecycles#2017-06-1912:45gfredericksthe thing I want to understand about a class to help me understand the whole codebase is how/why the objects get instantiated, what you do with them while they're alive, how they get discarded, how that relates to the lifecycle of the program as a whole; and I've never seen an OO codebase where that's remotely clear#2017-06-1912:46gfredericksperhaps there's a similar question about how functions are used in a functional codebase, but it seems like less crucial of a question for some reason#2017-06-1912:55Alex Miller (Clojure team)ā€œlifecycleā€ implies some degree of statefulness to me. mutable state requires coordination. all of that is often a mess in OO.#2017-06-1912:56Alex Miller (Clojure team)although Iā€™d say that aspect is a distant second problem for me after the notion of not having a generic data interface and/or clear equality semantics in OO#2017-06-1916:02carocadalexmiller: that is an interesting thought. I am wondering, is there a consensus about ā€œthis is not generic enoughā€. Currently I am working on a program were I need to squeeze every inch of performance that I can get so I ended up defining some custom interfaces to access the data. Nevertheless this puts the clear equality semantics and generic data interface a bit to the limit since the interface is no longer Clojure-like but rather case-specific.
#2017-06-1916:02carocadBtw I used some of the current benchmarks as inspiration šŸ™‚ https://github.com/jafingerhut/clojure-benchmarks/blob/master/binarytrees/binarytrees.clojure-rh.clojure#L18#2017-06-1919:37Alex Miller (Clojure team)I have no problem with breaking every rule in the book in very narrow areas where you are seeking the ultimate performance :) hopefully that is about 0.1% of the code#2017-06-1920:20carocadWell that depends on how you define 0.1% of the code. The loc that use those custom interfaces are very short but the ones that consume it are probably very long and very different. Which is what got me thinking. I think it is a similar situation to what Clojure does i.e defining algorithms and interfaces in java yet using them in Clojure.#2017-06-1912:58gfredericksthe "no generic data interface" complaint also applies to type-oriented functional programming (haskell etc.), right?#2017-06-1912:58Alex Miller (Clojure team)I havenā€™t done enough of it to say#2017-06-1912:59gfredericksthat feels like the biggest difference to me between clojure and most of the rest of FP, even more than static vs dynamic type system#2017-06-1912:59gfredericksthough I'm sure the static vs dynamic dimension is highly correlated with generic vs typey#2017-06-1918:07spiedenyeah not having equality, robust hashing and string representations built into your data structures is terrible#2017-06-1918:09gfredericksI had a giant dump of ruby data the other day and would have appreciated being able to read it into a repl and analyze it, but I just had to decide to not want to do that#2017-06-1919:02mobileinkgfredericks: that must'v hurt like hell.#2017-06-1919:04gfredericksso much #<...>#2017-06-1919:10mobileinkgfredericks: i hate sharps in my dumps.#2017-06-1918:13spiedenone of my complaints with clojure is actually that the default string representation isnā€™t EDN ā€” this decision confuses me. e.g.
(println {:foo "bar"})
{:foo bar}
=> nil
#2017-06-1918:14spiedeni use prn-str for everything, so itā€™s not a big deal#2017-06-1918:16spiedeni guess not all clojure data can be represented as EDN because of nested Java objects, but why not quote ā€œfooā€ above?#2017-06-1918:28gfredericksbecause then (println "foo") would be surprising#2017-06-1918:29gfredericksyou'd need some other mechanism for printing a raw string#2017-06-1918:31spiedenah yeah, that makes sense#2017-06-1918:32spiedeni guess you could special case strings inside of collections#2017-06-1918:33gfredericksthat would bother me even more#2017-06-1918:33gfredericksbut that might be a personality thing; some people like things that are less complicated to describe, others like things that are more likely to do what you wanted#2017-06-1918:34spiedenwell said. i lean towards the former too#2017-06-1918:35gfredericksironically, now that I think about it, I feel like gripes about clojure are pretty evenly split between people wanting the former and getting the latter, and vice versa#2017-06-1918:41spiedena good knifeā€™s edge to balance on =)#2017-06-1918:43gfredericksI suppose there are also gripes about situations where neither ideal is achieved#2017-06-1920:27shaun-mahood@alexmiller: Is there a place on the http://clojure.org site that would make sense for compiling links to blog posts, videos, and other external resources related to spec? There's a bunch of good ones on the Cognitect blog and elsewhere that I wouldn't know how to find if I were looking at it for the first time.#2017-06-1921:39Alex Miller (Clojure team)Depends what they are #2017-06-1921:41Alex Miller (Clojure team)The community/resources page is one place but may not be best for everything #2017-06-1921:41potetm@spieden Just to make sure it's said: pr does exactly what you want.#2017-06-1921:42potetmThe docs actually say: > By default, pr and prn print in a way that objects can be read by the reader#2017-06-1921:45Alex Miller (Clojure team)@shaun-mahood give me examples and I can help find answers#2017-06-1922:05shaun-mahood@alexmiller: Here's the list just gleaned from the Cognitect blog http://blog.cognitect.com/blog/2017/3/24/3xeif9bxaom78qyzwssgwz1leuorh4 http://blog.cognitect.com/blog/2017/1/3/spec-destructuring http://blog.cognitect.com/blog/2016/12/9/works-on-my-machine-self-healing-code-with-clojurespec-1 http://blog.cognitect.com/blog/2016/10/5/interactive-development-with-clojurespec http://blog.cognitect.com/blog/2016/9/29/agility-robustness-clojure-spec http://blog.cognitect.com/blog/2016/9/14/focus-on-spec-combining-specs-with-sor http://blog.cognitect.com/blog/2016/8/24/combining-specs-with-and https://swannodette.github.io/2016/06/03/tools-for-thought A bunch from searching Medium https://product.gomore.com/end-to-end-integration-tests-with-clojure-spec-d4a48cbf92b5 https://medium.com/@rplevy/temporal-higher-order-contracts-with-clojure-spec-e92e795665d https://medium.com/degree9/data-validation-schema-spec-5547e33596bd There are a bunch of conference videos as well, plus the spec screencasts on ClojureTV One other thing that I haven't seen anywhere are the notes from your spec workshop, though I don't know if that's meant to be publicly available. If having any of this stuff (or type of stuff) is desirable, I'm happy to work on a PR for it. I know I've seen other interesting and exciting stuff on Spec elsewhere too.#2017-06-1922:10shaun-mahoodI just noticed your previous reply - the main motivation for my question was realizing that if I only look at http://clojure.org as it stands right now, none of the really interesting or exciting use cases jump out at me. Might not be a goal though, so I didn't want to put any work into a PR without figuring out what kind of thing might be useful for others.#2017-06-1922:33naomariki could appreciate such a compilation#2017-06-2005:26Alex Miller (Clojure team)@shaun-mahood broadly, I think itā€™s hard to give a list of blog entries the right context to be useful and survive well over time. It just doesnā€™t make sense to try to track every spec blog (most of which are now out of date). Whatā€™s useful on the site is curated reference or tutorial content that is kept up to date. So I guess largely, I would say these should not be added to http://clojure.org. One thing that I have been (very slowly) working on is a reference page for spec which I think is a good complement in between the rationale, the guide, and the api docs.#2017-06-2005:30Alex Miller (Clojure team)and the spec workshop notes are not intended to be public - they are training materials copyright Cognitect and represent probably 100 hours of work over the last year.#2017-06-2005:38shaun-mahoodThat sounds reasonable to me. I love the idea of a reference page - I'm still trying to come to grips with the boundaries of the new clojure and clojurescript site. #2017-06-2005:41shaun-mahoodI hope the workshop notes at least made the book easier to write.#2017-06-2005:43shaun-mahoodIt was definitely worthwhile to take at last years Conj, I would consider taking it again to pick up the new stuff and the bits I missed. #2017-06-1921:50spieden@potetm yes thanks#2017-06-2011:17matanSorry but after all the discussion about encapsulation, I still find that hiding (encapsulating) internal implementation of "complex" mechanisms (not silly field values like in Java) is a reasonable thing for helping others/self use of components without shooting one's own foot or scanning dozens of fns to figure what is meant to be used from the outside#2017-06-2014:17Alex Miller (Clojure team)matan: well, youā€™re arguing against something I didnā€™t say. I didnā€™t say that all encapsulation was bad, I said needless encapsulation of data is bad and OO as typically practiced encapsulates all data.#2017-06-2014:30donaldballWhen I have a complex side-effecting machine with a simple use interface, I reach for a protocol. You can also convey intentions about var usage with namespace design.#2017-06-2014:47colinkahn@U04V4HWQ4 whatā€™s an example of using namespace design? I might be overthinking the meaning, but I havenā€™t heard that term before.#2017-06-2014:50donaldballMaybe better examples but one that jumps to mind is core.async, where there is a public api (clojure.core.async) and a bunch of impl namespaces (e.g. clojure.core.async.impl.buffers)#2017-06-2011:18matan(and that's why we have namespace private declarations I think)#2017-06-2014:12Alex Miller (Clojure team)matan: note that even private vars are still accessible - anyone can obtain and use them still. The private meta is really more documentation than encapsulation.#2017-06-2015:21matanRight! #2017-06-2013:47bbqbaronthatā€™s a good insight; i think the clojure difference is probably avoiding 1. colocating state with logic in memory 2. requiring encapsulated fn A to bring B, C, D, E, F and all their dependencies with it#2017-06-2014:05joshjones@matan this belongs in #off-topic since this is not about clojure.spec (or even clojure itself) FYI#2017-06-2019:34chilleniousHow would I rewrite this to avoid duplication? (s/def ::PrimaryContact (s/keys :req-un [::FirstName ::LastName ::Email] :opt-un [::Title ::Company ::Type ::Phone ::Fax])) (s/def ::SecondaryContact (s/keys :req-un [::FirstName ::LastName ::Email] :opt-un [::Title ::Company ::Type ::Phone ::Fax]))#2017-06-2019:43Alex Miller (Clojure team)
(s/def ::Contact (s/keys :req-un [::FirstName ::LastName ::Email]
                                :opt-un [::Title ::Company ::Type ::Phone ::Fax])
(s/def ::PrimaryContact ::Contact)
(s/def ::SecondaryContact ::Contact)
#2017-06-2019:45chilleniousah, well that's easy... thanks!#2017-06-2102:32bbrinckIs multi-spec intended to work with the :default implementation of the multimethod?#2017-06-2102:35bbrincki.e. is it intended that I could use the :default implementation to provide a default spec? https://gist.github.com/bhb/7eefe3034a0622b6499a5b5dd2f7e52a#2017-06-2104:39Alex Miller (Clojure team)yes, this should work (it does in Clojure)#2017-06-2104:41Alex Miller (Clojure team)so, that's a bug in cljs spec#2017-06-2104:45Alex Miller (Clojure team)actually, that's already been fixed if you update your cljs version#2017-06-2115:44bbrinck@alexmiller Ah, shoot, I should have checked JIRA. Thanks for the info!!!#2017-06-2118:58royalaidIs there a currently recommended way to integrate clojure.specs generative testing with clojure.test?#2017-06-2209:17carocadroyalaid: I wrote a similar macro for fdef which is basically a simplification of that one. Here it is:
(defmacro defspec-test
  ([name sym-or-syms] `(defspec-test ~name ~sym-or-syms nil))
  ([name sym-or-syms opts]
   `(t/deftest ~name
      (let [check-results#  (clojure.spec.test/check ~sym-or-syms ~opts)]
        (doseq [result# check-results#]
          (t/is (nil? (:failure result#)) (str (clojure.spec.test/abbrev-result result#)))
          (when (nil? (:failure result#))
            (println "[OK] - " (clojure.spec.test/abbrev-result result#))))))))
#2017-06-2209:17carocadyou would use it like this: https://github.com/n7a235/data.hypobus/blob/dev/test/hypobus/basic_test.clj#L27#2017-06-2118:59royalaidI have found this https://stackoverflow.com/questions/40697841/howto-include-clojure-specd-functions-in-a-test-suite#2017-06-2118:59royalaidBut it seems a bit hacky and the output on failure isn't very helpful#2017-06-2119:03mattly@royalaid I can't speak to clojure.spec, but with test.check I use test.chuck, which has a macro checking that behaves similar to testing#2017-06-2119:03mattlyI'd link you to it except slack's electron app decided I can't use my clipboard anymore#2017-06-2119:10sneakypeetHi all. started using spec in cljs today. I am running into a weird issue
(ns tenandsix.app.flow
  (:require-macros [cljs.spec :as s])
  (:require  [cljs.spec :as spec]))

(s/def :flow/action-type keyword?) => works
(s/def :flow/action (s/tuple keyword? keyword?)) => No such namespace: s, could not locate s.cljs, s.cljc, or Closure namespace ""
#2017-06-2119:11dpsutton:flow vs :flo possibly?#2017-06-2119:12sneakypeetno that was actually me testing if the keyword made a difference šŸ™‚#2017-06-2119:12sneakypeetbut no that does not fix it#2017-06-2119:13sneakypeetany thoughts? I mean this is as basic as it gets#2017-06-2119:13dpsuttonthoughts, perhaps s/tuple is a macro and the import macro statement is malformed#2017-06-2119:14dpsuttonbut only when actually trying to macroexpand#2017-06-2119:14dpsuttonhttps://github.com/clojure/clojurescript/blob/master/src/main/cljs/cljs/spec/alpha.cljc#L395#2017-06-2119:15dpsuttonlooks like tuple is a macro#2017-06-2119:15sneakypeetit is yes.#2017-06-2119:16sneakypeetso actually. it works, but my repl gives the error, when I use the spec it works#2017-06-2119:18dpsuttoni don't understand your statement#2017-06-2119:18sneakypeetI'll try again#2017-06-2119:18dpsuttonif you're on a recent cljs, it looks like you no longer need a require-macros part#2017-06-2119:19dpsuttontry just requiring spec#2017-06-2119:19dpsuttonhttps://groups.google.com/forum/#!topic/clojurescript/FoiqNV5nunQ#2017-06-2119:19sneakypeet1. evaluating that actually creates the spec#2017-06-2119:19sneakypeet2. I can use the spec#2017-06-2119:19sneakypeet3. It gives the error when I create the spec#2017-06-2119:20sneakypeetchecking the link#2017-06-2119:22sneakypeet@dpsutton thanks your link helped#2017-06-2119:22sneakypeetthats a pretty big improvement#2017-06-2121:27adamfreywhat's the cleanest way to make a generator that constantly returns the same value? Similar to constantly in core?#2017-06-2121:30adamfreyI came up with this, which works. I just wanted to make sure I wasn't missing an easier way
#(gen/fmap (constantly 1)
       (gen/int))
#2017-06-2121:33gfredericks@adamfrey (gen/return 1)#2017-06-2121:33adamfreythanks!#2017-06-2121:33gfredericksprobably with an extra # in a spec context#2017-06-2123:24Alex Miller (Clojure team)@adamfrey I would say (s/gen #{1})#2017-06-2201:27madstapIn Rich's speculation keynote he mentioned that there are ways to check if backwards compatibility is broken in both set theory and regex theory. The set one would presumably be something like:
(defn breaking-change? [old-required-keys new-required-keys]
  (not (set/subset? new-required-keys old-required-keys)))
Are there any links to info about the regex part? Or am I misunderstanding something?
#2017-06-2202:05Alex Miller (Clojure team)madstap: the regex part is based on Matt Mightā€™s papers with regex derivatives and I believe Rich has spent enough time with it to believe that the math there makes this computable. Iā€™m having a strong sense of deja vu - have you asked this before?#2017-06-2202:13madstapI'm pretty sure I haven't. Thanks for the pointers šŸ™‚#2017-06-2203:14Alex Miller (Clojure team)Must have been someone else, or my addled brain#2017-06-2201:35sooheonHey is there function to coerce all keys in a map to namespace-qualified keys, other than the #::{} syntax? Iā€™m receiving an unqualified map from an API call, and would like to spec the result.#2017-06-2201:37jjttjj@sooheon if you just need to spec un-namespaced keys you can just do (s/keys :req-un [::my-key1 ::my-key2]) to require those keys un-namespaced, ie {:my-key1 "x" :my-key2 "y"}#2017-06-2201:39sooheonThanks, I did see that in the docstring, and it did work#2017-06-2201:39sooheonBut now when I pass around this map and assoc my own keys to it, it will have a mix of qualified and un- keywordsā€¦#2017-06-2201:41jjttjjs/keys can take vectors for both :req-un and :req if you have both namespace qualified and unqualified keys#2017-06-2201:41sooheonYeah, itā€™s just a bit of overhead for me#2017-06-2201:41sooheonwould have preferred to just go all namespaced, you know?#2017-06-2201:41sooheonThanks anyways :)#2017-06-2202:02seancorfield@sooheon You could just use reduce-kv and produce a new map from the old one with namespace-qualified versions of the keys (assuming you want the same qualifier on all of them).#2017-06-2202:03seancorfieldSomething like (reduce-kv (fn [m k v] (assoc m (keyword "qualifier" (name k)) v)) {} api-result)#2017-06-2202:04sooheonah cool! hadnā€™t thought of that. Was just hoping something like (into #::{}) would magically work#2017-06-2202:04sooheonis this something that is recommended, btw? to use qualified keywords for everything?#2017-06-2202:05seancorfieldI think itā€™s a reasonable approach for handling domain data within your system, yes.#2017-06-2202:06seancorfieldBear in mind thoā€™ that you probably want to keep those API result keys separate from your actual domain keys in order to avoid conflicts.#2017-06-2202:06seancorfieldSo you might have :external.api/name and :my.domain/name with different specs.#2017-06-2202:08seancorfieldThis is one of the reasons that the latest java.jdbc lets you specify :qualifier in the options on almost every call so you can get namespace-qualified keys in your SQL result hash maps.#2017-06-2202:08sooheonI see. I think itā€™ll just take some experience (just like knowing how to split up and organize nsā€™es in the first place)#2017-06-2202:09sooheonthanks!#2017-06-2202:18seancorfieldBear in mind that the namespace-qualifiers on keywords do not need to correspond to actual code namespaces so you have a lot of flexibility.#2017-06-2202:19seancorfieldFor example, we use a qualifier of wsbilling for all entities that relate to our (World Singles) billing system but we donā€™t have a namespace called wsbilling.#2017-06-2202:21seancorfieldWhen we accept JSON/Clojure data from outside (APIs, databases), we can normalize it to whatever qualifier we want to distinguish it, within our system, from our billing data.#2017-06-2202:22seancorfield(if this was a public Clojure API weā€™d use a reverse-domain-name prefix, like com.worldsingles.billing, I expect)#2017-06-2202:23seancorfieldYou can choose how to keep unique groups of entity names separate from each other.#2017-06-2202:23joshjones@sooheon @seancorfield I made a function a while back to namespace-qualify a map. here's what i have, if it's useful to anyone
(defn map->nsmap
  "Creates a namespaced-map from a standard one and a namespace (obj or string)"
  [m n]
  (reduce-kv (fn [acc k v]
               (let [new-kw (if (and (keyword? k)
                                     (not (qualified-keyword? k)))
                              (keyword (str n) (name k))
                              k) ]
                 (assoc acc new-kw v)))
             {} m))
#2017-06-2202:23sooheonAh great, thatā€™s very useful#2017-06-2202:25sooheon@seancorfield Iā€™ve still got to experience first hand how ns qualifying keys works with stuff like destructuring, and making sense of how I want to organize things, but thanks for the pointers#2017-06-2202:25seancorfieldThatā€™s good for preserving existing name-qualification. Sometimes you need to override that. And of course my simplistic example only works for keys that you can call name on (and assumes you want all-keyword keys back out)!#2017-06-2202:25seancorfield@sooheon Yeah, it takes some getting used to, after so many years of unqualified keywords!!#2017-06-2202:27seancorfieldIn terms of destructuring:
user=> (let [{:foo/keys [a b c]} {:a 1 :foo/b 2 :c 3 :foo/c 4}] (println a b c))
nil 2 4
#2017-06-2202:27joshjonesyes, i wrote it to behave the way the #: reader macro behaves, namely, it does not disturb non-qualified keys#2017-06-2202:27joshjones
#:foo{:a 1 :bar/b 1}
=> {:foo/a 1, :bar/b 1}
#2017-06-2202:27sooheonyou mean already qualified ;)#2017-06-2202:28joshjonesyes, sorry#2017-06-2202:28joshjonesand non-keyword keys#2017-06-2202:28joshjones
#:foo{:a 1 :bar/b 1 42 100}
=> {:foo/a 1, :bar/b 1, 42 100}
#2017-06-2202:28seancorfieldalso
user=> (let [{:keys [a b foo/c]} {:a 1 :foo/b 2 :c 3 :foo/c 4}] (println a b c))
1 nil 4
So you can put qualifiers in the key vector ā€” and you get unqualified symbols back out!
#2017-06-2202:29sooheonhuh#2017-06-2202:29sooheonhow would you use the :: sugar w/in destructuring#2017-06-2202:30seancorfieldLike this:
user=> (let [{::keys [a b c]} {:a 1 :user/b 2 :c 3 :user/c 4}] (println a b c))
nil 2 4
Note that weā€™re in the user namespace so ::keys means :user/keys
#2017-06-2202:31seancorfieldYou can also do this (which Iā€™m surprised is allowed):
user=> (let [{:keys [a ::b ::c]} {:a 1 :user/b 2 :c 3 :user/c 4}] (println a b c))
1 2 4
#2017-06-2202:32joshjonesactually it seems to work by putting the :: in the vector of keys too, though i'm not sure it's a good idea#2017-06-2202:32joshjones
(let [{:keys [a b c]} {:a 1 :foo/b 2 :c 3 ::c 4}] (println a b c))
1 nil 3
=> nil
(let [{:keys [a b ::c]} {:a 1 :foo/b 2 :c 3 ::c 4}] (println a b c))
1 nil 4
#2017-06-2202:32sooheonso the last example would be the way to mix and match ::b :foo.bar/c and :d#2017-06-2202:32seancorfieldI expected that to be an error but apparently you can use keywords in the key vector? @alexmiller Is that supposed to be valid?#2017-06-2202:33sooheonYou canā€™t have multiple :keys ::keys and :foo/keys deconstructions separately at once, right? Youā€™d do {:keys [a ::b foo/c]}#2017-06-2202:55seancorfieldThatā€™s my understanding, yes.#2017-06-2203:10Alex Miller (Clojure team)@seancorfield yes, we support keywords there so that you can use autoresolved keywords #2017-06-2203:11seancorfieldIs that documented? (I guess the docs could have been updated since I last looked at themā€¦ šŸ™‚ )#2017-06-2203:11Alex Miller (Clojure team)And you can have multiple keys destructurings at the same time for different namespaces#2017-06-2203:12Alex Miller (Clojure team)It was in the changelog when it went in but that was a while ago#2017-06-2203:12seancorfieldYup, the destructuring guide contains examples.#2017-06-2203:13seancorfieldNot of ::k directly but of ::p/name#2017-06-2203:13Alex Miller (Clojure team)Rich and I worked on some updated docs for destructuring but I can't remember if those actually got all the way done#2017-06-2203:16seancorfieldItā€™s fairly comprehensive, TBH. Thank you. And Rich.#2017-06-2215:48hlship@alexmiller From my issue in the tweet, here's the tail end of a very long stack trace:
clojure.spec.alpha/regex-spec-impl/reify/gen*                 alpha.clj: 1672
                        clojure.spec.alpha/re-gen                 alpha.clj: 1601
                              clojure.core/every?                  core.clj: 2572
                                clojure.core/next                  core.clj:   64
                                              ...                                
                              clojure.core/map/fn                  core.clj: 2657
              clojure.spec.alpha/re-gen/ggens/gen                 alpha.clj: 1582
                        clojure.spec.alpha/re-gen                 alpha.clj: 1597
                        clojure.spec.alpha/gensub                 alpha.clj:  275
                             clojure.core/ex-info                  core.clj: 4617
clojure.lang.ExceptionInfo: Unable to construct gen at: [:exception] for: (instance? java.lang.Throwable %)
    clojure.spec.alpha/failure: :no-gen
       clojure.spec.alpha/form: (clojure.core/fn [%] (clojure.core/instance? java.lang.Throwable %))
       clojure.spec.alpha/path: [:exception]
#2017-06-2215:49hlshipWhat I don't get is why it is trying to create a generator. It just supposed to be validating inputs into our schema/compile function.#2017-06-2215:49hlshipBasically, the function I'm passing in doesn't have a fspec, so I'm going to add one and see if that helps.#2017-06-2215:53hlshipNope. Bad guess. Providing my own fspec didn't work.#2017-06-2216:25Alex Miller (Clojure team)Right, so that was my guess#2017-06-2216:25Alex Miller (Clojure team)the way that fspec args are validated in instrumentation is by using the fspecā€™s generator to generate values and invoke the function#2017-06-2216:26Alex Miller (Clojure team)and check the ret and fn spec to verify the function with random inputs is valid according to the fspec#2017-06-2216:28Alex Miller (Clojure team)thus all fspecā€™s should have an args spec that generates#2017-06-2216:28Alex Miller (Clojure team)the surprising bit here is that they need that not just for check, but also for instrument#2017-06-2216:29Alex Miller (Clojure team)same is also true of fdef for check, but not for instrument#2017-06-2216:29Alex Miller (Clojure team)(because checking an fdef involves generating using its args generator)#2017-06-2216:36hlship(in a meeting, will digest and get back to you)#2017-06-2216:41mpenetJust wondering, why not just wrapping args/ret of fspec fns with s/valid? when instrumented #2017-06-2216:42mpenetIt would fail "later" (invoke time) but might be more intuitive (and faster)#2017-06-2216:46mpenetCould be an option#2017-06-2217:05Alex Miller (Clojure team)not sure if Rich considered that or not, havenā€™t talked to him about it#2017-06-2312:35mpenetalexmiller: just wondering: wouldn't it be better to keep https://dev.clojure.org/jira/browse/CLJ-1936 open then?#2017-06-2217:31mpenetNot the first time this comes up: clj-1936#2017-06-2217:44hlshipSo I'm a bit lost .... is there no way I can simply declare what my callback should look like without coming up with a generator for the callback?#2017-06-2219:41Alex Miller (Clojure team)hlship: in short no, if you want to also validate the fdef. other options are to override the fdef args generator or to use a generator override on instrument#2017-06-2217:44hlshiphttps://dev.clojure.org/jira/browse/CLJ-1936#2017-06-2218:17mpenetYou can use fn? (meh)#2017-06-2219:27hlshipmpenet: Yes, that's where I'm headed.#2017-06-2219:40Alex Miller (Clojure team)actually, ifn? is better if you go that route#2017-06-2218:36hlshipI almost don't care about validation as much as documentation of my Argos and result. #2017-06-2219:18arohner
(gen/sample (s/gen (s/spec ::clojure.core.specs.alpha/map-bindings)))
ExceptionInfo Couldn't satisfy such-that predicate after 100 tries.  clojure.core/ex-info (core.clj:4725)
#2017-06-2219:20arohner@gfredericks in core.specs.alpha#2017-06-2219:20arohnerthe clojure.core specs that got split out into a separate lib when spec got split out#2017-06-2219:20arohner@alexmiller Iā€™m confused by the definition of ::core.specs/map-bindings. What does it mean for (s/every ... :into {}) I donā€™t see a guarantee that the every coll has an even number of elements, nor a requirement that the input coll is a map#2017-06-2219:20arohner(this is coming up because Iā€™m trying to run spectrum inference on clojure.core)#2017-06-2219:22gfredericksif you're wondering where the such-that error is coming from, I suspect https://github.com/clojure/core.specs.alpha/blob/c33689f75dbe3188ee00b32bb798bcd0cfd6cacc/src/main/clojure/clojure/core/specs/alpha.clj#L36#2017-06-2219:22gfredericks@arohner every branch of map-bindings seems to be a pair, does that explain your confusion?#2017-06-2219:23arohnerpartially#2017-06-2219:28arohner(s/conform ::clojure.core.specs.alpha/map-bindings '[[foo bar]]) => [[foo bar]]#2017-06-2219:28arohnerit seems very weird to me that :into is used for generating, but the resulting conform doesnā€™t change#2017-06-2219:29arohnerthe real source of my trouble is just below, when I have to handle (s/def ::map-binding-form (s/merge ::map-bindings ::map-special-binding))#2017-06-2219:29arohnerwhere s/merge takes maps#2017-06-2219:43Alex Miller (Clojure team)sorry, just reading this now. where is your question at now?#2017-06-2219:44Alex Miller (Clojure team)note that s/merge doesnā€™t flow conformed values so only the last map spec in the merge will matter for the conformed value#2017-06-2219:44arohner::map-binding-forms takes two maps, because of s/merge, but map-bindings is not guaranteed to return maps, because it uses :into and not :kind#2017-06-2219:49Alex Miller (Clojure team)I donā€™t agree with the last clause there?#2017-06-2219:50Alex Miller (Clojure team)both :into and :kind affect s/every gen#2017-06-2219:50arohner(s/conform ::clojure.core.specs.alpha/map-bindings '[[foo bar]]) is that expected?#2017-06-2219:51Alex Miller (Clojure team)Iā€™d say thatā€™s not a supported map binding form#2017-06-2219:51Alex Miller (Clojure team)I guess we could use :kind to narrow that#2017-06-2219:51arohnerright, spectrum is currently picky about that, and Iā€™m trying to understand it / make it work#2017-06-2219:52arohnerspectrum says map-binding isnā€™t necessarily a map, so the merge isnā€™t guaranteed to work in all cases#2017-06-2219:58Alex Miller (Clojure team)yeah, that seems good. if you want to file a jira, we can try to get that in next release#2017-06-2220:01arohnersure. in CLJ, or does spec have itā€™s own project now?#2017-06-2220:01Alex Miller (Clojure team)CLJ#2017-06-2220:01Alex Miller (Clojure team)if you want to make a patch, that will be faster than me making it (as I can screen it)#2017-06-2220:02Alex Miller (Clojure team)patch against core.specs.alpha of course#2017-06-2220:08arohnersure#2017-06-2221:40peejaWhat's the best way to define a function :args spec for a function which takes a single argument?#2017-06-2221:46joshjones@peeja - same way as you do for any :args spec
(s/fdef your-func
        :args (s/cat :single-arg some-pred?)
...)
#2017-06-2221:46peejaSo, still use s/cat then?#2017-06-2221:46joshjonesyes, as it's still a vector of one element#2017-06-2222:06Alex Miller (Clojure team)agreed and to take that a step further, itā€™s useful to use a regex op (s/cat) here because the conformed value will be a map with the arg name as the key, and thatā€™s useful in the :fn spec or in explanations#2017-06-2312:02odinodingiven a spec (obtained from ex-data of an exception thrown by inspect, i.e the :cljs.spec.alpha/spec), how can one obtain the key of that spec in the registry?#2017-06-2312:02odinodinI want to go from spec -> the name of the fdefā€™ed function that failed#2017-06-2312:28odinodinMy current solution is to parse it out of the exception message, which contains ā€œCall to #ā€˜name-of-var did not conform to specā€¦ā€œ. It would be nice if instrument included the key in the ex-data, but I might be misunderstanding something here.#2017-06-2313:13wilkerlucio@odinodin can you give please show the data you are trying to go from -> to?#2017-06-2313:13wilkerlucioI think you might want to use (s/form) or (s/get-spec), but I'm not sure yet what you are trying to get#2017-06-2313:16odinodin@wilkerlucio it is a spec record object, as provided by the exception thrown by instrument when a fdefā€™ed function is called with non-conforming input.#2017-06-2313:18odinodinit is the spec object that (s/get-spec 'the-fdefed-var) would return as far as I can tell#2017-06-2313:19wilkerluciothat's a long chain, hehe, I'm not sure what kind of data you are looking at, can you paste what your (ex-data) result looks like?#2017-06-2313:24odinodinThis is what (ex-data) returns:
{:cljs.spec.alpha/problems ({:some-description-of-the-problem "..."})
 :cljs.spec.alpha/spec #object[cljs.spec.alpha.t_cljs$spec$alpha56508],
 :cljs.spec.alpha/value ({:foo "bar"}),
 :cljs.spec.alpha/args ({:foo "bar"})
 :cljs.spec.alpha/failure :instrument}
#2017-06-2313:25odinodinI want to know what the :cljs.spec.alpha/spec object is referring to#2017-06-2313:26odinodinor it would probably be more correct to say, what key points to this spec in the registry#2017-06-2313:28wilkerlucio@odinodin gotcha, you can try calling (s/form) and (s/get-spec) on it, but I'm not sure if will work#2017-06-2313:28odinodinIā€™ll try that#2017-06-2313:29odinodinthanks#2017-06-2313:39odinodin@wilkerlucio (s/form) only returns the spec as defined on the (s/fdef) var. Iā€™ll just parse the var out from the error message instead. Anyway, thanks for your suggestions šŸ™‚#2017-06-2313:39wilkerluciono problem, I'm glad you can at least work around it, maybe other people will have a better solution šŸ˜‰#2017-06-2314:50Alex Miller (Clojure team)@odinodin there is a ticket filed for this at https://dev.clojure.org/jira/browse/CLJ-2166 - patches welcome#2017-06-2314:50Alex Miller (Clojure team)I just havenā€™t had time to look at it yet#2017-06-2314:54odinodin@alexmiller nice! Just what I was looking for :)#2017-06-2316:12samueldevhey all#2017-06-2316:13samueldevdoes anyone know of a library to convert clojure.spec problem explanations to more human-readable output?#2017-06-2322:02bbrinckhttps://github.com/bhauman/strictly-specking seems to have some work in this direction, but I donā€™t think itā€™s a drop-in solution right now#2017-06-2322:02bbrinckIā€™m also working on something now, but itā€™s early days and not yet ready for release#2017-06-2322:03bbrinck(inspired by the work in strictly-specking and elm errors)#2017-06-2322:03bbrinckIā€™m a bit confused by the value of ā€œinā€ when using ā€œmap-ofā€ https://gist.github.com/bhb/6f06dd07bcf5b275a7d4faf3167bfc85#2017-06-2322:04bbrinckI searched JIRA but could not find an issue, so it may be a misunderstanding on my part#2017-06-2322:05bbrinck@samueldev ^-- (sorry, forgot to mention you, see my comment above)#2017-06-2322:11bbrinck@samueldev I think @ericnormand is also maybe working on something for more human-readable error messages?#2017-06-2322:12samueldevthanks @bbrinck ! I'll do some digging with @ericnormand's work and bruces!#2017-06-2323:46richiardiandreaI think also the above odinodin is working on something for re-frame#2017-06-2417:46Drew Verleeany idea why evaling (clojure.spec.alpha/valid? (clojure.spec.alpha/coll-of int? :kind vector?) [1 2 3]) would cause a stack over flow error?#2017-06-2417:47joshjonesworks fine over here -- can you give a context for how you're using it?#2017-06-2417:49Drew Verlee@joshjones from within the cider repl#2017-06-2417:52Drew Verleehmm#2017-06-2417:52Drew Verleei restarted my repl and it works fine. I think i know the interaction thats causing it, if not whyā€¦#2017-06-2417:54Drew Verleenope, i was wrong, i have no idea what was causing it. šŸ˜#2017-06-2417:55joshjoneshmm strange#2017-06-2516:34jjttjjI know spec is maybe too new for idioms to develop but is there a clear answer to having :x.specs.user/email, :x.specs.user/password and then :x.specs.user/user for the whole user entity or is :x.specs/user a better name for the "entity", what's everyone's preferences here?#2017-06-2519:24jjttjj
(ns x.specs.user
  (:require [clojure.spec.alpha :as s]))

(s/def ::username string?)
(s/def ::password string?)

(s/def ::user ;;or is :x.specs/user better?
  (s/keys :req [::username ::password]))
#2017-06-2520:16jjttjjI'm personally convinced x.specs/user is clearly better than ::user which expands to :x.specs.user/user, just trying to get a second opinion before imposing my will on others.#2017-06-2520:18luchiniWe have been doing something similar to :x.specs.user/user in your example. It seemed like a natural choice at first and it quickly became a shared language across the team.#2017-06-2520:20luchiniSame pattern you suggested: always the last s/def of the file and one ns per entity.#2017-06-2520:21luchiniIā€™m also not sure if this will evolve into an idiom but surely has made the communication in our team much easier.#2017-06-2520:22jjttjjthanks for the input!#2017-06-2520:26luchiniThereā€™s an interesting side-effect thatā€™s been great for the specifics of our project. We receive entity names from a remote system in a simple string. We can then convert the string into :x.specs.<string>/<string> and boom: we have our spec šŸ™‚#2017-06-2520:34jjttjj@lucascs awesome yeah that's basically what I'm going for with the overall project set up but is there a reason you go with :x.specs.<string>/<string> instead of :x.specs/<string> for the composite entity at the end of the file? Because what you're saying could be done with either of those options just as easily right? (Sorry to painfully over analyze small details here)#2017-06-2520:55luchiniIndeed it could. In practice the reasoning behind our choice was purely aesthetics. Kind of ā€œeverything related to the user entity is namespaced with :x.specs.user.ā€ Otherwise we would need to always have the entity itself as a bit of an exception (defining it with :x.specs/user instead of autoexpanding with ::user).#2017-06-2520:55luchiniPretty much a personal taste thing#2017-06-2520:56luchiniFrom a communication perspective, itā€™s one less cognitive level.#2017-06-2520:57jjttjjcool thanks again šŸ™‚#2017-06-2609:30vikeriI have trouble running stest/check in a repl. It just returns an empty vector immediately, just as if the symbol was undefined. I have required the symbol and I can evaluate it in the REPL but it seems the check canā€™t find the symbol. How can I make sure that stest/check finds the function that I gave to it?#2017-06-2611:14degI'm very excited about the goal of clojure.spec leading to improved error messages, but there is still a ways to go. I just typo'd :require in
(ns raven.intro
  (:requre [clj-http.client :as http]
           [raven.secrets :as secrets]))
and got the following user-friendly error:
CompilerException clojure.lang.ExceptionInfo: Call to clojure.core/ns did not conform to spec:
In: [1] val: ((:requre [clj-http.client :as http] [raven.secrets :as secrets])) fails at: [:args] predicate: (cat :docstring (? string?) :attr-map (? map?) :clauses :clojure.core.specs.alpha/ns-clauses),  Extra input
:clojure.spec.alpha/spec  #object[clojure.spec.alpha$regex_spec_impl$reify__1200 0x1331d7d5 "
#2017-06-2611:16mpenetthere's room for improvement for sure, there isn't a lib that does human readable translation yet?#2017-06-2611:18degIt takes a lot less than that. Just a spec that had a list of valid keywords for the ns form could trivially pump out ":requre is not one of [:import :require :use ...]"#2017-06-2615:48Alex Miller (Clojure team)deg: Itā€™s actually a lot trickier than this from a spec perspective due to the ā€œformā€ structure of the dsl. If we were to design the ns api now, it is highly unlikely we would do it this way (would probably be a map). This turns out (due to the high fanout and nested differences) to be a particularly challenging case for spec to generically give a good error message.#2017-06-2611:19mpenetI guess it goes against the choice of making specs maps open to extension tho (which is arguably good or bad)#2017-06-2611:20degThe problem is that there are very few people in the community who are simultaneously likely to (1) hit this kind of error; (2) take more than a few seconds to spot the problem and feel the pain; and (3) have the comfort level to dive into the source of clojure core.#2017-06-2611:20mpeneteven tho it's not a map here#2017-06-2611:20mpenetsure I agree, personally I wish we could specify strict sets for some kind of specs#2017-06-2611:20mpenetnot sure how it's implemented for ns tho#2017-06-2611:21degAnd you are right too, of course, that spec's philosophy is generally against closed lists of keywords. But, enough of us have written that for our own purposes. And, I'm sure that very few people would be against tight checking for the sake of error checking of special forms or canonical macros.#2017-06-2615:49Alex Miller (Clojure team)deg: just because itā€™s hard for spec to produce a good generic error here does not mean that the ns macro canā€™t take matters into its own hands instead to provide a customized response. not a done deal.#2017-06-2611:21mpenetyeah but here it actually is strict, my bad šŸ™‚#2017-06-2611:22mpenetit's just the error that s cryptic#2017-06-2611:22degYup.#2017-06-2611:23degAnd, relatively easy to fix any one error. The challenge is creating a framework where, anytime a newbie is bitten by one of these messages and reports it, it triggers a process that makes that message be forever better for the next user.#2017-06-2611:23luxbockif you specify the legal keywords for the NS form as a set, then that's still very much open to extension, no?#2017-06-2611:24luxbockjust add more keywords to the set if the form ever gains more functionality#2017-06-2613:34wilkerlucio@jjttjj one other thing to consider is: what is user? because in my experience that can vary wildly inside of your system, a login user might require login and password while a registering user might require much more, think about this, and you end up having more specific entities (like: login-user and new-user) or you might drop the entities are all (that might vary a lot depending on much re-use you can give to those entities)#2017-06-2614:09bbrinckIā€™m a bit confused by the value of ā€œinā€ when using ā€œmap-ofā€. When there is something wrong a value in the map, the in path doesnā€™t actually seem to point to that value. For example: https://gist.github.com/bhb/6f06dd07bcf5b275a7d4faf3167bfc85#2017-06-2614:11joshjones@vikeri Are you sure you have fdef'd the function?
(defn foo [x] x)
=> #'sandbox.spec/foo
(s/fdef foo :args (s/cat :x int?))
=> sandbox.spec/foo
(stest/check `foo)
=>
({:spec ...,
  :clojure.spec.test.check/ret {:result true,
                                :num-tests 1000,
                                :seed 1498486054141},
  :sym sandbox.spec/foo})
#2017-06-2614:15joshjones@vikeri If you fail to either (1) fdef a defined function, or (2) give check a symbol which does not resolve to a function, it will exhibit the behavior you describe:
(defn bar [x] x)
=> #'sandbox.spec/bar
(stest/check `bar)
=> ()

(stest/check `not-a-function)
=> ()
#2017-06-2614:17stathissiderisis there any way to provide a custom message in the output of s/explain?#2017-06-2615:50Alex Miller (Clojure team)stathissideris: no#2017-06-2615:52stathissideristhanks. Are there any plans for it, or is it out of scope/a bad idea?#2017-06-2615:56Alex Miller (Clojure team)there are no plans for it right now. the idea is that specs should be able to give you generically consistent errors. The explain-data error you get has enough info that you should be able to build custom user errors from that if desired - I believe Sean Corfield is someone doing a lot of this right now.#2017-06-2616:17stathissiderisI get it. I guess I have a slightly more complex case where I use s/& to further validate a conformed value and s/explain reports the whole code of the anonymous function as the failing predicate, which is useful, but it would be even better to have a human readable message to say what was expected#2017-06-2616:17mpenetsomewhat related: https://dev.clojure.org/jira/browse/CLJ-2115#2017-06-2616:18stathissiderisIt looks like that: val: ... fails spec: :monitor.settings/aliases-header predicate: (fn [{:keys [aliases]}] (apply distinct? (map :alias aliases)))#2017-06-2616:19mpenetyou can create custom Spec impl but it's risky#2017-06-2616:19stathissideris@U050SC7SV I donā€™t want it that bad, itā€™s more of a ā€œnice to haveā€ for me šŸ™‚#2017-06-2616:36Alex Miller (Clojure team)s/& is indeed a special case and we are likely going to have to add support for custom forms there regardless (to support things like s/keys* which use it for implementation), but unlikely we would have a custom explain there beyond that#2017-06-2616:53stathissideris@alexmiller ok, thanks for the explanation and thanks for your efforts in general!#2017-06-2709:50stathissiderisOne way to make this a bit more self documenting would be for the second argument of s/& to be named function with a descriptive name instead of an in-place anonymous function#2017-06-2614:17vikeri@joshjones Hmm, I required the ns where the fn was defined but maybe that didnā€™t define the fdef. That is probably the issue.#2017-06-2614:18vikeriThanks for the pointer#2017-06-2614:30bbrinckA simpler example: https://gist.github.com/bhb/c8d01c455494921a3698a9cf951272ff of how :in works with map-of. I think Iā€™m beginning to understand how the path works. [:hi 0] means something like ā€œconstruct a key/value pair from the map and the key :hi, then navigate to the 0th element of that k/v pairā€#2017-06-2615:51Alex Miller (Clojure team)correct - maps are conformed as a sequence of map entries (specā€™ed as a tuple of k and v)#2017-06-2615:53Alex Miller (Clojure team)however, that path wonā€™t get you to the right place so I think that 0 is actually going to get you to the wrong place.#2017-06-2615:57Alex Miller (Clojure team)and that seems like a bug
#2017-06-2615:58Alex Miller (Clojure team)and would be happy to see a jira about that. it is related to https://dev.clojure.org/jira/browse/CLJ-2080 but not addressed by that ticket#2017-06-2616:08joshjonesI've been looking at it for a few minutes now @alexmiller -- and I was going to say it looked like a bug too (but was not confident enough that my understanding about it was correct). the second element of the :in in this case is which element of the tuple spec caused the error. But the first element (`"hi"`) is coming from the every-impl spec, and this part particularly does not seem correct#2017-06-2616:09Alex Miller (Clojure team)It's tricky #2017-06-2616:09joshjonesyes, it seems so#2017-06-2616:10joshjonesputting some println's in the spec code:
(clojure.spec.alpha/explain-data :foo/user-map {"hi" "foo"})
EVERY IMPL, mapping
i:  0 
v: [hi foo] 

TUPLE IMPL, mapping
i:  0 
form: clojure.core/string? 
pred: #object[clojure.core$string_QMARK___6415 0x674c583e 
#2017-06-2616:11Alex Miller (Clojure team)I would consider this on top of the patch for 2080, which improves things in the explain case, whereas this is the conform case which doesn't currently use the kfn#2017-06-2616:12Alex Miller (Clojure team)Rich and I have an ongoing discussion/argument about what happens here :)#2017-06-2616:13joshjonesi can see why, as it's not necessarily a clear cut answer as to what should be there. from one of your spec slides, I have that :in represents: "vector of specs from root spec to failing spec"#2017-06-2616:15joshjones@bbrinck it seems your confusion was justified šŸ˜‰#2017-06-2616:20bbrinck@alexmiller @joshjones I appreciate the info. I will look at the tickets and patches mentioned above and file a follow up bug. I agree itā€™s tricky - especially in the case where the key is wrong. Given a nested data structure, how do I provide a path to a map key? AIUI, it doesnā€™t work if I consider a ā€œpathā€ to equal ā€œa vector of keysā€ that would work with get-in.#2017-06-2616:21bbrinckWhat Iā€™ve started to do is try to construct a function (maybe this already exists?) that takes some data + an :in path and retrieves the value.#2017-06-2616:22bbrinckThatā€™s really what is driving this: Iā€™d like to be able to take some (invalid) data + a problem (which contains the :in) and be able to use the unique :in value to get the problematic value, which is likely deep inside the original data#2017-06-2616:23bbrinckI had originally (naively) thought that I could use get-in for that function, but I think itā€™s more subtle than that, so Iā€™m writing my own to use these special :in paths.#2017-06-2616:30Alex Miller (Clojure team)the idea is that you should be able to do that#2017-06-2616:32bronsaisn't that impossible if you also want to be able to reach keys?#2017-06-2616:34Alex Miller (Clojure team)itā€™s not possible in all cases#2017-06-2616:34Alex Miller (Clojure team)but I donā€™t know that itā€™s useful to specify [<key> 0] either#2017-06-2616:38bbrinckYeah, tricky. For my cases, Iā€™m considering changing my code to accommodate a ā€œspec pathā€ that allows me to reach keys i.e. writing new functions that work like get-in or update-in with this new type of path#2017-06-2617:14stathissiderisjust a thought, not a question: I think Iā€™ll write a few ā€œreadableā€ and size constrained generators for strings, keywords and symbols so that my eyes donā€™t bleed whenever I try to read the output of s/exercise (and I know itā€™s a good thing that it produces ā€œchallengingā€ output)#2017-06-2711:44abhiragHi, kinda new to clojure as well as clojure.spec#2017-06-2711:45abhiragWas trying out specifying the above function and stest/check hangs forever#2017-06-2711:45abhiragam I doing anything wrong here?#2017-06-2713:44Alex Miller (Clojure team)@abhirag how long did you wait? (presumably not ā€œforeverā€ :) it works for me, but takes a while.#2017-06-2713:44Alex Miller (Clojure team)ā€œa whileā€ being a couple minutes#2017-06-2713:48gfredericksšŸ˜³ why should that take a couple minutes?#2017-06-2713:49Alex Miller (Clojure team)a fair question :)#2017-06-2713:50abhiragI did try and restrict the count to 10 to reduce time, my dev machine might be slower than yours Alex, I will try again and try and not get afraid of the cooling fans revving up :)#2017-06-2713:50abhiragI also tried to restrict the :test-nums to 10#2017-06-2713:51abhiragBut still the running time wasn't getting shorter#2017-06-2713:51gfredericksthat definitely sounds like a problem to me#2017-06-2713:51Alex Miller (Clojure team)yeah, that seems weird#2017-06-2713:52Alex Miller (Clojure team)s/coll-of with a :count parameter will create a gen/vector with a size#2017-06-2713:53abhiragYeah I did learn that from your guide :)#2017-06-2713:54abhiragAnd also if I just specify the function using fdef and don't give an implementation#2017-06-2713:54abhiragAnd then try check#2017-06-2713:54abhiragIt just gives a null pointer exception#2017-06-2713:55abhiragNo error message regarding the fact that no implementation was provided :P#2017-06-2713:56abhiragThought I should mention that too#2017-06-2713:56abhirag:)#2017-06-2713:56Alex Miller (Clojure team)@gfredericks (g/sample (g/vector g/large-integer 10) 1000) seems like it should be close to what Iā€™d expect and thatā€™s fast#2017-06-2713:59gfredericksalexmiller: yeah that's what I was imagining#2017-06-2714:01Alex Miller (Clojure team)although itā€™s also going to put those in lists, then put that in a vector for the args, then invoke the function, then check the return is a list, reverse that list, and compare it to the input list#2017-06-2714:03Alex Miller (Clojure team)but even so#2017-06-2714:43Alex Miller (Clojure team)this is indeed pretty jacked and is already logged https://dev.clojure.org/jira/browse/CLJ-2103#2017-06-2714:43Alex Miller (Clojure team)but I did not fully understand this issue before#2017-06-2715:01gfredericksI don't understand it at a glance; let me know if it'd be helpful for me to look closer#2017-06-2713:57Alex Miller (Clojure team)@abhirag there is a ticket for that error case, which I think went into alpha17, not sure which youā€™re using#2017-06-2713:58abhiragThe version mentioned in the guide#2017-06-2713:58abhiragLet me check#2017-06-2713:58Alex Miller (Clojure team)I think the guide says alpha16 right now#2017-06-2713:59abhiragAlright I will upgrade, thanks for your help :)#2017-06-2714:00Alex Miller (Clojure team)maybe Iā€™m confusing that with something else. if you still see, feel free to log a jira for it#2017-06-2714:02abhiragWill do :)#2017-06-2714:29abhirag@alexmiller I am still getting this error#2017-06-2714:30abhiragI haven't ever logged a defect before in clojure, but if you feel that this deserves a ticket I'll log one šŸ™‚#2017-06-2714:31Alex Miller (Clojure team)yeah, I think so. you can create an account at https://dev.clojure.org/jira/secure/Signup!default.jspa and then log a defect at https://dev.clojure.org/jira/browse/CLJ#2017-06-2714:32Alex Miller (Clojure team)Iā€™m still looking at your gen stuff too. It shouldnā€™t be that slow. I believe itā€™s the :kind list? that is causing the slow-down.#2017-06-2714:34abhiragyeah I had tried property testing in other languages before, that snippet was an overkill, my main motive with that was that if we already have a function tested and need to rewrite it for optimization etc. we could just use the old function to test it#2017-06-2714:35abhiragnow I realize that with spec all I really need is that the rewritten function have the same spec#2017-06-2714:37wilkerlucio@abhirag I noticed on your REPL example that you didn't defined the my-reverse function#2017-06-2714:38abhiragyeah I realize that šŸ™‚ that was just to get that null pointer exception, I was gonna log a ticket to make that error message better#2017-06-2714:40wilkerluciocool, just wondered if was on purpose :)#2017-06-2714:43Alex Miller (Clojure team)@abhirag the slow gen is actually already logged https://dev.clojure.org/jira/browse/CLJ-2103 and is definitely in need of some work#2017-06-2714:45abhiragone more quick question (stest/check `my-reverse {:num-tests 10})#2017-06-2714:45abhiragis this the correct way to reduce the number of tests?#2017-06-2714:50Alex Miller (Clojure team)no :)#2017-06-2714:51abhiragthat actually didn't throw any exception, but as I was not getting any output, there wasn't any other way to know šŸ™‚#2017-06-2714:51Alex Miller (Clojure team)I think
(stest/check `my-reverse {:clojure.spec.test.check/opts {:num-tests 10}})
#2017-06-2714:52abhiragalright got it, thanks šŸ™‚#2017-06-2714:52abhiragI'll try and use :onto vector#2017-06-2714:53abhiragas described in the issue description#2017-06-2714:53abhiragto get away from the slow running time for now#2017-06-2714:58Alex Miller (Clojure team)I think using :into () in addition to :kind list? would help#2017-06-2714:59abhiragalright will give that a try šŸ™‚#2017-06-2816:43lwhortonwhats the proper way to define a spec referring to another namespace?
(s/def ::foo :bar.alice/bob)
(s/def ::my-spec (s/keys :req-un [::foo]))
#2017-06-2816:44lwhortonfor some reason i get complaints about unable to resolve spec :bar.alice/bob using the above method#2017-06-2816:44lwhortonmight i be required to be explicit in :require [bar.alice :as bar.alice] so load-order is taken care of properly?#2017-06-2816:50seancorfieldThe ns containing that spec (`:bar.alice/bob`) must be loaded before you can use it in another spec. Otherwise the s/def will not have been executed.#2017-06-2817:16stathissiderisis there a way to override the default generator of s/keys but only for a single key? (without renaming the key)#2017-06-2817:19bbrinckWe have some JSON data coming in over the wire that uses strings as keys e.g. {"city" "Denver" "state" "CO"}. Weā€™ve been converting all strings keys to keywords in order to spec them with s/keys, but of course, doing the mapping takes a call to clojure.walk/keywordize-keys and then, if there is a problem that we want to report to the dev, we might need to convert back to strings to make a sensible error about the data. In this case, do people do the conversion like this? It would seem like being about to have string keys would be useful e.g. (s/keys :req-str [:location/city :location/state]) where :req-str is like :req and :req-str-un would be like :req-un, but for string keys#2017-06-2819:27Alex Miller (Clojure team)bbrinck: would be reasonable to file a jira enhancement for htis#2017-06-2819:28bbrinck@U064X3EF3 Can do!#2017-06-2817:40grzmI had some slow checks that I was trying to performance tune. One approach I wanted to try was swapping out the fdef spec during the stest/check run, similar to how you can swap out generators using the :gen option. I tried instrumenting the function and using the :spec option prior to the stest/check run, but that didn't have an effect. Does this seem like a reasonable thing to do? Is there a way to do it?#2017-06-2819:28Alex Miller (Clojure team)grzm: check takes a generator override map - did you try that?#2017-06-2819:32grzmIt's not clear to me how providing a generator would replace the function spec: I'm not trying to change the values that are supplied to the function: I'm trying to change the spec that's applied to the function as a whole for the scope of the check. Or am I misunderstanding?#2017-06-2819:32grzmWhat would the generator be generating in this case?#2017-06-2819:33Alex Miller (Clojure team)oh, then instrument with a replacement spec should be what you want#2017-06-2819:34Alex Miller (Clojure team)however, calling instrument correctly is sometimes hard#2017-06-2819:40Alex Miller (Clojure team)in particular, anything that is being changed (including the thing whose spec is changing) needs to be in the list of instrumented vars. You should see that var in the return value from instrument as well - if you donā€™t, it wasnā€™t changed.#2017-06-2819:41grzmI'm working up a short example that describes what I'm trying.#2017-06-2819:43Alex Miller (Clojure team)hereā€™s a spec replacement example:
(defn a [x] (if (zero? x) "a" (inc x))) ;; special behavior on 0
(defn b [y] (if (zero? y) 0 (+ (a y) 10))) ;; but b guards this
;; so use simpler spec when testing b
(stest/instrument `a 
  {:spec {`a (s/fspec :args (s/cat :a int?) :ret int?)}})
(stest/check `b)
#2017-06-2819:48grzm
(defn a [x])

(s/fdef a
        :args (s/cat :x int?)
        :fn (fn [_] true))

(s/fdef b
        :args (s/cat :x int?)
        :fn (fn [_] false))

;; should pass
(stest/check `a)

(stest/instrument `a {:spec {`a `b}})
;; should fail
(stest/check `a)
#2017-06-2819:49grzmWhat I've been trying is more direct. In your example, you're checking b which calls a. I'd like to check a directly.#2017-06-2819:52Alex Miller (Clojure team)Iā€™m not sure the val of the :spec map will actually resolve that via the registry - did you try passing the actual spec there?#2017-06-2819:52Alex Miller (Clojure team)
(stest/instrument `a {:spec {`a (s/get-spec `b)}})
#2017-06-2819:54Alex Miller (Clojure team)doesnā€™t seem like that works either#2017-06-2819:55grzmI don't see it working, either. You're a faster typer than I am šŸ™‚#2017-06-2819:56grzmTaking a step back, does this seem like a reasonable thing to do?#2017-06-2819:56Alex Miller (Clojure team)yes#2017-06-2819:57grzm
(stest/instrument `a {:spec {`a (s/fspec :args (s/cat :x int?) :fn (fn [_] false))}})
#2017-06-2819:57grzmThat doesn't work either, btw (for completeness)#2017-06-2819:58grzmI'm considering taking your spec course at the Conj. How far into the weeds are you going to get? Do you think you'll have a syllabus available?#2017-06-2820:00Alex Miller (Clojure team)the example I gave you above is from the spec course#2017-06-2820:01Alex Miller (Clojure team)so about that far :)#2017-06-2820:03grzmGotcha. šŸ™‚#2017-06-2820:05grzmWould you like me to open a ticket for this?#2017-06-2820:07Alex Miller (Clojure team)sorry, I was looking at the code. I understand why it doesnā€™t work, still thinking about what that means though.#2017-06-2820:07Alex Miller (Clojure team)the spec override is used when building the wrapped var in instrument#2017-06-2820:08Alex Miller (Clojure team)but check still uses the version in the spec for checking ret and fn, not the overridden spec#2017-06-2820:09Alex Miller (Clojure team)and indeed check canā€™t even see that - itā€™s just part of the check-fn on the instrumented var#2017-06-2820:09Alex Miller (Clojure team)seems like it would be totally reasonable to want to do something like this in the context of check though#2017-06-2820:10Alex Miller (Clojure team)instrumented vars will only catch invalid invocations, not invalid results.#2017-06-2820:11Alex Miller (Clojure team)so I think a ticket to add the ability for check to take spec overrides etc would be reasonable. we have talked about making things like generator overrides, spec overrides etc a consistent api used across exercise, exercise-fn, instrument, check, etc which would be in this area as well#2017-06-2820:12grzmOkay. I'll open one. (No need to apologize, by the way. I suspected as much, and even if you were doing something else, I'm grateful for the time and attention you're providing.)#2017-06-2820:17grzmI was similarly surprised about generator overrides with instrument and check. I expected generator overrides to apply globally. From what I can tell, they only apply to the checked function, not the functions called by the checked function.#2017-06-2820:17Alex Miller (Clojure team)itā€™s a little more subtle than that#2017-06-2820:18Alex Miller (Clojure team)they do apply globally, but there are a couple cases where they donā€™t get picked up#2017-06-2820:18Alex Miller (Clojure team)there are some tickets on this#2017-06-2820:19grzmHappen to have pointers to those tickets handy to see if they cover my use case?#2017-06-2820:19Alex Miller (Clojure team)https://dev.clojure.org/jira/browse/CLJ-2079#2017-06-2820:20grzmCheers.#2017-06-2820:21Alex Miller (Clojure team)https://dev.clojure.org/jira/browse/CLJ-2095 is another, although Iā€™m not sure that this is something we actually can or will change#2017-06-2820:28grzmHere's the ticket I opened. https://dev.clojure.org/jira/browse/CLJ-2193#2017-06-2820:29grzmAre you guys happy with Jira? Is it worthwhile for someone to make a Clojure syntax highlighter plugin?#2017-06-2823:51grzmHere's a gist showing the instrument generator override behavior I mentioned above. https://gist.github.com/grzm/0ada176caee5bcdab39d820577ed3823 I'll bring this up in #clojure-spec as well and continue discussion there unless you bring in back here. Thanks again for your help today.#2017-06-2817:54lwhorton@bbrinck whatā€™s your concern, performance?#2017-06-2817:57bbrinckPerformance and the fact the validated data doesn't match the original, so error messages either need to be converted back or are somewhat confusing. #2017-06-2817:58lwhortonas always with performance i would test and verify that itā€™s going to be an issue before worrying about doing 2x conversions#2017-06-2818:00grzm@bbrinck with respect to the validated data doesn't match the original, you're already doing a conversion (I suspect) from JSON objects to Clojure maps. The similarity between the two (including syntax) obscures this a bit.#2017-06-2818:01bbrinck@lwhorton. Fair point about perf. My greater concern is complexity of implementation when converting to validate and then back to report for what seems like a common case of string keys (when working with JSON). #2017-06-2818:03bbrinckFrankly it's not a huge cost. Just wondering if this workaround is how others are tackling this case. #2017-06-2818:03lwhortonmaybe im looking at it wrong, but i dont see complexity around (keywordize-keys json) and (string-keys edn)#2017-06-2818:04lwhortonif you convert to edn, run a spec/validate and it fails, then convert it back to json and make a POSTā€¦ that all seems fairly straightforward?#2017-06-2818:06bbrinckI'd ideally like to give error messages that are relevant to the original data. Imagine an API that accepts JSON. If I want to give an errr message about non conformance, I want this to be about strings not keywords. #2017-06-2818:07bbrinckI suppose I can explain-data and then walk each problem and convert back to strings. The n format the data?#2017-06-2818:08lwhortonor hold a reference to the original json, right?#2017-06-2818:08bbrinckBut that's just the whole thing right? That's not the specific section that has the problem #2017-06-2818:08lwhortoni suppose if you wanted to get into the exact location where a conform failed, yes you might have to do explain-data#2017-06-2818:09bbrinckYou'd need to walk each problem and reconvert, then reformat. But you are correct, it's not a huge amount of work#2017-06-2818:09lwhortonyou could also write your spec to be some form of s/cat :value string? ... etc.#2017-06-2818:10bbrinckAnd I reformat I just mean reimplement 'explain-str'#2017-06-2818:10lwhortonbut that might be way more complex than what is already offered by (s/keys)#2017-06-2818:10lwhorton(essentially spec out the JSON implementation)#2017-06-2818:11lwhortoni would also look around for some libs that might already exist for this sort of thing#2017-06-2818:11bbrinckRight. Thanks for the ideas! Good to know I'm not missing an obviously simpler way :)#2017-06-2818:12lwhortonwith plumatic.schema there was coersion, but iā€™m not sure if spec went down that path as well#2017-06-2818:16bbrinckAIUI, spec is avoiding coercion but I could be mistaken. I might just end up implanting a version of 's/keys' that accepts strings (and produces a generator that uses string keys). Could be useful for JSON validation #2017-06-2818:17bbrinckNote I agree the specs themselves should be keywords. I'm just thinking that they could be included in maps as strings. #2017-06-2818:17lwhortonIā€™ve got a question for you šŸ™‚ ā€¦ have you ever used branching in a spec to define a spec? Say you have a key :foo and :bar, but :bar can pull from set A or B of valid values. Is there a way to make :bar read :foo and decide which is a valid spec?#2017-06-2818:19grzm@lwhorton Does multi-spec do what you want? Or are you getting at something different?#2017-06-2818:22lwhortonhah, that looks exactly like what Iā€™m talking about#2017-06-2821:32dealyspec newbie here, please excuse my ignorance. I have a data structure which I've defined a spec for. I've just needed to add a channel as one of the items. How do I update my spec for this type of a field?#2017-06-2821:44seancorfieldPretty sure thereā€™s no way to spec a channel (same as thereā€™s no way to spec atom/delay/etc) so you can just use any? (or not provide an actual spec for that key).#2017-06-2821:46seancorfieldSpecā€™ing something to be a channel wouldnā€™t be very useful: you couldnā€™t generate values for it (since it has type channel).#2017-06-2821:47dealyoh, ok, there's a lot of spec I'm still pretty confused about. So, spec'ing data structures is primarily for building up things from basic types (ints doubles bools and functions) as opposed to interfaces and complex objects?#2017-06-2821:49seancorfieldHmm, not reallyā€¦#2017-06-2821:50seancorfieldYou can spec domain-level entities ā€” so you can give things meaning ā€” but you need to think about what youā€™re trying to achieve with spec. Why are you specā€™ing a particular structure? What are you going to use the spec for?#2017-06-2821:50seancorfieldThereā€™s not much point in specā€™ing everything and I think itā€™s a common misunderstanding to try to treat spec like a type system and ā€œspec everythingā€.#2017-06-2821:52seancorfieldYou also need to consider whether you want specs to be ā€œgeneratableā€ (you probably mostly do) so you need to avoid specs with types that canā€™t be generated (since theyā€™re not very useful, in general).#2017-06-2821:54dealyyea, that makes sense. I haven't gotten into that part yet. Still only using it as a way of just making sure my datastructure remains in the format I expect/need.#2017-06-2821:57seancorfieldAre you using conform or valid? or some other way of checking against the spec?#2017-06-2821:58dealyvalid?#2017-06-2821:58dealythis is all part of my first GUI app using re-frame so its validating the main app-db as the UI changes#2017-06-2821:58seancorfieldSince spec checks are all runtime, if you really need to ā€œassertā€ your data structure has a specific key, you can always spec it as any? and if you use something as a channel and it isnā€™t, itā€™s going to blow up at runtime anyway.#2017-06-2821:59dealyits a lot to learn all at once#2017-06-2821:59seancorfieldI would be a bit suspicious of data including a channel thoā€™ā€¦#2017-06-2822:00dealynormally yes, but it is an artifact of a timer that needs to be cleaned up later#2017-06-2822:00seancorfieldafter all, thatā€™s not data you can serialize and pass around so itā€™s not really dataā€¦#2017-06-2822:00seancorfield(I may be taking a bit of a purist view here ā€” not sure how others feel?)#2017-06-2822:11lwhortoncan someone point out my (probable) misuse of multi-spec? what iā€™m trying to go for, in plain english, is to specify a map where some keyā€™s spec depends on the value of another key. If key a is one of ā€œcatā€ or ā€œdogā€, key b should be specā€™d differently in either case.
(defmulti b-depends-on-a :a)
(defmethod b-depends-on-a "cat" [_] (s/or :cat-only-thing ::cat-thing
                                          :cat-or-dog-thing ::shared-things))
(defmethod b-depends-on-a "dog" [_] (s/or :dog-only-thing ::dog-thing
                                          :cat-or-dog-thing ::shared-things))
(s/def ::a #{"cat" "dog"})
(s/def ::b (s/multi-spec b-depends-on-a :a))
(s/def ::foo (s/keys :req-un [::a
                              ::b]))

#2017-06-2822:14lwhortonI would expect to be able to run something like (s/conform :: foo {:a "cat" :b ...}) or (s/conform :: foo {:a "dog" :b ...}) and have the response [:cat-only-thing ā€¦] or [:dog-only-thing ā€¦] or [:cat-or-dog-thing ...] depending on the value of b.#2017-06-2822:14lwhortonIs this totally off?#2017-06-2822:24seancorfieldI think your multi-spec needs to be a hash map that has a key :a#2017-06-2822:24seancorfieldSo ::b would be spec a hash map ā€” do ::cat-thing etc spec hash maps?#2017-06-2822:25seancorfieldSee this page https://clojuredocs.org/clojure.spec/multi-spec#2017-06-2822:26lwhortoniā€™ve been banging my head on that page for quite a while now .. i dontā€™ really understand the ā€˜tagā€™ argument to be honest#2017-06-2822:26seancorfieldThis https://clojure.org/guides/spec#_multi_spec also shows the multi-specs as being hash maps.#2017-06-2822:27lwhortonthe call to s/multi-spec event-type in that example is calling on a defmulti event-type, unless Iā€™m misunderstanding#2017-06-2822:27seancorfieldThe :event/type key is common to all the (hash maps) that are valid for that set of specs.#2017-06-2822:29seancorfieldIn your example, you should have a hash map that has a key :a that is the ā€œtagā€ and it can have the value "cat" or "dog" and the defmethods should returns specs for the cat, or dog, hash map that includes the tag :a.#2017-06-2822:29lwhortonoh you mean the object on which iā€™m invoking the spec? i.e. s/conform ::foo {:a "cat" (or "dog") :b something-else}?#2017-06-2822:30seancorfieldRightā€¦ so your defmethods should be (s/keys :req-un [::a :cat/b]) and (s/keys :req-un [::a :dog/b])#2017-06-2822:31seancorfieldthen the multi-spec selects which spec to use based on the tag :a#2017-06-2822:31lwhortonah so i guess I cannot get away with what I was originally trying to do#2017-06-2822:31seancorfieldDepends what ::cat-thing, ::dog-thing, and ::shared-things areā€¦#2017-06-2822:32lwhortonwhich is something like ā€œif the multimethod matches on ā€˜catā€™, return a spec which is an (s/or ā€¦)#2017-06-2822:32lwhortonso you are saying ::cat-thing, ::dog-thing, ::shared-things would have to be maps containing the key :a, correct?#2017-06-2822:34seancorfieldWhat are you trying to do?#2017-06-2822:35seancorfieldIf you explain how you want :b to vary, I can probably show youā€¦#2017-06-2822:35lwhortoni have a piece of data that looks like this {:status "" :operation "" :state ""}#2017-06-2822:35lwhortonactually let me thin that down even more, i have a map {:status "" :operation ""}#2017-06-2822:36lwhortonoperation can belong to any member of the set #{:idle :downloading :patching}#2017-06-2822:37lwhortonbut depending on the value of :operation, the spec for status should be ā€œlimitedā€. that is, given the idle operation, the available statuses should only be one of #{:a :b :c}, and given the downloading operation, statuses should only be one of #{:c :d :e} (note the overlap)#2017-06-2822:37lwhortoni was trying to use spec to enforce this pattern-matching behavior#2017-06-2822:38lwhortonthe (s/or ..) comes into play because there is some overlap between :downloading / :patching / :idle, but only 1-2 states#2017-06-2822:39seancorfieldSureā€¦ so you need to spec the different types of status values, e.g., (s/def :downloading/status #{:c :d :e}) and (s/def :idle/status #{:a :b :c})#2017-06-2822:40seancorfieldand then your defmethod would return (s/keys :req-un [::operation :downloading/status]) for the downloading branch and (s/keys :req-un [::operation :idle/status]) for the idle branch#2017-06-2822:41lwhortonoof, so all this pain and confusion because i was trying to do an s/or (to union) where I should have just defined the full sets outright, regardless of overlaps?#2017-06-2822:42lwhortonill give that a whirl and get back to you.. regardless of the outcome thanks for your help#2017-06-2822:44seancorfield
(s/def ::operation #{:idle :downloading :patching})

(s/def :idle/status #{:a :b :c})
(s/def :downloading/status #{:c :d :e})
(s/def :patching/status #{:a :c :f})

(defmulti operation-spec :operation)
(defmethod operation-spec :idle
  [_] (s/keys :req-un [::operation :idle/status]))
(defmethod operation-spec :downloading
  [_] (s/keys :req-un [::operation :downloading/status]))
(defmethod operation-spec :patching
  [_] (s/keys :req-un [::operation :patching/status]))

(s/def ::machine (s/multi-spec operation-spec :operation))
#2017-06-2822:44seancorfield(s/conform ::machine {:operation :idle :status :a}) succeeds#2017-06-2822:45seancorfield(s/conform ::machine {:operation :idle :status :d}) fails#2017-06-2822:45seancorfield(s/conform ::machine {:operation :downloading :status :a}) succeeds#2017-06-2822:45lwhortonindeed it does, and a gen/sample gives me 10 good values !#2017-06-2822:46lwhortonwell itā€™s good to know now that defmethod has to return ā€œthe original matchā€ + the spec, not some hob-goblin pseudo spec#2017-06-2822:46lwhortonmany thanks for your guidance#2017-06-2822:46seancorfieldThanks for the question ā€” I hadnā€™t played with multi-spec before and this gave me a good opportunity to try it out.#2017-06-2822:47seancorfieldThe different branches could also have different keys etc.#2017-06-2822:48seancorfieldAnd, indeed, doesnā€™t have to be a hash map but then the tag function would need to be something other than a simple keyword.#2017-06-2822:49lwhortonso does the tag arg on (multi-spec spec tag) have to be identical to the fn/kwd in defmulti name fn/kwd?#2017-06-2822:50lwhortonnot really sure why the tag is there.. unless its a way to generate fields under a different key from the one of the multi-match?#2017-06-2822:53bbloomrandom thought: i kinda wish there was a way to alias keys, just as there is a way to alias attributes in datomic - would help with refactoring and ā€œsubtypingā€#2017-06-2822:58seancorfield@lwhorton I believe the tag is needed in multi-spec so that the generator can figure out what key to work with.#2017-06-2822:58seancorfield(since defmulti could take a function etc)#2017-06-2822:59lwhortoni see .. if you still have that ^above code in a register, what happens when you try to generate some samples?#2017-06-2822:59seancorfieldIf defmulti was some complex selector function, the argument passed to multi-spec would have to kind of reverse that I thinkā€¦#2017-06-2822:59lwhortondo you end up with {:data-state "" :data-status {:data-state "" :data-status ""}?#2017-06-2823:00seancorfield
([{:operation :idle, :status :c} {:operation :idle, :status :c}] 
 [{:operation :patching, :status :c} 
  {:operation :patching, :status :c}] 
 [{:operation :downloading, :status :c} 
  {:operation :downloading, :status :c}] 
 [{:operation :idle, :status :b} {:operation :idle, :status :b}] 
 [{:operation :downloading, :status :e} 
  {:operation :downloading, :status :e}] 
 [{:operation :patching, :status :f} 
  {:operation :patching, :status :f}] 
 [{:operation :idle, :status :c} {:operation :idle, :status :c}] 
 [{:operation :idle, :status :b} {:operation :idle, :status :b}] 
 [{:operation :patching, :status :c} 
  {:operation :patching, :status :c}] 
 [{:operation :idle, :status :b} {:operation :idle, :status :b}]) 
from s/exercise
#2017-06-2823:00lwhortoninteresting, i probably typo-d somewhere#2017-06-2823:00lwhortonthis spec gen stuff is so great#2017-06-2823:03seancorfieldYup, I pretty much always try to s/exercise my specs as a sanity check, and also to see the sort of crazy stuff they can produce. We use regex fairly heavily so we rely on @gfredericks test.chuck which has a generator for regex string specs ā€” awesome stuff!#2017-06-2823:16spiedengreat clojure.test integration too =)#2017-06-2910:52harriganIs the following behavior for s/& expected?
(s/conform (s/* int?) []) ;; => []
(s/conform (s/& (s/* int?)) []) ;; => nil
#2017-06-2910:53harriganAFAIKT, (s/& (s/* ...) ...) []) always returns nil no matter what predicates are passed to s/&.#2017-06-2912:13gmercer@harrigan not sure if it is relevant but there was an earlier discussion regarding the difference between s/& and s/and#2017-06-2912:16gmercer
(s/conform (s/and (s/* int?)) []) ;; => []
#2017-06-2912:21harriganI donā€™t think this is due to the difference between s/& and s/and. The actual instance that tripped me up was:
(s/conform (s/& (s/+ int?) #(= 3 (count %))) []) ;; => :clojure.spec.alpha/invalid
(s/conform (s/& (s/* int?) #(= 3 (count %))) []) ;; => nil
#2017-06-2912:27harriganI expected s/conform to return invalid in both cases.#2017-06-2912:31gmercerlooking back, it was your discussion ..#2017-06-2912:43gmercerdang!#2017-06-2912:43gmercer
(require '[clojure.spec.alpha :as s])
(s/conform (s/& (s/* (fn [i] (prn "pred of " i ) (int? i))) (fn [x] (prn "exec test of " x) (= (count x) 1))) [1])
(s/conform (s/and (s/* (fn [i] (prn "pred of " i ) (int? i))) (fn [x] (prn "exec test of " x) (= (count x) 1))) [1])
(s/conform (s/and (s/* (fn [i] (prn "pred of " i ) (int? i))) (fn [x] (prn "exec test of " x) (= (count x) 0))) [])
(s/conform (s/& (s/* (fn [i] (prn "pred of " i ) (int? i))) (fn [x] (prn "exec test of " x) (= (count x) 0))) [])
#2017-06-2912:47gmercer@harrigan curiouser and curiouser ... šŸ˜æ#2017-06-2913:18Alex Miller (Clojure team)This is a bug with s/& where it won't check the extra preds if the empty coll passes #2017-06-2913:18Alex Miller (Clojure team)There is a ticket with a patch waiting to go in#2017-06-2913:20Alex Miller (Clojure team)https://dev.clojure.org/jira/browse/CLJ-2182#2017-06-2913:21Alex Miller (Clojure team)@harrigan ^^ I am bad at notification#2017-06-2913:22gmercerthanks Alex - any idea about the double 'exec test' on the
(s/conform (s/& (s/* (fn [i] (prn "pred of " i ) (int? i))) (fn [x] (prn "exec test of " x) (= (count x) 1))) [1])
#2017-06-2913:24Alex Miller (Clojure team)Can happen - there is no guarantee on how many times a predicate may be called#2017-06-2913:25Alex Miller (Clojure team)I'd have to spend more time to explain why here#2017-06-2913:25Alex Miller (Clojure team)To understand why that is#2017-06-2913:25gmercernw - but interesting!#2017-06-2915:12stathissiderishas anyone noticed multi-spec being slow to generate values? it also sometimes fails randomly#2017-06-2915:14stathissiderisIā€™m doing a gen/sample with a size of 1, and I get ExceptionInfo Couldn't satisfy such-that predicate after 100 tries.#2017-06-2915:14stathissiderisbut not always#2017-06-2915:15stathissideristhe spec is a multi-spec that contains maps with strings, nothing fancy#2017-06-2915:15stathissiderisone of the keys is a multi-spec itself#2017-06-2915:15gfredericksthe such-that exceptions happen when you have an s/and somewhere and the generator for the first clause isn't likely to satisfy the remaining clauses#2017-06-2915:19gfredericksin the worst case the problem is solved by writing a custom generator for the (s/and ...)#2017-06-2915:19gfredericks(the slow generation is probably a different symptom of the same problem)#2017-06-2915:20stathissideristhe only suspect thing in there was this:
(def email-re #"[a-zA-Z0-9._%+-]
#2017-06-2915:20stathissiderisbut I removed it and still got the same behaviour#2017-06-2915:20gfrederickswell that looks like it already has the custom generator#2017-06-2915:20stathissideriscorrect#2017-06-2915:21gfredericksis it possible you're using some higher-level spec that expands to an s/and?#2017-06-2915:22stathissiderishm, I think s/merge might#2017-06-2915:24gfredericksšŸ¤” that sounds strange#2017-06-2915:25stathissiderisI think I know what it is: Iā€™ve been using this macro to disallow ā€œunknownā€ keys (I know itā€™s discouraged):
(defmacro only-keys
  [& {:keys [req req-un opt opt-un] :as args}]
  `(s/merge (s/keys 
#2017-06-2915:26stathissiderisI donā€™t think it plays well with generation#2017-06-2915:26stathissiderisremoving it fixed my problem#2017-06-2915:26stathissideris(it seems)#2017-06-2915:26gfredericksI would've expected s/and to be used there instead of s/merge šŸ˜•#2017-06-2915:27gfredericksif that's equivalent, I bet the s/and would actually generate just fine#2017-06-2915:27gfrederickssince I don't think s/keys generates any extra keys#2017-06-2915:37EdI'm having a problem with macro expansion and specs ... sometimes the specs are not checked#2017-06-2915:37Edhas anyone else seen this?#2017-06-2915:37gfredericksspecs on macros?#2017-06-2915:37Edi'm having trouble working out the circumstances where it works or doesn't#2017-06-2915:37Edyes#2017-06-2915:38gfredericksthese are specs about the compile-time forms, or the runtime data?#2017-06-2915:38Edi've narrowed down this test case:#2017-06-2915:39Edwhich fails if i use lein test#2017-06-2915:39Edbut passes if I load the code with C-c C-k in cider#2017-06-2915:39Ed@gfredericks compile-time forms#2017-06-2915:40gfredericks"load the code in cider" means actually running the test, not just compiling it?#2017-06-2915:41Edno ... load the code in cider means running cider-load-buffer#2017-06-2915:41Edand then either running the tests with C-c C-t C-n ... or running the tests at the repl#2017-06-2915:41stathissideris@gfredericks thanks, Iā€™ll give it a try#2017-06-2915:41Edwith run-tests#2017-06-2915:42gfredericks@l0st3d okay, so if you add (is false "dangit") as a second assertion in your test, it will fail there but not with the first assertion?#2017-06-2915:42Edyup#2017-06-2915:43gfredericksokay, I have no idea then#2017-06-2915:43Edit reports that the test was run, but that succeeded ...#2017-06-2915:44Edok ... @gfredericks ... thanks for your help#2017-06-2915:45Edi'm stuck then#2017-06-2915:50lwhortonspec is my new addiction. how did i ever do front-end development without you, my love?#2017-06-2915:51lwhortonif i need to build a feature but the backend team is busy and wonā€™t get to an api data endpoint in time, no worries: spec can auto-generate a static api response for me and I can go on my way building the html.#2017-06-2916:02gfredericks@l0st3d if I were you I would try to reproduce the problem without cider; I assume you've tried restarting the cider process?#2017-06-2916:03Edyes ... the 3 files I posted above always fail to check macro expansion when run from the terminal using lein#2017-06-2916:04Edcider seems to fix it ... which is how i didn't notice at first šŸ˜‰#2017-06-2916:10gfredericksoooh#2017-06-2916:10gfredericksI think I read the whole thing backwards#2017-06-2916:10gfredericksdue to ambiguity of "fail" and "pass" in this case#2017-06-2916:10gfredericksso it's leiningen that has the problem, not cider#2017-06-2916:11gfredericksin that case I'd try reproducing the problem using just java -cp ... clojure.main <file>#2017-06-2916:22Edyeah ... started doing that ... what's the minimal classpath that you need to start a repl with 1.9.0-alpha17?#2017-06-2916:23Edi've clearly got something missing#2017-06-2916:28gfredericksyou could try running lein classpath and using that#2017-06-2916:28gfrederickswhich reminds me that if you have deps in your ~/.lein/profiles.clj then it can be helpful to turn that stuff off via LEIN_NO_USER_PROFILES=1#2017-06-2916:28Edfair point#2017-06-2916:29Edthanks
#2017-06-2916:29Edthere's nothing in my lein profile#2017-06-2916:30Edah ....#2017-06-2916:30Edit's the ~'s#2017-06-2916:30Edzsh is expanding the first ~ but not the rest#2017-06-2916:30Edso it's not finding the jar on the class path ... ~/.m2 is not a dir#2017-06-2916:30Eddoh#2017-06-2916:31gfredericksšŸ˜­#2017-06-2916:31gfrederickscomputers are terrible#2017-06-2916:35EdšŸ˜‰#2017-06-2916:35Edyeah ... and the minimal case there works fine#2017-06-2916:53rickmoynihanare there any plans to attach doc strings to s/defs?#2017-06-2917:25mpenetIt might happen some day https://twitter.com/stuarthalloway/status/867362631216230400#2017-06-2917:29mpenetIn theory you could hack your own doc registry, but it d be nicer to have this built-in #2017-06-2917:30mpenetProper metadata support would be nice, not just docs imho#2017-06-2917:31rickmoynihanone other small issue I have is that if I have two namespaces foo.bar.baz and foo.bar.baz.impl and I want to add a spec alias for foo.bar.baz so I can do ::baz/blah inside foo.bar.baz.impl there doesnā€™t appear to be a way to do it when foo.bar.baz requires ā€¦impl. Basically Iā€™d like a version of alias that doesnā€™t require the namespace to be loaded.#2017-06-2917:33rickmoynihannot sure if thereā€™s a solution to that one yetā€¦ other than fully qualify the keywords#2017-06-2917:34rickmoynihanI guess that would also allow you to alias keywords to namespaces which donā€™t exist ā€” which would also be useful as keywords donā€™t have to map to namespaces#2017-06-2917:39Alex Miller (Clojure team)@rickmoynihan @mpenet yes, Rich, Stu, and I all want this, just havenā€™t gotten to it yet (and I think itā€™s trickier than it appears). would be happy to see someone dig in and suggest alternatives to move it forward#2017-06-2917:40rickmoynihan@alexmiller: doc strings or aliasing, or both?#2017-06-2917:40Alex Miller (Clojure team)that was about doc strings#2017-06-2917:41rickmoynihanand other metadata?#2017-06-2917:41Alex Miller (Clojure team)on aliasing, Rich has some thoughts on an approach to making keyword aliasing better, but I have not gotten info on it yet#2017-06-2917:41mpenetWhat's so tricky? #2017-06-2917:41Alex Miller (Clojure team)@rickmoynihan maybe. needs someone to dig in to see.#2017-06-2917:41Alex Miller (Clojure team)@mpenet where do you put them?#2017-06-2917:42Alex Miller (Clojure team)a spec can be a keyword reference to another spec - in that case, there is nowhere to hang meta#2017-06-2917:43mpenetCould be a simple metadata registry, atom with a map from kw -> metadata no?#2017-06-2917:43Alex Miller (Clojure team)I donā€™t think itā€™s useful to do so here in this chat, but someone needs to evaluate what the options are. if you want to write something up in the ticket or elsewhere, please do.#2017-06-2917:44mpenetSure, I can do that tomorrow morning (cooking dinner atm :))#2017-06-2917:44Alex Miller (Clojure team)off the top of my head some options include: explicit support in the spec protocol, hanging meta off the values in the spec registry, wrapping the values in the spec registry, creating a separate registry for docs or meta#2017-06-2917:45Alex Miller (Clojure team)these all have various pros and cons#2017-06-2917:45Alex Miller (Clojure team)they need to be checked against all the things that can be specs - sets, pred functions, anon functions, keyword refs to other specs, things that implement the Spec protocol, etc#2017-06-2917:47Alex Miller (Clojure team)if other kinds of meta, what are some examples? would some meta be set automatically by spec or is it all custom per use?#2017-06-2917:47Alex Miller (Clojure team)all of this stuff needs to be answered#2017-06-2918:00rickmoynihanyeah it seems like quite a big job - but spec has a pretty big surface area so not surprising#2017-06-2919:17lwhortoniā€™ve looked around at s/conformer and browsed a bit for conforming data with specā€¦ but it seems like a not-great idea to use spec as a sort of ā€œapi parserā€. is this correct?#2017-06-2919:17lwhortoni.e. if a service provides me some json, i canā€™t really write a spec on the client saying ā€œthis is my shapeā€, then somehow use spec to convert json -> my shape, can I?#2017-06-2919:18seancorfieldThe general advice is donā€™t coerce/transform data via s/conform unless you intend all potential clients of your spec to need that conversion!#2017-06-2919:19seancorfieldAt World Singles, we do some coercion with our API specs, but we already have JSON converted to Clojure data structures as input (using JSON middleware for Ring), and so we only coerce strings to keywords and a few other ā€œminorā€ things in our specs.#2017-06-2919:20seancorfieldSo Iā€™d recommend keeping the JSON -> Clojure data step separate from the Clojure data -> conformed input step.#2017-06-2919:20lwhortonokay, so considering that advice I guess i will write custom (parse-*-entity) which converts edn (xformed in a handler) from the wrong shape to edn of the client shape.#2017-06-2919:21lwhortonwould be really neat if thereā€™s a tool similar to spec that handles this, though iā€™m not really sure what it would look like. the real answer is have a properly shaped api response, but beggars cant be choosers#2017-06-2921:07wilkerluciolwhorton: I made a tutorial where I provide one solution to have coercion on top of specs (without affecting the conformance), check the section "Coercing results with spec" at https://medium.com/@wilkerlucio/implementing-custom-om-next-parsers-f20ca6db1664#2017-06-2923:19lwhortonthanks for the link. this is a good read regardless of coercing because i donā€™t know om very well#2017-06-2921:09wilkerlucio@lwhorton besides the simple version of coercions on the article, I have code for a more sophisticated one (but no explanations) at: https://gist.github.com/wilkerlucio/08fb5f858a12b86f63c76cf453cd90c0#2017-06-3008:16mpenet@alexmiller I got the ball moving here: https://dev.clojure.org/jira/browse/CLJ-2194. Looking forward to seeing what comes out of this.#2017-06-3008:31danielstocktonI have the following om.next style spec:
(s/def ::ident (s/and vector? (s/cat :ident keyword? :value #(not (coll? %)))))
(s/def ::join-key (s/or :prop keyword? :ident ::ident))
(s/def ::join (s/and (s/map-of ::join-key ::query) #(= (count %) 1)))
(s/def ::union (s/and (s/map-of keyword? ::query) #(> (count %) 1)))

(s/def ::param-expr
  (s/cat :query-expr ::query-expr
         :params map?))

(s/def ::mutation-expr
  (s/or :no-params (s/cat :mutate-key symbol?)
        :with-params (s/cat :mutate-key symbol?
                            :params map?)))

(s/def ::query-expr
  (s/or :prop keyword?
        :ident ::ident
        :mutation-expr ::mutation-expr
        :union ::union
        :join  ::join
        :param-expr ::param-expr))

(s/def ::query
  (s/or :recursion (s/or :depth number?
                         :unbounded #(= % '...))
        :query     (s/and vector?
                          (s/+ ::query-expr))))

(println (s/conform ::query [:one :two :three]))
(println (s/unform ::query (s/conform ::query [:one :two :three])))
Why does the first print give [:query [[:prop :one] [:prop :two] [:prop :three]]] but the second give (:one :two :three) (sequence not a vector)?
#2017-06-3008:32danielstocktonIf query is spec'ed as being a vector? shouldn't unform also return a vector?#2017-06-3010:01jwkoelewijnhi there, Iā€™m having some troubles regarding circular refs in my specs. I have a spec defining an office which has employees in the office ns, and I have an employee (in an employee ns), which should have an office. As you cn guess, this results in circular dependencies (or when I donā€™t require one of the namespaces, it results in Unable to resolve spec. Is there any recommended method to deal with issues like this?#2017-06-3013:44Alex Miller (Clojure team)@jwkoelewijn code would help. What you're describing should be possible#2017-06-3014:01rickmoynihanDoes anyone know of any ring middleware that pretty prints ex-infoā€˜s that raise spec explain-data? Thinking of something like prone for dev but for spec problems.#2017-06-3014:02rickmoynihanprone currently gives me a huge wall of text#2017-06-3014:02bbrinckHas anyone worked on a partial spec for specs? Or a test.check generator? Iā€™m mostly interested in generating specs rather than conformance or validation#2017-06-3014:52Alex Miller (Clojure team)bbrinck: https://dev.clojure.org/jira/browse/CLJ-2112 is about specs for specs. Those also are a step towards spec generators which Iā€™ve also done some more work thatā€™s not public yet.#2017-06-3015:01bbrinckExcellent news. Iā€™m voting for that one now šŸ˜‰#2017-06-3015:02bbrinckI may work on some generators soon. Iā€™m guessing it will be a subset of what youā€™ve done, but itā€™ll be public, so Iā€™m happy to share in case itā€™s useful.#2017-06-3015:25Alex Miller (Clojure team)Iā€™ve in particular worked on the generators in the direction of testing spec itself. Namely, if you have spec specs with working generators, you can generate specs, then use those to generate data, then validate that the generated values are valid according to the generated specs#2017-06-3015:26Alex Miller (Clojure team)the last time I worked on this I mostly convinced myself there was a bug in multi-spec gen for the non-keyword case that was giving me grief, but I never ran it down#2017-06-3018:50eraserhdI wonder how you spec that s/conform returns :clojure.spec/invalid as a valid result.#2017-06-3014:06bbrinck@rickmoynihan This isnā€™t directly applicable, but data-frisk includes some nice-looking spec messages. I wonder if that could be ported.#2017-06-3014:07bbrinckDo you need to return explain-data or could your raise an ex-info that contains explain (or some other string representation of the errors)?#2017-06-3014:10bbrinckIā€™m just wondering if maybe you could solve the issue by formatting the errors at the source. That might not be desirable though.#2017-06-3014:14danielstocktonCan I destructure metadata? For example, if I have a vector [1 2 3] which has meta {:some :meta} -> [:vector [1 2 3] :meta {:some :meta}]. I can't work out how I'd spec this.#2017-06-3014:15mpenet@danielstockton with a predicate I guess#2017-06-3014:16danielstockton@mpenet Tried that but I couldn't get destructuring as above when conforming. My understanding of spec is still fairly superficial.#2017-06-3014:19mpenetyou can do this with a custom conformer I think#2017-06-3014:20danielstocktonOk, I'll look into that, thanks.#2017-06-3014:21danielstocktonI don't need the conformed output exactly as above but I need to be able to easily identify the two parts.#2017-06-3014:22mpenet(s/conform (s/and vector? (s/conformer #(hash-map :vector % :meta (meta %))))) (with-meta {} {:foo 1}) -> {:vector [] :meta {:foo 1}}. this example conformer is a bit broken but that's the idea for the coercion part#2017-06-3014:28danielstocktonCool, why do you say it's broken?#2017-06-3014:29mpenetit's ok actually, before the edit it was šŸ™‚#2017-06-3014:29danielstocktonAm I right that conformer takes a second argument to unform? This might address the issue I had above with unforming to a seq.#2017-06-3014:29mpenettrue#2017-06-3014:30mpenetnever used unform but yes, it might work#2017-06-3014:30danielstocktonStill curious to know exactly why it unforms to a seq. I noticed for simpler specs (just vector?) it unforms correctly so it must be a side effects of something.#2017-06-3014:30mpenetI think that's a bug#2017-06-3014:30danielstocktonShould I be calling it a vector in the conformed output or is it a 'tuple'?#2017-06-3014:36danielstocktonThe source (if this is the right place) seems to unform to a vector: https://github.com/clojure/clojure/blob/d920ada9fab7e9b8342d28d8295a600a814c1d8a/src/clj/clojure/spec.clj#L874#2017-06-3014:37danielstocktonIf it is a bug, I can't work out where it comes from.#2017-06-3014:38wilkerlucio@danielstockton for that case I think a tuple makes more sense than a cat, a cat is for more regexy stuff#2017-06-3014:39wilkerlucioI think it unforms to a seq because you are using a cat, which is for regexy stuff, and I believe it will always conform to a seq#2017-06-3014:39danielstockton@wilkerlucio Which cat are you referring to?#2017-06-3014:40danielstocktonThe only ones I see are deeper in the spec#2017-06-3014:40wilkerluciothis one: (s/def ::ident (s/and vector? (s/cat :ident keyword? :value #(not (coll? %)))))#2017-06-3014:40danielstocktonOh, ident?#2017-06-3014:40wilkerlucioor I'm looking at the wrong place?#2017-06-3014:41danielstocktonI don't think the query I'm unforming has any idents: [:query [[:prop :one] [:prop :two] [:prop :three]]]. The [:prop :one] is the conformed value for a :prop ::query-expr which is just a keyword?#2017-06-3014:41danielstocktonThe original query is just [:one :two :three]#2017-06-3014:42danielstocktonI did think it might be related to a cat or similar, somewhere, but I couldn't explain it.#2017-06-3014:44wilkerlucioah, I think I was reading it wrongly, sorry, so it seems something is not aligned between conform and unform?#2017-06-3014:45danielstocktonI mean, perhaps I just misunderstand unform because it doesn't promise to give back exactly what you put it, only undo the destructuring..#2017-06-3014:45danielstocktonI'm just curious exactly what's causing this behaviour.#2017-06-3014:55Alex Miller (Clojure team)itā€™s unclear to what is not working as you expect from the back chat - can you restate?#2017-06-3014:56danielstocktonWhy does the first print give [:query [[:prop :one] [:prop :two] [:prop :three]]] but the second give (:one :two :three) (a lazy sequence, not a vector)?#2017-06-3015:01Alex Miller (Clojure team)looks like this is sufficient repro:
(s/def ::query (s/and vector? (s/+ keyword?)))
(println (s/conform ::query [:one :two :three]))
(println (s/unform ::query (s/conform ::query [:one :two :three])))
#2017-06-3015:02danielstocktonYep thanks, I made it look unnecessarily complicated.#2017-06-3015:03Alex Miller (Clojure team)so in conform, the general approach is to avoid rebuilding a collection if at all possible, so there you just end up with the original vector on conform#2017-06-3015:03Alex Miller (Clojure team)unform of a regex spec however will always (I think) yield a seq#2017-06-3015:05Alex Miller (Clojure team)unform of an s/and walks the preds in reverse#2017-06-3015:09danielstocktonMake sense, so it's the s/+ specifically. And if I wanted unform to yield the same collection type, I'd use a customer conformer with a second argument? I'm trying to use spec to implement a parser and I want to be able to go from query <-> ast. I don't know if this is a good use case for spec or not.#2017-06-3015:10Alex Miller (Clojure team)the broader problem here is: how do I use a regex spec but constrain to vectors and this is a known problem area that Rich and I have discussed at length as it came up a lot in specā€™ing the core macros#2017-06-3015:11Alex Miller (Clojure team)there is currently no simple solution that works as youā€™d like for all of conforming, generation, explain, and unconform (although you can make subsets of those work)#2017-06-3015:11Alex Miller (Clojure team)Iā€™m trying to work out if in this case you could actually somehow supply a custom unform function to the vector? pred as thatā€™s not something Iā€™ve tried#2017-06-3015:23Alex Miller (Clojure team)looks like there is no easy way to do so although there is certainly the potential to do so#2017-06-3015:43danielstocktonOk, I'll keep my ear to the ground.#2017-06-3018:53eraserhdCan i recursively override a generator? e.g. pass a function to s/spec's :gen that overrides just the recursive part, and close over such this also overrides the recursive part, making a depth-limited generator?#2017-06-3018:54eraserhdI've done this from scratch with clojure.test.check.generators, just trying to see if I can reuse most of what spec would have done.#2017-06-3022:05Alex Miller (Clojure team)Recursive generators are already depth limited based on the dynamic var in spec#2017-06-3022:05Alex Miller (Clojure team)In case that solves the problem#2017-07-0103:37bbloomsaw some javascript/flowtype code today that has this type:#2017-07-0103:37bbloomdeclare type Callback<T> = (error: ?Error, result: ?T) => void;#2017-07-0103:37bbloomand the code calling it had the common pattern: if (err) { log(ā€¦); return; } process(data)#2017-07-0103:37bbloombut to appease the type checker, had to add if (data) { ...#2017-07-0103:38bbloommeanwhile, spec: (s/alt :failure (s/cat :err ::error) :success (s/cat :err nil? :data ::whatever))#2017-07-0103:38bbloomsure, it isnā€™t checked statically - but at least it represents the actual right thing!#2017-07-0117:57arohner@bbloom soon šŸ™‚#2017-07-0117:57arohnerwell, not soon, but Iā€™m actively working on it#2017-07-0118:25bbloomšŸ™‚#2017-07-0202:13wildermuthnI recently read an article about how Reddit (or parts of it?) are switching to TypeScript, and one of their main criteria for choosing a compile-to-js lang was it needed to be typed. Been thinking about that, and having a lack of experience with typed languages, what other Clojurists think about that argument, particularly with respect to Clojure Spec. Link: https://redditblog.com/2017/06/30/why-we-chose-typescript/ Any articles or thoughts out there about how Spec fits into this question of typed vs. dynamic?#2017-07-0210:59carocadwildermuthn: you should probably check the core.typed library from clojure. It adds static type annotations on top of clojure code. Also afaik one of the current initiatives in google summer of code for clojure is precisely to use specs to create those static types. I think another project about that is spectrum which is already pursuing that initiative.#2017-07-0218:49seancorfieldI think developers either naturally lean towards types or not, and that influences their language choices. At World Singles, weā€™ve historically worked with dynamically typed languages and our forays into typed languages (C#, Scala) have not felt very ā€œnaturalā€ to us. Since we adopted Clojure in 2011, weā€™ve also tried Schema and core.typed, and given up on them, then tried them both again and given up on them again. Weā€™re heavy users of Spec, however, and we focus mostly on specifying data structures and using conform to validate (with a little coercion).#2017-07-0218:53seancorfieldMyself, personally, Iā€™ve worked with both statically typed and dynamically typed languages across most of my career and, whilst I like the idea of static types, I donā€™t generally like the reality of them. I was an early advocate of Haskell (because of my PhD work on FP language design in the 80's) but I have something of a love/hate relationship with Haskellā€™s type system šŸ™‚ So, overall, I think Iā€™m one of those that leans naturally to dynamically typed languages.#2017-07-0219:30carocad@U04V70XH6 would you mind sharing the reason why you abandoned the type annotations for Clojure? Although I have not used them I don't see a reason to abandon them completely. I think specially for libraries api it could be quite useful, even though personally I would prefer core.typed to infer the type of my arguments based on their spec I.e, go from the more generic approach to a more specific one and definitely avoid having to do both#2017-07-0219:32seancorfieldItā€™s very hard to provide type annotations that satisfy core.typed for some idiomatic Clojure code. Also, core.typed doesnā€™t do a great deal of type inference so you have to annotate a lot of code to satisfy core.typed. The former means you often need to rewrite Clojure to a less idiomatic style to satisfy the type checker. The latter means you end up with a lot of ā€œnoiseā€ in your code.#2017-07-0219:34seancorfieldI donā€™t remember how many of my discussions with Ambrose about this were done on public mailing lists. Some. So maybe look in the archives (I think thereā€™s a core.typed mailing list?).#2017-07-0219:37seancorfieldBy contrast, Spec is very much purely opt-in. You can specify just what you want, and test for validity/conformance exactly where you want. You can specify just data or just functions or a bit of both. You can choose exactly what to instrument and when. And you also get the benefit of being able to generate conforming test data and conduct generative testing as and when you need it.#2017-07-0219:44carocadI guess the marketing idea of core type is greater that its implementation. Have you actually tried spectrum? I know it is quite experimental but I think it child be a great tool, although from what I have read you also have to annotate a lot of code for it to stop complaining#2017-07-0219:45seancorfieldWe have not tried Spectrum yet. It seemed a bit too experimental for production code at present.#2017-07-0219:48seancorfieldAnd to be fair to core.typed, itā€™s improved dramatically over the years and it is an amazing piece of engineering work. Clojure is just a really hard language to perform full and useful type inference on so itā€™s a brutally hard problem to solve. I understand that the latest work being done focuses handling the boundary between typed code and untyped code and that Ambrose is working with the Typed Racket folks on that?#2017-07-0219:48carocadYeah I'm looking forward to the results of this gsoc and see the result of spec/type projects (hopefully they are implemented) :)#2017-07-0219:49seancorfieldPart of my PhD work was looking at type inference so I know how difficult this problem can be with some languages šŸ˜#2017-07-0219:52seancorfieldThis was a fascinating talk at Clojure/West BTW http://2017.clojurewest.org/emina_torlak/ ā€” back in the late 80's I was working with MALPAS which was a program verification system and we were transforming C code for analysis and verification. Eminaā€™s talk showed both how far this area of research has come in three decades, as well as how difficult it is and how slow progress really is.#2017-07-0219:53carocadMind if I ask: in your opinion is the type inference that core.typed is doing "better" than for example flow for JavaScript. I guess I don't have to ask about java :/#2017-07-0219:53seancorfieldI donā€™t follow anything in the JS world ā€” canā€™t stand the language šŸ™‚#2017-07-0219:56carocadWell funny because flow is written in Ocaml so I thought the techniques might be familiar or general for type inference. So I was wondering about their difference. I guess I ignore too much about the topic šŸ˜«#2017-07-0220:17seancorfieldThe implementation language has no bearing on the language you're analyzing, interpreting, or compiling. I implemented both APL and my various experimental FP languages using Pascal, and I wrote the type inference engine in Prolog simple_smile #2017-07-0220:18seancorfieldGiven a choice back then I'd have preferred Algol 68 to Pascal but my reviewers were not familiar enough with Algol 68 so I wasn't allowed. #2017-07-0220:31carocadJesus I don't even know those languages ... :/#2017-07-0220:48seancorfieldThe "Pragmatic Programmer" suggests learning a new language every year -- good advice! šŸ˜ˆ #2017-07-0301:03wildermuthnGood info here! @U04V70XH6, did you find that using typed langs helped to do the things the article said it wanted: catch bugs earlier, make it easier to refactor, have a larger team working on the codebase?#2017-07-0301:03wildermuthnAnd related, whether Spec can actually address those issues?#2017-07-0301:04wildermuthnMy cljs team uses Spec mostly to conform/validate api request/responses. But thinking about whether to expand it to be a keyword namespaced kind of typing.#2017-07-0301:37seancorfieldSure, a type system will catch some bugs earlier (by definition, since it takes effect before your code even runs) but a type system canā€™t catch all classes of bugs. In the context of Clojure ā€” working with small, pure functions on immutable data (for the most part), and building up code with a REPL ā€” there are whole class of bugs you wonā€™t introduce in the first place. I donā€™t think thereā€™s been enough analysis in the FP arena comparing static/dynamic type systems for any conclusion to be drawn. If you compare, say, Java and Kotlin, you have two languages with static types and one catches potential NPEs (Kotlin) and one doesnā€™t. But if you look at Clojure, nil-punning is idiomatic and makes annotating code much, much harder because ā€œnilableā€ infests all the types in many places in your code. Type signatures for generic code tend to become very complex ā€” try writing a Spec for map (and remember its return type will be different with just a function argument compared to with one or collection arguments).#2017-07-0301:39seancorfieldSpec isnā€™t a type system and youā€™re not really comparing apples to apples if you try to compare it to one. I donā€™t know what Typed Clojure (in a true language sense, not core.typed) would look like but I donā€™t think it would be much like the Clojure we know and love today.#2017-07-0307:13carocad@U04V70XH6 I partially disagree with what you said. Building up code in the REPL and having small pure functions sure helps but there are times where the datastructures are soo deep that creating those by hand becomes a nightmare. Specially in those cases I find a type system to be way more useful than a repl. It gives you a first overview of whether a big refactoring is going in the right direction. It is not something to rely on, of course, but I think it eases those tasks#2017-07-0310:29seancorfield@U0LJU20SJ I'm curious about the business domain where such deeply nested structures occur? It's not something I've ever run into. I'd expect Spec to be very helpful in such a situation then?#2017-07-0313:56carocad@U04V70XH6 take a look at the response from Mapbox Directions API: https://www.mapbox.com/api-documentation/?language=JavaScript#route-object. They need to returns lots of objects in a json api. In my case I am trying to provide a customized routing which means that I have to mimic their api response. The problem obviously arises once they change (currently they are at v5). In those cases you have to throw away pretty much every spec that you had for that and start refactoring lots of code. That is what I meant with deeply nested objects involved in a big refactoring process. I guess @U07CTDKT7 tried to raise that topic as well.#2017-07-0314:00carocadTo mention another example. Since we are not all senior developers ( šŸ˜‰ ) we might just go with the first idea of what you should return at a specific place. Later on as you learn more both about the topic and about the language you realize that you need a better design for it. More often than not, those are deeply nested objects which require quite some refactor. In that case, having a static type analyzer to back you up is great. Obviously that means that the system was poorly designed from the beginning but then again ā€¦ we beginners šŸ˜›#2017-07-0315:21wildermuthn@U04V70XH6, what you are saying about immutablity, pure functions, and the repl being a bug-reducer has been my experience too (js vs ClojureScript). @U0LJU20SJ, I've had the same case with denormalized api responses. We're using DataScript to renormalize the data, but that's really tedious (using Spec has helped). In terms of refactoring, though, my impression is that not having to deal with types makes refactoring easier (but not simpler?). I'm impressed by Rich Hickey's point in his Spec talks that it is probably better to just have new functions that to modify old ones in a breaking manner (which Spec should make easier to identify). #2017-07-0315:22wildermuthnBut this would mean Spec'ing every function you ever write! :) Which makes me wonder if it is worthwhile. #2017-07-0315:25carocad@U07CTDKT7 I actually tried to spec every function before as well and it was a nightmare. But at some I realized that specifying the API functions is actually enough. If the user facing function works as expected, then all other internal ones should also do it, otherwise the external would have failed. At least that is the way that I am approaching the problem now šŸ™‚#2017-07-0316:46seancorfieldAgreed, re: specā€™ing lots of functions. We spec key data structures (ā€œdata as APIā€) and we spec some functions. Nearly all of our use of Spec is explicit (`conform`, valid?) rather than implicit (`instrument`). Regarding deeply nested API results ā€” Iā€™d probably wrap the API and flatten results out quite a bit and then work with that everywhere else in my code, probably with namespaced keys. Easier to read and manipulate. Inside the wrapper, Iā€™d probably use Spec to describe the exact API result to better support the transformation to a flatter data structure.#2017-07-0217:20stathissiderisJust released spec-provider v0.4.9, now with optional numerical range predicates https://github.com/stathissideris/spec-provider#inferring-specs-with-numerical-ranges#2017-07-0309:51yendais there a "native" way in spec to specify a map with either key a or b ?#2017-07-0309:52mpenet(s/keys :req-un [(or ::a ::b)]) i think#2017-07-0309:53yenda@mpenet I forgot to mention it is either a or b not the two so currently I'm using a xor
(defn xor [x y]
  (or (and x (not y))
      (and y (not x))))
#2017-07-0309:56yendamaybe there is no way around it because it is going against spec to forbid things#2017-07-0309:56mpenetI was about to say that, especially for maps#2017-07-0316:42ikitommiHere too: [metosin/spec-tools "0.3.0"] is out with support for Swagger2. More info at #announcements #2017-07-0316:55joshjones@yenda your xor function used in this way gives you what you want
(s/def ::a (s/nilable neg-int?))
(s/def ::b (s/nilable pos-int?))
(s/def ::x string?)

(s/def ::mapspec (s/keys :req-un [::x] :opt-un [::a ::b]))

(s/def ::abspec #(let [a (find % :a)
                       b (find % :b)]
                   (xor a b)))

(s/def ::finalspec (s/and ::mapspec ::abspec))
using find to ensure that even if a and b are nilable, the xor will still be enforced
#2017-07-0316:56joshjonesand using opt-un in the mapspec ensures that non-namespaced keywords a and b will still be validated#2017-07-0316:59joshjones
(s/valid? ::finalspec {:x "yo" :a 100})
=> false
(s/valid? ::finalspec {:x "yo" :a -100})
=> true
(s/valid? ::finalspec {:x "yo" :a 100})
=> false
(s/valid? ::finalspec {:x "yo" :a -100 :b 100})
=> false
(s/valid? ::finalspec {:x "yo" :a nil :b nil})
=> false
(s/valid? ::finalspec {:x "yo" :b 42})
=> true
#2017-07-0317:00joshjonesThough it's still preferable to not add a restriction like this, as having both keys should do no harm, it's good to know that spec provides the flexibility for this to work#2017-07-0410:17dm3how do you properly spec a structure where the contents depend on type of one of the fields? E.g.
(def m {:standard/source :the-source, :standard/content {:standard.content/details {... specific to :the-source ...}})
Iā€™d like to make a spec for :standard.content/details based on :the-source
#2017-07-0410:19dm3I canā€™t see the way out except for having :standard.content/details renamed to :the-source/details - is there any?#2017-07-0410:21andrewmcveighSounds like a job for multi-spec#2017-07-0411:04rickmoynihanis it just me or does (s/keys :req [(or ::foo ::bar)]) default generator always generate both ::foo and ::bar rather than one or the other, and both?#2017-07-0411:09rickmoynihanI appreciate there are other ways to write this by moving the or over two keys specs#2017-07-0411:13bronsathere's alraedy a ticket for this IIRC#2017-07-0411:15bronsahttps://dev.clojure.org/jira/browse/CLJ-2046#2017-07-0411:46rickmoynihanthanks#2017-07-0414:11danielstockton
(s/def ::test (s/conformer (fn [q] q) (fn [q] (apply list q))))
(s/def ::test2 (s/map-of keyword? ::test))

(println (s/unform ::test [:a :b :c]))
(println (s/unform ::test2 {:some-key [:a :b :c]}))
The first unform works but not the second. Can anyone help?
#2017-07-0414:17rickmoynihanshouldnā€™t it be {::test [:a :b :c]}#2017-07-0414:22rickmoynihanforgive me itā€™s map-ofā€¦ seems strange then that it doesnā€™t work.#2017-07-0414:25danielstocktonI also tried (println (s/unform ::test2 (s/conform ::test2 {:test [:a :b :c]}))) just in case.#2017-07-0414:28danielstocktonSorry, I'll rename that keyword to avoid confusion.#2017-07-0414:39bronsa@danielstockton it's already been fixed in the latest spec version#2017-07-0414:40danielstocktonI'm on alpha17, has it been released?#2017-07-0414:40bronsatry adding an explicit dependency to [org.clojure/spec.alpha "0.1.123"]#2017-07-0414:40danielstocktonOk, will do, thanks.#2017-07-0414:40bronsaaltho I thought alpha17 depended on that#2017-07-0414:42bronsauh#2017-07-0414:43bronsanevermind, this seems to be a different bug than the one I was thinking about (https://dev.clojure.org/jira/browse/CLJ-2076)#2017-07-0414:44danielstocktonIt looks pretty similar.#2017-07-0414:48bronsaah yeah, it is. sorry, I tested on a broken repl#2017-07-0414:49danielstocktonMe too, I think I need cljs.spec#2017-07-0414:52danielstocktonYes, I updated to the latest cljs and it works. Thanks!#2017-07-0420:12danielstocktonI think I may have found another bug in conform/unform:
(s/def ::join-key (s/or :prop keyword? :ident ::ident))
(s/def ::join (s/map-of ::join-key ::query))

(s/def ::query-expr
  (s/or :prop keyword?
        :join  ::join))

(s/def ::query
  (s/or :query (s/and (s/conformer #(if (vector? %)
                                      %
                                      :clojure.spec/invalid)
                                   list)
                      (s/+ ::query-expr))
        :not-query nil?))

(println (s/conform ::join {:a [:b]}))
(println (s/unform ::join (s/conform ::join {:a [:b]})))
Uncaught Error: nth not supported on this type cljs.core/Keyword
It seems to be due to ::join-key in map-of, if I replace that with keyword? then it works.
#2017-07-0420:13danielstocktonOh no, sorry. I need to read the documentation better. There is an option to conform keys for map-of.#2017-07-0420:16danielstocktonPerhaps it shouldn't throw an error though?#2017-07-0420:16danielstocktonIf the key isn't conformed, unform could just return it as is.#2017-07-0420:46jcfHi all. Has anyone come up with an elegant way of specifying :args to a zero-arity function?
(s/fdef f
  :args empty? ; kinda cryptic, no?
  :ret map?)
empty? kinda does what I need I think because it'd end up applying an empty list of args to the function, but that seems a little cryptic.
#2017-07-0420:47jcfI guess I could use (s/coll-of any? :count 0) but again that doesn't sit well with me.#2017-07-0420:48jcfMaybe I'll create ::none somewhere in my codebase to return an empty collection (s/coll-of any? :count 0 :into []).#2017-07-0420:48jcfOr ::zero-args to make it clear what I'm doing.#2017-07-0420:50jcf
(s/def ::zero-args
  #{[]})
That works but I don't ā¤ļø it.
#2017-07-0421:29gfredericks@jcf (s/cat)?#2017-07-0510:13curlyfryHi, I just realized that spec/instrument is incompatible with event handler-type systems like re-frame (systems where functions are stored in a data structure prior to instrument being called). Since instrument wraps the original var, the function calls that are made through lookup in a data structure (e.g a map of event->handler-fn) aren't spec checked. Since this is a clear majority of the calls being made in for example re-frame, the value of spec:ing (with fdef) the handler functions is decreased. Is there any way around this at all? I guess you could add the vars themselves in the map instead of the functions they point to, but that seems like a big change to make in an existing system, and feels pretty unidiomatic.#2017-07-0510:35carocadcurlyfry: why do you need to fdef the function? doesnt it suffice to just spec your :db such that it is always in the right state? you could also avoid instrumenting your functions and have them tested separatedly#2017-07-0511:02curlyfryI'd like to use the philosophy of spec: instrument ensures that functions are called correctly, tests ensures that they have the correct return values. Checking the whole app-db after every event is also not viable when the app gets large.#2017-07-0511:30carocad> tests ensures that they have the correct return values. not really. You can fdef a function without instrumenting it. Then on your test spec will generate random input against which you can check the output of the function and the relation between them. > Iā€™d like to use the philosophy of spec: instrument ensures that functions are called correctly If you only interested in that then I dont know how to šŸ˜ž#2017-07-0511:40curlyfryWhat do you mean by "not really"? You're describing exactly what I mean :) Some discussion on the topic: https://groups.google.com/forum/m/#!msg/clojure/JU6EmjtbRiQ/WSrueFvsBQAJ#2017-07-0514:40carocadhehe sorry for the confusion. In retrospective my message is quite confusing šŸ˜„. What I meant is that if you can test your function with generated random input and it returns what you expect then instrumenting at runtime is not necessary since you can be sure of the integrity of your function. It is definitely not so dynamic since changing your functions would break that certainty but so far I dont know any other option#2017-07-0515:01curlyfryI guess I'd just prefer to be able to do both... Anyway, thanks a lot for your input! :)#2017-07-0515:12carocadoh another idea. Can you not redefine the handlers so that re-frame uses the new instrumented functions? I use Cursive so I usually just do sync file to repl and it automatically loads all changed files which would trigger the reg-event-handler ā€¦ I guess. Have not tried it yet šŸ™‚ Hope it helps#2017-07-0600:07danielcompton@U0J30HBRS with re-frame in particular I suspect we'll have to add support for turning on spec checking so you can do this. Would be nice to integrate with spec/instrument, but doesn't seem like its possible#2017-07-0810:05stathissiderisa similar problem is when you have a chain of ring handlers and you reload the code for one of them, but because the data structure is already referring to the original var, reloading the handler has not effect. In cases like that, Iā€™ve seen people adding the original vars as you suggested and I donā€™t think itā€™s unidiomatic. Just a bit unusual!#2017-07-0510:16andrewmcveighI guess you could potentially spec the data structure and use fspec at the leaves where the event-handler functions are.#2017-07-0510:16andrewmcveighSeems complicated though#2017-07-0510:21danielstocktonNot possible to def the functions outside the datastructure?#2017-07-0510:22danielstockton(defn my-fn [x y z] ...) (def handler {:event1 my-fn})#2017-07-0511:30curlyfry@danielstockton That's how I do it. The problem is that the functions, not the vars, are what are added to the datastructure. If you try it out, you'll notice that the function calls made from lookups in the map aren't checked. In my particular case the functions are added to a data structure behind the scenes, by re-frames reg-event.#2017-07-0512:39rickmoynihanIā€™m trying to instrument an fdefā€™d function with a stub and a custom generator but get an exception ClassCastException clojure.test.check.generators.Generator cannot be cast to clojure.lang.IFn clojure.spec.alpha/gensub (alpha.clj:268)
(defn foo [a]
    (+ a a))

  (s/def ::num number?)

  (s/fdef foo
          :args (s/cat :a ::num)
          :ret number?)

  (st/instrument `foo {:stub #{`foo} :gen {::num (g/elements [1])}})
#2017-07-0512:40rickmoynihanthe instrument call itself raises the exception#2017-07-0512:42rickmoynihanahh I see the problem - I need to obviously gen the return value not the argsā€¦ doh#2017-07-0512:43rickmoynihanchanging it to :ret ::num works#2017-07-0513:01andrewmcveighIs it that (g/elements [1]) should be #(g/elements [1])?#2017-07-0513:01rickmoynihanahh yeah sorry Iā€™d fixed that locally too#2017-07-0513:01andrewmcveighnp#2017-07-0513:02rickmoynihanactually thatā€™s one thing Iā€™m never really clear onā€¦ when to pass a generator and when to pass a 0-arg function returning a generatorā€¦#2017-07-0513:02rickmoynihanis there a resource somewhere explaining that?#2017-07-0513:04andrewmcveighI think you always need to give a function to something that comes from spec (and is deffed). As far as I worked out, it's to do with the way spec lazy-loads test.check. Not seen it written down anyway though.#2017-07-0513:05andrewmcveighI can't think of a place where spec requires a plain generator.#2017-07-0513:05andrewmcveighOnly if you're calling test.check stuff directly#2017-07-0513:21rickmoynihanok#2017-07-0513:21rickmoynihanat least I know the heuristic now šŸ™‚#2017-07-0513:51rickmoynihanIs it just me or does instrument with a stub also try and generate conforming values for the :args spec; even though itā€™ll never need a generator for them - as it only needs to generate return values?#2017-07-0513:53rickmoynihanbasically Iā€™m getting a such-that error on instrument but if I replace the :args spec with something like (s/* any?) it works.#2017-07-0514:05jannisHi folks! Is there a good approach to writing function specs and running clojure.spec.test.alpha/check in a way that allows functions to either return a value (checked with :ret) or throw an exception?#2017-07-0514:09jannisOr, simplified, is there a way to spec a function so that the fact that it throws under certain conditions is part of the spec?#2017-07-0514:24wilkerlucio@jannis I remember reading something here about spec not contemplating exceptional cases, I believe the idea is that specs are about data only#2017-07-0514:25jannisYes, thatā€™s what I was thinking.#2017-07-0514:26jannisI think Iā€™ve found a solution for myself: have the spec verify the data is ok, then have the function only operate on valid data. That way generative tests only generate valid data but invalid data would still be caught at runtime if I use instrumentation.#2017-07-0514:26jannisThat completely eliminates the need for exceptions. šŸ™‚#2017-07-0514:35joshjonesYou donā€™t want to use instrument at runtime, generally speaking @jannis#2017-07-0514:36jannisI think I doā€”but only while developing.#2017-07-0514:36jannisAnd while not testing performance perhaps šŸ˜‰#2017-07-0514:36joshjonesyes, thatā€™s what i meant and should have said ā€œproductionā€ šŸ˜‰#2017-07-0514:38jannis@joshjones No worries, I assumed thatā€™s what you meant already.#2017-07-0520:38hkjelsIs there a clever way to get the merged form of a spec that uses spec/merge?#2017-07-0522:20rickmoynihanJust ran into this issue ( https://dev.clojure.org/jira/browse/CLJ-1989 ) and surprised that the lazy load coverage doesnā€™t extend to all of test.check.generators#2017-07-0601:42gfrederickslet in particular is a macro, and the only macro of the test.check generators#2017-07-0601:42gfredericksand it looks like the patch doesn't lazy-load it, it completely reimplements it, with different behavior#2017-07-0601:44gfredericksI'm not sure if lazy-loading a macro is even possible#2017-07-0610:05jimmyhi guys, is there any good way to test all fdef in my project instead of using clojure.spec.test.alpha/check for each one explicitly.#2017-07-0610:18rickmoynihan@nxqd: guessing you mean fdef not fspec but if you call check with 0-args it should run all var specs that have been loaded#2017-07-0610:19jimmyrickmoynihan: really, nice. I will give it a try. Thanks#2017-07-0610:20jimmynice, it works šŸ™‚#2017-07-0610:20rickmoynihancool#2017-07-0610:27jimmybtw, does stest/check run all the tests in parallel ?#2017-07-0610:51rickmoynihanno idea - guessing not#2017-07-0614:48englishstest/check does parallelize generative tests using pmap https://github.com/clojure/spec.alpha/blob/8f4e6f7d53db30b20d1f7b1c190f21f40d12ca49/src/main/clojure/clojure/spec/test/alpha.clj#L410-L411#2017-07-0909:19jimmy@U06A9614K thanks for your info ( And sorry for my lazy ass ... )#2017-07-0610:21rickmoynihanHmmmā€¦ wondering if generator errors like ExceptionInfo Couldn't satisfy such-that predicate after 100 tries. clojure.core/ex-info (core.clj:4725) would be better if they told you which generator/spec failed to generate a value. The stack trace doesnā€™t seem to contain many useful clues.#2017-07-0610:24rickmoynihanWould be nice if the ex-info data contained something like {:generator #function[foo.bar/bar-gen] :spec :foo.bar/valid-bar?}#2017-07-0611:19gfredericksrickmoynihan: test.check didn't provide a mechanism for this until 0.10.0-alpha1; now that that's released, clojure.spec can be modified to make use of it#2017-07-0611:25rickmoynihanCool! One thing Iā€™m not quite clear on is the precise relationship between clojure.spec and test.check. I obviously understand spec uses it, and wraps it via the lazy-load mechanism. But is spec promising to expose everything inside test.check (plus its own stuff), or is it deciding on what to expose/not-expose? If so how are those decisions being made?#2017-07-0611:30gfredericksthe exposing-vs-not distinction isn't very impactful since you can reference test.check generators directly in your spec usage; I don't think that was ever disintended#2017-07-0611:31gfredericksI don't know about why some generators don't have proxy versions in spec (other than let)#2017-07-0611:33rickmoynihanYeah, Iā€™ve been thinking I might be better just using test.check.generators rather than specs lazy loaded onesā€¦ What would the disadvantage be of doing that? Presumably Iā€™d have to make sure the non-lazy-loaded generators were only ever included in dev envs?#2017-07-0611:34rickmoynihanIt really frustrates me that the lazy loaded ones donā€™t have docstrings.#2017-07-0611:34gfredericks"have to" is strong, depends on how much you care about your production deps#2017-07-0611:34gfredericksthat sounds fixable#2017-07-0611:35rickmoynihan> depends on how much you care about your production deps Yeah, I guess I donā€™t when its an app. A library should probably be more conservative though.#2017-07-0611:35gfredericksoh definitely, yes#2017-07-0611:35gfredericksbut yeah, you could stick the generators in /test or /dev or something, and then have your own sort of lazy loading thing if you need to reference them from /src#2017-07-0611:36rickmoynihanits a shame about let not being included. It feels much nicer than fmap and others.#2017-07-0611:37rickmoynihanbut understand the macroness of it complicates it#2017-07-0611:37gfredericksyeah; I suppose you could have a weird thing where the macro tries to require test.check and if it fails it expands to exception-throwing code#2017-07-0611:38rickmoynihanThat seems like a nice idea#2017-07-0611:41rickmoynihanAs an aside are you involved at all in the development of clojure.spec beyond your work on test.check?#2017-07-0611:51gfredericksnope#2017-07-0611:58gfredericksnot beyond the details of the integration#2017-07-0611:59rickmoynihancool#2017-07-0612:00rickmoynihanThanks for explaining, and all your work on test.check. btw, I updated the CLJ-1989 with your comments on let. Hope you donā€™t mind, and that they accurately reflect your opinion: https://dev.clojure.org/jira/browse/CLJ-1989?focusedCommentId=46211#comment-46211#2017-07-0612:17gfredericksonly nitpick is that the macro wouldn't expand to a require call, it would actually call require before constructing any expanded code#2017-07-0612:18gfrederickse.g.,
(defmacro let [& args] (try (require ...) `(...) (catch Exception e `(throw ...))))
#2017-07-0612:23rickmoynihanyeah fair point, thatā€™s what I meantā€¦ will update the ticket#2017-07-0612:28rickmoynihanok updated https://dev.clojure.org/jira/browse/CLJ-1989?focusedCommentId=46211&amp;page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-46211#2017-07-0610:53rickmoynihanHmm also just noticed a small problem with check which is that you canā€™t both run it on all fdefs and provide custom options#2017-07-0610:53rickmoynihanahh its ok checkable-syms is exposed so can do it by hand#2017-07-0612:15rickmoynihan@gfredericks: Do you know if test.check 0.10.0-alpha2 is compatible with the latest clj 1.9 / spec?#2017-07-0612:18gfredericks@rickmoynihan should be, yeah#2017-07-0612:22rickmoynihanawesome#2017-07-0617:48peejaAre we not supposed to be able to add generators to predicates? Is that a closed set?#2017-07-0617:48peeja(That is, if I define a predicate, am I not able to add a generator for it?)#2017-07-0618:02wilkerlucio@peeja I'm not sure what you asked, but I think you might want to check (s/with-gen)#2017-07-0618:02wilkerluciothen you can set any generator you want to your spec, no mater the predicate#2017-07-0618:02peejaRight, I mean I want to register that as the generator to use everywhere (if it's not overridden)#2017-07-0618:02hlshipI'm missing something about combining s/and and s/or: ` (s/def ::identifier (s/and (s/or :keyword simple-keyword? :symbol simple-symbol?) graphql-identifier?))#2017-07-0618:03wilkerlucio@peeja what you can do is define a spec, and then re-use that spec instead, eg:#2017-07-0618:03wilkerlucio
(s/def ::my-spec (s/with-gen my-pred? #(gen-fn)))

(s/def ::other-spec ::my-spec)
(s/def ::another-one (s/and ::my-spec other?))
#2017-07-0618:04wilkerluciothen it will inherit the generator#2017-07-0618:05wilkerlucio@hlship on the and case, the next predicate will receive the conformed version from the previous one#2017-07-0618:06hlship(s/explain ::identifier :foo) java.lang.ClassCastException: clojure.lang.MapEntry cannot be cast to clojure.lang.Named ` I'm guessing that the entire map conformed by s/or (e.g., {:keyword :foo}) is being passed to graphql-identifier? (which expects a String, Keyword, or Symbol). So, what's the correct way to deal with this situation? I'd rather not do the identifier check first. I guess I could change graphql-identifier? to expect the conformed result from the s/or?#2017-07-0618:06wilkerlucioso, your graphql-identifier? will receive something like {:keyword :bla} or {:symbol some-sym}#2017-07-0618:06peeja@wilkerlucio Yeah, that makes sense. I was hoping to do it straight on the predicate because it looks like a nicer API, but if that's not how it's meant to work, then that works.#2017-07-0618:08hlshipIt seems like it gets the Map$Entry, e.g, [:symbol foo].#2017-07-0618:08wilkerlucio@peeja if you look at the sources for the spec generators, it uses a map to convert from symbol -> generator, I guess we can suggest this to be replaced with a multi-method making it extensible from the outside#2017-07-0618:08hlshipok, I'm on that track.
#2017-07-0618:09wilkerlucio@peeja check this: https://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/gen/alpha.clj#L132-L185#2017-07-0618:50peejaIs there any way to use with-gen without blowing away generator overrides?#2017-07-0619:01peejaIt seems like it can't, since it doesn't have access to the overrides like gen* does#2017-07-0619:01peejaAre we meant to define our own macros a la keys/`map-spec-impl` etc?#2017-07-0717:15englishIā€™ve been working on a Ruby port of spec: https://github.com/english/speculation Implementation-wise itā€™s got all the important stuff: data validation, instrumentation and generative testing. Iā€™m currently working on improving the docs and writing up some guides. If there are any Rubyists here jealous of clojureā€™s spec, Iā€™d been keen to hear your thoughts and feedback!#2017-07-0909:21jimmyenglish: hey this is amazing. It's a good lib for me to train some junior Ruby dev before converting them to Clojure completely šŸ˜„#2017-07-0916:58englishThanks! The api is pretty much the same so transitioning should be straightforward#2017-07-0719:53ghadiWhat is the farthest along library with making nice experiences for spec-based errors?#2017-07-0802:07xiongtxIs the file/line/col of spec definitions stored anywhere, as they are for vars when the latter are read? Itā€™d be nice to have that information so we can jump to a specā€™s definition.#2017-07-0806:04richiardiandrea@xiongtx there has been some plan for that if I remember correctly, unfortunately I cannot find the reference (I think it was exactly in this channel)#2017-07-0817:44jsselmanHow can I use the result of clojure.spec.test/check to see if a test failed? The doc seems to hint at checking the value of the :failure key, but that value is false in my test despite there being a clear check error#2017-07-0817:47gfredericks@jsselman is it false or [false]?#2017-07-0817:47jsselmanprintln shows it as false#2017-07-0817:48gfrederickshmm#2017-07-0817:48jsselmanI should mention this is ClojureScript not Clojure#2017-07-0817:48gfrederickswhat's the full return?#2017-07-0817:51jsselmanthe function call is just (stest/check ``sut/normalize)`#2017-07-0817:51jsselmanarg how do i insert a literal backtick#2017-07-0817:51gfredericksthe value under [:clojure.test.check/ret :result] should work#2017-07-0817:52gfredericksI have no idea what the failure key is meant to mean#2017-07-0817:52jsselmanthe value there is false, does that indicate the test failed?#2017-07-0817:52gfredericksyes#2017-07-0817:52jsselmani guess i can quickly check that#2017-07-0817:56jsselman@gfredericks that does work, thanks!#2017-07-0817:59jsselmanThe spec guide shows an example using
(stest/abbrev-result (first (stest/check `ranged-rand)))
, but here it loses the failure. Guess I can manually check
#2017-07-0817:59jsselman(I get {:sym raytrace-weekend.vec3/normalize, :failure false})#2017-07-0900:15mathpunkThis didn't seem fancy, but it gave me a stack overflow: (s/coll-of map? :kind seqable?)#2017-07-0900:16mathpunkwhat am I failing to understand here?#2017-07-0900:20mathpunkIn case it's not clear, I'm attempting to express, "This here data is a sequence of maps"#2017-07-0900:22mathpunkSince another way to express that is, (s/* map?), I'm using that. But what a strange error that was. (SOLVED)#2017-07-0900:22gfredericksis seqable? a valid :kind?#2017-07-0900:56mathpunk"`:kind` - a predicate or spec that the incoming collection must satisfy, such as vector?"#2017-07-0900:57mathpunkmind you, I'm getting seqable? out of clojure.future.spec#2017-07-0900:57mathpunkmaybe i'm holding it wrong somehow#2017-07-0911:56gfredericksI'm planning to release schpec with defn+spec today https://github.com/gfredericks/schpec/commit/4b0de0bc836111fc9953e37c68ca1756a0d09bb8 /cc @olivergeorge#2017-07-0914:12mpenetCouldn't it be made more efficient by conforming 1 arg at a time instead of creating a s/cat at invoke time? That would also allow to avoid & args?#2017-07-0914:13mpenetAh but you do this to pattern match depending on arg type for same arg length. Nevermind#2017-07-0914:14mpenetCool stuff#2017-07-0923:07Oliver Georgehey wow. that goes beyond typing the defns. interesting stuff.#2017-07-0923:09Oliver GeorgeI've been looking at porting some ocaml code. they rely heavily on functions with match statements which looks a lot like this. (I've been using core.match)#2017-07-0923:10Oliver GeorgeHere's a random example: https://github.com/links-lang/links/blob/master/queryshredding.ml#L1234#2017-07-0923:11Oliver GeorgeThat's pretty much sugar for the match statement#2017-07-0923:11Oliver Georgehttps://github.com/links-lang/links/blob/master/queryshredding.ml#L1153#2017-07-1001:01gfrederickscool#2017-07-0920:31gfredericksokay, I released defn+spec in case you like that sort of thing https://github.com/gfredericks/schpec#comgfredericksschpecdefnspecdefn#2017-07-1005:28anticrisisHiya folks, does this look like a valid spec? (s/def ::test (s/cat :k keyword? :rest (s/* any?)))#2017-07-1005:28anticrisisThe reason I ask is I get different results with s/valid? and an instrumented function#2017-07-1005:32anticrisisI.e. (s/valid? ::test [:foo]) => true but when used in s/fdef test-fn :args (s/cat :t ::test) it fails instrumentation#2017-07-1005:38seancorfield@anticrisis regex-style specs combine unless you wrap them with s/spec as I recall.#2017-07-1005:39seancorfieldSo (s/fdef test-fn :args (s/cat :t (s/spec ::test)))#2017-07-1005:39anticrisisoh wow, ok#2017-07-1005:40anticrisisthere's something I have a great deal of trouble wrapping my head around with these regexp specs I think#2017-07-1005:42anticrisisthat does the trick, many thanks -- I will meditate on why, for a few hours.#2017-07-1005:43seancorfieldIf you do (s/exercise (s/cat :t ::test)) you'll see what's going on...#2017-07-1005:44seancorfield...it generates sequences and they conform to {:t {:k first-element :rest [other-elements]}}#2017-07-1005:47anticrisishmm#2017-07-1005:50seancorfieldPerhaps easier to see with this example:
boot.user=> (s/def ::pair (s/cat :a keyword? :b pos-int?))
:boot.user/pair
boot.user=> (s/def ::four (s/cat :ab ::pair :cd ::pair))
:boot.user/four
boot.user=> (s/valid? ::four [:x 1 :y 2])
true
boot.user=> (s/conform ::four [:x 1 :y 2])
{:ab {:a :x, :b 1}, :cd {:a :y, :b 2}}
boot.user=>
#2017-07-1005:53anticrisisthat part makes sense, but I'm not connecting it to the use of (s/spec) to change the behavior of the s/* operator when defining a function spec#2017-07-1005:54anticrisisI'm scrutinizing the output of s/exercise with and without s/spec right now#2017-07-1005:58anticrisisTo me, the conformed output of (s/exercise (s/cat :t (s/spec ::test))) and (s/exercise (s/cat :t ::test)) look the same#2017-07-1006:01anticrisisThanks for tip about s/specwith regexp ops and s/exercise, they point me in the right direction, even if I don't completely understand it. Many thanks.#2017-07-1006:25anticrisisI still think there's something inconsistent going on, either because I'm misusing this simple s/* form, or because something's broken. Here's a short walkthrough to demonstrate:#2017-07-1006:25anticrisis
(s/def ::test (s/cat :k keyword? :rest (s/* pos-int?)))

(s/conform ::test [:foo 1 2 3 4])
;; => {:k :foo, :rest [1 2 3 4]}

(s/conform (s/spec ::test) [:foo 1 2 3 4])
;; => {:k :foo, :rest [1 2 3 4]}

(s/conform (s/cat :t ::test) [:foo 1 2 3 4])
;; => {:t {:k :foo, :rest [1 2 3 4]}}

(s/conform (s/cat :t (s/spec ::test)) [:foo 1 2 3 4])
;; => :clojure.spec.alpha/invalid

(defn test-fn [test] nil)
(s/fdef test-fn :args (s/cat :t ::test) :ret nil?)

(stest/instrument `test-fn)
(test-fn [:foo 1 2 3 4])
;; => exception

(s/fdef test-fn :args (s/cat :t (s/spec ::test)) :ret nil?)

(stest/instrument `test-fn)
(test-fn [:foo 1 2 3 4])
;; => nil

;; Note that in order to get test-fn to pass instrumentation,
;; its :args must use the same form which conform considers invalid
#2017-07-1006:45seancorfield
(s/conform (s/cat :t (s/spec ::test)) [[:foo 1 2 3 4]])
One element, of type ::test.
#2017-07-1006:45seancorfieldsimilarly your function accepts one argument, of type ::test#2017-07-1006:45seancorfieldIt's consistent @anticrisis#2017-07-1006:46seancorfieldThe :args in s/fdef is the sequence of arguments.#2017-07-1006:50seancorfieldFor your first s/fdef -- :args (s/cat :t ::test) -- your test-fn would have to be (defn test-fn [kw & nums] nil) -- first argument :k, a keyword, and the :rest of the arguments are numbers.#2017-07-1006:50seancorfielddoes that help @anticrisis ?#2017-07-1006:52anticrisisI see what you're saying... but I thought :args (s/cat :x ::whatever) meant "the arguments are a list of one element, labeled 'x', which can be conformed with spec '::whatever'#2017-07-1006:57anticrisisYour example of fdef'ing [kw & nums] is helpful to tell me I've been thinking about this all wrong. I would have fdef'd it like this: :args (s/cat :k keyword? :nums (s/* pos-int?)) -- which you're telling me is the same as :args ::test-- which it is, but those particular neurons in my brain apparently weren't connected.#2017-07-1007:04anticrisisFor the record, I'd like to point out that nesting data structures is confusing. Is there a version of clojure I can use that doesn't do that? Thanks. šŸ™‚#2017-07-1007:14seancorfieldIf you want actual sequences (and nested sequences) then use coll-of etc, not cat šŸ˜ø #2017-07-1007:39hkjelsHow can I stub a function? Typically Iā€™d like to require that on-click is a function, but it fails when I later exercise it since theres no generator for it#2017-07-1007:40hkjelsCreating a custom generator would be overkill I think#2017-07-1011:12gfredericks@hkjels would a custom generator of (gen/return (constantly nil)) be overkill?#2017-07-1011:41hkjels@gfredericks ohh, hehe. no#2017-07-1011:41hkjelsšŸ™‚#2017-07-1011:41hkjelsthanks!#2017-07-1013:13rickmoynihanHmmmā€¦ Iā€™ve just made an observation about specs which Iā€™m wondering if others could validate for meā€¦ Basically I have a layered architecture where we do something like this: (->> input transform-1 transform-2 transform-3) and Iā€™ve just realised that in layout terms it seems better to organise the input specs alongside the transforms, rather than the :ret specs. It also seems to more closely align with things like instrument checking :args rather than :rets Has anyone else noticed this?#2017-07-1013:55rickmoynihanWhat is the most idiomatic way to spec something to be a single value? Iā€™m guessing: (s/def ::single-value #{:foo})#2017-07-1013:58mpenetyep, you get free gen with that one#2017-07-1014:51rickmoynihanmpenet: yeah the free gen is a big motivator šŸ™‚#2017-07-1013:58mpenetin theory (constantly :foo) would work for validation but no gen#2017-07-1014:01gfredericks(constantly :foo) would work? That seems identical to any? to me#2017-07-1014:01gfredericksit's a predicate that always returns truthy#2017-07-1014:01mpenetright#2017-07-1014:01mpenetbad example#2017-07-1014:01gfredericks#(= % :foo) I guess?#2017-07-1014:01mpenetyes or #(identical? :foo %)#2017-07-1014:02gfredericksjust for keywords#2017-07-1014:02mpenetyup#2017-07-1014:02mpenetbut anyway, the set solutions is better in most case#2017-07-1014:03gfredericksI can imagine people stumbling on #{nil} relatively often#2017-07-1014:04Alex Miller (Clojure team)use nil? then#2017-07-1014:04mpenetwell I guess you want nil? in that case#2017-07-1014:04gfredericksyep#2017-07-1014:04gfredericksI guess that sort of error isn't likely to go unnoticed#2017-07-1017:41ghadihttps://clojurians.slack.com/archives/C1B1BB2Q3/p1499457189858308#2017-07-1017:41ghadiBumping that question -- I was looking for some prior art for custom printing mostly#2017-07-1018:41bbrinck@ghadi Iā€™m working on something now, but itā€™s still pre-alpha#2017-07-1018:41bbrinckhope to have something ready by thursday šŸ™‚#2017-07-1018:43bbrinckThe code is under change right now, but a preview of what Iā€™m trying to achieve is in the tests e.g. https://github.com/bhb/expound/blob/master/test/expound/spec_test.cljs#L120-L137#2017-07-1019:06ghadibbrinck: very cool#2017-07-1018:43bbrinckExcuse the lack of documentation or sane organization#2017-07-1018:46bbrinckHas anyone found a use case for specing a map with coll-of? I see that you can create a spec like (s/def :foo/map (s/coll-of int? :kind map? :into {})) but I canā€™t see why this would be useful since everything but the empty map would always fail.#2017-07-1018:48bbrinckI suppose you could maybe replace int? in that example with a spec for the key/value pairs, but in that case, map-of seems superior.#2017-07-1018:48Alex Miller (Clojure team)there are some cases where it is useful to spec a map as a sequence of entry tuples#2017-07-1018:49Alex Miller (Clojure team)ā€œhybridā€ maps (combination of s/keys and s/map-of styles) is one case#2017-07-1018:49Alex Miller (Clojure team)example here: http://blog.cognitect.com/blog/2017/1/3/spec-destructuring#2017-07-1020:09souenzzoalexmiller: When I look to the result of (s/conform ::binding-form it remember me about ast's ... may in the future, can spec/conform be used as analyzer?#2017-07-1020:18Alex Miller (Clojure team)If you use that in the general sense, then yes absolutely - itā€™s a great tool for turning s-expressions into a tree of maps describing a DSL syntax#2017-07-1018:54bbrinck@alexmiller Ah, I see. Youā€™re using a tuple as the ā€œinnerā€ spec. That makes sense. Thanks for the info!#2017-07-1018:54Alex Miller (Clojure team)thatā€™s actually how s/map-of and s/every-kv are implemented#2017-07-1018:57bbrinckInteresting. Your example was instructive. I can see how oring together several tuples would let me define different valid key/value pairs#2017-07-1018:58bbrinckBasically, it lets me create a clear relationship between the spec for certain keys and the specs for their values. But combine them all into one big map spec#2017-07-1018:59bbrinckā€œvalid key/value pairsā€ Sorry, I meant ā€œvalid key/value pairs of specsā€#2017-07-1020:03lfn3Is fspec meant to handle cases where the provided function throws? Itā€™s causing spec errors when instrumented.#2017-07-1020:05lfn3Iā€™ve got a relatively minimal repro:
(t/use-fixtures :once #(do (s.test/instrument)
                           (%1)
                           (s.test/unstrument)))

(defn takes-fn [f]
  (try (f true)
       (catch Error e
         false)))

(s/fdef takes-fn
  :args (s/cat :f (s/fspec :args (s/cat :bool boolean?)
                           :ret boolean?))
  :ret boolean?)

(deftest can-handle-throws
  (is (false? (takes-fn #(if %1
                           (throw (ex-info "scary exception!" {}))
                           true)))))
#2017-07-1020:06lfn3That yields: In: [0] val: (true) fails at: [:args :f] predicate: (apply fn), scary exception! when the test is run#2017-07-1020:24Alex Miller (Clojure team)when you have an fdef with an fspec arg, the instrumented var will check the fspec by generating from its args spec and invoking the function to validate it conforms to the spec#2017-07-1020:24Alex Miller (Clojure team)in this case it will generate boolean values and invoke f#2017-07-1020:25Alex Miller (Clojure team)specs cover non-exceptional use (as those are the exceptions :) so I think a more accurate fspec args spec in this case would use false?, not boolean?. it would then invoke f with only false and not trigger the exception#2017-07-1021:38lfn3Right I guess I just wasn't expecting the fspec'd arg to be invoked? Wouldn't it be better to use with-gen so the spec still captures the allowed range of values? #2017-07-1022:28Alex Miller (Clojure team)well the idea is that you said this function takes a function that responds a certain way - the only way to validate that is to invoke it#2017-07-1022:33lfn3Yeah agreed. I donā€™t think that was really a question, I was just kinda surprised. I would have assumed that the fn would be instrumented and validated when it was invoked rather than when it was passed into another function. Might be something worth mentioning in the docs, if it isnā€™t already?#2017-07-1022:45Alex Miller (Clojure team)This has been brought up several times but I haven't had a chance to discuss it with Rich#2017-07-1023:14lfn3Ok. Thanks for the clarification and explanation in any case, itā€™s much appreciated.#2017-07-1101:46danielcomptonI'm trying to use spec.test/check to check all my functions, but I'm getting a couldn't satisfy such-that predicate exception. Is there any way to figure out which spec wasn't able to be satisfied?#2017-07-1102:03seancorfieldI don't know of any way to debug that -- other than to s/exercise your specs as you write them...#2017-07-1102:04seancorfieldYou could call s/registry and go through all the keys in that (which are specs) I guess...#2017-07-1102:04seancorfield...I've just gotten kinda paranoid about exercising each spec in the REPL as I write it.#2017-07-1102:26bfabryI think there's a request open to include the spec in that message. iterating the registry sounds like a good option in the meantime#2017-07-1107:39danielcompton@bfabry do you know which issue it is? I took a look in JIRA but couldn't see anything#2017-07-1108:39hkjelsHow do I express, many strings and/or many vectors? And I donā€™t care which one it is, so I donā€™t need any branching#2017-07-1109:12rickmoynihan@danielcompton: Iā€™ve had that same issueā€¦. Supposedly the latest version of test.check exposes more information on that, infact I updated my test.check to the latest version which provides a bit more info, but still doesnā€™t tell you the spec that failed (as it knows nothing of spec).#2017-07-1109:15danielcomptonšŸ˜ž#2017-07-1109:31rickmoynihanfinding them is a bit of a PITA, lucky for you bisecting specs with exercise is logarithmic šŸ™‚#2017-07-1109:31rickmoynihanbut better to exercise them as you write them bottom up like seancorfield suggests#2017-07-1111:12gfredericks@danielcompton @rickmoynihan @bfabry what's needed is to find the spot[s] where clojure.spec calls gen/such-that and get it to pass that debugging info along; I have no idea how difficult that would be, might be super easy#2017-07-1111:12gfredericksthe test.check change is that you can now pass an option (`:ex-fn`) that lets you customize the exception#2017-07-1115:14rickmoynihanSome of my spec generators seem to grow to pretty big values under the default 1000 generations used by clojure.spec.test.alpha/check. I donā€™t know how big theyā€™re getting in numbers of items, but theyā€™re OOMing. Is there an easy way to make specs flatten out at a certain size, but keep iterating over random generations?#2017-07-1115:16rickmoynihanobvs I could add an extra constraint e.g. #(> 1000 (count %)) to maps etcā€¦ but it feels like itā€™d be better done in the map/sequence generators#2017-07-1115:23gfredericks@rickmoynihan under default configurations your tests should max out their size at 200 trials and cycle back to small at that point; there should be a way to restrict the size further than that#2017-07-1115:23gfredericksSee this option to quick-check that I expect spec provides a mechanism for passing through: https://github.com/clojure/test.check/blob/05a53600aab21576420984a54f3ff75714981bc3/src/main/clojure/clojure/test/check.cljc#L46#2017-07-1115:23rickmoynihanhmmmā€¦ not sure why Iā€™m seeing so much heap then#2017-07-1115:24gfredericksthe underlying problem here relates to collection sizing, which we have an open test.check issue for#2017-07-1115:24gfredericksbut the workaround is to explicitly limit sizing via the option I just pointed to#2017-07-1115:25rickmoynihanyeah you can pass those through via (check ,,, {:clojure.spec.test.check {:max-size 10}})#2017-07-1115:25gfredericksAll I was saying with 200 vs 1000 is that you should probably see the same issues with only 200 trials#2017-07-1115:25gfredericksI'd try 50 for starters#2017-07-1115:27rickmoynihan100 for num-tests seems to work but more than that GC seems to dominate the run time#2017-07-1115:28rickmoynihanwill try reducing max-size#2017-07-1115:29gfredericksnum-tests can be as high as you want as long as max-size is low enough#2017-07-1115:29gfredericksmax-size defaults to 200, and the sizes used throughout the run are (cycle (range max-size))#2017-07-1115:29rickmoynihancool#2017-07-1115:30gfredericksso num-tests can "fix" sizing issues only when num-tests < max-size#2017-07-1115:31rickmoynihanmakes sense#2017-07-1115:32rickmoynihancool. That max-size trick at 50 seems to work nicely.#2017-07-1115:33rickmoynihan@gfredericks: so does :max-size 50 mean maps have a maximum of 50 pairs, strings have a maximum of 50 chars, vectors have a maximum of 50 items etc?#2017-07-1115:33rickmoynihanor is it a weight/multiplier?#2017-07-1115:33gfredericksthat tends to be true, but there's no true definition of size; it's an abstract thing that could theoretically be interpreted differently by any given generator#2017-07-1115:34rickmoynihanok thatā€™s what I thought#2017-07-1115:34gfrederickse.g., you can easily defy that by generating vectors with a specified length of 75#2017-07-1115:34rickmoynihanšŸ™‚#2017-07-1115:34rickmoynihanbut is it true of the default generators?#2017-07-1115:35gfredericksyeah#2017-07-1115:35gfrederickspresumably the fix for all this will be that lower-down nested structures will be substantially smaller#2017-07-1115:35gfredericksbut that wouldn't contradict what you just said#2017-07-1115:39rickmoynihanlowering it to 50 seems to work nicelyā€¦ it also means I can run more generations in the same time which seems like the right trade off for my data.#2017-07-1116:47lwhortondoes anyone know a way to express ā€œa map of keys of spec A or B, where A key points to spec A0 and B key points to spec B0ā€?
(s/def :foo (s/map-of #{:A :B} ...maybe (s/or :a :A0 :B0)
something along those lines?
#2017-07-1116:53seancorfield@lwhorton Sounds like a multi-spec to me.#2017-07-1116:54lwhortonheh, back to that olā€™ zinger again#2017-07-1116:56seancorfieldTBH, I'm not sure what you're actually trying to do, based on what you said. Can you give a concrete example?#2017-07-1119:43lwhortonmulti-spec turned out to be the right thing again. thanks!#2017-07-1118:15bbrinck@lwhorton You might also want to use tuples inside a coll-of. Something like (untested):
(s/coll-of (s/or :a (s/tuple ::A ::A0) :b (s/tuple ::B ::B0)) :into {} :kind map?)
#2017-07-1201:04danielcomptonI just realised that if you're using a tools.namespace based workflow, the spec registry isn't cleared when you refresh. Is there a recommended way to "clean the registry" safely?#2017-07-1201:05danielcomptonI suspect that if I just reset! the registry I might be missing internal/clojure.core.specs specs?#2017-07-1201:05twashingQuestion. This will break with Clojure telling me that ā€œNo such var: eidā€#2017-07-1201:05twashing
(s/def ::eid (s/with-gen (s/and number?
                                pos?
                                #(instance? java.lang.Long %)
                                (complement nil?))
               gen/int))

(s/fdef foo/bar
        :args (s/cat :a any?
                     :b ::eid)
        :ret (s/coll-of ::some/things))
#2017-07-1201:06danielcomptonwhat is es/eid?#2017-07-1201:06twashing::eid is the first form#2017-07-1201:06twashingā€¦ sorry wrong namespace#2017-07-1201:07twashingfixed#2017-07-1201:07twashingWhy canā€™t I use ::eid in my function spec :args?#2017-07-1201:09danielcompton@twashing I can load this into a REPL fine:
(defn bar [])

(s/def ::eid (s/with-gen (s/and number?
                                pos?
                                #(instance? java.lang.Long %)
                                (complement nil?))
                         gen/int))

(s/fdef bar
        :args (s/cat :a any?
                     :b ::eid)
        )
how are you generating the error?
#2017-07-1201:12twashingItā€™s the function that generates generative tests, based on the function spec.#2017-07-1201:12twashing... one sec, lemme find it.#2017-07-1201:15twashingclojure.spec.test/check#2017-07-1203:12souenzzo::stuff is a map with a fixed key :data that will never change,.
(s/def :foo/data (s/keys :req [:relevant/key :another.relevant/key]))
(s/def ::stuff (s/keys :req-un [:foo/data]))
There is some how to "omit" :foo/data definition?
#2017-07-1210:10joost-diepenmaatwhatā€™s the recommended way to run spec checks during automated testing? weā€™re currently using lein test with clojure.test and it would be nice to use something like the solution here https://stackoverflow.com/questions/40697841/howto-include-clojure-specd-functions-in-a-test-suite#2017-07-1210:11joost-diepenmaatif I have to I can copy & paste and adapt that code but it would be nice to have a supported library#2017-07-1211:50gfredericks@joost-diepenmaat if you paste it into https://github.com/gfredericks/schpec and make a PR, I'll release it not sure if that counts as "supported" or not#2017-07-1214:23jpmonettashi everyone! I'm trying to understand how to match :clojure.spec.alpha/problems inside :clojure.spec.alpha/value in explain-data result independent of the specs that were applied and I'm having some trouble#2017-07-1214:27jpmonettasso the thing is that seeing a problem in [::k 1] you don't know if it means, it's the value of that key, or it's the element at pos 1 of a seq in the value of that key#2017-07-1214:27jpmonettasdoes it make sense?#2017-07-1214:40bronsasee https://dev.clojure.org/jira/browse/CLJ-2192#2017-07-1214:41bronsayou're right in your analysis, it's been discussed before and it's logged in that ticket#2017-07-1214:43jpmonettasgreat, thx a lot @bronsa#2017-07-1216:58urbankSo I'm wondering, is spec with s/def + s/valid? supposed to replace predicate functions? Otherwise I find it hard to decide what should be a spec and what should just be a function#2017-07-1217:04urbankOh, I suppose that isn't really a good question. More pertinent would be whether new predicates should be written as anon functions in s/def or should s/def refer to a named predicate function#2017-07-1310:10rickmoynihandependsā€¦ sometimes the predicate function is generic and reusable but the s/def can give it a more specific name/abstraction. e.g. (s/def ::user-id integer?) (s/def ::post-id integer?)#2017-07-1310:10rickmoynihanothertimes the predicate is too specific to be reusable, so just keep it anonymous.#2017-07-1310:11rickmoynihan(No idea if this is best-practice btwā€¦ but itā€™s what Iā€™ve been doing)#2017-07-1217:30assoc-inI am wondering how I would write a spec for a nested map. I have something like this {:1 {:user-id 1 :username "bob"} :2 {:user-id 2 :username "jim"}} Where I have a (s/def ::user (s/keys :req-un [::user-id ::username])). I know how to do this if I had a vector of users by using (s/coll-of ::user), but how would I approach this with having the map keys being the ::user-id spec and the values of each map being the ::user?#2017-07-1217:57bfabry@assoc-in (s/map-of keyword? ::user)#2017-07-1217:58assoc-in@bfabry Ah yes I didn't see that looks like just what I needed. Thanks!#2017-07-1217:58bfabrynp#2017-07-1300:14bbrinckSeveral people on this channel have asked about pretty-printing spec failures. Iā€™ve just released Expound, a library that formats clojure.spec errors https://github.com/bhb/expound#2017-07-1307:07richiardiandreabbrinck: great great great thanks!#2017-07-1300:15bbrinckIā€™ve got a lot more features planned, but hopefully this first release is useful to people on this channel#2017-07-1300:15bbrinck^- cc @ghadi#2017-07-1300:17bbrinckcc @samueldev (who also asked about this awhile back)#2017-07-1301:20samueldevAwesome @bbrinck I'm going to try this out tomorrow morning!#2017-07-1306:13naomarikHow can I constrain a multispec to a specific key? With the event example in spec guide Iā€™ve tried this: (gen/sample (s/gen :event/event {:event/type #(s/gen #{:event/search})})) which doesnā€™t work#2017-07-1306:28naomarikSolved it using gen/fmap to constrain the type, wondering if more succinct solution is available though.#2017-07-1310:13rickmoynihan@bbrinck: awesome!! Iā€™ve been wanting something like this for a long timeā€¦ Another thing that might be useful is integrating this into prone for rendering spec errors in stack traces over http (in dev): https://github.com/magnars/prone#2017-07-1312:56bbrinckGood idea!#2017-07-1401:26ghadiNice job @bbrinck https://clojurians.slack.com/archives/C1B1BB2Q3/p1499904883413529#2017-07-1401:34bfabryya looks really nice @bbrinck#2017-07-1403:21bbrinckThanks! Let me know if you run into any problems.#2017-07-1418:04adamfreyis there a way to generate an args seq from an fdef with an :args spec?#2017-07-1418:15bbrinck@adamfrey would this work?
(s/fdef ::foo :args (s/cat :i int? :s string?))
(s/exercise (:args (s/spec ::foo)))
#2017-07-1418:16bbrinckor
(require '[clojure.test.check.generators :as gen])
(gen/sample (s/gen (:args (s/spec ::foo))))
#2017-07-1418:26adamfreythat does work, thanks#2017-07-1520:16arohnerhereā€™s a fun crash-the-process bug:
(s/fdef clojure.core/hash-map :args (s/* (s/cat :k any? :v any?)) :ret map?)(clojure.spec.test.alpha/instrument) 
#2017-07-1520:25arohnerhrm,
(clojure.spec.test.alpha/check 'clojure.core/hash-map)
also throws OOME, on a -Xmx1024m JVM
#2017-07-1522:17joshjonesI can replicate the instrument stack overflow -- however, for the check, it's just creating huge maps. So, best to run check with some constraints IMO usually:
(stest/check `hash-map {:clojure.spec.test.check/opts {:num-tests 500 :max-size 30}})
#2017-07-1622:27lwhortonwhatā€™s the proper way to spec protocols? iā€™m assuming I can just spec some client fn instead of the protocol? but Iā€™m having a conceptual issue about how to have multiple implementations of the protocol with this approach:
;; protocols.cljs
(defprotocol IMyProto
  (-foo [this v]))

;; client.cljs
(defn foo [this v]
  (protocols/-foo this v))

;; impl-A.cljs
(defrecord AImpl []
  IMyProto
  (-foo [this v] ...))

(defn foo [this v] (-foo this v)) ;; this is the fn I would instrument for A types

;; impl-B.cljs
(defrecord BImpl []
  IMyProto
  (-foo [this v] ...))

(defn foo [this v] (-foo this v)) ;; this is the fn I would instrument for B types
in client.cljs I want to be able to call protocols/foo instead of prototols/-foo, because I can instrument (defn foo...). But this breaks the purpose of a protocol - how do I know which implementation to invoke from the client a-impl/foo or b-impl/foo? Whatā€™s the proper way to organize these guys so I can stest/check 'foo?
#2017-07-1622:37lwhortonI suppose I could turn it on its head so that client calls always invoke (protocols/foo this v), and each impl points to an internal function IMyProto (foo [this v] (-foo this v))? But this doesnā€™t seem friendly, particularly once the records start having fields and if any protocol fn needs to refer back to those fields.#2017-07-1622:46lwhorton*or perhaps I should re-apply s/fdef to the protocolā€™s foo when I actually want to run my checks? (s/fdef protocols/foo ::args (s/cat :type ::a-type), then re-def with ::b-type?#2017-07-1623:21seancorfieldProtocols are open for extension -- so you can't write a spec for all possible argument types. I think you could write a multi-spec for foo -- where the defmulti discriminator function returns the type of the first argument?#2017-07-1700:06danielcompton@lwhorton I don't think you can spec protocols, the recommendation I got at the time was to create a regular function in front of the protocol function and spec that#2017-07-1700:08danielcomptonhttps://clojurians-log.clojureverse.org/clojure-spec/2016-06-09.html#inst-2016-06-09T00:06:42.000541Z#2017-07-1700:18lwhortoncongratulations by the way. i hope you have a heck of a time with the kiddo. Iā€™ā€˜ll certainly miss The REPL.#2017-07-1702:55danielcomptonI won't be gone for too long šŸ™‚#2017-07-1700:08lwhortonyea .. thatā€™s what Iā€™m rolling with. I probably wasnā€™t clear, but in my question Iā€™m trying to address the issue with trying to test.check multiple ā€œregular functions in front of the protocol functionā€ for each possible argument type#2017-07-1700:09lwhortonas @seancorfield mentions, theyā€™re open for extension so you canā€™t cover every base. that being said, iā€™m only trying to cover my particular bases.#2017-07-1709:11rickmoynihanWorth remembering also that you can spec that the value your wrapping fn receives satisfies? the protocol. Youā€™ll need to write a custom generator to generate conforming values for each of the implementations you care about though. You donā€™t get that for free like you do with multi-spec.#2017-07-1709:15rickmoynihanThis said if your implementations had a common construction interfaceā€¦ e.g. they all took a string as a single argument to their constructor, you could easily write a dynamic generator by inspecting the protocol i.e. by using (keys (:impls foo.bar/Protocol))#2017-07-1709:15rickmoynihanthen mapping the implementation classes to constructors/generators#2017-07-1711:40samueldevdoes anybody have any specs around HTML5 filetypes?#2017-07-1711:40samueldevI have a fn I'd like to spec but am currently omitting spec'ing one particular argument, which I'm expecting to be an HTML5 File/Blob#2017-07-1711:53samueldev@bbrinck hey ben, do I have to do anything special to make expound work in CLJS?#2017-07-1711:59samueldevnvm I didn't realize the transition from cljs.spec -> cljs.spec.alpha#2017-07-1711:59samueldevhad happened#2017-07-1711:59samueldevall good now šŸ™‚#2017-07-1714:29bbrinck@samueldev Iā€™ll add that to the README#2017-07-1714:32bbrinckA user has also reported a issue on CLJS 1.9.671 (I had been testing on CLJS 1.9.542). Just FYI: https://github.com/bhb/expound/issues/3#2017-07-1715:08samueldev@bbrinck using the lib and converted all my spec explains to yours this morning; quite pleased šŸ™‚#2017-07-1715:09samueldevits aim seems to be to output developer-friendly error messages; have you considered the use case of going all the way and having it output non-developer-friendly error messages, that can be presented to an end-user? for example if I'm doing some simple validation on a POST route in my API to create an entity, and they omit a required key, an output-string provided along the lines of "Missing required key: _____"#2017-07-1715:09samueldevas it stands now, I'm using regular spec explains and iterating over the problems and generating these strings#2017-07-1715:10samueldev(with some hard-coded mappings from something like a str? predicate failing --> a string being generated along the lines of "Value provided for key ____ must be a string."#2017-07-1715:11samueldevthis is something I've considered lib-ifying for my own sake but could instead PR it into expound perhaps, if you see value in it#2017-07-1716:05bbrinckYes, in the future, I want to have good defaults for a number of contexts: test failures, REPL, end-user, etc#2017-07-1716:06bbrinckThat wonā€™t replace the current error message, but itā€™ll either be another namespace or perhaps some way to configure expound#2017-07-1716:06bbrinckBut more generally, I want to extract out a set of helper functions that work on clojure.spec.problems such that you can trivially build your own custom string representation without dealing with all the details of spec#2017-07-1716:09bbrinckThat way, if you donā€™t like the default ā€œuser-facingā€ error message, you can peek underneath and see how its built and build your own.#2017-07-1716:10bbrinck@samueldev If you want to create a GH issue with an example of a spec, a value, and the error string youā€™d like to see for a user-facing error, that would be very helpful!#2017-07-1715:17adamfreyfor a spec that can potentially generate huge values (like giant nested maps with long keywords), what is the most efficient way to get a small values (for playing around with at the repl)#2017-07-1715:21schmeefor some reason, my spec errors are printed without newline, which makes them completely unreadable:
:error-while-loading kleinheit.datomic.impl
#error {
 :cause "Call to clojure.core/refer-clojure did not conform to spec:\nIn: [2 1] val: :as fails at: [:args :exclude :op :quoted-spec :spec] predicate: #{:exclude}\nIn: [2 1] va......
#2017-07-1715:21schmeeany idea what might cause this?#2017-07-1716:15bbrinck@schmee It looks like youā€™re printing out the entire error record. Is it possible to just get the :cause and print that?#2017-07-1716:15schmeeI hope so šŸ˜„#2017-07-1716:15schmeeI donā€™t get why it prints the whole error record in the first place#2017-07-1716:16bbrinckSorry, what is ā€œitā€?#2017-07-1716:16schmeelein repl šŸ™‚#2017-07-1716:17bbrinckAh, I see. Can you perhaps write a wrapper function that calls the function in a try/catch?#2017-07-1716:21bbrinckI have not tested this, but you could#2017-07-1716:21bbrinck@schmee Or, after you see an error like that, could you just do something like (println (:cause (ex-data *e))) to print out the cause of the last error?#2017-07-1716:22schmeeI suppose so, but I would prefer if I could get the REPL to always do that for me#2017-07-1716:22bbrinck(just put that on the REPL after the error has been caused)#2017-07-1716:24bbrinck@schmee I havenā€™t tried this, but it looks like lein letā€™s you configure the error handler https://github.com/technomancy/leiningen/blob/master/sample.project.clj#L357-L370. Note that by only printing the cause, youā€™ll presumably be losing other contextual information about the error, which may or may not be useful.#2017-07-1716:25bbrinckIn any case, this isnā€™t really related to spec, so you may have more luck tracking down an answer in #leiningen . Good luck!#2017-07-1716:25schmeeahh, now thatā€™s what Iā€™m looking for, thanks! šŸ˜„#2017-07-1717:34schmeeis there a ā€œrecursiveā€ version of describe, that resolves non-fn specs?#2017-07-1717:39Alex Miller (Clojure team)not currently#2017-07-1717:49schmeewhatā€™s the easiest way to constrain a s/cat into a vector?#2017-07-1717:50schmee(s/and vector? (s/cat ...)) seem to fail to generate data with exercise#2017-07-1717:51schmeehereā€™s how Iā€™m doing it now, just curious if there is a better way:
(spec/def ::pred* (spec/cat :k any? :pred any?))
(spec/def ::pred (spec/with-gen (spec/and vector? ::pred*) #(gen/fmap vec (spec/gen ::pred*))))
#2017-07-1718:21bbrinck@schmee My understanding is that vector sequences is a known problem that isnā€™t yet solved: https://clojurians.slack.com/archives/C1B1BB2Q3/p14988354#2017-07-1718:23bbrinckOr, perhaps I should say ā€œregex specs that are also vectorsā€#2017-07-1718:43gfredericksI feel like in some platonic sense the regex specs fit seqs much more than vectors but that doesn't mean it wouldn't be terribly useful to have regexes for vectors#2017-07-1718:43gfredericksprobably mostly/entirely for macros?#2017-07-1718:53bbrinckIā€™ve also had use cases when using conform/unform on vectors#2017-07-1718:54bbrinckWe have cases where we like the succinctness of using vectors e.g. something like hiccup: [:p {} "hi"]. When we manipulate it, itā€™s handy to have named parts by using a regex spec, but then we can unform it back to the vector form#2017-07-1718:55bbrinckRight now we have to unform, then convert the list to a vector#2017-07-1718:57schmeeyeah, a catv would be nice#2017-07-1718:57schmeeis there a JIRA or something for this?#2017-07-1719:00bbrinckhttps://dev.clojure.org/jira/browse/CLJ-2021#2017-07-1719:00schmeecheers :+1:#2017-07-1719:40taylorare there/will there be specs for the return values of explain-data?#2017-07-1719:56schmeethereā€™s a JIRA for ā€œspecs-for-specā€: https://dev.clojure.org/jira/browse/CLJ-2112#2017-07-1721:54lwhorton@schmee I was doing exactly that yesterday with clojure.test.check.generators and the gen/vector api.#2017-07-1721:54lwhortonhttps://clojure.github.io/test.check/clojure.test.check.generators.html#var-vector#2017-07-1721:55lwhortonyou can define the spec with (s/with-gen (s/cat ...) and hand the with-gen something that always returns a vec#2017-07-1813:38stathissiderisis there any way to ā€œmergeā€ two map generators?#2017-07-1813:40mpenetI guess you can cheat your way into it with (s/gen (s/merge x y))#2017-07-1813:41stathissideriswell, I donā€™t have specs for the maps I need to merge (so itā€™s not strictly a spec question!)#2017-07-1813:42gfredericks(gen/let {m1 gen-m1, m2 gen-m2} (merge m1 m2))
#2017-07-1813:42stathissiderisoh thereā€™s gen/let!#2017-07-1813:42stathissideristhatā€™s great, thanks @gfredericks!#2017-07-1819:46plinshello everyone, how can i validate a month inside a string? is the regexp approach the best? it should be a string containing 2 digits ranging from ā€œ01ā€ to ā€œ12"#2017-07-1819:49gfredericksit's certainly easy as a regex #"0[1-9]|1[0-2]"#2017-07-1819:51plinsthx#2017-07-1822:37jcf
(into #{} (map #(format "%02d" %)) (range 1 13))
You could build a set of the strings as the count is small too. šŸ™‚
#2017-07-1822:39jcfThen with clojure.spec you can do something like (s/def ::month-number-str (into #{} (map #(format "%02d" %)) (range 1 13))). And/or just write it out like so:
(s/def month-number-str
  #{"01" "02" "03" ...}
#2017-07-1823:08creeseI'd like to generate a spec dynamically, where the name of the spec is returned from a function. Is there any prior art on this?#2017-07-1823:11creese
(defn register-spec
  [spec-k attr-ks]
  (s/def (keyword "foo" spec-k) (s/keys :req-un attr-ks)))
#2017-07-1823:11creeselike this#2017-07-1823:12jcf@creese I'm not sure it's a good idea or that it'd work as expected, but the spec registry is an atom you can swap! etc. #2017-07-1909:04ikitommi@schmee spec-tools has a spec visitor, which traverses specs recursively, one use case is the recursive describe. Has regression (& progression) tests for all core-specs: https://github.com/metosin/spec-tools/blob/master/test/cljc/spec_tools/visitor_all_test.cljc#2017-07-1909:10schmeecool, Iā€™ll try it out later today!#2017-07-1910:47danielnealI'm still new with spec. Is it considered unidiomatic to use conformers to coerce, as in the following. What would be the idiomatic approach
(s/def :conformers/int
  (s/conformer (fn [x]
                 (cond
                   (integer? x) x
                   (re-matches #"\d+" x) (edn/read-string x)
                   :else :cljs.spec.alpha/invalid))))

(s/def :data.subscription/quantity :conformers/int)
#2017-07-1911:12danielnealI've asked a stackoverflow question just so in case someone answers it won't get lost in the mists of slack history#2017-07-1911:12danielnealhttps://stackoverflow.com/questions/45188850/is-use-of-clojure-spec-for-coercion-idiomatic#2017-07-1911:39schmee@danieleneal AFAIK the jury is still out on that one#2017-07-1911:39schmeebut spec-tools mentioned above has a lot of stuff for that#2017-07-1911:44danielnealah cool - so it might be a case that the boundaries around idiomatic use will emerge with time#2017-07-1911:44danielnealmakes sense#2017-07-1912:03ikitommimy 2 cents go to separating conforming from specs. There is CLJ-2116 for it.#2017-07-1913:56jcfhttps://dev.clojure.org/jira/browse/CLJ-2116#2017-07-1914:32danielneal@ikitommi what do you do for the coercing bit#2017-07-2006:03ikitommi@danieleneal spec-tools but I'm biased as co-authoring the lib ;) haven't seen anything else for it, really. Cognitect people have said many times that runtime transformations are not is scope of spec :(#2017-07-1915:28tcouplandGot a bit of a spec design and validation question: We've got a few messages flying around and are working on adding spec's for them now (i know, wrong way round), the messages generally look like this:
{:type "msg1"
 :version "1.0.0"
 :payload {:msg1-specific-key "val"}}
The trouble is the payload key being reused across all the the different :type fields, making writing a spec for :payload very tricky as it lacks the context of the :type value. I'm thinking we should have gone for a flat structure:
{:type "msg1"
 :version "1.0.0"
 :msg1-specific-key "val"}
A multi spec would then have something to go on. Just wondering if there's anything I've missed that would let us write a decent :payload spec? Feels like the current format just doesn't work with spec very well at the moment.
#2017-07-1915:47jcf@tcoupland if you're using a multimethod with your multi-spec you can implemtent arbitrarily complex dispatch logic.#2017-07-1915:48jcfYou can get at what's in the :payload, and transform it, use or to pick from multiple options.#2017-07-1915:49jcfOf course, if your data looks really similar that might be tricky, but then how specific does your spec really need to be? šŸ™‚#2017-07-1915:53tcouplandšŸ™‚#2017-07-1915:55tcouplandyeah, i'm worried that the dispatch function would get a bit nuts, also it wldn't really solve the problem of attaching the type to a specific payload#2017-07-1915:58jcfTime for version "2.0.0" šŸ˜‰#2017-07-1916:00nwjsmithI was fiddling with some spec tooling last night, and there is something I'd like to express that is difficult in spec. I'd like to be able to express "this is a collection of As for some comparable type A".#2017-07-1916:03nwjsmithI don't know much about types, but I think in Haskell this can be expressed as Comparable a => List a. It would be useful in situations like spec-ing the args of clojure.core/sort. Especially since vectors are comparable#2017-07-1916:03nwjsmithThat way you could express "this is a collection of As or a collection of vectors of As for some comparable scalar A"#2017-07-1916:18andrewmcveighCan't you write:
(s/coll-of #(instance? Comparable %))
#2017-07-1916:18andrewmcveigh?#2017-07-1916:19andrewmcveighSeeing how spec is working on values, you don't really need type constraints#2017-07-1916:20nwjsmithYes, totally. It's on the data/test-generation side that this becomes a little awkward.#2017-07-1916:21andrewmcveighAh, OK. Then yes I see your point šŸ˜‰#2017-07-1916:21nwjsmithI'd like my cake and I'd like to eat it too šŸ™‚#2017-07-1916:25andrewmcveighI think that (s/coll-of (s/and any? #(instance? Comparable %))) is closest in meaning to Haskell's Ord a => List a#2017-07-1916:25andrewmcveighProbably will get super slow though#2017-07-2006:05jimmyhi guys, I'm in a situation that we have two kind of specs:
# "correct" spec, this one is used to do validation, instrument 
and # "test" spec, this one is mainly to use in gen test. Since when doing test, sometimes I want a function to handle all kind of input instead of the "correct" one generated by "correct" spec.
How do you guys manage these in terms of code organization ( "test" spec in test namespace, "correct" spec in the same ns of the function def ) or ... ? I would love to hear your idea on this.
#2017-07-2006:06seancorfield@nxqd I would have a single spec but override the generator.#2017-07-2006:07seancorfieldYou want the same "correct" spec in both production code and test code -- but it's reasonable to have a generator that produces simpler data. You do not want the other way around.#2017-07-2006:07seancorfieldPerhaps you're not explaining what you're really trying to do?#2017-07-2006:14seancorfieldIf you're explicitly doing validation inside the function, then the arguments it takes are a different spec, by definition, since the function takes a broader range of data -- it's expecting a broader range of data so that it can validate a small range of data and do something different and observable for the "invalid" data.#2017-07-2006:17seancorfieldi.e., (if (s/valid? ::some-spec input-data) (do-good-stuff input-data) (do-something-else input-data)) -- the function is anticipating input-data that does not conform to ::some-spec and it is expected, defined, testable behavior that the function does something else -- and the function accepts, as part of its core production spec input data that is beyond just ::some-spec. Does that make sense @nxqd ?#2017-07-2006:23jimmyyeah you are correct. The approach I'm doing atm is basically the same as you mentioned above. what I have in mind is to design it like this:
code-ns
(s/fdef ... :args (s/cat :args ::args))
(defn ... )

test-ns
(s/fdef ... :args (s/cat :args ::args)) ;; the only different of this ::args is the gen function as you said.
;; register this fdef instead of the one above and run tests.
;; since the real implementation will still use the "correct" spec, there is no problem regarding valid? in function.
Is there any better solution to swap different gen fn in different env, so we don't have to create another test ns.
#2017-07-2006:24seancorfieldNo, what I'm saying is the function itself has one spec that is the same for test and production. It may be a different spec from what it checks (validates) inside the function.#2017-07-2006:25seancorfieldA generator for a spec cannot generate data that fails to conform to the spec, so the generator may generate a subset of acceptable data, but not a superset.#2017-07-2006:26jimmyhmm, interesting. I got your point now#2017-07-2006:28seancorfieldConsider an API function that accepts arbitrary strings as arguments but then conforms the arguments to numeric values -- the function argument spec is basically string? and then it will have a separate spec for conforming numeric strings to numbers.#2017-07-2006:29seancorfieldSince you want the arguments to function to be mostly numeric, you need a custom generator that mostly generates numbers and calls str on them, but sometimes generates arbitrary strings (which will fail the other spec) and cause the function to do whatever error handling you expect.#2017-07-2006:30jimmyagree. this clears my mind. What is your naming convention for fdef args spec and separate spec?#2017-07-2006:30jimmy
(s/def :fdef/arg )
and (s/def :correct/arg ) ?
#2017-07-2006:31seancorfieldIt depends on the business domain. In this case, I'd probably have a :domain/spec for the conforming spec and a :api/spec for the function argument spec.#2017-07-2006:32jimmygreat ! Thanks a lot. Have a nice day sir šŸ˜„#2017-07-2006:32seancorfieldGlad to help!#2017-07-2017:39phreedCan I get some help with writing a spec? I want a vector of of alternating types, similar to a tuple spec but repeating where the types of the first and last are the same. The following is invalid but hopefully suggests what I am looking for. (s/def ::graph-path (s/tuple-ish ::node ::edge ::node ::edge ... ::node))#2017-07-2018:02phreedIn think I figured it out... (s/def ::graph-path (s/cat :h ::node :r (s/* (s/cat :e ::edge :n ::node)))#2017-07-2109:25ikitommi@danieleneal @alex - gave a shot at the CLJ-2116: https://github.com/clojure/spec.alpha/pull/1. Idea is to support a optional conforming callback for conform & explain, allowing specs to be separated from conforming. Would be used like this:
(deftest conforming-callback-test
  (let [string->int-conforming
        (fn [spec]
          (condp = spec
            int? (fn [_ x _]
                   (cond
                     (int? x) x
                     (string? x) (try
                                   (Long/parseLong x)
                                   (catch Exception _
                                     ::s/invalid))
                     :else ::s/invalid))
            :else nil))]

    (testing "no conforming callback"
      (is (= 1 (s/conform int? 1)))
      (is (= ::s/invalid (s/conform int? "1"))))

    (testing "with conforming callback"
      (is (= 1 (s/conform int? 1 string->int-conforming)))
      (is (= 1 (s/conform int? "1" string->int-conforming))))))
#2017-07-2110:50scaturrAre there any known issues with clojure.test.check and multi arity functions?#2017-07-2110:52scaturrfor some reason when I try to run (st/check make-action) my repl just hangs forever#2017-07-2110:53scaturrsame thing out of the repl too - lein test just hangs (running generative tests via clojure.test.check in my normal test suite)#2017-07-2110:54scaturrI did verify that I could generate data for all the (s/def) expressions individually - that seems to work#2017-07-2111:12gfredericks@scaturr did you try generating (s/cat :type ::type :error? ::error? :payload (s/? ::payload)) as well?#2017-07-2111:13scaturrI did not - I will try that now#2017-07-2111:14scaturryes that worked šŸ˜•#2017-07-2111:14gfredericksI'm trying this out too#2017-07-2111:15gfrederickscan't reproduce; check ran fine for me#2017-07-2111:15scaturrI changed ::payload back to map? for that example#2017-07-2111:15scaturrinstead of string? - though both produce the error#2017-07-2111:16scaturrwhat version of test.check are you running?#2017-07-2111:16scaturrthank you for your help by the way šŸ™‚#2017-07-2111:17gfredericksprobably 0.10.0-alpha2#2017-07-2111:17gfredericksyeah#2017-07-2111:17scaturrIā€™m on 0.9.0#2017-07-2111:17scaturrIā€™ll try updating#2017-07-2111:18scaturrthat fixes it#2017-07-2111:19scaturrthank you so much#2017-07-2111:20gfrederickswell I'd certainly like to know what the difference is, but I don't think it matters enough to look into šŸ˜•#2017-07-2111:20gfredericksbut if you figure it out, let me know šŸ™‚#2017-07-2111:20scaturrwill do!#2017-07-2113:45Alex Miller (Clojure team)@ikitommi noted. I'm not going to do anything with it until I get a chance to ask Rich about it#2017-07-2116:22wilkerluciohello people, for those looking at how to do coercion with specs, spec-coerce leverages your spec definitions to coerce value types šŸ™‚ https://github.com/wilkerlucio/spec-coerce#2017-07-2201:21danielcomptonLooks neat, I see there is "Coercion overrides map to specify contextual coercions" on the TODOs, will that let you specify different contexts like a JSON payload or form encoding, e.t.c.?#2017-07-2203:04wilkerlucio@danielcompton thanks for the interest šŸ™‚ for that I was thinking more like when you do generator overrides, like you do on specs, I think the main use case would be to set a custom parser for a different date/time format depending on the source#2017-07-2205:01thedavidmeisterhey, is there a way to and additional predicates onto existing keywords when used in spec/keys for a new def?#2017-07-2205:02thedavidmeistere.g. i have :db/id which can be any int? but when i have a :db/id in a (spec/def :item/new ...) i also want to ensure that it is the specific int -1#2017-07-2205:55Alex Miller (Clojure team)No, but you can s/and additional constraints over the s/keys#2017-07-2206:11thedavidmeister@alexmiller like this?#2017-07-2206:11thedavidmeister
(spec/def ::item
 (spec/and
  (spec/keys :req [:db/id])
  (comp #(spec/valid? ::id %) :db/id)))
#2017-07-2206:11thedavidmeisterthat's a little different to what i asked#2017-07-2206:11thedavidmeister
(spec/def ::id
 (spec/or
  ::id--new #{new-item-id}
  ::id--existing pos-int?))
#2017-07-2206:14thedavidmeisterbut seems to be what i want when i run exercise, thanks šŸ™‚#2017-07-2206:22Alex Miller (Clojure team)I would try to avoid the comp valid? ā€¦ and instead state it as an additional spec or predicate#2017-07-2206:23Alex Miller (Clojure team)you can also use s/merge instead of s/and to combine two map specs together#2017-07-2206:23Alex Miller (Clojure team)one general, and one more specific#2017-07-2206:24Alex Miller (Clojure team)s/keys already checks that the value for the key conforms to the spec, so the extra check in your ::item is duplicating that#2017-07-2206:28thedavidmeister@alexmiller ah, i'm pretty new to this, could you give me a simple example of that?#2017-07-2206:29thedavidmeister(spec/def :db/id int?) this is :db/id#2017-07-2206:29thedavidmeisterso that's any int#2017-07-2206:30thedavidmeisterbut the item id has to be either a positive int or new-item-id (which is -1)#2017-07-2206:30thedavidmeisterit can't be just any negative int#2017-07-2206:30Alex Miller (Clojure team)I think you mostly have it above already#2017-07-2206:30Alex Miller (Clojure team)your ::id looks good#2017-07-2206:31Alex Miller (Clojure team)and then just (spec/def ::item (spec/keys :req [::id])) is sufficient#2017-07-2206:31Alex Miller (Clojure team)spec/keys checks that the map has the required keys and that every registered key spec has a value that conforms to that spec#2017-07-2206:32thedavidmeisterexcept that when i do exercise it generates maps with ::id instead of :db/id#2017-07-2206:33thedavidmeisteri can't actually pass it ::id because the key :db/id is coming from upstream#2017-07-2206:45thedavidmeisterso, related question#2017-07-2206:45Alex Miller (Clojure team)sorry, it was unclear whether those were the same or different things#2017-07-2206:45thedavidmeisteroh well yeah#2017-07-2206:45thedavidmeisteri've got something related#2017-07-2206:45thedavidmeisteri have a :project/id#2017-07-2206:45thedavidmeisterand i want to reference it with :item/project#2017-07-2206:45thedavidmeisterthey have the same predicate so i thought i could do (spec/def :item/project :project/id)#2017-07-2206:45thedavidmeisterbut it doesn't like that#2017-07-2206:46Alex Miller (Clojure team)should work fine - whatā€™s not working?#2017-07-2206:46thedavidmeisterUnable to resolve spec: :project/id#2017-07-2206:47Alex Miller (Clojure team)could you give a larger example that produces that?#2017-07-2206:48thedavidmeisterhmm#2017-07-2206:48thedavidmeisterso#2017-07-2206:48thedavidmeister
(spec/def :project/id uuid?)
(spec/def :item/project :project/id)
#2017-07-2206:48thedavidmeisterworks#2017-07-2206:48thedavidmeisterbut when :project/id is in a different ns (that i required) it gives the error#2017-07-2206:48Alex Miller (Clojure team)can you give that example?#2017-07-2206:49Alex Miller (Clojure team)there is a known issue right now with ordering of something like that such that the aliased spec needs to be defined first#2017-07-2206:49Alex Miller (Clojure team)are you running into something like that?#2017-07-2206:50thedavidmeisterpotentially#2017-07-2206:50thedavidmeisteri'm stopping and starting my test runner now#2017-07-2206:50thedavidmeistermaybe it got confused with me changing things#2017-07-2206:50Alex Miller (Clojure team)https://dev.clojure.org/jira/browse/CLJ-2067#2017-07-2206:51thedavidmeisteryeah it seems fine now#2017-07-2206:51thedavidmeisterso i cannot reproduce#2017-07-2206:51thedavidmeisterweird#2017-07-2206:51Alex Miller (Clojure team)probably an ordering issue#2017-07-2206:51Alex Miller (Clojure team)Iā€™m taking off! Laterā€¦#2017-07-2206:54thedavidmeisterthanks for the help!#2017-07-2207:14ikitommiok @alexmiller, added the current changes as a patch too if that helps the conversation with Rich.#2017-07-2308:14thedavidmeisteris there something like spec/or that just takes a list of predicates rather than the k/v list?#2017-07-2308:14thedavidmeisteri don't totally understand why spec/or and spec/and are different in this way#2017-07-2308:29thedavidmeisteralso, what's the best way to use spec to see if something only has a certain set of keys?#2017-07-2308:32schmeespec/or takes a k/v list so that when you conform, you see which of the branches was taken#2017-07-2308:32schmeecheck out the ā€œComposing predicatesā€ here: https://clojure.org/guides/spec#2017-07-2308:33schmeethis is not needed for s/and, since there is no branching there (all predicates must be true)#2017-07-2308:35schmeechecking that something only contains a certain set of keys is sort of against specs pricinples, since key sets are supposed to be open#2017-07-2308:35schmeeof course you can do that anyway by using a custom predicate fn#2017-07-2309:35thedavidmeister@schmee i have a map that contains some keys that represent "machine data" like ids/references and some that represent "user data" which is what the user actually typed in#2017-07-2309:35thedavidmeisterwhen an item has only machine data left in it i want to be able to clean it out of my db#2017-07-2309:36thedavidmeisterbecause that item no longer represents anything useful to the user#2017-07-2309:36thedavidmeisterso i want a spec that can tell when only machine keys are left#2017-07-2309:36thedavidmeisterso it is open when adding, but closed when making a decision about whether to "clean up"#2017-07-2309:38schmeethat can be done with a predicate fn spec#2017-07-2309:39schmeebut for instance s/keys has no way to say ā€œthese keys are not allowedā€#2017-07-2309:40thedavidmeisterit's not that they're not allowed#2017-07-2309:40thedavidmeisteri mean they are#2017-07-2309:40thedavidmeisterbut that's not the right way to think about it in context#2017-07-2309:42thedavidmeisteri'm trying to detect something not restrict it#2017-07-2309:49thedavidmeisterit's like#2017-07-2309:49thedavidmeister(if (spec/valid? :item/item--machine-only) (do ...) (do ...))#2017-07-2309:50thedavidmeister@schmee the extra keys are totally allowed but i do want to treat items that have only machine keys a bit differently#2017-07-2311:14schmeeperhaps something like this would work?
(s/def ::machine-keys (s/keys :req [:a :b :c]]))
(s/def ::human-keys (s/keys :req [:d :e]]))
(s/def ::all-keys (s/merge ::machine-keys ::human-keys))
#2017-07-2311:14schmeethen you could validate thing separately when needed but still have a spec for the combination#2017-07-2312:21thedavidmeister@schmee the problem is that ::machine-keys is valid when there are human keys too#2017-07-2312:22thedavidmeisterand human keys is valid for machine keys#2017-07-2312:36schmeethen do something like (and (s/valid? ::machine-keys foo) (not (s/valid? ::human-keys foo))#2017-07-2400:15thedavidmeister@schmee but then :f would trigger#2017-07-2400:15thedavidmeisteri actually do want my keys to be open#2017-07-2400:16thedavidmeisteri don't want to have to stipulate every human key, and none of them are required, they're all optional, it's just important that there is at least one#2017-07-2310:52thedavidmeister@schmee also, when you're doing spec/or and using keywords it still seems odd to force k/v#2017-07-2310:53thedavidmeistere.g. (spec/or :foo/bar :foo/bar :a/b :a/b)#2017-07-2311:11schmeethe first keyword there is just a tag#2017-07-2311:11schmeee.g
dev=> (s/def ::test (s/or :foo keyword? :bar int?))
:dev/test
dev=> (s/conform ::test 1)
[:bar 1]
#2017-07-2311:12schmeespec makes you name all the branches so that you can use that in error messages etc.#2017-07-2312:17thedavidmeister@schmee but if the branch is just a keyword that is already registered it already has a tag, so it could use that for error messages#2017-07-2312:17thedavidmeisterhmmm how is it possible that#2017-07-2312:18thedavidmeister(valid? ::foo a) and (valid? ::bar a) are both true#2017-07-2312:18thedavidmeisterbut (valid? (spec/and ::foo ::bar) a) is false?#2017-07-2312:37schmeeit would be weird if there was a different syntax for registered keywords, better to have a uniform interface and accept the extra typing#2017-07-2312:38schmeeyou can make a macro that does what you want#2017-07-2313:10thedavidmeister@schmee mmk, any idea why spec/and can be false when each of the individual items are true?#2017-07-2313:11schmeewell, in your example you test that a is valid for ::foo and b is valid for ::bar#2017-07-2313:12schmeethen you test if ::foo and ::bar is true for b#2017-07-2313:12schmeeso youā€™re testing two different things#2017-07-2313:12schmeethe first doesnā€™t imply the second#2017-07-2313:22thedavidmeister@schmee typo, it's all supposed to be a#2017-07-2313:23thedavidmeister
boot.user=> (spec/valid? :item/item--exists {:item/project #uuid "4e2db038-f64d-4805-95cf-720e9e7f1419", :db/id 1, :item/title "85", :item/list-id {:db/id 10}})
true
boot.user=> (spec/valid? :item/item--user-input {:item/project #uuid "4e2db038-f64d-4805-95cf-720e9e7f1419", :db/id 1, :item/title "85", :item/list-id {:db/id 10}})
true
boot.user=> (spec/valid? (spec/and :item/item--user-input :item/item--exists) {:item/project #uuid "4e2db038-f64d-4805-95cf-720e9e7f1419", :db/id 1, :item/title "85", :item/list-id {:db/id 10}})
false
#2017-07-2313:25schmeewhat does item/item--exists and :item/item--user-input look like?#2017-07-2313:28thedavidmeister
(spec/def :item/item--exists
 (spec/and
  :item/item
  (comp #(spec/valid? :item/id--exists %) :db/id)))
#2017-07-2313:28thedavidmeister
(spec/def :item/item--user-input
 (spec/and
  :item/item
  #(not (all-keys-in? % base-keys))))
#2017-07-2313:28thedavidmeister
(defn all-keys-in?
 [coll allowed-keys]
 (= #{}
  (clojure.set/difference
   (set (keys coll))
   allowed-keys)))
#2017-07-2313:31thedavidmeisterbut does it matter?#2017-07-2313:31thedavidmeisterhow can (and true true) be false?#2017-07-2313:36schmeethat is a good question#2017-07-2313:46thedavidmeisterinterestingly, {:item/project #uuid "4e2db038-f64d-4805-95cf-720e9e7f1419", :db/id 1, :item/title "85", :item/list-id {:db/id 10}} was generated by exercise for the spec/and#2017-07-2313:48thedavidmeisterwait, that's a lie šŸ˜›#2017-07-2313:48thedavidmeisteri generated it with slightly different code#2017-07-2313:54thedavidmeisterhmmm, so if i do exercise for :item/item--exists or :item/item--user-input independently i can generate :item/list-id#2017-07-2313:54thedavidmeisterbut together i cannot#2017-07-2419:39devthcontinually bitten by not knowing if instrumentation is on during interactive dev (e.g. turn it on, then redefine a symbol - now it's off) and tests for a given symbol. i think it's on, then find out i had an invalid spec in a lib for weeks because the upstream lib turned on instrumentation and threw an exception. šŸ¤”#2017-07-2419:43caiomutable shared state is the root of all evil šŸ™‚#2017-07-2419:40devthi think i want a single jvm flag to turn it on globally#2017-07-2419:47wilkerlucio@devth a flag would be complicated, if you think about how instrumentation works, it actually overrides your var, much like a monkey patch in Ruby if you are familiar with, so it would have to re-trigger on every def situation. a simpler solution would be to find a way to handle it on your editor, make it automatically call s/instrument on file save or something#2017-07-2419:48devthfamiliar with how it works and monkey patching. that puts the onus of using this thing correctly on users, and the many tools/editors they use#2017-07-2419:50wilkerlucioyup#2017-07-2419:50devthso i'm not satisfied with that solution šŸ˜›#2017-07-2419:52devthi kinda like how schema had their own s/defn. that would provide the means to flip a global switch.#2017-07-2419:53devthyes i know i'm free to build my own wrapper / macro / lib thing#2017-07-2419:53devthbut first have to answer "should i build my own wrapper marcro lib thing?"#2017-07-2422:11wilkerluciothe main issue on having a global toggle would be that it requires a conditional test overhead, imagine if you have to had this overhead on each function call to check if the parameters should be validated or not#2017-07-2422:12wilkerluciothat can be quite expensive, the way instrumentation works right now adds no overhead at all when you are not using it#2017-07-2422:12wilkerluciocurrently can be annoying to use, but editors and other features on top of it can solve this issue, while having the global check maybe not have a way to solve from the user code base#2017-07-2423:41devthwith a macro it could be zero cost#2017-07-2419:57tetriscodesHello, Iā€™m trying to gen some JSON for the Netflix Eureka API and it has @ā€˜s in the keys#2017-07-2419:58tetriscodes
(spec/def :port-def/$ int-str-gen)
(def enabled (keyword (symbol "port-def" "enabled")))
(spec/def enabled boolean?)
(spec/def ::port-def (spec/keys :req-un [:port-def/$ :port-def/enabled]))
#2017-07-2420:04tetriscodesproduces :port {:$ "9", :enabled false},#2017-07-2420:05tetriscodesIā€™ve been trying
(spec/def :port-def/$ int-str-gen)
(def enabled (keyword (symbol "port-def" "@enabled")))
(spec/def enabled boolean?)
(spec/def ::port-def (spec/keys :req-un [:port-def/$ enabled]))
#2017-07-2420:06tetriscodesproduces :port {:$ "9"}#2017-07-2420:06moxaj@tetriscodes I think you should use a qualified keyword instead of a symbol for the enabled spec#2017-07-2420:08moxajoh, you can't, the reader throws#2017-07-2420:09tetriscodesI thought :port-def/@enabled was qualified#2017-07-2420:17moxaj@tetriscodes ugly workaround:
(def enabled (keyword (symbol "port-def" "@enabled")))
(defmacro foo []
  `(s/def ~enabled boolean?))
(foo)
#2017-07-2420:18moxajhopefully you don't have too many references to the enabled spec ^^#2017-07-2420:32tetriscodesstill no enabled key#2017-07-2420:32tetriscodesspec/keys takes keys that are registered in the catalog as a spec#2017-07-2420:34avihi all šŸ‘‹ anyone happen to know why clojure.spec.gen doesnā€™t include let, and if thereā€™s an idiomatic approach to doing what clojure.test.check.generators/let does?#2017-07-2420:36tetriscodesHow are you using it?#2017-07-2420:37avi
(def ^:private gen-datetime
  (tc-gen/let [year (gen/large-integer* {:min 2017 :max 2117}) ; we should be so lucky
               month (gen/large-integer* {:min 1 :max 12})
               day (gen/large-integer* {:min 1 :max 28}) ; damn you February
               hour (gen/large-integer* {:min 0 :max 23})
               minute (gen/large-integer* {:min 0 :max 59})
               second (gen/large-integer* {:min 0 :max 59})]
           (str (string/join "/" [year month day])
                " "
                (string/join ":" [hour minute second]))))
#2017-07-2420:38avihmmm maybe thereā€™s an altogether better approach hereā€¦ I should probably just generate a unix timestamp integer within a certain range, then use clj-time to convert it into the desired string formatā€¦#2017-07-2420:39aviIā€™m still curious about let though#2017-07-2420:42tetriscodesyou could use spec/with-gen to make your clause a regex and then have your generator create the date formatted as a string and then use clojure.spec.gen/sample to generate data.#2017-07-2420:44aviThanks! Not sure I 100% follow but Iā€™ll try to parse that (mentally) and eventually report back. Thank you!#2017-07-2420:44tetriscodesLooks like you are trying to generate data#2017-07-2420:45tetriscodesSo writing the generator can create some random data for you, or the library test.chuck has a string-from-regex function that you can use but it may be harder to get the ranges on your numbers.#2017-07-2420:45aviaha, string-from-regex sounds promising, I didnā€™t know about that!#2017-07-2420:46avithank you!#2017-07-2420:46tetriscodeshttps://github.com/gfredericks/test.chuck#2017-07-2420:46avioh ah I see#2017-07-2420:46avivery cool#2017-07-2420:46aviwill check it out#2017-07-2420:46avithank you!#2017-07-2420:47tetriscodeshttps://clojurians.slack.com/archives/C1B1BB2Q3/p1500927450713826 that still gives me errors because they vector needs to look up the keywords#2017-07-2420:47tetriscodesThat helps get the spec defined but the lookup of the specā€™s keyword return nil#2017-07-2420:48tetriscodesI had heard that in the future spec/keys may take a function, but I donā€™t see that in the latest alphas#2017-07-2420:51moxaj@tetriscodes well, anywhere you'd use that spec, you'll have to wrap that form in a macro to bypass the reader#2017-07-2420:51moxajhence my comment#2017-07-2420:51moxajbut i'm sure there are cleaner solutions#2017-07-2420:51tetriscodesBut how does the definition of the s/keys vector find that key?#2017-07-2420:52tetriscodesBecause I still have to do :port-def/@enabled#2017-07-2420:52tetriscodeswhich isnā€™t a valid keyword#2017-07-2420:52moxaj
(defmacro bar []
  `(spec/def ::port-def (spec/keys :req-un [:port-def/$ ~enabled])))

(bar)
#2017-07-2420:53tetriscodesoh#2017-07-2420:53tetriscodesthat worked#2017-07-2420:53moxajmaybe you could do some preprocessing with spec/and, afaik conformed values flow through#2017-07-2420:53tetriscodesI should use macros everywhere#2017-07-2420:54moxajdefinitely not šŸ™‚ this is last resort#2017-07-2420:54tetriscodesThanks. Its in one spot, so Iā€™ll keep it hidden.#2017-07-2420:54tetriscodesYes, iā€™ve been encouraged to avoid them, so Iā€™m not familiar.#2017-07-2420:55moxajwell, macros in general, but particularly in this case, since this is a 'hack'#2017-07-2423:41aviany chance anyone can see whatā€™s wrong with this?
(s/def ::stream-of-string-rows
  (s/with-gen
    #(instance? InputStream)
    (fn [] (gen/fmap (fn [v] (->> (map (partial string/join ",") v)
                                  (string/join "\n")
                                  .getBytes
                                  io/input-stream))
                     (gen/vector (s/gen ::string-row))))))
I keep getting errors like this when I run stest/check on the namespace this is in:
java.util.concurrent.ExecutionException: clojure.lang.ArityException: Wrong number of args (1) passed to: events/fn--4703
            clojure.lang.ArityException: Wrong number of args (1) passed to: events/fn--4703
Not sure whatā€™s happening, because when I define these specs in the REPL they work just fine with gen/generate ā€¦
#2017-07-2423:47aviah crap I figured it outā€¦ the spec should have been #(instance? % InputStream)#2017-07-2423:47avipicard-facepalm#2017-07-2423:48avior, rather (partial instance? InputStream)#2017-07-2423:48avipicard-facepalm picard-facepalm#2017-07-2502:21jpmonettas@bbrinck I remember yo were working on a lib for formatting clojure.spec errors, I've been also experimenting with the same but with GUIs#2017-07-2502:21jpmonettashttps://github.com/jpmonettas/inspectable#2017-07-2502:21jpmonettasthat's a link to it#2017-07-2502:22bbrinckLooks great!#2017-07-2512:50mishaHow do I spec various length tuples? For example datomic's datoms, which can be [e a v t op] [e a v t] [e a v] [e a]#2017-07-2512:50mishas/or multiple s/tuples (or s/cats)?#2017-07-2512:54mishabtw, what are more suitable use cases for s/cat as opposed to s/tuple?#2017-07-2512:58mishaI think s/cat is suitable when you need to destructure seq into map, for example for clojure.pprint/print-table#2017-07-2513:17schmee@misha use s/? for variable length tuples#2017-07-2513:18schmees/cat requires you to tag the elements which can be useful for conforming while s/tuple doesnā€™t#2017-07-2513:19misha@schmee s/? s/+ expect homogeneous elements as far as I can tell reading docs. I, on the other hand, need to spec a seq where particular elements are of specific types#2017-07-2513:19schmeenot sure what that means, can you give an example?#2017-07-2513:20misha
(defmacro +
  "Returns a regex op that matches one or more values matching
  pred. Produces a vector of matches"
  [pred-form]
#2017-07-2513:22mishait'd be ok to use it to spec, say, seq of keywords. but I need to specify, seq of [int keywords type-a type-b bool] or [int keywords type-a type-b] or [int keywords type-a]#2017-07-2513:22misha(basically each element can have its own spec)#2017-07-2513:23mishaI end up with or and tuple
(s/def :fsm/transition
  (s/or
    :6-tuple (s/tuple :fsm/from-state :fsm/to-state :fsm/trigger :fsm/guard :fsm/behavior :fsm/internal-transition?)
    :5-tuple (s/tuple :fsm/from-state :fsm/to-state :fsm/trigger :fsm/guard :fsm/behavior)
    :4-tuple (s/tuple :fsm/from-state :fsm/to-state :fsm/trigger :fsm/guard)
    :3-tuple (s/tuple :fsm/from-state :fsm/to-state :fsm/trigger)))
#2017-07-2513:24schmeesomething like (s/def ::datom (s/cat :e int? :a (s/? keyword?) :v (s/? map?) :t (s/? inst?) :added (s/? boolean?)))?#2017-07-2513:25mishayeah, I might replace tuple with cat if I will require seq-to-map destructuring.#2017-07-2513:26mishabut in your example you need to wrap it in the s/or and add more "arities", because it'll fail validation on shorter seqs with Insufficient input#2017-07-2513:26schmeenope:
dev=> (s/conform ::datom [1 :type])
{:a :type :e 1}
dev=> (s/conform ::datom [1 :type {:asdf 1}])
{:a :type :e 1 :v {:asdf 1}}
#2017-07-2513:27mishathat's odd. how did I get "Insufficient input" then?#2017-07-2513:27schmeecan you post your spec?#2017-07-2513:28mishaI wiped it out already opieop#2017-07-2513:28mishaah, lol, you wrapped each subspec in (s/?)#2017-07-2513:29mishayour spec probably will accept missing-element-specs too.#2017-07-2513:29mishalike [e a t]#2017-07-2513:29mishawhich are not the droids I am looking for#2017-07-2513:30schmeedang, you got me on that one#2017-07-2513:32mishathe only thing I don't like about or - those :6-tuple labels always feel like a hack. It's like I put effort into designing/naming specs, but those labels? just barf some random name out to make compiler stop complaining.#2017-07-2513:33misha(or rather about me using or and similar specs)#2017-07-2513:36schmeewell, itā€™s great for some things, like when writing parsers and you want to dispatch on the tags#2017-07-2513:36schmeewell, itā€™s great for some things, like when writing parsers and you want to dispatch on the tags#2017-07-2513:38hmaurercan you give an example of that please? I am interested#2017-07-2513:40mishawhen you conform, and then pass conformed value to any multimethod
#2017-07-2513:45schmeeI did a toy macro to create maps which has some special syntax for not including nil values#2017-07-2513:45schmeefirst I wrote regular clojure code to parse it, but then I thought, why not do it with spec?#2017-07-2513:45schmeeso the conformed input looks like this:
dev=> (spec/conform ::the-spec '(:a 1 :b 2 (maybe :c) nil [:d (pred-fn)] 123))
[{:k [:any :a] :v 1}
 {:k [:any :b] :v 2}
 {:k [:maybe {:k :c :maybe maybe}] :v nil}
 {:k [:pred {:k :d :pred (pred-fn)}] :v 123}]
#2017-07-2513:46schmeeso you can see the :any, :maybe and :pred tags in there which come from s/cat specs#2017-07-2513:47schmeewhich I then use to dispatch and parse to the appropriate form#2017-07-2513:48schmeeslack is acting up on me, excuse the double posts#2017-07-2513:51hmaurer@schmee itā€™s acting up on me too; probably a server issue#2017-07-2513:51hmaurerthanks for the explanation šŸ™‚#2017-07-2513:36schmeebut in your case I agree it doesnā€™t add much value#2017-07-2513:38mishaI just have not had enough experience with spec to be actually using those tags much "later" in the code, so I don't really have naming intuition developed yet.#2017-07-2514:23mishaare there any apparent downsides of reusing spec names as dispatch keys in s/or/`s/cat`?
(s/def :foo/bar
  (s/or
    ::spec-one ::spec-one
    ::spec-two ::spec-two))
(apart from verbose error messages because of all the long qualified dispatch kw namespaces)
#2017-07-2515:39mishaIs there a preferred way to coerce value during conforming? If it'd become or's dispatch value ā€“ it's fine. something like:
(s/def ::ft/internal?
  (s/or
    false #{false nil :fsm/external}
    true #{true :fsm/internal}))

(s/conform ::ft/internal? :fsm/internal)
;; => [true :fsm/internal]
#2017-07-2515:50Alex Miller (Clojure team)there is a currently undocumented s/nonconforming that you could wrap around the s/or for that#2017-07-2516:08mishathe exact problem I have in the s/or above is "Assert failed: spec/or expects k1 p1 k2 p2..., where ks are keywords", and I used booleans hoping to get this kind of coercion#2017-07-2516:29mishafound this! https://gist.github.com/Deraen/6c686358da62e8ed4b3d19c82f3e786a#2017-07-2516:33Alex Miller (Clojure team)oh sorry, I didnā€™t even read your spec! the ks need to be keywords. And sets of falsey values wonā€™t work (as they will return falsey values even when thereā€™s a match)#2017-07-2516:38Alex Miller (Clojure team)hard to suggest an exact rewrite without knowing your goals re conforming#2017-07-2516:38mishaI wrote a conformer for that:
(defn internal-transition? [t]
  (case t
    nil false
    false false
    true true
    :fsm/external false
    :fsm/internal true
    #?(:cljs :cljs.spec.alpha/invalid
       :clj  :clojure.spec.alpha/invalid)))

(s/def ::ft/internal? (s/conformer internal-transition?))

(s/conform ::ft/internal? :fsm/internal)   ;=> true
(s/conform ::ft/internal? :fsm/external)  ;=> false
(s/conform ::ft/internal? :foo/bar)        ;=> :clojure.spec.alpha/invalid
#2017-07-2516:39Alex Miller (Clojure team)bleh#2017-07-2516:39mishaopieop#2017-07-2516:39Alex Miller (Clojure team)you donā€™t need a conformer#2017-07-2516:40Alex Miller (Clojure team)the problem with conformers is that you are doing a lossy conversion that throws away the original value, and making that decision for all future users of your spec#2017-07-2516:41mishathis is true. I can return [true :fsm/internal] though, right? and write an unformer for such values#2017-07-2516:42Alex Miller (Clojure team)I would spec it as (s/def ::ft/internal? (s/or :b (s/nilable boolean?) :k #{:fsm/external :fsm/internal}))#2017-07-2516:42Alex Miller (Clojure team)the tagged result tells you how to coerce the value if needed#2017-07-2516:43mishaI just wanted to avoid dispatching on arbitrary keyword later, instead of doing something like:
(->> huge-map ... ::ft/internal? first)
;;or just
(->> huge-map ... ::ft/internal?)
#2017-07-2516:44Alex Miller (Clojure team)I would at the very least write the spec as I have it above, then write a second spec that applies a conformer to the result of the first if needed#2017-07-2516:44mishayeah, and so I'd need to
(->> huge-map ... ::ft/internal? coerce-internal)
#2017-07-2516:44Alex Miller (Clojure team)yeah#2017-07-2516:46mishajust going through the options I have atm. Baked in conformer is indeed "bleh" comparing to just using coerce-fn there. I just don't really know how many client call sites there will be, don't want to forget calling coercion somewhere.#2017-07-2517:27wilkerlucio@misha also you can check the spec-coerce project: https://github.com/wilkerlucio/spec-coerce#2017-07-2518:36mishathanks @wilkerlucio#2017-07-2520:26avišŸ‘‹ Hi all! Iā€™m new to spec and Iā€™ve got a fairly complex fdef thatā€™s taking a very long time to runā€¦ ~30 seconds. Are there any tips, articles, best practices, etc on debugging this sort of thing? Thanks!#2017-07-2520:27avi(I can try to reduce it down to a gist if anyoneā€™s interested in seeing my slow ugly code)#2017-07-2520:45english@aviflax a gist might be useful. also, whatā€™s taking a long time to run? is it stest/check?#2017-07-2520:47aviAh, yes, it is stest/check ā€” thanks!#2017-07-2520:47aviI will try to put together a gist that isolates the problem#2017-07-2601:29Alex Miller (Clojure team)@aviflax there are a couple known issues related to s/coll-of and the other collection specs. if you posted your args spec, I might be able to suggest something#2017-07-2613:59avi@alexmiller great, thank you! Iā€™m working on a gist now ā€” almost done, but having a snag with tools.deps.alpha. Where/how should I report this snag?#2017-07-2615:49Alex Miller (Clojure team)@aviflax The jira link for tools.deps is on the readme page #2017-07-2615:55aviGot it ā€” will report there. Thanks!#2017-07-2616:12avi@alexmiller this reproduces and demonstrates my problem and how Iā€™m currently working around it by setting :num-tests low: https://gist.github.com/aviflax/1a9ba7e73d45157bfc03f6b11c3b9b18#2017-07-2616:47Alex Miller (Clojure team)Two things: 1) there is a known issue when using s/coll-of with :kind but without :into (https://dev.clojure.org/jira/browse/CLJ-2103) - this should be fixed soon, but adding your own :into clause will work around it. This could easily be enough. 2) using s/coll-of with a :gen-max option can also help constrain the size of generated collections#2017-07-2616:48Alex Miller (Clojure team)actually, #2 may be a non-issue for you since youā€™re using :count#2017-07-2616:48Alex Miller (Clojure team)so Iā€™d recommend in line 20ish, adding :into []#2017-07-2618:29avi@alexmiller will do ā€” thanks so much!#2017-07-2619:08gfrederickswould a conj talk on test.check be useful? if so, what angle exactly? usage patterns? implementation details? something else? I haven't really considered this before because I figured that reid had already talked about it a few years back and it hadn't changed much since then, but maybe that's not true and/or not a good reason#2017-07-2619:30avi@alexmiller I added the :into but it doesnā€™t seem to have made a difference ĀÆ\(惄)/ĀÆ#2017-07-2619:35aviadding :gen-max doesnā€™t seem to make a difference either šŸ˜ž#2017-07-2623:01Alex Miller (Clojure team)Well on the upside, you can rule those out as the problem :)#2017-07-2700:11aviShould I open a JIRA ticket?#2017-07-2700:18aviIā€™ve shrunk down the example just a bit by removing the CSV parsingā€¦ itā€™s still slow šŸ˜“#2017-07-2700:18avihttps://gist.github.com/aviflax/1a9ba7e73d45157bfc03f6b11c3b9b18#2017-07-2620:00misha@gfredericks I'd listen about approaching to property based testing, something in between cookbook and best practices, to improve test-design intuition (what ever that means)#2017-07-2620:01gfredericks@misha cool; I think that's more or less what I meant by "usage patterns", maybe#2017-07-2620:01mishamaybe kappa#2017-07-2620:10wilkerlucio@gfredericks a fresher way to go about it might be using spec as reference, how to use and extend generators for specs, and how to write tests using those#2017-07-2620:11gfredericksyeah, intersecting with spec would probably be more relevant. potentially harder for me since I haven't used spec very much#2017-07-2620:12wilkerlucioyou can use that as a reason to get more into spec too šŸ™‚#2017-07-2620:12gfredericksright šŸ˜„#2017-07-2620:32danielcompton@gfredericks the hardest thing I've found with property based testing is figuring out useful properties that can be calculated, without reimplementing the original function#2017-07-2620:32gfredericksyeah I think that's common. that would fit better as a pure test.check patterns talk#2017-07-2622:19dadairis there a way to do nested/consecutive multi-specs? e.g., {:type :a}, then also expect :subtype :b, and now multi-spec based off :subtype?#2017-07-2623:00Alex Miller (Clojure team)You can have a multispec return a spec that is another multispec or you can dispatch on both at the same time using a more complicated dispatch function#2017-07-2623:02Alex Miller (Clojure team)@gfredericks a talk on building more complex generators would be very useful and relevant to both spec and test.check#2017-07-2623:05gfredericks@alexmiller so limiting the scope to generators, enabling a deeper exploration?#2017-07-2623:05Alex Miller (Clojure team)I think there is plenty to talk about#2017-07-2623:06Alex Miller (Clojure team)Property test patterns is good too but less relevant re spec and has somewhat been done#2017-07-2623:08gfredericksokay cool; that happens to also be probably the most enjoyable for me to put together#2017-07-2623:20dadair@alexmiller do you have an example of what a multispec returning a multispec would look like? I've tried that and didn't seem to be able to get it to work#2017-07-2713:22misha@alexmiller @gfredericks I'd listen about "highjacking" specs with generators, and generators strategies in general, e.g. for cases, where you need to define some root rules, and all downstream generators need to inherit it. Which is, probably, the same issue/situation, as "I wrote a lib, and need to either hardcode some resource to use it implicitly (bad, but easy), or need every function to accept it as an argument (good, but messy verbose)". Think taxi-driver's browser arg, component vs. mount, etc. One use case would be testing datomic/datascript, where on the one hand you have all the specs about db/`tx-report` structure, but on the other, when you test actual transaction, ā€“ an extra layer of testing the data flow is required, like "my temp ids appear in tx-report, and transacted data appears in db". Such "flow" specs can be described, and tested generatively, but it is somewhat messy, and is hard to keep de-complected from "structure" specs.#2017-07-2713:24gfredericks@misha is this a situation where you're doing a higher level test and want to model some external system? #2017-07-2713:27mishathe closest to real life I got ā€“ is datascript testing, which I described. It probably is "higher level testing" from some pov, but feels like essential for any lib a larger than few util functions#2017-07-2713:29mishaYou might perceive datascript as being external to the app code, yes. Not sure if I answered your question#2017-07-2713:31gfredericksIs "flow" vs "structure" partially about one call vs multiple calls?#2017-07-2713:33mishayes, though, those multiple calls could be a single fn.#2017-07-2713:33gfredericksRight; okay cool that makes sense#2017-07-2713:39mishaok, how'd you go about testing datscript/datomic's transact!? I'd have specs for db, tx-report. I'd see, if structure conforms the specs. But then I'd need to test that, all temp-ids are permanent, that mapped temp-ids - are the same and for same entities, as in submitted data. That extra data in new db - is in fact submitted data. I feel, like it can be done on "generators" level, and not in just custom test code, which does not leverage, say, spec. Because after I'm done testing datascript, I want to model it (lol now I understand your question opieop) to enable testing app code build on top of it, and I would not be able to do this with custom test code, which does not leverage generators#2017-07-2713:42gfredericks@misha so when you say "custom test code" you're thinking of example-based tests, not something like vanilla test.check properties?#2017-07-2713:42mishayeah, I think so.#2017-07-2713:44mishaI think the question can be narrowed down to "how to use system-wide seed data, and set of rules, which affect all(most) of the generators"#2017-07-2713:46mishaI don't have an intuition developed in this area much, so I might be talking gibberish here and there. That's why I'd listen to something structured on the topic opieop#2017-07-2713:48gfredericksyeah, it sounds like you need a higher level overview; which is somewhat at odds with alex's more focused talk idea, but it might be that going over certain generator patterns would illuminate other aspects as a side effect#2017-07-2713:59mishaanything helps, and generative testing is so huge, but "under-appreciated" from talks perspective. Everyone's just "here, you can generate strings, ints, and booleans. Boom! Off you go."#2017-07-2721:45misha@gfredericks check this out, interesting approach to closures, to solve what I was trying to describe earlier today: https://clojurians.slack.com/archives/C03S1KBA2/p1501187926443798#2017-07-2721:46mishamore on this in this thread https://clojurians.slack.com/archives/C03S1KBA2/p1501179951981871?thread_ts=1501070045.404030&amp;cid=C03S1KBA2#2017-07-2721:50misha~alternative to closures and dynamic vars#2017-07-2722:08gfredericks@misha this is related to test.check generators somehow? #2017-07-2722:30misha@gfredericks no, but this might be useful in a highjacking generators with seed data. Seemed relevant to what I wrote earlier today...#2017-07-2722:35gfredericks@misha I'm having trouble figuring out what you mean by hijacking a generator#2017-07-2722:43mishaI am having trouble describing it opieop#2017-07-2722:47mishaI had a problem with my spec usage, where I wanted to have a spec for, say datascript entity. This spec contains datascript id spec, which can specify either id-before-transaction (actual id or temp id), or id-after-transaction (actual id only). So I have this nested spec, at the bottom of which there is an id-spec. For some tests I wanted that id-spec to be id-before-transaction, and for the others ā€“ id-after-transaction.#2017-07-2722:49bfabryok I can't find a gif to demonstrate it. but imo that sounds like a sign that you have two different things that should have different names#2017-07-2722:52misha@bfabry that's sort of true, but I doubt, that redefining entire entity spec with only id spec change ā€“ is THE solution.#2017-07-2722:52seancorfieldWhat about s/merge?#2017-07-2722:53seancorfieldDefine a datascript-core spec and then have datascript, datascript-before-transaction, and datascript-after-transaction for the three cases where you 1) don't care, 2 & 3) do care.#2017-07-2722:53bfabry^#2017-07-2722:54seancorfield(or possibly you only need datascript and transacted-datascript since the ID in the first one is either anyway)#2017-07-2722:54seancorfieldGiven that you have two ID specs, presumably there are situations where you care which type of ID you have?#2017-07-2722:54seancorfield(if you don't care, you don't need two specs -- you're over-specifying!)#2017-07-2722:56misha@seancorfield in this example, you are might be 100% right. But I just want to illustrate a situation, where you might have a composed spec more than 2 levels deep, and you want to replace a leaf sub-spec or a generator for it.#2017-07-2722:57mishain that case I cared about pre/post transact ids, hence the "story"#2017-07-2722:58mishaall of it is to describe "what I'd like to hear about in test.check conj talk" opieop#2017-07-2817:26bfabry@alexmiller is there anywhere to look to find out which core functions are spec'd / will be spec'd?#2017-07-2817:26bfabryactually there's an obvious answer to that, just grep for fdef in master, doh#2017-07-2818:56jpmonettas@bfabry check out https://github.com/jpmonettas/inspectable/ , specially the browse-spec functionality#2017-07-2820:04nopromptdoes anyone know of existing code which produces fully resolved spec descriptions with respect to the namespace from whence they came?#2017-07-2820:05noprompte.g. (spec/describe some-spec) => (clojure.spec.alpha/coll-of foo.bar/bean-sprout?) not (`(coll-of bean-sprout?)`)#2017-07-2820:06nopromptiā€™m asking before i embark on the journey.#2017-07-2820:29jpmonettasif I understood correctly (spec/form some-spec) is what you are looking for#2017-07-2820:30jpmonettasalso check https://github.com/jpmonettas/pretty-spec if you want to pretty print it#2017-07-2820:39wilkerlucio@noprompt I did some stuff that does reflection on the specs, I would be interesting to know if you can figure that out. afaik, this can be tricky, because to correctly resolve that you need to know the namespace definitions for the place where the spec was defined (which can be different from the spec namespace itself), so I'm not sure if that is even possible, given that you might not be able to disambiguate the non-qualified names (or even the qualified, considering aliases can change). but anyway, please let me know if you figure it #2017-07-2820:42jpmonettas@noprompt @wilkerlucio but isn't (spec/form ...) just that?#2017-07-2820:47wilkerlucio@jpmonettas that returns the original form as data, but it doesn't give you the context about, so imagine that I have a spec like (s/def ::some-spec my-pred?), just by reading the form, how can you tell from where my-pred? comes from?#2017-07-2820:48wilkerluciomaybe by getting the var from the function reference, might be possible there, but then I have no idea on how to do that on CLJS#2017-07-2820:51jpmonettas@wilkerlucio sorry still don't get it, s/form returns everything qualified, I think the idea is you can serialize, deserialize specs#2017-07-2820:52wilkerlucio@jpmonettas ha, you are right, hahha, man, I was assuming it didn't because of what noprompt said, but I did the test now and you are right, it returns with the full name šŸ˜…#2017-07-2820:54noprompt@jpmonettas @wilkerlucio thank you!#2017-07-2917:15stathissiderisdoes stest/check expect the :gen option to contain a map of spec-names to generator-returning functions?#2017-07-2923:49stathissiderisfor the record, the answer is yes#2017-07-3000:13camdezApologies because Iā€™m sure this gets asked all the time butā€¦if I have nested data which reuses keys, like thisā€¦
{:id 214566,
 :name "Fake Co",
 :phone_number {:id 141683, :phone_number "555 123 4567"}}
ā€¦if I donā€™t want to do something gross like a semantic-defying s/or, is my only option to make have two phone_number specs in separate namespaces?
#2017-07-3000:30camdezAnd, assuming that is right, any advice on where those additional namespaces live? Do you actually go with a proliferation of tiny files? Or create the namespaces without creating files? And then do you alias those namespaces or just go with something like this?:
(ns foo.specs
  (:require [clojure.spec :as s]))

(s/def ::id integer?)
(s/def ::name string?)
(s/def :foo.specs.phone_number/phone_number string?)
(s/def ::phone_number (s/keys :req-un [::id :foo.specs.phone_number/phone_number]))
(s/def ::business (s/keys :req-un [::id ::name ::phone_number]))
#2017-07-3001:13bfabry@camdez you don't need to actually create namespaces to use them, you can just
(alias 'foo.customers 'customers) 
(alias 'foo.customers.phone_numbers 'phone_numbers)
and then do
(s/def ::phone_numbers/phone_number string?)
(s/def ::customers/phone_number (s/keys :req-un [::phone_numbers/phone_number))
#2017-07-3001:16bfabrystandard disclaimer that if you can figure a way to make your app use namespaced keywords using :req and :opt rather than :req-un and :opt-un you'll probably have a better time in the long run#2017-07-3002:22camdezThanks, @bfabry. A couple small notes for posterity after experimenting a bitā€¦ 1. The argument order for alias is actually the other way: (alias 'phone_numbers 'foo.customers.phone_numbers) 2. The namespace actually needs to exist before you can alias it (one could (create-ns 'foo.customers.phone_numbers)), even though the namespaced keywords used for the spec registry donā€™t necessarily need to exist. I appreciate the disclaimer but Iā€™m trying to wrap a spec around an API response I donā€™t control, so it is what it is.#2017-07-3002:24camdezThis all makes me wonder even more how people are actually doing things. Does your spec registry tend to match actual namespaces? Are those in the same namespaces as your code? Or a parallel set of namespaces? Inquiring minds want to know. šŸ˜›#2017-07-3002:27camdezIMHO Spec seems broad enough in its scope to require not just how can I but how should I documentation.#2017-07-3004:35seancorfield@camdez We have a combination of keywords where the namespace matches a code namespace and others where they just use a unique single-segment prefix.#2017-07-3004:42camdez@seancorfield Thanks! Whatā€™s the deciding factor between the former and the latter?#2017-07-3004:42seancorfieldI think the recommended approach for alias currently, if you want to introduce a new ns is (alias 'alias (create-ns 'the.full.name)) FYI.#2017-07-3004:43seancorfield@camdez It Depends(tm) šŸ™‚#2017-07-3004:46seancorfieldFor specs that are internal to a ns, the obvious choice is just to use ::name and it'll be in that ns. For specs that are intended to be used across nses, then you need a "unique enough" name for the intended usage. So if that's a subsystem of your own application, you can just use a simple qualifier. For example, we use wsbilling for World Singles Billing. You could use a prefix that matched a DB table, for example, or a component in your app.#2017-07-3004:47camdez@seancorfield Thanks. Good explanation.#2017-07-3004:47seancorfieldFor specs that are intended to be more global, then the prefix also needs to be more global.#2017-07-3004:52seancorfieldI think it's a bit early yet with clojure.spec for "best practices" to have truly settled down.#2017-07-3004:57seancorfieldWe've been using it heavily in production for quite a while. We've defined the data model for a few systems with it and we use those for both test generation of data and validation of some inputs. We also have a series of coercing specs defined around one of our REST APIs: we s/conform the raw parameters and get either ::s/invalid or the valid parameters conformed to longs, doubles, strings, keywords etc.#2017-07-3005:17camdez@seancorfield I definitely agree RE best practices but it feels like such a buffet that we definitely need to be having the conversations about what might become best practices.#2017-07-3005:18camdezDespite some valiant documentation efforts (largely around whatā€™s possible) I think itā€™s still hard for a spec beginner to know where to start.#2017-07-3006:27seancorfieldYes, definitely hard. I actually submitted a talk for one of the Clojure conferences about our use of spec, but after talking to Alex Miller, I pulled it because I wasn't sure we were doing things a good way (I've since changed my mind again šŸ™‚ )#2017-07-3006:28seancorfield(to be fair, I had a few reasons for pulling the talk but that was the main one)#2017-07-3006:29seancorfieldI'm looking forward to hearing other people's spec talks at future Clojure conferences!#2017-07-3008:19matanHi all, are we approaching a release of clojure 1.9 with the finalized spec in it? I still see in the docs a mention of :clojure.spec.alpha/invalid, will I have to change my code to use a differently named value later on or is it just out-of-date documentation? https://clojure.org/guides/spec#2017-07-3008:19matanThanks!#2017-07-3015:10camdez@matan Iā€™m not speaking with any authority here, but I donā€™t believe the docs are out of date (see latest here: https://github.com/clojure/spec.alpha). I believe you will have to use a different named value later, but if youā€™re requiring spec with an alias (e.g. (:require [clojure.spec.alpha :as s])) and using that alias to refer to the keyword (e.g. :s/invalid) then youā€™ll only need to change the require statement later.#2017-07-3015:12camdezIā€™m assuming that code living under an alpha namespace is not subject to the standard guidelines about accretion and breakage (indeed, thatā€™s likely the exact meaning of the alpha namespace), and thus clojure.spec.alpha will become clojure.spec in a non-backwards compatible way.#2017-07-3018:11stathissideristhis is how to generate params for a function based on its spec:
(gen/generate (s/gen (:args (s/get-spec (resolve `my-fn)))))
#2017-07-3018:11stathissideris(1 sample)#2017-07-3101:30camdez@seancorfield Aha! Thank you!#2017-07-3105:51Oliver GeorgeI'd like to check some javascript data based on a regular spec which uses s/keys and s/coll-of.#2017-07-3105:52Oliver GeorgeI know it's not in scope for clojure-spec but perhaps someone has written something which does this?#2017-07-3105:52Oliver George
(s/assert-obj ::typical-spec #js {:a [1 2 3]})
#2017-07-3106:31ikitommi@camdez if you are dealing with deeply nested legacy models and only want to validate those in the api layer - after which you only transform them to conform your own application specs - you could check out data-specs (in spec-tools lib). Builds on top of clojure.spec. Itā€™s *not* a good/best practise, but food for thought anyway. Something like:
(require '[clojure.spec.alpha :as s])
(require '[spec-tools.data-spec :as ds])

(s/def ::business
  (ds/spec
    ::business
    {:id integer?
     :name string?
     :phone_number {:id integer?
                    :phone_number string?}}))

(s/valid?
  ::business
  {:id 214566,
   :name "Fake Co",
   :phone_number {:id 141683, :phone_number "555 123 4567"}})
; true
#2017-07-3106:32ikitommia sample bare-bones http api with it: https://github.com/ikitommi/business, added a :tags (a set of keywords) to demonstrate the automatic coercion (string->keyword & vector->set here)#2017-07-3112:26camdezThanks! Iā€™ve used Plumatic (nĆ©e Prismatic) Schema quite a bit, so this schema-resembles-data approach is quite familiar. But Iā€™ve been using keyword prefixes divorced from the extant namespaces and itā€™s working pretty well for me, if a tad verbose.#2017-07-3116:20bfabry@olivergeorge simplest thing that comes to mind is to just js->clj and run spec over the result#2017-07-3117:50misha@bfabry @olivergeorge js->clj and clj->js are not symmetric, beware#2017-07-3119:19plinshello everyone, im trying to spec a map where all keys are optional but you should have at least one of them, is this achievable?#2017-07-3119:20bfabry@plins sure, (s/and (s/keys :opt [...]) not-empty)#2017-07-3119:21plinsi wasnt aware i could use not empty as a spec, thx!!!!#2017-07-3119:23bfabryyou can use any predicate function as a spec šŸ™‚ though not-empty isn't technically a predicate function but it's good enough#2017-07-3119:24plinsso spec treats nil as false and truthy values as true ?#2017-07-3119:25bfabrywell, clojure treats nil and false as false and everything else as true. so yeah spec does too#2017-07-3119:25bfabry
cljs.user=> (s/valid? (s/and (s/keys :opt [::foo]) not-empty) {::foo nil})
true
cljs.user=> (s/valid? (s/and (s/keys :opt [::foo]) not-empty) {})
false
cljs.user=> (s/explain (s/and (s/keys :opt [::foo]) not-empty) {})
val: {} fails predicate: not-empty
:cljs.spec.alpha/spec  #object[cljs.spec.alpha.t_cljs$spec$alpha16935]
:cljs.spec.alpha/value  {}
nil
#2017-07-3119:56mfikesIt doesnā€™t appear to be possible to use an s/cat spec for an argument. Does this instrument failure make sense?
user=> (require '[clojure.spec.alpha :as s] '[clojure.spec.test.alpha :as st])
nil
user=> (s/def ::ingredient (s/cat :quantity number? :unit keyword?))
:user/ingredient
user=> (s/valid? ::ingredient [0 :teaspoon])
true
user=> (defn none? [ingredient] (zero? (first ingredient)))
#'user/none?
user=> (s/fdef none? :args (s/cat :ingredient ::ingredient))
user/none?
user=> (st/instrument)
[user/none?]
user=> (none? [0 :teaspoon])

ExceptionInfo Call to #'user/none? did not conform to spec:
In: [0] val: [0 :teaspoon] fails spec: :user/ingredient at: [:args :ingredient :quantity] predicate: number?
:clojure.spec.alpha/spec  #object[clojure.spec.alpha$regex_spec_impl$reify__1200 0x379e0a93 "
#2017-08-0101:12athosYou should wrap ::ingredient with s/spec, like (s/fdef none? :args (s/cat :ingredient (s/spec ::ingredient)))#2017-07-3119:58plins@bfabry, i think i found a problem
(s/def ::my-spec (s/and not-empty (s/keys :opt-un [::a ::b])))
(s/valid? ::my-spec {} => false
(s/valid? ::my-spec {:A 1 :c 1}) => true
#2017-07-3119:59bfabry@plins well, that's not so much a problem you just didn't fully explain your requirements to me šŸ˜›#2017-07-3119:59plinsim sorry for that#2017-07-3120:00plinslet my try express myself better#2017-07-3120:00bfabryno it's ok I get it now#2017-07-3120:00plinsšŸ™‚#2017-07-3120:01bfabryyou probably want (s/def ::my-spec (s/and (some-fn :a :b) (s/keys :opt-un [::a ::b])))#2017-07-3120:02bfabrythere's various ways to get around that duplication, fwiw#2017-07-3120:04plinsiā€™ll dig deeper into the docs, thx šŸ™‚#2017-07-3120:09plinsive settled with
(s/def ::my-spec (s/and (s/map-of #{:a :b} any?)
                        (s/keys :opt-un [::a ::b])))
thx!
#2017-08-0104:55athos@plins I think (s/keys :req-un [(or ::a ::b)]) would be a smarter solution#2017-08-0110:39misha@athos is this even legal?#2017-08-0110:41mishait is!#2017-08-0110:43athos@misha Of course! s/keys docstring refers to this feature.#2017-08-0110:44mishanot sure how I missed it then. anyway: wow, nice#2017-08-0111:33mishais "mutual exclusiveness" and/or "orthogonality" of a set of the predicates is even approachable, let alone provable automatically (e.g. during testing with spec)? Does it imply complete exhaustion of the possible input combinations?#2017-08-0111:42mishaContext is: In Harel's charts (or UML state diagrams), state can have multiple transitions to mutually exclusive states, via the same trigger. What ensures that system ends up in a valid state, is a combination of guards (predicates) on those transitions, which should guarantee mutual exclusiveness of the guard outcome. I want to try to automatically test whether guards on a set of transitions are mutually exclusive. e.g. :
(def FSM
  {:A {:B {:trigger :a :guard pos-int?}
       :C {:trigger :a :guard neg-int?}}})
Current state is :A. Some event triggers transition :a. Machine has to switch to either :B or :C, but not both. I want to make sure guards (`pos-int?`, and neg-int? in this case) are mutually exclusive. Automatically. At compile time.
#2017-08-0111:44mishaIs it even possible? Or the best I can do is to just issue a "dude, make sure [:A :B :guard] and [:A :C :guard] are mutually-exclusive" warning?#2017-08-0111:44gfredericksthere's no general solution to that, but you could write a test.check property that asserts it#2017-08-0111:44gfredericksif you have a meaningful generator for the state#2017-08-0111:44gfredericksor whatever it is that's getting passed to the guards#2017-08-0111:45misha@gfredericks the complication with this, is the huge arity of the guards: they need to accept app-db, whole-machine, and event trigger with its args harold#2017-08-0111:46mishawhere for machine I may supply generator, but app-db and trigger args structure, contents, and specs are entirely up to user#2017-08-0111:48gfrederickssounds tough šŸ™‚ you can validate at compile time of course#2017-08-0111:48mishaand for even as simple as a single string arg, exhausting 2 arbitrary predicates (e.g. with regexps inside) is doubtful, I think(?)#2017-08-0111:49gfredericksbut for a turing-complete language you can't in general check if two predicates are mutually exclusive#2017-08-0111:49gfredericksregexes might be doable, depending on the details#2017-08-0111:49gfredericksnot worth it though#2017-08-0111:49gfredericksjust a theoretical curiosity#2017-08-0111:49mishaI feel... relief šŸ˜„#2017-08-0111:53mishawhat if the predicate's source code is available? Is there something to, say, reverse-engineer things like (select-keys app-db [:foo :bar]) to reduce relevant input scope? Or at least to deduce, that guards, in fact, are orthogonal (which is useful too, but I forgot why just now)?#2017-08-0111:54gfredericksdefinitely for special cases#2017-08-0111:54gfredericksbut the general case impossibility I mentioned earlier applies to source or compiled or whatever else#2017-08-0111:55mishait'd be useful to cover at least some basic cases, because most of (UI) FSMs would actually use pretty basic guards#2017-08-0111:59misha(There are at least 2 implementation options: 1) pseudo states, where you define a fork in the transition as a separate state, and just ask to for cond/case guard there. 2) duplicate transitions with separate boolean guards, which (implicitly) need to be mutually-exclusive. I am exploring (2) at the moment, to make literal FSM definition less verbose.)#2017-08-0112:32mfikesThe solution to my s/cat in s/cat question yesterday was, of course, s/spec:
(s/fdef none? :args (s/cat :ingredient (s/spec ::ingredient)))
#2017-08-0115:19jpmonettasif someone finds it useful, just released a version of https://github.com/jpmonettas/inspectable with clojure and clojuresript support, also with an enhanced browser that shows the spec together with a sample#2017-08-0211:40NiclasCould someone help me understand why for the given example fn
(defn my-fn [selector]
  (condp = selector
    1 :b
    2 :c
    false))
it fails on the :ret spec when calling the fn with the following spec
(s/fdef my-fn
        :args (s/cat :selector any?)
        :ret #{false :b :c})
but not for the following spec
(s/fdef my-fn
        :args (s/cat :selector any?)
        :ret (s/or :a #(= % false)
                   :b #(= % :b)
                   :c #(= % :c)))
#2017-08-0211:48mpenetprolly comes down to this: (s/valid? #{false šŸ˜› :c} false) -> false or (#{false} false) -> false#2017-08-0211:49mpenetsince it will return the value matching in the set (false) this will break#2017-08-0211:52mpenettypical quirk with Sets#2017-08-0211:52scaturrIs there some established/idiomatic way to use spec with core.async? Particularly when reading from a channel?#2017-08-0211:52scaturrOr would it come down to adding a spec to a function that pushes on to the channel you are reading from?#2017-08-0211:53scaturrProbably way too broad of a questionā€¦ lol#2017-08-0211:56mpenet@looveh so in short, use contains? (and specify a custom :gen if you need to)#2017-08-0211:56Niclas@mpenet Ah, thatā€™s quirky indeed, thanks!#2017-08-0212:48schmeeunless you have a transient collection, for which contains? doesnā€™t work#2017-08-0213:39matan@camdez thanks for last week's answer about clojure.spec.alpha/invalid in the context of spec's code stability šŸ™‚#2017-08-0213:39camdez@matan Youā€™re very welcome. I hope it was helpful.#2017-08-0213:46matan@camdez yes, it certainly was!#2017-08-0213:47matan@camdez will I bump into a lot of stuff coming from its "alpha" namespace when using spec today?#2017-08-0213:47camdez@matan I donā€™t think youā€™ll run into much.#2017-08-0215:04cddrAm I correct in thinking that spec would be the wrong tool to assert relationships between keys in a map. For example, lets say we're representing a bill split between n folks, can/should you leverage spec to assert that the sum of amounts under the payees key is greater than or equal to the top-level amount? If not spec, is there some other library that allows you to organize validations like these in a way that lends support to generating good error messages
{:amount 60
 :payees [{:name :andy
           :amount 20}
          {:name :rich
           :amount 20}
          {:name :alex
           :amount 20}]}
#2017-08-0215:08mpenetseems fine to me, could be a (s/and (s/keys ...) sum-amounts>=total?)#2017-08-0215:13mpenetthen it's just pattern matching on the explain to return something more "flat" for error messages (depending on your usage), which you can do with/without spec (you could build it with a spec errors conformer that maps to your precise error types)#2017-08-0215:15ghadiagree with @mpenet. i don't think it's the wrong tool at all. just a predicate!#2017-08-0215:25cddrDoh! It never occurred to me that (s/keys) could be anded.#2017-08-0219:45caio@cddr https://github.com/xsc/invariant this looks better for that#2017-08-0219:48ghadion the contrary, I really don't think you need a library to describe a datastructure -- this is literally the rationale for clojure.spec#2017-08-0219:49ghadiwhat cddr wants is a predicate that applies to a collection#2017-08-0219:49ghadiSimilar to how fspec provides the :fn spec that relates a function's :args and :ret#2017-08-0219:49caioyou're trying to define an invariant about a data structure, not a contract. I like the idea of separating those concepts (thus using a library to define invariants looks like a good idea)#2017-08-0219:50caiobut yeah, that's just being pedantic, so for a simple structure like this, I agree adding a new dependency is probably an overkill#2017-08-0219:51cddrI definitely prefer using the standard lib when it supports what I'm trying to do. Cheers for the suggestion though. Looks interesting šŸ™‚#2017-08-0219:53bfabryI think invariant is just a library for defining data structure predicates. if you have to write enough/complicated enough data structure predicates I think it would be useful to use with spec. similar to how specter is useful if you've got to do enough deeply nested data manipulation#2017-08-0219:53caio@cddr yeah, it is! I'm probably biased towards it because I used it and liked it a lot hahaha. In my case, I had to define invariants on a custom markup language#2017-08-0219:54caioso spec wasn't going to help there#2017-08-0221:16rgdelatois it possible to define a spec for map keys without defing a separate spec for each key? I guess what I have in my head is something like:
(s/keys :req-lit {:first-name string?
                  :last-name string?
                  :email (s/and string?
                                #(re-matches email-regex %))})
...where you could spec map keys inline?
#2017-08-0221:17gfredericksI think that's an intentionally excluded feature#2017-08-0221:18rgdelatois there anywhere where I could read up on why that's the case?#2017-08-0221:18bbrinckhttps://clojure.org/about/spec - ā€œMap specs should be of keysets onlyā€#2017-08-0301:37bbrinckIā€™m trying to make a patch to spec.alpha, but Iā€™m running into problems testing my changes (with provided tests) and building a local jar (so I can include in my own projects for further testing). Iā€™m used to doing everything with lein, so Iā€™m less familiar with ā€œbareā€ clojure development. Does anyone know of a good guide? My googling is failing me.#2017-08-0301:40gfredericksmvn test maybe?#2017-08-0301:47bbrinck@gfredericks yes, that works! thank you so much. Iā€™m realizing I never bothered to learn much about the underlying java foundations before jumping straight into lein for managing clj projects.#2017-08-0316:27bmaddyDoes anyone see what I'm doing wrong here?
user> (defn foo [x] :bar)
#'user/foo
user> (clojure.spec/fdef foo :args (clojure.spec/coll-of int?) :ret int?)
user/foo
user> (clojure.spec.test/instrument `foo)
[user/foo]
user> (foo :fail)
clojure.lang.ExceptionInfo: Call to #'user/foo did not conform to spec:
                            In: [0] val: :fail fails at: [:args] predicate: int?
                            :clojure.spec/args  (:fail)
                            :clojure.spec/failure  :instrument
                            :clojure.spec.test/caller  {:file "form-init7754260867026619201.clj", :line 230, :var-scope user/eval98211}
                            
user> (foo 1)
:bar
user> *clojure-version*
{:major 1, :minor 9, :incremental 0, :qualifier "alpha12"}
I would expect (foo 1) to throw an exception with a message about how (int? :bar) is false.
#2017-08-0316:29mpenetyou need to call check#2017-08-0316:29bfabry@bmaddy instrument does not check :ret specs#2017-08-0316:29mpenetinstrument only checks args (sadly)#2017-08-0316:30mpenetI think everyone hits that one once. Not really intuitive I guess#2017-08-0316:30mpenetarguably it can be considered a naming issue at least#2017-08-0316:31bmaddyInteresting.... Is there a reason for instrument not looking at :ret or is it just not implemented?#2017-08-0316:32mpenetit's intentional#2017-08-0316:32mpenetI think it's to allow granularity with gen overrides and such, in theory bundling the 2 in a single function isn't really difficult, it's just not provided#2017-08-0316:35bmaddyOk. Thanks for the help you two!#2017-08-0317:18misha@bmaddy https://groups.google.com/forum/#!msg/clojure/jcVnjk1MOWY/UwP5bc1oCAAJ https://groups.google.com/d/msg/clojure/JU6EmjtbRiQ/uND70kAFBgAJ#2017-08-0317:28matanJust went through the spec guide today... and looking forward to heavily using it for better code and better testing workflow. How do you control whether to instrument, and whether to run test code reliant chiefly on stest.check, using leiningen? What would a leiningen workflow look like?#2017-08-0317:29bmaddyInteresting, thanks @misha. Those comments were very helpful.#2017-08-0318:44seancorfield@matan We're mostly using spec for defining and validating data structures, rather than testing. I've suggested to people that they call instrument either in their test fixtures or directly in each test namespace as appropriate (to instrument functions in the namespace under test).#2017-08-0318:45seancorfieldI think there are lots of ways to go about using clojure.spec in tests -- and no "best practices" have arisen yet because it's still alpha and early days.#2017-08-0318:46seancorfieldAs Rich and others have said, generative testing -- via test.check and clojure.spec.test/check -- should probably be viewed as separate from your "unit testing", since it can take a while and you want "unit tests" to run very quickly (for fast feedback).#2017-08-0318:47seancorfieldSo it's probably a good idea to separate those out into "tests" that don't run as a normal part of your "unit" test suite, but can be run via a separate Leiningen/Boot task (so they can still be run automatically as needed).#2017-08-0318:48seancorfieldI will say that with clojure.java.jdbc, which has optional specs, the "unit tests" run much, much slower with instrumentation in place so even that probably needs to be something you optionally enable for testing.#2017-08-0318:50gfredericksI wish there was a good way of specifying different profiles for test.check properties#2017-08-0318:51gfredericksyou want to do different things with it at different times#2017-08-0318:52seancorfieldFor example, clojure.java.jdbc tests on Clojure 1.8 take about 30 seconds (user; 12.5s real) whereas tests on Clojure 1.9 with instrumentation of all java.jdbc functions take 1 minute 30 seconds (user; 1 minute real). So that's a huge overhead.#2017-08-0318:56royalaid@seancorfield thanks for clarifying that! I have been trying to add specs to an internal tool at where I work and seems like a massive uphill battle to get them to work in the way that I was expecting. Having the generative tests be a separate thing all together make so much more sense#2017-08-0320:08seancorfieldWe use s/conform a lot for validating (and coercing) input to our REST API and also for user input validation, where we can pick apart s/explain-data to generate informative error messages. That stuff's all in production.#2017-08-0320:10seancorfieldWe use test.check and stest/check for a small handful of what would otherwise be "unit tests" where they don't introduce a drag on our tests. We use s/exercise and rand-nth to get random, conforming input in some of our tests.#2017-08-0320:11seancorfieldSo far we've generally kept instrument and stest/check primarily for manual, isolated test runs. We'll probably integrate that based on environment variables or Boot task flags at some point.#2017-08-0320:12seancorfield@royalaid One thing to be careful of: don't try to spec everything! Use spec where it provides the most leverage, at system boundaries, and for key APIs, and/or key arguments to those APIs. One of the really nice aspects of spec (compared to, say, Typed Clojure) is that you really can opt-in one piece at a time, where you need it most.#2017-08-0321:24andrewboltachevHello. Is it possible to specify a complex spec inline (w/o s/defing something)?#2017-08-0321:30seancorfield@andrewboltachev You mean directly in a s/conform or s/valid? call, for example? Sure, specs are "just" predicates.#2017-08-0321:31andrewboltachev@seancorfield To be specific, I want to validate a map#2017-08-0321:31andrewboltachevbut e.g. (s/keys :req-un [::a]) refers to a symbol defined in (some) ns#2017-08-0321:32seancorfieldFor s/keys you must s/def the keys if you want them validated.#2017-08-0321:32andrewboltachevi.e. it takes both name and value from it#2017-08-0321:32seancorfieldIf you read the spec rationale, it explains why you can't specify both key names and value "types" together.#2017-08-0321:32andrewboltachevgot it, thanks! looks anyway slightly opinionated solution for me šŸ˜‰#2017-08-0321:32seancorfieldAt least it's the correct opinion šŸ™‚#2017-08-0406:13matanThere's not really something like that, a "correct opinion" for how to provide an API for humans šŸ™‚#2017-08-0321:35mishad#2017-08-0402:59bbrinckBesides core.specs.alpha, does anyone know of an open-source project with a fair number of specs?#2017-08-0403:00ghadi@bbrinck https://github.com/ring-clojure/ring-spec/blob/master/src/ring/core/spec.clj#2017-08-0403:01bbrinck@ghadi Perfect#2017-08-0403:01bbrinckThanks!#2017-08-0406:11matanThanks for yesterday's short discussion of spec.check and all of that. I guess I'd use custom leiningen tasks and all those related leiningen features, to fiddle my workflow. As for test duration, a quick glimpse reveals that the number of generated inputs/runs is a parameter, and I'd gladly use slightly longer-running tests while controlling it#2017-08-0414:46yendaIs there a way with spec to validate maps like {:a 1 :b {:c 2}} were a is optional and c is only required when a is not present#2017-08-0417:23Alex Miller (Clojure team)you can s/and a custom predicate to make any check you like#2017-08-0417:23Alex Miller (Clojure team)something like (s/and (s/keys :opt-un [::a ::b]) #(if (:a %) (contains-key? (:b %) :c) true)) (can be improved, but you get the idea)#2017-08-0417:24Alex Miller (Clojure team)I flipped the case on :a but you know#2017-08-0423:53leongrapenthinIs there a way to prevent spec from checking fspeced lambdas with generative testing when instrumentation is enabled? It's not helpful when those functions are Callbacks that do side effects#2017-08-0423:58leongrapenthinI'd really like a knob to disable validation of fspeced lambdas through generative testing#2017-08-0423:58leongrapenthinAugmentation with a wrapper that checks the fspeced lambda during invocation against the fspec would be fine.#2017-08-0423:58leongrapenthinWould be great, even more so#2017-08-0500:26wilkerlucio@leongrapenthin I think you can do that, you have to give the fspec a name (make a s/def with it), use that name on your usages, then use spec overrides during the run of check, as documented here: https://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/test/alpha.clj#L381#2017-08-0500:34leongrapenthin@wilkerlucio Not sure thats what I'm looking for. My case is like this: A global functions :args are speced like this (s/cat ::config (s/keys :req [::on-click-save])), then there is (s/def ::on-click-save (s/fspec :args (s/cat :cust ::customer)))#2017-08-0500:34leongrapenthinNow when I invoke the global function with {::on-click-save (fn [cust] (persist-on-server cust))}, on-click-save will be invoked a hundred times with generated ::customers by specs validation code#2017-08-0500:35wilkerlucioah, true, that's for generators#2017-08-0500:36wilkerluciowell, not sure if there is an API for that, but something you could do is to change the spec registry directly, maybe that's a bad idea, but I guess it could work#2017-08-0500:36leongrapenthinYeah sure#2017-08-0500:36wilkerlucioits just a map in an atom, nothing fancy#2017-08-0512:59leongrapenthin@wilkerlucio it appears that setting s/fspec-iterations is a workaround, but it disables it for all#2017-08-0512:59leongrapenthinsetting it to 0#2017-08-0518:37leongrapenthincreated a ticket for it https://dev.clojure.org/jira/browse/CLJ-2217#2017-08-0522:39souenzzoAnyone using spec to specify strings/routes/path/ns? I do not know if it was made for this, but it looks promising
(s/def ::path (s/or :path1 (s/cat :start #{\/}
                                  :name (s/* (s/and char?
                                                    #(not= % \/))))
                    :path2 (s/cat :start #{\/}
                                  :name1 (s/* (s/and char?
                                                     #(not= % \/)))
                                  :div1 #{\/}
                                  :name2 (s/* (s/and char?
                                                     #(not= % \/))))))
=> :user/path
(s/conform ::path (map identity "/foo"))
(s/conform ::path (map identity "/foo/bar"))
=> [:path1 {:start \/, :name [\f \o \o]}]
=> [:path2 {:start \/, :name1 [\f \o \o], :div1 \/, :name2 [\b \a \r]}]
#2017-08-0522:44souenzzoEven generators work: (map (partial s/conform ::path) (gen/sample (s/gen ::path)))#2017-08-0616:40scaturrHow would s/conform work in an http context? Like how would I conform JSON payloads for specs defined with namespaced keys? Like if I have a spec for ::entity/id instead of just :id?#2017-08-0616:42scaturr(s/conform ::entity {:id some-uuid}) -> invalid#2017-08-0618:22dergutemoritz@scaturr You can use :req-un instead of :req to achieve what you want in this case#2017-08-0618:42scaturrwould that lead to a situation where you have specs exist just for json conforming? or can :req and :req-un co-exist?#2017-08-0618:42scaturrseems like they could not#2017-08-0618:43scaturrthank you for the suggestion though šŸ™‚#2017-08-0621:04stathissiderisJust released spectacles: lenses for #clojure -- checked at runtime using #spec https://github.com/stathissideris/spectacles#2017-08-0622:11schmee@scaturr have a look at this: https://github.com/metosin/spec-tools#dynamic-conforming#2017-08-0622:11schmeeand the blog post: http://www.metosin.fi/blog/clojure-spec-as-a-runtime-transformation-engine/#2017-08-0719:05souenzzoMy actual spec says (s/def ::my-fns (s/coll-of fn?)). Can I specify that it's a array of functions with arity 2?#2017-08-0719:11souenzzo(s/def ::my-fns (s/coll-of (s/fspec :args (s/cat :foo integer? :bar integer?)))#2017-08-0719:14bfabryseems like that should work, does it not?#2017-08-0719:16souenzzoworks šŸ˜„. But i think that it's not on docs. I dig into code...#2017-08-0802:00didibusCan I spec a map in a way where I want it to have at-least one of two keys. So a valid map would be: {:x :y} {:x} or {:y}, but {} would not be valid?#2017-08-0802:24souenzzo@didibus
(s/valid?
  (s/or :a (s/keys :req [:foo/bar]
                   :opt [:bar/foo])
        :b (s/keys :req [:bar/foo]
                   :opt [:foo/bar]))
  {}
  )
#2017-08-0802:26didibusAh yes, that should work!#2017-08-0802:26didibusthx#2017-08-0803:25joshjones@didibus @souenzzo it's even more concise:
(s/valid?
  (s/keys :req-un [(or ::x ::y)])
  {:y 42 :x 33})
#2017-08-0813:26souenzzoWhy does it do not generate`{:x 33}`?
(gen/sample (s/gen
                (s/keys :req-un [(or ::x ::y)])))
=>
({:x -1, :y 0}
 {:x 0, :y -1}
 {:x 0, :y -1}
 {:x 0, :y 1}
 {:x 0, :y -1}
 {:x -2, :y 0}
 {:x 6, :y -21}
 {:x 6, :y 0}
 {:x 4, :y -23}
 {:x -8, :y 117})
#2017-08-0823:11madstap@souenzzo https://dev.clojure.org/jira/browse/CLJ-2046#2017-08-0814:58mbarbieriI'm playing for the first time with spec. I want to generate a spec from a contract map and test if another map is conforming. As first step I would like that all keys in the map are present in the contract map. But i read the contract from a json file, converting to unqualified keywords, so
(s/keys :opt-un (keys contract-map))
gives me Assert failed: all keys must be namespace-qualified keywords Do I have to convert them to qualified? Or is it a better way to handle it?
#2017-08-0815:01misha@mbarbieri :opt-un expects coll of qualified keys (which might point to their specs), -un part indicates that data will have no namespaces in keys.#2017-08-0815:04mbarbieri@misha ok data will have no namespaces, but not even the keys I'm passing in the spec will have, hence the failure. If I'm not wrong, that exception happen because my code will evaluate in something like:#2017-08-0815:04mbarbieri
(s/keys :opt-un [:a :b :c])
#2017-08-0815:06mishait is ok for data to have no namespaces, but in spec declaration you need to supply namespaces, so later you can assign actual specs to those qualified keys, and s/keys will check not only keys presence, but will check values against those specs as well#2017-08-0815:08mishayou need to supply qualified keys, yes. how exactly - depends on your use case. If you are want to generate spec once and put in the source file ā€“ just come up with a namespace and list those manually. If you need to generate that spec anew dynamically every time ā€“ that'd seem odd to me#2017-08-0815:08mishaanyway. check out this https://github.com/stathissideris/spec-provider#2017-08-0815:09mishamight be useful in any case#2017-08-0815:09stathissiderisIā€™m here in case there are any questions šŸ™‚#2017-08-0815:10mishaI have one! not lib related tho :)
(s/explain-data
  (s/map-of qualified-keyword? fn? :min-count 0)
  {:foo/bar 9})
;;=>
#:clojure.spec.alpha{:problems ({:path [1],
                                 :pred fn?,
                                 :val 9,
                                 :via [],
                                 :in [:foo/bar 1]})}

(get-in {:foo/bar 9} [:foo/bar 1]) 
;;=> nil
(replaced string with number to illustrate my confusion with 1 as path inside 9(?))
#2017-08-0815:11mishawhat 1 in :in [:foo/bar 1] is supposed to point to?#2017-08-0815:11mishahow do I actually use this :in path? Which data structure do I apply it to?#2017-08-0815:12mbarbieri@misha ok thanks#2017-08-0815:13misha
from 
:in - the key path through a nested data val to the failing value. In this example, the top-level value is the one that is failing so this is essentially an empty path and is omitted.
#2017-08-0815:22misha
(s/def :foo/bar (s/tuple int? int? fn?))
(s/explain-data
  (s/map-of qualified-keyword? :foo/bar :min-count 0)
  {:foo/bar [1 2 3]})
gives :in [:foo/bar 1 2], where 2 makes sense as index of val 3, but 1 still does not, at least while I am thinking in terms of (get-in data in)
#2017-08-0815:28mishahttps://github.com/bhb/expound/blob/master/src/expound/alpha.cljc#L488-L494 harold#2017-08-0817:05souenzzoI'm about 5 minutes waiting this repl command
(-> (test/check `spec-utils/atributos-do-pattern)
        (test/summarize-results))
There is some way to limit the number of tests that test/check do?
#2017-08-0817:41spinningtopsofdoomI think
(-> (test/check 10`spec-utils/atributos-do-pattern)
        (test/summarize-results))
#2017-08-0817:41spinningtopsofdoomis what you want#2017-08-0817:56souenzzoIllegalArgumentException Don't know how to create ISeq from: java.lang.Long clojure.lang.RT.seqFrom (RT.java:542)#2017-08-0817:57souenzzoAnother question: how to define zero args? (s/fdef foo :args (s/cat)) works, but not sure if it's right...#2017-08-0818:03spinningtopsofdoomOops my mistake I was thinking of the test.check library
(-> (test/check spec-utils/atributos-do-pattern {:num-tests 10})
        (test/summarize-results))
see this link for details (look for ::stc/opts) https://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec.test/check
#2017-08-0818:06spinningtopsofdoomI think (s/fdef foo :args (s/cat)) is correct. s/cat constructs a sequence on 0 - N specs (like it's regex namesake) so no specs given to it should mean zero arguments expected#2017-08-0818:39kennyIs there a best practice you guys are following regarding enabling of instrumentation? I am finding that many times if I don't enable or forget to enable it, my functions will silently fail and I receive an obtuse error message. Normally I would have an assertion at the beginning of the function to produce a clear error message but I figured spec instrumentation would replace this.#2017-08-0818:43souenzzosubscribe#2017-08-1012:48souenzzoBump!#2017-08-1013:44souenzzoCheckout (clojure.spec.test.alpha/instrumentable-syms) and (stest/instrument)#2017-08-0820:30souenzzoWhat is the current status of clojure.spec.specs.alpha? (just curious/studding)#2017-08-0822:06mrkaspais there a library to convert clojure.spec/explain-data to something friendly to respond in a json api? I was using the struct library I want something similar#2017-08-0822:12Oliver GeorgeHas anyone written a spec function coverage tool? e.g. report the percentage of functions with a spec defined by namespace or project.#2017-08-1013:52souenzzoIt's easy to write... Checkout clojure.spec.test.alpha/instrumentable-syms and clojure.repl/dir-fn#2017-08-1023:17Oliver GeorgeThank you I'll try those#2017-08-1101:50Oliver GeorgeThanks for the help. I'm writing clojurescript which isn't quite as self aware but I had success on a few fronts.#2017-08-1101:50Oliver GeorgeSimple case is checking a namespace.
(defn var->sym [var]
  (let [{:keys [ns name]} (meta var)]
    (symbol ns name)))

(defn interns->fn-syms [interns]
  (->> (vals interns)
       (filter (comp fn? deref))
       (map var->sym)))

(deftest sims-logic-coverage
  (let [instrumentable-syms (stest/instrumentable-syms)
        instrumentable? #(contains? instrumentable-syms %)
        fn-syms (interns->fn-syms (ns-interns 'sims.logic))]
    (doseq [fn-sym fn-syms]
      (is (instrumentable? fn-sym) "No s/fdef for function"))))

(test/run-tests)
#2017-08-1101:51Oliver GeorgeThe "total coverage" option was possible but messy. I had to use cljs.analyzer.api to know what namespace and interns existed.#2017-08-0901:15Oliver George@mrkaspa clojure.spec provides a basis for that sort of thing but leaves other tools to craft user friendly messages.#2017-08-0901:15Oliver GeorgeExpound is getting a bit of attention: https://github.com/bhb/expound#2017-08-0901:15Oliver GeorgeThere are others.#2017-08-0901:15Oliver GeorgeThe other JSON api consideration is that clojure.spec deals in clojure data structures (maps, sequences...) not javascript objects and arrays.#2017-08-0901:16Oliver GeorgeYou can do something like js->clj to translate the data into a format clojure.spec likes#2017-08-0901:17Oliver GeorgeThe error messages might be a little confusing where they say "this keyword is missing" when a JS consumer might expect "this property is missing"#2017-08-0910:21ikitommi@mrkaspa I'm returning everything (problems + extra info of the cause) to clients, specs can be serialized via s/form. Not sure how usefull this is thou. Spec-tools has utility for adding human readable :reason for specs (like struct does).#2017-08-0912:03mishahow can I properly delete the spec? (undo s/def, actually, undo s/fdef)#2017-08-0912:08mishause case is: first, I get some function symbols from config, resolve them vars, and see if those are actually functions. next, I want to enforce common function signature on those, so I want to dynamically do (s/def 'some-handler :lib.specs/handler). This is all cool, but I want to clean specs registry from those dynamically set specs on some tear down event (some reload in REPL, or what not).#2017-08-0912:11misha(or at least to know I don't have to do cleaning up at all, because spec registry thing was designed to not worry about it.)#2017-08-0912:12mishaso far I found only this https://dev.clojure.org/jira/browse/CLJ-2060#2017-08-0912:16mishaactually, I control the call site, and can ensure args are ok, but does fspec actually check the arity of a function? or does it check args, and if a function (not args) does not conform to the spec ā€“ it will blow up, but not in the spec-way, but in usual invalid arity one?#2017-08-0912:34mishayup
(s/fdef user/foo  :args (s/cat :x int? :y int?)  :ret nil?)
;; => user/foo

(defn foo [x])
;; => #'user/foo

(st/instrument)
;; => [user/foo]

(apply foo [1 2])
;; clojure.lang.Compiler$CompilerException: clojure.lang.ArityException:
#2017-08-0912:39mishaso the best UX I can provide, is to make "handler" fspec available, so user could instrument handlers during development at will.#2017-08-0913:37mishaGenerating queue, am I doing it right?
(:import
   #?(:clj  [clojure.lang PersistentQueue])))

(s/exercise
  (s/coll-of int?
       :kind #(instance? PersistentQueue %)
       :into (PersistentQueue/EMPTY))
  1)

;([#object[clojure.lang.PersistentQueue 0x66b3bed3 "
#2017-08-0914:01bbrinck@misha Looks OK to me. I would have written :into PersistentQueue/EMPTY (no parens), but in practice, it doesnā€™t seem to make a difference#2017-08-0914:41misha@bbrinck you might be right (originally I had a (q) cljc function call there, edited for brevity). Still need to see how it'd work in cljs.#2017-08-0914:50mishato create spec for keyword with arbitrary namespace, I need to (create-ns ...) first, right (if there is no file for that ns in a project)?#2017-08-0915:00bbrinck@misha In CLJS you can do :kind #queue []#2017-08-0915:01bbrinck@misha Iā€™m not sure if this works for your use case, but specs just need to be namespaced, but that namespace doesnā€™t have to be a clojure namespace. e.g. (s/def :user/name string?) is totally valid.#2017-08-0915:02bbrinckThe jury is still out on the pros/cons, but Iā€™ve tended to build my specs based on my logical domain, not my Clojure namespaces. I never get to use the :: syntax, but the plus side is that when I print out my specs (or get spec failures), things are a bit more succinct#2017-08-0915:03bbrinckFor the example above, namespacing further by app or company helps avoid conflicts e.g. :my-app.user/name#2017-08-0915:52misha@bbrinck ah, I went step further, and it complained about aliased namespace, which I need to create before aliasing. So if I want to use ::u/name, I need to (create-ns 'user) and (alias 'u 'user) first.#2017-08-0917:43Alex Miller (Clojure team)thatā€™s totally fine (and something Rich has ideas about ways to improve specifically re working with keyword namespace aliases)#2017-08-0917:45misha@alexmiller any news on unregistering specs?#2017-08-0917:45Alex Miller (Clojure team)yeah, thereā€™s a ticket out there, should be in next spec batch#2017-08-0917:45Alex Miller (Clojure team)plan is to have (s/def ::foo nil) do this#2017-08-0917:46Alex Miller (Clojure team)https://dev.clojure.org/jira/browse/CLJ-2060#2017-08-0917:56didibusQuestion: I'm not sure I understand what fmap is for. When would I want to use it. Its one of the few functions that has no doc.#2017-08-0917:58Alex Miller (Clojure team)all of the gen namespace functions are dynamically loaded and thus donā€™t have doc, but they are just aliases for the equivalent function in clojure.test.check.generators#2017-08-0917:58Alex Miller (Clojure team)fmap is used to construct a generator based on applying an arbitrary function to some other generator#2017-08-0917:59Alex Miller (Clojure team)there is an example in the guide https://clojure.org/guides/spec#2017-08-0918:00Alex Miller (Clojure team)test.check.generators doc is at https://clojure.github.io/test.check/clojure.test.check.generators.html#2017-08-0918:00Alex Miller (Clojure team)and more examples at https://clojure.github.io/test.check/generator-examples.html#2017-08-0918:00Alex Miller (Clojure team)although fmap is not really covered there#2017-08-0921:41dspiteselfit appears that s/? breaks up the merging of the s/cat#2017-08-0922:21ghadi@dspiteself https://dev.clojure.org/jira/browse/CLJ-2105 & https://dev.clojure.org/jira/browse/CLJ-2003#2017-08-0922:22ghadiYou may want to add that test case if it's substantially different#2017-08-0922:23ghadi(I haven't looked closely, just aware of the bug)#2017-08-0922:46dspiteselfThanks#2017-08-0923:17didibus@alexmiller Thanks. I meant test.check doesn't have a doc for fmap. I get it now though.#2017-08-0923:26didibusQuestion: Is there a way I can specify that a spec can be one of a set of specs? Such as (s/def ::user-info #{::name ::address}). Where (s/def ::name (s/keys [::first ::last])) and (s/def ::address [::street ::city ::country]))#2017-08-0923:57didibusOkay, I think multi-spec is what I want: https://clojure.org/guides/spec#_multi_spec#2017-08-1000:37gfredericks@didibus the alpha versions of test.check have a docstring for fmap#2017-08-1000:43bfabry@didibus I don't think you want multi-spec, that's for multimethods. you just want s/or I think#2017-08-1000:44didibusHum, what would multi-spec do that s/or wouldn't?#2017-08-1000:47bfabryrequires a defined multimethod by the looks#2017-08-1000:48didibusI guess its just that multi-spec is open for extension#2017-08-1000:49didibusSo maybe useful for libraries?#2017-08-1000:49bfabryseems to require a tag of some sort on your entity#2017-08-1000:49bfabrywhich you don't have in your original example?#2017-08-1000:50didibusOh right, hum. I guess I see. I think multi-spec is more what I'm imagining in my head, since I'm thinking of it as an inheritance of some sort. Like a shape can be a block square or cube.#2017-08-1000:51didibusBut, I guess I don't really care about the type name, what's that called, when you have types defines by there shape and not a name?#2017-08-1000:52bfabryin static typing land I believe it's called "structural typing", but I've been writing in dynamic languages for like 8 years at this point#2017-08-1000:52bfabryso not the person to ask šŸ™‚#2017-08-1000:53didibushaha, anyways, thanks for the alternative, I'll think about which one works best for what I'm trying to do#2017-08-1002:46dspiteself@alexmiller I submitted a patch on https://dev.clojure.org/jira/browse/CLJ-2003 . I tried to find a spec.alpha JIRA project. Am I correct to submit the patch to Clojure?#2017-08-1012:07Alex Miller (Clojure team)Yep!#2017-08-1006:50mpenetquestion of style: when you spec (deeply) nested maps inside a namespace. do you create tons of new deep ns for keys specs corresponding to the map shape ex: ::foo.bar.baz.etc/* or do you use a more flat naming scheme (like inner classes in java) so ::foo$bar$baz/* (spec-tools seems to do something like this for the Schema like api I think)?#2017-08-1012:09Alex Miller (Clojure team)I would definitely not do the latter and would potentially not even do the former but instead use the same namespace for many of the levels#2017-08-1012:21mpenethow would you do that when you spec a json api for instance, there often are duplicates in key names#2017-08-1012:21mpenetthing.id foo.otherthing.id that.id etc#2017-08-1012:23mishayeah, like in
(s/def :foo.bar/name one-name-pred?)
(s/def :foo.baz/name different-name-pred?)
(s/def :foo/bar (s/keys :req-un [:foo.bar/name]))
(s/def :foo/baz (s/keys :req-un [:foo.baz/name]))
#2017-08-1012:25mishaI try to keep all specs in a single file, at least in active design/development phase, so I end up with something like above (or your 1st approach, @U050SC7SV )#2017-08-1012:26mpenetyup, involves a lof of create-ns juggling#2017-08-1012:26mpenetsame, I tend to create foo.clj foo_specs.clj pairs#2017-08-1012:27mishanot, if you are willing to use long qualified keywords, instead of aliases.#2017-08-1012:28mpenethence the create-ns stuff#2017-08-1012:28mishaI still did not calibrate my preference of aliases over long kws. Converted 300 lines of specs yesterday to aliases ā€“ can't understand shit now dafuq#2017-08-1012:29mishae.g.
(alias 'fc 'fsm.compiled)
(alias 'fcs 'fsm.compiled.state)
(alias 'fce 'fsm.compiled.event)
(alias 'fcg 'fsm.compiled.guard)
(alias 'fcb 'fsm.compiled.behavior)
(alias 'fcm 'fsm.compiled.machine)
(alias 'fctran 'fsm.compiled.transition)
(alias 'fctrig 'fsm.compiled.trigger)
#2017-08-1012:29mpenetI don't abuse aliases (I never create them manually)#2017-08-1012:30mishanow quick, whose ids are those? ::fcb/id and ::fcs/id kappa#2017-08-1012:30mpenetI just create-ns under the current ns (via a macro) and add dotted childs foo.clj -> :foo/* :foo.bar/* foo.bar.baz/*#2017-08-1012:33mishaI might revert all of the aliases, and settle down with just long qualified kws. Will have to spend ~same amount of time to comprehend same spec, but at least I will not need to jump around the file in process#2017-08-1012:34mpenetif you do it "locally" aliases are not really necessary you can use ::bar.baz.prout within 'foo ns for foo.bar.baz.prout etc#2017-08-1012:34mpenetit's what I have found to be the less horrible to deal with that stuff, and semantically it's kinda clean#2017-08-1012:35mishaon the other hand, those namespaces are not that long. I'd not like to have 1kloc of specs for, say, ring.middleware, where nss are 100 chars long harold#2017-08-1012:36mishalet me try that. (sorry, for highjacking thread, and driving chances of @alexmiller reading all of it to zero opieop )#2017-08-1012:37misha@U050SC7SV
::foo/bar
Invalid token: ::foo/bar, compiling (/.../src/kek/fsm_spec.cljc:15:1)
#2017-08-1012:38mpenetyou still have to create-ns the ns yep#2017-08-1012:39mpenetno way around this (for now)#2017-08-1012:39mishasame after
(create-ns 'kek.fsm-spec.foo)
=> #object[clojure.lang.Namespace 0x80d220d "kek.fsm-spec.foo"]
#2017-08-1012:40mishabasically, I just need longer alias names#2017-08-1007:19mpenet(I usually go with the former)#2017-08-1015:11mishadoes not really feel "reuse", man harold#2017-08-1017:03misha@souenzzo interesting, but I will still require to copy/paste dispatch keys by hand#2017-08-1019:43didibusQuestion: What versioning scheme does clojure.spec uses. Specifically, what part of the version do indicates non-backward compatibility if it changes? "1.9.0-alpha17". Is it the number following alpha? Is it the minor 0 version, or the 9 or the 1?#2017-08-1019:49souenzzoinside -alphaXX, any change can break i think#2017-08-1020:15seancorfieldAlso note that Spec is currently clojure.spec.alpha and has a different version number to Clojure itself, as do the also separate core specs.#2017-08-1020:23didibusOh I see: 0.1.17#2017-08-1020:24didibusSo, the way they have it now, is there a strict notion of breaking change on the version? Like 0.2 would be breaking, but not 0.1.18 ?#2017-08-1020:32bfabryI would say while it's still labelled alpha any release could be breaking, as it's more just a "new build"#2017-08-1021:36seancorfieldI get the impression -- from talking to Rich and Alex and others, and from what they've said in public talks at conferences -- that "The Clojure Way" doesn't really pay much attention to version numbers... After all, "major" releases of Clojure are all 1.x so far, and nearly all the Contrib libraries are 0.x.y but are production-ready...#2017-08-1021:37seancorfieldI suspect we'll see a bigger push -- across the whole community -- for non-breaking future releases while the artifact/namespace name is unchanged.#2017-08-1021:37seancorfieldFor example, Rich's Spec-ulation talk.#2017-08-1021:38seancorfieldI've struggled with this with clojure.java.jdbc because I've had several releases that have been breaking ones because the API took a long time to drift toward "best practices" as they've evolved over the last 5-6 years I've been maintaining it.#2017-08-1021:39didibusIts true, but all Clojure versions have historically maintained backwards compatibility. Also, while Rich Hickey doesn't seem to like versions, he does like clearly indicating when things break backward compatibility, though he suggest a whole new name instead of a version bump. At least that's what I got from his talk.#2017-08-1021:40seancorfieldAt one point, I pushed the entire API into a "deprecated" namespace, allowing for a simple cross-application edit for users, if they wanted to upgrade without changing all their calls. I think the new way means we'll get "versions" encoded in the namespace names instead, at least until we figure out a better approach.#2017-08-1021:40didibusNot really an aversion for versions, just that we could do much better I guess#2017-08-1021:41seancorfieldAt least with clojure.spec.alpha, you know it's subject to change and can (and has!) cause(d) code breakage.#2017-08-1021:41seancorfieldOnce it's ready to become stable, it'll get renamed back to clojure.spec (and folks can continue using clojure.spec.alpha until they're ready to switch).#2017-08-1021:42didibusYa, I'm assuming this now. I think its the right assumption for spec when in alpha#2017-08-1021:42didibusthanks#2017-08-1021:44seancorfieldWe've just had a long discussion about this with clj-time because I'd originally pushed to switch its implementation from Joda Time to Java Time. With discussions over about a year, it became clear that wouldn't be possible while retaining the clj-time/clj-time coordinate due to possible conflicts in versions across multiple libraries that depend on clj-time -- so it's sort of become the poster child for "don't break code when you only change the version".#2017-08-1021:48didibusHum, ya#2017-08-1021:50didibusWell, in fact, my reason for asking was that at my company, we internally maintain our own package repo. And our version scheme actually goes like this: Any.Number.Affects.Backwards.Compatibility. And we drop all part of a version that doesn't affect backwards compatibility.#2017-08-1021:50bfabrylol, I appreciate you not breaking clj-time sean, that would definitely give me a headache#2017-08-1021:50didibusIn that way, conflicts auto-resolve to the newest version that does not break backwards compatibility. But for that, you have to tell the build tool what numbers in the library versioning scheme is the backward breaking one.#2017-08-1021:52didibusSo, I think its actually more in line with Rick's vision. Just freely upgrade things, but don't break people. And if you need to break people, then and only then, just make a fork as a whole new name. So sure, that could be clj-time_v2, but the version is now only a way to indicate incompatibility, and nothing else.#2017-08-1021:55didibusP.S.: Aren't you stuck using joda-time until clojure drops supports for Java 1.6 and 1.7?#2017-08-1022:56seancorfieldHeh... read the discussion (that started in November 2015!) about that: https://github.com/clj-time/clj-time/issues/196 /cc @U050MP39D#2017-08-1022:57seancorfieldMy solution to this at work was to introduce clojure.java-time and start switching code over to use that instead (of clj-time and date-clj which we were previously using).#2017-08-1022:58seancorfieldGiven the presence of clojure.java-time, I'm not very inclined to create a "somewhat API-compatible version of clj-time" based on Java Time -- I'd rather support clojure.java-time instead.#2017-08-1100:11didibusAh, didn't know about clojure.java-time. Looks good. Though what I'd like to see is a common clojure/clojurescript API for date and time.#2017-08-1100:16seancorfieldWell, there's cljs-time which mirrors clj-time, and in the clojure.java-time repo there's an issue proposing a cljs API that matches...#2017-08-1100:18didibusOh, okay ya, I guess that's good enough.#2017-08-1105:51tianshuCan I add spec to a function before I define the function? like @spec in elixir. And is it possible to have a variable T to represent some type like generic types?#2017-08-1105:56seancorfield@doglooksgood Not sure what you mean about "generic types" in the context of Clojure?#2017-08-1105:57seancorfieldas for declaring a spec for a function before its defn, yes, that definitely possible...#2017-08-1106:09tianshumy mistake, I was thought that I have to write spec for function after that is defined. for my first question, maybe I shouldn't call it a generic type. I want to know something that can help me writing a spec for functions like map.#2017-08-1106:17seancorfieldmap is a pretty hard function to write a spec for -- especially since the one argument version returns a transducer šŸ™‚#2017-08-1106:18seancorfieldYou don't need to spec everything. Just spec at the boundaries of your subsystems and/or APIs.#2017-08-1106:20tianshuit seems impossible for writing spec for a function that receive T and returns T?#2017-08-1116:14Alex Miller (Clojure team)correct - specs are not parameterized#2017-08-1117:17didibusI think it doesn't make sense to have parameterized specs. Because they're predicate based, the parameters would be non intuitive. For each one you'd have to pass in a pred, but you wouldn't know how that pred would combine with the spec. And each spec could do it differently. That said, I'm not sure, it's a good idea. I'm trying to think if there's useful real use cases for it, nothing jumps at me right away, can you suggest one?#2017-08-1118:49Alex Miller (Clojure team)well, weā€™re not going to do it, so Iā€™m not going to invent a reason why :)#2017-08-1106:20seancorfieldI think spec is much more valuable for spec'ing data structures than functions, to be honest, but that might be the domain I'm working in (where I'm using spec mostly to describe what API parameters should be, so s/conform and s/invalid? are being called).#2017-08-1106:20seancorfieldSo you want the type of the return to match the type of the first argument?#2017-08-1106:21tianshuyes#2017-08-1106:21tianshuso I should use :fn in fdef?#2017-08-1106:21seancorfieldYou can spec that with :fn to check that yes.#2017-08-1106:22seancorfieldDo you really want identical types tho'? What sort of function are you talking about?#2017-08-1106:25seancorfieldFor example, map takes a function and a collection and returns a lazy sequence so the types don't match there -- but I guess you could check non-empty collection/sequence members being the same type? But you can't do much for empty collection/sequence. Would you want to check the whole input/output? What about lazy sequences not being entirely consumed, like (take 5 (map inc (range)))?#2017-08-1106:26seancorfieldYou need to be careful that the spec doesn't realize the whole sequence.#2017-08-1106:43misha@doglooksgood :fn will not be checked by instrumentation. you will need you run tests explicitly.#2017-08-1106:55tianshuI got a library for this#2017-08-1106:56tianshuBut I think it will be nice to have some spec inference in editor. but without type variable, it seems to be difficult.#2017-08-1108:31misha@doglooksgood you might like this https://github.com/stathissideris/spec-provider#2017-08-1117:25didibus@doglooksgood I'd give a look at https://github.com/arohner/spectrum It gives everything you spec a type based on the spec, and then uses that to validate types at compile time. Maybe you could use it to infer things in an editor.#2017-08-1119:16arohnerIā€™m slowly working on inference of untyped fns#2017-08-1119:29arohneralso, spectrum isnā€™t really usable for production use yet#2017-08-1119:59souenzzoFunctions with the signature like +: (fn [& args])... How to declare theirs specs? (s/fdef my-fn :args (s/coll-of integer?)) ?#2017-08-1120:02bfabry@souenzzo (s/cat (s/* integer?))#2017-08-1120:08bfabry@souenzzo actually minus the cat, just (s/* integer?)#2017-08-1120:17bfabry
boot.user=> (defn +'' [& args] (apply + args))
#'boot.user/+''
boot.user=> (s/fdef +'' :args (s/* integer?))
boot.user/+''
boot.user=> (st/instrument)
[boot.user/+'']
boot.user=> (+'' 1 2 3)
6
boot.user=> (+'' 1 2 3 'a)
clojure.lang.ExceptionInfo: Call to #'boot.user/+'' did not conform to spec:
                            In: [3] val: a fails at: [:args] predicate: integer?
#2017-08-1205:28mattlyIā€™m trying to migrate a project from pre clojure.spec.alpha to post-that, and I get this error:#2017-08-1205:29mattlynowhere in the project is clojure.spec.gen.alpha referenced directly#2017-08-1205:29mattlyIā€™m on clojure 1.9-alpha17#2017-08-1205:29mattlyany ideas?#2017-08-1205:59mattlyhm, apparently I had a clojure version mismatch somewhere#2017-08-1205:59mattlyits working now I think#2017-08-1208:09danielstocktonIs it possible to remove all conforming on a spec so that it just returns identity (but still does the validation)?#2017-08-1208:53mpenetWith a conformer that calls unform? (yuk)#2017-08-1208:55mpenetSounds like the wrong solution to the pb. Maybe you need to decompose upstream spec(s) to build this kind of things#2017-08-1209:28danielstocktonYeah, I abandoned that approach. I'm trying to submit a patch for https://dev.clojure.org/jira/browse/CLJ-2199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel#2017-08-1209:29danielstocktonCame to what I think is a better solution.#2017-08-1211:14misha@danielstockton clojure.spec.alpha/nonconforming
(defn ^:skip-wiki nonconforming
  "takes a spec and returns a spec that has the same properties except
  'conform' returns the original (not the conformed) value. Note, will specize regex ops."
  [spec]
#2017-08-1221:38Alex Miller (Clojure team)As a note here, this was intentionally left out of the docs (the ā€œskip-wikiā€ meta) because Rich is still mulling whether to include it. Some time has passed since we last talked about it, but at this point I think we should either keep it, or we should have a nonconforming version of s/or (which is the really common case where you want it).#2017-08-1211:18danielstockton@misha thanks!#2017-08-1216:43mishahttps://www.infoq.com/presentations/clojure-spec toot#2017-08-1221:14Alex Miller (Clojure team)hey, itā€™s me! :)#2017-08-1219:18mishaTIL, regular-expression specs work like splicing, so nested reg-ex specs expect flat data structure dafuq#2017-08-1219:19mishaI read guide like 7 times already, and missed it complitely#2017-08-1221:15Alex Miller (Clojure team)tbh, it took me like flubbing this like 5 times before I really knew it#2017-08-1221:16Alex Miller (Clojure team)and I even wrote 2 implementations of it before the current one!#2017-08-1221:28mishathe spec is still alpha thing makes more sense now troll#2017-08-1309:02mishais there something out of the box for getting conforming subset of a data structure? e.g.
(s/def ::foo (s/keys :req-un [::a ::b]))
(s/valid? ::foo {:a 1 :b 2 :c 3})  ;; => true
(conformed-part ::foo {:a 1 :b 2 :c 3})  ;; => {:a 1 :b 2} (only keys from ::foo spec are kept)
#2017-08-1316:30Alex Miller (Clojure team)in this case, the data structure does conform to the spec#2017-08-1316:31Alex Miller (Clojure team)because s/keys supports open maps#2017-08-1316:31Alex Miller (Clojure team)so, essentially no. there is something here Rich is considering adding but not sure if and when that will happen#2017-08-1406:31mishathe use case is to re-use ui-approved-spec in a select-keys to not leak any sensitive or just extra information about entity, exactly because of s/keys's support of open maps#2017-08-1408:09ikitommi@misha try spec-tools:
(require '[clojure.spec.alpha :as s])
(require '[spec-tools.core :as st])

(s/def ::a int?)
(s/def ::b int?)
(s/def ::foo (st/spec (s/keys :req-un [::a ::b])))
(s/valid? ::foo {:a 1 :b 2 :c 3})  ;; => true

(st/select-spec ::foo {:a 1 :b 2 :c 3}) ;; => {:b 2, :a 1}
#2017-08-1408:47misha@ikitommi will check it out, thanks#2017-08-1412:15hmaurerHi! I have a quick question. I have a spec for a map, which defines a set of required and a set of optional keys (`(s/keys :req [...] :opt [...])`). I would like to generate a version of that spec suitable for ā€œpartial updatesā€. By that I mean that the spec should pass if a required key is missing, but it should not pass if the required key has value nil.#2017-08-1412:16hmaurerIs that possible?#2017-08-1412:17hmaurerFor a real-world use-case, think of a CRUD scenario. When creating an entity I need to validate the presence of some attributes, but when updating I only need to make sure that an attribute marked as required isnā€™t set to nil#2017-08-1412:38Alex Miller (Clojure team)The vals for the required keys (actually all keys) in s/keys will be verified according to the spec for that key. If the spec does not allow nil, it is not valid.#2017-08-1416:17hmaurerOk, thanks!#2017-08-1412:40Alex Miller (Clojure team)
user=> (s/def ::i int?)
:user/i
user=> (s/def ::m (s/keys :opt [::i]))
:user/m
user=> (s/valid? ::m {::i nil})
false
user=> (s/valid? ::m {})
true
#2017-08-1412:41Alex Miller (Clojure team)If the key is required, then its absence will cause validation to fail as required keys are ā€¦ required. :)#2017-08-1416:17hmaurerHi againā€¦ I am getting this error whenever I run the ā€œdocā€ command (and others) in the REPL: java.lang.NoClassDefFoundError: clojure/spec/alpha$get_spec. Any ideas?#2017-08-1416:18bronsathere's already a ticket + patch on jira#2017-08-1416:37hmaurerah, thanks @bronsa#2017-08-1501:55bbrinckFor anyone that uses Orchestra to instrument :fn and :ret specs, Expound 0.2.0 now supports pretty error messages for Orchestra errors. https://github.com/bhb/expound#using-orchestra#2017-08-1510:36hmaurerHi. I am having an issue with clojure.spec. This code throws an error: https://gist.github.com/hmaurer/d030f9ff9250482c6dea3e3738737863#2017-08-1510:37hmaurer(Iā€™ve added the error as a comment on the gist)#2017-08-1510:37hmaurerI think this is because s/keys is a macro, but I am not sure how to get it to do what I want#2017-08-1510:37hmaurer(dynamically build that spec from the map)#2017-08-1510:59moxaj@hmaurer left a comment on the gist#2017-08-1511:01hmaurer@moxaj thank you! So I assume I cannot / should not define specs at runtime? e.g. I canā€™t define specs based on my database schema after opening a connection?#2017-08-1511:01moxajeval is another option#2017-08-1511:02hmaurerah, Iā€™ll try that. Is it an outright terrible idea to define specs at runtime though?#2017-08-1513:31Alex Miller (Clojure team)no#2017-08-1511:10urbankDoes anyone here define specs in the same file as the functions? Above the functions like haskell type annotations#2017-08-1511:16souenzzoI just started to write my specs like that. Not sure if is the best practice or not.#2017-08-1511:17gfredericksI usually do#2017-08-1511:31urbankThe second answer here gives a good arguments#2017-08-1511:31urbankhttps://stackoverflow.com/questions/37942495/where-to-put-specs-for-clojure-spec#2017-08-1513:02mishaI lean towards: data specs in a separate file(s), function specs ā€“ next to actual implementation. (Yet to see how it'll turn out in longer run)#2017-08-1513:08urbank@U051HUZLD Right, that makes sense to. Because functions might share data specs, right?#2017-08-1513:13misha@U489U338R yes, and, functions are more "implementation detail"-y than data, I think. But I'd start with everything in one file, and see what's up later in project.#2017-08-1518:58didibusYa, data spec in their own namespace gives you a clean domain model. And fn specs I put below the functions.#2017-08-1519:13seancorfieldMost of our specs are for data structures and they go in whatever is the appropriate namespace for those data structures -- which is often a separate ns but sometimes is a ns with some "utility" functions specific to the data structure. Where we spec functions we generally put the spec right above the function (like a type annotation).#2017-08-1519:13seancorfieldBut, as noted, if you want your code -- a library -- to be usable by Clojure < 1.9 then you have to put specs in a separate and completely optional namespace.#2017-08-1519:14seancorfieldSee, for example, how clojure.java.jdbc handles this (specs in a separate namespace on their own -- even for functions).#2017-08-1512:24stathissiderishttps://gist.github.com/stathissideris/8d892c78c76b6d615ced53004ea32840#2017-08-1512:24stathissiderisMake clojure.spec behave more like schema in terms of keys strictness (no unknown keys allowed)#2017-08-1512:31stathissiderisdisclaimer: untested, most likely buggy#2017-08-1512:36gfredericksschpec has excl-keys: https://github.com/gfredericks/schpec#things-it-has#2017-08-1514:03stathissiderissure, but I wanted to be able to be strict conditionally#2017-08-1514:47gfredericksoh sorry, I didn't look closely#2017-08-1512:41hmaurerRe-iterating my earlier question, does anyone have a big objection to defining specs at runtime (with eval), based, .e.g, on the appā€™s database schema? I am a beginner so I am not sure if it is ā€œgood taste/practiceā€ in the clojure world#2017-08-1513:27mishait possible, but keep in mind, you cannot yet retract specs from registry as of now @hmaurer#2017-08-1513:28mishaso if your specs will change during application run time - make sure you migrate those appropriately#2017-08-1513:32Alex Miller (Clojure team)well, you can if you reach in and munge the registry atom :) but thatā€™s coming soon.#2017-08-1513:32mishaI meant in official way opieop#2017-08-1513:33Alex Miller (Clojure team)@hmaurer I think dynamic generation of specs is fine#2017-08-1516:29scaturrI am having a hard time understanding s/keys. I keep my specs organized in separate namespaces - i.e something like entity.clj and entity/spec.clj. I have the following used to spec a generic entity map#2017-08-1516:30scaturrI define ::id to make importing it elsewhere easier, and then use it in ::entity/id because its my understanding that is how the :req field in ::entity is able to conform that specific key?#2017-08-1516:30scaturris that correct? am I doing this in a weird way?#2017-08-1516:33scaturrmainly so some other ā€œentity specā€ can merge in ::entity#2017-08-1516:33scaturrwhich implies todos would need an ::entity/id key#2017-08-1519:01didibusI'm not sure I follow your logic. It seems to me your overcomplicating it.#2017-08-1519:15didibusIf conceptually the id of entity is supposed to be the same as the id of todo, the you s/def :some-ns/id. And in your spec for entity you say keys req [:some-ns/id]. And do tje same in your todo spec.#2017-08-1519:39didibusBasically s/keys describes an associative structure which when used with :req says that the structure must have as keys the given list of namespaced keywords. If the keyword has a spec defined, it will be used to validate and conform the value of the key. #2017-08-1615:31scaturrThe todo example does use the :some-ns/id approach#2017-08-1615:32scaturrBut if I were to define (s/def ::entity/id ...) how can I then import that elsewhere?#2017-08-1615:32scaturrThis might just be a misunderstanding of clojure O_o#2017-08-1615:32scaturr(:require [todos.entity.spec :refer [?])#2017-08-1615:33scaturror even if you use :as - since its already namespaced in the spec - is there a way to get at it?#2017-08-1615:33scaturrthank you for the reply by the way šŸ™‚#2017-08-1616:43didibusYou don't need to refer anything, just require it. (:require [todos.entity.spec]) Specs are always defined globally. All that needs to happen is your app must eval the s/def form before you use it. Require will do that. #2017-08-1616:44didibusAlso, ::entity/id doesn't make sense. It would be :entity/id#2017-08-1616:47didibusSo specs are namespaced. They're globally accessible from anywhere after they've been deffed, but they need to be prefixed with a namespace. This helps reduce name clash.#2017-08-1616:49didibusSo if you are in namespace todos.entity.spec and you s/def a spec, and you use the double colon ::, then you created :todos.entity.spec/your-spec#2017-08-1616:51didibusNow if you are in say todos.app namespace, you can require todos.entity.spec and refer to the spec as :todos.entity.spec/your-spec#2017-08-1616:54didibusIf what you want it :entity/id instead of :your-ns/is you have two options#2017-08-1616:56didibusEither define the spec like that, so don't use the double colon, you can (s/def :entity/id int?) for example#2017-08-1616:58didibusThis will be make it less portable though, since the chance of name clash are higher.#2017-08-1617:01didibusOr you can define it prefixed with the full namespace using ::, and then when you use it elsewhere, you can alias the namespace as entity.#2017-08-1617:15didibus`(ns ns1) (s/def ::id int?)` Then in ns2 `(ns ns2 (:require [ns1 :as entity]) (s/def ::user :entity/id)` #2017-08-1617:18didibusAs far as I know, you can't refer a spec yet. But you can kinda manually do it if you want#2017-08-1617:18didibus(s/def ::id :some-other-ns/id)#2017-08-1518:06csmSo I have a cat spec that contains a UUID for one field, and another field is a coll-of maps, and each map has a :guid->UUID; the requirement is that one of the maps contain the UUID. I can express this in a spec, but Iā€™m not sure how to make a generator for such a scenario
#2017-08-1518:29aaelonyis this useful? https://github.com/lbradstreet/cljs-uuid-utils#2017-08-1519:07csmclojure.spec.gen.alpha/uuid is sufficient for generating UUIDs, the problem is ensuring the first UUID appears in the second collection#2017-08-1519:12gfredericks(gen/let [recs (gen/not-empty recs) special-uuid (gen/elements (map :guid recs))] [recs special-uuid])#2017-08-1521:15hmaurerHi. Is there a straightforward way to serialize the result of explain-data to JSON? I am getting an error on trying to serialize spec predicate functions#2017-08-1522:14didibusCan you serialize to edn instead? That generally works better for Clojure.#2017-08-1522:53souenzzocheshire do it without extra settings#2017-08-1610:41hmaurer@U2J4FRT2T it looks like it cannot encode functions (`:pred` key in the spec problems)#2017-08-1610:42hmaurer@U0K064KQV I am consuming it from a JS frontend so EDN does not seem ideal#2017-08-1611:34mishadepends on what you plan to do with predicates.#2017-08-1615:51hmaurer@U051HUZLD not much, I just want a string so I can know why validation failed for a specific field#2017-08-1615:53mishathen (map #(update % :../pred pr-str)) or something, prior to json-serialization#2017-08-1616:00dergutemoritzOr just use s/explain-str instead of s/explain-data#2017-08-1616:01dergutemoritzOr would that be too opaque? šŸ™‚#2017-08-1616:35didibusI always wished cheshire offered custom encoders.#2017-08-1616:56hmaurer@U06GVE6NR too opaque; I need to know which fields contain errors šŸ™‚#2017-08-1616:57hmaurer@U051HUZLD I didnā€™t know the function pr-str; thank you!#2017-08-1621:27didibusQuestion: Is there a way I can find the spec of a data-structure at runtime? I know data-structure are not tagged with their spec, but can you like run a structure through all specs and get the spec it validates back?#2017-08-1621:29gfredericksI bet it'll pass any?#2017-08-1621:29gfredericksthough I guess that's not in the registry#2017-08-1621:29gfredericksbut there could be a lot of keywords in the registry that just have any? as their spec#2017-08-1621:30didibusWell, I guess it would need to be smarter, like return the strictess spec that it validates or something. Not sure how that could work#2017-08-1621:30joshjoneswhat specifically do you want to accomplish?#2017-08-1621:31gfredericksspecs like (s/keys :opt [...]) would validate unrelated maps#2017-08-1621:31didibusI want conditional logic based on the "type" of my data.#2017-08-1621:32joshjoneshave you considered using metadata?#2017-08-1621:32didibusI thought about it, and then I wondered if there's a way to have spec validate that some data-stucture has particular meta on it#2017-08-1621:33joshjonesspec can validate most anything ā€¦ if you can write a function that tests whether or not something is ā€œvalidā€, you can use it with spec#2017-08-1621:34didibusHum, ya I guess that's true, forgot its just predicates. I was looking for some prefab like s/meta, but I can just build my own#2017-08-1621:34gfredericksI'd just make it explicit if you can {:type :something, ...the rest of the data...}#2017-08-1621:35didibusYa, I thought I might need to go down that path. It just forces everything to now operate over the data in a special way though.#2017-08-1621:35didibusI'll try tagging my data with the spec keyword#2017-08-1621:35didibusin their meta#2017-08-1621:35didibussee if that works#2017-08-1621:36joshjonesi think @gfredericks meant not using metadata at all, but just using a map with a :type keyword. it seems to me you donā€™t may not really need spec, or metadata, but maybe just a good olā€™ multimethod#2017-08-1621:36didibusI always worry about using meta like this, because I feel a lot of functions in Clojure don't handle meta very well, like if they modify the data they can return a copy with the meta lost#2017-08-1621:38didibus@joshjones Ya, I got it. But then everything is a map. Say I have a set, I can't union it directly anymore for example, I have to unwrap and re-wrap manually. Maybe there's a monad I can use to help with this though#2017-08-1621:39joshjonesyou donā€™t need to make it a map to use a multimethod to implement different logic (at the function level) based on your data ā€” but i donā€™t have any specifics so iā€™m not quite sure what you really want#2017-08-1621:40joshjonesif you can write a function that can determine what ā€œtypeā€ of data a particular input is, thatā€™s your dispatch function ā€” but again, maybe you donā€™t even want function-level conditional logic, so just guessing#2017-08-1621:46didibusMy problem is I guess the structure and values of my data aren't unique enough. So really I want my data-structures to have a name or a tag of some sort. So say I could distinguish between two vector of ints, because one is an age vector, and one is a height vector.#2017-08-1621:46didibusThen based on if the vector is of age or of height, I want to do something different#2017-08-1621:48didibusadding a metadata tag to my vectors would be one way, but like I said, I know some functions over vectors don't preserve meta. I could wrap my vectors in a map and have {:tag :age :value [12 23 34]}, but then for every operation over the vector I have to first unwrap the vector, pass it to the function expecting a vector, then wrap it back again.#2017-08-1621:50didibusFor some reason, spec always fools me in thinking it can do this, because of the name you give to specs, I assume that data of that spec will have that name on them too, but its not the case.#2017-08-1621:52didibusIt would be great if all data-structure could have a pointer to a spec, like that's what I would want. So then given a vector, you could do (spec [1 2 3]) and it would return the spec for [1 2 3]. Unfortunately, it works the other way around.#2017-08-1621:56seancorfieldYour problem there is that your raw data (a vector of numbers) isn't meaningful on its own...#2017-08-1621:56seancorfield(as you said "...not unique enough...")#2017-08-1621:58seancorfieldMetadata never seems to be a powerful enough solution -- except where it is attached to a Var or to code (that a macro can process). Metadata on values seems outside of the problem space imagined in its design...#2017-08-1622:01didibusHum, that's a good rule of thumb for metadata.#2017-08-1622:04joshjonesbased purely on data alone, how would you know if a vector is a vector of ages, or heights? my guess is, you donā€™t really know, so you need to tag it as such somehow. even if you did what you imagined, (spec [1 2 3]), the spec that youā€™re utilizing has to know somehow what you say the ā€œtypeā€ is ā€¦ the simplest solution is to use a map. thereā€™s overhead in any solution, whether thatā€™s explicitly declaring a type when itā€™s created and tailoring functions to only operate on those types, or whether itā€™s doing what youā€™ve suggested. you canā€™t get around the overhead of differentiating data#2017-08-1622:09didibusI guess I'm not sure what's the best way right now to do that tagging. Basically, I want to add semantic meaning on my data-structures, but I'd like to retain the ability to use all of clojure's functions that operate over that structure. You would think metadata is exactly for this, but not really. Like doing ^:age [1 2 3] would be very natural. So you've got data structured in a vector, and you're giving it semantic meaning by saying it represents the concept of age. So what are other options I have?#2017-08-1622:18joshjoneswhat is the actual source of these vectors? where are they coming from?#2017-08-1622:22joshjonessometimes people will have a function return different shapes/types of data, and the advice is usually to have a function return a consistent shape/type. in the same way, if your data comes from a source which can itself be split such that when you receive data, you already know what it looks like, this is an option. for example, at some source level you must know whether this data represents ages or heights. so, depending on your design and requirements, you may be able to ask for only data that represents ages, and then operate with the knowledge that you have ages. ditto for heights.#2017-08-1622:53seancorfieldIf you have a TaggedValue record that has a tag and a value then you could have functions that accepted functions and turned them into tag-preserving operations (and could verify consistency of tags in multiple arguments too).#2017-08-1719:48Alex Miller (Clojure team)fyi, there is already tagged-literal and tagged-literal? that use clojure.lang.TaggedLiteral. Those instances also respond to :tag and :value keyword lookups.#2017-08-1719:49Alex Miller (Clojure team)and print as tagged literals#2017-08-1721:32seancorfieldResponds to :tag and :form -- I tried :value and got nil. Definitely useful. I did not know about that @U064X3EF3 Thank you!#2017-08-1721:36Alex Miller (Clojure team)itā€™s used by reader conditionals in the case where you read and keep the conditionals, but donā€™t have a tagged literal to represent a tagged literal version. it would ideally be the fallback reader if you didnā€™t find a reader rather than the error you get now - you can install that yourself though.#2017-08-1622:54seancorfieldBut you're straying off into a hundred different types (with one function each) instead of one type with a hundred functions. So it's beginning to sound non-idiomatic.#2017-08-1701:55didibusQuestion: Is there a place on a spec you can add a doc-string, so that it is printed when the spec fails?#2017-08-1701:55didibusI would like to have literate english prose explain what the spec predicates are asserting.#2017-08-1703:19seancorfieldI believe there's a JIRA issue open to consider docstrings for specs -- but it wouldn't be printed (by default) when a spec failed.#2017-08-1704:50mpenetThere s a jira for metadata as well#2017-08-1704:51mpenethttps://dev.clojure.org/jira/browse/CLJ-2194#2017-08-1704:59seancorfieldI think better third-party "explain" functions are the right way to go here.#2017-08-1705:07seancorfield(but I think that each application is going to need its own customization so I don't know how much generic goodness we're going to get here)#2017-08-1711:51mishais there builtin predicate for exceptions?#2017-08-1717:12souenzzoI think that "spec is about what you function can do. You shouldn't specify what your function cant do." (under quote cos it's about what I understand when I see the core team talking about spec)#2017-08-1711:57misha
(s/def ::throwable #(instance? Throwable %))
?
#2017-08-1714:06scaturrI asked this a bit ago but I have yet to stumble upon a solution šŸ™‚ - is there a way to specify a function of arity 1? If i have a function that takes another function as an argument - and that function argument is expected to take a single argument?#2017-08-1714:07scaturr(defn mycool-func [fn1-handler] ...)#2017-08-1714:07scaturrhow would I spec mycool-func?#2017-08-1714:07scaturrIā€™ve just been using fn? until I learn what a computer is#2017-08-1714:31gfrederickssomething like (fspec :args (s/cat any?))#2017-08-1719:45Alex Miller (Clojure team)(s/fspec :args (s/cat :x any?))#2017-08-1714:52scaturr@gfredericks Iā€™ll give that a whirl. Thank you!#2017-08-1715:03samueldevhow can I check if something is a spec?#2017-08-1715:03samueldevI am writing a DSL and one of the values needs to be a clojure.spec#2017-08-1719:43Alex Miller (Clojure team)s/spec?#2017-08-1716:36samueldevI'm opting for (s/get-spec) for my problem#2017-08-1717:58joshmillerIs there an equivalent to s/describe for fdefā€˜ed fns?#2017-08-1718:16souenzzo"(s/describe `rules.core/rules-in-eid)"works for me#2017-08-1718:19joshmillerOh, duh, I didnā€™t quote the fn name so it evaluated it. Thanks!#2017-08-1719:44Alex Miller (Clojure team)also note that (doc rules.core/rules-in-eid) will print the function specs#2017-08-1719:44Alex Miller (Clojure team)as part of the docstring#2017-08-1719:44Alex Miller (Clojure team)doc also knows about specs so (doc ::foo) works for data specs#2017-08-1720:00joshmillerOh awesome, didnā€™t know about that either. Thanks!#2017-08-1721:48souenzzo(s/cat :foo string? :bar (s/spec (s/cat :barbar string?))) is it right? there is a shorter way? I'm not sure if s/spec + s/cat is right ["foo string" ["barbar string"]] << sample#2017-08-1721:50joshjonesšŸ‘:skin-tone-2: ^^#2017-08-1721:52bfabry@souenzzo the second half could be (s/tuple string?) if you are specifically expecting a vector of length 1#2017-08-1721:56souenzzoBut tuple is some kind of anonymous Thinking in docs (in my case, the main propose), give a name for the elements of the tuple is better...#2017-08-1721:58souenzzo(in the real case, it's a tuple 3 elements where each is a different tuple of 3 elements)#2017-08-1722:09bfabryaye, it's very dependent on your use case. with that much nesting I'd say names could definitely be helpful#2017-08-1721:53bfabry(without the need for wrapping in s/spec)#2017-08-1810:08lmergenwhat would be the idiomatic way to override certain spec generators just for my test cases ? it feels a bit icky to be doing this in my source namespaces#2017-08-1810:08lmergeniā€™m thinking of doing a couple of s/with-genā€˜s in my test namespace to update the existing spec definitions, would this work ?#2017-08-1810:11lmergenoh, i just noticed s/gen accepts a gen overrides map#2017-08-1810:12lmergenand that answers my question šŸ™‚#2017-08-1901:48potetmSo, I would like to do something like #"[abc]{0,9}d?" in a spec regular expression.#2017-08-1901:49potetmIs there a recommended way of doing a {0,9} quantifier?#2017-08-1901:50potetmOr do I just have to check the conformed value in a separate predicate?#2017-08-1903:51ghadispec/& is what you want @potetm #2017-08-1903:54potetmThanks @ghadi!#2017-08-1907:35mishahow do I conform strings with spec? I don't? Or should I just split string into some data structure first, and then proceed with spec? an example would be a csv file (let's pretend there is no parser for csv)#2017-08-1909:07seantempestaIs there a tool to generate documentation based on Specā€™ed functions?#2017-08-1914:10pwrflxhi! is it possible to share specs between clojure and clojurescript?#2017-08-1914:41scaturrit sure is!#2017-08-1914:41scaturras long as the underlying predicates work on both platforms#2017-08-1914:42scaturryou can use reader conditionals to include the cljs version#2017-08-1914:42scaturrhttps://github.com/brianium/clean-todos/blob/master/src/todos/core/entity/todo/spec.cljc#L4#2017-08-1915:17pwrflxthanks, I'll look into this.#2017-08-2008:21pwrflxthanks again, the example in your project worked nicely for me!#2017-08-2221:51scaturrglad it helped šŸ™‚#2017-08-1922:18didibusIs there a variant of s/def that also returns the spec?#2017-08-1922:47tbaldridgeif it did the spec returned would have the same name that you just gave it....so I'm not sure how that helps much#2017-08-1922:48tbaldridge(s/def ::foo ...) would just return ::foo#2017-08-1922:54didibusI'm thinking what get-spec returns#2017-08-2008:41misha@didibus s/def already returns spec kw#2017-08-2008:42misha(s/def :foo/bar int?) => :foo/bar#2017-08-2016:17souenzzoHow to fdef a multimethod? I will need my-mm and my-mm', right? Is there some standard pattern? namming pattern?#2017-08-2020:33didibus@misha Hum, I swear I thought I had tried it, oh well, you're right#2017-08-2100:45Alex Miller (Clojure team)@souenzzo multimethods are not currently support with spec fdef#2017-08-2113:31mgrbyte@alexmiller Does ā˜ļø also hold for protocols? currently trying to spec a protocol fn that dispatches on type with s/or and not succeeding...#2017-08-2113:37ghadii think so @mgrbyte#2017-08-2114:17Alex Miller (Clojure team)yes#2017-08-2114:18Alex Miller (Clojure team)functions that canā€™t be instrumented right now include: multimethods, protocols, inline functions, and primitive type hinted functions#2017-08-2114:18Alex Miller (Clojure team)some of those are fixable, some arenā€™t (easily)#2017-08-2115:25mgrbyteattempting to spec a fn that has one argument, but can accept either a string or keyword, my attempt:
(s/fdef species->ident
        :args (s/or :string-val string? :keyword-val keyword?)
        :ret keyword?)
(defn species->ident [s-or-kw]
  (-species->ident s-or-kw))
(clojure.spec.test.alpha/check 'species->ident) gives the error (snipped):
:failure #error {
 :cause "Don't know how to create ISeq from: clojure.lang.Keyword"
 :via
 [{:type java.lang.IllegalArgumentException
   :message "Don't know how to create ISeq from: clojure.lang.Keyword"
   :at [clojure.lang.RT seqFrom "RT.java" 547]}]
I think I'm probably using the wrong spec for args; any pointers please?
#2017-08-2115:25mgrbyte(assuming it's safe to spec a regular fn that delegates to a protocol fn)#2017-08-2115:37andrewmcveighYou'll need something like
(s/fdef species->ident
        :args (s/cat :arg1 (s/or :string-val string? :keyword-val keyword?))
        :ret keyword?)
#2017-08-2115:37andrewmcveigh:args will always be a sequence#2017-08-2115:39andrewmcveigh@mgrbyte ^^#2017-08-2210:11mgrbyte@andrewmcveigh thanks!#2017-08-2210:58cddrIs there any way to leverage spec while writing "interpreters/transpilers" like hiccup which seem to be typically implemented by writing a defmethod for each operator. One disadvantage I've noticed about having these things as data is that when authoring the data, you don't get niceties like editor completion that you would get if it was actual code.#2017-08-2213:30cddrOh. I see this was discussed literally yesterday. Sorry. Are multi-methods one of the cases that aren't easily fixable?#2017-08-2215:08Alex Miller (Clojure team)I think itā€™s possible to make them work, but havenā€™t worked on doing so.#2017-08-2407:35cddrIt occurred to me today that for my use-case, I think I can work around it with a macro that expands to a (defmethod, defn) pair and then write fspecs against the defn.#2017-08-2414:07Alex Miller (Clojure team)yes, that should work#2017-08-2214:19yonatanelIs it customary to spec string length with coll-of characters with :count?#2017-08-2215:09Alex Miller (Clojure team)In most cases, I wouldnā€™t use coll-of for strings as that implies anything that is a sequential view of characters (lists, vectors, seqs)#2017-08-2215:09Alex Miller (Clojure team)prob better to do something like (s/and string? #(= 3 (count %)))#2017-08-2218:27souenzzo@alexmiller is almost possible* specify strings/routing with spec For example: (s/or :nums (s/coll-of #{"1" "2" "3" "4"}) :not-nums (s/coll-of #{"a" "e" "i" "o" "u"})) * with almost possible I mean: you can do with collections of chars. There is plans to do (s/string string-spec) or something like?#2017-08-2218:29Alex Miller (Clojure team)no plans currently#2017-08-2218:30Alex Miller (Clojure team)we already have string regex matching - use that#2017-08-2218:30Alex Miller (Clojure team)specā€™s regex stuff will never be anywhere near as fast or as full-featured as the excellent regex engine already built into java#2017-08-2218:31Alex Miller (Clojure team)if you want gen, Garyā€™s regex stuff in test.chuck is pretty cool#2017-08-2219:11seancorfieldYeah, a big +1 for test.chuck's regex generator -- we love that!#2017-08-2220:16souenzzoI thought about using the specifications to do string routing. Trying to do something better than bidi, I realized spec would do that. Bidi gives you a data/dsl form do describe strings [[:foo :bar :my-path1] [:foo :my-path0]] Then, bidi "conform" this string and returns the conformed [:my-path1 {:foo "username" :bar "project-name"}] It's very close to spec (s/or :my-path (s/cat :foo string-wo-bar? :bar string-wo-bar?))...#2017-08-2220:24dealyHi, I'm just getting started w/spec. I just added clojure 1.9 to my project.clj and immediately get a huge spec error at startup. Is it usually difficult to upgrade a project from 1.8 to 1.9?#2017-08-2220:26hiredmankind of hard to say, in general, anything that spec complains about 1.9 is also broken in 1.8, 1.8 just isn't checking#2017-08-2220:29seancorfield@dealy One place to start looking is https://dev.clojure.org/display/design/Errors+found+with+core+specs which lists libraries that clojure.spec found bugs in, most of which have been fixed/updated.#2017-08-2220:29dealyIt seems to be failing on my require statement, the error msg doesn't hlep much: In: [2] val: (quote :as) fails at: [:args :exclude :op :spec] predicate: #{:exclude}#2017-08-2220:30dealyit repeats that error several times#2017-08-2220:30dealyits on the first line of the first file it tries to compile#2017-08-2220:30seancorfieldWhat does your require look like?#2017-08-2220:32dealyits about 21 lines, should I post it here?#2017-08-2220:32seancorfieldUse a snippet (via the + button on the left)#2017-08-2220:33hiredmanare you using core.async I vaguely recall there being a gnarly bug that looks something like that if you aren't on the latest or so release#2017-08-2220:33dealyyes using core.async#2017-08-2220:35hiredmanhttps://github.com/juxt/yada/issues/156#2017-08-2220:35seancorfieldInteresting -- core.async isn't listed on that page above so I guess it had been updated/fixed before that page was created -- good catch @hiredman !#2017-08-2220:39souenzzohttps://github.com/pedestal/pedestal/issues/505 Reported this bug, but dont submitted to http://dev.clojure.org#2017-08-2221:02seancorfieldYeah, I suspect a lot of people working with core.async in any way are adding it as an exclusion and pulling in a new version directly.#2017-08-2220:35hiredmanso that is an example of a similar issue, fixed by bumping core.async versions#2017-08-2220:36dealywell that helped some, at least the program went further before spewing errors, I didn't realize that there were gonna be these kinds of problems just upgrading to 1.9 ugh#2017-08-2221:02Alex Miller (Clojure team)1.9 has specs on core that will find errors that were silently ignored in the past. so, itā€™s good! but also frustrating at times. many of the most popular libs have been fixed up long ago.#2017-08-2220:51dealyany idea what is causing this: Error refreshing environment: java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;)V, compiling:(closure.clj:100:1) I'm using cljs 1.9.908#2017-08-2220:57hiredmanyou should check your build tool for conflicting versions of whatever com.google.common comes from#2017-08-2221:11dealyit was guava, upgrading to the latest cljs caused a conflict I guess, its working now#2017-08-2223:43Oliver GeorgeIt might be mild OCD but I find it quite inconvenient to come up with qualified keywords when writing function specs.#2017-08-2223:43Oliver GeorgeI think what I want is a more adhoc/flexible way to create aliases.#2017-08-2223:46Alex Miller (Clojure team)Likely coming in the future (but not 1.9)#2017-08-2223:48Oliver GeorgeYay#2017-08-2315:23souenzzoWaiting to 2.o#2017-08-2317:17Alex Miller (Clojure team)you mean 1.10 :)#2017-08-2223:43Oliver GeorgeFor example:
(s/def ::session (s/keys :req [:oauth2/csrf-token]))
(s/def ::params (s/keys :req-un [::code ::params]))
(s/fdef oauth2-success :args (s/cat :req (s/keys :req-un [::params ::session])))
#2017-08-2223:44Oliver GeorgeThis is a ring handler. If I want to spec several handlers in one namespace the ::params key needs to be unique.#2017-08-2223:44Oliver GeorgeSo I end up wanting qualified keywords with a namespace unique to the function symbol#2017-08-2223:45Oliver Georgee.g. ::oauth-success/params#2017-08-2223:45Oliver GeorgeI can do it long hand. :my-app.handlers.oauth-success/params#2017-08-2223:46Oliver GeorgeIn CLJ I can declare the alias (but not CLJS) and then do ::oauth-success/params but even that is pretty heavy handed.#2017-08-2301:24didibusAm I right in thinking that the generator for (s/keys :req [(or ::a ::b ::c)]) will always return a map with all keys? I think that will leave out potential edge cases when doing generative testing.#2017-08-2302:08Alex Miller (Clojure team)there is a pending enhancement to fix that#2017-08-2302:09Alex Miller (Clojure team)https://dev.clojure.org/jira/browse/CLJ-2046#2017-08-2302:29didibusgreat, thx#2017-08-2303:27didibusQuestion: I've got a spec which is a map-of string? to itself. It works, but sometimes the generator takes like 5 seconds or more to generate a single example. Is there a way I can limit the number of recursive generation it does?#2017-08-2303:28didibus
(s/def :a/b (s/map-of string? (s/or :1 :a/b
                                                               :2 string?)))
#2017-08-2303:31didibusShould have googled it first: https://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec/*recursion-limit*#2017-08-2303:31didibusnvm#2017-08-2310:05martinklepschHey all šŸ™‚#2017-08-2310:06martinklepschIā€™m getting errors when starting my repl: clojure.lang.ExceptionInfo: Unable to resolve spec: :app.spec/thing#2017-08-2310:07martinklepschI require my app.spec namespace in my main namespace so Iā€™m thinking it should be loaded before other namespaces depending on it.#2017-08-2310:11rodDo you have a repo you can link to martinklepsch?#2017-08-2310:11martinklepsch@rod unfortunately not but I may have just found the issue#2017-08-2310:12martinklepschanother namespace except core required app.spec resulting in it (+ some others) being loaded before the spec namespace was loaded#2017-08-2317:20jrychterI just realized that multi-specs use the raw data, not conformed data, for their multimethods. I didn't expect that, but now that I think about it, it seems it's the only way to avoid dependency issues. Is this expected, or am I doing something silly? To clarify what I'm talking about:
(s/def ::type (s/and (s/conformer keyword) keyword?))
(defmulti part-type :type)
(defmethod part-type "meta" [_]
  ...)
The defmethod has to be defined on "meta", not :meta, because that's what the database returns and that's what the value is before conforming.
#2017-08-2317:26Alex Miller (Clojure team)that seems like the expected behavior to me#2017-08-2317:27Alex Miller (Clojure team)multi-spec chooses a spec based on the data#2017-08-2317:27Alex Miller (Clojure team)note that the multimethod does not need to be based on a keyword though, it can be an arbitrary function#2017-08-2317:45jrychterYes, after thinking about this for a moment, I realized that it would be unreasonable to expect anything different. And yet it surprised me, I grew used to thinking of conform as a conversion layer between database representation (JSON in my case) and native Clojure data. I expect the "conversion" to happen first, before the method gets selected.#2017-08-2317:46jrychterI think I read somewhere that s/conform isn't really intended for this role, but I find it amazingly useful.#2017-08-2318:36potetmomg this is amazing: https://github.com/gfredericks/test.chuck#string-from-regex#2017-08-2318:36potetmThanks for the tip @alexmiller!#2017-08-2320:09bbrinckDoes anyone know of work to generate human-readable documentation from specs?#2017-08-2320:28adamfreyit is possible to set the generator of a spec after it's been defined?#2017-08-2320:43Alex Miller (Clojure team)@adamfrey not without re-registering it#2017-08-2320:44Alex Miller (Clojure team)@bbrinck autodoc is used to create the clojure docs and it has rudimentary support for including specs#2017-08-2320:44Alex Miller (Clojure team)there are also one (or maybe more) tools for creating diagrams representing specs#2017-08-2320:45Alex Miller (Clojure team)https://github.com/jebberjeb/specviz#2017-08-2320:45bbrinck@alexmiller Thanks!#2017-08-2320:46jebberjeb@bbrinck PRs welcome there šŸ™‚#2017-08-2415:21jpmonettashi everybody, what's the way to spec something is a promise of some kind#2017-08-2415:35Alex Miller (Clojure team)not really any good way right now#2017-08-2415:36Alex Miller (Clojure team)I would focus on specā€™ing the place where you deliver the value and receive the value#2017-08-2415:40joshjonesyou could do
(s/def ::promise #(= (class %) (class (promise))))
?
#2017-08-2415:42joshjonesyou could put the class of the promise elsewhere if you wanted to avoid creating a promise each time you ran the predicate#2017-08-2415:47Alex Miller (Clojure team)Iā€™m not sure thereā€™s much value in checking just that something is a promise. I presume what is of interest is that itā€™s a promise that returns something that matches a spec#2017-08-2415:47Alex Miller (Clojure team)however, checking for that would require you to block waiting for the promise to be delivered#2017-08-2415:48Alex Miller (Clojure team)so I think you instead want to validate the spec at the point where you are already retrieving the value#2017-08-2415:48Alex Miller (Clojure team)or where you are delivering the value#2017-08-2416:09jpmonettasthanks @joshjones @alexmiller#2017-08-2416:09jpmonettasyeah I'm trying to spec a function return that returns a promise with an int?#2017-08-2416:11jpmonettassince it's a :ret I'm not worried about instrument blocking on deref#2017-08-2416:11jpmonettasI'm more interested in the doc aspect#2017-08-2416:35seancorfieldSeveral times I've wanted to be able to spec "this should be an atom containing an X" so I feel the lack @jpmonettas -- but as @alexmiller says, a lot of the times where you would want this, spec would need to deref the container to check the value inside and that's not the correct semantics (it would block).#2017-08-2417:12Alex Miller (Clojure team)No, that's different. Atom deref is not blocking#2017-08-2417:16Alex Miller (Clojure team)We might add atom-of eventually#2017-08-2417:17seancorfieldI meant more in the general "derefable" situation -- but you're right that each type of derefable is different.#2017-08-2417:19Alex Miller (Clojure team)All of the Clojure reference types do nonblocking read#2017-08-2417:23seancorfieldGood point. Future/Promise are the outliers.#2017-08-2417:27gfredericksand delay#2017-08-2417:36Alex Miller (Clojure team)none of Future/Promise/delay are reference types#2017-08-2417:39Alex Miller (Clojure team)that is, by ā€œreference typeā€ I mean, IRef (not IDeref)#2017-08-2416:36seancorfieldIt would make me want to have the function return int, and instead call it in a future (which has the advantage that the caller controls both the wrapping and the dereference).#2017-08-2416:36seancorfield(since you don't mind the checker blocking on a deref to check the value anyway)#2017-08-2416:43mpenetYou can put a validator on the atom with a valid? check i guess#2017-08-2416:43mpenetWont help with gen tho#2017-08-2416:59mpenetGolang is interesting for chan args, you can say "this arg is a chan of int that can only be take!en in that scope" etc#2017-08-2417:00mpenetChan args have a concept of "direction"#2017-08-2417:18Alex Miller (Clojure team)As do core.async chans via the port protocols#2017-08-2417:18mpenetYes it s very similar#2017-08-2417:19mpenetxforms on chans might make specing them more tricky tho#2017-08-2505:32thedavidmeisterwhen i use spec/exercise for things like string? and pos-int? i mostly get blank or 1 character strings and numbers 1 and 2#2017-08-2505:32thedavidmeisteris there a way to get it to "try harder"#2017-08-2505:33thedavidmeisterand give me longer strings more often, with more variety, like emojis and non english alphanumeric characteres#2017-08-2505:33thedavidmeisterand give me random ints anywhere in the full range of available positive integers?#2017-08-2505:34thedavidmeisteri really like the idea of using spec to help me generate tests, but it doesn't give me a lot of confidence to know that most of my tests are just passing "" and 1#2017-08-2505:41thedavidmeisteroh hmmm#2017-08-2505:42thedavidmeisteri just realised that if i (spec/exercise string? 1) i almost always get ""#2017-08-2505:42thedavidmeisterbut if i (spec/exercise string? 100) then the later strings are much longer#2017-08-2505:44thedavidmeisterso i could do something like (ffirst (shuffle (spec/exercise pos-int? 100))) but this seems inefficient...#2017-08-2505:48seancorfield@thedavidmeister I'm a bit puzzled about how you're running tests here?#2017-08-2505:48seancorfieldAre you using spec.test's check on functions? Or test.check with properties etc?#2017-08-2505:53thedavidmeisteruh, nothing like that atm#2017-08-2505:53thedavidmeisteri've just been working on creating specs for my existing codebase#2017-08-2505:54thedavidmeisterand i've got existing tests that i've handrolled examples for#2017-08-2505:54thedavidmeisterbut i thought i could use spec/exercise as a "drop in" replacement for my examples#2017-08-2505:54seancorfieldWe have a couple of places where we do (rand-nth (map first (s/exercise ::my-spec 100))) which is an equivalent to what you have so it's a reasonable approach for getting a specific random piece of test data.#2017-08-2505:56thedavidmeisterok cool#2017-08-2505:56thedavidmeisterjust checking that i'm not crazy šŸ™‚#2017-08-2505:56seancorfieldI suspect rand-nth is going to be faster than first of shuffle (and (first (rand-nth ...))` is probably the fastest).#2017-08-2505:56thedavidmeisteris there a way to get it to be lazy?#2017-08-2505:56seancorfieldLazy how? You only want one value and you want a random one out of 100...#2017-08-2505:57thedavidmeisteryes but it's clearly doing something different for the 100th to the 1st item that it generates#2017-08-2505:57thedavidmeisteri want the logic of the 100th iteration#2017-08-2505:57thedavidmeisterbut not the 99 values that come before it#2017-08-2505:58seancorfieldSince we're talking about tests I wouldn't be too concerned about efficiency.#2017-08-2505:58thedavidmeisteri know...#2017-08-2505:58thedavidmeisterit just seems so wasteful#2017-08-2505:58thedavidmeisteri'm not normally "that guy" but...#2017-08-2506:00seancorfieldFWIW, (first (rand-nth (s/exercise string? 100))) takes between 6 and 30ms on my low-powered laptop#2017-08-2506:00thedavidmeisterok#2017-08-2506:00thedavidmeisterwell the other half of my q#2017-08-2506:00thedavidmeisteris there a way to get string? to be less alphanumeric-y?#2017-08-2506:01seancorfieldWhat do you want it to be?#2017-08-2506:01thedavidmeisteri sort of expected utf-8#2017-08-2506:01thedavidmeisterso :poop: emojis and japanese kanji, etc.#2017-08-2506:01seancorfieldSome of the built-in generators are a bit conservative. Take a look at test.chuck and its regex generator.#2017-08-2506:02seancorfieldWe have a crazy email address regex and use test.chuck to produce random email addresses and it generates absolutely wild UTF-8 strings šŸ™‚#2017-08-2506:03thedavidmeisteroh yeah string-from-regex looks cool#2017-08-2506:04seancorfieldWe also have a wild password spec (with complex rules about UTF-8 character classes and length etc), but for testing we override to use a much simpler ASCII regex generator.#2017-08-2506:04thedavidmeisterso how do i plug that into spec?#2017-08-2506:04seancorfieldWe're testing different things there.#2017-08-2506:04seancorfields/exercise lets you specify overrides, or you can use s/with-gen to provide a custom generator for any spec.#2017-08-2506:05thedavidmeisterok cool, so can i do like (s/with-gen string? (partial string-from-regex #".*")#2017-08-2506:05thedavidmeisteror is that not right?#2017-08-2506:05seancorfieldIt needs to be a zero arity function that returns a generator.#2017-08-2506:06gklijslike: (s/def :nl.klijs.person/first-name (let [re #ā€œ^[A-Z]{1}[a-z]{2,20}ā€œ] (s/spec string? :gen #(sg/string-generator re))))#2017-08-2506:07seancorfieldan example from our code
(s/def ::date-of-birth (s/with-gen age-18-120?
                         (fn [] (s/gen (s/inst-in (wdt/years-ago 120)
                                                  (wdt/years-ago 18))))))
#2017-08-2506:09seancorfieldWe wrap test.chuck so we don't drag it in for production code but it can come in during testing so we have
(defn fn-string-from-regex
  "Return a function that produces a generator for the given
  regular expression string."
  [regex]
  (fn []
    (require '[com.gfredericks.test.chuck.generators :as xgen])
    (let [string-from-regex (resolve 'xgen/string-from-regex)]
      (string-from-regex regex))))
#2017-08-2506:09seancorfieldand then
(s/def ::password (s/with-gen (s/and string? password-valid?)
                    (wgen/fn-string-from-regex #"[a-zA-Z0-9 
#2017-08-2506:09seancorfield(`password-valid?` is a gnarly predicate)#2017-08-2506:10thedavidmeisterok cool#2017-08-2506:10thedavidmeisterand then once you've done that, exercise will know what to do with ::password#2017-08-2506:12seancorfieldYup, it will generate based on that regex.#2017-08-2506:12thedavidmeisterthat sounds much more thorough than alphanumeric strings#2017-08-2506:12seancorfieldWhich will satisfy our rules (`password-valid?`) but much more easily -- and still readable.#2017-08-2506:14thedavidmeistersoz, i have another question while i go through all this#2017-08-2506:14thedavidmeisteris there a predicate for date/times?#2017-08-2506:14gklijsI needed to use it to be able to even generate maps containing an email#2017-08-2506:15thedavidmeister@gklijs yeah i'm running into limitations with the default string? generator pretty fast too#2017-08-2506:22seancorfield@thedavidmeister We have a set of specs and generators for date/times but it's a fair bit of code because we actually need to coerce from strings to dates, and we want to generate formatted dates.
#2017-08-2506:23seancorfieldBut for basic stuff you can just use inst?#2017-08-2506:24thedavidmeisteroh yeah, i'm slowly working through moving between js dates, goog dates, joda dates and iso8601 strings#2017-08-2506:24thedavidmeisterjust wanted something basic for now though šŸ™‚#2017-08-2506:24thedavidmeisterah inst? looks great#2017-08-2506:25thedavidmeisteralthough hmmm#2017-08-2506:25thedavidmeisterit seems to strongly prefer 1970-01-01#2017-08-2506:26thedavidmeistereven if i do (first (rand-nth (spec/exercise inst? 1000))) it rarely moves away from 1970#2017-08-2506:27seancorfieldYeah, the inst? generator is pretty conservative.#2017-08-2506:27thedavidmeisterhmmm#2017-08-2506:27thedavidmeisterknow of anything like that regex lib?#2017-08-2506:28thedavidmeisteri feel like even using a random int as a timestamp would be better šŸ˜•#2017-08-2506:29gklijsmaybe with dates you want one based on the current date and add/substract some random year/month/seconds etc?#2017-08-2506:30seancorfieldHence our age range spec / generator šŸ™‚#2017-08-2506:31thedavidmeisteryeah#2017-08-2506:31thedavidmeisteri think i'm going to go for a jog and think about how i can adapt to my stuff šŸ™‚#2017-08-2506:33seancorfieldOne thing I've found important with clojure.spec is to be careful not to over-specify things and to override generators to produce more "sane" data (for your problem domain) -- generating just a small subset of possible conforming values sometimes.#2017-08-2506:34thedavidmeisteri think that will be ok for me atm#2017-08-2506:34thedavidmeisteras i'm replacing hard-coded values in tests anyway#2017-08-2506:34thedavidmeisterit will be an incremental process#2017-08-2506:34thedavidmeisteras i gradually introduce more complexity of what is specced#2017-08-2509:19thegeez@thedavidmeister spec generator have the notion of a size parameter (100 here). However, this is not a minimum length: (clojure.test.check.generators/generate (clojure.spec.alpha/gen string?) 100) and clojure.test.check.generators/resize#2017-08-2509:21thedavidmeister@thegeez wow that is great#2017-08-2509:21thedavidmeisterthanks!#2017-08-2515:47martinklepschhow do you express that all keys of a map should be keywords? I want to spec a structure that looks like this:
{vec {keyword {string {:some-known-key ::x
                         :another-known-key ::y}}}}
#2017-08-2515:49seancorfield@martinklepsch map-of?#2017-08-2515:50seancorfield(map-of keyword? ::value-spec)#2017-08-2515:51martinklepsch@seancorfield thanks, Iā€™ll look into map-of#2017-08-2518:19ikitommi@martinklepsch map-of is the right answer, but your pseudo looks much like the thing that the data-specs are eating šŸ˜‰ (and generating map-ofs & friends)
(require '[clojure.spec.alpha :as s])
(require '[spec-tools.data-spec :as ds])

(s/def ::x string?)
(s/def ::y int?)

(let [martin {vector? {keyword? {string? {:some-known-key ::x
                                          :another-known-key ::y}}}}
      value {[1 2 3] {:a {"b" {:some-known-key "nice"
                               :another-known-key 18}}}}
      spec (ds/spec ::martin martin)]
  (s/valid? spec value))
;; true
#2017-08-2518:24martinklepsch@ikitommi ah that indeed seems like a nice utility!#2017-08-2519:46eraserhdSo I just discovered that (s/def :foo/bar :baz/quux) doesn't work if :baz/quux hasn't already been defined. Is this new?#2017-08-2519:46eraserhdIs it just s/keys that defers lookup?#2017-08-2519:48eraserhdAnd if so, does that mean that I can't redefine :baz/quux later?#2017-08-2519:52Alex Miller (Clojure team)almost everything defers lookup. this is a known (and unresolved) issue.#2017-08-2519:53eraserhdIf I wrap it with s/spec, will it defer?#2017-08-2519:54Alex Miller (Clojure team)maybe? try it#2017-08-2519:55eraserhdapparently... more testing in half an hour or so#2017-08-2622:46souenzzothere is some wayt to (s/def :req-un [:foo/bar :as :new-name])?#2017-08-2623:55Alex Miller (Clojure team)no#2017-08-2717:00hmaurerHi! Is there a way to get only the keys specified in a spec (req and opt) and nothing else after validating a map? And to do so with nested maps too? #2017-08-2717:00hmaurerI expected conform to do this but it doesn't appear to #2017-08-2721:16hmaurerthe spec-tools package seems to offer a solution to my problem#2017-08-2718:36ikitommiHi. The s/conformer seems to take the optional unf function as a second argument for unforming. It defaults to nil currently. Would an identity be a better default?#2017-08-2718:38ikitommie.g. this would not fail by default:#2017-08-2800:35Alex Miller (Clojure team)yes, it would still fail by giving you the wrong result#2017-08-2800:36Alex Miller (Clojure team)given that conformers are used to transform the input, using identity is virtually always as wrong as nil. much better to fail and tell you how to fix imo#2017-08-2914:01ikitommithanks.#2017-08-2718:38ikitommi
(require '[clojure.spec.alpha :as s])

(s/def ::spec (s/conformer str))

(s/conform ::spec 1)
; => "1"

(s/unform ::spec (s/conform ::spec 1))
; CompilerException java.lang.IllegalStateException: no unform fn for conformer
#2017-08-2721:45hmaurerAnybody familiar with spec-tools here?#2017-08-2807:21ikitommi@U5ZAJ15P0 I am, one of developers of the lib.#2017-08-2819:28hmaurer@U055NJ5CC Hi! I wanted to ask if there is a utility function to walk a (deeply nested) map together with its associated spec#2017-08-2819:28hmaurerI just realised itā€™s precisely what has been discussed here https://github.com/metosin/spec-tools/issues/65#2017-08-2819:29hmaurerdoes such a function exist now?#2017-08-2905:01ikitommi@U5ZAJ15P0 Not yet. Just commented on the issue. If you have ideas, please share them on the issue. Contributions welcome too#2017-08-2803:21danielcomptonIs there a way to spec a nested map that dispatches on type? e.g.
(s/def :app/command (s/keys :req [:command/id
                              :command/params
                              :request/id
                              :idempotency/id]))

(defmulti command-params :command/id)
(s/def :command/params (s/multi-spec command-params :command/id))

(defmethod command-params :test/command [_]
  (s/keys :req [:material/id]))

;; This is the kind of map shape I want

{:command/id     :test/command
 :request/id     (random-uuid)
 :idempotency/id (random-uuid)
 :command/params {:test/id (random-uuid)}}
#2017-08-2803:22danielcomptonI thought I could do this with a multi-spec, but I'm not sure how to define a multi-spec where the data is at one level, and the tag is at the parent level#2017-08-2807:28ikitommiIdea: s/conform could be changed into s/walk that could be parametrised to do either :conform (as now) or :coerce (like conform, but would not return the branch-information, e.g. same results as with conform + unform).#2017-08-2807:29ikitommiI think this would yield much better perf, which matters at runtime.#2017-08-2813:38scaturrIs there any sort of consensus on where to put specs? Separate ns? Same ns as things being specced?#2017-08-2815:41souenzzo+subscribe+#2017-08-2815:59seancorfieldIt Depends(tm).#2017-08-2816:00seancorfieldData specs can usually be in their own ns, possibly with a few predicate functions or closely related helper functions.#2017-08-2816:01seancorfieldFunction specs probably should be in the same ns as the functions they are spec'ing -- unless you need to support Clojure 1.8 (or earlier) as well as Clojure 1.9, such as for a library, in which case they must go in a separate, optional ns, that only 1.9 users will load.#2017-08-2816:38scaturrthat is super helpful. thank you!#2017-08-2813:39scaturrTooling makes me feel like the same ns might be more convenient - not necessarily better?#2017-08-2813:40scaturrfor instance ciderā€™s doc command doesnā€™t work unless you explicitly have the spec required#2017-08-2813:40scaturrĀÆ\(惄)/ĀÆ#2017-08-2817:49hmaureris there a way to ā€œoverrideā€ an already defined spec?#2017-08-2817:49hmaurer(with s/def)#2017-08-2817:50hmaureritā€™s a no-op for me if the spec is already defined#2017-08-2818:37seancorfieldRemember that s/def will only update the spec -- not all the other specs that have been defined in terms of that spec.#2017-08-2900:48Alex Miller (Clojure team)yeah, this should work (once you re-def the other specs)#2017-08-2817:58hmaurerFound my answer: hacky (reset! @#'spec/registry-ref {})#2017-08-2818:00hmaureractually that seems to break thingsā€¦#2017-08-2820:17joshjonesiā€™m not shocked#2017-08-2900:48Alex Miller (Clojure team)there is a ticket on the way such that (s/def ::foo nil) will be sufficient to remove a spec#2017-08-2910:53hmaurer@alexmiller thank you!#2017-08-2910:55hmaurer@ikitommi I just started working with spec-tools but I am finding it a bit hard to write functions dispatching on (s/form spec) as most are wrapped in spec-tools.core/spec. Am I doing something wrong?#2017-08-2910:57hmaurer@ikitommi ah, I just looked at https://github.com/metosin/spec-tools/blob/master/src/spec_tools/visitor.cljc and it seems you are handling that case by extracting the inner spec#2017-08-2911:24ikitommi@U5ZAJ15P0 wrapping into spec records is unfortunate, there is https://dev.clojure.org/jira/browse/CLJ-2116 to enable runtime coercion natively. But, yes the original spec is found in :spec#2017-08-2912:19hmaurer@ikitommi thank you! Iā€™ll take a look. Overall great work on spec-tools by the way; it appears to solve a lot of the issues I had with clojure.spec šŸ™‚#2017-08-2912:57hmaurerOut of curiosity, what is the consensus regarding issues described here @ikitommi @alexmiller ? https://dev.clojure.org/jira/browse/CLJ-2116. More specifically @alexmiller , you mention > While I think some interesting problems are described in the post, I donā€™t agree with most of the approaches being taken there. Do you have an alternative approach to suggest? I am a beginner with clojure/clojure.spec but I did find that spec-tools answered some real needs of mine (e.g. strip extra keys when conforming a map)#2017-08-2913:42Alex Miller (Clojure team)if you want to transform data, then transform data. why does that need to be in spec?#2017-08-2913:42Alex Miller (Clojure team)if the selective conforming only happens at the end, why does it need to be in spec?#2017-08-2913:46Alex Miller (Clojure team)the purpose of conforming is not transformation or coercion. itā€™s to tell you how a value conforms to a spec,#2017-08-2913:58hmaurerAh I see. So these sort of tools should be built on top of spec, not as a part of it#2017-08-2913:59hmaurerHow would you approach annotating a spec with additional data that can be used later on by utility functions?#2017-08-2913:59hmaurer(what spec-tools does by wrapping specs in a Spec record)#2017-08-2914:53Alex Miller (Clojure team)I would like to have arbitrary meta on specs and we have some tickets around that#2017-08-2914:54Alex Miller (Clojure team)but barring that, specs are named with a fully-qualified name - anyone can make a registry with those keys to store arbitrary stuff#2017-08-2914:59hmaurer@alexmiller good point; I guess those things will come more naturally to me as I get familiar with specs šŸ™‚#2017-08-2922:08tap@alexmiller Then this is a wrong use case example for conforming? Transforming some numbers to fizz, buzz, fizzbuzz. https://gist.github.com/stuarthalloway/01a2b7233b1285a8b43dfc206ba0036e#2017-08-2922:33Alex Miller (Clojure team)Fizzbuzz is not a use case#2017-08-3001:21tapDo you mean itā€™s a fun little problem which we can use whatever we want, right? Itā€™s good to know for me because I misunderstood the purpose of conformer by this. Already did a csv validating and parsing using conform and additional conformers#2017-08-2916:40seancorfieldA use case we've found very valuable is using spec for conforming REST API parameters and form parameters -- which all start off as strings but we need numbers or Boolean etc. For those specs, we coerce string input to conformed values. Then we have a single step to s/conform the input map to a validated parameters map.#2017-08-2916:41seancorfieldSo at the core, we'll have things like (try (Long/parseLong v) (catch Exception _ ::s/invalid)) as a named spec (predicate).#2017-08-2916:43seancorfieldI would agree with @alexmiller that coercion/transformation beyond that level is probably "doing it wrong".#2017-08-2916:46ikitommi@seancorfield So you allow clients to send numbers as strings over JSON too?#2017-08-2916:47ikitommi(and over EDN or Transit too)#2017-08-2916:54seancorfieldWe accepted form-encoded POST bodies so it isn't always JSON.#2017-08-2916:54hmaurer@seancorfield well, a simple case which brought me to try spec-tools was having the ability to strip all keys from a map that were not part of the spec#2017-08-2916:55hmaurerI assume that, were I to do this myself, I would have to write a function to traverse my spec together with the data and strip keys appropriately?#2017-08-2916:57seancorfieldWhy do you need to strip other keys? The only case I can think of is if you send the full hash map to a 3rd party that would object to the extra keys -- and it's safer to explicitly build the map you need at those points.#2017-08-2916:58seancorfieldIf your code never uses :x in a map, why do you care that you're passed :x in the first place, is my question.#2017-08-2916:58hmaurerprecisely that; I am transacting the data to Datomic#2017-08-2916:58seancorfieldBut you'd be building a namespace-qualified map specific to your schema at that point -- I wouldn't imagine the namespace-qualifier would be the same as your JSON input?#2017-08-2916:59hmaurerno, but I have a step which namespace-qualifies my JSON input#2017-08-2917:01ikitommiI think it's a good default to strip extra keys when reading external input. Like rest/http parameters from client#2017-08-2918:17souenzzoBit off, but I have a experiment that I use for example /user/%5B%3Auser%2Fuuid%20%23uuid%22123%22%5D (encoded form of /user/[:user/uuid #uuid"123"]) to avoid transformation problems (simple edn/read-string transform types). Not sure if it's a good solution/it's sacale, but it's working well on my tests.#2017-08-3102:11tjscollinsI think I'm slowly losing my mind trying to get my function specs to work right.
(s/fdef example-fn
    :args string?
    :ret string?)

(defn example-fn [s] s)
This does not work if I try stest/check example-fn. Why? I'm clearly missing something. I've been re-reading the guide for two days, but something's not clicking, and I can't see where why s/fdef's are going wrong.
#2017-08-3102:20seancorfield@tjscollins :args is always a sequence of argument specs so you want (s/cat :s string?) there.#2017-08-3102:25tjscollinsHow does s/cat work? Do the keys have to match the variable names, or are they just used as informational labels in the output of s/explain?#2017-08-3102:28seancorfieldHave a read of this https://clojure.org/guides/spec#_spec_ing_functions -- it has some examples that should help.#2017-08-3102:29tjscollinsYeah, I've read through that repeatedly. It's just not making sense to me how s/cat and s/alt work.#2017-08-3102:29seancorfieldThe keywords in s/cat don't have to match the argument names but it's common convention that they do (otherwise your error messages will be less than helpful).#2017-08-3102:30tjscollinsSo what is actually happening when the args are checked against s/cat? Do the args pass if they match any spec that's "s/cat"ed together? Match every spec?#2017-08-3102:31seancorfields/cat is a "sequence regex" so it matches the sequence of arguments passed into the function#2017-08-3102:31tjscollinsThe guide calls it a regex op, but I don't know what that means in this context#2017-08-3102:31tjscollinsSo the first pair passed to s/cat has to match the first arg?#2017-08-3102:31tjscollinsAnd the second pair matches the second arg?#2017-08-3102:32seancorfieldSort of... consider
(s/fdef clojure.core/declare
    :args (s/cat :names (s/* simple-symbol?))
    :ret any?)
#2017-08-3102:32seancorfieldThat matches zero or more simple symbols as an argument sequence and labels that sequence as :names#2017-08-3102:33seancorfieldSo in (declare foo bar quux), (s/* simple-symbol?) will match the three symbols and :names will be (foo bar quux)#2017-08-3102:33seancorfieldIt's quite literally "regex" for sequences (instead of strings).#2017-08-3102:34tjscollinsSo something like (s/cat :s string? :i int?) would match both ("1" 1) and (1 "1")?#2017-08-3102:34seancorfieldNo, it's sequential.#2017-08-3102:34seancorfieldIt would only match "1" 1#2017-08-3102:35seancorfieldHow familiar are you with regex for strings?#2017-08-3102:35tjscollinsAlright, would (s/cat :s (s/* string?) :i int?) match both ("1" 1) and ("1" "1" 1)?#2017-08-3102:35seancorfieldYes, I believe it should#2017-08-3102:35tjscollinsYeah, I'm familiar with string regexes#2017-08-3102:36tjscollinsOkay, that makes more sense now#2017-08-3102:37seancorfieldYou can try it out directly in the REPL, without worrying about arguments: (s/conform (s/cat :s (s/* string?) :i int?) ["1" "1" 1])#2017-08-3102:37seancorfieldand (s/conform (s/cat :s (s/* string?) :i int?) [1]) => {:i 1} because there were no strings at the beginning#2017-08-3102:39tjscollinsAlright that makes a lot more sense now. Is s/alt basically s/or for a sequence?#2017-08-3102:43seancorfieldYup.#2017-08-3102:44tjscollinsThat helps a lot. Thanks.#2017-08-3102:44seancorfieldAlso from that guide:
(s/cat :prop string? :val  (s/alt :s string? :b boolean?))
#2017-08-3102:47tjscollinsFirst element has to be a string, second can be boolean or string, right?#2017-08-3113:48favilaI'm trying to spec a sequence where there is a fixed predicate/spec based on position, but the sequence may omit things at the end; and I'd like it to conform to a flat map, one key per position, nil or no-key for the omitted items#2017-08-3113:50favila(s/conform ??? [1 ""]) => {:pos1 1 :pos2 ""}#2017-08-3113:50favila(s/conform ??? [1 "" :kw]) => {:pos1 1 :pos2 "" :pos3 :kw}#2017-08-3113:51favila(s/conform ??? [1 :kw]) => ::s/invalid#2017-08-3113:51favilaI can't use cat plus nested s/? because that backtracks (last example would be accepted)#2017-08-3113:53favilathe best I can think of is a linked-list type structure of nested s/or that a conformer flattens#2017-08-3113:53favilaany other ideas?#2017-08-3113:58Alex Miller (Clojure team)s/or of s/tuple ?#2017-08-3113:59favilas/tuple won't work because variable-width#2017-08-3113:59Alex Miller (Clojure team)Iā€™m saying you could s/or multiple s/tuple together#2017-08-3113:59Alex Miller (Clojure team)itā€™s unclear how much variability there is here#2017-08-3114:00Alex Miller (Clojure team)I guess maybe s/alt of s/cat would be better for your conformed value#2017-08-3114:00Alex Miller (Clojure team)I think Iā€™d say: donā€™t fight the conformed value. write the right spec, then transform the conformed value to what you want.#2017-08-3114:01Alex Miller (Clojure team)always come back to: how can I state the truth about the data?#2017-08-3114:01favilathat's the part that is difficult#2017-08-3114:02favilaI essentially want s/tuple, with keyed names for each slot (not numbers), and possible truncation on the right side for optional values#2017-08-3114:03favilafor background, this is parsing an edi segment, if you are familiar with those#2017-08-3114:04Alex Miller (Clojure team)nah#2017-08-3114:04Alex Miller (Clojure team)sounds like you really want s/cat with nested s/?#2017-08-3114:05Alex Miller (Clojure team)(s/cat :1 int? (s/? (s/cat :2 string? (s/? (s/cat :3 keyword?))))) etc#2017-08-3114:05favilawait that's legal?#2017-08-3114:05Alex Miller (Clojure team)sure#2017-08-3114:05Alex Miller (Clojure team)the downside of that is the result will be nested maps#2017-08-3114:05favilai thought it had to be key+spec pairs#2017-08-3114:06Alex Miller (Clojure team)oh, I missed the keys in there (always do that)#2017-08-3114:06Alex Miller (Clojure team)(s/cat :1 int? :more (s/? (s/cat :2 string? :more (s/? (s/cat :3 keyword?)))))#2017-08-3114:07favila
(s/conform
  (s/cat
    :a number?
    :tail/next (s/? (s/cat
                          :b string?
                          :tail/next (s/? (s/cat :c keyword?)))))
  [1 "" :k])
#2017-08-3114:07favilawas the best I could come up with#2017-08-3114:07Alex Miller (Clojure team)yeah#2017-08-3114:07favilaand I was thinking maybe an s/& could flatten the keys out#2017-08-3114:07favila?#2017-08-3114:08Alex Miller (Clojure team)I would separate that from the spec, but yes should be able to regularize both the creation of these (with a macro) and the transformation to something you want#2017-08-3114:32bbrinck@alexmiller When using a predicate as a spec (which can occur in, say, s/assert), the value of :clojure.spec.alpha/spec is a function. However, if a proper spec is defined, the :pred is a symbol. https://gist.github.com/bhb/de31b73133d6322158a3c0a5d47d67a2#2017-08-3114:32bbrinckDo you happen to know why there is a difference?#2017-08-3114:32bbrinck(My goal is to provide a consistent printer that can say ā€œ<value> did not match <predicate>ā€ in all cases)#2017-08-3114:33bbrinckI did not see a JIRA ticket, but would be happy to create one if it is helpful šŸ™‚ Iā€™m not sure if there is a technical limitation here.#2017-08-3114:35bbrinckI suspect this is related to the fact that s/def is macro whereas explain-data is a function, so it only gets the function value#2017-08-3114:39bbrincki.e. the symbol is already resolved by the time explain and explain-data see it. Hm. I wonder if the right solution is to spend time making a pretty-printer for functions that works reliably across CLJ and CLJS#2017-08-3114:44Alex Miller (Clojure team)there is a ticket and a patch for that#2017-08-3114:44Alex Miller (Clojure team)but your reasoning is correct#2017-08-3114:48bbrinckMy ticket searching skills need work šŸ˜‰#2017-08-3114:48Alex Miller (Clojure team)https://dev.clojure.org/jira/browse/CLJ-2068 is what Iā€™m thinking of#2017-08-3114:48Alex Miller (Clojure team)but maybe itā€™s not covering what youā€™re asking#2017-08-3114:49Alex Miller (Clojure team)you should not have any expectation about the concrete type of :clojure.spec.alpha/spec, other than that it can be used as a spec#2017-08-3114:50bbrinckNo, youā€™re correct, this is what I want: I am not concerned about the concrete type, I just want a way to consistently pretty print the problem.#2017-08-3114:51Alex Miller (Clojure team)that ticket fixes the :pred in the explain-data#2017-08-3114:51Alex Miller (Clojure team)by unmunging the function name#2017-08-3114:51bbrinckAlthough in my case, Iā€™d probably print the set differently (instead of a set literal, Iā€™d enumerate options), but that should not be a problem#2017-08-3114:51Alex Miller (Clojure team)Iā€™m hoping it will be in the next spec release#2017-08-3114:52bbrinckDoes it apply the same unmunge to :clojure.spec.alpha/spec at the top level of the explain-data?#2017-08-3114:52bbrinckIOW, would it work for (s/explain odd? 10)?#2017-08-3114:52Alex Miller (Clojure team)
user=> (s/explain odd? 10)
val: 10 fails predicate: odd?
#2017-08-3114:53Alex Miller (Clojure team)(running with the patch)#2017-08-3114:53bbrinckMuch appreciated! I will definitely be using this in expound#2017-08-3114:57Alex Miller (Clojure team)I have been too busy to look at expound, but hope to soon#2017-08-3115:01bbrinckDonā€™t look too closely: the internals are a bit of a mess, but I hope to refactor soonish šŸ™‚ . Most importantly, I want to cleanly separate the pretty printing part (hopefully useful to end users, but not really reusable) from the stuff I did related to adding information to the ā€œinā€ paths, which could be useful for any library that wants to consume explain-data#2017-08-3115:06Alex Miller (Clojure team)seems wise#2017-08-3116:14didibusCan I trust describe to be consistent on specs of the same kind. Like will all s/keys describe itself in a consistent way?#2017-08-3116:15didibus I'm going to build reflective logic on spec descriptions, but I want to be sure its reliable.#2017-08-3116:41hmaurerHi! Is there a built-in function to conform a spec and throw an error if the spec does not conform?#2017-08-3116:42hmaurerMy use-case is basically this: I want to pipe data through a series of steps, and I would like to throw an error if a spec does not conform#2017-08-3116:42hmaurerI could easily write a function to do this#2017-08-3116:42bbrinck@hmaurer Would s/assert work?#2017-08-3116:42hmaurerjust wondering if itā€™s in clojure.spec#2017-08-3116:43bbrinckfollowed by conform?#2017-08-3116:43hmaureractually s/assert does exactly what I want; I donā€™t care too much about conforming#2017-08-3116:43hmaurerthank you!#2017-08-3116:45bbrinck@hmaurer Just make sure to call (s/check-asserts true) to turn on assert checking for spec#2017-08-3116:46hmaurer@bbrinck ok. Is it sensible to use s/assert when validating user input, e.g. an API payload?#2017-08-3116:50bbrinck@hmaurer I would prefer to just use explain-str in that case personally, since a) assert will raise an exception that youā€™ll need to catch and b) turning on asserts will enable asserts throughout your program, which you probably donā€™t want. Iā€™d only use assert for dev-time checks/invariants#2017-08-3116:51bbrinckIā€™m assuming the flow is check data against spec if it works, proceed else, show message to user?#2017-08-3116:52hmaurerpretty much#2017-08-3116:52hmaurerI would like to have some flow like this: check data again spec => validate through authorization filters => assign UUID => transact to database#2017-08-3116:52bbrinckIf your user is a dev at dev-time, I think assert is fine. But if you want to show end-user error messages, Iā€™d avoid it#2017-08-3116:52hmaurerany step can throw an error which should be reported back to the user#2017-08-3116:52hmaurerend-user, not dev#2017-08-3116:53bbrinckRight, in that case, Iā€™d use explain-str and check to see if it success.#2017-08-3116:53bbrinckThen return that string to whatever is going to show it to a user#2017-08-3116:54hmaurerI am quite new to Clojure; would you in general avoid to throw errors and return error conditions instead? Aka use a ā€œEitherā€-like type, similar to Haskell/etc?#2017-08-3116:55hmaurere.g. return {:ok ...} or {:fail ...}#2017-08-3116:56bbrinckGood question. I donā€™t see a lot of use of ā€œeitherā€ in Clojure code. Exceptions are fine if itā€™s actually an exceptional case , but AIUI, in your case, you want to validate user input so you expect things to not conform#2017-08-3116:56bbrinck(correct me if Iā€™m wrong)#2017-08-3116:57hmaurerYou are right. I would like to be able to chain my pipeline steps with (-> ...), but breaking out of the pipeline on error seems hard to do without exceptions#2017-08-3116:57bbrinckI donā€™t have great advice here for general error patterns, but Iā€™d lean towards ā€œuse exceptions for exceptional cases, not control flowā€#2017-08-3116:57bbrinckRight#2017-08-3116:57bbrinckThere are libraries for this, but I donā€™t know how idiomatic they are. I havenā€™t used them myself#2017-08-3116:58bbrinck@hmaurer Some discussion here: https://www.reddit.com/r/Clojure/comments/6wmnfm/thoughts_on_failjure_vs_funcoolcatseither_for/#2017-08-3116:59didibusIt's more idiomatic to use exceptions with ex-info. I would wrap explain-str or explain-data in a function which throws an ex-info, where I handle it at the place where my code can return an error msg to the user. There I'd parse out what I need from the exception into the msg I want to show the user and return it.#2017-08-3116:59hmaurerThanks! Iā€™ll read this. I like the philosophy of using errors for exceptional cases only though; considered following that approach in a previous javascript projct#2017-08-3117:00hmaurer@didibus yep, that sounds a bit more idiomatic/practical#2017-08-3117:01didibusWhat you do is in the ex-info map, you put a key such as :type :validation-error and so where you catch it, you know this isn't a bug, but a validation error. Normally, the exceptions should not log error or push failure metric, as they are actually expected behaviour#2017-08-3117:01bbrinckEven if you use exceptions in this case (not necessarily bad IMO), Iā€™d still probably throw an exception manually instead of relying on assert since turning on/off asserts is a global thing. IOW, do you want to enable all asserts in production?#2017-08-3118:22hmaurerThat seems reasonable#2017-08-3117:02bbrinckThatā€™s a valid question, but you probably donā€™t want it tied to your user-facing errors messages#2017-08-3117:02didibusI mean exceptions of type validation-error#2017-08-3117:02didibusAgree with bbrinck#2017-08-3117:02bbrinckGood suggestion @didibus#2017-08-3117:03bbrinck@hmaurer Shameless plug, but if you are considering showing spec error messages to users, take a look at https://github.com/bhb/expound#2017-08-3118:24hmaurerI stumbled on it yesterday actually! Itā€™s looking very useful#2017-08-3117:06didibusBig fan of expound, the default spec errors are so full of noise.#2017-08-3117:08didibus I'm actually disappointed in spec for improving clojure error msgs. They promote the idea, but it really doesn't help because the error is so long. Its only slightly friendlier then a stack trace#2017-08-3117:09bbrinckIMO, the default error messages are a good default, since they need to be useful ā€œout of the boxā€ in a variety of contexts: shown to a end user, shown to a dev, logged, returned as an API response, etc. Expound makes a different set of tradeoffs that are very much optimized for the ā€œdisplay to devā€ case - *much* more verbose, and omits some information, but hopefully helps devs find the root problem faster#2017-08-3117:10bbrinckLuckily we can swap out the default for custom printers like expound when we want to customize behavior šŸ˜‰#2017-08-3117:11bbrinckExpound is probably still not quite optimized for end users, but I am looking to make it more suitable for this in the future.#2017-08-3117:11didibusI don't know. Id like more beginner friendly defaults in Clojure. Easier to swap out to something else when you're not a beginner anymore. Its a lot to ask to a beginner to learn how to setup expound to figure out why his code doesn't work.#2017-08-3117:13bbrinck@didibus I agree with you that itā€™d be nice to get something easier for beginners. I have been wondering if it would be possible to create a lein plugin or library that would set up beginner-friendly REPL / logs by default#2017-08-3117:14bbrinckAdding a single dependency that loads core specs, turns on instrumentation (with orchestra?) and configures spec to use expound without any further config could be really useful#2017-08-3117:16bbrinckmaybe pretties up/shortens stack traces as well#2017-08-3117:17didibusThat would be really nice. Maybe if it can also add a coloured repl and preload clojure.repl #2017-08-3117:19didibusI've heard of https://github.com/venantius/ultra/blob/master/README.md#2017-08-3117:19didibusBut since at my work we use Ant, it doesn't help me promote Clojure to other teams#2017-08-3117:22bbrinckWhat editor would you recommend for other teams? Just curious about the constraints.#2017-08-3117:24didibusWell, I tend to recommend eclipse + counterclockwise or cursive with intelliJ.#2017-08-3117:25didibusThe latter less so, because of its paid nature#2017-08-3117:27didibusYou'd just be surprised how many top class engineers don't bother to learn how to setup any environment properly.#2017-08-3117:28didibusSo when I tell them, if you want to do Clojure and like it, you need to learn at least how to setup a working repl. And already I've lost the interest of half of them#2017-08-3118:12gdeer81@didibus I think @sekao is the best person to talk to about this since he has written three IDEs so far and each one tries to lower the burden of getting everything set up#2017-08-3118:16gdeer81this is a side topic not really related to spec so I broke it out into this thread#2017-08-3120:21didibusNightlight looks pretty cool #2017-08-3118:15gdeer81my favorite is nightmod since I can send someone the download link and they can be in a new project with a repl, starter code, and a live game environment in less than 5 minutes#2017-08-3118:38favilaIn clojure.spec.alpha, s/invalid? is defined like (identical? ::invalid ret). I thought we couldn't rely on keyword interning anymore?#2017-08-3118:40ghadiit still is a thing#2017-08-3118:40ghadi(not in cljs afaik)#2017-08-3118:40favilaI thought at some point even in clj it was no longer guaranteed#2017-08-3118:41favilaso (identical? (sym|kw) x) became a bad idiom#2017-08-3118:41ghadisymbols are not interned#2017-08-3118:41ghadijust keywords#2017-08-3118:41favilaah maybe that's what I'm thinking of#2017-08-3118:41favilaor I'm just confused with cljs#2017-08-3118:41favila(I do a lot of cljs)#2017-08-3118:41ghadithough the codepath goes still through a method called intern#2017-08-3119:05ikitommihi. I have a problem creating a recursive spec. the s/spec requires the spec to exist:
(require '[clojure.spec.alpha :as s])

(def data
  ["/api"
   ["/ping" {}]
   ["/pong" {}]])

(s/def ::route
  (s/cat
    :path string?
    :arg (s/? (s/map-of keyword? any?))
    :childs (s/* (s/spec ::route))))
; CompilerException java.lang.Exception: Unable to resolve spec: :user/route
#2017-08-3119:06ikitommiSetting something temporary as a spec and double-evaluating the original spec seems to work thou, but donā€™t think this is the way to do this šŸ˜‰#2017-08-3119:06ikitommi
;; declare'ish
(s/def ::route int?)

(s/valid? ::route data)
; false

;; resetting the spec works now (but the childs is wrong)
(s/def ::route
  (s/cat
    :path string?
    :arg (s/? (s/map-of keyword? any?))
    :childs (s/* (s/spec ::route))))

(s/valid? ::route data)
; false

;; now it might work?
(s/def ::route
  (s/cat
    :path string?
    :arg (s/? (s/map-of keyword? any?))
    :childs (s/* (s/spec ::route))))

;; ....
(s/valid? ::route data)
; true
#2017-08-3119:59favila@alexmiller My attempt to spec the s/cat s/tuple hybrid with optional trailing positions https://gist.github.com/favila/67c869b3c92b1a40559f639b065734e2#2017-08-3120:02favilaworks for me (so far)#2017-08-3120:04ikitommiOh, my recursive spec works when I wrap the child spec into something, like s/and
(s/def ::route
  (s/cat
    :path string?
    :arg (s/? (s/map-of keyword? any?))
    :childs (s/* (s/spec (s/and ::route)))))

(s/valid?
  ::route
  ["/api"
   ["/ping" {}]
   ["/pong" {}]])
; true
Is this the correct way to do this?
#2017-08-3121:33hmaurerHi! Quick question: I have a spec for a map which has required and optional keys (`(s/keys :req [..] :opt [..])`). One of my functions takes ā€œpartialā€ versions of those maps, which essentially means that every key is optional (and every key that was optional in the full spec can be nil, but I can handle that separatly)#2017-08-3121:34hmaurerit there a neat way to do this, how should I define two versions of the map specs? (e.g. :foo/my-map and :foo.partial/my-map)#2017-08-3121:42plinshello everyone, im reading the spec docs (`https://clojure.org/guides/spec`) is it possible to achieve with spec/fdef something similar to this? (catching spec errors at run time):
(defn person-name
  [person]
  {:pre [(s/valid? ::person person)]
   :post [(s/valid? string? %)]}
  (str (::first-name person) " " (::last-name person)))
#2017-09-0104:20didibusYou can instrument, but it doesn't do the post#2017-09-0104:40didibusYou can use orchestra a 3rd party library, that one will instrument the post also#2017-09-0114:17plins@didibus#2017-09-0114:18plinsany recommendations on libraries that already do that?#2017-09-0221:59didibus@U3QUAHZJ6 https://github.com/jeaye/orchestra#2017-08-3122:04wilkerluciohello, I'm having an issue trying to define the following specs:#2017-08-3122:04wilkerlucio
(s/def :user/accessors (s/keys :opt [:user/id :user/name :user/address]))
(s/def :user/id string?)
(s/def :user/name string?)
(s/def :user/address :address/accessors) ; <- trouble maker

(s/def :address/accessors (s/keys :opt [:address/street :address/user]))
(s/def :address/street string?)
(s/def :address/user :user/accessors)
#2017-08-3122:05wilkerluciothis is a simplified case where I need to set a spec to something that still are going to be defined#2017-08-3122:05wilkerlucioright now this throws an error, as:#2017-08-3122:05wilkerlucioCompilerException java.lang.Exception: Unable to resolve spec: :address/accessors, compiling:(2017.clj:10:1)#2017-08-3122:06wilkerlucioI remember reading that the definitions should be lazy to allow this kind of things, am I missing something here?#2017-09-0104:12didibusI always known it as not being lazy. Can't you just reorder your definitions? Being lazy would make it risky I feel, what if specs are accross namespaced and lazy, and you forgot to load another namespace.#2017-08-3122:17wilkerlucio@hmaurer you can define multiple versions, create a simpler one and then you can merge it with other to create a more fully-featured, example:
(s/def :foo.partial/my-map (s/keys :req [:foo/a :foo/b]))
(s/def :foo/my-map (s/merge :foo.partial/my-map (s/keys :req [:foo/c])))
#2017-09-0108:43lmergeni have a question about project organisation and specs ā€” specifically, iā€™ve settled on the use of a separate specs.clj file to put all my specs in, that works quite well. but, i have multiple small subprojects, of which some share the same specs (i.e. the output of one project is the input of another). whatā€™s the best strategy here ? just copy/paste these common specs into these projects (because theyā€™re different domains), or share the code using a common library ?#2017-09-0108:52mpenetseparate repo for shared specs works well#2017-09-0108:52mpenetdepends what it is#2017-09-0109:00ikitommicould there be something like s/lazy which would work like s/spec but would not fail if the given qualified keyword doesnā€™t link to registered spec? could be used with the recursive specs (my example above). (s/lazy ::route) (vs (s/spec (s/and ::route)). The error messages be easier to understand. Happy to do a PR if.#2017-09-0113:59wilkerlucioI have the same problem when trying to spec some distant relations where I can't guarantee the load order#2017-09-0413:02wilkerlucio@ikitommi just one thing, you can do (s/and ::route) instead of (s/spec (s/and ::route)) and it will work the same#2017-09-0413:08ikitommi@U066U8JQJ indeed, thanks!#2017-09-0109:21lmergen@mpenet in this specific instance itā€™s the spec of what a job input should look like ā€” itā€™s produced by one component, and consumed in another.#2017-09-0109:21lmergenso it strikes me that a common spec library might be a good approach for this, but it also feels like it might create more spaghetti / overhead
#2017-09-0113:04aisamuHi! Is there a non-macro alternative to specā€™s sequence operators? (`cat`, *, ? ā€¦). To avoid the XY problem: I have to generate medium-sized specs at ā€œruntimeā€, and my current solution is giving me ERROR java.lang.RuntimeException: Method code too large!. Iā€™ve read that it happens because there is an eval generating classes too big for the JVM. The eval exists because I couldnā€™t use the spec macros at runtime without wrapping them with make-fn. Is it unfeasible or just not currently implemented? Or am I approaching this backwards?
;; 
(defmacro make-fn [m] 
 `(fn [& args#]
    (eval 
      (cons '~m args#))))

#2017-09-0114:13ghadi@aisamu what specifically are you evalling that is causing the error?#2017-09-0114:17aisamuThe sequence macros, as in (make-fn s/?), which later gets called with apply and a vector of specs#2017-09-0114:28dealyHi, I've been trying to write a spec for a function which takes another function as a parameter. However it always fails with an Unable to construct gen error, is there a way to spec a function like this?#2017-09-0114:33ghadi@aisamu Are you doing something like calling eval on a long collection [(s/?....)......] ? How many before it blows with an error?#2017-09-0114:57aisamu@ghadi No. It's a small collection (~4), but the specs inside can get rather large. (The last input collection I managed to trace before the blow-up had 50k chars)#2017-09-0115:05ghadiyeah... spec macros can generate a lot of code#2017-09-0115:05ghadithe clojure compiler doesn't do method splitting#2017-09-0115:06ikitommi@aisamu had the same problem. See the source code, most specs have non-documented functional versions beneath. Hopefully will be part of the final public api.#2017-09-0115:11aisamu@ikitommi I looked into that, but it quickly became clear I'd have to bring along a dozen private functions to get the top level to work =/#2017-09-0115:23Alex Miller (Clojure team)the downside of the functional versions is that you lose forms for reporting#2017-09-0115:23Alex Miller (Clojure team)as those are captured by the macro#2017-09-0115:24Alex Miller (Clojure team)I have talked here in the past about how specs for spec forms (CLJ-2112) can be used to programmatically construct specs via unform#2017-09-0115:24Alex Miller (Clojure team)still a work in progress#2017-09-0115:47ikitommiI don't mind that the forms need to be manually passed to the functions. I'm using a pred->form multimethod to auto-resolve the forms, with most/all core preds mapped.#2017-09-0115:50ikitommiAnd looking forward on progress on the CLJ-2112.#2017-09-0116:33hlshipI think clojure.spec could do a bit better job of reporting failures to dynamically load clojure.test.check.generators. My production code doesn't include the org.clojure/test.check dependency, but does instrument a particular function at runtime; that seems to be forcing the code down a path that involves org.clojure/test.check ... possibly because some of the values being specs are functions? Very hard to say.#2017-09-0116:33hlshipPossibly this can be circumvented by providing an explicit generator for those function specs within our giant spec.#2017-09-0118:04mpenetfunctions as argument of instrumented fn get tested via gen yep#2017-09-0118:05mpenetyou can overwrite :gen on them with ifn? but it kinda defeats the purpose after all, personally I would prefer if we could make it so that the argument (which is a fn) has its args checked at "invoke" time#2017-09-0118:05mpenetthere's also a ticket on hold somewhere about this if I recall#2017-09-0119:03hlshipI've basically changed my spec from a useful fspec to just fn?. https://github.com/walmartlabs/lacinia/pull/112 That'll do for now.#2017-09-0412:32mpenetseems like I hit an interesting bug: direct-linking breaking a multispec : [...] foo is not a fn, expected predicate fn#2017-09-0412:33mpenetstill trying to understand the why/how#2017-09-0412:33mpenet :at [clojure.spec.alpha$dt invokeStatic alpha.clj 756]}]#2017-09-0412:33Alex Miller (Clojure team)and you canā€™t reproduce without direct linking?#2017-09-0412:33mpenetthe bug goes away without dl yes#2017-09-0412:33Alex Miller (Clojure team)well would be good to see a jira on that if you can build a reproducible case#2017-09-0412:34mpenetthe code is quite involved I ll try to create a minimal repro#2017-09-0412:34Alex Miller (Clojure team)I donā€™t think multimethod calls are direct linked#2017-09-0412:34Alex Miller (Clojure team)the line you have above is part of regex specs, not multispec#2017-09-0412:35mpenetbut it mentions a fn that's used as first arg to a multi-spec#2017-09-0412:36mpenet(it's a multimethod to be more precise)#2017-09-0412:36Alex Miller (Clojure team)sorry, just guessing from too little info#2017-09-0412:36mpenetnp#2017-09-0419:54gklijsmanaged to get spec working on clojurescript also after some searching/trying, but did not find any direct help on the internet. Iā€™n now using a cljc file starting with `(ns m-venue.spec (:require [#?(:clj clojure.spec.alpha :cljs cljs.spec.alpha :default clojure.spec.alpha) :as s]))` and on clojure I only need to source path, for clojurescript i need to put it in the source paths and require it. Now I can use the same spec front and back-end šŸ™‚.#2017-09-0422:45Alex Miller (Clojure team)cljs automatically rewrites the clojure namespace as the cljs one, so you shouldnā€™t actually need any of this afaik - just require the clojure ns#2017-09-0420:42plinshi everyone, im trying to use :pre and :post to produce a more detailed error in run time
(defn person-name
  [person]
  {:pre  [(if-not (s/valid? ::person person)
            (throw (IllegalArgumentException. (s/explain ::person person)))
            true)]
   :post [(if-not (s/valid? string? %)
            (throw (Exception. (s/valid? string? %)))
            true)]}
  (str (::first-name person) " " (::last-name person)))



(person-name 42)
;;=> CompilerException java.lang.IllegalArgumentException
val: 42 fails spec: :payment-gateway.util/person predicate: map?
:clojure.spec.alpha/spec  :payment-gateway.util/person
:clojure.spec.alpha/value  42

;; works fine, s/explains works properly

(person-name {::first-name "Elon" ::last-name "Musk" ::email "
#2017-09-0502:50souenzzomissing # ?#2017-09-0420:44plinswhy this works for :pre but not for :post?#2017-09-0422:47Alex Miller (Clojure team)shouldnā€™t both of those pre and post be functions? I think theyā€™re both getting evaluated at compilation time, not runtime#2017-09-0509:32AvichalHello, I am trying to spec some higher order functions using fdef and fspec For eg:
(s/fdef adder
  :args (s/cat :x number?)
  :ret (s/fspec :args (s/cat :y number?)
                :ret number?)
  :fn #(= (-> % :args :x) ((:ret %) 0)))
is it possible to get the value of argument x inside the spec of the inner function which is returned ?
#2017-09-0511:49Alex Miller (Clojure team)No #2017-09-0514:38fothaggendaHello there! Quick question: is it possible to create an "inline" s/keys inside a s/cat in fdefs? Like:
(s/fdef something :args (s/cat :first-argument (s/keys :req-un [::a ::b ::c])))
#2017-09-0514:38Alex Miller (Clojure team)no#2017-09-0514:38fothaggendašŸ˜ž Ok, thank you!#2017-09-0514:39Alex Miller (Clojure team)or wait, maybe? what are you trying to do?#2017-09-0514:39Alex Miller (Clojure team)you might want s/keys*#2017-09-0514:39Alex Miller (Clojure team)if youā€™re matching varargs in a function#2017-09-0514:39fothaggendaHmmm, I'll check that out šŸ˜ƒ#2017-09-0514:40Alex Miller (Clojure team)and even my prior answer is wrong - you can use s/keys there (I just donā€™t think itā€™s what you want)#2017-09-0514:40Alex Miller (Clojure team)are you taking a map or a series of key/value options in something?#2017-09-0514:40fothaggendaOdd; I tried that way before and my generative test exploded#2017-09-0514:40fothaggendaI'm taking a map#2017-09-0514:41Alex Miller (Clojure team)oh, then you should be fine with the above#2017-09-0514:41Alex Miller (Clojure team)if you want to test it, run (s/exercise (s/cat :first-argument (s/keys :req-un [::a ::b ::c])) 1000) - thatā€™s basically what youā€™re getting with a generative test on something#2017-09-0514:42fothaggendaNice catch, I'll inspect this way#2017-09-0514:43Alex Miller (Clojure team)often pushing that out to 1000 samples reveals more :)#2017-09-0514:43Alex Miller (Clojure team)if a/b/c are big colls, that can provoke some problematic samples - bounding their size often helps#2017-09-0514:43Alex Miller (Clojure team)with :gen-max#2017-09-0514:44fothaggendaActually that's not the case, in my scenario ::a, ::b and ::c are just very specific definitions, but I got your point#2017-09-0518:47arohnerare there any projects for associating docstrings with spec definitions?#2017-09-0518:52Alex Miller (Clojure team)we will eventually get around to implementing it in spec, just kinda far down the list atm#2017-09-0608:41gklijs@arohner I have a private project which I use to serialize specced data, where I also store the used spec in either the bytes or the string. I might make it public. It also makes it possible to set defaults, in order to get correctly specced data back, in case you added required keys.#2017-09-0608:47gklijsIt works both with clojure and clojurescript, but it really needs some more work (liks tests for the clojurescript) before I put it public#2017-09-0611:46ikitommi@gklijs can you re-create the spec instances from serialized format in cljs without (form) eval?#2017-09-0614:06gklijs@ikitommi Doentā€™t know what you mean exactly, i use the form function to re-create the instance, but by assuming the spec is a map with either keys, or merge, and by storing the data in a certain way, I only store the values. If there is an additional required key which wasnā€™t there when the instance was serialized, either nil is filled in, or the default is it is set.#2017-09-0712:14seantempestaIs there a version of s/or that just conforms the value and doesnā€™t tell you what value it is? I have a map where certain entries can be either a string or a function that returns a string and I just want to get back either a conformed map or :cljs.spec.alpha/invalid. Am I supposed to traverse the returned map and remove the named vectors?#2017-09-0713:54misha@seantempesta
(s/def ::key (s/or :int int? :str string?))
(s/conform ::key 1)
=> [:int 1]
(s/conform (s/nonconforming ::key) 1)
=> 1
(s/conform (s/nonconforming ::key) :foo)
=> :clojure.spec.alpha/invalid
#2017-09-0713:55seantempesta@misha oh cool. so just call s/nonconforming first?#2017-09-0713:56mishawrap spec you don't want vectors from in nonconforming#2017-09-0713:58seantempestaOh wait, s/nonconforming says that it returns the original value. I want the conformed value though.#2017-09-0713:59mishawhat do you think original and conformed values are?#2017-09-0713:59seantempestaoriginal is the original value, but a conformer function can return a modified value.#2017-09-0714:00mishaif you have custom conformer, I think you can "avoid vector" there#2017-09-0714:02seantempestaWell I was using all custom conformers. Writing separate functions like str-or-fn-ret-str, but I realized I was duplicating a lot of code. Also, I think that requires me to write custom generators for everything.#2017-09-0714:03mishacan you share str-or-fn-ret-str?#2017-09-0714:03seantempestaIt looks like (s/unform ::key (s/conform ::key val)) is working#2017-09-0714:04seantempestaWell, itā€™s actually string-or-react-class. Hereā€™s what it looks like:
(defn string-or-react-class?
  "Accepts either a string, react class, or a fn that returns a react class.  If it's a fn, props will automatically
  convert (js->clj) when the fn is called."
  [s-or-c]
  (cond
    (helpers/is-react-class? s-or-c) s-or-c
    (string? s-or-c) s-or-c
    (fn? s-or-c) (fn [props & children]
                   (let [clj-props (js->clj props :keywordize-keys true)
                         react-c (s-or-c clj-props children)]
                     react-c))
    :else :cljs.spec.alpha/invalid))
#2017-09-0714:04mishaif you supply custom unformer - it might work. But I have a feeling you are doing something redundant#2017-09-0714:05mishawhy do you have it as a conformer, rather than spec composed from predicates?#2017-09-0714:05seantempestaThatā€™s what I was asking. If I do it as s/or predicates I get back vectors#2017-09-0805:42didibusFrom my perspective, you conform to get back a vector, or a custom projection of the specced data.#2017-09-0805:43didibusIf you want to know if something validates, you do (when (s/valid? ...) do), or one of the explain fns.#2017-09-0714:08mishatry to wrap s/or in nonconforming. this way you can have more granular conformer for just (fn? s-or-c) case.#2017-09-0714:08seantempestaOkay. Iā€™ll give that a shot. Thanks.#2017-09-0714:14mishaalthough I'd like to see examples of coercion in spec done right, myself#2017-09-0714:17misha@seantempesta it seems like nonconforming or prevents any conformation inside its branches#2017-09-0714:18mishaduh... opieop#2017-09-0714:26bbrinck@seantempesta Can you talk more about why a call to conform is useful in this case? If you just want the original value, could you just do something like (if (s/valid? ::string-or-int x) x :cljs.spec.alpha/invalid)#2017-09-0714:27mishaI think to wrap fn? in (fn [props & children]#2017-09-0720:57cgrandHow do you prevent a :ret spec to be described by doc? I have a big spec and Iā€™d like to be displayed by name rather than dumping it on the user. Currently I wrap it into spec/and. Is there a more elegant solution?#2017-09-0721:04Alex Miller (Clojure team)name it?#2017-09-0721:05Alex Miller (Clojure team)then youā€™ll just see the name in the doc#2017-09-0721:14cgrandIā€™ll stick with spec/and then. (btw I also abuse spec/and for late-bound aliases)#2017-09-0721:25Alex Miller (Clojure team)I guess I donā€™t understand why thatā€™s better#2017-09-0722:07cgrandIf I put it in only in the doc then it's not in the spec anymore. If I put it in doc and in spec it doesn't prevent doc from dumping the whole description. So s/and is the hack to have :ret spec'ed without doc being verbose#2017-09-0722:50Alex Miller (Clojure team)sorry. I went and tried it and I understand what youā€™re saying now. So the current behavior is not the intention. there is a ticket for the early aliased spec resolution, donā€™t know the #.#2017-09-0721:28ddellacostawhat is the best approach for validating a tuple that has optional fields? E.g. [:required :required :optional]?#2017-09-0721:35Alex Miller (Clojure team)use (s/cat :a int? :b int? :c (s/? int?)) etc#2017-09-0721:37ddellacostaah thanks Alex, s/? was the missing piece I was trying to find.#2017-09-0721:39Alex Miller (Clojure team)itā€™s like regex - s/* s/+ s/? s/cat s/alt#2017-09-0721:30misha@alexmiller can you comment on @seantempesta's question, please? what is a better way to do coercion with spec? what is expected way to work with conformed compound values (map conformed to some the nested spec)? e.g. when you get map values as an or-conform-vector [:branch conformed-value]. Is "conform-client" code supposed to know the spec structure, and be able to walk the conformed data structure?#2017-09-0721:35Alex Miller (Clojure team)donā€™t do coercion with spec#2017-09-0721:36Alex Miller (Clojure team)use conform to tell you how a value is parsed. then use clojure to transform that to what you want.#2017-09-0721:38Alex Miller (Clojure team)for the specific case of wanting to avoid conforming an s/or (one of the most common), spec as (s/def :foo (s/nonconforming (s/or :a int? :b string?)))#2017-09-0721:39Alex Miller (Clojure team)that is, use s/nonconforming at the leaves, not at the top#2017-09-0721:40Alex Miller (Clojure team)for this particular case, Iā€™ve come to believe that nonconforming or is a reasonable use case and either we should keep s/nonconforming or add that as a variant/option on or#2017-09-0721:41mishaso, going back to Sean's example, conform map to spec (will return [:fn? s-or-c] for some key), and then post-process conformed map with a coerce-function, which will replace [:fn? s-or-c] with (fn [props & children] ...?#2017-09-0721:42Alex Miller (Clojure team)without having completely read or grokked all of the backchat, sure#2017-09-0721:42Alex Miller (Clojure team)Clojure is like, pretty good at transforming data. use it.#2017-09-0721:48mishaI think the issue/question is: "should one wrap coerce-fn in s/conformer and conform to that, or should one just apply coerce-fn after conforming data to simpler spec?"#2017-09-0722:01Alex Miller (Clojure team)the latter#2017-09-0722:01Alex Miller (Clojure team)if you do the former, you are registering specs that throw away information about the original value#2017-09-0812:32ikitommiwe are running our spec libs tests also with :advanced cljs optimizations. Seems that the cljs.spec.test.alpha/instrument doesnā€™t work under it. Is there a way to enable it?#2017-09-0822:05jjttjjlet's say the spec decimal-in defined here: https://github.com/SparkFund/useful-specs/blob/master/src/specs/number.clj#L24 is exactly what i need, except that the generator it comes with results in a bigdec, where I need a double. Is there a way to make a new generator that just coerces the output of an existing generator?#2017-09-0822:06jjttjjbasically i just need a spec and a generator for a double with a specified precision and scale#2017-09-0822:16Alex Miller (Clojure team)Make a spec with s/decimal-in#2017-09-0822:16Alex Miller (Clojure team)Then make a spec that wraps it with s/with-gen#2017-09-0822:17Alex Miller (Clojure team)If you want build on the existing gen, use gen/fmap over the s/gen of the first spec#2017-09-0923:58samedhiI have the following
;; This ::post does not work, recursion limit
;; (spec/def ::post (spec/keys :req [::post-content ::user-id ::posts]))

;; This ::post does work, never seems to recurse though...
(spec/def ::post (spec/keys :req [::post-content ::user-id]
                            :opt [::posts]))

(spec/def ::posts
  (spec/with-gen
    (spec/map-of ::post-id ::post)
    #(spec/gen (spec/map-of ::post-id ::post :min-count 0 :max-count 1))))
which defines a spec recursively; ::post refers to ::posts .However, if I remove :opt [::posts] and add ::posts to the :req fieldā€™s ::post, I get a Maximum call stack size exceeded. Is this the expected behavior?
#2017-09-1000:04samedhiSeems like the two should be equivalent, since ::posts would (probabilistically) stop its recursion when it generates the empty map (`{}`).#2017-09-1000:09samedhiPleased to see that the generated data is in fact recursive! I guess I will just let it be, just to close out, my question mostly has to do with why ::posts must be put in the :opt and cannot be included in the :req.#2017-09-1000:11samedhiAlso, it is rather interesting that ::posts can be referenced before being used, which I guess implies that spec/keys does not ā€œdereferenceā€ ::posts at its declaration, but waits to do so until the ::post spec is used.#2017-09-1000:11samedhiCuriouser and curiouser#2017-09-1118:48bfabry@samedhi well... they're not equivalent, because a optional key is different to a required key which is a (possibly empty) map. it seems like the generator produced by map-of just doesn't produce empty maps often enough to avoid blowing the stack#2017-09-1118:51bfabryyou need to be able to reference something before it's defined in order to support recursive specs, which is an intended feature#2017-09-1118:52samedhiHmm, interesting. It appears that I am referring to ::posts from within ::post before ::posts actually existā€¦#2017-09-1118:53bfabryyes, that's intended to be supported#2017-09-1118:59bfabryI don't know enough about how generators work, I'm guessing it's more sophisticated than just random so that's why it's managing to find the pathological case of a very deeply nested tree every time#2017-09-1120:57tbaldridgeYes, a lot of the complexity in generators comes from the feature of shrinking. When you get a test failure with a generator, test.check will try to shrink the possible data set to give you the smallest failing test. So instead of giving you "Failure when I tried: [123123121 55 3 9 5 23 4]" It may say: "Failure when I tried: [0 2]". And that tells you a lot about went wrong:#2017-09-1120:57tbaldridgefor example you know that the vector has to be at least two items long (it didn't fail when there was only one value in the vector). And that the second number has to be larger than one: (since 0 and 1 didn't cause it to fail).#2017-09-1120:58tbaldridgeThat sort of information is very helpful in debugging, but requires a lot of backtracking in the generator, and that's what makes them so hard to write/understand.#2017-09-1123:15bfabryI would love to know the "right" way to make the generators work for the recursive structure above#2017-09-1200:04gfredericksyou could try writing your own with gen/recursive-gen#2017-09-1200:05gfredericksor...is this a DAG?#2017-09-1200:06gfredericksor I guess it's a proper tree?#2017-09-1200:07bfabryI did try with recursive-gen and got the same error. seems like it should be a proper tree yeah, which I guess is a type of DAG#2017-09-1200:07gfredericksyeah I think recursive-gen would work#2017-09-1200:07gfrederickscan you share the recursive-gen code?#2017-09-1200:07bfabrythis is what I tried:
boot.user=> (spec/def ::posts
       #_=>   (spec/with-gen
       #_=>     (spec/map-of ::post-id ::post)
       #_=>     #(tc-gen/recursive-gen (fn [post-gen] (tc-gen/map (spec/gen ::post-id) post-gen)) (spec/gen ::post))))
#2017-09-1200:08gfredericksI think the problem there is the (spec/gen ::post)#2017-09-1200:09gfredericksyou can't use that generator as part of what you're building, since it's your original problem#2017-09-1200:09gfredericksif you had a ::post that had everything but the ::posts key, so that it didn't recurse, then you could use that#2017-09-1200:09gfredericks::post-with-empty-posts#2017-09-1200:17bfabrybut a post without the ::posts key is not valid, the valid value for the leaf would be {}#2017-09-1200:20gfredericksYeah I meant a version where the key always has {}#2017-09-1205:59levitanong@gfredericks out of curiosity, could you try: (s/with-gen ::post #(s/gen (s/or :post ::post :empty #{{}})))#2017-09-1210:50gfredericksno, because it will still try to generate the default generater for a ::post, which can overflow the stack or whatever it was#2017-09-1207:20lmergenwhat would be the best way to spec a collection of tuples, e.g.
[[:k1 :v1] [:k2 :v2]]
i'm thinking of just doing an into {} ... before speccing and speccing it as if it were a map, but is there perhaps a more elegant way to do this ?
#2017-09-1207:20lmergenthe alternative i came up with was (s/* (s/or (s/cat ...) (s/cat ....))), but that's also ugly#2017-09-1207:24lmergen(as a bit of background, the reason i'm not using a map is that the ordering of the elements is important)#2017-09-1207:30tapMaybe this?
(s/coll-of (s/tuple keyword? keyword?))
#2017-09-1207:31lmergenyeah that's similar to the cat use case... the problem is that for a specific key, i need a specific value so then you would end up with
(s/or :k1 (s/tuple #(= :k1 %) ::v1)
           :k2 (s/tuple #(= :k2 %) ::v2)
#2017-09-1207:31lmergenetc#2017-09-1207:32lmergenthat is, if you want to have a similar effect as
(s/keys :opt-un [:k1 :k2])
#2017-09-1207:40tapOk. I also donā€™t know a better way than your s/or way#2017-09-1207:42lmergenok. guess i'll wrap this in a macro or something to make it pleasant.#2017-09-1207:44lmergenmaybe a multi spec is the answer though...#2017-09-1207:44lmergenyes, i definitely think that can work#2017-09-1210:01mpenetyou can shorten it a bit with (s/tuple #{:k1} ::v1)#2017-09-1210:02mpenetotherwise maybe keys* can be used#2017-09-1210:07mpenetbasically (s/keys* :req-un [(or ::k1 ::k2)])#2017-09-1210:07mpenetconforming it will return a map tho#2017-09-1210:07mpenet(s/valid? (s/keys* :req-un [(or ::foo ::bar)]) [:foo 1]) -> ok (::foo is #{1} here for the example)#2017-09-1210:08mpenetbut it will happily take "longer" tuples so not sure it's the best thing tbh#2017-09-1210:10mpenetseems like keys* exists (among other things) to validate unspliced kw args#2017-09-1210:34lmergen@mpenet thanks! that looks like what i want#2017-09-1211:12Jon
:data #:clojure.spec.alpha{:problems [{:path [:args], :reason Extra input, :pred (clojure.spec.alpha/cat :docstring (clojure.spec.alpha/? clojure.core/string?) :attr-map (clojure.spec.alpha/? clojure.core/map?) :clauses :clojure.core.specs.alpha/ns-clauses), :val ((:require-macros [respo.macros :refer [defcomp div span input <> cursor->]]) (:require [clojure.string :as string] [hsl.core :refer [hsl]] [respo.app.comp.task :refer [comp-task]] [respo.core :refer [create-comp]] [respo.comp.space :refer [=<]] [respo.comp.inspect :refer [comp-inspect]] [respo.app.comp.zero :refer [comp-zero]] [respo.app.comp.wrap :refer [comp-wrap]] [polyfill.core :refer [text-width* io-get-time* set-timeout*]] [respo.app.style.widget :as widget])), :via [], :in [1]}], :spec #object[clojure.spec.alpha$regex_spec_impl$reify__1200 0x60330e39 
#2017-09-1211:12Jonhow to read this error please?#2017-09-1212:33ikitommi@jiyinyiyong try https://github.com/bhb/expound#2017-09-1212:38ikitommiThere is a good sample of it in https://github.com/metosin/reitit/blob/master/README.md#validating-route-trees. Original :problems was totally unreadable (recursive specs). Would like it just to report the deepest error, but still it's awesome.#2017-09-1217:04seancorfield@jiyinyiyong According to the :args and :in [1], it's not expecting that :require-macros form -- so I'm thinking you tried to run this in Clojure, rather than ClojureScript? (I think :require-macros is ClojureScript-only?)#2017-09-1223:52Jonit turned out shadow-cljs compiler did not handle it correct šŸ™‚#2017-09-1217:04seancorfield(but, yes, some of ns spec output is a bit overwhelming)#2017-09-1217:05seancorfieldExtra input generally means a sequence (`s/cat`) ran into a form that was not recognized by the syntax specified in the spec.#2017-09-1217:07seancorfield:value (respo.app.comp.todolist (:require-macros [... is the value that failed and :in [1] gives you the position within that sequence.#2017-09-1218:57kevin.lynaghIs validating data with runtime context in the purview of spec? For example, instead of just saying that :foo/id is an integer, can I specify that "it's an integer that corresponds to an entity in the provided database?"#2017-09-1218:57ghadinot recommended#2017-09-1218:57kevin.lynaghFrom what I can tell, even though specs are being checked at runtime, there's no idiomatic way (short of dynamic var based hacks) to pass additional context into the validation logic.#2017-09-1218:58kevin.lynagh@ghadi Is there any kind of rationale or discussion on this topic somewhere that you know of? I haven't been able to find anything in my research more specific than "don't use spec as a runtime transformation system"#2017-09-1218:58kevin.lynaghwhich is sort-of-related, but not quite.#2017-09-1218:59ghadiif you look up in a DB your predicates / specs might change their contract#2017-09-1219:00ghadithe meaning of a spec might change based on state, which loses a whole bunch of value (inverting a spec into a generator)#2017-09-1219:00ghadiI think specs should not depend on any ambient state#2017-09-1219:01kevin.lynaghbesides the implicit spec database#2017-09-1219:01ghadie.g. just check that the email address is valid looking, don't try to open up an SMTP connection#2017-09-1219:02kevin.lynaghYeah, but that's something that can't be technically enforced by the API of spec#2017-09-1219:02kevin.lynaghgiven that it accepts arbitrary predicate fns#2017-09-1219:02ghadii get that, it's just a terrible idea to curry in state into your predicate#2017-09-1219:03ghadinothing I can do to prevent it ĀÆ\(惄)/ĀÆ#2017-09-1219:03kevin.lynagh= )#2017-09-1219:03kevin.lynaghRight, I don't mean to challenge the underlying design decision#2017-09-1219:03kevin.lynaghas a consumer, it just wasn't clear to me whether what I wanted to do was in the design space at all#2017-09-1219:04kevin.lynaghI thought it might be, since conceptually it's just further validation#2017-09-1219:04ghadii think you'd use spec to check data at the door, then do extrinsic checks afterwards#2017-09-1219:04ghadii don't think it's a design goal to be a total system#2017-09-1219:04kevin.lynaghAt that point, it's easier to do the entire validation / generation myself, rather than using spec.#2017-09-1219:05kevin.lynaghsince in my domain pretty much all of the incoming data is just ids#2017-09-1219:05ghadithere's so many more benefits you'd be dropping#2017-09-1219:05kevin.lynaghso I need to realize those IDs into entities, then check properties of those entities.#2017-09-1219:05kevin.lynaghditto on the generation front, just getting integers won't help. I need to be able to generate integers that correspond to entities of type X#2017-09-1219:05kevin.lynaghor whatever.#2017-09-1219:06ghadidon't you need to check the entities?#2017-09-1219:06kevin.lynaghunless I'm missing something where spec might still be able to help in here?#2017-09-1219:07ghadiyou're right the IDs are trivial, but I doubt that's the extent of the applicability of spec for your use case#2017-09-1219:07ghadiI'd certainly want something to describe one of the "entities" and get a free generator#2017-09-1219:08ghadiA non-trivial generator#2017-09-1219:08kevin.lynaghI can walk you through a concrete example if you have 5 minutes#2017-09-1219:08ghadigo for it; I might not be able to help, but there are many šŸ‘€ in here#2017-09-1219:09kevin.lynaghcool, thx = )#2017-09-1219:09kevin.lynaghI'm using datascript as the db, so it's all in-memory.#2017-09-1219:09kevin.lynaghthe application architecture is event sourced#2017-09-1219:10kevin.lynaghbasically, I have (step db event) ;;=> new-db#2017-09-1219:10kevin.lynaghbasically, create a new database from the old database, given an event#2017-09-1219:11kevin.lynaghThe events implement a few protocol methods like (valid? [this, db]) which says whether or not the event is valid for a given database.#2017-09-1219:13kevin.lynaghSo, for example, #events.AddChild{:parent-id 5} is a record that represents the event "add a child to node 5". The valid? function needs to look and see if node 5 exists in the db and, if it does, make sure it's the type of node that is allowed to have children#2017-09-1219:13kevin.lynaghAll this make sense so far?#2017-09-1219:18kevin.lynaghAnyway, I don't understand how I can implement valid?w/ spec, since everything depends on having the runtime db value. Likewise, I cannot generate these events with spec, because it'd need to only generate integer IDs that correspond to nodes that can have children.#2017-09-1219:19kevin.lynaghIf wrote some specs to define what it means for a node to be allowed to have children, then I could turn the ID into a datascript entity at runtime and validate the entity using spec.#2017-09-1219:23kevin.lynaghbut it doesn't seem like those specs would help me generate events to test against the system#2017-09-1219:24ghadiyour scenario makes sense. I think the fact that you and spec named something the same thing (valid?) but has a different contract is confusing the situation#2017-09-1219:24kevin.lynaghAh, I only did that once I gave up on spec =P#2017-09-1219:30ghadiYou might make a spec to check a JWT. (s/def :my.security/jwt ....) Doesn't mean that the token is good enough to get through authentication. Spec will check that it is structurally sound#2017-09-1219:31ghadiThe spec might be pretty involved, but you can't have something that is valid? one minute and not the next#2017-09-1219:31ghadispec predicates should not be hooked up to state. a point of the specs is to have enduring, sharable meaning#2017-09-1219:31kevin.lynaghRight, that makes sense#2017-09-1219:32ghadiI wish I had a more succinct rationale. Maybe @alexmiller will chime in later#2017-09-1219:32kevin.lynaghI didn't realize that it was a deliberate decision to prioritize that sharable meaning by disallowing runtime context.#2017-09-1219:33kevin.lynaghyeah, I hope I don't sound cranky about spec at all --- I think it's a really interesting idea and solves some hard problems.#2017-09-1219:33ghadiit doesn't disallow it, it's the runtime context (AKA mutable state) that will work against spec's goals#2017-09-1219:33kevin.lynaghthat's why I'm trying to use it in this context#2017-09-1219:33kevin.lynaghI don't want to rely on implicit runtime context or mutable state#2017-09-1219:34kevin.lynaghI'm just trying to pass that in#2017-09-1219:36kevin.lynaghAnyway, thanks for the chat and all your work on this stuff#2017-09-1219:36kevin.lynaghThere are some other places where I've very happily used spec for validation/generation = )#2017-09-1220:24Alex Miller (Clojure team)I donā€™t have a lot to add to Ghadiā€™s comments which I mostly agree with. Some people are doing runtime generation and registration of specs (by loading values from a db for example). That seems fine to me (it does get a little trickier with s/keys).#2017-09-1220:26Alex Miller (Clojure team)in constrast, specs that use stateful stuff (dynvars, atoms, etc) seem bad to me#2017-09-1220:27Alex Miller (Clojure team)what is spec buying you in this case vs just writing code to assert whatever predicate you have?#2017-09-1221:12kevin.lynagh@alexmiller I have the same kinds of problems that spec solves very well: composing predicates and giving back specific errors about why things failed validation. E.g., the AddChild event I mentioned earlier actually needs to take multiple parent-ids, and it would be great to get the detailed validation errors from spec.#2017-09-1221:13kevin.lynagh"Cannot add child because the third item in the list is of type X, but you can only add children to types U, V, W"#2017-09-1221:13Alex Miller (Clojure team)ok#2017-09-1221:13ghadiyou can easily express that in spec if you write a predicate#2017-09-1221:14ghadiit's the db extrinsic stuff that is a anti-recommendation#2017-09-1221:16kevin.lynagh@ghadi The predicate can only run against reified entities from the DB. Are you suggesting something like (->> parent-ids #(d/entity db %) #(s/valid? (s/coll-of ::parent))?#2017-09-1221:17kevin.lynaghWhere ::parent is a spec that has the relevant predicates?#2017-09-1221:18ghadisure, that's better -- fetch the data before asking spec#2017-09-1221:19ghadi(I was suggesting just building plain, offline predicates)#2017-09-1221:21ghadiif there are different types of nodes, some that have parent-ids and some that don't, you should be able to represent that in the data directly, without querying something external#2017-09-1221:21kevin.lynaghJust to clarify, all of this is happening in an in-memory database, so there is no network fetching or anything like that.#2017-09-1221:21ghadiRight but if that thing can change, YMMV#2017-09-1221:21kevin.lynaghit's immutable.#2017-09-1221:22ghadiok as long as a payload that is valid? right now, is valid? in the future, that's good.#2017-09-1221:22ghadiIf that can flip, bad kitty#2017-09-1221:23ghadii'm assuming you wouldn't call it a database if the data could never change#2017-09-1221:25ghadithis will probably get me in trouble: specs should only check stuff intrinsic in the data.#2017-09-1221:28ghadi
(s/def :my.expiration/date #(re-matches #"\d{4}-\d{2}-\d{2}" %)
Spec can tell you if something conforms to :my.expiration/date... but not if it's expired
#2017-09-1221:28ghadi(aside: are regexes valid specs?)#2017-09-1221:29ghadiSpec will generate you a bunch of :my.expiration/date's, some might be expired, some not#2017-09-1221:30kevin.lynaghAnother way to phrase that question: Should spec allow you to validate {:time #inst"..." :expr #inst"..."}#2017-09-1221:30kevin.lynaghpart of the spec is that both values should be dates#2017-09-1221:30kevin.lynaghbut should you check in spec whether the :expr date is before or after the :time date.#2017-09-1221:31ghadisure#2017-09-1221:31ghadiit's a bad idea to call (Clock/now) while validating the expiration#2017-09-1221:32ghadibecause that's extrinsic#2017-09-1221:32kevin.lynaghYeah, I understand that part#2017-09-1221:32kevin.lynaghI've been drinking the clojure kool aid for a while, don't worry = )#2017-09-1221:32ghadiheh#2017-09-1221:32ghadiThen you should have no problem extrapolating (Clock/now) to a database šŸ˜ƒ#2017-09-1221:33kevin.lynaghcalling (d/entity some-db some-id) essentially turns my integer id into an immutable view into the whole database#2017-09-1221:33kevin.lynaghso reifying the IDs into entities, then calling spec makes sense.#2017-09-1221:33ghadiit's weird though, when you check {:time #inst"..." :expr #inst"..."} it's a totally different entity that you're checking. Something like a token-request#2017-09-1221:34ghadiwe're in sync#2017-09-1221:34ghadii think originally I was thinking the db lookup was happening within the spec definition#2017-09-1221:35kevin.lynaghwell, that was my question, if it'd be possible to do that.#2017-09-1221:35kevin.lynaghbut you can't without dynamic var tomfoolery#2017-09-1221:35kevin.lynaghwhich I don't want to do.#2017-09-1221:35kevin.lynaghIt's still awkward, though, since I can't really spec out my events, since I can't spec their integer-id arguments#2017-09-1221:36kevin.lynaghThe best I can do is write my own validation fn that hydrates the ids into immutable entities, then calls spec on those entities.#2017-09-1221:36kevin.lynaghditto for generation, I can't use spec to generate anything.#2017-09-1221:37kevin.lynaghThe best I could do is, for a given DB, enumerate all entities, see which ones are valid? against a specific spec, then return their ids#2017-09-1221:38kevin.lynaghAnyway, I'll give that a shot and see how tidy I can make things. I may come back in a week or two with a follow up report + gist example#2017-09-1221:38kevin.lynaghThanks again for the conversation!#2017-09-1313:14thedavidmeisterhow would i put together a spec for a map with string keys and values that are vectors of strings?#2017-09-1313:15Alex Miller (Clojure team)(s/map-of string? (s/coll-of string? :kind vector?))#2017-09-1313:19thedavidmeisteris (spec/map-of string? (spec/+ string?)) ok?#2017-09-1313:21Alex Miller (Clojure team)I would recommend coll-of in this case#2017-09-1313:21Alex Miller (Clojure team)it gives you more control over size constraints, generation, etc and is a better match for what you described#2017-09-1313:22thedavidmeisterok cool#2017-09-1313:22Alex Miller (Clojure team)s/+ is for describing sequential collections with internal structure along with the other regex ops. while what you have will work, youā€™re not really using any of those features#2017-09-1313:22thedavidmeisterthanks#2017-09-1318:21bfabryno doubt this has been asked 1k times, but it's probably evolving. what are people currently doing to turn on instrumentation for their test run? top of the file in test_helpers.clj that's required by every test, fixture that's used by every test?#2017-09-1318:44Alex Miller (Clojure team)once fixture seems like a good place for that#2017-09-1318:54bfabrymakes sense#2017-09-1319:35joost-diepenmaat@bfabry weā€™re using :once fixtures,#2017-09-1319:35joost-diepenmaatworks ok, unless you forget to use the fixture šŸ˜•#2017-09-1320:53jumar@bfabry we're using lein profiles and injections - something like this:
:injections   [(require 'clojure.spec.test)
                                             (clojure.spec.test/instrument)
                                             (.println System/err "Instrumented specs")]
#2017-09-1420:18akielIā€™ve created a lib to support things like form validation using spec. I would be super happy if someone can look at it and give me feedback. Thanks! https://github.com/alexanderkiel/phrase#2017-09-1506:59mbjarlandI have a left field question about spec. There are a number of libraries in clojure for describing binary formats (buffy, octet, etc). Essentially you create a "binary spec" which describes the byte layout of your binary format and then ask the library to parse a byte array, byte buffer etc and back come strings, numbers etc parsed data types. The wish list for the language where you create these binary specs is that they be composable, support references from one spec to another (string length stored in another field), be terse etc. It strikes me that this wish list has a lot of overlap with spec. Would it be sane to try to use spec as the language for describing these kinds of binary formats or is it just a bad fit? You would get a fair amount of power for free and in years to come you would be using a language people are already familiar with...so it seems there are some upsides here. One example of a good fit is multi-specs which in spec parse maps of different types depending on a 'type' field. There is a very close correspondence in a lot of binary formats with a type int16 or int32 followed by some binary data conforming to the type specified. Collections also match well. Anyway, somebody please call me crazy and tell me why this doesn't work : )#2017-09-1510:35gklijsIt would be great if you would have a higher-level format describing the data, which could be serialized to bytes, and deserialized to language-specific data-structures of different languages. However it is not easy, I had some experience with avro, https://avro.apache.org/ but even for bigintegers it doesnā€™t work for java out of the box..
#2017-09-1512:40Alex Miller (Clojure team)I do think there is some overlap in goals and you could build custom spec types for this similar to the regex ops part of spec. Byte level stuff has some special considerations so I don't think you'd want to use what's in spec now for that.#2017-09-1516:48cgrand@alexmiller when/if you have time could you explain how my hack ruins s/form? #2017-09-1516:51Alex Miller (Clojure team)if you have doc, then I guess itā€™s fine. really Iā€™d just rather actually fix this.#2017-09-1611:30OlicalAfternoon šŸ™‚ I was wondering what sort of performance I could expect from s/fdef? Like, if I don't instrument, is it just a normal defn?#2017-09-1611:32OlicalAlso, is it "normal" to automatically enable instrumentation during development within a library? Wondering how much spec goodness I can expose to the consumer of the library automatically without the performance hit in production builds.#2017-09-1611:35gfredericks@olical the fdef is independent from the defn until instrumentation, so it should have no effect at all#2017-09-1611:35OlicalAwesome, that's what I was hoping for. Apologies if it was mentioned in the documentation and I didn't spot it.#2017-09-1611:38OlicalI was thinking of wrapping (s/instrument) within some sort of (when dev? ...). Not sure if that's possible or a good idea.#2017-09-1611:39Olical(also I realise I was being dumb, you do defn and fdef next to each other, fdef does not replace defn)#2017-09-1816:11bbrinck@olical Keep in mind you can place the fdef above the defn (which I prefer for readability). Sometimes poeple donā€™t realize this#2017-09-1819:43christianromney@alexmiller love these useful type predicates in clojure 1.9! I'm putting (ex-info?) on my wish list for Clojure Claus... šŸ˜„#2017-09-1819:46Alex Miller (Clojure team)Ticket and patch would be useful#2017-09-1820:14christianromneyhttps://dev.clojure.org/jira/browse/CLJ-2237 as requested! Thanks for considering this...#2017-09-1819:48christianromney@alexmiller will do!#2017-09-1911:56stathissideriswhy does this result in the function to be called several times?
(s/valid? (s/fspec :args (s/cat))
                          (fn []
                            (println 'foo)))
#2017-09-1911:56stathissideris21 times in fact#2017-09-1911:58gfredericksI think it's quickchecking your function to make sure it matches the spec#2017-09-1912:01stathissiderisok, is there any way to say that itā€™s fine if the function throws an exception?#2017-09-1912:02stathissiderisbecause:
(s/valid? (s/fspec :args (s/cat))
                          (fn []
                            (throw (ex-info "foo"))))
#2017-09-1912:05stathissiderisfalse#2017-09-1912:46Alex Miller (Clojure team)no, exceptions should be ā€¦ exceptional#2017-09-1912:47danielnealCongrats on the 1.9 beta release! I'm just wondering how much of clojure is planned to be specced? I'm guessing it's just the macros - or are there plans to spec clojure core functions as well?#2017-09-1912:48Alex Miller (Clojure team)tbd both from a short-term and long-term perspective#2017-09-1912:49Alex Miller (Clojure team)there are several categories of fns that canā€™t currently be instrumented (protocols, multimethods, primitive fns, inline fns) so that limits some things#2017-09-1912:49Alex Miller (Clojure team)Iā€™ve worked on others and I think it is valuable to have them. it is also pretty tricky to spec some of them, esp the higher order stuff#2017-09-1912:50danielnealyeah that makes sense#2017-09-1912:50danielnealso "everything" - probably out of scope - but there are plans over the long term to spec what would be useful, including the functions#2017-09-1912:51Alex Miller (Clojure team)I would hesitantly say yes as there are also performance things to be aware of when you load a large number of specs#2017-09-1912:52Alex Miller (Clojure team)right now, the core.specs.alpha has only macro specs in it and thatā€™s loaded implicitly on first macro expansion#2017-09-1912:52Alex Miller (Clojure team)I think for fns, we may want to put those in a separate ns that you explicitly load if you want to instrument core#2017-09-1912:52danielnealah I see that makes sense#2017-09-1912:53Alex Miller (Clojure team)since core.specs.alpha is a separate lib, this is something that can evolve and be released in between clojure releases too. so if it doesnā€™t get done for 1.9 ga, can still make progress before the next major release#2017-09-1912:53danielnealThanks alex#2017-09-1915:46seancorfield@danieleneal I have almost everything spec'd in clojure.java.jdbc and run the test suite with it instrumentd -- it's quite an overhead. When I get to my desk I can get numbers. #2017-09-1916:59seancorfieldSo I have rough numbers now: clojure.java.jdbc has about 600 assertions. Against 1.8 it takes about 30 seconds (about 14 seconds "real" time). Against 1.9 with instrumentation it takes about 100 seconds (about 70 seconds "real" time). @danieleneal /cc @alexmiller#2017-09-1917:14Alex Miller (Clojure team)what does it take on 1.9 without instrumentation? :)#2017-09-1917:14Alex Miller (Clojure team)comparing apples to orange juice there#2017-09-1917:33seancorfieldLet me just comment out the instrument call to check that...#2017-09-1917:36seancorfieldPretty much identical. 1.8 looks a hair faster but I'd have to run the test suite a lot more times to be certain.#2017-09-1917:36seancorfieldAnd the 30/100 seconds above is "user" (cpu) time. The "real" time is elapsed.#2017-09-1919:53jeayeYeah, jdbc was heavy for me, when instrumenting as well. Much, much heavier than all of our intrumenting combined.#2017-09-1919:53jeayeHad to disable it during development.#2017-09-1919:55seancorfield@jeaye It has specs for all the public API and JDBC stuff tends to get called a lot in DB-centric apps šŸ™‚ If you think it's doing something suspect in its specs that causes undue performance overhead, let me know.#2017-09-1920:47richiardiandreahello folks! is there somewhere a wrapper for passing spec keys directly to generate? I mean without the extra nested call to s/gen, ideally (somewhere/generate :my-ns/key)?#2017-09-1921:15bfabry@richiardiandrea exercise?
boot.user=> (s/def ::foo integer?)
:boot.user/foo
boot.user=> (s/exercise ::foo 1)
([0 0])
#2017-09-1922:40jeaye@seancorfield Will do. I did end up looking, when I saw how long it was taking, and the specs seemed sane to me. It might be worth digging deeper, to see why these specific specs are taking so long. If I had to guess, it's the extensive usage of s/or and s/cat with s/*, wheras just about every spec we have is a s/keys or a clojure.core predicate.#2017-09-2003:43seancorfieldGood point. I might look at refactoring the specs to make them more efficient...#2017-09-2007:04odinodinIs the JVM opt to disable spec asserts still -Dclojure.spec.compile-asserts=false or did this change with clojure.spec.alpha?#2017-09-2007:49mpenetit's a compile time thing#2017-09-2007:49mpenetso doing that at runtime with compiled asserts will do nothing I believe#2017-09-2007:50mpenetotherwise you might be looking for clojure.spec.check-assertsif asserts were compiled#2017-09-2012:44Alex Miller (Clojure team)I believe in both cases we did not put alpha in the property name, but you can check the source#2017-09-2012:47mpenetno alpha yes#2017-09-2014:12wagjoHi, let's say I have a function that takes a nested map and adds a new key somewhere deep.
(defn add-bar
  [m]
  (assoc-in m [:model/foo :model/bar] 42))
Given the spec for the input argument (and its :model/foo key), what's the best practice to spec out the the return value with :model/bar as required key for :model/foo?
#2017-09-2019:40dbushenkoHi all!#2017-09-2101:12richiardiandreaHello! I have probably asked this already at some point in the past, but is there a way to create a Generator that basically uses a function of mine? I am trying with (tcgen/generate (tcgen/->Generator #(faker/name.findName))) but no luck, it always returns nil#2017-09-2101:14gfredericks@richiardiandrea what sort of thing are you trying to generate?#2017-09-2101:15richiardiandreafake data generated from a function...it looks like this works:
(def fake-generators
  {:order/customer #(tcgen/return (faker/name.findName))})
#2017-09-2101:16richiardiandrea(gen/generate (s/gen :order/customer fake-generators))#2017-09-2101:17richiardiandreaI was basically thinking of it in terms of repeatedly#2017-09-2101:17gfredericksif you're doing property-based testing at all, it's generally better to build things from the combinators#2017-09-2101:17gfrederickswhat does faker/name.findName do?#2017-09-2101:17richiardiandreawell, I am not using it for property testing with this specific case#2017-09-2101:18richiardiandreaI am bending it a bit, so that I can generate intelligible data structures#2017-09-2101:19gfrederickstest.check can generate intelligible data structures šŸ™‚#2017-09-2101:20richiardiandreaso now I get a very nice:
{:name "saladman",
 :key "ia",
 :timestamp #inst "1970-01-01T00:00:00.942-00:00",
 :aggregate "order",
 :data {:customer "Velma Doyle"}}
#2017-09-2101:22gfredericksis this because you want to use the generators for friendly examples?#2017-09-2101:22richiardiandreayep exactly#2017-09-2101:22gfredericksI almost feel like that should be a different piece of functionality all together#2017-09-2101:26gfredericksmaybe not; I guess it's just the strings that were bothering you?#2017-09-2101:28richiardiandreayes I completely agree with you, it could be a separate piece functionality, but generators by definition can be a good fit for that as well#2017-09-2101:29richiardiandreaif I have a restricted set of things I can use tcgen/elements and that's nice, but for names...something like the above works well#2017-09-2111:01gfredericksNow I'm thinking this belongs in spec, if anywhere. A function called s/example which will use custom code if registered and fall back on the generator otherwise#2017-09-2118:35bbrinckIf often find myself redefining fdefs a few times to get them right and AFAICT, you need to call instrument after each redefinition. Has anyone else run into this? I suppose I could define a new macro named fdef! that did both steps and then just remove the ! before I commited the code?#2017-09-2118:36bbrincksome sort of auto-instrumentation mode that worked as I redefined things would be handy. Has anyone written something like this?#2017-09-2120:40danielcompton@bbrinck I have instrument called in my user namespace that is reloaded after all of the other dependencies are loaded#2017-09-2120:41danielcomptonSo whenever I reload the code, (with clojure.tools.namespace) there is an instrument call at the end#2017-09-2120:41danielcomptonI still haven't figured out what's the best way to do this for testing though#2017-09-2120:43bbrinckAh, youā€™re right clojure.tools.namespace would be useful here. Thanks!#2017-09-2121:15danielcomptonI've got a lazy way of doing it, which is just to put the instrument call directly in the user ns, but a fancier way to do it would be to attach it to the c.t.n.r/refresh :after handler#2017-09-2200:29jeayeGiven a fn like (defn foobar ([a] (inc a)) ([a b] (str a b))) where arity 1 expects a number and, let's say, arity 2 expects two strings, can I spec this so that arity 1 doesn't accept a single string?#2017-09-2200:30jeayeEverywhere I see people spec'ing multiple arities, they'll just use s/or. That leaves all sorts of open holes.#2017-09-2200:30jeayeI'd see something like this: (s/fdef foobar :args (s/cat :a (s/or :int integer? :str string?) :b (s/? integer?)))#2017-09-2200:30jeayeBut that permits (foobar "bad")#2017-09-2200:39seancorfield(s/fdef foobar :args (s/or :one (s/cat :a number?) :two (s/cat :a string? :b string?)))#2017-09-2200:41jeayeDamnit, that worked perfectly. Cheers, @seancorfield. I think I had tried s/alt there, with two s/cats, but it failed.#2017-09-2200:41seancorfields/alt will alternate multiple sequence regexes <-- this is poorly worded but I'm not quite sure how to describe its interaction with s/cat#2017-09-2200:42jeayeThis would be a good thing to have in the spec guide, I think; multiple arities aren't covered at all.#2017-09-2200:44seancorfieldIt probably doesn't help that the docstrings for s/alt and s/or look so similar. Seems to be a common point of confusion.#2017-09-2200:44jeayeYep, agreed.#2017-09-2200:49seancorfieldI just tried it with s/alt in place of s/or and it worked -- but the spec failure you get for (foobar "a") is different...
boot.user=> (foobar "a")

clojure.lang.ExceptionInfo: Call to #'boot.user/foobar did not conform to spec:
                            In: [0] val: "a" fails at: [:args :one :a] predicate: number?
                            val: () fails at: [:args :two :b] predicate: string?,  Insufficient input
                            :clojure.spec.alpha/spec  #object[clojure.spec.alpha$or_spec_impl$reify__810 0x55365ffb "
compared to
boot.user=> (foobar "a")

clojure.lang.ExceptionInfo: Call to #'boot.user/foobar did not conform to spec:
                            val: () fails at: [:args] predicate: (alt :two (cat :a string? :b string?)),  Insufficient input
                            :clojure.spec.alpha/spec  #object[clojure.spec.alpha$regex_spec_impl$reify__1200 0x373ea576 "
#2017-09-2200:50seancorfieldIn the first case, it fails to match :args :one :a because "a" is not a number? and it also fails to match :args :two :b because only one string was provided.#2017-09-2200:51seancorfieldIn the second case, it only reports the failure to match on alt :two (again, because only one string was provided).#2017-09-2200:51seancorfieldThe spec I used was (s/fdef foobar :args (s/alt :one (s/cat :a number?) :two (s/cat :a string? :b string?)))#2017-09-2200:53seancorfield^ @jeaye#2017-09-2201:15jeayeAh, I was getting the second one, but I was also likely testing with invalid inputs and ... bah, I don't remember. Awesome to see it works well; thanks again.#2017-09-2201:16jeayeNot having to calculate the various alternations between arities for use with s/or is going to make my macro writing significantly simpler as well.#2017-09-2205:11jeayeIs there a good spec floating around for specs themselves? They can be ifn? but also things like s/coll-of return something else. I've also tried s/spec?.#2017-09-2211:50Alex Miller (Clojure team)See https://dev.clojure.org/jira/browse/CLJ-2112 for wip#2017-09-2211:50Alex Miller (Clojure team)For spec forms that is#2017-09-2211:51Alex Miller (Clojure team)Currently spec instances should just be considered to be opaque and respond to true for s/spec?#2017-09-2217:03jeayeAwesome, thanks. Good to see there's work being done here.#2017-09-2205:36seancorfieldThat would be a good question for Alex when he's around tomorrow @jeaye#2017-09-2207:17jeayeAlright, I'll bug him in the morning.#2017-09-2208:12dbushenkowill Alex be today?#2017-09-2209:23vikeriI canā€™t seem to get my mutimethods to become intstrumented. Anyone else also having this issue?#2017-09-2209:39rovanionMultimethods are weird, one of those things I sometimes restart my repl for and it just works.#2017-09-2209:48andrewmcveigh@vikeri see above (hope the link works)#2017-09-2209:51vikeri@andrewmcveigh Ok thanks!#2017-09-2218:43richiardiandrea@gfredericks is it possible to write a generator that continues to generate until the sum of an operation on the generated stuff meets a condition?#2017-09-2218:43richiardiandreafor instance generate integers until their sum is 5#2017-09-2218:46tbaldridge@richiardiandrea that doesn't make a lot of sense, since there's a almost unlimited set of numbers that would sum up to 5#2017-09-2218:46gfredericksyou want a collection of integers in particular?#2017-09-2218:46richiardiandrearight, let's say I restrict the domain#2017-09-2218:47gfredericksand you want the sum to be =5, or <=5?#2017-09-2223:25richiardiandreaI see now the reason of this question, I am basically now generating a list of things such that their sum <= total randomly, if < then the last element is manually crafted#2017-09-2301:11gfredericksyeah, that's definitely a legit way to do it#2017-09-2218:47richiardiandreabasically I would want to have a such-that, but where I accumulate results first, then I apply pred on the result#2017-09-2218:47gfredericksone idea is to generate larger sequences and fmap it to the largest prefix that meets your criteria#2017-09-2218:47gfrederickswould that be sufficient?#2017-09-2218:48richiardiandreauhm, let me try that, I will be back with the result šŸ˜„#2017-09-2319:43jstewWhere is the canonical place to put specs? I read that clojure.core has them in a clojure.core.spec namespace . I'm thinking of making a separate namespace of .spec for each namespace in my app. But I will have some common specs (boilerplate stuff), but don't know where to put those. I'm not sure it matters since all of the defined specs are global.#2017-09-2320:03gfredericksclojure is canon-poor#2017-09-2320:57bbrinckI donā€™t know if there is a standard, but Iā€™ve been putting them in the same namespace as the functions that manipulate the data. That way, other namespaces can require a single ns and get functions+specs#2017-09-2322:04jstew@bbrinck What about specs that are shared between namespaces? Say you have a spec definition for a core.async channel or a map structure that's shared all over your app, do you duplicate that everywhere or keep those sorts of things in a single namespace#2017-09-2322:07bbrinckIn that case, Iā€™d probably extract to a common ā€œspecsā€ namespace like youā€™ve suggested.#2017-09-2322:08bbrinckItā€™s kind of like generic util functions. Iā€™ll put them in a generic ns until some natural grouping emerges, then extract a more well-defined ns. YMMV#2017-09-2513:12acronI notice that a value which returns :clojure.spec/invalid under conform does not necessarily return false from a valid? call - is this reasonable? Am I misunderstanding something about valid??#2017-09-2513:16acron
user> (spec/valid? (spec/conformer #(if (string? %) :clojure.spec/invalid %)) "an invalid string")
true
#2017-09-2513:20Alex Miller (Clojure team)itā€™s :clojure.spec.alpha/invalid now#2017-09-2513:21Alex Miller (Clojure team)
user=> (s/valid? (s/conformer #(if (string? %) ::s/invalid %)) "an invalid string")
false
#2017-09-2513:22Alex Miller (Clojure team)Iā€™d recommend using the auto-resolved keyword here ^^ as this will change again when spec eventually comes out of alpha#2017-09-2513:22acron@alexmiller picard-facepalm thanks#2017-09-2613:22benhI want to generate strings of length 64. This code does the job, but seems a bit unwieldy. Is there a nicer way to do this?
(gen/generate (gen/fmap clojure.string/join 
                        (gen/vector clojure.test.check.generators/char-ascii
                                    64)))
#2017-09-2613:23mpenetsame question as yesterday, I dont think so no#2017-09-2613:23gfredericksif you like gen/let better syntactically, you can
(gen/let [chars (gen/vector test.check.generators/char-ascii 64)]
  (apply str chars))
#2017-09-2613:24gfredericksmodulo getting all the namespaces right#2017-09-2613:24mpenetI guess we could have something better in test.check for this#2017-09-2613:25mpenetin the same vein as gen/vector#2017-09-2613:25gfredericksSomething specialized for strings? #2017-09-2613:25mpenetyeah#2017-09-2613:26gfredericksThe whole string thing needs an overhaul if anybody can think of a reasonable approach to unicode#2017-09-2613:26mpenetI know we can abuse string-for-regex in test.chuck for some of it#2017-09-2613:31gfredericksThat is a good example of the issues. Try generating matches for #".*" and see how much you like it#2017-09-2616:36uwois it likely or unlikely that s/def will accept docstrings in the future?#2017-09-2617:55Alex Miller (Clojure team)Likely#2017-09-2617:13bfabryI wish there was a way to mark a function as impure in an fdef/fspec that had the consequence of turning on :ret validation for instrumentation and turning off validation of things against that fspec#2017-09-2617:16fedregHi all, is there a way in spec to define a map with something like this: (s/def ::my-map :req-un [::name ::id (one-of ::role ::title)]?? ...So the data could be {:name "tom" :id "1" :role "mgr"} or {:name "tom" :id "1" :title "ceo"} for example. Thx!#2017-09-2617:18bfabryso xor?#2017-09-2617:19bfabryit's been a while but what little is left of my Maths for CS class in my brain is saying you can't create an xor with just or and and. so I'd say you'd have to just use or + a predicate#2017-09-2617:21fedregI need an s/or but for the keys instead of just the vals#2017-09-2617:23bfabrywell s/keys supports or, so (s/def ::my-map :req-un [::name ::id (or ::role ::title)]) is perfectly valid#2017-09-2617:24bfabrymy point was just that {:name "tom" :id "1" :role "mgr" :title "ceo"} would be valid with that spec (as would your two examples)#2017-09-2617:25fedregahh, missed that. Thanks for clarifying. Didn't realize it was that simple. Thx!#2017-09-2617:27bfabrynp#2017-09-2620:00mrchanceHi, is there an automated way to get a spec for the output of conform, called with another given spec?#2017-09-2620:03mrchanceI mean:
> (def sp (s/cat :x double? :y double?))
> (s/conform sp [1.0 2.5])
{:x 1.0, :y 2.5} <- I'd like a spec describing this map
#2017-09-2620:15Alex Miller (Clojure team)no, although thatā€™s been asked a few times.#2017-09-2620:18mrchanceThanks! So if I wrote one myself, how would I check that they fit together? generative tests?#2017-09-2620:20Alex Miller (Clojure team)sure!#2017-09-2620:20mrchanceOk, will play with it more, thanks šŸ™‚#2017-09-2701:28waffletowerIs there a better way to parameterize generators than to use macros? In this case, Iā€™d like to reuse a generator in different specs with different parameters:
(defmacro limited-string-m [lim]
  `(gen/such-that (fn [s#]
                    (not (blank? s#)))
                  (gen/sized
                   (fn [size#]
                     (gen/resize (dec ~lim) gen/string-alphanumeric)))))
#2017-09-2710:56gfredericks@waffletower at a glance that looks like a macro that could trivially be a function you don't generally need to write any macros to build generators#2017-09-2711:38stathissideris@mrchance maybe you could do it with https://github.com/stathissideris/spec-provider#2017-09-2711:45mrchance@stathissideris oh nice, that looks like a helpful recommendation in any case!#2017-09-2716:24waffletower@gfredericks Thanks! works as a function as well#2017-09-2716:24waffletower
(defn limited-string-fn [lim]
  (gen/such-that (fn [s]
                    (not (blank? s)))
                  (gen/sized
                   (fn [size]
                     (gen/resize (dec lim) gen/string-alphanumeric)))))
#2017-09-2716:45gfredericks@waffletower btw, if you want to retain the gradual-growth property of the builtin generator, you could replace (gen/resize (dec lim) ...) with (gen/scale #(min % (dec lim)) ...)#2017-09-2716:46waffletowerthanks, I had noticed that side effect#2017-09-2802:28shark8meHi! is there a (short) way to reference a spec defined in a different namespace?
(ns a.b.c 
   (:require [cljs.spec.alpha :as s]))
(s/def ::foo int?)
;;; in a different ns
(ns a.d 
 (:require [cljs.spec.alpha :as s])
(s/valid :a.b.c/foo 10) 
Is there a way to avoid typing the fully qualified :a.b.c/foo by using :require :as ?
#2017-09-2802:38gfredericks(require [a.b.c :as a-really-short-name-that-doesn't-take-a-while-to-read]) ::a-really-short-name-that-doesn't-take-a-while-to-read/foo @shark8me#2017-09-2802:40shark8meThanks! šŸ™‚#2017-09-2802:41shark8meI couldn't find this documented, I assumed it would be :ns/foo instead of ::ns/foo#2017-09-2806:54jumarNo, it's not, the proper syntax really is ::specs-ns/my-spec#2017-09-2806:20witekHello. I have a function: (defn f [m]). I expect m to be a map containing a value unter the key :name. The value has to be a keyword. How do I spec this function using (s/fdef f :args #!%&)? What do I have to put in :args?#2017-09-2807:01jumar@witek If I understand your problem, then s/keys should work just fine:
(s/def ::name keyword?)
(s/valid? (s/keys :req-un [::name]) {:name :john}) ;=> true
(s/valid? (s/keys :req-un [::name]) {:name "john"}) ;=> false
#2017-09-2807:04jumarOr rather for the function:
(defn f [m]
  (:name m))

(s/fdef f
        :args (s/cat :m (s/keys :req-un [::name])))
#2017-09-2807:26hkjelsit seems that coll-of messes with ordering, how do I specify a sorted set of maps#2017-09-2809:57curlyfry@hkjels What do you mean by "messes with ordering"? Do you mean when conforming?#2017-09-2812:00hkjels@curlyfry yeah#2017-09-2815:50bfabrykinda seems like a sorted-map? predicate that takes the comparator makes sense#2017-09-2817:43souenzzomaybe (s/coll-of string? :kind set? :sorted true). It will be able to sort map, set, vec, list...#2017-09-2823:35jeaye:sorted? true -- now I'm just nitpicking#2017-09-2900:02johanatans/gen (or s/with-gen) seems to unnecessarily use such-that#2017-09-2900:02johanatancausing its associated spec's gen to be usable for only a few elements#2017-09-2900:02johanatan
(s/def ::kebab-str
  (s/with-gen
    #(= %1 (csk/->kebab-case %1))
    #(gen/fmap clojure.string/join (gen/list (gen/frequency [[1 (s/gen #{\-})] [9 gen/char-alpha]])))))
#2017-09-2900:03johanatan[`csk` is camel-snake-kebab]#2017-09-2900:03johanatan
core> (s/gen ::kebab-str)
#clojure.test.check.generators.Generator{:gen #function[clojure.test.check.generators/such-that/fn--17362]}
#2017-09-2900:03johanatanthe such-that above is suspicious#2017-09-2900:04johanatanand the result is usuable only for a few elements:
core> (gen/sample (s/gen ::kebab-str) 10)
("" "" "i" "rc" "sh" "s" "gh" "q" "" "od")
#2017-09-2900:07johanatanand not for many:
core> (gen/sample (s/gen ::kebab-str) 100)
ExceptionInfo Couldn't satisfy such-that predicate after 100 tries.  clojure.core/ex-info (core.clj:4725)
#2017-09-2900:29seancorfield@johanatan Generators always validate against the spec. Your generator probably generates values that do not satisfy your predicate.#2017-09-2900:30seancorfieldWhat does your predicate return for "-a" or "---"?#2017-09-2900:37johanatan@seancorfield ah, good call. looks like first digit needs to be non-dash#2017-09-2900:39johanatan@seancorfield interesting that I wasn't getting a more obvious exception though (which I did get with a more egregious disparity between the validator and generator)#2017-09-2900:47johanatanFYI... this version works:
(s/def ::kebab-str
  (s/with-gen
    #(= %1 (csk/->kebab-case %1))
    #(gen/fmap (comp clojure.string/lower-case clojure.string/join)
               (gen/such-that (fn [c] (and (not-empty c) (not= \- (first c)) (not= \- (last c))))
                              (gen/list (gen/frequency [[1 (s/gen #{\-})] [9 gen/char-alpha]]))))))
#2017-09-2901:01seancorfieldYou probably ought to look at Gary Fredericks' test.chuck library -- it contains a generator for regex strings which would make this a lot simpler!#2017-09-2901:03seancorfieldAside from anything else, you could then define ::kebab-str as a regex predicate and use test.chuck's regex generator directly on the same regex.#2017-09-2902:20amarHello! Any ideas on what Iā€™m doing wrong here. (s/valid? ::bar-path ["foo" 1]) is ok, but not so in an s/fdef
(s/def ::foo-path
  (s/cat :foo (partial = "foo")))

(s/def ::bar-path
  (s/cat :foo ::foo-path :idx nat-int?))

(s/valid? ::foo-path ["foo"]) ; => true
(s/valid? ::bar-path ["foo" 1]) ; => true

(s/fdef my-fn
        :args (s/cat :path ::bar-path)
        :ret int?)

(defn my-fn
  [path]
  (second path))

(stest/instrument)
(my-fn ["foo" 1]) ; => my-fn did not conform to spec
#2017-09-2905:34seancorfield@amar The s/cat calls combine to make one sequence regex. I think you can say (s/spec ::bar-path) as the :path argument spec and it will work.#2017-09-2905:35seancorfieldYou might also want to look at s/tuple for your ::bar-path spec instead of s/cat.#2017-09-2911:57amarseancorfield: Thanks. The following worked.
(s/fdef my-fn
        :args (s/cat :path (s/spec ::bar-path))
        :ret int?)
#2017-09-3017:44eggsyntaxI feel as though I remember reading that it's OK to have advance spec definitions (ie to define specs whose s/def refers to a spec defined later in the same ns. Can anyone confirm or deny that?#2017-09-3019:16Alex Miller (Clojure team)yes (although there are some known cases where that does not yet work right)#2017-10-0209:21borkdudeIs it me or is clojure.future.spec not checking macro specs at compile time?#2017-10-0211:04gfredericksit couldn't without patching some monkeys, right? I have no idea if it attempts to do that or not.#2017-10-0221:27ajsIs it typical to use spec during runtime in production apps, or is more of a debugging tool in development to ensure data is flowing as expected in your app, but turned on off in production?#2017-10-0221:34bfabryturned off in production#2017-10-0221:34bfabryexcept possibly in very specific spots (system boundaries etc)#2017-10-0221:34bfabrywe use it in development, and in our master and staging environments#2017-10-0221:39guyHow big a performance inpact is it to have it running?#2017-10-0221:48ajsit would seem to me that spec/conform is designed with runtime use in mind for coercing data into a desired format, is this not accurate?#2017-10-0221:49bfabry@ajs no it absolutely is designed for that, those are the "specific spots" I was talking about#2017-10-0221:50bfabryie it'd be odd to be validating/conforming data somewhere other than a system boundary, unless you're looking for bugs in your own code#2017-10-0221:50bfabrywhich is what instrument/test.check are for, but they're expensive so just use in lower envs#2017-10-0221:50ajsis something like goog.debug used for turning it off in production, or is there a spec-specific way to do so#2017-10-0221:51bfabryyou still need to validate/conform at system boundaries though, because you can't trust other people's code#2017-10-0221:52bfabrythere's a few different levers to tweak. instrument is used to add input validation to every function with an fdef. so just not calling that in production. spec/assert is a more explicit validation check, which can be disabled using compile-asserts or clojure.spec.check-asserts#2017-10-0311:06dbushenkohi all!#2017-10-0311:07dbushenkoI'm specifying an anonymous function, and its return value does not match the spec. Instead of normal error report or something it says like this: clojure.lang.ExceptionInfo: Can't convert path {:form (#function[.../fn--20377/fn--20384]), :val {}, :in [0], :in' []}#2017-10-0311:07dbushenkois it normal or I'm doing something wrong?#2017-10-0321:51bbrinckIt sounds like a bug in Expound? https://github.com/bhb/expound/blob/b9fc0d46dc3534bd77cdfebcf771cfecf163f058/src/expound/paths.cljc#L160-L166 If you have a repro and some time, a bug report would be appreciated šŸ™‚#2017-10-0313:01Alex Miller (Clojure team)hmm, not sure. repro?#2017-10-0315:00dbushenkosorry? do you want a gist?#2017-10-0315:04Alex Miller (Clojure team)yeah, want to know how to reproduce that error#2017-10-0315:05Alex Miller (Clojure team)and then if you can follow it with a (pst *e)#2017-10-0315:05dbushenkowhat is this (pst *e) ?#2017-10-0315:07guyprevious stack trace ?#2017-10-0315:08guymy guess#2017-10-0315:08Alex Miller (Clojure team)print stack trace#2017-10-0315:08guydang#2017-10-0315:08Alex Miller (Clojure team)it is a function in clojure.repl#2017-10-0315:08dbushenkoknow what, looks like spec went ok, it was test who produced this strange error report...#2017-10-0315:08dbushenkošŸ˜ž#2017-10-0315:09Alex Miller (Clojure team)good, because I didnā€™t know where the ā€œconvert pathā€ error was coming from :)#2017-10-0315:14dbushenko@alexmiller are there any plans to support specing the protocol implementations in records?#2017-10-0315:17Alex Miller (Clojure team)not near term. If you could spec them, Iā€™m not sure you have the opportunity to actually do useful stuff with those specs.#2017-10-0315:25dbushenko@alexmiller maybe you remember I tried to invite you to our conf in December and you couldn't make it šŸ™‚ Can you probably recommend someone from clojure core developers who might be able to visit us with a talk?#2017-10-0315:28Alex Miller (Clojure team)lots of fine people - take a look at any of the recent clojure confs (EuroClojure, Clojure/west, Clojure/conj, ClojureX, clojureD, clojuTRE, etc)#2017-10-0315:28dbushenkook, thanks#2017-10-0315:28Alex Miller (Clojure team)maybe post something here in #events, on the mailing list, on the reddit, etc#2017-10-0318:37jonasIs there a function in spec similar to spec/assert but that conforms the value? (spec/assert ::a-spec x) returns x (or throws) and not the conformed value of x.#2017-10-0318:39jonasPerhaps the reason is that you can set *compile-asserts* to false which results in all assertions to compile to x?#2017-10-0318:49souenzzoinstrument?#2017-10-0318:41bfabry@jonas that would seem like a good reason why that wouldn't work#2017-10-0507:39dbushenkohi all!#2017-10-0507:39dbushenkodo we have any means of specifying records?#2017-10-0507:41mpenets/keys should just work on them#2017-10-0508:17dbushenkothanks!#2017-10-0515:19guyWhats the best way to test a spec'd function that has side effects?#2017-10-0516:04jaymartin@guy Iā€™m not sure about the best way, but mocking can help speed things along if thatā€™s what youā€™re after. https://github.com/benrady/specific#2017-10-0516:26guyThanks @jaymartin have you tried it with the latest version of clojure.spec.alpha (17) ?#2017-10-0516:26guyit looks like specific is using alpha13#2017-10-0518:05jaymartinNo, Iā€™ve just messed about with it playfully. But the ideas should be rather generic.#2017-10-0518:08guykk thanks šŸ‘#2017-10-0522:20aengelbergIs there an equivalent of declare for specs?#2017-10-0522:20aengelbergalternatively, is there a way to disable the check to see if a spec name exists when used in other specs?#2017-10-0522:24aengelbergbasically I would like to structure my code in the following order:
(s/def ::x ::y)
(s/def ::y ...)
#2017-10-0522:24aengelbergreplacing the first ::y with (s/and ::y) seems to work.#2017-10-0523:08bfabryfeels kinda odd that it doesn't just work, seeing as most of things in spec are lazy evaluated#2017-10-0523:08bfabryfeels kinda odd that it doesn't just work, seeing as most of things in spec are lazy evaluated#2017-10-0602:07Alex Miller (Clojure team)There is a ticket about this. The intent is that it should work #2017-10-0615:35bfabrygood to know, thanks alex#2017-10-0619:59aengelbergOh cool, I'm glad that that is the intent#2017-10-0523:36guy@aengelberg what do you mean#2017-10-0523:38bfabry@guy he means
boot.user=> (s/def ::foo ::bar)
clojure.lang.Compiler$CompilerException: java.lang.Exception: Unable to resolve spec: :boot.user/bar, compiling:(boot.user1422927036988753503.clj:1:1)
#2017-10-0523:39guySorry im probably being slow#2017-10-0523:39guyhe wants to use a different spec and call it something new?#2017-10-0523:43bfabryhe wants to say that the spec for ::foo is the same as the spec for ::bar, and it be ok that the spec ::bar has not been defined yet#2017-10-0523:44bfabry@aengelberg when I've wanted that I've just used (s/def ::bar any?)#2017-10-0620:02aengelbergthat works too. thanks#2017-10-0607:39guykk thanks @bfabry#2017-10-0609:16kennethkalmerI need some advice, Iā€™ve hit a strange situation while trying to writes specs for data coming in from an external sourceā€¦ The source returns a map that roughly translates to this: {:productStatus {:productStatus "active" ...} ...}#2017-10-0609:17kennethkalmerhow on earth would I spec :productStatus the map, and the string value? I have no control over the output generated by the service#2017-10-0609:18mpeneteither you overspecify by creating separate nses for each, or use something more generic like map-of, if it's only one key its fine, if that s a lot of keys that s so so#2017-10-0609:23kennethkalmerthink separate nsā€™s are maybe the way to go here, will test that out, thanks!#2017-10-0609:28kennethkalmerthanks @mpenet! simple, it works, donā€™t know why I got myself stuck in another loop#2017-10-0617:58jaymartinThis may be highly subjective, but will Spec replace :pre and :post conditions as the idiomatic way to validate inputs and outputs? For example, something like this:
(ns rna-transcription)


(def m
  {\G  \C
   \C  \G
   \T  \A
   \A  \U})


(defn to-rna [xs]
  {:pre [(every? #(contains? m %) xs)]}
  (apply str (map m xs)))
#2017-10-0618:00jaymartinOr will :pre and :post still have their place and time?#2017-10-0618:04bfabryI could still see myself using :pre and :post to check non-local invariants, but that'd be pretty uncommon#2017-10-0618:23seancorfield@jaymartin You could define specs for your DNA/RNA stuff and use s/valid? in :pre...
(s/def ::dna m)
(s/def ::dna-sequence (s/coll-of ::dna))
(defn to-rna [xs]
  {:pre [(s/valid? ::dna-sequence (seq xs))]}
  (apply str (map m xs)))
#2017-10-0618:24guythat looks nice#2017-10-0618:24seancorfieldor go further and define ::dna-string as (s/and string? #(s/coll-of ::dna (seq %))) and then {:pre [(s/valid? ::dna-string xs)]}#2017-10-0618:27seancorfield(I hardly ever use :pre or :post, I must admit -- but I'm an advocate of leaving assertions on in production if you're going to use them at all)#2017-10-0618:27guywhat do you think about instrument in production? šŸ‘€#2017-10-0618:28seancorfieldinstrument is a different beast, not least because it can trigger generative testing on higher-order function.#2017-10-0618:29seancorfieldIn addition, instrument can introduce a big performance overhead depending on how much is spec'd and how they're written. The specs for java.jdbc are a case in point: they introduce a huge overhead.#2017-10-0618:29jaymartinSuper helpful!#2017-10-0618:29seancorfield(Hmm, that reminds me that I forgot to add :keywordize? to the optional specs in release 0.7.3!)#2017-10-0618:30guythanks!#2017-10-0618:47jeaye@guy We're using it in production.#2017-10-0618:47jeayeGiven our specs, I've measured that we spend about 2% of our time with instrumentation.#2017-10-0618:48jeayeAs Sean said, that can very wildly, depending on how the specs are written. Primarily whether or not there are a lot of alternate paths which need to be considered.#2017-10-0618:48jeayeFor us, it's almost entirely clojure.core predicates and s/keys though, so it ends up being very fast and entirely practical for our production builds.#2017-10-0619:40guynice thanks @jeaye!#2017-10-0707:18ajsSince instrument does not check the return value, it would seem to me that a :post condition is still quite useful.#2017-10-0708:10mpenetI tend to use s/assert instead nowadays #2017-10-0709:08ajswith :post at least, you donā€™t have to let your fnā€™s return value so you can assert it before you return it#2017-10-0709:25ajsdoes s/valid? work on all components of fspec?#2017-10-0709:26guywhat do you mean
#2017-10-0709:27guyoh right got you#2017-10-0709:27guyhttps://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec/fspec#2017-10-0709:29ajsi know that instrument only looks at args, but i havenā€™t tried out how valid? works on it. will try that when i get home.#2017-10-0709:29ajswould be cool if that worked#2017-10-0709:32guywell instrument works on the return value too ?#2017-10-0709:32guythats why u have :args, :ret :fn#2017-10-0711:59ajs@guy no it doesn't. Are you sure? Check the docs. Only args are supported for instrument, though you can specify ret and fn for other purposes. #2017-10-0712:01ajs"Note that the :ret and :fn specs are not checked with instrumentation..."#2017-10-0712:08jaymartin@ajs Where exactly are you seeing that note, as I canā€™t find it.#2017-10-0712:09ajsThe spec guide itself, in instrument section. Why do you think libraries like Orchestra exist?#2017-10-0712:10jaymartinSilly me, I was looking in the doc string for instrument.#2017-10-0712:11jaymartinSo then, is fdef orthogonal in some way. Its doc string says: ā€œOnce registered, function specs are included in doc, checked by instrument, tested by the runner clojure.spec.test/run-tests, and (if a macro) used to explain errors during macroexpansion.ā€#2017-10-0712:12jaymartinI think I need to start over at the spec guide itself and re-read.#2017-10-0712:12ajsThe are tested by instrument for :args only I think#2017-10-0713:39Alex Miller (Clojure team)correct#2017-10-0714:00ajsOther than tests, I gather there's really no way to perform validation on the entirety of a function's signature, including its return value, and also validating that the entire fn received in a higher order function, is what is expected, is that right?#2017-10-0715:50Alex Miller (Clojure team)stest/check is the only thing in spec that validates :ret and :fn specs#2017-10-0715:51Alex Miller (Clojure team)Iā€™m not sure I fully understand the question re hof#2017-10-0716:25ajsThat answers it!#2017-10-0713:47taylorIā€™m trying to write a spec for a collection where I need the first item to conform to another spec. s/cat works great, but when I fdef and check an unary function of that collection it looks like test.check is applying the collection to the function, and so I get arity exceptions. Should I change the function to take varargs?#2017-10-0713:48gfredericksare you using s/cat to express the whole arglist or just the first arg?#2017-10-0713:49gfredericksI think to use a regex for a single arg you'll need to wrap in s/spec, maybe that's what you're missing#2017-10-0713:49gfredericks(s/cat :first-arg (s/spec (s/cat ...))) something like that#2017-10-0713:49tayloryeah, my :args is (s/cat :clause (s/spec ::group))#2017-10-0713:50gfredericksI don't know then. You certainly should be able to spec that.#2017-10-0713:56tayloractually it does work when I wrap with s/spec, but I think I need to limit the test cases because itā€™s running forever#2017-10-0713:57taylorthe spec is recursive and I guess is getting really complex samples#2017-10-0713:57gfredericksyeah šŸ˜ž the Big Problem of spec/test.check#2017-10-0713:59taylorwell the good news is this is a toy problem šŸ™‚#2017-10-0714:13taylor
(binding [clojure.spec.alpha/*recursion-limit* 1]
  (stest/check `clause-str {:num-tests 1}))
#2017-10-0714:13tayloris ā˜ļø what recursion limit is there for?#2017-10-0714:15tayloror I could read the docsā€¦ https://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec/*recursion-limit*#2017-10-0714:13taylorit runs pretty quickly like that#2017-10-0714:18gfredericksI think so. what's it default to?#2017-10-0714:20tbaldridgeThe depth of spec checking is somewhat limited by the JVM stack size, that keeps the generators from blowing the stack. Without it, youā€™d get a stack overflow from almost any recursive structure.#2017-10-0714:25gfredericks@tbaldridge you're saying *recursion-limit* refers to checking rather than generation?#2017-10-0714:28tbaldridgeIt only refers to generation. Checking is assumed to be bounded by the depth of the input#2017-10-0714:29gfredericksokay. I think there could be a more automatic approach to handling the recursion, but I'm not sure.#2017-10-0714:29gfredericksI have to come up with something for the vanilla test.check collection generators, so whatever works there might apply to how spec assembles generators#2017-10-0714:30tbaldridgeThere probably could be. That dynamic var got added in one of the pre-alpha builds when I tried to build a recursive spec and it would stack overflow on the first structure generated.#2017-10-0714:31tbaldridgeChecking is fine since that's GIGO, so it's on the user not to check data that's millions of levels deep.#2017-10-0714:31tbaldridge@gfredericks any insight in the past year about shrinking recursive generators?#2017-10-0714:32tbaldridgeThey currently shrink by width (and a b) -> (and a), but not by depth: (and (and (and a))) -> (and a). I know we discussed this once, not sure if anything has changed in the past year#2017-10-0714:33gfredericksI don't think I have any thoughts that I didn't put on the ticket#2017-10-0714:34gfrederickshttps://dev.clojure.org/jira/browse/TCHECK-110#2017-10-0714:59taylorHa. This is exactly the type of structure Iā€™m generating too#2017-10-0715:04gfredericks@U3DAE8HMG important to note here that spec doesn't use recursive-gen#2017-10-0715:12gfredericksBut spec's approach has the same problem, but even worse#2017-10-0714:35gfredericksthe unfortunate part is that after generating a collection, test.check can't (in general) tell what parts the children are#2017-10-0714:36gfredericksand you'd need their shrink-trees in any case, not just the data#2017-10-0714:37gfredericksoh ummmm#2017-10-0714:39gfredericksI was thinking that there's an unsound approach where you wrap the child generator in an observer that keeps track of how it was called and use that to add shrinking info I thought it was unsound because you can't tell just because the generator was called, that the result actually appears in the final structure but I just realized that that probably doesn't matter. anything generated by the child generator should be a valid value from the final generator, so even if it doesn't appear in the actual data, it's not an invalid thing to shrink to in the strict sense#2017-10-0714:41gfredericksso now the worst thing about this approach is that it's a bit messy#2017-10-0714:42gfredericksbut I think everything still ends up being deterministic#2017-10-0714:42gfredericksI'll go comment on the ticket. I don't have time to work on it until at least thursday#2017-10-0714:48gfredericks@tbaldridge ā‡‘#2017-10-0715:12gfrederickshttps://clojurians.slack.com/archives/C1B1BB2Q3/p1507389135000078?thread_ts=1507386899.000005&amp;cid=C1B1BB2Q3#2017-10-0715:14gfredericksWell maybe not worse....#2017-10-0715:30tbaldridgeah right, I forgot about that. Spec's stack overflow happened during the creation of the generator#2017-10-0715:48gfredericksah yeah#2017-10-0715:49gfredericksthe thing about spec's approach is that it's committing to a particular depth in the generator rather than describing the recursion the way the spec itself does#2017-10-0715:49gfredericksit might be making other commitments in the process, I'm not sure#2017-10-0715:49gfredericksI'm not sure this is a significant problem, it just complicates things#2017-10-0801:04taylorthanks for the advice & discussion! FWIW hereā€™s the code I was working with https://gist.github.com/taylorwood/232129ccd3cb809281fea591d46f1b8a#2017-10-0910:05jwkoelewijnHi all, does someone know if there is a way to limit generators? I have created specs with generators, and generating a top-level entity now takes multiple seconds, which in turn seems to limit me from using generative testing#2017-10-0911:06gfredericks@jwkoelewijn I'm not familiar with all the entry points, but test.check has a :max-size option that spec will pass through, that you could set to 50 or 15 depending on what you're doing exactly when the multiple-second thing happens#2017-10-0911:13jwkoelewijncool, will look into that options, thanks!#2017-10-1001:50erichmondIs there a good doc/faq/something on using spec's ability to generate mock data? Generating a string like "001212314" in particular? (meaning, n length, chars consisting of just 0-9?#2017-10-1002:47Alex Miller (Clojure team)spec itself does not really have something like that built in#2017-10-1002:47Alex Miller (Clojure team)you can use fmap to build a set of characters, then apply str to them#2017-10-1002:48Alex Miller (Clojure team)or use the regex gen support in the test.chuck library#2017-10-1003:09erichmondI had been using test.chuck, but was curious if there was a more spec-native solution. thanks, I'll keep going with what I have!#2017-10-1003:31Alex Miller (Clojure team)test.chuck is rad#2017-10-1013:53jwkoelewijnquestion, when i run (clojure.spec.test/check 'merge-users {:clojure.spec.test.check/opts {:num-tests 15 :max-size 2}}) it seems the num-tests option seems to be ignored. Am I passing the options for quick-check wrong? or is this to be expected and should I address my own expectations? šŸ™‚#2017-10-1013:55jwkoelewijnwhere I use the backtick instead of the quote (to help in markdown slack formatting)#2017-10-1014:06taylor
(stest/check `foo {:clojure.spec.test.check/opts {:num-tests 1}})
this works for me
#2017-10-1015:13borkdudeIn Clojure spec, how could I generate symbols or strings of max length 2 that contain only lowercased alphabetic characters? This works, but maybe it can be simplified?
(s/def ::varname
  (s/with-gen symbol?
    #(gen/fmap (fn [[i j l]]
                 (symbol
                  (str/lower-case (.substring
                                   (str i j)
                                   0 l))))
               (gen/tuple (gen/char-alpha)
                          (gen/char-alpha)
                          (gen/choose 1 2)))))
#2017-10-1015:14gfredericksmin length 1?#2017-10-1015:14borkdudeyes#2017-10-1015:14gfredericksI don't think that's terrible#2017-10-1016:28nwjsmith
(s/def ::varname
  (s/with-gen symbol?
              #(gen/fmap (fn [characters]
                           (symbol
                             (string/lower-case
                               (apply str characters))))
                         (gen/vector (gen/char-alphanumeric)
                                     1
                                     2))))
#2017-10-1017:05jonasWith spec/keys (or spec in general), whatā€™s the best way to express: One and only one of the keys ::foo or ::bar is required?#2017-10-1017:06jonasOr alternatively ā€œEither ::foo or ::bar or both is requiredā€#2017-10-1017:06bfabry(s/keys :req [(or ::foo ::bar)]) and then and'd with a spec to enforce only one if you want that#2017-10-1017:07jonasThanks, let me try that#2017-10-1017:07bfabry
The :req key vector supports 'and' and 'or' for key groups:
(s/keys :req [::x ::y (or ::secret (and ::user ::pwd))] :opt [::z])
#2017-10-1017:07jonasI think this is exactly what I was looking for :+1:#2017-10-1020:40borkdude@nwjsmith Thanks šŸ™‚#2017-10-1020:47borkdude@nwjsmith https://github.com/borkdude/aoc2015_day7/commit/936987c86451d9032e28e6151f0dd061ab58a0be#2017-10-1020:48nwjsmithCrazy, I was just reading your blog post. Great stuff!#2017-10-1023:59nwjsmithHow are you folks debugging Couldn't satisfy such-that predicate errors?#2017-10-1100:00nwjsmithIā€™m having trouble figuring out which generator is causing the error#2017-10-1100:06bfabry@nwjsmith one option would be to walk the registry using s/registry and try and generate an example for each one until you fail#2017-10-1100:06bfabryI believe there's a jira open to figure out a way to add the offending spec to the error message you're seeing#2017-10-1100:07gfrederickss/and is a common culprite#2017-10-1100:08gfredericksthe way to add the offending spec to the error message is implemented in the latest alpha of test.check, but I doubt has been incorporated into spec. I assume there's an open ticket for that.#2017-10-1100:12nwjsmith@bfabry @gfredericks thanks!#2017-10-1100:12nwjsmithIā€™ve narrowed my current run-in with that error down to this:
(s/def ::header-name
  (s/with-gen
   (s/and string?
          (complement string/blank?)
          utilities.string/lower-case?
          utilities.string/ascii?)
   #(gen/fmap (comp string/lower-case (partial string/join "-"))
              (gen/vector (gen/not-empty (gen/string-ascii))
                          1
                          100))))
#2017-10-1100:16nwjsmithAh, figured it out#2017-10-1100:16nwjsmithmy utilities.string/ascii? function is broken#2017-10-1100:18gfredericksphew#2017-10-1107:27msolliIā€™m specing a function that calls another function, stubbing out that second function with instrument. When Iā€™m doing stest/check on the function, itā€™s really slow. Takes ~30 s with :num-tests 1000. With num-tests 100, itā€™s sub-second. What gives?
(require '[clojure.spec.alpha :as s])
(require '[clojure.spec.test.alpha :as stest])

(defn fetch-foo-from-somewhere
  [a]
  [,,,])

(defn get-foo
  [b]
  (fetch-foo-from-somewhere b))

(s/def ::foo string?)
(s/fdef fetch-foo-from-somewhere
        :args (s/cat :a any?)
        :ret ::foo)
(s/fdef get-foo
        :args (s/cat :b any?)
        :ret ::foo)

(stest/instrument `fetch-foo-from-somewhere {:stub #{`fetch-foo-from-somewhere}})

(stest/summarize-results (stest/check `get-foo {:clojure.spec.test.check/opts {:num-tests 1000}}))
#2017-10-1110:58gfredericks@msolli how about {:num-tests 1000 :max-size 100}?#2017-10-1111:07msolliYeah, that took the time down to ~4 s.#2017-10-1111:09msolliSo I understand :max-size somehow controls the way generated values ā€œgrowā€ as the number of tests increase. How does this explain why larger sizes takes so much more time?#2017-10-1111:09gfredericksit's probably this: https://dev.clojure.org/jira/browse/TCHECK-106#2017-10-1111:10gfredericksprobably stemming from the use of any? in this case#2017-10-1111:12gfredericksI suppose you could argue that the generator underlying any? should be designed to never get very big#2017-10-1111:16msolliAh, I see. Iā€™m just getting into Spec, and Iā€™m following along to https://clojure.org/guides/spec#_combining_code_check_code_and_code_instrument_code. It seemed strange that the official docs endorse something this slow.#2017-10-1111:17msolliI can confirm that the values generated for any? really do become huge.#2017-10-1111:20msolliSo I guess itā€™s the GC-ing of those large objects that is the bottleneck.#2017-10-1111:22Alex Miller (Clojure team)There are some tickets and work to do in this area still#2017-10-1111:25Alex Miller (Clojure team)@gfredericks the most critical fix was the bug in s/coll-of that went into the spec release last week actually. That was the cause of one set of problems (not the one here though).#2017-10-1111:27gfredericks@alexmiller this thing? https://github.com/clojure/spec.alpha/commit/739c1af56dae621aedf1bb282025a0d676eff713#2017-10-1111:27gfredericksI can't see anything that looks particularly relevant in the last 6 months of commits#2017-10-1111:28gfredericksoh was it generating a bunch of stuff and throwing it away?#2017-10-1111:28msolliThe conj, well, enjoy yourselves! And thanks for the explanation and all the great work youā€™re doing.#2017-10-1111:32gfredericks@alexmiller after reading the ticket, I assume CLJ-2103 is what you were referring to Any thoughts (ha!) about any?? The underlying gen/any-printable in test.check could have its size scaled down. I feel like that would make it more useful for almost any conceivable use case.#2017-10-1111:38Alex Miller (Clojure team)Yeah that would be good#2017-10-1111:39Alex Miller (Clojure team)And yes 2103#2017-10-1111:41gfredericksalthough now that I think about it, making sizing better in collections and gen/recursive-gen might take care of the gen/any-printable problem#2017-10-1113:30mmerIs there anyway to get the context of a element being validated by a spec. I need to get hold of the overall datastructure that is being validate by a set of specs so that I can do some cross validation in my own predicate?#2017-10-1114:13taylorI commented on your Stack Overflow question too. I think maybe the solution to your problem is to enforce that invariant from your outer spec instead of your inner spec?#2017-10-1114:20taylorposted example on SO#2017-10-1114:32taylor
(s/def ::tag string?)

(s/def ::inner (s/keys :req-un [::tag]))

(s/def ::outer
  (s/and
    (s/keys :req-un [::inner ::tag])
    #(= (:tag %) ;; this tag must equal inner tag
        (:tag (:inner %)))))

(s/conform ::outer {:tag "y" ;; inner doesn't match outer
                    :inner {:tag "x"}})
;=> :clojure.spec.alpha/invalid

(s/conform ::outer {:tag "x"
                    :inner {:tag "x"}})
;=> {:tag "x", :inner {:tag "x"}}
#2017-10-1114:33taylorhttps://stackoverflow.com/questions/46685778/clojure-spec-accessing-data-in-hierarchical-spec/46690677#46690677#2017-10-1115:46mmerThank you - I kind of understand. I need to work through some details.#2017-10-1122:05mmerWhat seems strange is that with the whole conformed model that it is not easier to walk the model. In other words it woul dbe good to be able to apply the equivalent of an xpath to the specs to be able to find associated data and perform link and checks between them#2017-10-1119:13ikitommiHi. Wrote an suggestion of the spec coercion: https://dev.clojure.org/jira/browse/CLJ-2251#2017-10-1121:45dealytrying to spec a function that takes 2 params, a keyword and a set of string (which can be nil), but my spec always fails when a set is passed to the function, how should it be spec'd#2017-10-1121:46dealy(s/fdef cs! :args (s/cat :mis keyword? :table-names (or nil? (s/coll-of string?)))) is the spec I was using#2017-10-1121:50bfabryyou want (s/or (s/coll-of string? :kind set?) nil?) but really you want (s/nilable (s/coll-of string? :kind set?)#2017-10-1204:24uwoIs there a way to ask of a spec, ā€œwhat keys can you have?ā€#2017-10-1204:25uwoI know we can call (s/describe :my/spec) and get back a form. I would need to parse that s expression to get that answer - which is fine of course#2017-10-1214:48guyHi i've got a newbie spec question. Should you reuse specs from one ns to another. Say i have some data that has :name :email and wanted to validate it with spec. Would i create a ::name ::email in each namespace that checks it, or would i create a spec namespace and import it from there. Or should i do something else?#2017-10-1215:29taylorYes, itā€™s normal and sometimes necessary to use specs across namespaces. If you have common/shared specs then you should definitely consider defining them in one place and referencing them where theyā€™re needed.#2017-10-1215:31guyok thanks!#2017-10-1218:51the2bearsHi, I'm starting out with spec and adding it to a small project we have. However, the project uses Incanter and a 'lein compile' now fails.
Caused by: clojure.lang.ExceptionInfo: Call to clojure.core/defn did not conform to spec:
 {:clojure.spec.alpha/args (matrix "\n  Returns a matrix or vector, in a valid core.matrix format. You can use the slices function to\n  access the rows.\n\n  Equivalent to R's matrix function.\n\n  Examples:\n    (def A (matrix [[1 2 3] [4 5 6] [7 8 9]])) ; produces a 3x3 matrix\n    (def A2 (matrix [1 2 3 4 5 6 7 8 9] 3)) ; produces the same 3x3 matrix\n    (def B (matrix [1 2 3 4 5 6 7 8 9])) ; produces a vector with 9 elements\n\n    ; since (plus row1 row2) adds the two rows element-by-element\n    (reduce plus A) ; produces the sums of the columns\n\n    ; and since (sum row1) sums the elements of the row\n    (map sum A) ; produces the sums of the rows\n\n  " ([data] (m/matrix data)) ([data ncol &] (m/matrix (partition ncol (vectorize data)))) ([init-val rows cols] (m/compute-matrix [rows cols] (constantly init-val))))}
#2017-10-1218:51the2bearsI imagine Incanter has not been updated, but I also don't read the spec errors very well yet.#2017-10-1218:54bfabrywhat version of incanter?#2017-10-1218:54bfabryand do you have more error/stacktrace?#2017-10-1218:54the2bears1.9.0#2017-10-1218:55the2bearsI do have more trace, hold on#2017-10-1219:00the2bearsShoot, that cut off some#2017-10-1219:02guyu cud try putting it in a gist#2017-10-1219:03the2bearsyeah, would be better I think.#2017-10-1219:06bfabryah, yes, bug in incanter#2017-10-1219:06bfabrythis is not a valid args list
([data ^Integer ncol &]
     (m/matrix (partition ncol (vectorize data))))
#2017-10-1219:08bfabryI'll open a PR#2017-10-1219:08the2bearsThanks!#2017-10-1219:09bfabryoh actually it's fixed in master#2017-10-1219:09bfabrytry the latest snapshot maybe#2017-10-1219:09the2bearsokay, I can do that for testing this out. Thanks again for the help.#2017-10-1219:09bfabryoh no I'm on the wrong branch#2017-10-1219:09bfabry1s#2017-10-1219:10bfabryyeah fixed on 'develop' branch#2017-10-1219:12the2bearsokay, I'll pull it down and test a local.#2017-10-1309:49mishagreat talk, @gfredericks, thank you! now we need to persuade you into writing a book on test.check and gen testing#2017-10-1312:03Alex Miller (Clojure team)that is a fantastic idea#2017-10-1501:24codonnellam I missing something about conformers? I'm confused as to why (s/valid? (s/conformer (fn [_] :clojure.spec/invalid)) nil) is true. If the conformer function returns :clojure.spec/invalid, shouldn't the value be invalid?#2017-10-1501:27codonnellah, it should be clojure.spec.alpha/invalid or just ::s/invalid :duck:#2017-10-1507:26jumarwhat exactly is the relationship/difference between clojure.spec.gen.alpha and clojure.test.check.generators? So far, I've found two differences: 1. missing nat generator in clojure.spec.gen.alpha 2. cannot reference generators directly, but has to call them as functions: e.g. with clojure.test.check.generators I can do this:
(gens/hash-map
   :boolean gens/boolean
   ;; note: `nat` generator is not in spec 
   :small-integers (gens/vector gens/nat)
   :large-integer  gens/large-integer
   :double         gens/double
   :color          (gens/elements [:red :green :blue])
   :uuid           gens/uuid
   :string-or-keyword (gens/tuple gens/string gens/keyword))
But with clojure.spec.gen.alpha I have to do this:
(gen/hash-map
   :boolean (gen/boolean)
   ;; note: `nat` generator is not in spec 
   :small-integers (gen/vector gens/nat)
   :large-integer  (gen/large-integer)
   :double         (gen/double)
   :color          (gen/elements [:red :green :blue])
   :uuid           (gen/uuid)
   :string-or-keyword (gen/tuple (gen/string) (gen/keyword)))
#2017-10-1507:34jumarclojure.test.check.generators/let is also missing in clojure.spec.gen.alphanamespace#2017-10-1507:37jumarRelated to the 2.) When I actually try to use (gens/double) it throws an exception so the usage pattern is a bit inconsistent#2017-10-1513:28gfrederickswhat kind of exception?#2017-10-1518:13jumar
1. Unhandled java.lang.ClassCastException
   clojure.test.check.generators.Generator cannot be cast to clojure.lang.IFn
#2017-10-1518:14jumarI don't have anything against either approach, I just thought that they behave in the same way O:-)#2017-10-1518:37gfredericksoh, you mean between the two namespaces it's inconsistent yes absolutely, and that's unfortunate#2017-10-1507:51jumarbtw. @gfredericks thanks a lot for the great Conj talk. I missed it at the conference but watched it now. Really useful!#2017-10-1507:58jumarjust a small thing. imho, gen/large-integer doesn't accept min/max options. I had to use gen/large-integer* (slide 43). Funny thing is that if you use clojure.spec.gen.alpha/large-integer it doens't throw an exception but generates wrong data#2017-10-1513:28gfredericksthat's disappointingly error-prone#2017-10-1513:29gfredericksthe other thing is definitely a slide typo šŸ˜ž#2017-10-1513:30gfredericksmy org-modeā‡’beamer workflow is not yet sophisticated enough to automatically verify that the code works#2017-10-1513:48stathissideriswhat do people normally do for specā€™ing var-args#2017-10-1513:48stathissiderisdo they put the whole thing in a s/?#2017-10-1513:49stathissiderisit feels slightly wrong because that means that itā€™s all or nothing#2017-10-1513:51gfredericks@stathissideris does s/* not work?#2017-10-1513:54stathissiderisok some context: Iā€™m trying to extend spec-provider to support inference of function specs, and when it comes to varargs, if I see an example of a function call where the varargs happen to be 1 "foo" Iā€™m not sure if I should infer a spec of (s/cat ... :rest (s/? (s/cat :integer integer? :string string?)))#2017-10-1513:54stathissideriswhich says that there are up to 2 varargs but theyā€™re both optional#2017-10-1513:55stathissiderisfeels like itā€™s more restrictive than the function signature#2017-10-1513:55gfredericksif all you know is one example call, I don't think you can know very much at all for sure#2017-10-1513:56stathissiderisagreedā€¦#2017-10-1513:56stathissiderisbut even if I get more calls, Iā€™m not sure whatā€™s the ā€œbestā€ spec to infer#2017-10-1513:56stathissiderisfor example letā€™s say I get one more example where the varargs is just 20#2017-10-1513:57stathissiderisdoes it then become (s/cat ... :rest (s/? (s/cat :integer integer? :string (s/? string?))))#2017-10-1513:58stathissiderisor maybe I always wrap the individual var-args with s/? because theyā€™re optional#2017-10-1514:01gfredericksthere's no right answer, it's all heuristics#2017-10-1514:01stathissiderisI know šŸ™‚#2017-10-1514:03gfrederickswhich I guess means I have no further advice šŸ™‚ those kinds of problems make me just throw up my hands and give up#2017-10-1514:04stathissideristhey are hard, but they make you think hard about idioms and coder expectations#2017-10-1514:05stathissiderisI had to do a lot of that for spec-provider#2017-10-1514:05stathissideris@gfredericks great talk btw#2017-10-1514:05gfredericksthanks#2017-10-1514:54stathissideriswhatā€™s the difference between s/alt and s/or?#2017-10-1514:54stathissiderisI saw Ambrose used alt for multiple arities#2017-10-1514:54stathissiderisIā€™ve always used or#2017-10-1514:55taylor
;; unlike `or`, `alt` works on sequence elements, not on values themselves!
#2017-10-1514:56taylorthe example here https://clojuredocs.org/clojure.spec/alt makes that a little more obvious#2017-10-1515:00stathissiderisok, so itā€™s like with alt you have an implicit s/cat around each option#2017-10-1515:00stathissiderisok, so itā€™s like with alt you have an implicit s/cat around each option#2017-10-1515:08taylorI think it's more like all the "regex" spec types (`cat`, alt, *, +, ?) are specifically meant to specify elements of a sequence. I wouldn't say there's an implicit cat around alt or any of the regex specs, they all work on sequences whether you use them alone or in combination#2017-10-1515:11taylorI'm probably missing some context for your use case though? Writing specs for multi-arity functions?#2017-10-1515:18taylorThe fdef docstring has a nice example:
(s/fdef clojure.core/symbol
  :args (s/alt :separate (s/cat :ns string? :n string?)
               :str string?
               :sym symbol?)
  :ret symbol?)
#2017-10-1515:18taylorUsing an alt case for each arity#2017-10-1515:19stathissiderisok, but apart from conciseness, whatā€™s the benefit over s/or in this case?#2017-10-1515:19stathissideris(Yes, Iā€™m writing specs for multi-arity functions)#2017-10-1515:22taylor
(s/fdef clojure.core/symbol
        :args (s/or :separate (s/cat :ns string? :n string?)
                    :str (s/cat :str string?)
                    :sym (s/cat :sym symbol?))
        :ret symbol?)
#2017-10-1515:22stathissiderisyeah, itā€™s longer, but are there any other disadvantages?#2017-10-1515:27taylorNot sure what other disadvantages there might be. The :args spec isn't just longer with or, it's more complex. They also produce slightly different output e.g. when incorrectly calling a instrumented function#2017-10-1515:28taylorThe or version has additional, unnecessary tags in its explanation#2017-10-1515:30taylorfrom the :clojure.spec.alpha/problems: :path [:args :sym] vs :path [:args :sym :sym]#2017-10-1515:36Alex Miller (Clojure team)Regex specs compose together to describe a single sequential context and are generally preferred for describing args#2017-10-1515:37Alex Miller (Clojure team)Generally it's best to use a top level cat as you will then get a map conformed to component names that you can use in the :fn spec#2017-10-1515:38stathissideris@U064X3EF3 but are different arities a ā€œsingle sequential contextā€?#2017-10-1515:38Alex Miller (Clojure team)For any given case, yes#2017-10-1515:38Alex Miller (Clojure team)I would actually write that example now as#2017-10-1515:39Alex Miller (Clojure team)Sorry, I can't type it all, on the phone#2017-10-1515:43stathissiderisno problem, would be grateful to have your example when you get some time#2017-10-1515:44Alex Miller (Clojure team)Use alt for alternatives, ? for optional args, * for varargs etc. you can describe any set of arities I've encountered with a regex spec#2017-10-1515:45stathissiderisalright, thanks#2017-10-1515:45stathissiderison a bit of tangent: what about keyword args?#2017-10-1519:34Alex Miller (Clojure team)Use keys* - that's what it's for#2017-10-1608:21stathissiderisok, many thanks!#2017-10-1613:58bbrinckIā€™m unclear on how conformers are intended to be used.#2017-10-1613:59bbrinckIā€™ve seen cases like: - converting a string into a seq so spec regex operators can validate a string - converting a string into an int - converting a string into a UUID#2017-10-1614:01bbrinckBut from my understanding of https://dev.clojure.org/jira/browse/CLJ-2116, conformers arenā€™t intended to be used for transformation#2017-10-1614:02bbrinckAre the above cases unintended uses of conformers? What would be an intended use case? I ask because Iā€™m trying to figure out how to include support for conformers in Expound#2017-10-1615:31Alex Miller (Clojure team)the intended use case was for writing new composite spec types (like s/keys*)#2017-10-1615:31Alex Miller (Clojure team)so basically not what most people try to use them for#2017-10-1907:41jrychterFWIW, I also use them for things like converting strings to keywords in JSON.#2017-10-1907:42jrychterI think that while this might not have been the intended usage, it shows that there is a need here. I am having hopes that Rich will take this on as something to think about in the future.#2017-10-2322:13bbrinckFor what itā€™s worth, using conformers in this way also makes it harder to give good error messages. Iā€™m not sure what the best solution is, but perhaps two phases where first the data is validated, then transformed#2017-10-1614:03mpenetto give you information about what you are validating for reuse "later", this makes more sense in the context of validating/destructuring data passed to a macro such as defn#2017-10-1614:05bbrinckDo you happen to have an example? My main source of examples of macros specs is https://github.com/clojure/core.specs.alpha, but I donā€™t see any conformers there#2017-10-1614:07mpenetmy bad I read conforming and I got carried away#2017-10-1614:10mpenetI am guilty of using it for transformation, not sure my example is clean but here you go: we have a dsl a bit like sql for a rule engine, we validate the syntax with spec and use a conformer to spill the ast (parsed form)#2017-10-1614:14bbrinck@mpenet Hm, that is interesting. Is the DSL originally a string then? Do you use spec to parse the string?#2017-10-1614:14mpenetalso guilty of using it do to json transforms, "old" stuff#2017-10-1614:14mpenetyes#2017-10-1614:14mpenetthe spec is basically a conformer that wraps the parse fn#2017-10-1614:14bbrinckRight#2017-10-1614:15mpenetin reality it s a lot hairier because we have custom explain and whatnot but that s the idea#2017-10-1614:15bbrinckI see, thanks for the example. One reason itā€™s hard to add support for conformers is that a) Iā€™m unclear on the intended use and b) in practice, it seems like they are often uses for transformation. Iā€™m not sure if itā€™s practical to not support a common use case.#2017-10-1614:15bbrinckRight#2017-10-1614:16mpenetit s made for transformations, I guess their validity depends on the context#2017-10-1614:21bbrinckThis is the part I donā€™t fully understand: conformers do transform values, but what are examples of recommended transformations? @alexmiller said: ā€œI donā€™t think we are interested in turning spec into a transformation engine via conformersā€, but Iā€™m not sure what that includes/excludes#2017-10-1614:22mpenetit's a good question indeed#2017-10-1614:24mpenetneither the guide or rationale pages mention it#2017-10-1614:24bbrinck@mpenet In any case, I appreciate you explaining how you use them in practice. This is very helpful for my research, since I may end up wanting to support different use cases#2017-10-1614:25mpenetyou're welcome. love expound. thanks for that#2017-10-1615:32Alex Miller (Clojure team)the guide page intentionally does not mention conformers as we consider them to be primarily useful for writing new custom composite spec types (not for general data transformation)#2017-10-1615:44bbrinckBy ā€œcomposite spec typesā€ do you mean new spec types that work similar to or or regex specs? i.e. where I would want to name the parts?#2017-10-1619:11bbrinckWhoops, sorry I missed your comment addressing exactly this question in the thread above. Thanks!#2017-10-2010:31carocad@alexmiller would you be so kind to put an example of the ā€œexpected useā€ in ? šŸ™‚ Currently only the ā€œunintented useā€ is documented so I guess that this leads people in the wrong direction (and might continue to) http://clojuredocs.org/clojure.spec/conformer#2017-10-1615:33Alex Miller (Clojure team)I think having a tool for transforming data informed by specs would be awesome. I donā€™t think conformers are that tool.#2017-10-1617:24ikitommi@alexmiller any ideas how could we find such a tool? It looks like we could do that with conformers, with just small changes to how it works. Or is there something new coming from Rich?#2017-10-1615:50danielnealI'm trying to use spec to document a map that contains a :key-fn that returns a collection i.e. on this data {:aliases #{:bob :robert}}, {:key-fn :aliases} would be suitable. However this happens
(s/def ::key-fn
  (s/fspec :args (s/cat :item any?)
           :ret (s/coll-of any?)))

(s/def ::my-map (s/keys :req [::key-fn]))

(s/valid? ::my-map {::key-fn :aliases}) ;;false
(s/valid? ::key-fn :aliases) ;; false
Am I doing this all wrong?
#2017-10-1615:50danielnealI'm trying to use spec to document a map that contains a :key-fn that returns a collection i.e. on this data {:aliases #{:bob :robert}}, {:key-fn :aliases} would be suitable. However this happens
(s/def ::key-fn
  (s/fspec :args (s/cat :item any?)
           :ret (s/coll-of any?)))

(s/def ::my-map (s/keys :req [::key-fn]))

(s/valid? ::my-map {::key-fn :aliases}) ;;false
(s/valid? ::key-fn :aliases) ;; false
Am I doing this all wrong?
#2017-10-1615:56taylorI think the problem is that a keyword (`:aliases`) wonā€™t conform to your fspec#2017-10-1615:57taylorYou could add an s/or case for keywords in your ::key-fn spec#2017-10-1615:57danielneal#2017-10-1615:57taylor(s/or :kw keyword? :fn (s/fspec ...))#2017-10-1615:58danielnealthe thing is it's the fact that the function returns a collection that is the important thing I'm trying to capture#2017-10-1615:58danielnealI wouldn't mind if I have to help it along with a generator or something#2017-10-1615:59danielnealbut I don't know where or how or if I'm approaching the whole thing in a stupid way#2017-10-1616:00danielnealoh no wait#2017-10-1616:00danielnealthe inlined function with constants worked#2017-10-1616:01danielnealI would like to specify that if you're gonna give a keyword it's got to look up data that is a collection#2017-10-1616:05taylorhmm maybe need another approach if you need that guaranteeā€¦ a keyword could return anything that happens to be in the map#2017-10-1616:08danielnealmm yes#2017-10-1616:09danielnealas far as I can make out specifying the return types of functions <as> parameters is a bit of an edge case#2017-10-1616:09danielnealbut would love to be able to provide it to help readers understand the contract#2017-10-1616:10taylorso I guess youā€™d need more context if you were going to validate that the supplied keyword would return a collection; youā€™d need to know 1) what the keyword is and 2) the value for that keyword in the other map itā€™d be applied to?#2017-10-1616:14danielnealyeah you're right#2017-10-1616:14danielnealI suppose I could just put it in the doc string#2017-10-1616:15taylorone weird idea: if you had all this info (including the ā€œsubjectā€ map) in one data structure, you could write a spec for the whole structure to ensure any ::key-fn refers to a collection value in the ā€œsubjectā€ map#2017-10-1616:15danielnealooh interesting#2017-10-1616:32taylorAnyway, the return value spec for your function isnā€™t going to matter for valid?. Itā€™d be useful if you were using test.check to check the function though #2017-10-1620:38akhudekIs there a way to write a spec that checks properties of pairs of elements in a sequence? E.g. in [{:a 1 :b 2} {:a 2 :b 3} {:a 3 :b 4}] Iā€™d like to check that :b is equal to :a for each pair of items from the sequence.#2017-10-1620:38akhudekI can do this as a function on the sequence.#2017-10-1620:39akhudekBut then on failure explain returns the entire sequence as the value causing the problem.#2017-10-1620:39akhudekerm, let me fix that#2017-10-1620:40ghadiyou can do it but not as a function on the sequence, but each element of it#2017-10-1620:40ghadiunless I understand you wrong#2017-10-1620:41ghadiDo you mean there is some relationship between adjacent maps? or within individual maps?#2017-10-1620:42akhudekI mean that for {:a 1 :b 2} and {:a 2 :b 3} :b of the first and :a of the second must match, and then for {:a 2 :b 3} and {:a 3 :b 4}, we have the same check and so on#2017-10-1620:42ghadigotcha#2017-10-1620:42ghadiyou have a tradeoff here#2017-10-1620:43ghadiYou can individually check pairs by running (map #(s/valid? ....) (partition 2 1 collection))#2017-10-1620:43ghadiiff you want more targeted error messages#2017-10-1620:44ghadiotherwise you can continue to check the whole collection and get errors at the collection level#2017-10-1620:44ghadidepends on what you want#2017-10-1620:44akhudekok, thanks, thatā€™s what I thought but wanted to make sure there wasnā€™t some other way šŸ™‚#2017-10-1620:44ghadiI don't believe that (at this time) there is a have-your-cake-and-eat-it-too...#2017-10-1705:51talgiatAre there ways to validate a form with spec, with user friendly error messages (not developer friendly). Iā€™ve found this: https://github.com/alexanderkiel/phrase but I donā€™t want to do field by field validation but validate a whole spec that is a map with field dependent error messages.#2017-10-1713:54bbrinckCan you talk more about the map of fields? Do you have an example?#2017-10-1706:24seancorfield@talgiat See if this might be closer to what you want https://github.com/bhb/expound#2017-10-1712:07tatuthow do I override a generator for something thats deep within some nested maps, without doing s/def on that thing?#2017-10-1712:07tatutam I missing something#2017-10-1713:36Alex Miller (Clojure team)You can override by path
#2017-10-1713:37Alex Miller (Clojure team)But it is tricky to find exactly the right path to use #2017-10-1722:24taylordo you need the same spec to work for both those test cases, or just the first?#2017-10-1722:29taylor
clojure
(s/def ::field
  (s/cat :field-name string?
         :field-type #{'Currency 'Text}))
(s/def ::entity
  (s/cat :entity-name keyword?
         :fields (s/+ (s/spec ::field))))
(s/valid? ::entity '(:columns ("Price" Currency) ("Description" Text)))
;;=> true
#2017-10-1722:31taylorcould replace s/+ with s/* if you donā€™t require anything after the initial keyword#2017-10-1722:32fentonJust the first#2017-10-1722:33fentonOk I'll try that thanks#2017-10-1809:59joost-diepenmaathow can I combine two specs like s/or but without the labels so it conforms as a single value?#2017-10-1810:14andrewmcveigh@joost-diepenmaat you could wrap it in a s/nonconforming spec https://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/alpha.clj#L1761#2017-10-1810:17joost-diepenmaatthatā€™s interesting. thanks @andrewmcveigh#2017-10-1817:39seancorfield@joost-diepenmaat bear in mind that s/nonconforming is undocumented and may disappear. I believe the Clojure/core folks have said the most likely "solution" to this is that they may provide a specific non-conforming version of s/or, rather than exposing general non-conforming machinery... /cc @andrewmcveigh#2017-10-1817:47ikitommicalling s/unform on the conformed value will also remove the branching info.#2017-10-1817:49ikitommis/nonconforming also stops all conforming, so the conformers on the or branches are not run.#2017-10-1817:50joost-diepenmaatI could possibly use a multi spec. For now I could make it work with a single spec #2017-10-1907:46jrychterI am having problems with circular dependencies. Spec strongly encourages splitting your data entities into separate namespaces. I have projects (`project`) which contain entries (`entry`). But entries also refer back to :project/id via a :entry/project-id attribute. If I split entries into their own namespace, I have to refer to it when defining what a project is, but the entry namespace also needs to know what a :project/id is.#2017-10-1914:01Alex Miller (Clojure team)the qualifiers used in spec names do not have to correlate to an actual namespace nor do they have to be defined in that namespace. The word ā€œnamespaceā€ is dangerously overloaded in this area.#2017-10-1914:09Alex Miller (Clojure team)I think itā€™s most useful to declare stuff like this in some totally different ā€œdomain dataā€ namespace#2017-10-1915:36jrychterInteresting. But if they don't correlate to namespaces, I lose the (rather convenient) :: syntax. I guess the namespaced map reader syntax could make up for that.#2017-10-1915:45bbrinckschec lets you create aliases to non-existing namespaces, and then you could use the alias to shorten writing specs#2017-10-1915:45bbrinckhttps://github.com/gfredericks/schpec#things-it-has#2017-10-1917:30Alex Miller (Clojure team)you can do this without a library too#2017-10-1917:31Alex Miller (Clojure team)
(create-ns 'my.cool.thing)
(alias 't 'my.cool.thing)
::t/foo
#2017-10-1917:32Alex Miller (Clojure team)support for handling keyword aliasing in better ways (without doing that ^^) is something Rich is working on for future release btw#2017-10-1907:53jrychterAlso, I'm not sure about the new reader syntax for namespaced maps, specifically how it should work with destructuring:
(let [{a :some.ns/a} {:some.ns/a 1}] a) => 1
(let [#:some.ns{:keys [a]} {:some.ns/a 1}] a) => 1
(let [#:some.ns{a :a} {:some.ns/a 1}] a) => error
Is map destructuring with key renaming not supported?
#2017-10-1911:15gfredericksa problem with that last one is that the binding form is read in as {some.ns/a :a} which is sort of backwards from what you want, because the symbol gets namespaced but not the key#2017-10-1912:29jrychter@gfredericks Right. But what now, do we give up key renaming? Is that expected/intended?#2017-10-1912:32gfredericksyou can do it the long way, at worst#2017-10-1912:34gfredericksI have no idea whether there was any intention to make something like this work#2017-10-1912:36jrychterHmm. In that case, the push to use namespaced keys everywhere seems slightly premature.#2017-10-1913:57Alex Miller (Clojure team)thatā€™s not valid syntax#2017-10-1913:58Alex Miller (Clojure team)the ā€œleftā€ part of destructuring is always the unqualified name of the local being bound#2017-10-1913:59Alex Miller (Clojure team)so you want: (let [{a :some.ns/a} {:some.ns/a 1}] a)#2017-10-1913:59Alex Miller (Clojure team)(this has worked since Clojure 1.6 I believe)#2017-10-1914:11jrychter@alexmiller I am moving to ns-qualified keys everywhere and have many places where I do {:keys [a b c d e f]}. I was hoping I could do #:some.ns{:keys [a b c d e f]} and this works well, but what if I additionally want to extract {renamed-g :some.ns/g}?#2017-10-1914:11Alex Miller (Clojure team)you can do both#2017-10-1914:12Alex Miller (Clojure team){:some.ns/keys [a b c d e f], renamed-g :some.ns/g}#2017-10-1914:13Alex Miller (Clojure team)I think using the #:some.ns prefix is kind of confusing in this context - I usually put it inside the map#2017-10-1915:31jrychter@alexmiller Oh, so that's the syntax! Thank you, I've never seen it until now.#2017-10-1915:49Alex Miller (Clojure team)Means the same thing - just alternate syntax#2017-10-1915:50Alex Miller (Clojure team)You can do multiple keys destructuring from different namespaces too#2017-10-1916:44jrychterThank you for your time and help!#2017-10-1918:58radscan anyone explain to me why spec doesn't check :ret specs during instrumentation by default? I know there was a discussion about it a while back, but I'm curious if anyone has a succinct explanation for the decision#2017-10-1919:09bbrinckIIRC, the rationale is that instrumentation is for checking that you called function X correctly, not for checking that the function X implemented correctly#2017-10-1919:09bbrinckThe recommendation is to use generative testing to confirm that function X works correctly (based on the :ret and :fn specs)#2017-10-1919:10bbrinckI personally disagree with that distinction, since I have functions that canā€™t be realistically verified with generative testing due to a) side effects or b) disproportional work to get the specs perfectly tuned#2017-10-1919:10bbrinckWhich is why I use https://github.com/jeaye/orchestra šŸ™‚#2017-10-1919:12bbrinckIn my limited experience, instead of using check to test function X, we ended up just not speccing :ret or :fn because they were unverified and I preferred no documentation rather than documentation that was potentially incorrect or out of date#2017-10-1919:16bbrinckIn particular, using orchestra to instrument functions during non-generative testing gives me confidence that my :ret and :fn functions are correct, which makes them more valuable as documentation#2017-10-1919:19radsthanks, that's very helpful#2017-10-1921:50eriktjacobsen@alexmiller Is there any plan to allow multiple, or nested namespaces in spec? Example: A spec ::bar/type declared in the foo.core namspace that could be referenced as :foo.core/bar/type? We are currently struggling with a seeming limitation of spec, and while the underlying problems deserves a longer form post, before heading down that path, was just curious if this was something being thought about.#2017-10-1922:03seancorfieldA spec of ::bar/type would need a namespace alias of bar available in order to be legal. That could alias any namespace @eriktjacobsen but could be foo.core.bar if you wanted it to be. Then ::bar/type would be :foo.core.bar/type.#2017-10-1922:04seancorfieldSince namespaces are (supposed to be) globally unique and can be arbitrarily "nested" already, I'm not sure what you're asking for...#2017-10-1922:16eriktjacobsenThank you Sean. I realize what the current system provides, and how to work around it. I was mostly curious if that specific solution has been talked about... will write up an explanation in a bit.#2017-10-1922:21Alex Miller (Clojure team)There is no plan (or need) for any kind of nesting#2017-10-1922:21eriktjacobsenThanks for answer. put another way, has there been any thought to allowing a component of a spec identifier that isn't tied to a namespace or part of the unqualified key#2017-10-1922:22Alex Miller (Clojure team)No, why?#2017-10-1922:23Alex Miller (Clojure team)That is, please help me understand the problem you're trying to solve#2017-10-1922:23eriktjacobsenwriting the why up now, mostly due to tradeoffs in workflow my team is seeing. Perhaps it just needs a better framework for how we're structuring things, but it could also be solved by what I mentioned.#2017-10-1922:25Alex Miller (Clojure team)I think sometimes it's also unclear that keyword qualifiers can refer to Clojure namespaces but don't have to have any relation to them#2017-10-1922:25Alex Miller (Clojure team)There is an unfortunate overload and common use for the word "namespace"#2017-10-1922:27Alex Miller (Clojure team)Currently namespace aliases also have the unfortunate downside of being tied to the particular case where a qualifier is actually a namespace (I do not expect this to be as constrained in the future)#2017-10-2000:03eriktjacobsen@alexmiller Thanks for the quick answers. Explained: https://redd.it/77if23 If the answer is "you need a separate namespace per data type with colliding keywords", so be it.... just wanted to have the discussion.#2017-10-2000:42seancorfield@eriktjacobsen Yes, the answer is to provide unique namespace-qualifiers for each of the colliding keywords -- remember that those namespaces do not have to exist, but they should have names that represent the different concepts being modeled.#2017-10-2000:47seancorfieldThe whole point of namespaces is to provide unique ways to name things that are distinct (but otherwise have the same unqualified name).#2017-10-2001:01eriktjacobsenRight, it's just unfortunate to lose the relationship of between using the spec and knowing where it's defined, losing ability to use the :: and aliases (until patched), and other limitations mentioned. I understand that might be the answer, its just my team is unhappy if that's the final answer.#2017-10-2001:07gfredericks:foo.bar/baz.hullabaloo is a valid keyword I think#2017-10-2001:07gfredericksI don't suppose it's likely to make anybody happy though#2017-10-2001:09eriktjacobsenCorrect, but when used in a (spec.keys :req-un) it will require the keyword to be :baz.hullabaloo, correct? Rather than :hullabaloo. That is basically the same solution as my first proposal, just with . instead of /#2017-10-2001:11gfredericksyep. it's just sadness all over#2017-10-2001:16eriktjacobsenit does seem like differentiation between specspace and namespace seem slightly confused, or dare I say complected.#2017-10-2001:26gfredericksOTOH it would be weird and confusing to have many parallel namespace systems e.g. those lisps that keep functions separate from the other kind of thing#2017-10-2001:10potetmWhy use :req-un for a namespaced key?#2017-10-2001:13eriktjacobsenBecause the incoming data I'm trying to spec is generally using unqualified keys. Such as from parsing json.... I suppose I could add a step before doing spec validation that transforms all the unqualified keys into qualified keys using some mapping scheme, validates with spec, then unqualifies them again for usage downstream, but that's basically the functionality i'm looking for from spec and seems like a lot of overhead#2017-10-2001:33potetmAh, okay. It sounded like you had control over the keyword. That makes sense though.#2017-10-2007:03jrychter@eriktjacobsen FWIW, I am struggling with similar issues. I tried (briefly) using spec keywords in "fictional" namespaces (e.g. not corresponding to clojure ns), but then I end up with an explosion of long namespaced keywords. Lots of typing, lots of things to read and process when looking at code. The :: really helps in reducing that.#2017-10-2007:05jrychterSomewhat related: I'm also thinking about dropping :req-un and moving to fully-qualified keys everywhere. But there is a price to be paid in storage (JSON database), transmission (websocket trafic isn't compressed, Transit helps but I'm not sure to what extent), code complexity (need to include keyword namespaces in all destructuring code), and ClojureScript code building keywords from strings. I'm not sure what that price is.#2017-10-2008:05mpenetif you throw in some utils it's not so bad#2017-10-2008:07mpeneti use a macro that creates "relative" aliased ns and tend to spec maps using it following a pattern like ::map ::map/foo ::map/bar ::map/baz etc#2017-10-2008:07mpenet(defmacro rel-ns [k] `(alias k (create-ns (symbol (str k))))))#2017-10-2008:10mpenetyou can just do (rel-ns 'foo.bar.baz.bad) if deeply nested and then use it as ::foo.bar.baz.bad/xxx, it's quite readable imo#2017-10-2018:06eriktjacobsenThank you! We have thought of similar workarounds, mostly wanted to bring the discussion up for a solution from the core.specs team, though we could incorporate this if we have to workaround it.#2017-10-2016:45metametadataHi! Given ::allowed-val "enum spec", is there a way to print all the allowed values in the error message? Currently it prints (into #{} allowed-vals). And let's consider it's the requirement that I need allowed-vals extracted as a var because I need to iterate over it in other places.
(def allowed-vals [1 2 3])
(s/def ::allowed-val (into #{} allowed-vals))
(s/explain ::allowed-val 5)

=>
val: 5 fails spec: :cljs.user/allowed-val predicate: (into #{} allowed-vals)
:cljs.spec.alpha/spec :cljs.user/allowed-val
:cljs.spec.alpha/value 5
#2017-10-2017:24hiredman
user=> (s/def ::foo #{:a :b :c})
:user/foo
user=> (s/form ::foo)
#{:c :b :a}
user=> (doseq [i (s/form ::foo)] (prn i))
:c
:b
:a
nil
user=> 
#2017-10-2017:49metametadatayeah, it's a bit backwards but should work, thanks#2017-10-2020:04ajsThere is no forwards or backwards with a set, right? Unordered. #2017-10-2100:03metametadataright šŸ™‚ I meant that it's backwards in a sense that I'd like the variable to be the source of allowed values instead of putting them into spec#2017-10-2018:13metametadataand what if the enum values cannot be hardcoded like that in spec? e.g. when I need to read them from file#2017-10-2018:46ghadiright now calling eval to build specs is probably the best choice, but work is apparently underway on making spec more "programmable" -- make specs from external stuff#2017-10-2020:04bsimahow would I spec a function with variadic keyword args like (myfn :a 1 :b 2)? Iā€™m trying to use (spec/* (spec/keys :req-un [::a ::b])) but thatā€™s not right#2017-10-2020:04taylorkeys*#2017-10-2020:05taylorhttp://clojuredocs.org/clojure.spec/keys*#2017-10-2020:06bsimaah thanks#2017-10-2022:19falakCan somebody help me out with writing a fdef for a multi-method using defmulti?#2017-10-2022:22taylorI might be able to in a little while. Do you have an example?#2017-10-2022:26falakI have a function which takes 2 maps as input and returns a vector with 2 maps.
defmulti my-function (fn [map1 map2] (:type map2)

defmethod my-function :type1
  [map1 map2]
  (let .... 
        ...
        ...
        ...)
[ret-map1 ret-map2]

defmethod my-function :type1
[map1 map2]
.
.
.
#2017-10-2022:31falakThis is what I tried for fdef -
(s/fdef my-function
             :args (s/cat :map1 map? :map2 map?)
             :ret    vector?)
#2017-10-2023:02taylorNot sure exactly whatā€™s not working but this works for me:
(defmulti my-function (fn [map1 map2] (:type map2)))
(defmethod my-function :type1 [map1 map2]
  [map1 map2])
(defmethod my-function :type2 [map1 map2]
  [map2 map1])
(s/fdef my-function
        :args (s/cat :map1 map? :map2 map?)
        :ret vector?)
(stest/instrument `my-function)
#2017-10-2100:11falakSo how do I test the dispatch-value of the dispatch-fn in the defmulti definition?#2017-10-2102:55James VickersI'm trying to make a spec for a function that takes in a map with un-qualified keys (I know how to do this with qualified keywords, just use s/keys). The function arguments look like this: [{:keys [interest term balance] :as m}], and for that I make a function spec with (s/cat :m (s/keys :req-un [::interest ::term ::balance])) - I have specs named ::interest,`::term`, ::balance. Is there a more direct way to make a s/fdef spec for :args? I don't use the whole map (`:m`) in the function body and only added that part to use in the spec. I tried the example in the clojure.spec documentation for s/keys with unqualified keywords but couldn't get it to work. Alternatively, am I just way off by using un-qualified keywords as keys in the map the function takes?#2017-10-2102:55James VickersI'm trying to make a spec for a function that takes in a map with un-qualified keys (I know how to do this with qualified keywords, just use s/keys). The function arguments look like this: [{:keys [interest term balance] :as m}], and for that I make a function spec with (s/cat :m (s/keys :req-un [::interest ::term ::balance])) - I have specs named ::interest,`::term`, ::balance. Is there a more direct way to make a s/fdef spec for :args? I don't use the whole map (`:m`) in the function body and only added that part to use in the spec. I tried the example in the clojure.spec documentation for s/keys with unqualified keywords but couldn't get it to work. Alternatively, am I just way off by using un-qualified keywords as keys in the map the function takes?#2017-10-2102:57taylorthe :as m in the function argslist shouldnā€™t matter to spec, you should be able to remove it if youā€™re not using it#2017-10-2103:01James VickersThanks for answering so fast. So s/cat needs key/pred argument pairs, so if I remove :m from the definition of the function spec, what would I put instead?#2017-10-2103:07taylorthe :m in your function spec doesnā€™t have any relation to the :as m in your arglist, it could be named anything really#2017-10-2103:07taylors/cat requires each ā€œelementā€ be tagged with some keyword though, so you canā€™t omit it. The actual keyword name doesnā€™t matter so much unless youā€™re interested in the conformed version of it i.e. it could be (s/cat :tgif ::map-spec) and itā€™d still work#2017-10-2103:07taylorit looks like youā€™re doing it right#2017-10-2103:10taylorif it doesnā€™t work when you remove :as m from the fn arglist, something else must be wrong#2017-10-2103:11James VickersYou are totally correct - took out :as m in the function args list, still works. Can change :m to any keyword (e.g. :foo) in the :args spec and that works too. But that's so weird! In this usage, I guess spec doesn't use the keyword arg for this spec?#2017-10-2103:13James VickersThat's weird. I wonder if there'll be other functions coming in spec like s/cat that don't have that setup, it seems weird that for this case (which seems common?), it doesn't matter what keyword you put there - just a placeholder. Thanks!#2017-10-2103:14taylors/cat does use the keyword to tag conformed outputs, it just doesnā€™t matter for your use case#2017-10-2103:16taylor
(s/def ::opts (s/* (s/cat :opt keyword? :val boolean?)))
(s/conform ::opts [:silent? false :verbose true])
;;=> [{:opt :silent?, :val false} {:opt :verbose, :val true}]
example from spec guide, notice the output has :opt and :val ā€œtagsā€
#2017-10-2103:23James VickersI guess the reason I didn't understand is I don't quite get s/cat and the other regex ops in spec.#2017-10-2104:57seancorfield@jamesvickers19515 So your function has one arg, a hash map? (s/cat :m ::account) perhaps, with (s/def ::account (s/keys :req-un [::interest ::term ::balance]))#2017-10-2105:10James VickersThanks @seancorfield. @taylor showed me that when using s/cat in this instance, the first argument actually didn't matter - could be :foo for all it mattered.#2017-10-2105:10seancorfieldRight, s/cat takes a sequence of (whatever) argument names and specs.#2017-10-2105:11seancorfieldIt's convention to use the same (keyword) name for each argument as the function but there's no reason to.#2017-10-2105:11seancorfieldIn your case, you have a destructuring as the first (only) argument so its name is somewhat arbitrary anyway.#2017-10-2105:12James VickersI guess what was surprising was that there wasn't a spec function for this case that doesn't require the unused keyword as the first arg - I was under the impression that functions that take a single map were common in Clojure.#2017-10-2105:13seancorfieldYes... not sure what you're asking...#2017-10-2105:13James VickersSorry, wasn't a question šŸ™‚ just a comment#2017-10-2105:14seancorfield(s/cat ...) is how you specify an argument list.#2017-10-2105:18seancorfieldSpecs name things, as part of their conformance. The names don't have to correspond to anything in the source code (`s/or` is a good example, s/cat is similar).#2017-10-2105:23James VickersI think I sort of see now. I did something like this at the REPL:
(s/def ::account (s/cat :m (s/keys :req-un [::interest ::term ::balance])))
(s/conform ::account [{:interest 4.25 :term 360 :balance 261250}])
=> {:m {:interest 4.25, :term 360, :balance 261250}}
#2017-10-2105:27seancorfieldYeah, argument lists are sequences. s/cat matches a sequence of (named) specs. So, in this case :m is the name and the s/keys is the spec for it.#2017-10-2105:27James VickersAnd this call to valid? with a vector that looks like the function signature:
(s/valid? ::account [{:interest 4.25 :term 360 :balance 261250}])
=> true
#2017-10-2105:27seancorfieldRight.#2017-10-2105:31James VickersThanks, I think I understand s/cat better after playing with it in the REPL a bit.#2017-10-2300:11Drew Verlee@alexmiller in the clojure spec workshop you said you wouldnt use constantly with with-gen in this example:
(def id-gen
  (gen/fmap #(str "ID-" %)
    (s/gen (s/int-in 100000 999999))))

(s/def ::id
  (s/with-gen (s/and string? #(re-matches id-regex %))
    (constantly id-gen)))
what was your advice on that?
#2017-10-2319:46Alex Miller (Clojure team)Well first, nothing actually wrong with this example. But I think I would prefer (fn [] id-gen) now.#2017-10-2300:11Drew Verleeugh, i shouldn't ping ppl at 8pm on sunday. got excited.#2017-10-2302:51seancorfield@drewverlee It's OK to be excited at weekends šŸ™‚ Just needs to be tempered with a little patience until most folks get to their desks on Monday morning šŸ™‚#2017-10-2315:44mmerIssues with : Call to clojure.core/let did not conform to spec: In: [0] val: () fails spec: :clojure.core.specs.alpha/bindings at: [:args :bindings :init-expr] predicate: any?, Insufficient input - Should I be using clojure.spec or clojure.spec.alpha in my namespace?#2017-10-2315:45taylorif youā€™re using the latest version, the namespace is clojure.spec.alpha#2017-10-2315:46mmerThanks - but ...why am I seeing this error?#2017-10-2315:47taylorcan you post a code snippet of your let?#2017-10-2315:51mmerThanks Taylor - I had been getting errors all the time just when I added the clojure.alpha module to my project that I assumed the spec error was in spec not my code! But of course it was my code!#2017-10-2317:53hiredmanI am using spec as a parser, and have just noticed that when I parse a seq with 40 entries (all already realized in memory) I get out of memory errors, any ideas how to avoid this?#2017-10-2317:55bfabryseems weird, lots of alternates?#2017-10-2317:57hiredman2 alternates with 2 and 4 cases#2017-10-2317:58hiredmanI haven't played with changing how the spec is structured yet to see if that improves things#2017-10-2317:59hiredmanI do use s/& a few times, which I can imagine being expensive for backtracking#2017-10-2318:48ghadi@hiredman any way you can file an issue?#2017-10-2318:49hiredmanmaybe#2017-10-2319:43Alex Miller (Clojure team)donā€™t know that I can help without a repro case or more info#2017-10-2322:12Drew VerleeAny idea how you can get the keys from a spec?
(s/def ::foo (s/cat :name ::name))

(??? ::foo)
=> [:name]
#2017-10-2322:14bfabry@drewverlee there's a few different options. s/form will give you the spec definition as data that you could walk#2017-10-2322:32Alex Miller (Clojure team)Spec form specs will eventually make this easier to answer (see clj-2112)#2017-10-2322:53hiredmanwhile I was fiddling with coming up with a minimal test case, I realized I could change the format slightly to have an explicit delimiter instead of an (s/& ...) check, which seems to have solved my performance issues#2017-10-2408:52msolliWhere is a good place to put (test/instrument) so that all my specā€™ed functions are instrumented in dev? I have a fairly standard Luminus-based webapp.#2017-10-2411:25stathissideris@drewverlee I had to do this for the spectacles library, see here: https://github.com/stathissideris/spectacles/blob/master/src/spectacles/impl.clj#L4-L36#2017-10-2411:55ikitommi@stathissideris @drewverlee there is also a parse-spec in spec-tools doing about that: https://github.com/metosin/spec-tools/blob/master/test/cljc/spec_tools/parse_test.cljc#L28-L39#2017-10-2411:55ikitommispec parsing is the new black šŸ˜‰#2017-10-2411:58stathissideris@ikitommi do you think we could use spec to parse spec forms or would reality collapse under the weight of self-reference? šŸ˜„#2017-10-2412:01ikitommi@stathissideris https://dev.clojure.org/jira/browse/CLJ-2112 ?#2017-10-2412:01stathissiderisway ahead of us šŸ™‚#2017-10-2412:07ikitommibut even when the spec-of-specs ships, we still need to utilities to do something with the parsed data, e.g. collect keys out of (s/merge ::a-map ::another-map), spec-tools is one bin collecting this kind of stuff. And there are many others.#2017-10-2414:29taylorIā€™m curious about this too#2017-10-2414:43andre.stylianosso am I#2017-10-2415:17jeayeI'm just using re-frame, not Luminus, but it's in my-app.core behind a macro.#2017-10-2415:23jeayehttps://gist.github.com/jeaye/9a57740fc474a60a050ae68acf714c4d#2017-10-2415:24jeayeThat'll do the trick, allowing you to specify bits in your leiningen profiles and then conditionally bring in code by reading the profile using environ in a macro.#2017-10-2415:24taylorthanks, the only caveat is that any namespaces you want to instrument must have already been loaded before thatā€™s called, right?#2017-10-2415:24jeayeThat's right.#2017-10-2415:25jeayeI don't think you can get around that though.#2017-10-2415:26jeayeOur my-app.core requires every model ns we have. Our my-app.core-views requires every view ns we have, as well as requiring my-app.core (so view code isn't needed for unit testing).#2017-10-2415:26taylorI wouldnā€™t imagine so, was just curious what other peopleā€™s approaches to that might be. I was thinking another approach would be to just put the (conditional) instrument calls at the bottom of each specā€™d namespace#2017-10-2415:26jeayeHm, that would be every single ns for us, which seems less manageable. Sounds like it would work though, if that's your preference.#2017-10-2418:00msolliThanks, @U4986ECDQ, seems like a reasonable solution. Those are some handy macros! šŸ‘#2017-10-2415:55tony.kayI have a project where I use tools-ns to reload code. If I donā€™t carefully set refresh dirs then suddenly defn will start causing spec failuresā€¦as in defn- doesnā€™t conform to spec. Is this a known issue? Narrowing refresh dirs seems to fix it.#2017-10-2512:53Alex Miller (Clojure team)I havenā€™t heard of that before#2017-10-2513:22robert-stuttafordis the right way to spec a map that must have at least one of several keys (s/or (s/keys :req []) (s/keys :req []) (s/keys :req [])) ?#2017-10-2513:22tayloryou can use a single keys spec with or inside of the :req vector#2017-10-2513:22robert-stuttaford(s/keys :opt []) says 0..n. i want 1..n#2017-10-2513:22tayloroh sorry, misread#2017-10-2513:23mpenetsounds ok, but if they are sharing the same context and have something like a :type key it might make sense to make a multi-spec out of it#2017-10-2513:24robert-stuttafordit truly is a situation where you can provide 1, 2 or 3 of the keys, but providing 0 of them makes the whole map redundant#2017-10-2513:25robert-stuttafordmodelling a rule system where there are 3 possible outcomes - no-value-yet, yes, or no. you have to model at least one outcome, or thereā€™s no point to testing for the rule#2017-10-2513:26taylorthose three states should be mutually exclusive?#2017-10-2513:27robert-stuttafordthe rule can be reused for multiple cases in a doc#2017-10-2513:27robert-stuttafordsome may 0, some may 1, some may 2#2017-10-2513:27robert-stuttafordiā€™ll try s/or + s/keys !#2017-10-2513:33robert-stuttaford@taylor you were right!#2017-10-2513:33robert-stuttaford(s/keys :req [(or :a :b :c)]), plus the fact that all ks are validated gives me what i need#2017-10-2514:17ajslooking at this example from @cemerick do i read correctly that specs default to passing valid? even if the namespaced keyword was never def'd? https://twitter.com/cemerick/status/875748591310168065#2017-10-2514:19taylorIā€™ve only seen this behavior w/`key` specs, but yeah it doesnā€™t require that the keyword have a registered spec#2017-10-2514:19mgrbytenope. (require '[clojure.spec.alpha :as s]) (s/valid? ::some-undefined-spec [}) will throw an exception (unable to resolve some-undefined-spec)#2017-10-2514:20mgrbyteyep, (s/valid? (s/keys :req [::some-spec]) {}) will pass tho#2017-10-2514:20mgrbyteeven when ::some-spec has not been defined yet#2017-10-2514:21ajshmm, i wonder what the reasoning is for allowing that keys#2017-10-2514:21ajsthat's doesn't make any sense to me#2017-10-2514:21taylormaybe so that you can easily spec required map keys without writing specs for the keysā€™ values, or having to define the key specs before the keys spec#2017-10-2514:22taylorotherwise youā€™d have to do (s/def ::foo any?) for every key before using it in a keys spec?#2017-10-2514:22ajslot of room for user error though, as @cemerick has noted. as i am very prone to typos (perhaps we all are), i can see this biting me#2017-10-2514:23tayloritā€™s true, and Iā€™ve actually seen some code in the last week that checks map spec keys for ā€œtyposā€ but I canā€™t remember where šŸ™‚#2017-10-2514:23ajsi'm currently evaluating a variety of validation libraries like funcool/struct and truss and others. the one that most ably catches my typos is the one i will use.#2017-10-2514:24taylorfound it https://gist.github.com/stuarthalloway/f4c4297d344651c99827769e1c3d34e9#2017-10-2514:24taylor
;; The example program below lets you identify "missing" keys specs at
;; the time and place of your choosing, and then handle them as you
;; deem appropriate, without imposing those decisions on other
;; users of spec.
#2017-10-2514:25ajsi find it a little bizarre that you have to jump through those hoops and write that code just to have spec do what you are expecting; perhaps that leaves room for a little library on top of spec to help mitigate disaster#2017-10-2514:28ajsi don't understand why specifying the presence of a key uses the same syntax/naming as specifying the presence of an actual spec/predicate -- those are two different things, but they are expressed the same in spec, hence the room for easy errors#2017-10-2515:01bbrinckIt seem like most of my typos are around keywords. It seems like a tool that would macroexpand code and look for keywords that are unique would catch a large number of my typos.#2017-10-2515:11andre.stylianosFrom https://clojure.org/about/spec#_sets_maps_are_about_membership_that_s_it
Sets (maps) are about membership, thatā€™s it

As per above, maps defining the details of the values at their keys is a fundamental complecting of concerns that will not be supported. Map specs detail required/optional keys (i.e. set membership things) and keyword/attr/value semantics are independent. Map checking is two-phase, required key presence then key/value conformance. The latter can be done even when the (namespace-qualified) keys present at runtime are not in the map spec. This is vital for composition and dynamicity.
#2017-10-2515:14andre.stylianosFrom what I understand there are two different things. -- Membership: "This map has (required or optional) such and such keys" (s/def ::map (s/keys :req [::foo])) -- Value: "Anything with this key should conform to this value" (s/def ::foo string?)#2017-10-2515:16andre.stylianosThat's the way I understood this at least#2017-10-2516:55ajsPerhaps one could consider spec a lower level tool on top of which you could build an API to better express concrete structural requirements that eliminate some of that ambiguity, and use that instead of spec directly. #2017-10-2516:56ajsOtherwise there seems ample room for user error, which is typically what you're trying to avoid in the first place by using a validation library#2017-10-2815:04metametadata@U5YHNV0EA I also questioned this behavior on the mailing list recently: https://groups.google.com/forum/#!topic/clojure/i8Rz-AnCoa8 And put my current solution into the gist here: https://gist.github.com/metametadata/5f600e20e0e9b0ce6bce146c6db429e2#2017-10-2516:38mmerApart from expound is there a way to associate an error message with spec definition so you can force an error message that makes sense and also allows for globalisation of error messages?#2017-10-2516:47bbrinckIs this for devs or end users?#2017-10-2516:47bbrinck@mmer If the messages are for end users, you probably donā€™t want Expound. https://github.com/alexanderkiel/phrase is likely a better fit#2017-10-2516:54mmerWould it not be reasonable to include some method of defining the error message directly alongside the definition?#2017-10-2517:16bbrinckHow would you want to use those errors messages? In explain?#2017-10-2517:21bbrinckor would you want to show the error messages to end users?#2017-10-2517:25bbrinckMy understanding is that specs are getting metadata, so if you just need a place to add some info and you want to later pull that out, you could potentially use metadata? https://dev.clojure.org/jira/browse/CLJ-2194#2017-10-2518:46mmerI guess I need to get a usable error message for end users. My usecase revolves around validating yaml files edited by people. As these are nested structures the explanation from explain? get complex. I was hoping for a simple way to add a message to def that explain in human terms what is required. In my case the errors from spec are about a format the user does not see. Why use spec? I have found no other suitable tool that allows me to define in effect a schema for Yaml.#2017-10-2518:51bbrinckOK, that makes sense.#2017-10-2518:52bbrinckSo, you could assign an error message to each spec. You could then call explain-data and, for each problem, convert the failing spec into the error message.#2017-10-2518:53bbrinckThat may be sufficient for your use case. However, remember that problems are flat - if you have a ā€œorā€ spec, youā€™ll get two different problems for the same value#2017-10-2518:54bbrinckAdditionally, itā€™s not trivial in my experience to show where the non-conforming is located within the larger data structure#2017-10-2518:55bbrinckSpec also will print errors in terms of the Clojure data structure, which may not be useful if your users are expecting to see errors in terms of Yaml#2017-10-2519:03bbrinckIt really depends on your spec, but probably the simplest approach is to assign errors to each spec (you could just build a static map for now, since each spec name is a unique keyword), then call explain-data, then take the first problem for each in, then print out the in path + your custom string error (you can find the relevant spec in via). That might be sufficient, although possibly misleading if you have a lot of ā€œorā€ specs.#2017-10-2519:39stathissideris\|||||||||#2017-10-2522:31Brendan van der EsAnyone know a better way to conform an or spec without keywords? This is the best I got:#2017-10-2522:33mmer@bbrinck Thank you sir for a long and consider answer - I like the map of spec errors approach as even with an or you can explain the options. . Much appreciated.#2017-10-2522:41mmerA follow up from my previous question - is it posible to get the set of specs that are currently in registered?#2017-10-2600:37bbrinck(require '[clojure.spec.alpha :as s]) (s/registry)#2017-10-2522:42hiredmanthere is a registry function that returns the registry (a map of names to specs)#2017-10-2522:51Brendan van der Es@hiredmanThanks. I am trying to s/or arbitrary specs. Essentially I'm trying to confom different argument lists, e.g.[& args], to the same format of input.#2017-10-2522:53Brendan van der EsSpecifically for datomic, [db entity-hash-map] conforms to EntityMap & EntityMap conforms to EntityMap. Maybe this approach is flawed anyways šŸ˜›, I'll keep playing around with it. Thanks#2017-10-2522:54hiredmanI would use spec to recognize (tag) different formats so you know which formats to deal with, not to try and smoosh all the formats in to one#2017-10-2523:06Brendan van der EsMakes sense, you do than have to handle the different inputs in your function, which arguably is the better place for this logic anyways. I'm hoping this logic might be more reusable if I can embed the fact that the different arg-list formats essentially carry the same information in the spec.#2017-10-2600:39bbrinck@b.j.vanderes FWIW, my understanding is that conformers are not intended to be used to change the shape of the data, so you may run into problems here. Perhaps itā€™d be easiest to just conform as normal, then write a transformation function that uses the conformed values and converts them#2017-10-2601:59Brendan van der Es@bbrinck @hiredman Yea, this is all probably too counter to intention to be usable (performance might also be an issue). But just for fun this is what I came up with as far as defining functions that process the conformed arguments:#2017-10-2602:04Brendan van der EsCan combine with above to mitigate an outer layer of branching in the function.#2017-10-2608:23tapI get OutOfMemoryError when calling instrumented function with wrong number of arguments where one of the argument is a huge collection. I know itā€™s considered user mistake but itā€™s a kind of mistake that is quite hard to figure out the cause. Should clojure prevent that, or break circuit or something?#2017-10-2612:49Alex Miller (Clojure team)Would help to know more about the spec and understand why its generating huge collections#2017-10-2612:49Alex Miller (Clojure team)There are some known issues in this area#2017-10-2702:16tapSure. Iā€™ll try to come up with a sample project replicating the issue I found this weekend#2017-10-2611:01danielnealWhat is the best way of getting the specs of arguments to an fdef'd function?#2017-10-2611:01danielnealI've got an approach here#2017-10-2611:01danielnealbut it uses eval and is probably all wrong šŸ™ƒ#2017-10-2612:44Alex Miller (Clojure team)Yeah, donā€™t do that :)#2017-10-2612:45Alex Miller (Clojure team)Just (-> sym s/get-spec :args)#2017-10-2612:46Alex Miller (Clojure team)Or do you want the spec of each argument separately? #2017-10-2612:48Alex Miller (Clojure team)In the most general sense (considering multi arity and variadic), I think that is difficult#2017-10-2612:52danielnealyeah - the spec of each argument separately#2017-10-2612:53danielnealthanks for the get-spec tip šŸ™‚#2017-10-2612:55danielnealjust doing a very rough and ready thing to see if I can go from a value to the (specced) functions that you can invoke that take that value as an arg#2017-10-2612:55danielneallike intellisense, but stupid#2017-10-2612:55danielnealstupidisense, if you will#2017-10-2614:04Alex Miller (Clojure team)this is kind of whatā€™s happening inside of spec regex validation checking, but itā€™s really happening left to right and every possibility is being evaluated and narrowed based on the prior matches (of which there may be multiple)#2017-10-2614:05Alex Miller (Clojure team)so a priori before starting the parse there is not necessarily a single spec but really a set of specs that may apply at each point#2017-10-2614:05Alex Miller (Clojure team)that said, there is the very common case that the args spec is a single non-variadic arity and each arg matches up 1-for-1 with an s/cat args spec#2017-10-2614:06Alex Miller (Clojure team)in which case, you can just pull the s/cat spec apart to match it up#2017-10-2614:07danielnealmm yes I follow you. Technically the args spec for fns could be any regex (or anything) but in practice there will be a common case like you described#2017-10-2614:10danielnealthis is what I've got so far - https://gist.github.com/danielneal/d82c142c9eab9f8caec0fa93b87ff7f3 - I'd like to see if I can get it completing in emacs to see if it 'feels' useful. Dumb but proof of concept, like when IDEO demonstrated the talking kitchen with people hidden behind the fridge saying stuff and responding to voice commands.#2017-10-2619:17zcljSometimes I get errors such as: `java.lang.IllegalStateException: Attempting to call unbound fn: #'clojure.test.check.generators/choose, compiling:(generators.cljc:499:3) Exception in thread "main" java.lang.IllegalStateException: Attempting to call unbound fn: #'clojure.test.check.generators/choose, compiling:(generators.cljc:499:3)` It happens with different test.check unbound fns, it is not the same every time and a rebuild usually resolves the error. The file that is beeing compiled is a quite basic cljc file with some specs in it. Does anyone have any advice to point me in the right direction?#2017-10-2621:49Alex Miller (Clojure team)Any chance this is happening during check?#2017-10-2621:51Alex Miller (Clojure team)There is a dynamic load race that can happen with loading the test check generator namespace that we have a ticket on. I donā€™t remember /cdn-cgi/l/email-protection presents, but it was weird looking#2017-10-3012:03zclj@U064X3EF3 Sorry for the late reply. The stack trace only show line 1 for my files i.e. in the ns loading. The file do include a stest/check test but from the stack trace I do not seem to hit that. It blow up on the first line. I'll try and find the ticket you mentioned and see if that holds any clues#2017-10-2619:26gfredericksO_O#2017-10-2619:35billin an s/fdef :fn spec for a tree function of mine, it seems like I need to get access to the un-conformed argument (a tree) to the function so I can pass it to some other tree functions to build up my validation. is there a way to gain access to the un-conformed args or perhaps to un-conform the conformed args?#2017-10-2619:44billis the right word for ā€œun-conformā€ ā€œunformā€?#2017-10-2619:45billyessss unform https://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec/unform#2017-10-2619:45gfredericksdeconformificate#2017-10-2708:53mmerIs there a way to associate a string of text with a spec definition. I would like to annotate my definitions#2017-10-2709:17gklijsYou can do something like (def label (s/and (s/spec string?) #(> (count %) 3) #(< (count %) 40))) and then use hat spec somewhere else like (s/def ::nl-label label)#2017-10-2709:45danielnealI think doc-strings for specs are coming later, too#2017-10-2713:47souenzzohttps://dev.clojure.org/jira/browse/CLJ-1965 https://dev.clojure.org/jira/browse/CLJ-2194#2017-10-2819:19mike706574Is it intentional that the functions in clojure.spec.gen that alias functions in clojure.test.check.generators aren't considered generators?#2017-10-2819:19mike706574Or am I doing something wrong?#2017-10-2819:20mike706574(clojure.test.check.generators/generator? clojure.spec.gen.alpha/int) seems to be returning false#2017-10-2819:22mike706574In my case, I'm trying to write a spec for an 5-10 character alphabetic string with a custom generator that uses clojure.spec.gen.alpha/char-alpha#2017-10-2819:23mike706574Sampling is failing with an AssertionError with a message of Assert failed: First arg to vector must be a generator (generator? generator)#2017-10-2819:24mike706574It works fine when I use clojure.test.check.generators/char-alpha - just seems counterintuitive that the aliased function doesn't work#2017-10-2819:40hiredmanthe spec aliases are actually single argument functions that return generators#2017-10-2819:41mike706574oh#2017-10-2819:41mike706574that makes more sense#2017-10-2819:41hiredmanthis is to support lazy loading of clojure.test.check#2017-10-2819:41mike706574so i shouldn't be using them?#2017-10-2819:42hiredmanyou can use them, you just have to be aware of that fact when mixing with plain test.check generators#2017-10-2819:43hiredmanand if I recall correctly, most places in clojure.spec where you can supply a custom generator, you supply a no arg function that returns a test.check generator#2017-10-2819:43mike706574yup#2017-10-2819:44mike706574i just put parens around it - (gen/vector gen/char-alpha 5 10) to (gen/vector (gen/char-alpha) 5 10) - and it worked#2017-10-2819:44hiredmanthat's lisp for you#2017-10-2819:45mike706574cool, thanks#2017-10-2910:17Oliver GeorgeQuick sanity check. I think this code should stub a function (in second case). It does on CLJ but doesn't on CLJS.#2017-10-2910:17Oliver George
(defn add2 [a b] 1.1)
(s/fdef add2 :args (s/cat :a int? :b int?) :ret int?)
(println ::a (add2 1 2))
(stest/instrument `add2 {:stub #{`add2}})
(println ::b (add2 1 2))
(stest/unstrument `add2)
(println ::c (add2 1 2))
#2017-10-2910:18Oliver GeorgeAm I doing something obviously wrong?#2017-10-2915:24eoliphantIs it possible to do ā€˜crossā€™ specification/validation with spec? If I have a map
{:a "even"
:b 2}
Where say :b should be odd? if :a is "odd" and vice versa?
#2017-10-2915:25mingp@eoliphant Perhaps treat them as separate cases and combine with spec/or.#2017-10-2915:26mingpIt would depend more on what the more immediate problem you're trying to solve is.#2017-10-2915:26eoliphanthmm ah I see, when youā€™re doing the ā€˜mapā€™ spec cover both cases with the or#2017-10-2915:26eoliphantiā€™ll give that a try#2017-10-2916:08mrchanceHi, is there a way to generate self contained specs? I want to turn a swagger.json into a spec and run it on some data at runtime, and it looks like that requires eval with the way the spec registry currently works?#2017-10-2916:08mrchanceOh, and https://clojure.github.io/core.specs.alpha/ appears to be broken, I arrived there from the overall clojure api page https://clojure.org/api/api#2017-10-3016:26Alex Miller (Clojure team)This is a known issue in the doc build process#2017-10-3016:41mrchanceOk, just wanted to let you know. Any insights into the other problem? I feel pretty dirty if I have to fiddle with global mutable state just to get some spec validation working...#2017-10-2922:17Drew VerleeIf i want to use spec in order to pass the user of my application a error msg how would i do that? Is there a place to override the error msg with a custom one?#2017-10-2922:18Drew VerleeDoes that even fit with the rational? Like if i have a command line app and i want to validate the input and send back an error msg, would spec fit into?#2017-10-2923:03seancorfieldThe idea is to use the explain-data result and map it to whatever error messages you want in your app @drewverlee#2017-10-2923:15bbrinck@drewverlee Perhaps https://github.com/alexanderkiel/phrase would be a good fit?#2017-10-2923:16bbrinckBut yes, if your specs are fairly simple, it might be easies to just map explain-data to error messages. It gets a little trickier if your specs are more deeply nested
#2017-10-2923:24Drew Verlee@bbrinck phrase is in the spirit of what i want. Its odd i couldn't search it down.#2017-10-3013:26Drew VerleeI poked around with the idea of phrase and iā€™m not sure it make sense for what iā€™m doing. As @seancorfield suggested i can pull the information with explain-data. And if i want to reference something inside the spec it might make sense to pull it into a var to use across multiple domains rather then pulling it out via regex matching (which is what phrase seems to do). Iā€™m would be interested to hear how something like phrase worked in a larger project, as it seems prescriptive.#2017-10-3014:36mpenetdid anyone create a web based spec browser thing?#2017-10-3014:39bbrinck@mpenet Whatā€™s your use case? Do you want to browse specs at dev time, or, say, publish specs online similar to API docs?#2017-10-3014:40mpenetyep exactly#2017-10-3014:40mpeneti know of https://github.com/jpmonettas/inspectable but i d like to ship a browser based thing#2017-10-3014:41mpenetI guess this could be adapted, but just asking in case somebody did that work already#2017-10-3014:42bbrinckHm, not that I know of. On a related note, I wonder if itā€™s be useful to start a wiki that collects these spec helper libraries as a resource#2017-10-3014:50jebberjeb@drewverlee Iā€™ve been working on something like this which tries to map a human readable message by a problemā€™s spec keyword, and falls back to a symbolic predicate match (like Phrase) if it canā€™t. Trying to embloginate it now.#2017-10-3015:00Drew VerleeI was thinking this over last night. I donā€™t think there is any generic relationship between a human readable message and the code. The only way i could see to do this that was better then plain old clojure would be some sort of semantic analysis of the function words. I feel like a neural net might be able to make something out of the function names and values. I was curious about phrase, but felt it wasnā€™t doing much for me that i couldnā€™t get from core spec and a mapping to human readable message. Iā€™m curious about your work.#2017-10-3015:06jebberjebI suspect its there, but Iā€™m not even close to finding it. My work is mostly a straight dispatch based on the problemā€™s spec keyword, falling back to a non-macro based Phrase implementation if that fails. And only the latter out of necessity. s/keys specs for example, generate problems that really require you to look at the predicate to know exactly what went wrong.#2017-10-3015:07jebberjebSo the predicate dispatch is mostly to catch cases where :via just doesnā€™t tell you enough (in the s/keys case, I mean the exact field) about what went wrong.#2017-10-3015:10jebberjebIā€™m hoping that approach casts a wide enough net to handle most of the problems generated by our specs. Chopping the specs up, and defā€™ing each piece seems to help.#2017-10-3015:29Drew VerleeI think dispatching on the problem specā€™s keyword and maybe some context importer to the end user might make sense. The advantage to just handing them as data and passing them around is that you could organize your code around them.#2017-10-3015:29Drew VerleeIs that in line with your thinking? I should get back to you tomorrow when im feeling better.#2017-10-3015:30jebberjebI think it is šŸ™‚ Yeah, would love to talk more.#2017-10-3014:51jebberjebItā€™s in use now in a fairly large project, but was introduced only about a month ago. So still waiting to see how it pans out.#2017-10-3015:06amannNot sure if this is the correct place to ask this question (if not, feel free to boot me to another channel!), but I recently stumbled across some performance related things with respect to spec generative testing specifically regarding every and the :kind and :into parameters. So the spec I had is loosely similar to this:
(s/def ::x (s/every string? :kind vector?))
(time (some-gen-testing ::x))

=> "Elapsed time: 93083.882115 msecs"

(s/def ::x (s/every string? :into []))
(time (some-gen-testing ::x))

=> "Elapsed time: 293.882115 msecs"
As you can see, :into is orders of magnitude faster than :kind for generative testing. My question is this: when should I be using :kind and when should I be using :into?
#2017-10-3016:20Alex Miller (Clojure team)Are you using latest spec? This problem has been fixed.#2017-10-3016:21Alex Miller (Clojure team)https://dev.clojure.org/jira/browse/CLJ-2171 fixed in spec 0.1.134#2017-10-3016:22Alex Miller (Clojure team)I would recommend preferring kind if thatā€™s sufficient#2017-10-3016:53bostonaholic@alexmiller I think you may have linked the wrong JIRA#2017-10-3016:53Alex Miller (Clojure team)Yup#2017-10-3016:54Alex Miller (Clojure team)https://dev.clojure.org/jira/browse/CLJ-2103#2017-10-3016:54bostonaholicmore better!#2017-10-3017:08amann@alexmiller looks like there was a very fun dependency error with pedantic being enabled in leinengen. I was pulling alpha17 when my deps required beta2. Rerunning things I got the following (which looks acceptable to me):
(s/def ::x (s/every string? :kind vector?))
(s/def ::y (s/every string? :into []))
(s/def ::n nil?)

(defn xfn
  [x]
  nil)

(defn yfn
  [y]
  nil)

(s/fdef
  xfn
  :args (s/cat :x ::x)
  :ret ::n)

(s/fdef
  yfn
  :args (s/cat :y ::y)
  :ret ::n)

(time (tu/stest-w-report `xfn))
(time (tu/stest-w-report `yfn))
#2017-10-3017:08amann
(time (tu/stest-w-report `xfn))
"Elapsed time: 974.417512 msecs"
=> true
(time (tu/stest-w-report `yfn))
"Elapsed time: 522.806027 msecs"
=> true
#2017-10-3017:08amannie, off by half a second rather than minutes#2017-10-3017:08amannmany thanks for calling that out!#2017-10-3015:25mpenet@alexmiller is that voluntary or a leftover https://github.com/clojure/spec.alpha/blob/739c1af56dae621aedf1bb282025a0d676eff713/src/main/clojure/clojure/spec/alpha.clj#L237 ? just lost a bit of time tracking success! spurious messages in logs that where coming from here#2017-10-3015:27mpenetthe originating code was a bit weird anyway (just calling s/explain-data in a when-let to validate a piece of content and grad the error in one go), but maybe printing here is not something desirable#2017-10-3016:22hiredman@mpenet that is what explain does, if you look at the code up above it as full of prs, the default for explain is to print stuff out#2017-10-3016:27mpenetNo prob with explain printing. Just found a bit odd that -data would#2017-10-3016:23Alex Miller (Clojure team)Youā€™ll get that if you ask for explain on something valid#2017-10-3016:23Alex Miller (Clojure team)Which leads me to why you would do that... :)#2017-10-3016:30mpenetThe code in question is gone. But since there an if branch in that code for the printing in case of success maybe it's expected people would do that#2017-10-3016:23mpenetyes that was the case, not very nice #2017-10-3016:23mpenetIndeed#2017-10-3016:25mpenetI was not expecting it to trigger a print regardless#2017-10-3016:25mpenetex-data doesnt for instance#2017-10-3016:29hiredmanare you sure you aren't just calling explain, not explain-data?#2017-10-3016:31mpenetNot in front of my computer anymore. I think it was -data, but not sure#2017-10-3016:39mpenetAh no it was explain, sorry about the noise šŸ™ All good then#2017-10-3017:13amannšŸ¬#2017-10-3018:38stexI'm was wondering if the following is possible with clojure.spec, since I'm not sure if it's one of the use-cases it tries to solve. Let's say we have a user which can have two roles: (s/def ::role #{:admin :normal}) and a admin specific field which is only required when the user is an admin, not a normal user. I currently have: (s/def ::user (s/keys :req [::role] :opt [::admin-specific-field]), but that's only partially true, since the second key is required depending on the value of ::role. Anyone have an idea how to spec this? Thanks šŸ™‚ šŸ™‚#2017-10-3018:39taylorI think thereā€™s a multi-spec example in the spec guide very similar to that#2017-10-3018:41taylorhttps://clojure.org/guides/spec#_multi_spec#2017-10-3018:44stex@taylor Thanks! Didn't catch that for some reason šŸ˜„#2017-10-3021:55abpHey, considering https://dev.clojure.org/jira/browse/CLJ-2116 is the top two voted ticket for Clojure right now, I thought about this general issue for quite a while last year and as a result wrote a spec that, given a root spec, recursively parses specs into reversed dependency order, to do a spec rewrite to a new root, while replacing certain parts of the spec-tree. The latter algorithm isn't implemented yet, I put it aside for quite a while. It shouldn't be too much work but maybe I'd need to get tools.analyzer into the mix for rewrites of multispecs. And then there was Rich announcing better programmability of specs in his conj keynote. So the question is, is it still worth pursuing this? Any ideas about how deep the rabbit hole might get? I'm aware of some libs doing spec parsing into records etc. but for some reason I'd like to avoid that (you're welcome to convince me). Also I haven't looked into term rewriting at all, but maybe that's what I'm trying to do.#2017-10-3022:07abpAh, https://github.com/jpmonettas/inspectable/blob/master/src/inspectable/spec_utils.cljc#L24 is a nice solution for the multispec-problem.#2017-10-3117:01ikitommi[metosin/spec-tools "0.5.1"] works now with the latest spec & clojure beta.#2017-11-0109:40hkjelsHow do I get the merged form of a spec?#2017-11-0114:42Alex Miller (Clojure team)You donā€™t#2017-11-0122:35hkjelshttps://github.com/lab-79/clojure-spec-helpers/blob/f3d7c48c55bbe4dc901cd7f1ace1e02b3777bd1b/src/lab79/clojure_spec_helpers.cljc#L61 This solved my issue for now#2017-11-0112:05rickmoynihanDoes clojure/spec have a predicate that matches g/simple-type?#2017-11-0112:05rickmoynihanbut no more#2017-11-0112:05rickmoynihani.e. doesnā€™t match complex (collection) types#2017-11-0114:43Alex Miller (Clojure team)no, but (def simple? (comp not coll?)) is a good first approximation#2017-11-0114:56mpenetOr (complement coll?)#2017-11-0114:56Alex Miller (Clojure team)yep, gooder#2017-11-0114:58Alex Miller (Clojure team)well, mine was shorter :)#2017-11-0114:58Alex Miller (Clojure team)ā€œcomplementā€ is a long word :)#2017-11-0115:30mpenetšŸ™‚#2017-11-0120:34uwoam I right to understand that you canā€™t document keys with (s/keys :opt [::some-key]) without also providing the spec for ::some-key?#2017-11-0120:47seancorfieldIt won't generate without a spec for ::some-key but it will validate just fine -- it will treat missing specs as any? I believe.#2017-11-0120:50seancorfield^ @uwo is that the scenario you are asking about?#2017-11-0121:06uwo@seancorfield yes! thatā€™s the way I thought/hoped it worked, but I was calling s/assert in some clojurescript code and it was complaining about some keys that I had declared in s/keys, but that did not have their own spec.#2017-11-0121:55ajs@alexmiller those were some meaningful tweets #2017-11-0203:51thedavidmeisteri have a really basic question...#2017-11-0203:51thedavidmeisterhow do i get#2017-11-0203:52thedavidmeister(def foo [:foo/bar]) (spec/def (spec/keys :req foo)) to not throw errors?#2017-11-0203:52thedavidmeisterDon't know how to create ISeq from: clojure.lang.Symbol#2017-11-0204:18seancorfield@thedavidmeister That's because spec/keys is a macro so it doesn't eval its arguments.#2017-11-0204:18thedavidmeisterhmm, is there a way to get around that?#2017-11-0204:18thedavidmeisterit would be nice to reuse foo elsewhere#2017-11-0204:18seancorfieldWhat problem are you trying to solve?#2017-11-0204:18thedavidmeisterjust generally keeping things DRY#2017-11-0204:19thedavidmeisteri have a list of keys that i'd like to put in a spec, but also use elsewhere#2017-11-0204:19thedavidmeistere.g. putting in a list of options for a dropdown menu#2017-11-0204:19seancorfield
(s/def ::foo (s/keys :req [::foo/bar]))
(last (s/form ::foo))
Try that.
#2017-11-0204:21thedavidmeisterhmm yes, that does work but#2017-11-0204:21thedavidmeisterthen if i add :opt in the positions of things in form change#2017-11-0204:21seancorfieldi.e., make the spec the system of record and get the keys from it elsewhere#2017-11-0204:21thedavidmeisteryeah i see, is there a more robust way to extract things from spec?#2017-11-0204:21thedavidmeisterk/v style?#2017-11-0204:21seancorfield(let [[& {:keys [req req-un opt opt-un]}] (rest (s/form ::some-spec))] ...)#2017-11-0204:22thedavidmeistermmm, because (::foo (s/form ::foo)) didn't work#2017-11-0204:23seancorfieldIt's just data shrug#2017-11-0204:23seancorfieldThe spec should be the system of record tho'...#2017-11-0204:23thedavidmeistermaybe#2017-11-0204:23thedavidmeisteri don't necessarily see why it should be#2017-11-0204:24thedavidmeisterand it seems like trying to use it that way is clunky at best#2017-11-0204:24seancorfieldThere are JIRA issues around providing a more programmatic API at some point.#2017-11-0212:47Alex Miller (Clojure team)Rich is working on a significant spec update that will include more support in this area#2017-11-0215:27uwosuper awesome! around the idea of clj-2112?#2017-11-0215:37Alex Miller (Clojure team)no, this is in the internals#2017-11-0204:25thedavidmeisterif it hasn't been prioritised to date, isn't that evidence against the idea of "should be used in this way"?#2017-11-0204:26seancorfieldSpec is designed for human writing and comprehension right now, not for programmatic writing. But at least it's data so you can programmatically read and comprehend them.#2017-11-0204:26thedavidmeisterit's true, it's better than nothing atm#2017-11-0204:27thedavidmeisteri can manually pick apart the form of the spec and alias it to something more convenient#2017-11-0204:27thedavidmeisterbut if the intention is truly that spec is a "one stop shop" for this type of thing, then this API needs some polishing >.<#2017-11-0204:28thedavidmeister(::foo (:keys (s/xxx ::foo))) seems to be what i'm reaching for here#2017-11-0204:29thedavidmeisterwhere s/xxx is not s/form but something along those lines#2017-11-0204:30seancorfieldI just tried this in the REPL. I think it's clearer:
(let [[& {:keys [req req-un opt opt-un]}] (rest (s/form ::foo))] req)
#2017-11-0204:30seancorfieldThat will destructure all four possible keys options.#2017-11-0204:30thedavidmeisteroh nice#2017-11-0204:30thedavidmeisteryeah something like that šŸ™‚#2017-11-0204:31thedavidmeisterthanks for the help#2017-11-0204:31seancorfieldIt won't deal with s/merge constructs tho'... but what we do is have a bunch of basic specs with s/keys and those are what we destructure to get the actual keys, and then we s/merge them for compound specs.#2017-11-0204:31seancorfield(funnily enough I was writing some code to do exactly this at work today)#2017-11-0204:32seancorfieldAnd we have other code that uses specs as the basis of CRUD-style data functions too...#2017-11-0204:32thedavidmeisteri think atm it's pretty good getting things in to spec but not so much getting them back out#2017-11-0204:32thedavidmeisteractually it would be pretty sweet if i could plug spec straight into my UI#2017-11-0204:33thedavidmeisterbut it seems to not quite be there yet#2017-11-0204:34seancorfieldI think it's mostly a matter of tooling -- a lot of which will come from the community.#2017-11-0204:34thedavidmeistertotally, i've seen a lot of cool tooling already just lurking in this chat#2017-11-0204:34seancorfieldGiven that 1.9 is still in pre-release, not everyone is using it yet, which is holding back tooling.#2017-11-0204:35seancorfieldWe have 1.9 Beta 3 in production and we're heavy users of spec but it's all still a bit of an adventure...#2017-11-0204:35thedavidmeisteryeah i know that hoplon had to make some changes to keep pace with 1.9, and also has started playing around with spec but it's WIP#2017-11-0204:35thedavidmeisteri imagine it is the same for everyone#2017-11-0204:36seancorfieldBeta 4 broke our code due to removing bigdec? which was added earlier in 1.9. But it was a small change. We expect to have Beta 4 in production on Monday.#2017-11-0204:36thedavidmeisternice šŸ™‚#2017-11-0204:36thedavidmeisterwell i g2g run some errands#2017-11-0204:36thedavidmeisterthanks for the help#2017-11-0216:11uwo@seancorfield I was able to create minimal reproduction. This only occurs in cljs. Should I make a ticket for this?
(s/def ::test (s/keys :opt-un [::unspeced-keyword]))

(s/def ::test-fn
  (s/fspec :args (s/cat :an-arg ::unspeced-keyword)))

(s/assert ::test-fn (fn [a]))
;; => #object[Error Error: Unable to resolve spec: :cljs.user/unspeced-keyword]

#2017-11-0216:14uwoguessing this is related to fspec invoking a generator, but dunno#2017-11-0221:28Alex Miller (Clojure team)I believe it is. fspecs will be checked by using the :args generator to verify the function can accept those args. In this case itā€™s trying to generate ::test instances that sometimes include ::unspeced-keyword instances, but it doesnā€™t know how to do that.#2017-11-0221:28Alex Miller (Clojure team)or at least thatā€™s my guess, maybe something else going on#2017-11-0313:43uwoShould I create a jira cljs ticket for this?#2017-11-0314:06Alex Miller (Clojure team)sure#2017-11-0216:54seancorfieldIf that works in Clojure but fails in cljs, raise a JIRA issue. If it failed in both I'd say it was related to generators. If it's only failing in cljs, I'm not so sure.#2017-11-0216:55seancorfieldConfirmed it works in Clojure, so that looks like a cljs bug.#2017-11-0216:58uwothanks. will do#2017-11-0219:07bmabeyHow do I use a sequence spec as a function argument? This is what I've tried so far without any luck:
(s/def ::int-then-strings (s/cat :num int? :strs (s/+ string?)))

(defn blah [stuff m]
  (first stuff))

(s/fdef blah
        :args (s/cat :list-of-int-then-strings ::int-then-strings :map map?))

(stest/instrument)

(s/valid? ::int-then-strings [2 "foo"]) ; => true

(blah [2 "foo"] 3)
;; clojure.lang.ExceptionInfo: Call to #'blah did not conform to spec:
;; In: [0] val: [2 "foo"] fails spec: :int-then-strings at: [:args :list-of-int-thenstrings :num] predicate: int?
;; :clojure.spec.alpha/spec  #object[clojure.spec.alpha$regex_spec_impl$reify__1200 0x37194692 "
#2017-11-0219:10taylor
(s/def ::int-then-strings (s/spec (s/cat :num int? :strs (s/+ string?))))

(defn blah [stuff m]
  (first stuff))

(s/fdef blah
        :args (s/cat :list-of-int-then-strings ::int-then-strings :map map?))

(stest/instrument)

(s/valid? ::int-then-strings [2 "foo"]) ; => true

(blah [2 "foo"] {:foo 3})
#2017-11-0219:10taylorā˜ļø this works#2017-11-0219:11taylortwo changes: your ::int-then-strings spec wrapped in s/spec to prevent it from getting ā€œflattenedā€ into one big sequence spec, and your test call was invalid according to your spec: it was an integer and not a map#2017-11-0219:11bmabeyThanks! Why is the extra s/spec needed around the s/cat?#2017-11-0221:30Alex Miller (Clojure team)because all regex specs combine to describe a single level of sequential collection. The s/spec forces a boundary such that the outer regex spec includes a collection which has an inner regex spec.#2017-11-0220:58xiongtxBeen getting some problems when using clojure.spec.alpha, clojure.test.check.generators, and boot-cljā€™s cljs compilation. See issue: https://github.com/clojure-emacs/cider/issues/2104 and minimal repo: https://github.com/xiongtx/reload-error-boot @alexmiller Is it not recommended to use clojure.test.check.generators directly when using clojure.spec? It seems to me thereā€™s weird interaction b/t the lazy combinators, CLJS compilation, and namespace reloading. I suspect we can avoid this problem by using only clojure.spec.gen.alpha.#2017-11-0221:33Alex Miller (Clojure team)I donā€™t understand the problem - too much tooling and other stuff for me to get it. Shouldnā€™t be any issue with using clojure.test.check.generators directly - thatā€™s the same thing clojure.spec.gen.alpha is doing, just with a delayed load step.#2017-11-0221:40Alex Miller (Clojure team)generators in test.check are records iirc - maybe youā€™re running into a case where the record is being redefined and the old generators are no longer instances of the new (reloaded) record class?#2017-11-0223:19xiongtxGood insight! Yes, that seems to be precisely whatā€™s happening. After a refresh:
(let [spec (s/int-in 0 10)
      g (s/gen spec)
      h (gen/->Generator (constantly 1))]
  (println "clojure.spec.alpha generator's classloader: " (.getClassLoader (type g)))
  (println "clojure.test.check.generators generator's classloader: " (.getClassLoader (type h))))

;; clojure.spec.alpha generator's classloader:  #object[clojure.lang.DynamicClassLoader 0x73867ca9 
The problem is that clojure.test.check.generators was reloaded (and the defrecord re-evaluated), but the same was not done for the lazy combinators in clojure.spec.gen.alpha. This is probably b/c clojure.spec.gen.alpha does not :require clojure.test.check.generators, so the dependency tracking in clojure.tools.classpath... isnā€™t working properly.
#2017-11-0223:55xiongtxI believe there was some talk of giving clj a way to lazily load :requireed namespaces. That, if implemented, would seems like the ideal solution here.#2017-11-0301:00James VickersWhat is the shape of the data that gets passed to the :fn argument of s/fdef? Does someone have an example of that (or a way to print it)?#2017-11-0301:00tayloritā€™s a map with :ret and :args keys#2017-11-0301:01taylorthe :args value is the conformed value of the args I think#2017-11-0301:02taylor
:fn (fn [{:keys [args ret]}]
              ...
#2017-11-0301:03James VickersThanks. Is that in the docs somewhere?#2017-11-0301:04Alex Miller (Clojure team)That is all correct - the values of the map are the confirmed values of the args and ret specs #2017-11-0301:05Alex Miller (Clojure team)Iā€™m not sure where it would be docā€™ed in the docstrings#2017-11-0301:05James VickersThanks#2017-11-0301:05taylor
:fn A spec of the relationship between args and ret - the
  value passed is {:args conformed-args :ret conformed-ret} and is
  expected to contain predicates that relate those values
#2017-11-0301:05taylorfdef docstring#2017-11-0301:06James VickersAh, thanks.#2017-11-0301:07James VickersSo, if the function returns a single value (like a number), then (% :ret) in the :fn spec should yield the return value?#2017-11-0301:08Alex Miller (Clojure team)Depends on the ret spec#2017-11-0301:08taylorit might be tagged/conformed though I guess?#2017-11-0301:08Alex Miller (Clojure team)If itā€™s a simple pred then yes #2017-11-0301:09Alex Miller (Clojure team)Right#2017-11-0301:10James VickersSweet, thanks.#2017-11-0301:10James VickersI was surprised that putting a do expression with a println in the :fn spec didn't print anything when the function was called (instrumentation on)#2017-11-0301:11taylorthe :fn spec doesnā€™t come into play re: instrument calls#2017-11-0301:11taylorit does get used if you check the function though#2017-11-0301:11James Vickersoh#2017-11-0301:12taylor> Instrumentation validates that the :args spec is being invoked on instrumented functions ā€¦#2017-11-0301:12taylor> check will generate arguments based on the :args spec for a function, invoke the function, and check that the :ret and :fn specs were satisfied.#2017-11-0301:13James VickersCool. I ran a check and it printed out what was passed to :fn.#2017-11-0313:17zcljIf I have a production spec of say an email address, it will require a custom generator if I want to generate test data. Since a generator typically will not be used in production would it be a good practice to redefine the production spec in i test ns that will include the generator, keeping any generators out of production code? Can this lead to problems if I have nesting and to be able to generate the top level spec I need to redefine lower levels with the generator in my tests?#2017-11-0314:04Alex Miller (Clojure team)Another thing to consider is using generator overrides at the point of testing. That way you donā€™t have to redefine a spec, you just supply a set of alternate generators.#2017-11-0314:19zcljBut in the case of a nested spec such as a person having an email, and I am testing the person don't I have to redefine the email spec refered by the person spec? Is generator overriding done by using with-gen or are there other ways I have missed?#2017-11-0314:29Alex Miller (Clojure team)no, you can supply custom generators in exercise, instrument, check, etc#2017-11-0314:31Alex Miller (Clojure team)in stest/check for example, you can supply a :gen map in the options: ā€œmap from spec names to generator overridesā€#2017-11-0314:31Alex Miller (Clojure team)so that way you can supply overrides just in the context of a test#2017-11-0314:53zcljah I see, will try that out, thanks for your help!#2017-11-0321:30seancorfieldGood to be reminded of that. So far, we've tended to write wrappers for generators so we can lazy load the testing library (and therefore keep the actual "testing generator" in the production code without needing the testing libraries in production).#2017-11-0313:25wilkerluciohello, has anyone here though about the idea of specing nested structures? currently s/keys only supports flat structures, if I need a nested I have to define the nested structure ahead of time on that key, this prevents different nesting structures depending on the context#2017-11-0313:26wilkerluciomaybe we could support via a syntax like the datomic pull syntax, eg: (s/keys :req [:user/name {:user/address [:address/line1 :address/city]}])#2017-11-0314:04ikitommi@U066U8JQJ have you checked https://github.com/metosin/spec-tools/blob/master/README.md#data-specs ? nests also vectors and sets. In the end, just generates specs with alternative (macro free) syntax.#2017-11-0314:07ikitommialso would like to see support for nested keys-specs in spec itself. #2017-11-0314:07Alex Miller (Clojure team)thatā€™s not in line with the ā€œset of attributeā€ thinking and no plans for that#2017-11-0314:55wilkerluciothe issue I'm facing is that for the "container" specs I might want different subsets of keys, depending on the context#2017-11-0314:55wilkerlucioso by not having a fixed for the children, it gets tricky#2017-11-0315:06Alex Miller (Clojure team)you donā€™t need a fixed spec for the children#2017-11-0315:07Alex Miller (Clojure team)an empty (s/keys) is sufficient to cover an open set of attributes#2017-11-0315:07Alex Miller (Clojure team)or use s/multi-spec to select the spec based on the contents#2017-11-0315:07Alex Miller (Clojure team)or s/or multiple choices, etc#2017-11-0315:08wilkerluciothanks for the tips Alex, I like the open one, but at same time, if I want to require different sets of keys depending on the context, that doesn't work, the multi-spec can work, but its a lot more involved#2017-11-0315:11wilkerlucioI love the idea of living by just attributes, but if I need to give a name to a context of sub-attributes, I feel like backing again to the "box" (class, entity, whatever...) constraints again#2017-11-0315:17wilkerlucio@U055NJ5CC thanks for pointing that out, I'll look it up#2017-11-0315:18Alex Miller (Clojure team)what youā€™re saying is that they key at the top level does not have a stable semantic meaning so I would think about what that means and whether itā€™s a good idea#2017-11-0315:19wilkerlucioyeah, in general terms, any sub-set might be unstable, if what you care is just about the leaf attributes validation#2017-11-0315:21wilkerlucioI've been writing a considerable amount of code regarding to data fetching apis (om.next style), and many times I see that the sub-set of keys of a child element can vary wildly, in my case I'm embracing this and it's working pretty good, but I can't get the specs around to match it in the way it is#2017-11-0315:21Alex Miller (Clojure team)(s/keys) ! :)#2017-11-0315:21wilkerluciothe problem is the children, for example, working in micro-service architecture#2017-11-0315:22wilkerlucioI have many endpoints across services that can return different sub-sets of the data#2017-11-0315:22wilkerlucioand altough they share some root keys, what comes in the children is variable, some endpoints give more, some give less information#2017-11-0315:22wilkerlucioand in current spec way, I can't define what is required for each return (or input) on a case-to-case bases#2017-11-0315:23wilkerlucioin the same way we can't have a good definition of what are the required fields for an user (a login might be user/password, a signup would require much more)#2017-11-0315:24wilkerlucioI feel the same problem when trying to specify requirements for a nested item#2017-11-0315:24Alex Miller (Clojure team)I think thatā€™s all ok, and you should not try to define every key set aggregation#2017-11-0315:24Alex Miller (Clojure team)lean on the attributes#2017-11-0315:27wilkerluciobut that's the problem, if I have one key like :user/address, I expect this key to be the same always, but what I expected to be inside of it can change from case to case#2017-11-0315:27wilkerlucioon the top level, just create a new set and we are all good#2017-11-0315:27wilkerluciofor the nested, there is no way to override the requirements#2017-11-0315:28wilkerlucioto say that the sub-set is different a separated case, makes sense?#2017-11-0315:29Alex Miller (Clojure team)why not just (s/def :user/address (s/keys)) ?#2017-11-0315:29Alex Miller (Clojure team)then rely on your address attribute specs to do the work#2017-11-0315:34wilkerluciothe only issue there is that we can't make some attributes required for a context that way#2017-11-0315:34wilkerlucioso the validation gets too loose#2017-11-0315:36Alex Miller (Clojure team)if you need that then it sounds like s/multi-spec to me, or state all the possibilities with s/or, or add outer constraints s/andā€™ed at the top level#2017-11-0315:36wilkerluciohumm, the top level constraint sounds like a good path for the cases I'm thinking#2017-11-0315:36wilkerluciothanks, I'll try that and see how it goes#2017-11-0315:36Alex Miller (Clojure team)always come back to ā€œstate the truth about what itā€™s in your dataā€ - what can occur in your actual data? say that in the spec.#2017-11-0313:27wilkerluciohad anyone felt the desire to specify the kinds on that way?#2017-11-0520:26Drew VerleeI just spenT a day thinking about how to use specs for human error msgs. i think phrase is interesting but it seems to lack a way to do some rather useful things and makes others somewhat more difficult then then should be. The one problem i can't think around is naturally combining your "for user erorrs" together when you concat specs together.#2017-11-0613:27dominicm@drewverlee I think the human error messages also need to include a location which is separate from their spec position. Sometimes you need to do validation across keys for humans, but place the error message on a particular key (password & password_confirm must match, but if they don't, the error is in password_confirm)#2017-11-0614:24gfredericks#2017-11-0618:13seakoi find them noisy and like that test.chuck turns them off when using the checking macro#2017-11-0616:51Drew VerleeIs there a rational for the :reason key in the problem map produced by spec/explain-data?. I cant find any discussion on it#2017-11-0620:34Drew Verleeis there a function that you can call on a spec to get the code for the spec?
(s/data some-spec) 
;;=> (s/def ::some-spec string?)
#2017-11-0620:35taylorhttps://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec/form#2017-11-0620:36taylor(s/form some-spec) in your example should work @drewverlee#2017-11-0620:37Drew VerleeIt seems to only return part of the spec.
(s/def ::kid (s/keys ::req [::name]))
(s/form ::kid) 
;;> (spec/keys)
#2017-11-0620:38taylorI think that keys spec is invalid#2017-11-0620:39taylortry :req instead of ::req#2017-11-0620:39Drew Verleegood catch#2017-11-0621:09Drew VerleeHmm, how about something similar for spec functions? those defined using fdef?#2017-11-0621:10taylor
(s/form `foo)
where foo is your function
#2017-11-0716:14Drew VerleeHow does that work? backtick stops evaluation of things and qualifies their namespace. Why does that help?#2017-11-0716:17taylorhttps://clojure.org/guides/weird_characters#syntax_quote#2017-11-0716:17taylor> However, symbols used within a syntax quote are fully resolved with respect to the current namespace#2017-11-0716:19taylor
(defn foo [x] x)
=> #'sandbox.core/foo
foo
=> #object[sandbox.core$foo 0x111da369 "
#2017-11-0716:38Drew Verleegotcha. I was actual experiencing another mis understanding the same time concerning lists.#2017-11-0621:12taylors/fdef is using s/def ā€œbehind the scenesā€ to register the spec to the function symbol, so you can use the function symbol to resolve the spec (instead of a namespaced keyword)#2017-11-0709:26mattiaswmap-like? I have an application and I use maps a lot. Added spec, most of them using s/keys. In some cases I want to apply operations to the maps like filter. But this breaks the specs, since it isn't a map any more, but a sequence of pairs. The simple solution is of course to convert the result of filter back to a map, but is there a better way? Here is a small sample
(s/def ::test-id int?)
(s/def ::test-data string?)

(s/def ::test1
  (s/keys :req-un [::test-id ::test-data]))

(s/fdef spec-test
        :args (s/cat :m ::test1)
        :ret  int?)

(defn spec-test
  [m]
  (count m))

(def test1-sample {:test-id 1 :test-data "hello"})

(defn works
  []
  (spec-test test1-sample))

(defn works-not-which-is-ok
  []
  (spec-test (dissoc test1-sample :test-data)))

(defn works-not-which-is-not-ok
  []
  (spec-test (filter (fn [_] true) test1-sample)))
#2017-11-0713:17Alex Miller (Clojure team)You can spec them as s/coll-of an s/tuple of key value pairs. Thatā€™s not great if you are still relying on attribute keys. I guess you could also use s/keys* on the kv tuple.#2017-11-0714:19mattiaswIf I understand it correctly, s/keys* wants the structure [:a 1 :b 2], but I have [[:a 1][:b 2]]#2017-11-0813:18Alex Miller (Clojure team)Yeah, youā€™d want coll-of keys*#2017-11-0712:08rickmoynihanWhat is the best way to spec a map which has a single required key (s/keys :req-un [::id]) (s/def ::id int?) where every other (optional) map key is a string? with a string? value? Such that (s/valid ::spec {:id 123}) ;; => true, (s/valid ::spec {"foo" "bar" :id 123}) ;; => true, (s/valid ::spec {"foo" "bar"}) ;; => false#2017-11-0713:08taylorhereā€™s a really naive way to do it:
(s/def ::my-map
  (s/and (s/keys :req-un [::id])
         #(every? (fn [[k v]]
                    (or (= :id k)
                        (and (string? k) (string? v))))
                  %)))
maybe thereā€™s a better way, but Iā€™m not sure how youā€™d combine keys + map-of specs like this
#2017-11-0713:13Alex Miller (Clojure team)These are sometimes called hybrid maps - I have a blog about the spec for destructuring which covers the techniques for handling them. http://blog.cognitect.com/blog/2017/1/3/spec-destructuring#2017-11-0713:36taylorvery nice, then something kinda like this might work:#2017-11-0713:36taylor
(s/every (s/or :id (s/tuple #{:id} int?)
               :str (s/tuple string? string?)))
#2017-11-0713:37rickmoynihantaylor: yes I had something similar to your first, but the use of and & or in a single monolothic predicate bothered me, as it kills error message granularity further down the tree.#2017-11-0713:38rickmoynihanI think as youā€™ve discovered the use of every and or looks to be how to do it šŸ™‚#2017-11-0713:38rickmoynihanThanks @U064X3EF3 for the pro tips#2017-11-0712:31sushilkumarHow to write spec for ā€œstring of integerā€? I want to create a generator of "string of integer" which should only generate valid string of integer (value within Integer/MIN_VALUE and Integer/MAX_VALUE).#2017-11-0712:31sushilkumarHow to write spec for ā€œstring of integerā€? I want to create a generator of "string of integer" which should only generate valid string of integer (value within Integer/MIN_VALUE and Integer/MAX_VALUE).#2017-11-0712:45tayloryou could write a predicate function int-str? that returns true/false if the given string can be parsed as an integer, then itā€™s trivial to use that predicate as a spec#2017-11-0712:45gfredericksThe generator would be (gen/fmap str an-appropriate-int-generator)#2017-11-0714:36sushilkumarThanks @U3DAE8HMG and @U0GN0S72R for sharing your ideas. Now I am able to do it as follows and it will also help in writing fdefs.
(defn- in-integer? [x]
 (and (>= x Integer/MIN_VALUE) (<= x Integer/MAX_VALUE)))                      (s/def ::in-integer?
 (s/and number? in-integer?))           (s/def ::str-long?
 (s/spec string?
         :gen #(gen'/fmap str (s/gen ::in-long?))))
#2017-11-0718:54seancorfieldQuick Q about double-in -- if I specify :min 0.0 and :max 1.0, does that automatically exclude NaN and infinity? Or do I also need to specify :NaN? false :infinity? false?#2017-11-0718:56taylorlooking at https://github.com/clojure/clojure/blob/d920ada9fab7e9b8342d28d8295a600a814c1d8a/src/clj/clojure/spec.clj#L1630 it doesnā€™t look like specifying min/max has any effect on NaN/infinity (and they both default to true)#2017-11-0718:57taylorbut
(s/valid? (s/double-in :min 0 :max 1) Double/NaN)
=> false
#2017-11-0718:59taylorso I guess it implicitly excludes NaN/infinity by virtue of those not passing the range comparator checks?#2017-11-0719:01taylor
(s/valid? (s/double-in :min 0.0) Double/POSITIVE_INFINITY)
=> true
#2017-11-0719:24seancorfieldThanks @U3DAE8HMG That's sort of what I intuitively expected to happen but I wasn't sure how "special" NaN was... I guess that begs the question of what do you do if you want 0.0 .. 1.0 or NaN? I suppose you have to :or two specs together... but what would that second spec look like, i.e., how would you allow only NaN or a range?#2017-11-0719:27taylor#2017-11-0719:27taylora more qualified person could very well have a better answer!#2017-11-0719:59seancorfieldThat allows any double, it seems.#2017-11-0720:33taylorha! yeah it doesā€¦ disregard#2017-11-0720:34taylorlooking at double-in impl. :NaN? true is a no-op#2017-11-0720:36taylor
(s/or :range (s/double-in :min 0.0 :max 1.0)
      :nan #(and (double? %) (Double/isNaN %)))
#2017-11-0817:12akhudekWhat is the rational for s/cat returning a map when conformed?#2017-11-0817:13akhudekI suppose there is no way to conform a sequence and get a sequence rather than a map?#2017-11-0817:13akhudekI know you can use coll-of but that doesnā€™t work if you want a sequence rather than a collection#2017-11-0817:16bfabryyou can make it non conforming#2017-11-0817:17bfabrythe rationale is that s/cat starts a regex spec, and regexes have alternates, so you need to give everything names to know which part of the regex it matched#2017-11-0817:18bfabrythere's a non-documented not currently supported function that makes a spec nonconforming https://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/alpha.clj#L1761#2017-11-0817:30rickmoynihanare there any docs/guides etc on conforming / conformers and tweaking output etc? I understand roughly how they work, but am curious about how to apply them in practice. What itā€™s ok to use them for, what itā€™s not a good idea to use them for etcā€¦#2017-11-0818:13Alex Miller (Clojure team)conformers exist primarily to build custom composite spec types (s/keys* for example)#2017-11-0818:27rickmoynihaninteresting hadnā€™t seen s/keys* before#2017-11-0819:04Alex Miller (Clojure team)s/nilable was originally written this way too although I ended up rewriting a custom impl for better performance#2017-11-0818:13Alex Miller (Clojure team)generally I would say you should not use them for data coercion or tweaking output#2017-11-0818:13zcljI have a case where I try to spec a HOF but I can not get check to satisfy. Here's a toy example of the problem:
(def uuid-regex #"^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$")

(s/def ::name string?)
(s/def ::age pos-int?)
(s/def ::id (s/and string? #(re-matches uuid-regex %)))
(s/def ::person (s/keys :req-un [::name ::age ::id]))

(s/fdef do-stuff
        :args (s/cat :person ::person
                     :lookup (s/fspec :args (s/cat :p ::person) :ret int?)))

(defn do-stuff [person lookup-fn]
  (let [result (lookup-fn person)]
    :important-work-on-result))

;; works
(s/exercise ::person 1 {::id (fn [] (gen/fmap str (gen/uuid)))})

;; do not work
(s/exercise `do-stuff 1 {::id (fn [] (gen/fmap str (gen/uuid)))})

;; do not work
(stest/check `do-stuff {:gen {::id (fn [] (gen/fmap str (gen/uuid)))}})
I am also open to that there are better ways to run check on HOFs that I am missing
#2017-11-0818:20Alex Miller (Clojure team)youā€™ll need to supply a generator on s/fspec#2017-11-0818:22Alex Miller (Clojure team)the thing thatā€™s breaking down is really the gen of ::id I think#2017-11-0818:25zcljis there anyway to define an override? For example if the fspec was provided by a lib?#2017-11-0818:32Alex Miller (Clojure team)what youā€™re doing should be working - I think this is a bug#2017-11-0818:34Alex Miller (Clojure team)(s/exercise (s/fspec :args (s/cat :p ::person) :ret int?) 10 {::id #(gen/fmap str (gen/uuid))}) is minimally sufficient#2017-11-0818:37zcljGood to know, then I can stop scratching my head. Yes, that example yields the same result#2017-11-0818:50Alex Miller (Clojure team)yeah, I see the bug in the code. canā€™t say I know how to fix it though.#2017-11-0818:52Alex Miller (Clojure team)when the fspec is conformed by exercise (or by check), it ā€œchecksā€ the generated function spec, but that conform does not have access to the gen overrides so it gens without them.#2017-11-0818:54zcljI see, that explains the observed behavior#2017-11-0818:59zcljshould I file a ticket? Have not done so before but I am willing to try if that would be helpful#2017-11-0818:59Alex Miller (Clojure team)Iā€™m filing one#2017-11-0819:02Alex Miller (Clojure team)https://dev.clojure.org/jira/browse/CLJ-2264#2017-11-0819:05zcljthanks, will follow the activity, and thanks for your help and explanation of the problem#2017-11-0819:07Alex Miller (Clojure team)itā€™s kind of a deep problem - weā€™ve got other tickets that are related but I never took the time to really get it.#2017-11-0819:26zcljI would be happy to help, but it sounds like its above my knowledge of the code base if it is a deep problem, not a novice ticket#2017-11-0818:15seancorfield@rickmoynihan Where I've found conformers useful is dealing with input data that is all strings, but is expected to conform to numeric, boolean, date, etc types.#2017-11-0818:16rickmoynihan@seancorfield: yeah I understand the input coercion at boundaries case#2017-11-0818:16seancorfieldThat's pretty much the only place I'd "recommend" them, having worked with spec in production since it first appeared.#2017-11-0818:19rickmoynihanok thanks, thatā€™s useful. I just remember rich saying that conformed values are basically the parsed/labelled output you want; and itā€™s very close to a shape I want for a specific caseā€¦ thinking itā€™s better to conform then post-process, rather than use the conformers to do it.#2017-11-0818:20rickmoynihanbut it occurred to me that if I used conformers then Iā€™d really need to write unformers too, and my spec output would no longer be spec outputā€¦ so what youā€™re saying seems to agree my gut feeling#2017-11-0818:39rickmoynihanso one thing Iā€™d quite like (for myself) is to write a variant of s/keys and s/def that uses URIs as keys instead of clojure keywords. I work with RDF, so mapping to keywords just to spec something feels somewhat redundant, when the URI straight out of the database is basically the same thing so Iā€™d like to basically save a redundant mapping and write: (def rdfs:label (URI. "http://,,,/label")) (rdf/def rdfs:label string?) (rdf/keys :req [rdfs:label]) And have it do the same thing as keys. Is there an easy way to do this? Thinking I need to extend s/Spec/`s/Specize` to URI and then rewrite s/keys? s/keys looks pretty hairy, is there any easy way?#2017-11-0819:03Alex Miller (Clojure team)I think this is roughly how you would do this, yes, and it would not be super easy. Also, I expect some of that plumbing to possibly change soon.#2017-11-0819:11rickmoynihanIā€™m assuming there are no plans/proposals in the pipeline to let you plugin and you use arbitrary values as keys/keywords?#2017-11-0818:40rickmoynihanor am I thinking the thoughts of a madman?
#2017-11-0818:43rickmoynihanI think what rich says about the design influence of RDF on clojure and spec is very clear; in many ways theyā€™re almost identical. Iā€™d basically like to reduce the friction of working in clojure with RDFā€¦ Also compare SHACL/SHEX to spec, theyā€™re almost just different syntaxes for the same abstract model.#2017-11-0818:58akhudekok, thanks for the comments bfabry and alexmiller, Iā€™ll avoid using spec for transforms.#2017-11-0900:02rickmoynihanI think Iā€™ve found a bug in clojure 1.9 / clojure.core.specs.alpha:
(if-let [foo 1] foo :clojure.spec.alpha/invalid)

CompilerException clojure.lang.ExceptionInfo: Call to clojure.core/if-let did not conform to spec:
In: [2] val: :clojure.spec.alpha/invalid fails at: [:args :else] predicate: any?
 #:clojure.spec.alpha{:problems ({:path [:args :else], :pred clojure.core/any?, :val :clojure.spec.alpha/invalid, :via [], :in [2]}), :spec #object[clojure.spec.alpha$regex_spec_impl$reify__1188 0xeabf22e "
#2017-11-0900:05rickmoynihanissue appears to be that you canā€™t return the value :clojure.spec.alpha/invalid from macros. Itā€™s kinda like a macro hygiene issue, but for s/invalid#2017-11-0900:06bronsait's known#2017-11-0900:07rickmoynihancool. Figured it would be#2017-11-0900:08bronsasee https://dev.clojure.org/jira/browse/CLJ-1966#2017-11-0900:12rickmoynihanthanks#2017-11-0912:26sushilkumarHello everyone, I am learning Spec'ing higher order functions. Here are the examples which I am experimenting with (actual HOFs I am working on are quite complex) :
(Works fine with exercise-fn.)
(defn hof1 [x f] (f x))
(s/fdef hof1
        :args (s/cat :x number?
                     :f (s/fspec :args (s/cat :y number?)
                                 :ret number?))
        :ret number?
        :fn #(= ((-> % :args :f) (-> % :args :x)) (:ret %)))

(exercie-fn doesn't work :(. Here I am expecting input f to be either + or - or * fn.)
(defn hof2 [f x] (f x 1))
(s/fdef hof2
        :args (s/cat :x number?
                     :f (s/fspec
                          :args (s/cat :y number?)
                          :ret number?))
        :ret number?
        :fn #(= ((-> % :args :f) (-> % :args :x) 1) (:ret %)))
How write spec to ensure correctness in the samples generated using exercise-fn for HOFs? (I want to use spec's check fn to test HOFs on generated inputs based on written spec.)
#2017-11-0914:29Shantanu KumarHi, Iā€™m a spec noob here ā€” can somebody tell me an easy way to automatically (st/instrument) in all namespaces?#2017-11-0914:30joost-diepenmaatyou only need to call st/instrument once#2017-11-0914:30joost-diepenmaatbut you have to make sure youā€™ve loaded all the namespaces before you do#2017-11-0914:30joost-diepenmaatif itā€™s in your dev setup, just call st/instrument in your main or dev namespace at the bottom#2017-11-0914:31joost-diepenmaatif itā€™s in tests, your best bet is probably to use a fixture#2017-11-0914:31Shantanu Kumar@joost-diepenmaat Ah OK, thanks!#2017-11-0915:07Shantanu KumarAlso, how do I generally make ā€œdid not conform to specā€ errors easier to read? Iā€™m using Orchestra+Expound and have specified the following, but to no avail:
(st/instrument)
(set! s/*explain-out* expound/printer)
as mentioned here: https://github.com/bhb/expound#using-orchestra
#2017-11-0915:20bbrinck@kumarshantanu Can you post an example of the code youā€™re calling and the ā€œdid not conform to specā€ message you get?#2017-11-0915:22bbrinckSetting expound/printer seems to work for me https://gist.github.com/bhb/f538f2b1a47eb7a1514da0fe62d46265#2017-11-0915:34Shantanu KumarThis gist works for me at the REPL. @bbrinck The error reporting Iā€™m dealing with extracts the exception message and stack trace and sends it as HTTP response. Is Expound supposed to work in that use case?#2017-11-0916:03bbrinckYes, it should. For instance, building on my previous example, here is code to catch the instrumentation exception and return the message as a string: (try (foo "") (catch Exception e (.getMessage e)))#2017-11-0916:04bbrinckAnother option is to use Expound to validate incoming data directly. For instance, letā€™s say the HTTP endpoint takes some JSON as incoming data#2017-11-0916:06bbrinckThen you could validate that data with something like:
(binding [s/*explain-out* expound/printer]
  (s/explain-str :my-app/http-data-spec http-data))
#2017-11-0916:08bbrinckBy default, Expound writes out all the Clojure specs that failed, but there is an (undocumented, but soon to be documented) option to omit this#2017-11-0916:09bbrinckif youā€™re interested in that, I can give some example code#2017-11-0918:49bbrinckIf anyone else is interested, the issue was the dynamic var *explain-out* is only set in the current thread, and servers like HTTP Kit spawn new threads for requests. The fix is to replace (set! s/*explain-out* expound/printer) with (alter-var-root #'s/*explain-out* (constantly expound/printer))#2017-11-0917:38naomarikthere a better way of having a generator that just executes a function with no regard to the input? this does what i want, but notice the argument here is unnecessary #(gen/fmap (fn [_] (utils/uniform 1e12 1e13)) (s/gen pos-int?))#2017-11-0918:58taylorI was going to suggest gen/return but do you even need the output of the pos-int? generator?#2017-11-0919:04taylorfor example, doesnā€™t matter what the previous generator is doing if youā€™re just constantly returning the same thing regardless of input
(gen/sample (gen/fmap (constantly -1) gen/pos-int))
=> (-1 -1 -1 -1 -1 -1 -1 -1 -1 -1)
#2017-11-0919:23gfredericksthis is a nonstandard generator construction The "right way" would be to use combinators somehow rather than calling a function that supplies its own randomness. That might be a bit more effort, depending on what you want. If you aren't doing things the "right way", then there's not really a better way.#2017-11-0919:24gfredericksdo you really need a uniform distribution between two doubles?#2017-11-0919:24gfredericks(gen/double* {:min 1e12 :max 1e13}) would not be very nonuniform I don't think#2017-11-1305:56naomarik@U3DAE8HMG yeah my point was i didnā€™t need the output of the pos-int? generator. gen/return was EXACTLY what i was looking for, thanks šŸ˜‰#2017-11-1305:57naomarik@U0GN0S72R yeah i need uniform distribution between two large numbers because made a spec that expected a number beteween those two ranges and this issue causes it to fail without a custom generator https://dev.clojure.org/jira/browse/CLJ-2179#2017-11-1305:58naomariki also stumbled upon #(gen/choose 1e12 1e13) that does uniform distribution#2017-11-1312:55gfredericksthat ticket is interesting#2017-11-1312:55gfredericks@U0DHHFEDP what is your spec for exactly?#2017-11-1312:55gfredericksI'm having trouble seeing why something would fail#2017-11-1313:27naomarik@U0GN0S72R itā€™s a ghetto spec for a unix timestamp in ms, the generator fails because it attempts to make ints starting at very low numbers. hereā€™s the code
(s/and pos-int?
          (s/int-in 1e12 1e13))
#2017-11-1313:30naomarikit passes most the time when ran by itself but i have it nested in another datastructure and it fails 100% of the time#2017-11-1313:30gfredericksoooooh; this is an s/and issue, not an s/int-in issue#2017-11-1313:30gfredericksif you swap the order of those two you should have it succeed#2017-11-1313:31gfredericksor set the generator to be the default generator that the s/int-in expression generates#2017-11-1313:31gfredericksI think CLJ-2179 is a red herring here#2017-11-1313:33naomariktried swapping the order, didnā€™t work#2017-11-1313:35naomariki think the issue is legit, if you try running the code (gen/sample (s/gen (s/int-in 0 100))) you get a ton of low numbers#2017-11-1313:35naomarik(0 0 0 1 0 4 0 52 3 34)#2017-11-1313:35naomarikthings like this#2017-11-1313:36naomarikmy final code works well enough though (s/with-gen (s/and pos-int? (s/int-in 1e12 1e13)) #(gen/choose 1e12 1e13) ))#2017-11-1313:36naomariksorry for formatting šŸ˜‰#2017-11-1317:29gfredericks@U0DHHFEDP getting low numbers that are in the range is one thing, but I thought you were describing getting numbers that were out of range#2017-11-1402:33naomarik@U0GN0S72R thatā€™s what I thought was going on, i think my issue is that i was plugging in doubles using the short notation. It seems (s/int-in) doesnā€™t cast those for you but will verify that the input is in range, this is why I needed to pair it with pos-int#2017-11-1402:36naomarik
;; Fails reliably
(s/def ::test2 (s/and pos-int? (s/int-in 1e12 (- 1e13 1))))

;; Fails sometimes
(s/def ::test3 (s/and pos-int? (s/int-in 1e9 (- 1e10 1))))

(s/def ::works (s/int-in (encore/as-int 1e13) (encore/as-int (- 1e14 1))))

#2017-11-1402:38naomarikbumping the numbers up in the first example makes it fail a lot more often, I guess pos-int is the only thing generating numbers here?#2017-11-1402:43naomarikin any case was my fault for plugging doubles in something that is clearly for ints šŸ˜‰#2017-11-1412:06gfredericksyeah, when you use s/and the generator is taken from the first spec and the remaining specs are used to filter#2017-11-1019:02uwoI know this question comes up all the time, and that multi-spec is a way to solve it. but just asking again to see if thereā€™s another solā€™n: we want to ensure that a data structure at some point in our system conforms to a certain shape. The data is nested and the keys already have fixed namespaced names. Itā€™s easy to specify required keys at the top level, but at every nested level the key already has a fully qualified name, and that fully qualified key has different requirements in different system contexts. The only two solā€™ns I see atm are multi-specs and key-name rewriting. Using an empty (s/keys) is not a solā€™n in this case, because we need conform to ensure that required fields at nested levels are present.#2017-11-1019:49Alex Miller (Clojure team)when you say ā€œfully qualified key has different requirements in different system contextsā€ you have started off by violating specā€™s assumption#2017-11-1019:49Alex Miller (Clojure team)unless you spec it to cover all possible contexts#2017-11-1020:52uwo@alexmiller gotcha. I figured that was the answer. Those fully qualified key names are the ones we place in datomic, and they get shuffled all over the app. Itā€™s just different requirements for the ā€œsameā€ data in different contexts#2017-11-1020:55qqqI'm looking at:
(s/def ::ingredient (s/cat :quantity number? :unit keyword?))
(s/conform ::ingredient [2 :teaspoon])
;;=> {:quantity 2, :unit :teaspoon}
I'm starting to get the impression spec is NOT ONLY for unit testing / debugging, but also something intended to be used in live production code ?
#2017-11-1020:55qqqIt's not clear to me how this s/conform is useful, unless it is meant to be used in production to 'reshape' data#2017-11-1021:14taylorI donā€™t think itā€™s intended to arbitrarily reshape data, but the ā€œtaggedā€ output from conform can be useful. For example, if youā€™re expecting different types of inputs, s/or will tag the matched spec in the conform output, otherwise itā€™d be hard to tell which spec matched#2017-11-1021:15taylorI do use it in production for similar cases, one example is a message handler that receives a few different types of messages that donā€™t have a natural ā€œkeyā€. conform output tells me exactly which kind of message it is#2017-11-1021:38seancorfield@qqq We use spec very heavily in production code. We s/conform input values from forms and REST APIs to the desired shape/types.#2017-11-1021:40seancorfieldWe define a spec for each API endpoint, and we s/conform the raw Ring :params to the spec. If it's s/invalid? then we use s/explain-data to guide us in returning error code and messages, otherwise we have a validated parameters map.#2017-11-1021:41seancorfieldWe limit the coercions tho'... We do string->long, string->boolean, string->date, and very little else.#2017-11-1023:17qqq@seancorfield: interesting, I was thinking that something vaguely along those lines would be possible ; nice to see someone do it in practice#2017-11-1023:17qqqunrelated spec question: how does s/or and s/alt differ ?#2017-11-1023:19qqqI'm running them with s/conform ... and getting identital results#2017-11-1023:31seancorfields/alt is for alternatives in a (sequence) regex; s/or is for specific alternatives in a spec.#2017-11-1023:31seancorfield(so s/alt mostly goes with s/cat etc)#2017-11-1023:41qqqIs there a way to define a spec, then selectively, on valid? calls turn "recursion on / recursion off" ? Practically, imagine we are speccing SVG. For a given SVG node, there are (1) properties of this node and (2) the children of this node. I want to be able to define the spec once, and then selectively say: 1. check that this node itself is valid (but don't care much about its children) vs 2. check that this node itself is valid, and that all its children (and all descendants) are valid too so one is a "shallow, just this level check" another is a "deep, check the entire tree check"#2017-11-1023:55bfabryI'm 99.4% sure that the answer is "define 2 specs"#2017-11-1102:07qqqthe problem with two specs is that in the case of maps, the key is used to pic, the spec for the value#2017-11-1102:07qqqwhich mean the data 'decides' whether the check is recursive or non-recursive#2017-11-1105:16seancorfield@qqq But that's exactly the point: with two separate specs you control whether the keys are deep or shallow -- assuming you have unqualified keys in the map and can use different namespace-qualifiers for shallow vs deep.#2017-11-1105:25qqq@seancorfield: but I want to determine whether to make a shallo/dep check at the function call, not at the point when I construct the data#2017-11-1105:26qqqI want a single piece of svg data in some functions, I want to make an expensive deep check taht it's an valid svg tree ikn other functions, I want to make a shallow check that "current node is good; we know nothing abut chiklren"#2017-11-1109:30mattiaswI need just 1 running one call to the function to check at a time. Inside the function, there is a db call, so I need to run checking in single-thread, so (check 'test-many)doesn't work. I tried to use check-fn, but this doesn't even start calling the function. (stest/check-fn 'test-many (s/fspec :args (s/cat :samples (s/coll-of :hashdb.db.commands/data)) :ret nil?)) (I changed backquote to quote when publishing to slack.#2017-11-1201:52James VickersIs there a 'normal' way to use function specs in your automated test suite? I feel like I haven't found a good solution to this - likely because I haven't written a regular test.check suite either.#2017-11-1201:54gfrederickswrapping a call in clojure.test/is is pretty easy#2017-11-1202:08James VickersI was wondering if there was already (`test.check`?) library function that would do something like that (some 'idiomatic' way).#2017-11-1202:09James VickersIs using function specs in the test suite a normal thing that people do, or are folks just using spec for development time help?#2017-11-1202:15gfredericksI would certainly put them in the test suite if I were using spec#2017-11-1202:31seancorfield@jamesvickers19515 We use spec in all sorts of ways. We use it heavily in production code for validating and conforming input parameters, specifying data structures (and then leveraging the form of the spec to "reflect" and get the set of keys that are "permitted", e.g., when storing hash maps in a SQL table), in tests via s/instrument, in tests via s/exercise to generate conforming test data.#2017-11-1202:32seancorfieldIn some cases we use test.check as part of a regular test, but generally we have those as separate tests we run apart from our unit tests -- generative testing often takes quite a while so it probably shouldn't be part of your unit test suite.#2017-11-1203:44jeayeI'm seeing an inconsistency between Clojure and ClojureScript for multi-arity fdefs. https://gist.github.com/jeaye/eaa8da70171a538e23b3c5dbfd9797fd#2017-11-1203:45jeayeThat spec catches the errors just fine, on Clojure, for all the arities. On ClojureScript, bad calls don't get caught by instrumentation at all.#2017-11-1203:50jeayeUpdated the gist; looks like a simple defn, with the same spec, throws for both cases (as it should). So this is specific to something happening in ClojureScript's multi-arity defn.#2017-11-1216:35qqq
(def color-regex #"^#[0-9a-z][0-9a-z][0-9a-z][0-9a-z][0-9a-z][0-9a-z]")

(s/def ::stroke-color (s/and string? #(re-matches color-regex %)))

(s/valid? color-regex "#ffaa88")


what am I doing wrong here, it is complaining that regex can not be converted into a function
#2017-11-1216:45qqqthe last line should be: (s/valid? ::stroke-color ...)#2017-11-1304:53qqqis there a way in spec to say ::x is a vector of numbers ::y is a vector of numbers ::s is a string and the three have the same length (thisxis regarding to an s/keys)#2017-11-1304:55qqqthis is what I have so far:
(s/def ::text-x (s/coll-of number? :kind vector?))
(s/def ::text-y (s/coll-of number? :kind vector?))
(s/def ::text-str string?)
but it's not clear to me how to specify the three have the same length
#2017-11-1306:24seancorfield@qqq Are you using them in a context together? Sounds like you could s/and a length check on top of whatever structure uses all three?#2017-11-1306:29qqq@seancorfield
(s/def ::text-x (s/coll-of number? :kind vector?))
(s/def ::text-y (s/coll-of number? :kind vector?))
(s/def ::text-str string?)
(s/def ::svg-text (s/keys :req [::text-x ::text-y ::text-str ]))

is what I have so far
#2017-11-1306:30qqqYou are recommending (s/and (s/keys ...1) ...2) ?#2017-11-1306:30qqqin ...2 , how do I access the text-x, text-y, text-str fields ?#2017-11-1307:34jumar@qqq something like this?
(defn- vals-same-length?
  [m]
  (->> (vals m)
       (map count)
       (apply =)))
(s/def ::svg-text-same-length (s/and (s/keys :req [::text-x ::text-y ::text-str ])
                                     vals-same-length?))

(s/valid? ::svg-text-same-length {::text-x   [1 2 3]
                                  ::text-y   [10 20 30]
                                  ::text-str "abc"})
(There might be more straightforward implementation of vals-same-length?)
#2017-11-1307:38qqq@jumar: I did not realize I can just pass in an arbitrary clojure function.#2017-11-1307:38qqqThis is very useful, thanks!#2017-11-1308:07seancorfield(sorry, was distracted by the amazing availability of early 80's German electronica on iTunes these days... yes, @qqq what @jumar said... when you use s/and you can have any predicate and it has access to the entire form being processed, so you could access just ::text-x, ::text-y, ::text-str if you wanted, or vals to get all the map's values.#2017-11-1308:07seancorfieldI was thinking you were using this with s/cat for argument specifications, but the same principle applies. Specs are just predicates. It's all functions, all the way down šŸ™‚#2017-11-1319:37degI need to define a spec for a map that holds keys owned by multiple parts of my code. So, looking a a tasteful way to build it incrementally from multiple files. (So that each namespace adds in the parts it understands). This strikes me as non-trivial, especially since I can't just have a defonce'd atom that I assoc into. Because s/keys is a macro, I need to assemble only after I've gathered all the parts. Sounds like I need to think about file loading order and other issues that I've ignored until now in my Clojure/Script. Is there a straightforward way to do this? Or am I barking up the wrong trees?#2017-11-1319:44hiredmanI am not sure what you are asking?#2017-11-1319:45hiredmanif each key is owned by a different part of your code, and you want to spec each key do that#2017-11-1319:45hiredmanand then in whatever central place, create your keys spec from each key#2017-11-1319:46hiredmanthe structure of clojure requires you to have a central place, the way the ns macro and everything else works requires you to have a defined load order and explicitly require the code you need access to#2017-11-1319:52qqqdoes s/coll-of check every element, or does it randomly pick a few and test them ?#2017-11-1319:52taylor> coll-of will exhaustively conform every value#2017-11-1319:53tayloras opposed to every: > Note that ā€˜everyā€™ does not do exhaustive checking, rather it samples coll-check-limit elements.#2017-11-1320:05bbloombeen away for a bit - i remember core had some private convenience macro for checking macro invocations. whatā€™s the latest recommendation on that? i actually want to use the spec for parsing inside the macro, but itā€™s a bit awkward to conform and then if that fails explain, etc#2017-11-1320:23deg@hiredman - I was looking for a way to avoid having to know about all the parts at one central place.#2017-11-1320:25degThat is, I don't want to say (s/keys :opt [ns1/key1 ns2/key2 ,,,]), but instead create the spec dynamically. This seems trickier than it should be, since keys is a macro, so I can't just do a trivial atom/assoc/apply dance.#2017-11-1320:46hiredmanit sounds like you want the already existing feature in spec where a s/keys spec will check specs for keys that exist in the map that have a spec even if they aren't part of the s/keys spec#2017-11-1321:15qqqI assume this is more appropriate for blog post reading rather than discussion: is there a good writeup somewhere of qualified vs unqialified names for specs ? Everything I've done so far in clojure is unqualified, i.e. :foo kws, but looking at spec, it seems like the default / preferred wa yis to be qualified i.e. ::foo or :blah.namespace/foo#2017-11-1321:45arohnerI have a uuid. Is there a smart way to bias the generator so I can control the distribution of uuids? i.e. completely random vs. power law?#2017-11-1321:46arohnerIā€™m aware of e.g. gen/elements, was wondering if thereā€™s something else#2017-11-1321:46gfredericksyou're trying to get collisions?#2017-11-1321:46arohneryes, some of the time#2017-11-1321:47arohnerbut Iā€™d really like e.g. power laws on repeat UUIDs, so out of 1000 uuids, one appears 100 times, one appears 20 times, etc.#2017-11-1321:48arohnerreading the spec source was informative. Iā€™ll just with-gen with a UUID constructor#2017-11-1321:49gfrederickswhere will you get the args for the constructor?#2017-11-1321:50arohnerhttps://github.com/danlentz/clj-uuid#2017-11-1321:50arohnertype 5s are deterministic, and I suspect I can create v1s by specifying a unix time#2017-11-1321:50gfredericksif you use gen/large-integer and gen/fmap that to the type-3 uuid constructor, that will give you something like the distribution you want#2017-11-1321:51gfrederickskeep in mind how distribution relates to size#2017-11-1321:51arohnercan you expand on that?#2017-11-1321:53gfrederickshave you seen this? https://github.com/clojure/test.check/blob/master/doc/growth-and-shrinking.md#2017-11-1321:55gfredericksI also talked about it some here if you liked faces and voices and slides and things https://youtu.be/F4VZPxLZUdA?t=25m26s#2017-11-1321:56arohnervery useful, thank you#2017-11-1400:58qqq
(defn tag-is [t]
  (fn [x]
    (= (:tag x) t)))

;; ** vec2 
(s/def ::x number?) 
(s/def ::y number?)
(s/def ::vec2 (s/and (s/keys :req [::x ::y])
                     (tag-is ::vec2)))

;; ** bbox
(s/def ::x0 number?)
(s/def ::x1 number?)
(s/def ::y0 number?)
(s/def ::y1 number?)
(s/def ::bbox (s/and (s/keys :req [::x0 ::x1 ::y0 ::y1])
                     (tag-is ::bbox)))

(defn vec2-new [{:keys [x y]}]
  {:tag ::vec2
   ::x x
   ::y y})

(defn bbox-new [{:keys [x0 x1 y0 y1]}]
  {:tag ::bbox
   ::x0 x0
   ::x1 x1
   ::y0 y0
   ::y1 y1})
Am I using spec right nere? the vec2-new and bbox-new seem very UN-clojurish 'every is data' approach. However, given that I'm using ::x ::y ::x0 ::y0 ... I don't know how else to nicely create vec2/bbox objects
#2017-11-1401:15taylormulti-spec seems like it might better fit that use case https://clojure.org/guides/spec#_multi_spec#2017-11-1401:17taylorthat would obviate the tag-is thing. the *-new fns at the bottom seem like theyā€™re just converting map keys from unnamespaced to the current namespace, why not use the namespaced keys upstream?#2017-11-1401:17taylor(using unnamespaced keys in your spec is also an option)#2017-11-1401:33qqq@taylor: because namespace 'object' is constructed != namespace 'object' is specced#2017-11-1402:08seancorfield@qqq In the upstream code, do you already have maps with unqualified keys? If so, where did they come from?#2017-11-1402:09seancorfieldIf you don't want the keys namespaced, change :req to :req-un (then bbox-new becomes just (assoc data :tag ::bbox))#2017-11-1402:11qqq@seancorfield: all code is code I have written (and can change -- am rewriting with spec -- and trying to figure out best practices)#2017-11-1402:11qqqI actually think qualified makes more sense than unqialified, since a word may mean different things n different namesapces#2017-11-1408:16deg@hiredman I did not know that, thanks!. Was it always the case that s/keys checks other keys too? Where is this documented? Meanwhile, I realized that another part of the solution for me is to use s/merge. That way, the central spec needs to only about the existence of each module, not its behavior.#2017-11-1413:05madstap@deg It's documented here https://clojure.org/guides/spec#_entity_maps#2017-11-1420:06ajmagnificoQuestion: In general, do you find itā€™s better to define specs in the namespaces that use them? Or in a separate ā€œappname.specsā€ namespace?#2017-11-1420:07ghadipersonally, separate#2017-11-1420:35ajmagnificoI think that the advantage of defining them in the namespaces where they are used is one of brevity.#2017-11-1420:35ajmagnificoFor example, with nested data structures#2017-11-1420:35ajmagnifico#:longappname.specs{:a 1 :b 2}#2017-11-1420:36ajmagnificothis is less verbose than {:longappname.specs/a 1 :longappname.specs/b 2}#2017-11-1420:36ajmagnificoBut the #: syntax only applies to the top level of the map#2017-11-1420:36tayloryou can alias the namespaces for brevity#2017-11-1420:41ajmagnificoSo, Iā€™m loading a config file that looks like this#2017-11-1420:41ajmagnificoand Iā€™m trying to spec it#2017-11-1420:41ajmagnificoI know I can use (spec/keys :req-un ā€¦)#2017-11-1420:41ajmagnificoBut Iā€™d like to use namespaces#2017-11-1420:43ajmagnificoAnyway, all of the pieces of this config are currently in my appname.specs namespace#2017-11-1420:43ajmagnificoWhen I load it and try to validate it, it fails, because all of my specs are namespaced#2017-11-1420:43ajmagnificoBut Iā€™d rather not have to put #:appname.specs in front of each map in the config#2017-11-1420:44ajmagnificoSo if I use the alias strategy like taylor said, Iā€™d need to choose an arbitrary alias ahead of time that I would use in the config file, and that would need to match up with an alias that I would create in the namespace where Iā€™m actually using and validating this config data.#2017-11-1420:45ajmagnificoOr, if I defed the specs in the namespace where the data will be loaded and used,#2017-11-1420:45ajmagnificothen I can just put a :: in front of each of the keys and everything will work as intended#2017-11-1420:46ajmagnificoBut am I going to get burned down the road if I have all of my spec definitions scattered throughout a bunch of namespaces instead of keeping them all in one location?#2017-11-1420:46taylorI guess Iā€™d ask myself what problem am I solving by namespacing all the keywords in my config map#2017-11-1420:46tayloror is it instead creating more problems#2017-11-1420:47taylorwhereā€™s your config defined, how is it loaded?#2017-11-1420:48ajmagnificoOkay, thatā€™s a fair question taylor. I guess at this point itā€™s me still trying to learn the idiomatic way of doing spec as it was intended by its creators. And they seem to think that namespaced keywords are a beautiful thing. Iā€™m still trying to figure out how theyā€™re beautiful. šŸ™‚#2017-11-1420:48ajmagnificobest practices#2017-11-1420:48ajmagnificoso Iā€™m wondering if ā€œbest practicesā€ here are, like you said, creating more problems#2017-11-1420:48ajmagnificoor if Iā€™m just doing it wrong.#2017-11-1420:49taylorI use specs for maps with unnamespaced keywords all the time, personally. I donā€™t think thereā€™s anything necessarily wrong with it#2017-11-1420:49ajmagnificoAnyway, the config is stored in edn format in a hand-edited file, loaded with edn/read-string during app initialization#2017-11-1420:50taylorbut then again most of the maps I work with are deserialized from some format that doesnā€™t support namespaces#2017-11-1420:52taylorand FWIW you could use any ā€œmade upā€ namespace for your config keywords, it doesnā€™t necessarily have to match a real namespace in your project#2017-11-1420:53ajmagnificothatā€™s an interesting idea#2017-11-1420:53ajmagnificothanks for the discussion, taylor#2017-11-1420:53ajmagnificoIā€™ll probably just end up using :req-un#2017-11-1420:54taylorthatā€™s what Iā€™d do, but Iā€™m lazy šŸ˜†#2017-11-1420:54ajmagnificothe hallmark of every great programmer#2017-11-1508:40tbaldridge@ajmagnifico something to consider, Spec also recommends flat maps:#2017-11-1508:42tbaldridge{:cnx-info.source.db/host :env/DB_HOST :cnx-info.source.db/port :env/DB_PORT :cnx-info.dst.db/host "localhost" ... :job/dir "some-directory}#2017-11-1514:35uwo@tbaldridge interesting, is that implicit in its design, or did I miss that recommendation somewhere?#2017-11-1514:39slipsetHow would you spec a map of ints to maps, eg#2017-11-1514:39slipset
{1 {:foo "bar"} 2 {:foo "qux"}}
#2017-11-1514:39taylorhttps://clojure.github.io/clojure/branch-master/clojure.spec-api.html#clojure.spec/map-of#2017-11-1514:40slipset@taylor Thanks!#2017-11-1514:41taylor(s/map-of int? map?) but you could get more specific (s/map-of int? ::some-complex-keys-spec)#2017-11-1514:41slipsetI did the second šŸ™‚#2017-11-1514:50ajmagnificothanks @tbaldridge. Thatā€™s a really good idea, and it will actually solve a couple of other problems for me as well. I didnā€™t recall that keyword namespaces donā€™t actually need to correspond to real namespaces, even though thatā€™s exactly what taylor told me! I guess I just needed to see it concretely for it to click.#2017-11-1514:50ajmagnificoI guess the only thing you miss out on when you make up namespaces is use of the :: syntax#2017-11-1514:50ajmagnificobut in my case, thatā€™s a small price to pay#2017-11-1523:27dpkpI'm trying to spec a variadic function that has a gnarly java-esque method signature like (f str int) (f str int bool) (f str int bool int) . I've tried to model this w/ s/cat but I'm having trouble building a spec that says (f str int int) is invalid but (f str int bool int) is valid. Example:
(s/def ::foo (s/cat :a string? :b int? :c (s/nilable boolean?) :d (s/nilable int?)))
#2017-11-1523:29dpkpthis works if args are supplied as nil. but I can't figure out how to also support the smaller arg list. (s/valid? ::foo ["a" 1]) is false#2017-11-1523:29dpkpany pointers?#2017-11-1602:45Alex Miller (Clojure team)Other suggestions were fine but also could use nested s/? for optional stuff#2017-11-1605:13dpkpSomething like this?
(s/def ::foo (s/cat :a string? :b int? :opts (s/? (s/cat :c boolean? :opts (s/? (s/cat :d? int?))))))
#2017-11-1612:47Alex Miller (Clojure team)Yep#2017-11-1523:29xandrewsgive https://clojuredocs.org/clojure.spec/alt a try#2017-11-1523:30dpkpok, will do! thanks#2017-11-1523:31xandrewslike (s/def ::foo (s/alt :two-arg-version (s/cat :a string? :b int) :three-arg-version (s/cat :a string? :b int? :c boolean?) ...etc)#2017-11-1523:31xandrewslike (s/def ::foo (s/alt :two-arg-version (s/cat :a string? :b int) :three-arg-version (s/cat :a string? :b int? :c boolean?) ...etc)#2017-11-1523:41xandrews@U1NCSDGDQ in my haste I forgot to include s/cat in the mix#2017-11-1523:41xandrewshere was an example that works for me#2017-11-1523:41xandrews
(s/def ::foo
  (s/alt
   :two-args (s/cat :a string? :b string?)
   :three-args (s/cat :a string? :b int? :c int?)))

(s/conform
 ::foo
 '("hey" "foo"))

(s/conform
 ::foo
 '("hey" 1 2))
#2017-11-1523:32dpkpaha!#2017-11-1612:39stathissiderisdoes anyone have a good way to instrument everything before running lein test from the command line?#2017-11-1612:47gfredericks@stathissideris leiningen injections ought to do it#2017-11-1612:59stathissideris@gfredericks thanks, Iā€™ll read up on that šŸ™‚#2017-11-1614:27souenzzo
(defmulti mkop :op)
(defmulti mkop' :op)
(s/fdef mkop
        :args (s/cat :args (s/multi-spec mkop' :op)))
(defmethod mkop :sum
  [{:keys [a b]}]
  (+ a b))
(defmethod mkop' :sum
  [_]
  (s/keys :req [::a ::b]))
(defmethod mkop :inc
  [{:keys [a]}]
  (inc a))
(defmethod mkop' :inc
  [_]
  (s/keys :req [::a]))
There is something wrong with this "pattern" for spec on multispec? Someone else doing something like this? Other options?
#2017-11-1616:37gklijsNot really sure what your trying to do. @souenzzo#2017-11-1617:16souenzzo@gklijs I'm trying to make a spec to a multimethod#2017-11-1620:00jumar@souenzzo I don't have an experience with multi-spec per se, but if you are trying to to instrument multimethod, that's not currently possible with spec. You'll need to create a wrapper function for multimethod and spec that one.#2017-11-1617:17gklijsok, no experience with that, I was busy using multi methods handling specs, so I can edit certain data based on itā€™s spec in the browser#2017-11-1708:14otfromwhen writing the :fn bit of an fdef, doesn anyone have any tips or examples to figure out which bit of the fn failed if there are multiple properties that should hold?#2017-11-1708:15otfromor with labels almost gives what I want, but obviously isn't logically correct. I want something like all#2017-11-1708:30tcouplandsomething abit like:
(s/all [{{:keys [first]} :args second :ret}]
           :value-mismatch #(= (:value first)
                                        (:value second)))
would be very tidy for checking a value was copied correctly in the function
#2017-11-1710:43trissis there a way to write a spec that says: a collection containing two or more integers and anything else?#2017-11-1710:46gklijswould became something like, and it would need to be a collection, and when filtered on number? the size is minimal of 2#2017-11-1711:09tcoupland@triss sounds like an s/cat which makes use of some of the regexp elements#2017-11-1712:15OlicalDoes anyone know if you can spec a map and have it conformed into multiple parts depending on the key. For example, conform {:A 1 :B 2 :a 1 :b 2} into something like {:upper {:A 1 :B2} :lower {...}}. Basically a group-by during a conform.#2017-11-1712:15OlicalI essentially want different rules for different kinds of keys in a map. So certain styles of keys would have different value requirements.#2017-11-1712:17andre.stylianosFrom what I've been seeing of the usage of spec, I'd guess not. Seems like it falls under the "data transformation" camp which is not something they intend spec to be used for.#2017-11-1712:18andre.stylianosbut I might be wrong#2017-11-1712:18OlicalYep, that's how I was beginning to feel about it. I'm currently doing that transformation after the fact, but it's hard to find them in a recursive spec, so I can't do the transform up front ahead of time.#2017-11-1712:18OlicalI don't want to walk the tree multiple times is all.#2017-11-1712:19OlicalI would quite like different specs for different kinds of keys though, not sure if I can do that.#2017-11-1712:19OlicalIn my case, if it starts with on- I want it to be restricted to a function value. If it's anything else it should be a string.#2017-11-1712:20andre.stylianosmakes sense. Does it have to be a tree? You could {:upper/A 1 :upper/B 2 :lower/a 1...}#2017-11-1712:21andre.stylianosand not quite sure what you mean by "different specs for different kinds of keys though, not sure if I can do that."#2017-11-1712:23OlicalYup, I'm not sure either šŸ™‚ It's like, I don't care about the actual keys, as long as they're keywords. But if it starts with on- it should be a function. I wonder if I can combine map-of and keys. It's almost like I want two stages, conform as much as you can with this spec, THEN conform with this spec.#2017-11-1712:23OlicalRight now I'm just doing map-of keywords to string or function.#2017-11-1712:24OlicalI just need constraints between key and value. Otherwise if I exercised this spec it would generate invalid runtime data, but it would match the spec.#2017-11-1712:24andre.stylianosYeah, for what you're saying it seems like map-of would be a fitting choice#2017-11-1712:26andre.stylianosBut I'm not sure if there's a way to treat the key/value pair as a whole with map-of#2017-11-1712:28OlicalI think I need a hybrid that doesn't exist šŸ˜…#2017-11-1712:28andre.stylianosAnother option might be coll-of if map-of doesn't work, since that way you could probably spec each entry as [k v] pairs#2017-11-1712:31OlicalFor context, this is an attribute map that will be applied to DOM nodes in this little Reagent/React like thing I'm building.#2017-11-1712:31OlicalSo most keys will just be strings that go to setAttribute on a DOM node, but event listeners are special things that are handled separately.#2017-11-1712:31OlicalI'm using spec as a parser of my hiccup like data structure basically.#2017-11-1712:35OlicalAh, hah! https://groups.google.com/forum/#!topic/clojure/WiMV5EEAVhM#2017-11-1712:35OlicalPeople with the same issue, linking back to this Slack.#2017-11-1712:35OlicalFound a strange loop šŸ˜„#2017-11-1712:36OlicalAaaaand bingo https://stackoverflow.com/questions/38151446/how-can-i-spec-a-hybrid-map#2017-11-1712:36OlicalAlex to the rescue.#2017-11-1714:01andre.stylianosnice one!#2017-11-1714:42otfromanyone out there have good examples of complex fn's on a :fn for a fdef?#2017-11-1714:42otfrom(esp ones that do good reporting of what bit of the fn has failed)#2017-11-1714:47otfromwhich is really just me bumping this: https://clojurians.slack.com/archives/C1B1BB2Q3/p1510906494000281#2017-11-1800:45qqqdoes spec have a way to capture: this object is an ATOM and when you deref the ATOM, it should satisfy this spec#2017-11-1800:57taylorI havenā€™t seen anything like that. I suppose you could wrap an atom and hook spec on derefs?#2017-11-1800:57qqqWe can set a validator-function on the atom, so it'll auto reject on swap! / reset! if it fails the spec.#2017-11-1800:57taylorinteresting#2017-11-1800:57qqqHowever, I now want a way to get the validator function of an atom, so we can say things like "assert that this atom has FOOBAR validator"#2017-11-1801:01danielcomptonYou also have the same problem for things like manifold deferreds#2017-11-1801:03qqqwtf are manifold deferreds? šŸ™‚#2017-11-1801:03qqqELI-don't-have-math-PhD#2017-11-1801:10danielcomptonhttps://github.com/ztellman/manifold/blob/master/docs/deferred.md#2017-11-1801:15qqq@danielcompton: this looks interesting ; does it also have cljs support ? [ I can't find mention of it anywhere on the page ]#2017-11-2217:03jeroenvandijkhttps://github.com/dm3/manifold-cljs#2017-11-2217:29qqq@U0FT7SRLP: thanks for the manifold-cljs link; much appreciated#2017-11-1803:15qqq1. Please ignore the camelCase. 2
[(s/assert ::appState 23) 
 (s/valid? ::appState 23)]
(comment
 [23 false])

3. How is this possible? if 23 is not a valid ::appState, shouldn't assert throw an exception ?
#2017-11-1803:57qqqResolved: I forgot to use (s/check-asserts .... )#2017-11-1807:35danielcomptonThereā€™s a CLJS port of it somewhere#2017-11-1807:48mattiaswI have a simple function that takes data, and creates test data from it. Currently, it uses (rand-int ..), what minimum changes do I need to do to this code, in order to use the same seed for random as the rest of the generators.
(defn create-update-operation
  "Change or delete or add a new key to `m`.
   In 30% of the operations, then value will be nil, i.e. delete."
  ;; Do not change or delete any of the pre-defined keys
  ([m] (let [res (create-update-operation (filter #(not (#{:id :tenant :entity :k :updated :version} (key %))) m) {})]
         ;; (println (count res))
         res))
  ([m changes]
   (if (<= (count m) 3) changes
       ;; find the next key to change (update or delete)
       (let [nxt      (rand-int (count m))
             next     (nth m nxt)
             rest     (nthrest m (min 1 nxt)) ;0 just mean that I am regenerating the first key.
             k        (key next)
             _        (assert (some? k))
             v        (update-or-delete-value k)
             changes2 (assoc changes k v)]
         ;; see if we should add a new key to the map
         (if (< (rand-int 10) 2)
           (recur rest (let [k2 (keyword (random-string))
                             v2 (update-or-delete-value k2)]
                         (assoc changes2 k2 v2)))
           (recur rest changes2))))))
where update-or-delete-value also uses rand-int
#2017-11-1900:00qqqIs there a better way to write: (s/def ::p1 #(s/valid ::vec2 %)) ?#2017-11-1900:00qqq(s/def ::p1 #(s/valid ::vec2 %))#2017-11-1900:32taylor(s/def ::p1 ::vec2)?#2017-11-1901:18qqqIn the spec guide, we see: (s/def ::suit #{:club :diamond :heart :spade}) which asserts that item is ELEMENT OF given set is there a corresponding builtin for 'item is SUBSET OF given set`#2017-11-1901:38taylorprobably just use https://clojuredocs.org/clojure.set/subset_q as a predicate#2017-11-2005:08qqqthey very act of writing specs is making my code clearer#2017-11-2005:08qqqit's forcing me to think: what condition does this key have to satisfy to be valid, and documenting it as code#2017-11-2007:39qqqthis is kind of insane, but is there a way to (while only holding weak references), tell clj/cljs to memoize calls to (s/assert SPEC OBJ) ?#2017-11-2012:20gfredericksthe jvm has a hashmap for that sort of use case I think dunno how easy monkeypatching s/assert to get that effect would be#2017-11-2012:20gfrederickswhy're you calling multiple times?#2017-11-2020:25johanatanare there any known gotchas involving partial application around a stest/instrumented function ?#2017-11-2021:15hiredmanit depends what you mean#2017-11-2021:15hiredmaninstrument works by setting the root value of vars to an instrumented value of the function, but if you have already taken the value of a var before instrumenting, of course that will have effect#2017-11-2021:17hiredmanso if you have something like (def f (partial g 1)) and then later instrument, calling f won't call the instrumented version of g#2017-11-2101:44johanatan@hiredman yea, that makes sense. unfortunately wasn't my problem but i figured it out anyway (something unrelated)#2017-11-2101:45johanatanthe error message i was getting from spec was unintelligible but i just massaged the code, fixed any issues i saw via manual inspection and it eventually went away#2017-11-2103:02johanatannarrowed it down to instrument actually doing more than I figured it would. looks like it exercises the instrumented with some values that the user itself isn't passing in. this was a bit surprising to say the least#2017-11-2103:02johanatankept trying to find where the ((nil)) value was coming from in my code/ how that was even possible etc etc#2017-11-2103:40seancorfield@johanatan If you instrument a higher-order function -- one that takes a function as an argument -- then it will use generative testing, as I understand it.#2017-11-2107:58qqqwith coll-of, is there a way to say: this collection ca be empty, but it can't be nil i.e. [] is okay, but nil is NOT okay#2017-11-2110:53trisshey all. Is there a spec for something I can use as a predicate for s/valid?#2017-11-2110:54trissI want a spec that matches both predicate functions and the names of specs.#2017-11-2111:06mpenet@qqq it works like this by default#2017-11-2111:07mpenet(s/valid? (s/coll-of any?) nil) returns false#2017-11-2117:08qqq@mpenet: yeah, thanks, I should have run that test, turns out bug was elsewhere and I was misattributing it#2017-11-2118:32johanatan@seancorfield it was a higher-order function in this case. any idea the reasoning behind using generative testing for this case? is it just a 'hack' since the function inputs cannot be inspected to a level of detail necessary for verification?#2017-11-2118:46seancorfieldI thought that was explained in the Spec Guide on the web site @johanatan? I certainly wouldn't say it was a 'hack'. It seems perfectly reasonable to me. How else would you verify that a function you're passing as an argument conforms to its spec in this situation?#2017-11-2118:47mpenetIt s doable at invoke time#2017-11-2118:47mpenetThere is/was a ticket about this#2017-11-2118:49mpenetI personally wished it was like this, now I almost always spec hof as ifn? for instrumentation because of this, which makes it a bit useless#2017-11-2118:50mpenetImho both make sense in some cases, we should just have the choice#2017-11-2118:58mpenetIt was clj-1936 not sure if there is a new ticket started from the discussion there#2017-11-2119:09johanatan@seancorfield it probably is explained but i haven't read the guide from beginning to end so could've missed it. well, yea, in a dynamic language this probably can't be done (except perhaps by inserting metadata onto anonymous functions at their definition sites and then reading it back out at the spec check sites [which now that I mention it, does sound like a better way if it would be possible to impl])#2017-11-2119:10johanatan[but by 'hack' i meant, an unfortunate edge where the abstraction leaks due to limitations of the underlying system being modified]#2017-11-2119:13johanatan@triss isn't spec? what you are asking for?#2017-11-2122:05triss@johanatan Not quite.#2017-11-2122:05trissIā€™m validating on (s/or :fn fn? :keyword keyword? :spec s/spec?) is there a neater way?#2017-11-2122:09qqqI'm starting to love spec's dynamic attributes.#2017-11-2122:09qqqI'm doing some GUI layout, and I can write a spec of :assert that the sum of the width of the elemes of this vector is <= width of the screen
#2017-11-2122:09qqqhit is nearly impossible to do in static type systems#2017-11-2123:16seancorfield@triss Do you want to validate that something is a registered spec or that it looks like it could be a spec?#2017-11-2123:16trissis a registered spec would be best for me#2017-11-2123:17seancorfieldFor the former, you could use #(try (s/get-spec %) true (catch Exception _))#2017-11-2123:17seancorfieldThat would be false for predicate functions tho'...#2017-11-2123:17seancorfield...but it would validate that you actually had a named, registered spec (keyword).#2017-11-2123:18trissah i see. thatā€™s a handy snippet fro something no doubtā€¦#2017-11-2123:18seancorfield(so I guess you'd still want (s/or :fn fn? :spec #(try ...)) ?)#2017-11-2123:19trissIā€™m gonna have to scratch my head for a bit about what I want to match on reallyā€¦ I want anything that woks as the first argument for s/valid?#2017-11-2123:19trisswhere is the spec for s/valid?#2017-11-2203:37Alex Miller (Clojure team)you canā€™t actually spec most of the functions in spec without creating an infinite recursion when instrumented#2017-11-2123:20seancorfieldBased on the docstring for s/valid?, it doesn't look like there's a spec registered for it.#2017-11-2123:22seancorfieldThe source shows it calls specize on its spec argument and that in turn is
(defn- specize
  ([s] (c/or (spec? s) (specize* s)))
  ([s form] (c/or (spec? s) (specize* s form))))
at which point you'll have to dig in the source for specize*
#2017-11-2203:06qqqThe question is NOT "what is wrong with this code." The question is "how do I get a more useful error msg"
(s/def ::ys number?)
(s/def ::xs number?)
(s/def ::data any?) 
(defn mat-of [ys xs q? data]
  (assert (vector? data))
  (assert (= (count data) ys))
  (doseq [y (range  ys)]
    (assert (= (count (get data y)) xs)
            (str "count of row " y " is " (count (get data y)) " not " xs))))
(s/def ::keyPad (s/and (s/keys :req-un [::xs ::ys ::data])
                       (fn [obj]
                         (mat-of (:ys obj) (:xs obj) any? (:data obj)))))
(s/explain ::keyPad
          {:ys   3
           :xs   3
           :data [[7 8 9]
                  [4 5 6]
                  [1 2 3]]})



returns false. I expect it to return true. It appears to be eating the assertion error. Is there a way to get a more helpful msg on why it's failing the predicate ?
#2017-11-2203:15qqqreframing question: when using predicates with clojure spec, is there a way, instead of just returning true/false, also return a error msg on false ?#2017-11-2203:38Alex Miller (Clojure team)currently, no#2017-11-2204:01qqqis this a short coming of current spec design, or is this because I'm mis using spec and should be using spec primitives as much as possible instead#2017-11-2204:16seancorfieldOne thing I've found helpful @qqq is to break predicates down into smaller components and give them names. That way the explain refers to the named predicates. In your case, I'd write data-is-vector?, data-has-ys-rows?, and data-has-xs-cols? and have those accept the whole hash map and return true/false. Then you'd have (s/and (s/keys :req-un [::xs ::ys ::data]) data-is-vector? data-has-ys-rows? data-has-xs-cols?) and I believe you'll get better error messages -- without using assert.#2017-11-2204:19seancorfieldThe other thing we do is take the explain-data and parse it so we can map symbols and forms to English error messages (well, actually to i18n keys so that we can produce error messages in any language!).#2017-11-2208:50hkjelsSo, Iā€™ve written quite a lot of ui-components where I use spec for various things. Mostly checking that parameters used with the components are valid. Iā€™ve come to a stage where Iā€™m doing performance-testing and it seems that spec is sucking the life out of this project. Is it just a bad idea to use spec for such a task?#2017-11-2213:36Alex Miller (Clojure team)Yes#2017-11-2209:00andre.stylianosI believe the encouraged way is to use spec for production code only at the boundaries where you receive data from sources you don't control. Beyond that, the validation that you do for internal code should be used only for development.#2017-11-2209:14hkjelsOK.. Then Iā€™ll have to use something like spec/nonconform all over the place in dev then#2017-11-2209:14hkjelsI guess it makes sense to not have it on in production#2017-11-2210:34ikitommi@hkjels just turn off the validation for real app? e.g. instrument the component functions for dev. Or use assert and disable that for prod. Havenā€™t used spec for ui but did that at some point with Schema. WIth it, you can disable function validations and the schematized defs and fns will not emit any code related to Schemas => no penalty.#2017-11-2211:26hkjelsWhen I disabled validation, it definitely got a performance boost#2017-11-2211:27hkjelsconform is still one of the heavier tasks in the performance-tab og chrome, but itā€™s also doing quite some work, so that makes sense I guess#2017-11-2211:28hkjelsI have to do some more testing, but this might do#2017-11-2212:00guyWhen you fdef a function with no args, do you just omit the :args key?#2017-11-2213:35Alex Miller (Clojure team)No, you should use (s/cat) to validate 0 arity#2017-11-2213:45Alex Miller (Clojure team)Also, you should be suspicious of 0 arity functions. Either they have side effects (and maybe shouldnā€™t be specā€™ed) or you can just use a def instead.#2017-11-2213:51guyThanks!#2017-11-2213:38otfrom@seancorfield and @alexmiller are there any good examples out there of using relatively complex :fn functions in fdefs? I saw that and hints above and wonder if that is the best way to go to figure out which bit of the property is being violated#2017-11-2213:42Alex Miller (Clojure team)Iā€™ve written some here and there but not sure if in anything public. If they get too complicated, then you have to question whether thatā€™s the best way to test. Sometimes straight test.check is better#2017-11-2213:56otfromcool, that makes sense#2017-11-2217:31qqqI realize this is a bad idea in most cases, but is it possible to say: (s/maps ... key :buttonContainer satisfies spec ::container) ?#2017-11-2218:03Alex Miller (Clojure team)Donā€™t understand the question#2017-11-2218:07bronsaare you looking for :req-un?#2017-11-2220:55Alex Miller (Clojure team)FYI, Iā€™ve been working on fix the auto doc junk for Clojure and contrib and the docs have been updated for Clojure, spec.alpha, and links in the spec guide#2017-11-2221:07seancorfield@alexmiller That's great to hear -- thank you! Is there anything I can do to help get https://clojure.github.io/java.jdbc/ updated too?#2017-11-2221:10Alex Miller (Clojure team)nope. I can try to run it manually thoughā€¦#2017-11-2221:15Alex Miller (Clojure team)@seancorfield updated#2017-11-2221:17Alex Miller (Clojure team)@seancorfield Iā€™m not sure if you had specs in those vars in the prior docs, but they are showing up now#2017-11-2221:37seancorfieldOh cool! No, I don't think the specs were showing up before.#2017-11-2221:38seancorfieldThat is very nice! Thank you!#2017-11-2312:26danielneal@alexmiller ooh that's awesome#2017-11-2316:26qqqI have a number of specs: ::Foo ::Bar ::Cat ::Dog if I do (s/keys :req-un [::Foo ::Bar ::Cat ::dog]) the keys of the map become :Foo :Bar :Cat :Dog. Is there some way to redo this so the keys become :foo :bar :cat :dog instead ?#2017-11-2316:50misha(s/def ::foo ::Foo) #2017-11-2316:52mishaAnd then use ::foo in :req-un#2017-11-2316:53qqqobvious yet brilliant#2017-11-2320:32qqq(s/valid? any? nil) what is idiomatic way to say: this can be anything so long as it is not nik ?#2017-11-2320:32madstap@qqq some?#2017-11-2320:33qqq(some? false) => true -- this is surprising#2017-11-2320:47guy
(defn some?
  "Returns true if x is not nil, false otherwise."
  {:tag Boolean
   :added "1.6"
   :static true}
  [x] (not (nil? x)))
#2017-11-2402:52johanatanI'm getting ExceptionInfo Couldn't satisfy such-that predicate after 100 tries. clojure.core/ex-info (core.clj:4744) for a spec generator that seemingly should be fairly trivial:
(s/def ::snake-cased-alpha-numeric
  (s/with-gen
    (s/and string? #(re-matches #"[a-z|\_|0-9]+" %))
    #(gen/fmap (fn [v] (apply str v)) (gen/vector (gen/frequency [[1 (gen/return \_)] [9 gen/char-alphanumeric]])))))
(s/def ::required? boolean?)
(s/def ::property-attrs
  (s/keys :req-un [::required?]))
(s/def ::events-schema
  (s/map-of integer?
            (s/map-of ::snake-cased-alpha-numeric
                      (s/map-of ::snake-cased-alpha-numeric ::property-attrs))))
> (gen/sample (s/gen ::events-schema))
ExceptionInfo Couldn't satisfy such-that predicate after 100 tries.  clojure.core/ex-info (core.clj:4744)
#2017-11-2402:52johanatan[I added the with-gen on snake-cased-alpha-numeric with it being the most complicated primitive in there and still get the error]#2017-11-2402:58danielcompton@johanatan can you generate any of the leaves?#2017-11-2402:58johanatan
> (gen/sample (s/gen ::property-attrs))
({:required? false} {:required? false} {:required? true} {:required? true} {:required? true} {:required? true} {:required? false} {:required? false} {:required? false} {:required? true})
#2017-11-2402:59johanatan
> (gen/sample (s/gen ::required?))
(false true true false true true false true true false)
#2017-11-2402:59danielcomptonwhat about snake-cased? that looks like the tricky one#2017-11-2402:59johanatan
> (gen/sample (s/gen ::snake-cased-alpha-numeric))
("zno" "n7" "_" "3" "_" "c" "8_" "5i" "k7q" "i")
#2017-11-2402:59danielcomptonhuh#2017-11-2403:00johanatanyea šŸ™‚#2017-11-2403:00johanatanoh, this may be it:#2017-11-2403:00johanatan
> (gen/sample (s/gen ::snake-cased-alpha-numeric) 100)
ExceptionInfo Couldn't satisfy such-that predicate after 100 tries.  clojure.core/ex-info (core.clj:4744)
#2017-11-2403:01danielcomptontry (s/map-of integer? ::snake-cased-alpha-numeric)#2017-11-2403:01johanatanperhaps there is a bug where my regex doesn't match values generated with the with-gen-specified generator#2017-11-2403:01johanatani've seen that error when the two are mismatched: i.e., when the spec and the generator for it are not perfectly aligned on what is acceptable or not#2017-11-2403:02gfredericks@johanatan would this help? https://github.com/gfredericks/test.chuck#string-from-regex#2017-11-2403:02danielcomptonTry https://github.com/gfredericks/test.chuck#string-from-regex maybe?#2017-11-2403:02danielcomptonjinx#2017-11-2403:02gfredericksI forget what the implications of jinx are#2017-11-2403:02gfredericksdo I have to not talk now for some time period#2017-11-2408:58danielnealI think you can't talk until someone says your name three times#2017-11-2412:07gfredericksthe same person? back-to-back in quick succession?#2017-11-2413:49danielnealaw you're right that's totally underspecified#2017-11-2413:53danielnealoh wait#2017-11-2413:53danielnealmy mistake#2017-11-2413:53danielneal
Traditionally, a jinx is ended when anyone speaks the jinxed person's name. However, a common variation says that only the jinxer can free the jinxee from their obligation to remain silent. (This is sometimes called a "private jinx" or "jinx personal lock".)
#2017-11-2419:08gfredericksa private jinx sounds a lot less interesting#2017-11-2403:03johanatanhmm, yea, i'll give that a try. thx!#2017-11-2403:08johanatanthat fixed it. was able to successfully generate 1000 at once#2017-11-2403:10johanatanthere is a bug in my regex though. noticing that pipes are ending up in the output. probably was the original problem#2017-11-2403:15danielcomptonbuy me a coke I think, although I probably owe you the drink. Edit: this was about the jinxing#2017-11-2403:17johanatanhaha, definitely. i would even get you a beer if you prefer#2017-11-2403:17johanatanthis was my intended regex, btw: #"[a-z0-9\_]+"#2017-11-2403:17johanatan[no pipes]#2017-11-2405:20qqqdoes clojure have any naming convention regarding fn names that end in a '-' ?#2017-11-2405:21qqqfor ecample, I have (defn node->svg ...) and I would like to define a helper function named (defn node->svg- ...) but intuitively, functions that end in '-' seem like a bad idea#2017-11-2405:33seancorfieldThe most common convention seems to be to end in a *#2017-11-2405:33seancorfield@qqq I've seen that in a lot of libraries -- and started following it myself.#2017-11-2412:06gfredericksI have the uncommon preference for foo', imitating mathematical notation#2017-11-2413:34guyHi has anyone made a spec for a base64 encoded string before?#2017-11-2413:35gfredericksthat sounds like it could be a regex#2017-11-2413:35guyooh yes#2017-11-2413:35guythats a smart idea#2017-11-2413:35gfredericksthough perhaps not a trivial one if you want to really follow the spec#2017-11-2413:36gfredericksbut of course that's something somebody else has done somewhere, e.g. https://stackoverflow.com/questions/475074/regex-to-parse-or-validate-base64-data#2017-11-2413:36guyyeah was just looking at that! thanks šŸ‘#2017-11-2423:27hlshipI'm stumbling on something pretty basic. s/cat is not working the way I'd expect.
(s/conform ::field-def [:foo :text]) 
=> {:field-name :foo, :field-type :text}
(s/conform (s/cat :fd ::field-def) [:foo :text]) 
=> {:fd {:field-name :foo, :field-type :text}}
I would have expected that to fail; ::field-def is
(s/cat :field-name keyword?
                          :field-type field-types)
That is, my ::field-def expects a tuple of keyword and field-type (a set of keywords). So I'd expect s/cat to consume the first value from a list, then apply ::field-def as a predicate and deconstructor to it. Instead, it is applying ::field-def directly. From my (mis-?) understanding, I'd expect the following to succeed, but it fails:
(s/explain (s/cat :fd ::field-def) [[:foo :text]]) 
In: [0] val: [:foo :text] fails spec: :com.walmartlabs.genie.launchpad.db-access/field-def at: [:fd :field-name] predicate: keyword?
:clojure.spec.alpha/spec  {:clojure.spec.alpha/op :clojure.spec.alpha/pcat, :ps [:com.walmartlabs.genie.launchpad.db-access/field-def], :ret {}, :ks [:fd], :forms [:com.walmartlabs.genie.launchpad.db-access/field-def], :rep+ nil}
:clojure.spec.alpha/value  [[:foo :text]]
Again, my expectation is that s/cat consumes a seq. The first element in the seq should be a ::field-def, which itself is a seq of two elements. Instead, it is matching the treating the entire list [[:foo :text]] as the field-def tuple, and failing because the first value in the list, [:foo :text] is not a keyword.
#2017-11-2423:31hlshipBasically, my goal is this:
(s/def ::primary-key (s/cat :pk-field-def ::field-def
                            :cluster-defs (s/* ::cluster-def)))
That is, a single field def, followed by zero or more cluster-def's.
#2017-11-2423:35seancorfield@hlship Regex specs "unwind" when they are combined, unless you wrap them in s/spec (or some other delimiting operation).#2017-11-2423:36hlshipOk, that could be what I'm missing. It's right there in the docs, once I know what to look for!#2017-11-2423:37seancorfields/cat isn't meant for tuples -- there's s/tuple for that#2017-11-2423:37seancorfield(although you'll get no labels with that I think...)#2017-11-2423:38hlshipNice. I don't need the conformed value, I'm just validating.#2017-11-2423:39seancorfieldI think it catches everyone out at least once šŸ™‚#2017-11-2510:17qqqis there a way to hijack s/assert 's error messages? I often have situations where: 1. obj is a giant data sttructure 2. obj fails spec ::foo 3. the fact that obj fails can be verified by just loloking at (keys obj) 4. s/assert prints out all of obj, which actuall makes it more difficult to see what is going on#2017-11-2522:48bbrinckYou can also customize the printer e.g.
(defn my-assert [spec val]
  (binding [s/*explain-out* my-printer]
    (s/assert spec val)))
my-printer will take a single argument that is the explain-data (itā€™s data like the return value from s/explain-data).
#2017-11-2523:22qqqthis looks better than what I asked for#2017-11-2523:22qqqso I get to take the assertion failure, as clojure data, rewrite it, then print it out#2017-11-2523:22bbrinckYep, you can format it however you want#2017-11-2523:23bbrinckAlso, if you happen to want an out of the box experience, you can use expound#2017-11-2523:23bbrinckIt doesnā€™t actually help in this specific case (it prints out the entire value when it is missing keys), but you can provide a custom function to print out the value in this specific case#2017-11-2523:24bbrinckhttps://github.com/bhb/expound#configuring-the-printer#2017-11-2523:24bbrinckin that case, youā€™ll get the spec-name and some other data, which might be enough to detect this particular case.#2017-11-2523:25bbrinckOr you could even write your own printer which inspects the explain-data and does something custom for keys specs, but passes it along to expound otherwise.#2017-11-2523:42qqqwhen an assertion happens, I want it to pop up a new browser window, print the first level of the data, have + buttons next to parts that can be expand (and expands when I click on them), and highlights in a red DOM rectangle the key/value pair that is failing the spec#2017-11-2523:42qqqsomeone needs to build that šŸ™‚#2017-11-2523:45bbrinckWould https://github.com/jpmonettas/inspectable#specs-fail-explain-why work?#2017-11-2523:50qqqthis is amazing#2017-11-2523:50qqqwhy is https://github.com/jpmonettas/inspectable#specs-fail-explain-why not common knowledge ?#2017-11-2600:00bbrinckNo idea, it seems like a great tool, maybe we need a collection of these helper libraries#2017-11-2512:57gfrederickswrap it in a try/catch?#2017-11-2513:23trisshow can I look inside the result of an s/and? - Iā€™d like to dig out the s/cat contained within it.#2017-11-2513:24gfrederickss/form?#2017-11-2513:34trisslovely thanks!#2017-11-2520:40hmaurerHi! I am getting a cryptic error (to me) when attempting to build my ClojureScript project for production (with advanced opts). Could someone please take a look? https://gist.github.com/hmaurer/1d30b39c8e28e646ebdfa524cbcdedb6#2017-11-2520:41hmaurerwrong channel, sorry#2017-11-2611:36qqq(s/keys ::req []) <- notice the :: instead of :req is there a way to have that throw an error ? sometimes, I define that, then I create objects, and they satisfy the spec ... because it's a map with no requirements since ::req is silently ighnroed#2017-11-2613:42Alex Miller (Clojure team)Sure, file a jira#2017-11-2614:50gfrederickswould that not go against the "accept arbitrary extra keys" philosophy of spec?#2017-11-2618:08Alex Miller (Clojure team)I wouldnā€™t necessarily put it in the spec for keys, but itā€™s reasonable to validate it in the function. Itā€™s not like this is getting passed to anything else.#2017-11-2618:09gfredericksI spose I had always interpreted the openness as partially targeting forwards-compatibility#2017-11-2618:10Alex Miller (Clojure team)Itā€™s still forward compatible to accept more stuff in the future#2017-11-2619:17gfredericksright, but then code targeting the future API can't use the older version if it happens to be around maybe that's not a useful thing? you just want backwards compatibility so you can use the newest of all the versions that each part of the code wants?#2017-11-2621:21Alex Miller (Clojure team)I would say thatā€™s a non-goal#2017-11-2617:55souenzzo
(s/fdef s/keys
        :args (s/+ (s/cat :k #{:req :opt}
                          :v coll?)))
=> clojure.spec.alpha/keys
(s/keys ::req [])
CompilerException clojure.lang.ExceptionInfo: Call to clojure.spec.alpha/keys did not conform to spec:
In: [0] val: :user/req fails at: [:args :k] predicate: #{:req :opt}
 #:clojure.spec.alpha{:problems [{:path [:args :k], :pred #{:req :opt}, :val :user/req, :via [], :in [0]}], :spec #object[clojure.spec.alpha$regex_spec_impl$reify__2436 0x2fa7a99c "
@qqq @gfredericks sure. But you can do šŸ™‚ Maybe in future versions of spec, this spec will be wrong. But for now, it's a usefull build-time error.
#2017-11-2617:57gfredericksbut there's a difference between spec setting that spec on s/keys and a user doing it themselves#2017-11-2619:54arohnerBefore I go write one, is there a generator for specs? i..e one that will generate (s/cat :x int?)#2017-11-2621:24Alex Miller (Clojure team)The spec specs in CLJ-2112 are imo the right direction for that. The specs there are not yet good enough to be used for this in general#2017-11-2620:00hoopeshttps://github.com/stathissideris/spec-provider what you're looking for?#2017-11-2620:03arohnerNot quite. I'm testing https://github.com/arohner/spectrum, so I want to throw arbitrary specs at it#2017-11-2620:56gfredericksonce there's a spec spec you could presumably get a generator from it#2017-11-2620:56gfredericksshouldn't spectrum have a spec spec already anyhow? I say that as someone who has never used spectrum#2017-11-2623:02arohnerIt only needs s/spec? as a spec when using spectrum. To test it, I want a spec generator#2017-11-2621:00taylorcurious if thereā€™s been any thought/discussion/work for sharing specs (besides putting them in a shared library)#2017-11-2621:02taylorsomething akin to XSD? I suppose itā€™d be limited to what can be expressed in whatever format#2017-11-2621:22Alex Miller (Clojure team)The whole point of using fully qualified namespace keys is that everyone can share specs#2017-11-2621:35tayloryeah, I guess my curiosity is more about alternative mediums/mechanisms for sharing specs?#2017-11-2621:36gfredericksbut as soon as you use custom predicates, you need to share code as well#2017-11-2622:56Alex Miller (Clojure team)Which is also fully qualified #2017-11-2621:38taylorIā€™m thinking of something conceptually similar to Swagger I guess. And yeah it would be a pretty limited subset of specā€™s abilities.#2017-11-2621:40taylora ha https://github.com/metosin/spec-tools#spec-transformations#2017-11-2704:34fedregIs there a way to get spec information similar to how I can use meta to get info on a function? So if I have a spec like (s/def ::example string?) is there some function I can call to see the spec definition? Even better, is there anything that will give me a list of specs in the namespace? Thanks!!!#2017-11-2715:59Alex Miller (Clojure team)yes, you can call (doc ::example)#2017-11-2715:59Alex Miller (Clojure team)youā€™d have to write something to filter specs per namespace#2017-11-2707:33mishas/form#2017-11-2707:34mishas/registry#2017-11-2707:34misha@fedreg #2017-11-2712:50ikitommi@taylor related to ā€œsimilar to swaggerā€ - if specs could be read back from their s/form without eval (I think it could be done already with undocumented part of the spec - but Rich might be doing something functional for this?), we could have a pragmatic subset of specs that could be safely used between servers & clients. and build swagger-like docs for them.#2017-11-2712:54ikitommiwe could publish the s/form as vendor extension to the spec-tools swagger-docs. e.f. :x-spec "(clojure.spec/keys :req [:user/x])" + :x-specs on top-level as the spec-registry for the related specs. Non-clojure clients could use swagger/openapi and clojure-clients just the spec-part.#2017-11-2716:34r0manIs this a bug?#2017-11-2716:34r0man(if-let [x false] x :my-ns/invalid) (if-let [x false] x :clojure.spec.alpha/invalid)#2017-11-2716:34r0manthe first works, the second raises an exceptions from spec#2017-11-2716:35r0manCall to clojure.core/if-let did not conform to spec: In: [2] val: :clojure.spec.alpha/invalid fails at: [:args :else] predicate: any?#2017-11-2716:36r0manthe second form works also in Clojure 1.8#2017-11-2716:39r0manoh, and I was trying the above with Clojure 1.9.0-RC1#2017-11-2720:02bfabry@U0CKBRBD2 known issue https://dev.clojure.org/jira/browse/CLJ-1966#2017-11-2720:04r0man@U050MP39D ah, thanks!#2017-11-2717:00qqqis there a way to say: ::foo is a COLLECTION such that (map :get-some-key ...) creates an unique list and if this fails, print out the entire collection#2017-11-2717:23taylor
(s/def ::my-map (s/keys :req [::get-some-key]))
(s/def ::my-coll (s/and (s/coll-of ::my-map)
                        #(= (count %)
                            (count (set (map ::get-some-key %))))))
(s/explain ::my-coll [{::get-some-key 1} {::get-some-key 1}])
#2017-11-2717:00qqq(as a spec)#2017-11-2721:20fedregThanks! Completely missed that registry function.#2017-11-2722:29j-poIs there a minimally ugly way to compare specs from two projects (like two versions of the same codebase)? One idea that comes to mind is loading one set of specs into a separate registry, but that looks like it's been made deliberately hard. Is there maybe a way to blow away the spec registry entirely to restart from scratch?#2017-11-2722:31bfabryspecs are fully namespaced with the idea that they'll be unique across projects, so caveat is anything you do that's assuming different is going to end up weird. but, if you want, I'm pretty sure you could just reset! the atom that is the spec registry#2017-11-2722:37gklijs@j-po maybe first load the old one, write all the forms to a map, and serialize. Then start again with the new one, and compare against the stored map?#2017-11-2723:05j-poThanks!#2017-11-2810:37otfromI'm creating a look up hash map from a vector of records where the keys will be strings based on what I want to look up. Anyone seen an example of how to spec this? I can't think of how to do it with s/keys#2017-11-2810:39otfrom
[{:a "foo" :b "shiny"} {:a "bar" :b "dull"}]
=>
{"foo" {:a "foo" :b "shiny"} 
 "bar" {:a "bar" :b "dull"}}
#2017-11-2810:52guyIā€™m not sure but could you use map-of ?#2017-11-2810:53guy
(s/def ::scores (s/map-of string? int?))
(s/conform ::scores {"Sally" 1000, "Joe" 500})
#2017-11-2810:53guytaken from https://clojure.org/guides/spec if you ctrl+f map-of#2017-11-2811:23otfrom@guy thx šŸ˜Š#2017-11-2811:23guyhopefully thats helpfully! šŸ‘#2017-11-2811:25otfromit is#2017-11-2821:45sekaois there any library that can "unroll" a spec so i can display a given spec with all its component specs in a single data structure?#2017-11-2821:47taylormight be some interesting stuff in this project https://github.com/jpmonettas/inspectable#2017-11-2821:51sekaohmm the spec browser is neat but does that lib allow you to just get the whole spec as data?#2017-11-2821:51tayloryou can already get that with s/form if I understand you correctly#2017-11-2822:01sekaos/form doesn't recursively grab specs that a given spec refers to#2017-11-2822:02tayloryeah youā€™ll have to do some additional lookup, but I think that should be pretty easy#2017-11-2822:04taylormaybe using clojure.walk and s/get-spec?#2017-11-2822:09taylor
(walk/postwalk
  (fn [v]
    (if (keyword? v)
      (or (some-> v s/get-spec s/form)
          v)
      v))
  (s/form ::my-coll))
hereā€™s an idea, maybe not a good one
#2017-11-2822:10taylordoesnā€™t cover specs registered to symbols, and probably other cases, etc.#2017-11-2822:29ikitommithere is a spec visitor on spec-tools: https://github.com/metosin/spec-tools/blob/master/README.md#spec-visitors#2017-11-2916:19acronRe spec, should Clojure ensure that keys referenced in s/keys are registered? I just noticed an unregistered key in one of mine as was surprised I hadn't been made aware by the compiler#2017-11-2916:20mpenetit allows that so that you can define key sets before the key specs themselves, most of the stuff in spec in wrapped in delays#2017-11-2916:21mpenetI could be wrong, but this sounds like the "issue"#2017-11-2916:22acronI understand, kinda makes sense.#2017-11-2916:23madstapYou can use this to find those errors https://gist.github.com/stuarthalloway/f4c4297d344651c99827769e1c3d34e9#2017-11-2916:24acronThanks @madstap this is useful#2017-11-2919:13mishais there a good example of a generator, which generates map (record) where values of 2 fields are dependent? multispec solves opposite problem: conform/validate records with 2+ dependent fields, can it be leveraged for generating generator?#2017-11-2919:17mishathe nuance in a data model I have, is that :type key on record determines not presence/absence of an attribute, but shape of a particular attribute:
{:type :string :val "foo"}
;;vs.
{:type :media :val {:url "" :content-type"image/jpg"}}
so I am already jumping some hoops to create multispec for this (extra level of spec indirection between data shape and attribute name)
#2017-11-2919:20mishawould like to avoid having custom monster generator, if possible at all#2017-11-3000:01shaun-mahoodI'm trying to hook a custom generator to a spec and I can't quite get it working - I suspect it's something simple that I'm missing. Details in thread.#2017-11-3000:01shaun-mahoodI'm trying to hook a custom generator to a spec and I can't quite get it working - I suspect it's something simple that I'm missing. Details in thread.#2017-11-3000:01shaun-mahoodMy custom generator is
(def gen-7-digit-plan (gen/fmap #(apply str %)
                                (gen/tuple (gen/choose 0 9)
                                           (gen/choose 0 9)
                                           (gen/choose 0 9)
                                           (gen/choose 0 9)
                                           (gen/choose 0 9)
                                           (gen/choose 0 9)
                                           (gen/choose 0 9))))
#2017-11-3000:04shaun-mahoodSpec is (s/def ::plan (s/with-gen string? gen-7-digit-plan))#2017-11-3000:05shaun-mahoodError message starts with #object[TypeError TypeError: self__.gfn.call is not a function]#2017-11-3000:06shaun-mahoodWhat I'm actually looking for is a 7 digit string with a range from 0000000 to 9999999 - probably there's a way better way to do this.#2017-11-3000:06taylor(s/def ::plan (s/with-gen string? (constantly gen-7-digit-plan)))#2017-11-3000:07taylorI know test.chuck lib has regex->string generators https://github.com/gfredericks/test.chuck#string-from-regex#2017-11-3000:08shaun-mahoodAwesome, thanks!#2017-11-3006:59misha@U054BUGT4 I might be wrong, but it seems like every gen override place in spec expects no args fn returning actual generator, hence ā€œconstantlyā€ above. #2017-11-3007:00mishae.g. s/exercise is one of those places#2017-11-3005:09quanI'm trying to spec clojure.string/index-of
(defn- str-or-char? [s] (or (string? s) (char? s)))

(s/fdef clojure.string/index-of
        :args (s/alt :idx (s/cat :s string?
                                 :value str-or-char?)
                     :idx+from (s/cat :s string?
                                      :value str-or-char?
                                      :from-index int?))
        :ret (s/nilable int?))
but running into this weird exception when instrument
(stest/instrument)
(clojure.string/index-of "abc" "a" 0)
;=> CompilerException java.lang.ClassCastException: clojure.spec.test.alpha$spec_checking_fn$fn__2943 cannot be cast to clojure.lang.IFn$OOLO
#2017-11-3005:09quannot sure what's wrong here?#2017-11-3006:13seancorfield@quan I believe that happens when you try to instrument a function that has arguments type-hinted to primitives. The IFn$OOLO type is (Object, Object, long) -> Object and is optimized for the primitive argument and that breaks spec. It's a known issue (I don't recall the JIRA issue number).#2017-11-3006:17quanah yes, from-index has ^long hint#2017-11-3006:25shaun-mahoodI want to use spec generators to build sample data when I'm prototyping a front end app. The generators return nil when I try to use them at runtime after requiring the file, but they work fine if I run them directly in the repl. Is there a good way to get these to work at runtime?#2017-11-3014:55mikeyjcatHi. Iā€™m using spec to validate data being prepared for transactions on datomic, including tempidā€™s. All works good. There is now a spec registered for :db/id that checks to make sure it is a instance of datomic.db.DbId. Wonderful, and I even created a custom generator. However, now when I am using spec with data read in, I have data with :db/id 123345, rather than a instance of datomic.db.DbId. This obviously fails the spec in instrumentation. It seems as though I need two versions of the spec for :db/id but this cannot be, in my understanding. Anyone tried anything similar?#2017-11-3021:14eriktjacobsen@U1NL85J57
(s/def ::uuid (s/or :int integer? :uuid uuid? :sha256 ::sha256))
#2017-11-3015:59Alex Miller (Clojure team)your spec is not capturing all the possible forms of :db/id#2017-11-3019:18tcouplandhas anyone got a tidy way of writing core.test is checks of spec/check? I've got something that works, but it's pretty ugly!#2017-11-3020:00tayloris it uglier than this?
(is (not (:check-failed (st/summarize-results (st/check `foo)))))
#2017-11-3020:41misha-> !#2017-11-3021:15tcouplandVery similar tbh! Do you get good failure messages out of that?#2017-11-3021:16tcoupland
(defn check
  [sym]
  (-> sym
      (stest/check {:clojure.spec.test.alpha.check/opts {:num-tests sample-size}})
      first
      stest/abbrev-result
      :failure))

(defmacro is-check
  [sym]
  `(is (nil? (check ~sym))))
#2017-11-3021:16tcouplandthat's what i'm using at the moment#2017-11-3022:01taylornice#2017-12-0109:21tcouplandwhen i get a minute i might have a play with sumarize, might give a clearer failure message#2017-12-0105:25Shantanu KumarHi, has anybody successfully used spec for web validation and translated the validation failures to HTTP 400 for nested data models? Any example or references would be great to have.#2017-12-0106:47ikitommiDonā€™t know about success, but there are routing libs like compojure-api and reitit, which support spec as web (backend) validation. There is an example in https://github.com/metosin/c2. For invalid input, produces http 400 with this kind of body:
{
  "spec": "(spec-tools.core/spec {:spec (clojure.spec.alpha/keys :req-un [:c2.spec/x :c2.spec/y]), :type :map, :keys #{:y :x}, :keys/req #{:y :x}})",
  "problems": [
    {
      "path": [
        "y"
      ],
      "pred": "clojure.core/int?",
      "val": "a",
      "via": [
        "c2.spec/y"
      ],
      "in": [
        "y"
      ]
    }
  ],
  "type": "compojure.api.exception/request-validation",
  "coercion": "spec",
  "value": {
    "x": 100,
    "y": "a"
  },
  "in": [
    "request",
    "body-params"
  ]
}
#2017-12-0106:49ikitommiso, just the :problems polished so that it can be serialized: https://github.com/metosin/reitit/blob/master/modules/reitit-spec/src/reitit/ring/coercion/spec.cljc#L84-L87#2017-12-0106:51Shantanu KumarI was looking at https://github.com/metosin/spec-tools meanwhile and thought of pinging you!#2017-12-0106:54Shantanu KumarWe are using a different data-driven router, so looking for a standalone validation mechanism using spec. Is there any such library?#2017-12-0106:59ikitommiWell, reitit is kinda modular, if you take the metosin/reitit-spec isā€™s flattened deps tree is: spec-tools, clojure.spec & meta-merge#2017-12-0107:00ikitommiit bring the ring-module but it has no dependencies. Could move the Coercion protocols into own module.#2017-12-0107:02ikitommido you think the coercion should be totally out of reitit (modules)? I would be more polite towards other routing libs at least.#2017-12-0107:04Shantanu KumarThanks, let me try out Reitit-spec. Iā€™m sort of new to spec. And yes, compat with other routing libs would be very useful.#2017-12-0107:05Shantanu KumarThough canā€™t comment now about code organization.#2017-12-0107:08ikitommithere is already compat, but needing to include another routing lib to get coercion for another is kinda odd. Plan is to ship coercion interceptors too (as they donā€™t require any new deps), so they should be usable from ā€œpedestal-style interceptor web libsā€#2017-12-0107:10ikitommiare you using mw or interceptors? and if interceptors, which kind?#2017-12-0107:11Shantanu KumarI am using Ring with a routing lib, but right now just using plain app code to do validation (itā€™ a new project) - no m/w for now.#2017-12-0107:13ikitommiok, I think I removed the ā€œvanilla ring coercion middlewareā€ in favour of the reitit ā€œcompiledā€ one (much faster). Could push them back to support other routing libs. This: https://github.com/metosin/reitit/commit/7979c9de9d42afd611e13da40502ac8ea1c0c0e3#2017-12-0107:18Shantanu KumarI think I can form an opinion about the commit after some more experience with spec.#2017-12-0107:18Shantanu KumarIā€™m looking at https://metosin.github.io/reitit/ ā€” is there a section that describes reitit-spec in particular?#2017-12-0107:19ikitommihttps://metosin.github.io/reitit/ring/coercion.html#2017-12-0107:22ikitommiI could cook up examples of that with plain ring. Most likely have time on the weekend.#2017-12-0107:23Shantanu KumarThanks! Looking into it. More examples would be cool for sure.#2017-12-0108:51ikitommi:+1:#2017-12-0110:10Andreas LiljeqvistHow would I define keys and sharing them between specs? like `(def base [::id ::something]) (s/keys :req base)`#2017-12-0110:11Andreas LiljeqvistI suppose it fails because of compile-time evaluation#2017-12-0110:14andre.stylianoss/merge?#2017-12-0110:15andre.stylianoshttps://clojure.github.io/spec.alpha/clojure.spec.alpha-api.html#clojure.spec.alpha/merge#2017-12-0110:15andre.stylianosYou can define a base spec and merge that into others#2017-12-0111:44Andreas Liljeqvist@andre.stylianos Thank you, that seems to be exactly what I am looking for#2017-12-0113:15mpenetmost of the time you use s/merge you probably want s/and instead#2017-12-0113:15guywhy do you think that is?#2017-12-0113:15guyJust want to understand thats all#2017-12-0113:17mpenetfrom what @andreas862 asked, he wants to be able to combine specs, not necessary merge them#2017-12-0113:17Alex Miller (Clojure team)I donā€™t think I would agree with that#2017-12-0113:17Alex Miller (Clojure team)This is exactly when you want to merge#2017-12-0113:19mpenetwhat are the advantages ? genuinely curious. I know about gen#2017-12-0113:20mpenetI remember also merge having arguably odd behavior with conforming#2017-12-0113:25Alex Miller (Clojure team)gen and conform#2017-12-0113:27Alex Miller (Clojure team)in that conform doesnā€™t flow#2017-12-0113:28Alex Miller (Clojure team)gen is the big one though#2017-12-0113:39ikitommiIs there a reason that conform doesnā€™t flow?#2017-12-0113:39Andreas LiljeqvistJust adding that what I wanted was to merge even if my wording was a bit unclear#2017-12-0113:53ikitommi
(s/def ::a string?)
(s/def ::b (s/and (s/conformer str/upper-case) string?))

(s/conform
  (s/merge
    (s/keys :req-un [::a])
    (s/keys :req-un [::b]))
  {:a "kikka", :b "kukka"})
; => {:a "kikka", :b "KUKKA"}

(s/conform
  (s/merge
    (s/keys :req-un [::b])
    (s/keys :req-un [::a]))
  {:a "kikka", :b "kukka"})
; => {:a "kikka", :b "kukka"}
#2017-12-0114:13mpenetanother thing I prefer not to use lately: custom conformers šŸ™‚#2017-12-0114:15gfredericksspeaking of schema-related transformations, I've been experimenting with reviving the stuff in https://github.com/gfredericks/schema-bijections as a library that only deals with bijections, not referring directly to schema/spec at all I'm curious how useful something like that would be for transforming data to the format that a spec expects (and back? e.g. jdbc)#2017-12-0114:16gfredericksI think generating bijections from specs would be the ideal result#2017-12-0114:16mpenetthat'd be useful#2017-12-0114:16gfredericksI still have to decide if/how to incorporate surjections though#2017-12-0114:16mpenetI tend to write this kind of stuff with specter these days, but that's quite gory#2017-12-0114:16gfrederickssince those are also useful in some cases#2017-12-0114:19mpenettbh I didn't really like the form it took in schema, but that was practical. I am not sure I have seen a solution I like for these kind of transformations yet#2017-12-0114:19gfredericksthe form what took in schema?#2017-12-0114:19mpenetcoercers#2017-12-0114:20gfredericksoh right -- I don't like them because they represent surjections everywhere, even for uses when bijections can work and it makes it easy to test your function-that-coerces-its-inputs with the canonical form of the data, since that's easier, and never actually test the production form that has to be coerced#2017-12-0114:22danielnealbijections from specs would be awesome āž•#2017-12-0114:23gfredericksI am envisioning that you can define different styles for other representations, like json and jdbc, and then get bidirectional functions for converting your canonical spec form from one to the other#2017-12-0114:23gfrederickse.g., "I want camel-cased keys, I want date-times serialized like this" etc.#2017-12-0114:24gfredericksthat category of things, but hopefully arbitrarily flexible#2017-12-0114:24gfredericksso if you say your json representation has camelCased keys while your internal representation is kebab-cased, then a json input with kebab-cased keys would be considered incorrect and cause an exception rather than just silently be accepted#2017-12-0114:27mpenetThere was a hint of a next iteration of spec coming out at some point (I think in Rich's talk), I am wondering if there are changes related to these kind of things (and others often mentioned).#2017-12-0114:28mpenetI personally am curious to see how spec forms conforming (or whatever replaces it) will take form#2017-12-0114:29mpenetrelated to CLJ-2112#2017-12-0114:33gfredericksoh yeah, I forgot about that#2017-12-0114:59mpenetand specs metadata too, could help for a lot of things#2017-12-0115:17gfredericksyou mean enabling specs to have metadata attached?#2017-12-0115:18gfredericksI assume they already can so you must mean something else#2017-12-0115:19mpenetyou can't attach metadata to specs atm#2017-12-0117:35ikitommi@alexmiller related to the s/merge problem, there is a PR request in spec-tools of a merge that seems to fix the problem. Could that be pushed to Spec itself? Comments welcome: https://github.com/metosin/spec-tools/pull/91#2017-12-0117:36Alex Miller (Clojure team)fixes what problem?#2017-12-0117:37ikitommiMy comment on 15:53, that canā€™t be right?#2017-12-0117:38ikitommie.g. merge merges that values, so the unconformed value override the conformed ones.
#2017-12-0117:41Alex Miller (Clojure team)it doesnā€™t override, itā€™s just that only the last spec in the merge controls the conformed result#2017-12-0117:42Alex Miller (Clojure team)that is the intent, not a bug, but it can have some non-obvious effects, particularly with unqualified keys#2017-12-0117:43Alex Miller (Clojure team)I think there is a ticket for this already in jira, but not sure what should be done with it#2017-12-0117:44ikitommioh, so that works with qualified keys. Thatā€™s interesting.#2017-12-0117:45ikitommihttps://dev.clojure.org/jira/browse/CLJ-1981#2017-12-0117:48ikitommiWoudnā€™t @arttukaā€™s PR fix that? Could the issue be re-opened?#2017-12-0117:51Alex Miller (Clojure team)I wouldnā€™t re-open that, but if there is a good statement of a problem and a patch, we can take a look at it#2017-12-0117:54Alex Miller (Clojure team)that is, in a new ticket#2017-12-0117:52Alex Miller (Clojure team)I have no idea what that PR is doing at a glance#2017-12-0117:53Alex Miller (Clojure team)Rich has a pretty extensive re-work coming for spec (post 1.9 release) and I suspect we wonā€™t really look at any fixes till weā€™re on the other side of that#2017-12-0316:32danielnealAh interesting! What kind of rework? Is it on the implementation side or is there going to be some kind of change to the interface?#2017-12-0117:58ikitommiok, Iā€™ll ask Arttu to post a new issue out of that. And looking forward to the re-work, whatever that is. thanks!#2017-12-0211:53gklijsI had an issue with a spec I defined in a .cljc file. I use an or to make one of two keys required. Thatā€™s working fine. But when I inspect the form in clj the or is without namespace, while I was expecting the namespace to be clojure.core.#2017-12-0213:43Alex Miller (Clojure team)What do you mean by ā€œinspectā€?#2017-12-0214:15gklijsI debugged by adding logging. (namespace x) is empty in the .cljc one, but clojure.core when the spec is defined in a .clj file.#2017-12-0215:52Alex Miller (Clojure team)I still donā€™t understand a) what youā€™re doing b) what youā€™re seeing or c) what you expect to see#2017-12-0216:28gklijsWhat I do is have a library which for a subset of whatā€™s possible to spec, transform the namespaced data into a vector without namespaces in such a way that Iā€™m able to get back to namespaced map from the vector. Part of this is checking the form of the spec used. When you use clojure.spec.alpha/keys, you can use clojure.core/or to be able to spec ā€œone of these should be thereā€. Now when the spec id defined in a .clj file the or is namespaces eith clojure.core, but when the spec is defined in a .cljc file the or has no namespace. I now solved it by looking only at the name of the or symbol. So there is no real problem anymore, but I was surprised this was happening.#2017-12-0315:56ikitommi@kumarshantanu did an example of normal ring app with the spec-coercion via a middleware#2017-12-0315:57ikitommivia a middleware: https://github.com/metosin/reitit/blob/master/examples/just-coercion-with-ring/src/example/spec.clj#2017-12-0315:59ikitommisame with reitit, where the data is read from route data instead of middleware args: https://github.com/metosin/reitit/blob/master/examples/ring-example/src/example/spec.clj#2017-12-0316:00ikitommiif you have questions, please join #reitit#2017-12-0318:04Shantanu KumarThanks a lot!#2017-12-0414:03andrea.crottiis this actually valid Clojure?
(defn run-query [service query]
  (let [{::keys [result error]} (invoke-service service {::query query})]
    (or result error)))
#2017-12-0414:03andrea.crottijust noticed in the clojure spec documentation https://clojure.org/guides/spec#2017-12-0414:03andrea.crottishould it not be :clojure?#2017-12-0414:05bronsathat's valid clojure yes, why does that confuse you?#2017-12-0414:11danielnealmaybe it's those namespaced keywrods#2017-12-0414:47andrea.crottiwell it doesn't compile
1. Caused by java.lang.RuntimeException
   Unable to resolve symbol: result in this context
#2017-12-0414:47andrea.crottiwith ::keys, but it does with :keys#2017-12-0414:48bronsawhich version of clojure?#2017-12-0414:48andrea.crotti1.8#2017-12-0414:48bronsause 1.9#2017-12-0414:49bronsa::keys is 1.9 syntax#2017-12-0415:18denikIs there a way to get a function spec as data?#2017-12-0415:19taylor
(s/form `my-fn-name)
#2017-12-0415:22bbrinckIf youā€™re looking for the :fn part of an fdef spec, you can use this:
(s/form (:fn (s/spec `my-fn-name)))
#2017-12-0415:31stathissiderisAnd you can also dig deeper and get the :args part:
(s/form (:args (:fn (s/spec `my-fn-name))))
#2017-12-0415:44denikneat! I forgot to call s/form on the retrieved spec:
(s/form (s/get-spec `foo))
#2017-12-0417:01denikcan the spec from s/keys be adapted as s/keys* to avoid repetition?
(s/def :foo/id number?)
(s/def ::foo
  (s/keys :req [:foo/id]))              ;; <-----------

(defn new-foo [& {:as attr-vals}]
  ;; add custom attrs
  attr-vals)

(s/fdef new-foo
        :args (s/cat :attr-vals (s/keys* :req [:foo/id])) ;; <----------
        :ret ::foo)

(new-foo :foo/id 1 :foo/bar "baz")
#2017-12-0417:53denik
(defmacro spec-keys->spec-keys*
  [spec]
  (let [form (s/form spec)]
    (if-let [[_ & args] (and (seqable? form) (= (first form) `s/keys) form)]
      `(s/keys* 
#2017-12-0500:03Drew VerleeCan anyone recommend any good resources on clojure spec, or property based testing concerning side effects (like reading, writing from files). I just watched ā€œA Deep Specification for Dropbox - Benjamin Pierceā€ : https://youtu.be/Y2jQe8DFzUM and have to assume the ideas from it were a motivation for clojure spec (given the timing). In the video, their is an argument that that deep specifications lead to more reliable software, which i agree with. He also argues, that this requires a lot of time and effort, which, also makes sense. My experience with spec so far has been that its hard to see how you can adapt them to handle side effects (writing to files, calling apis, dbs, etcā€¦) In the talk, the side effects are handled by observing them, and checking if the observations follow a specification. In general, is the approach you need to take if you want a specification over side effects? My intuition is that spec is more about modeling the non side effect parts of your application. But it feels like so much work is generally done as side effects, that the cost is fairly high to create the specs and generators that ā€œstubā€ out the side effect. Or really, what an ā€œeffectiveā€ stub would be.#2017-12-0500:22hiredmanthat youtube talk is more a long the lines of using test.check directly, not using spec#2017-12-0500:24hiredmanI am sure some people are using spec for that kind of thing, and you might use spec as a nice dsl for creating generators for that, but you wouldn't use spec's instrumentation stuff to do that kind of testing#2017-12-0500:30hiredmanin general, when I am trying to test a whole process, I use test.check to generate operations, feed them in to the process, then compare the result with what my model says it should be. spec (at least how I use it, and there are a lot of parts to spec so it can be used in a lot of different ways) is more about describing data that is moving around in the system. and I end up using those descriptions in assertions to make sure what I expect is there, to parse/recognize which of some alternative data a function was passed, or to generate other descriptions of the data for other tools (schemas, serializers, etc). I pretty much never use spec to annotate a function and then instrument that function.#2017-12-0500:36hiredmanI think generative testing splits in to two classes. Class 1 is what you see in simple examples on line, you have a pure function that does something to a list of integers, so you generate a list of integers and pass it in and assert some property. Class 2 is like what you see in that dropbox video (which is very good), you end up generating a program (a sequence of operations) that you run on the system you are testing and then also against some reference implementation (say for a distributed key value store, the ref impl might be an in memory hashmap), and compare the results.#2017-12-0500:37hiredmanspec's instrument and generative stuff seems mostly aimed at class 1#2017-12-0500:37hiredmanclass 2 is definitely a big investment in testing#2017-12-0500:52Drew Verlee@hiredman thanks, I saved your comment and I'll get around to reading it in a bit šŸ•#2017-12-0600:03zalkyHey all: I'd like make some spec assertions regardless of whether check-asserts has been set. It's important these assertions are always made. To that end I'm looking to use assert*. It's part of the public api, but the doc-string says Do not call this directly, use 'assert'.. Is there something important that I need to know? Seems like it fits the bill otherwise.#2017-12-0600:23Oliver GeorgeCould you do something like (asset (s/valid? xxx)) or (binding [s/*assert-flag-thing* true] (s/assert ::x 1)).#2017-12-0604:02Alex Miller (Clojure team)you can use assert* but itā€™s api is subject to change as part of the implementation - itā€™s public because it needs to be accessible by the macro when expanded#2017-12-0619:26hlolliI'm a complete beginner in clojure-spec, my question is, why don't I get spec error here?
(s/def ::spec1 string?)

(defn silly []
  1)

(s/fdef silly
  :ret string?
  :fn string?)

(silly)
#2017-12-0619:27taylorif youā€™re looking to assert that calls to silly have valid inputs, you can use instrument. Although thatā€™s not going to care about your :ret or :fn function specs#2017-12-0619:28taylorhttps://clojure.org/guides/spec#_spec_ing_functions#2017-12-0619:28taylorhttps://clojure.org/guides/spec#_instrumentation#2017-12-0619:29hlolliI thought that was done with :args within fdef? I was hoping it would tell me that the number 1 is not string#2017-12-0619:29taylorthatā€™s right, but thereā€™s no :args spec in your example#2017-12-0619:29taylorand you still have to instrument the function for those assertions to happen#2017-12-0619:30hlolliah ok! thanks, I'm reading trough the manual and therefore had this question, I'll continue to read further.#2017-12-0619:31tayloroh, if youā€™re only concerned about the return value of your function, instrument isnā€™t going to assert on that. You can use check for that though.#2017-12-0619:32hlolliyes that's what I was looking for, but in real world cases I'd see spec being most helpful in the args.#2017-12-0621:23bbrinckFWIW, Orchestra can be used to check ret/fn specs on instrumentation#2017-12-0621:25bbrinckhttps://github.com/jeaye/orchestra#2017-12-0622:18hlolli@U053S2W0V how is it different from cljs.spec.test.alpha/instrument that said, that's exacly the reason why I want to use spec, to get these errors right away.#2017-12-0622:19taylorinstrument only checks the :args#2017-12-0622:19taylorlooks like you can use Orchestra to get the same effect for :ret and :fn specs though#2017-12-0622:19hlolliah, nice I see#2017-12-0622:22hlollialso like their macros, very neat!#2017-12-0702:56eriktjacobsenDoes anyone have any spec->documentation projects? Something that would, for instance, render in a browser a map spec as a map with the values as the underlying spec types, that hides some complexity via being expandable? Thinking something like https://github.com/jebberjeb/specviz but that is more geared towards in-place documentation.#2017-12-0715:33joost-diepenmaat@hlolli because: you need to use clojure.spec.test.alpha/check a function to test its :ret and :fn specs#2017-12-0715:34joost-diepenmaatif you switch on instrumentation it will only test the :args spec for afn#2017-12-0715:35joost-diepenmaatnever mind. I just noticed you have a bunch of answers.#2017-12-0813:44trissHi allā€¦ anyone got a nicer way of doing this?
(defn cat-specs
  "Takes a spec and finds the highest level 's/cat' from it and extracts the
  specs of the values it matches."
  [spec]
  (->> (s/form spec)
       (tree-seq seq? identity)
       (filter seq?)
       (filter #(= 'clojure.spec.alpha/cat (first %)))
       (first)
       (rest)
       (partition 2)
       (map second)))
#2017-12-0814:20souenzzo
(defn cat-specs
  "Takes a spec and finds the highest level 's/cat' from it and extracts the
  specs of the values it matches."
  [spec]
  (->> (s/form spec)
       (s/conform (s/cat :op '#{clojure.spec.alpha/cat}
                         :args (s/* (s/cat :name keyword?
                                           :value any?))))
       :args
       (map :value)))
Some like this. This issue will help https://dev.clojure.org/jira/browse/CLJ-2112
#2017-12-2111:30trissFantastic thankyou! Apologies I just got round to understanding this#2017-12-0911:40mbjarlandI have a question, Iā€™m using spec to validate the format of a ā€œlayout stringā€ and in the spec I use conformers to convert from string to seq of chars:
(s/def ::layout-string
  (s/and string?
         not-empty
         (s/conformer seq)
         (s/+ (s/alt :col-align-bracket-expr
                     (s/cat :left-bracket #{\[}
                            :col-align-char #{\L \C \R \l \c \r}
                            :right-bracket #{\]})
                     :col-padding-non-bracket-char
                     (complement #{\[ \]})))
         (s/conformer char-seq->str-conformer)))
, is conformers the way to go or is there some cleaner way to deal with strings as sequences of chars?
#2017-12-0911:44mbjarlandI was hoping to use https://github.com/bhb/expound to make the error messages more readable, but expound does not seem to support conformers#2017-12-0911:52mbjarlandI think Iā€™m landing at using instaparse for parsing strings instead of spec. Seems like coercing proper error messages (including the location in the string etc) from spec would be more work than itā€™s worth for this problem#2017-12-0913:10Alex Miller (Clojure team)While spec can be twisted into parsing strings, itā€™s not designed for that and itā€™s going to be way slower than either a proper regex or a parser.#2017-12-0917:09Drew Verleedoes anyone have an example of how you would conform a string into a data structure, like a hasmap: (conform ::foo "8/blue") => {:age 8 :eye-color "blue"} It seems i would need to specify almost like a regex over the string. This feels doable, but i'm not sure.#2017-12-0917:24mpenetSee s/conformer. You basically pass a fn to it that does the conforming or returns ::s/invalid#2017-12-0917:24mpenetIt can also take another arg to do the inverse (aka unform)#2017-12-0919:35Alex Miller (Clojure team)I would say, donā€™t use spec for this, use a parser like instaparse#2017-12-0919:36aengelberg:+1:#2017-12-0919:36Alex Miller (Clojure team)spec is about describing the structure of Clojure data, not strings#2017-12-0920:00borkdude@mbjarland I just used Instaparse for a similar problem: https://github.com/borkdude/aoc2017/blob/master/src/day9_instaparse.clj#L14#2017-12-0920:01borkdudeI was going to ask here, Instaparse allows to hide output or hide tags to prevent extra nesting. Is such a thing with Spec also possible?#2017-12-0920:05aengelbergYou could maybe use s/and and s/conformer to flatten things as they are being parsed / conformed.#2017-12-0920:06genecwhat's the correct set of dependencies for using spec?#2017-12-0920:06genec:dependencies [[org.clojure/clojure "1.9.0"] [org.clojure/spec.alpha "0.1.94"]#2017-12-0920:08Alex Miller (Clojure team)Just clojure #2017-12-0920:08Alex Miller (Clojure team)It includes spec so you donā€™t need to declare that#2017-12-0920:08Alex Miller (Clojure team)https://clojure.org/guides/spec#2017-12-0920:10Alex Miller (Clojure team)Use of generators is optional. If you want that, youā€™ll also need [org.clojure/tools.check ā€œ0.9.0ā€]#2017-12-0920:13genec@alexmiller thanks. I'm going to use spec to validate csv files, seems like a good fit#2017-12-0920:15Alex Miller (Clojure team)I donā€™t think thatā€™s a good fit#2017-12-0920:15Alex Miller (Clojure team)CSV is text with tricky quoting rules #2017-12-0920:16Alex Miller (Clojure team)The right tool is a parser, like instaparse#2017-12-0920:16Alex Miller (Clojure team)Spec is for Clojure data#2017-12-0920:18genechmm... I'll take a look at instaparse. I have fields that can either be an int or "Missing", which I need to create a log of an errors for the user to correct before importing the file, which is why I thought spec would be good for that#2017-12-0920:21guyCould you use instaparse to parse the data, then use clojure spec to just check it after?#2017-12-0920:21genecI suppose I could, I was just going to use one of the csv libs#2017-12-0920:21guyyeah#2017-12-0920:24genechttp://www.metasoarous.com/presenting-semantic-csv/#2017-12-0920:25guyYeah i just used https://github.com/clojure/data.csv which was pretty handy#2017-12-0920:36borkdude@aengelberg thank you ā€” and thanks for Instaparse, itā€™s a really amazing tool šŸ™‚#2017-12-0921:46aengelbergGlad you find it useful!#2017-12-0922:17borkdudeYes, just used it for the first time to solve a puzzle, inspired by someone who did it in PureScript using parser combinators.#2017-12-1000:12Drew VerleeI'm somewhat confused about the exchange here: https://dev.clojure.org/jira/browse/CLJ-2116 If specs aren't meant for coercion from non clojure data structures to clojure datastructures. Then what is the recommended solution for coercion? Is the message that we shouldn't mix coercion to clojure data with validation of that data? Someone with more experience chim in please, but i feel like those things might overlap a great degree and if you cant compose the tool that is doing one with the other, then it feels like there is going to be a lot of confusing repetition.#2017-12-1000:39seancorfield@drewverlee The philosophical reason is that if your spec does coercion, no clients of your spec can avoid that coercion, and that can lead to problems.#2017-12-1000:42seancorfieldFWIW, we use spec at work to validate form parameters which all come in as strings, so we have some very limited coercion in a few of our specs. We have ::long and ::double specs that accept both a number and a string that can be parsed to a number, a ::boolean spec that accepts both a Boolean and a string that can be parsed to a Boolean (under a documented set of rules), and a ::date spec that accepts both an instant and a string that can be parsed to an instant (again, under a documented, and limited, set of rules).#2017-12-1000:43seancorfieldThat's about as far as I'd go with coercions in specs. Anything beyond that should separate the parsing operation out (and use spec to validate the resulting data structure), IMO.#2017-12-1000:50seancorfieldWe also have a REST API where the payloads are parsed (with Cheshire, via wrap-json-params and wrap-json-body) and then validated -- separately -- which makes the specs much more robust and more usable, and makes explain-data much easier to process (for example, using it to turn spec failures into human readable messages). Coercion in a spec tends to make that last part harder (since you have more operations to pick apart).#2017-12-1000:51seancorfieldCoercion in specs also often causes more work in unform and generators since you need to be able to "de-coerce" validated data.#2017-12-1000:56Drew Verlee@seancorfield That's reasonable. I suppose it works against what compjuri-api is going for. More to the point, i can't get compojure api to work anyway. Is there another restful api service i should look at? The more hand holding solution the better probably, as my use case is really bare bones.#2017-12-1000:57seancorfieldWe've built our REST APIs with just bare bones Ring and Compojure. I haven't looked at compojure-api.#2017-12-1000:57seancorfieldWe've built our REST APIs with just bare bones Ring and Compojure. I haven't looked at compojure-api.#2017-12-1008:08misha@borkdude have a look at s/nonconforming#2017-12-1009:03ikitommi@seancorfield @drewverlee The CLJ-2116 tries to solve just that, by separating the coercion from specs. Specs to define the form and apply different coercion when needed, e.g. in the borders. int? should be coerced from string->number If provided as a path-parameter, not if provided as JSON body or within your own domain code. Solution to enable this would be simple.#2017-12-1009:04ikitommithere is also CLJ-2251 on too If that which would make things fast.#2017-12-1009:06ikitommi@drewverlee If you have trouble with the spec coercion with c-api, happy to help.#2017-12-1014:23Drew VerleeThanks @ikitommi i'll ask in another channel.#2017-12-1014:37Drew Verlee@seancorfield i suppose its hard to see why i would use a spec on a string i had already successful parsed in most cases
(parse "drew,2") 
=> {:name "a" :siblings 2}
I have already discovered this line meets most of the specification * has two entries * second entry is a number If i want to build a spec about the clojure data to assert it has two entries and the first one is a string and the second is a number, i feel like i'm repeating information. I think the key point here is that, if i build a spec that does coercion on a string it wouldn't verify on the resulting structure. So its not really any more flexible or less work then separating out the coercion function and the spec.
#2017-12-1015:01genecIs there a way to use spec so that it can type hint function parameters during development? Coming from F#, it's something I really miss during development. Or is there a good post about spec demonstrating the Clojure way to keep track of data / types during development?#2017-12-1015:01genecIs there a way to use spec so that it can type hint function parameters during development? Coming from F#, it's something I really miss during development. Or is there a good post about spec demonstrating the Clojure way to keep track of data / types during development?#2017-12-1015:06taylorI donā€™t have a good answer, but I also came to Clojure from F# ā¤ļø Personally, there were lots of things I missed early on from F# but over time I found myself not missing them so much#2017-12-1015:08taylorI imagine youā€™re thinking of something that integrates into the editor? Spec does solve this problem with instrument but it happens as your code is executed, not when youā€™re typing it in#2017-12-1019:06genecthanks - yes, I'm missing the type checking from F#. I don't see any reason why there could not be type hints for things that have been spec'd in Clojure. I'll keep looking into this. Out of curiosity, what's your development env? I was using Atom/Proto-Repl but found it a bit flaky so I've switched to Intellij / Cursive and really like it.#2017-12-1021:40taylorI mostly use IntelliJ w/Cursive#2017-12-1022:05Drew VerleeOk some high level questions: 1. Is there value in exchanging name-spaced data across different systems? I feel like rich might have made a big deal about this and its gotten lost in the noise of me working with spec. 2. What would be the best way to transform a map with one namespace into another one with different namespaces. e.g
(s/def ::zoo (s/keys :car int?) 
(??? {:foo/bar 5 :foo/car 6} ::zoo)
=> {:foo/bar 5 :zoo/car 6}
This feels like what conform does, only i want it to not care about the namespace. Also that idea might be imply i'm doing something else wrong, so i'm open to suggestions.
#2017-12-1022:10Drew Verleeanother way to handle my problem would be to unnamespace the map#2017-12-1022:12Drew Verleeor i could not apply the namespace in the first place. I feel like these approaches have tradeoffs that probably dont matter in my situation but are interesting to think about.#2017-12-1022:26Drew VerleeWhile i'm thinking about it. Is there a way to combine two specs for a hash-map into one?#2017-12-1023:04Alex Miller (Clojure team)s/merge#2017-12-1023:05Drew Verleeit would be that easy wouldnt it šŸ˜• thanks @alexmiller#2017-12-1023:08gfrederickspronounced "smerge"#2017-12-1110:17genec(doc s/merge) returns nil, are there docs for spec?#2017-12-1113:22Alex Miller (Clojure team)It shouldnā€™t be if you load clojure.spec.alpha :as s#2017-12-1113:23Alex Miller (Clojure team)Docs also at https://clojure.github.io/spec.alpha/clojure.spec.alpha-api.html#clojure.spec.alpha/merge#2017-12-1110:58guyhttps://clojuredocs.org/clojure.spec/merge this is pretty bare#2017-12-1110:58guybut you also have https://clojure.org/guides/spec#_entity_maps and its got s/merge inside of it too#2017-12-1110:59guyat the bottom of this entity map part#2017-12-1111:14ikitommi@gfredericks @mpenet bumped into the need of bijections too: In routing, if a path-parameter is declared as keyword? we can easily coerce it from string with string->keyword. But with reverse-routing, we need to turn that keyword param back to string to create a path. I seem to have resolved that earlier by using a json-encoder to write things to string-like, but this canā€™t be the right way to do this.#2017-12-1111:18ikitommiI guess a simple (and fast) way would be to introduce a IntoString Protocol. But I guess there can be N different formats to read from and same N formats to write to. Hmm.#2017-12-1111:24ikitommiActually, I think I can just introduce a new type-conformer for spec-tools for the opposite direction type->string.#2017-12-1111:30ikitommiNo, using the conform requires the end value to be valid. This would require something like the CLJ-2251 with a ā€œjust transform without validatingā€ options. Or something totally different.#2017-12-1112:21gfredericksso would still be a good use case for a bijections library?#2017-12-1218:04ikitommi@gfredericks Would be best if spec itself provided the needed hooks to implement the bijections, but not likely to happen. I need something now, so will hack something. If you have good ideas how to do this properly, Iā€™m all ears!#2017-12-1111:45misha@alexmiller when will clojure.core/long? be released? it is in master, but is not in clojure 1.9.0 as I expected (or am I missing something?). Thanks https://dev.clojure.org/jira/browse/CLJ-1298#2017-12-1111:58bronsa@misha see https://github.com/clojure/clojure/commit/20f67081b7654e44e960defb1e4e491c3a0c2c8b#2017-12-1111:58mishaharold#2017-12-1111:59mishathanks, @bronsa#2017-12-1112:04mishais there out of the box generator for System.currentTimeMillis? (apart from clojure.test.check.impl/gen-current-time-millis) opieop#2017-12-1112:11gfredericks@misha how is what you're describing different from generating a long?#2017-12-1112:11mishafar greater than 0? :D#2017-12-1112:12gfredericksthey weren't far greater than 0 in 1970#2017-12-1112:12mishahonestly, I did not think it through yet#2017-12-1112:12mishapos-int? then?#2017-12-1112:12gfredericksthey were negative in 1969#2017-12-1112:13gfredericksis there a (not clojure) spec for these numbers? I've never heard of one#2017-12-1112:13mishaoh, okay.#2017-12-1112:14mishaI think the "much greater than zero" part of a spec ā€“ is a specialization for my case.#2017-12-1112:14gfredericksanyhow, with time generators you have three options A) make a generator that is not focused on your current now (cleanest if you can get away with it) B) call (time/now) somehow before constructing your generators and pass that in (lots of boilerplate potentially) C) make a nondeterministic generators (with associated damage to reproducibility and shrinking)#2017-12-1112:15gfredericksfor B) you'd want to log the time you're using, for reproducibility#2017-12-1112:18mishathank you. at this point I think I might just use int? with s/with-gen, and supply (System.currentTimeMillis) as a generator, as my only goal (so far) with this particular value - is plausible human-readable exercise data.#2017-12-1112:19misha(to the extent time-millis could be "human-readable")#2017-12-1112:20gfredericksconverting to/from time-mills would be a cool savant mental trick#2017-12-1112:20mishadoubt I would want to master it opieop#2017-12-1112:50tjtoltonIs there a megathread somewhere about 1.9 discussions? Did spec get pulled from the release as a finished product? Are macros using spec in the stable release? what's going on?#2017-12-1113:07bronsaspec got pulled out, it's still alpha but no major breaking API changes are expected, currently there's specs for ns, let, fn, defn#2017-12-1113:20Alex Miller (Clojure team)Hereā€™s the spec split announcement which explains: https://groups.google.com/forum/m/#!msg/clojure/10dbF7w2IQo/ec37TzP5AQAJ#2017-12-1114:15tjtoltonThanks @alexmiller, I'll bet that was a bummer of an internal conversation šŸ˜•#2017-12-1200:43athosAs you might already see it on the Clojure ML, I just released Pinpointer, a clojure.spec error reporter similar to Expound, Inspectable, etc https://github.com/athos/Pinpointer#2017-12-1200:44athosPinpointer formats (and even colorizes!) spec errors in a human-readable way. The difference between Expound and Pinpointer is very subtle from the outside perspective, but Pinpointer is based on a systematic error analysis rather than heuristic approaches, and this makes it possible to report more complicated errors, such that s/conformer would transform part of the input value.#2017-12-1200:46athosHave a look at it if you're interested, and any feedback is welcome šŸ˜‰#2017-12-1210:25stathissideris@athos I tried pinpointer and it gave up on my complex error, it just used explain as a fallback. Under what circumstances does it do that?#2017-12-1211:22athosThank you for giving a try to Pinpointer, @stathissideris! šŸ˜† By default, Pinpointer falls back to s/explain if something bad happens during the error analysis. That mechanism exists to prevent Pinpointer itself from ruining the original spec error info.#2017-12-1211:23athosIf you would like to turn off the fallback behavior, call pinpoint with the option {:fallback-on-error false} like (pinpoint <spec> <input> {:fallback-on-error false})#2017-12-1211:23stathissiderisok šŸ™‚ in my case I have a pretty loose spec that describes a DSL, so if an expression is wrong, the errors include a lot or different options#2017-12-1211:24stathissiderisbecause the actual error is one of many things that can go wrong#2017-12-1211:24stathissiderisalright, Iā€™ll try again with this option and see what happens#2017-12-1211:47stathissideristhanks!#2017-12-1212:18rickmoynihancan pinpointer/expound/etcā€¦ handle exceptions with spec errors in them?#2017-12-1212:18rickmoynihanor would I need to hook into the REPLs exception printer?#2017-12-1212:24degIn a re-frame CLJS app, my app-db holds entries created by multiple parts of my system. I want to check total validity at start of each re-frame event. So, I have (s/def ::db (s/merge :ns1/db-keys :ns2/db-keys ,,,)) in a central db.cljs file and I validate it in each event. Most of these parts are small and can be reasonably each held by a single file holding its specs, subs, and events. But, this creates a problem. The events need to require the namespace of the merged spec, while that namespace needs the namespaces of each part. What is the cleanest way to tastefully break this namespace circularity?#2017-12-1212:44rickmoynihanOkā€¦ partly wanting this for exceptions integrant raises from pre-init-specā€¦ but it looks like integrant actually calls s/explain-out to generate the message in the exception so it does indeed work#2017-12-1212:51athosYes, s/*explain-out* plugin mechanism should handle exceptions well. If you rather want something like clojure.repl/pst, pinpointer.core/ppt would be useful.#2017-12-1212:51rickmoynihancool#2017-12-1214:47Andreas LiljeqvistWhat is the recommended approach for shared specs in several projects? One solution is to make the spec a separate project and then require it.#2017-12-1215:03taylorIā€™ve been curious about the same. I guess the options are 1) share a project 2) serialize to some other format which obv. limits what you can do in spec.#2017-12-1215:09Andreas LiljeqvistProbably going to use a project, on the plus side I get versioned specs#2017-12-1216:23misha1) monorepo 2) shared lib 3) send/receive as edn (youā€™d need to make sure specs are cljs compatible, if sharing with ui)#2017-12-1219:00jeayeWe have a separate repo for specs shared between the client and server (edn). It also contains common functions shared between the two, defs, etc.#2017-12-1310:27Andreas LiljeqvistThanks for your input, quite helpfull#2017-12-1214:48Andreas LiljeqvistBut I am not completely happy with that idea#2017-12-1214:53stathissideriswhy not?#2017-12-1215:15Andreas LiljeqvistI suspect it is the best solution, but I am worried about code reloading and such. -- I will just have to try it out#2017-12-1215:29stathissideris@andreas862 donā€™t forget about the leiningen checkouts setting (if thatā€™s what youā€™re using)#2017-12-1215:29stathissiderisit allows you to hack on multiple non-deployed projects in parallel#2017-12-1215:30Andreas Liljeqvist@stathissideris ah, that is a nice feature!#2017-12-1215:31stathissiderisšŸ™‚ @andreas862 https://github.com/technomancy/leiningen/blob/master/doc/TUTORIAL.md#checkout-dependencies#2017-12-1215:38Andreas LiljeqvistPerhaps I should save my future self a lot of time and read the documentation...#2017-12-1215:38Andreas LiljeqvistThank you#2017-12-1221:05johanatanfor putting a spec on a def'd var, is the standard way to have a fspec on a self-invoking function?#2017-12-1221:05johanatanis there some other way?#2017-12-1221:07bfabrynot sure I understand what "putting a spec on a def'd var" means. I'm guessing the answer isn't just "use fdef"?#2017-12-1221:07bfabryor do you mean a var that's defined to be something other than a function? if so then specing it isn't supported#2017-12-1221:07bfabryor... isn't useful#2017-12-1221:08johanatanyea, like a var that's bound to the result of a function#2017-12-1221:08bfabryyeah you can't spec that#2017-12-1221:08johanatane.g., (def ^:private keys-validators (events-schema->keys-validators events-schema))#2017-12-1221:08johanatanor
(def ^:private refinements
  (merge user-defined-refinements normalized-base-refinements))
#2017-12-1221:09bfabryspec is meant for validating ranges of possibilities, a def'd var is "permanent" in clojure land. but yeah putting your right hand side inside a function and then fdefing that would make the most sense to me#2017-12-1221:09johanatanmm, ok. i'll try that. i think last time i tried it couldn't get it to work with anonymous functions (and wasn't really excited about polluting the global namespace with another defn)#2017-12-1221:11johanatando you know if it is possible to fspec an anonymous function?#2017-12-1221:11bfabryI mean seeing as this is all only happening once you could just do (def foo (s/assert ::spec (your code here)))#2017-12-1221:11johanatanhmm, except that it would probably need a doto so that the result is returned?#2017-12-1221:12bfabryassert returns the result if the result was valid#2017-12-1221:12johanatanoh, ok#2017-12-1221:12johanatanyea, that works. thx!#2017-12-1221:13bfabryno worries#2017-12-1221:18hlolliHow does one do or in spec, I've tried googling
(s/explain (s/or :k1 (s/coll-of ::ar ::ar)
                 :k2 (s/coll-of ::ar ::kr)) [(AudioSignal. 1) (ControlSignal. 1)])
I also don't need these keys, but this function is probably wrong. Im makeing a transpiler to a language that has it's own dispatch functions, and since I'm automatically generating the metadata, I end up with different possibilites. Essentally, I'd want to be able to say; this vector can be a sequence of specs ::x ::y or sequence of specs ::foo ::bar.
#2017-12-1221:19taylors/tuple?#2017-12-1221:20hlollitroughout the ocean of functions I want to spec, they can be tuple/octet or whatever number of possibilites...#2017-12-1221:20Alex Miller (Clojure team)probably want to look at the regex specs (cat * + ? alt)#2017-12-1221:22hlolliok, will look there. Lot to take in when first stepping into spec šŸ™‚#2017-12-1221:24Alex Miller (Clojure team)if I read what you want to do there for example you could say (s/cat :x ::ar :y (s/alt :a ::ar :k ::kr))#2017-12-1221:25hlollicat defines a sequence?#2017-12-1221:25Alex Miller (Clojure team)to match sequential collections that start with ar and then have ar or kr. x, y, a, and k are all made up keys that affect the conformed result#2017-12-1221:25Alex Miller (Clojure team)anything sequential#2017-12-1221:25Alex Miller (Clojure team)seq, vector, list#2017-12-1221:27hlolliok nice, this is actually for the input parameters so it's importent, I'm tempted to do this check at the very beginning of my functions.#2017-12-1221:27Alex Miller (Clojure team)they are most commonly used in s/fdef to spec args to functions#2017-12-1221:27Alex Miller (Clojure team)so that is a good match#2017-12-1221:29hlolliyes I may end up doing that, it's just this instrument function that I'm skeptical of, but I should not be. I want immediate error in runtime on wrong spec, or no matching patterns of a provided arguments, it's kinda pattern matching, therefore looking at or#2017-12-1221:29Alex Miller (Clojure team)s/assert might be useful too#2017-12-1221:30hlolliOn extra question since I'm rambling, is there a way to have any effect on the error message In: [1] val: #object[csound.core.ControlSignal] fails spec: :csound.core/ar at: [:k1 :0] predicate: audio-signal? I'm in clojurescript and I know that the object has keys which could help the user locate his problem, as this could be within a nested ast.#2017-12-1221:32taylors/explain-data gives you a structure you can get useful stuff out of, and there are some libs for transforming that into something that could be useful for you#2017-12-1221:33taylorhttps://github.com/bhb/expound https://github.com/alexanderkiel/phrase#2017-12-1221:33hlolliah yes! in fact the object itself in question.#2017-12-1221:35hlolliThanks @U3DAE8HMG#2017-12-1221:35hlolliOr explain has it too, just didn't consider takeing advantage of that and throw my own error message, suited to help the user find the spec mismatch.#2017-12-1420:01bfaypssst @U0CAUAKCG whatcha doing with csound? Looks cool#2017-12-1420:12hlolliWho's asking @U0MF00YRX? I'm working on some interoperability between clojurescript and csound, mainly transpiler, that would be a good start, already been live-coding with this combination for few years#2017-12-1420:16bfayOh awesome @U0CAUAKCG! I'm trying to do some similar stuff with clojurescript/supercollider. I tried using Overtone for a bit, but found that the startup time is too painful, especially when I tried running on a Raspberry Pi#2017-12-1420:24hlolliYes, Overtone is bit over engineerd for Raspberry Pi. But it's way better software than Sonic Pi in my opinion.#2017-12-1420:34bfayI think Sam Aaron had a different vision for both. The vision for Sonic Pi is about live-coding and education, it's a fun way to teach newcomers how to code. But yeah possibilities are pretty limited compared to Overtone and plain SuperCollider#2017-12-1304:37johanatanhi, does anyone know how to diagnose the following error?
1. Unhandled java.lang.IllegalArgumentException
   No implementation of method: :specize* of protocol:
   #'clojure.spec.alpha/Specize found for class: nil

#2017-12-1304:42johanatani found this but it seems only tangentially related: https://dev.clojure.org/jira/browse/CLJ-2032#2017-12-1304:44johanatanhere's the code:
(s/def ::json-primitive
  (s/or :n nil :b boolean? :n number? :s string?))

(s/def ::json-structure
  (s/with-gen
    (let [checker (fn inner [primitives? v]
                    (cond
                      (map? v) (and (every? string? (keys v)) (every? (partial inner true) (vals v)))
                      (coll? v) (every? (partial inner true) v)
                      primitives? (s/valid? ::json-primitive v)
                      :else false))]
      (partial checker false))
    #(gen/recursive-gen (fn [inner] (gen/one-of [(gen/list inner) (gen/vector inner) (gen/map gen/string inner)]))
                        (s/gen ::json-primitive))))

(s/def ::validation-fn
  (s/or :func (s/fspec :args (s/cat :x ::json-structure)
                       :ret boolean?)
        :spec s/spec?))

(s/def ::message-fn
  (s/fspec :args (s/cat :x ::json-structure)
           :ret string?))

(defn- seq->gen
  "Takes a sequence of generators and produces a generator of sequences."
  [seq]
  (apply gen/tuple seq))

(defn- map-seq->gen
  "Takes a sequence of values and a function to apply over them
  and produces a generator of the sequence of mapped values."
  [f val-seq]
  (seq->gen (map f val-seq)))

(s/def ::refinements
  (s/with-gen
    (s/map-of keyword? (s/tuple (s/nilable keyword?) (s/tuple ::validation-fn ::message-fn)))
    #(gen/let [kwds (gen/vector gen/keyword 5 25)
               refinements
               (map-seq->gen
                (fn [kwd]
                  (gen/let [k (gen/frequency [[4 (gen/elements (clojure.set/difference (set kwds) (set [kwd])))]
                                                [1 (gen/return nil)]])
                            validation-fn (gen/frequency [[8 (gen/return (with-meta (fn [_] true) {:validates? true}))]
                                                          [1 (gen/return (with-meta (fn [_] false) {:validates? false}))]])
                            message-fn (gen/fmap (fn [s] (with-meta (fn [_] s) {:msg s})) (s/gen string?))]
                    [kwd [k [validation-fn message-fn]]]))
                kwds)]
       (into {} refinements))))
and the invocation:
(gen/sample (s/gen ::refinements) 1)
#2017-12-1304:45johanatanyou'll need appropriate imports; i.e., s and gen from the spec guide#2017-12-1313:17lopalghost(s/def ::json-primitive (s/or :n nil šŸ˜› boolean? :n number? :s string?))#2017-12-1313:18lopalghostshould that be nil?#2017-12-1313:19guynil? is a predicate whereas nil is not#2017-12-1313:19guyis that what you mean?#2017-12-1313:20guy (s/or :n nil? :b boolean? :n number? :s string?))#2017-12-1313:20guyi think s/or takes keyword, predicate pairs#2017-12-1313:20lopalghostyes, this was in response to @johanatan#2017-12-1313:21guyah sorry#2017-12-1313:21guyšŸ‘#2017-12-1313:21guyGood spot then!#2017-12-1317:25johanatan@lopalghost good catch! thx!#2017-12-1317:43johanatandoes anyone know a way for an fspec to consider relations between its target's inputs? like if one of the parameters is functionally constrained by the value of the other one?#2017-12-1317:44johanatani could do it by stuffing the args into a nested vector:
(defn f [[a1 a2]] ...)
and then writing a spec/generator for that vector which combines two other [independent] spec/generators but i was kind of hoping that there is a better story for this...
#2017-12-1317:52lopalghostthe :args argument already treats the args as a sequence, so you can use (s/and ...) to spec both individual inputs and relations between them#2017-12-1317:52lopalghostthere's a good example in the spec guide: https://clojure.org/guides/spec#_spec_ing_functions#2017-12-1317:53johanatanah, nice! thx again#2017-12-1318:21jcthalysHi, how can i let this second element of a tuple optional?#2017-12-1318:21jcthalys
(s/tuple ::string-not-blank
                                          (s/nilable pos-int?))
#2017-12-1318:22jcthalysthis is accepted ["ChLh2" nil] but this not ["ChLh2"]#2017-12-1318:32lopalghosttuple is for a collection of fixed size. for variable size you want to use cat#2017-12-1318:33lopalghosteg
(s/cat :string ::string-not-blank :number pos-int?)
#2017-12-1318:34lopalghostsorry, that should be
(s/cat :string ::string-not-blank :number (s/? pos-int?))
#2017-12-1318:34lopalghostif you want the second term to be optional#2017-12-1319:04jcthalysin my case itā€™s a vactor, with one string and when have another thing its that intā€¦#2017-12-1319:05jcthalysthatā€™s s/cat return a mapā€¦#2017-12-1319:53lopalghostafter you conform vector, you can use unform to turn it back into a sequence#2017-12-1319:53lopalghostkeep in mind you don't need to use conform at all#2017-12-1319:56lopalghostthe alternative would be something like:
(s/or :two (s/tuple ::string-not-blank) 
              :one (s/tuple ::string-not-blank pos-int?))
#2017-12-1319:57lopalghostor otherwise, pad the vector with a nil before conforming, if it only has one element#2017-12-1319:57lopalghostI think cat would be most idiomatic though#2017-12-1320:19johanatan@lopalghost actually s/and for that purpose will result in a such-that and you can be at risk of a couldn't satisfy such-that predicate after 100 tries error#2017-12-1320:20johanatanthere needs to be a way to generate the two values together so that we know the two will conform with one another#2017-12-1320:20johanatanhence my suggestion to use a nested vector of args#2017-12-1320:21johanatando you see any other way?#2017-12-1320:38tristefigureHi. This is my first time using spec, and taking inspiration from clojure.core.specs.alpha, I decided to write my own version to parse/produce macro definitions. I wanted it to return something less detailed in a flatter hash and I wasn't happy with the fact results from conform mirror the spec' structure and with other petty details, so this is what I came up with:#2017-12-1320:38tristefigureTo which extent is this a misuse of this library ?#2017-12-1321:35lopalghost@johanatan not sure how a nested vector would solve the problem, as any spec you apply to the nested vector could also be applied to :args#2017-12-1321:36johanatan@lopalghost it solves it by letting me introduce a generator specifically for that tuple#2017-12-1321:38lopalghostYou can't do that with the :arts spec? Sorry if I'm misunderstanding you#2017-12-1321:38lopalghost:args*#2017-12-1321:38johanatane.g.,
(s/def ::args-tup
  (s/with-gen
    (s/tuple ::arg1 ::arg2)
    #( ... enforce relations here )))

(s/fdef -refinement-kwd->validator
        :args (s/cat :tup ::args-tup)
        :ret ::ret-val)
(defn- -helper [[arg1 arg2]] ; tuple allows both inputs to be generated simult.
  ;; omitted
)

(defn- original [arg1 arg2]
  (-helper [arg1 arg2]))

#2017-12-1321:39johanatanyou can do it in :args but you are liable to hit the cannot satisfy such-that problem#2017-12-1321:47lopalghostI mean why not just use ::args-tup as the :args spec? Does that cause a problem? I'd try it out myself but I don't have a repl at the moment#2017-12-1321:50johanatans/cat is required to name the arg#2017-12-1321:50johanatanyea, without s/cat, stest/check tries to send in the wrong number of arguments#2017-12-1321:50johanatan[i.e., sends in 2 where 1 is expected]#2017-12-1322:09lopalghostok I cobbled together a stupid example, let me know if this is relevant:
(s/def ::args-tup (s/and (s/tuple pos-int? pos-int?)
                         #(> (first %) (second %))))

(s/fdef subtract
        :args ::args-tup
        :ret pos-int?)

(defn subtract
  [high low]
  (- high low))
#2017-12-1322:10lopalghostdidn't supply a generator of course, but stest/check works with no problems here#2017-12-1322:32hlolliWould it be wise to spec no argument, if just for the metadata, if so, how would someone spec :args for no args?
#2017-12-1406:59curlyfryJust (s/cat)!#2017-12-1410:08hlolliThusund takk!#2017-12-1322:35johanatan@lopalghost that use of s/and has an implied underlying gen/such-that which depending on circumstances may be impossible or too difficult to satisfy. thus it is always better to generate exactly the right structure you want from scratch rather than relying on such-that.#2017-12-1322:49johanatan@lopalghost also, your fdef won't work: :args is an alternating sequence of name and type/spec. thus you need s/cat#2017-12-1415:03lopalghost@johanatan seems to work fine
> (s/exercise-fn `subtract)
> ([[2 1] 1] [[7 1] 6] [[3 1] 2] [[4 3] 1] [[13 4] 9] [[5 4] 1] [[7 3] 4] [[17 1] 16] [[96 3] 93] [[14 3] 11])
> (stest/check `subtract)
> ({:spec #object[clojure.spec.alpha$fspec_impl$reify__2451 0x2bfc33f4 "
#2017-12-1415:04lopalghostis there a reason you need the args vector to conform to a map?#2017-12-1415:05lopalghostif you supply a generator for ::args-tup, you shouldn't have to worry about failure during generative testing#2017-12-1415:13gfrederickseasy generator for a strictly descending pair of numbers: (gen/let [[a b] (gen/tuple gen/large-integer gen/large-integer)] (let [[a b] (sort [a b])] [(inc a) b]))#2017-12-1415:47andy.fingerhutI was looking at updating the Clojure cheat sheet for v1.9, now that it is out. I was wondering -- is there a list somewhere of all Clojure predicates for which test.check has generators included, vs. those it does not have generators for?#2017-12-1415:48gfredericksI expect there's a data structure in the spec code mapping specs to generators#2017-12-1415:49andy.fingerhutIf someone beats me to finding a link to that, let me know. I may want to link to it.#2017-12-1415:50gfredericksif you have a list of predicates, then programmatically figuring out which have generators should be easy#2017-12-1415:52andy.fingerhutIs there some small expression you can evaluate in a REPL to determine if a predicate has a generator already?#2017-12-1415:53gfredericksI thought there was a function in the spec API that takes a spec and returns a generator#2017-12-1417:03Alex Miller (Clojure team)s/gen#2017-12-1417:03Alex Miller (Clojure team)There are some things in gen namespace too#2017-12-1417:04Alex Miller (Clojure team)And thatā€™s where the built in mapping is set#2017-12-1417:05lopalghostgen/gen-for-pred#2017-12-1417:47andy.fingerhutThanks, s/gen is very helpful. And dang, I could have really added a lot of value by helping with the generator for zero? šŸ™‚#2017-12-1417:50andy.fingerhutI couldn't have been very helpful, of course. Simply my way of expressing surprise that there was such a generator.#2017-12-1420:41gfredericks(gen/one-of (gen/elements [(float 0) (int 0) 0.0 0 0N (biginteger 0) -0.0]) (gen/let [scale (gen/large-integer* {:min Integer/MIN_VALUE :max Integer/MAX_VALUE})] (.setScale 0M scale))) I assume?#2017-12-1420:56andy.fingerhutYes, the most complex part about it is if you want to generate every value that can satisfy zero?#2017-12-1420:57andy.fingerhutI probably should have made my snide remark about the predicate nil? instead, since it would be more obvious.#2017-12-1420:57andy.fingerhutI am probably having a senior moment here on my memory -- is there any reasonably common term for any type that isn't a collection?#2017-12-1420:58tayloritā€™s not scalar is it#2017-12-1420:58andy.fingerhutIt is, and I am understanding more every day what memory lapses feel like šŸ™‚#2017-12-1420:59taylorI increasingly find myself forgetting what Iā€™m about to do as I walk across my tiny apartment to do it#2017-12-1421:44gfrederickstattoos are the answer put "things that aren't collections are scalars" on your left arm#2017-12-1421:47cgoreAnd ā€œthere are no scalarsā€ on your right arm šŸ˜„#2017-12-1422:00andy.fingerhutOn a related note, I watched the movie "Memento" for the first time recently. Sound advice.#2017-12-1422:00taylorare you sure it was the first time??#2017-12-1422:00gfredericksmore likely he will soon watch it for the last time#2017-12-1422:02andy.fingerhutIt felt like the first time, anyway#2017-12-1422:02andy.fingerhutIf only I can keep forgetting my favorite movies and books, I can get full enjoyment from them multiple times -- if I can remember which ones they are.#2017-12-1422:04andy.fingerhut@U3DAE8HMG LOLAHTEMLTMW#2017-12-1422:04taylorif ignorance is bliss then forgetfulness isā€¦ personal growth?#2017-12-1422:04andy.fingerhut(Laugh Out Loud And Had To Explain My Laughter To My Wife)#2017-12-1507:30andy.fingerhutAre there any handy examples someone can link to for writing a spec for a function, where one of its arguments is a predicate function, e.g. a spec for clojure.core/filter ?#2017-12-1507:43taylorthis example isnā€™t exactly taking a predicate but it demonstrates a HOF taking another fn https://taylorwood.github.io/2017/10/15/fspec.html#higher-order-functions#2017-12-1507:59andy.fingerhutThanks. Nicely written examples there.#2017-12-1507:56andy.fingerhutI've got the doc strings for seq? and seqable?, but I can't for the life of me think what the difference is between them, i.e. why would you pick one over the other in a spec?#2017-12-1508:05curlyfry@andy.fingerhut seqable? is for things that can be made into a seq (but aren't necessarily seqs themselves). For example, (seq? []) returns false, but (seqable? []) returns true.#2017-12-1508:08andy.fingerhutGot it. Thanks.#2017-12-1508:09andy.fingerhutseqable? seems far more often useful in :args specs, then.#2017-12-1508:14andy.fingerhutLooks like a spec like this probably needs a custom generator, if I see an error message like "Couldn't satisfy such-that predicate after 100 tries."? (s/def ::set/relation (s/and set? #(every? map? %)))#2017-12-1508:19tayloryeah with s/and I believe the generator is based off just the first spec#2017-12-1508:20taylorso, very unlikely it will generate sets with all maps#2017-12-1508:26andy.fingerhutThis page: https://clojure.github.io/test.check/generator-examples.html gives this example of a generator: (def sorted-vec (gen/fmap sort (gen/vector gen/int)))#2017-12-1508:26andy.fingerhutWhen I try to def sorted-vec in my REPL I get an assertion error "Arg to vector must be a generator"#2017-12-1508:28andy.fingerhutSorry, pilot error. I had my gen alias'ed to clojure.spec.gen.alpha, not clojure.test.check.generators as stated at the top of that examples page#2017-12-1508:44andy.fingerhutDoes a generator for predicate any? ever produce values that aren't equal to themselves in Clojure, e.g. Double/NaN ?#2017-12-1512:10gfrederickstest.check's gen/any sure can not sure what any? got wired up to#2017-12-1512:10gfredericksI feel like in both cases we need a secondary concept, like gen/any-sane#2017-12-1512:11gfredericksjust making NaN opt-in all the time seems like encouraging bad testing#2017-12-1512:30lopalghost@andy.fingerhut have you tried using coll-of to spec a set of maps?#2017-12-1512:30lopalghosteg (s/coll-of map? :kind set?)#2017-12-1512:31gfredericksgen/any-equatable#2017-12-1513:03andy.fingerhutgen/any-anumber#2017-12-1513:03andy.fingerhutbecause, you know, it is not, not a number#2017-12-1513:09gfredericksyeah but I'm talking about for something like gen/any where you are potentially generating large data structures and if there are any NaNs hiding anywhere in that giant tree then the whole thing becomes sometimes equal to itself but sometimes not#2017-12-1513:11andy.fingerhutright. gen/any-anumber was a bad joke šŸ™‚#2017-12-1513:12lopalghostreminds me of one of my favorite clojure expressions: > (number? ##NaN) > true#2017-12-1513:13andy.fingerhutI feel a JIRA coming on ...#2017-12-1513:14andy.fingerhut@lopalghost I have not tried coll-of, but thanks for the suggestion. Will do.#2017-12-1513:16andy.fingerhutI guess the generator for any? also sticks with things considered "values" in Clojure, i.e. it doesn't generate instances of java.util.Set ?#2017-12-1513:17andy.fingerhutthose also play havoc with one's usual ideas for clojure.core/= if they are inside of other sets, or keys of maps.#2017-12-1513:59gfredericksyeah, definitely#2017-12-1514:00gfredericksgenerating literally anything would be quite a tall order#2017-12-1521:28didibusIs there a way to override the generator of the args spec of an fspec? Using instrument and :gen I thought would be able to do it, but I can't figure how.#2017-12-1521:33Alex Miller (Clojure team)I don't think so, or at least it's not easy (and I think there is a ticket about this)#2017-12-1521:35didibusOk, thanks#2017-12-1609:40andy.fingerhutAre there any common practices on how detailed people like to make their specs, in terms of trying to specify precisely which :args should be allowed, vs. which should not? In particular, I've been looking at clojure.set/rename-keys and how some values of the second argument lead to hard-to-predict return values.#2017-12-1609:41andy.fingerhutExamples: (set/rename-keys {:a 1 šŸ˜› 2 :c 3} {:a šŸ˜› :c :b}) => {:b 3}#2017-12-1609:41andy.fingerhutbut: (set/rename-keys {:a 1 šŸ˜› 2 :c 3} {:c šŸ˜› :a :b}) => {:b 1}#2017-12-1609:42andy.fingerhutIf you see happy faces, those should be keywords with "b"#2017-12-1609:42andy.fingerhutI could write a spec simply saying that the second parameter is a map?, but that would allow maps like the above to be included, which have unpredictable behavior.#2017-12-1609:43andy.fingerhutI could also write a more restrictive spec for the 2nd arg that prevents any 2 values in the 2nd argument map from being equal to each other, which would avoid the unpredictable behavior.#2017-12-1609:45andy.fingerhutFor that same function, it is also clear that for generating random inputs that are "interesting", i.e. that don't simply always return their first argument with no changes, the keys in the first and second maps should often contain common values. So a useful generator that tries to exercise bugs in the implementation needs to steer inputs in that direction with high probability.#2017-12-1609:46andy.fingerhutAnyway, looking for thoughts and recommendations there.#2017-12-1610:13gklijs@andy.fingerhut I think a great deal comes down to how much effort you want to put into it. As sone as you start using very specific specs, you will also need to supply generators for those. While if you just want to have some edge cases tested, it would probably be less work to write specific tests for those. Depending on how you use specs, making them complex might also slow things down.#2017-12-1611:51guyyou can use ` to make it ignore emojiā€™s @andy.fingerhut#2017-12-1611:51guydo one ` then another after you have finished#2017-12-1611:51guyit should look like {:a 2 :b 3}#2017-12-1618:36bbrinckIā€™ve just released Expound 0.4.0. Now Expound will describe missing key specs https://gist.github.com/bhb/5abeff11f5252e915a07ea20acadd7fc#2017-12-1703:18jeaye@bbrinck Sweet. Nice work!#2017-12-1820:12donaldballIā€™m struggling to figure out a good way to express a spec category that has come up before, but I canā€™t recall what the best recommendations are any longer. The value is a map with namespace keyword keys, two of which each require values in their own sets, but further the pair of values must occur its own set. For example, :pet/type could be #{:cat :dog :snek} and :pet/fur? could be boolean?, but {:pet/type :snek :pet/fur? true} is not allowed. This is trivial with a predicate spec but then I lose any hope that s/explain can identify the paths to the keys contributing to the violation.#2017-12-1821:27Alex Miller (Clojure team)multispec?#2017-12-1821:35misha@donaldball yeah, doable with multispec, but verbose. Especially if you'd overload second value based on type of the first (different value sets for :pet/fur? depending on :pet/type#2017-12-1821:38mishaplus, you'd need to supply custom generators to avoid "could not satisfy predicate after # tries", when amount of valid value invariants is noticeably smaller than count of combinations in cartesian product of both value sets.#2017-12-1821:44donaldballI thought about multispec but I wasnā€™t even sure how I could tell spec ā€œhey use a smaller spec for this namespaced key in s/keys this timeā€ and couldnā€™t figure out a good way to express it using s/map-of šŸ™‚#2017-12-1821:44mishaspeaking of "verbose", how you, ladies and gentlemen, organize your specs, when there is fair amount of "intermediate" specs, which are not exactly useful in and of themselves, but are used to build larger composite specs. E.g. when "leaf" specs are "useful", and "root" spec is "useful" (for eyeballing), but "branch" specs in between are sort of just glue and are noisy. "Just suck it up" ā€“ is an acceptable answer, I guess opieop#2017-12-1821:45misha@donaldball you can "alias" specs#2017-12-1821:48misha
(s/def :cat/color #{:grey :white})
(s/def :dog/color #{:black :white})
(s/def :pet/color :dog/color)
#2017-12-1821:51mishathe thing is, when you want to overload a :pet/foo key value based on :pet/type value, you can get away with this in spec by using unqualified keywords in your entity maps {:foo ..., :type}. but if you want to overload value of a namespaced key ā€“ it will be a mess to spec it, if possible at all#2017-12-1821:57misha@donaldball you still have an option to return these from multispec
(s/keys :req [:pet/type :cat/fur?])  ;; for type=cat
(s/keys :req [:pet/type :snake/fur?]) ;; for type=snake
where
(s/def :pet/fur? boolean?)
(s/def :cat/fur? :pet/fur?)
(s/def :snake/fur? false?)
#2017-12-1822:34donaldballYeah, but all these options seem to require adjusting the form of my inputs. While the input form is not necessarily desirable, Iā€™d like to try to spec is it is, not as I have transformed it to be.#2017-12-1919:38falakI am trying to write specs for a function whose argument can be nil or a collection. However, I am unable to combine those two using s/or. This is what I tried -
(s/fdef my-func
            :args (s/cat :arg1 (s/or :nil nil? :collection list?)))
This definitely won't work but it'd be great to have the option -
..... (s/or nil? list?)
#2017-12-1919:42guycanā€™t you do (s/def ::arg1 (s/nilable list?)) then just use that with s/cat :arg1 ::arg1 ?#2017-12-1919:50falakAh! I wasn't aware of s/nilable. Thanks! However, I am not sure why but (s/nilable coll?) works and s/nilable list? doesn't when the data structure is actually a list of maps (or nil).#2017-12-1919:52falakWhen I run type on the args, it says clojure.lang.LazySeq#2017-12-1920:10seancorfield@fravani because list? is only true for things that implement IPersistentList -- which a lazy sequence does not.#2017-12-1920:11seancorfieldBut coll? is true for IPersistentCollection and LazySeq does implement that.#2017-12-1920:18falakThat makes sense. No wonder it worked when I converted the lazy seq to non-lazy using into () ... Thank you so much for the help.#2017-12-2021:10sparkofreasonIf I want to have a macro that will be able to process spec forms defined in ClojureScript, how do I get the forms in the macro definition? s/form will only get the form at run time, and I need it at compile time.#2017-12-2021:30mishawill you use it instead of s/def?#2017-12-2023:15sparkofreasonNo, after s/def. Basically trying to auto-generate destructuring forms based on the spec.#2017-12-2022:02agCan someone give me an insight to the issue I'm having with our project please. Because of the way how clojure.spec.alpha interacts with clojure.test.check, specifically I'm guessing this part https://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/gen/alpha.clj#L38 it breaks on attempt to use clojure.tools.namespace.repl/refresh, in a project where clojure.test.check.generators being used in .cljc file(s). And that's because Generator record gets redefined, resulting in two different classes. Here's a sample repo that repros that bug (created by my colleague) https://github.com/xiongtx/reload-error-boot. I thought the easy way of fixing it is to stop using clojure.test.check and just use clojure.spec.gen.alpha everywhere. But that doesn't seem to include bunch of things: macros like gen/let and gen/bind, gen/return, gen/pos-int, etc. Basically not using it at all - seems not possible (?) Because of that, CIDER's refresh and clojure.tools.namespace.repl/refrsh are broken, that makes it sort of painful to work with. Can someone suggest a workaround?#2017-12-2022:03gfrederickscan you use refresh-all instead?#2017-12-2022:03gfrederickswait why does Generator get redefined at all?#2017-12-2022:08gfredericksrefresh shouldn't be reloading library code#2017-12-2101:00agcan you please take a look at https://github.com/xiongtx/reload-error-boot there's something strange going on, I think this deserves to be a JIRA ticket (unless someone already filed one) I can't formulate though exactly what's happening#2017-12-2101:12gfredericksat a glance, my guess is that cider is more enthusiastic about reloading things than tools.namespace is? certainly seems like bad behavior. I don't think tools should reload libraries under normal circumstances. I've never seen tools.namespace do that.#2017-12-2101:19agit happens also with tools.namespace#2017-12-2101:20agit's not bug in CIDER#2017-12-2101:20agit's because of this https://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/gen/alpha.clj#L38#2017-12-2112:06gfredericksyou're saying the lack of a standard require makes tools.namespace act as if t.c.generators was a project namespace?#2017-12-2116:07agĀÆ\(惄)/ĀÆ#2017-12-2116:07agit seems failing only for .cljc files#2017-12-2116:08agwith shared generators #2017-12-2117:23gfredericksdoes tools.namespace explicitly announce that it's reloading the generators namespace?#2017-12-2117:23gfredericks(it usually prints which things it's reloading)#2017-12-2118:02agthis is in our project, when you do (require '[clojure.tools.namespace.repl :refer [refresh]]) (refresh)
:reloading ... then list of namespaces
:error-while-loading finops-admin.specs.shared
#error {
 :cause "Assert failed: Second arg to such-that must be a generator\n(generator? gen)"
 :via
 [{:type clojure.lang.Compiler$CompilerException
   :message "java.lang.AssertionError: Assert failed: Second arg to such-that must be a generator\n(generator? gen), compiling:(shared.cljc:72:34)"
   :at [clojure.lang.Compiler$InvokeExpr eval "Compiler.java" 3700]}
  {:type java.lang.AssertionError
   :message "Assert failed: Second arg to such-that must be a generator\n(generator? gen)"
   :at [clojure.test.check.generators$such_that invokeStatic "generators.cljc" 346]}]
 :trace
and long stacktrace
#2017-12-2118:02agnote, there's nothing wrong with that generator in finops-admin.specs.shared#2017-12-2118:17agthis is a bug that appears in the .cljc code that's trying to use clojure.test.check.generators#2017-12-2118:17agif you read the readme in https://github.com/xiongtx/reload-error-boot you'll see detailed explanation#2017-12-2119:13gfredericksBut does the list of namespaces you elided contain test.check?#2017-12-2119:34agyup it does#2017-12-2119:49gfredericksokay, that is what seems buggy to me#2017-12-2121:08agposted here https://www.reddit.com/r/Clojure/comments/7lbplj/clojuretoolsnamespacereplrefresh_tumbles_on_cljc/#2017-12-2121:22xiongtxSeems that this is a known issue w/ tools.namespace. Thanks to @U09LZR36F for pointing this out! https://dev.clojure.org/jira/browse/TNS-45?actionOrder=desc#issue-tabs#2017-12-2100:15sparkofreasonFound an answer, albeit hacky-feeling, to getting compile-time spec forms for CLJS. At compile time, the CLJS form of the spec is stored in cljs.spec.alpha/registry-ref, an atom containing a map keyed by spec-name. This works, but feels like I'm coupling to internal implementation details, so if there's a better way, I'd be thankful.#2017-12-2100:15johanatan@lopalghost ah, yea, sorry re: the s/cat: it apparently is redundant/ unnecessary when the spec itself is already a tuple#2017-12-2100:15johanatanand yea, generating the entire tuple at once is what i'm doing in fact#2017-12-2100:15johanatanwas just hoping there'd be a better way because that implies that i need wrapper functions that take the untupled inputs and call the tupled ones#2017-12-2100:16johanatani'm hitting a weird error where spec itself is trying to construct an ExceptionInfo in a haram way#2017-12-2100:16johanatanthrowing this exception: https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/ExceptionInfo.java#L31#2017-12-2100:17johanatanhere's the full stack:
[[clojure.lang.ExceptionInfo <init> "ExceptionInfo.java" 31]
   [clojure.lang.ExceptionInfo <init> "ExceptionInfo.java" 22]
   [clojure.core$ex_info invokeStatic "core.clj" 4739]
   [clojure.core$ex_info invoke "core.clj" 4739]
   [clojure.spec.test.alpha$explain_check invokeStatic "alpha.clj" 277]
   [clojure.spec.test.alpha$explain_check invoke "alpha.clj" 275]
   [clojure.spec.test.alpha$check_call invokeStatic "alpha.clj" 295]
   [clojure.spec.test.alpha$check_call invoke "alpha.clj" 285]
   [clojure.spec.test.alpha$quick_check$fn__2986 invoke "alpha.clj" 308]
   [clojure.lang.AFn applyToHelper "AFn.java" 154]
   [clojure.lang.AFn applyTo "AFn.java" 144]
   [clojure.core$apply invokeStatic "core.clj" 657]
   [clojure.core$apply invoke "core.clj" 652]
   [clojure.test.check.properties$apply_gen$fn__16139$fn__16140 invoke "properties.cljc" 30]
   [clojure.test.check.properties$apply_gen$fn__16139 invoke "properties.cljc" 29]
   [clojure.test.check.rose_tree$fmap invokeStatic "rose_tree.cljc" 77]
   [clojure.test.check.rose_tree$fmap invoke "rose_tree.cljc" 73]
   [clojure.test.check.generators$fmap$fn__9199 invoke "generators.cljc" 101]
   [clojure.test.check.generators$gen_fmap$fn__9173 invoke "generators.cljc" 57]
   [clojure.test.check.generators$call_gen invokeStatic "generators.cljc" 41]
   [clojure.test.check.generators$call_gen invoke "generators.cljc" 37]
   [clojure.test.check$quick_check invokeStatic "check.cljc" 94]
   [clojure.test.check$quick_check doInvoke "check.cljc" 37]
   [clojure.lang.RestFn invoke "RestFn.java" 425]
   [clojure.lang.AFn applyToHelper "AFn.java" 156]
   [clojure.lang.RestFn applyTo "RestFn.java" 132]
   [clojure.core$apply invokeStatic "core.clj" 657]
   [clojure.core$apply invoke "core.clj" 652]
   [clojure.spec.gen.alpha$quick_check invokeStatic "alpha.clj" 29]
   [clojure.spec.gen.alpha$quick_check doInvoke "alpha.clj" 27]
   [clojure.lang.RestFn applyTo "RestFn.java" 137]
   [clojure.core$apply invokeStatic "core.clj" 661]
   [clojure.core$apply invoke "core.clj" 652]
   [clojure.spec.test.alpha$quick_check invokeStatic "alpha.clj" 309]
   [clojure.spec.test.alpha$quick_check invoke "alpha.clj" 302]
   [clojure.spec.test.alpha$check_1 invokeStatic "alpha.clj" 335]
   [clojure.spec.test.alpha$check_1 invoke "alpha.clj" 323]
   [clojure.spec.test.alpha$check$fn__3005 invoke "alpha.clj" 411]
   [clojure.core$pmap$fn__8105$fn__8106 invoke "core.clj" 6942]
   [clojure.core$binding_conveyor_fn$fn__5476 invoke "core.clj" 2022]
   [clojure.lang.AFn call "AFn.java" 18]
   [java.util.concurrent.FutureTask run "FutureTask.java" 266]
   [java.util.concurrent.ThreadPoolExecutor runWorker "ThreadPoolExecutor.java" 1149]
   [java.util.concurrent.ThreadPoolExecutor$Worker run "ThreadPoolExecutor.java" 624]
   [java.lang.Thread run "Thread.java" 748]]
#2017-12-2100:20johanatanseems like this may be a bug: https://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/test/alpha.clj#L279#2017-12-2100:20johanatanif when-not returns nil, ExceptionInfo doesn't like that#2017-12-2100:26johanatan(apply ex-info (remove nil? ["Specification-based check failed" (when-not ...)])) would seem to be a fix#2017-12-2100:27johanatanalthough that's going to seriously limit the person on the receiving end's ability to debug šŸ™‚#2017-12-2100:27johanatanso there is likely a higher-level explanation for why we got here#2017-12-2100:27johanatanwhich might be good to pass on to the user#2017-12-2107:24misha@dave.dixon you can call any defined function inside macro as part of the data transformation, canā€™t you? Which happens at compile time. It is the form macro returns who is evaluated at run time#2017-12-2107:25mishaDid you try to use s/form inside macro, but outside of quoted return value?#2017-12-2111:29trisshey all - how do I turn something like (clojure.spec.alpha/coll-of clojure.core/number?) back in to spec I can use in s/valid?. Iā€™m deconstructing my specs and using bits of them elsewhere.#2017-12-2111:42guyCould you just do (s/def ::your-spec (s/coll-of number?)) Then you can do (s/valid? ::your-spec ..)#2017-12-2111:42guyOr canā€™t you just use it in s/valid?#2017-12-2111:42guy(s/valid? (clojure.spec.alpha/coll-of clojure.core/number?) ā€¦)#2017-12-2111:48trissthanks guyā€¦ Iā€™ll check that in a momentā€¦. I just realised a quick call to eval does the job for me.#2017-12-2111:49guykk!#2017-12-2114:18sparkofreason@misha Yes. That doesn't find the spec. Looking at the CLojureScript spec code, the form function is only defined at run-time, and basically just wraps the specize* method on the Specize protocol. The reified instance of Specize wouldn't be available at compile-time anyway. The registry-ref atom defined in cljs/spec/alpha.cljc appears to be the mechanism by which the s/def macro communicates between compile and run-time. I don't see any function which would abstract access to registry-ref for use from macros.#2017-12-2121:14agpossible to have a "parametrized spec", where you can specify generator parameters? e.g.: making the following more flexible, by letting it generate dates in specified range (instead of hardcoded values):
(s/with-gen (s/int-in (inst-ms #inst "1980-01-02")
                                         (inst-ms #inst "2050-12-31"))
                     #(gen/choose
                       (inst-ms #inst "2015-01-01")
                       (inst-ms #inst "2016-12-31"))) 
?
#2017-12-2121:15Alex Miller (Clojure team)no, other than via a macro wrapping this#2017-12-2121:16Alex Miller (Clojure team)there is s/inst-in which does ranges?#2017-12-2121:16Alex Miller (Clojure team)not sure if that would serve your needs#2017-12-2121:17agalright... thanks @alexmiller#2017-12-2121:56seancorfield@ag Yeah, that's been a bugbear for us at World Singles since we have to track a constantly moving time window for certain valid time ranges.#2017-12-2122:11agso this (gen/sample (s/gen (s/inst-in #inst "2017-10-01" #inst "2018-12-31"))) returns bunch of dates, but they are mostly are in 2017-10, what gives?#2017-12-2122:12shaun-mahood@ag: There was a great talk by @gfredericks at this years conj about generators that had a good section on better generation of datetimes (the most relevant part starts at about 30 mins in if you want to skip right to it) - https://www.youtube.com/watch?v=F4VZPxLZUdA#2017-12-2122:12seancorfield@ag generators for ranges tend to start at the beginning and use small increments at first. Yup, what @shaun-mahood said!#2017-12-2122:13seancorfieldThat talk was super helpful for us at World Singles!#2017-12-2122:13agwe've been also using test.chuck I think it's @gfredericks project, right?#2017-12-2122:19shaun-mahoodYep - I haven't tried it yet but more than once someone has pointed me to it to answer a question about doing something more complicated. I guess I should try it šŸ™‚#2017-12-2122:21seancorfieldWe use it for regex generators. Wonderful!#2017-12-2123:08misha@ag you can provide map of override generators on spec "call" site#2017-12-2123:10mishahttps://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/alpha.clj#L272-L281#2017-12-2123:10misha
...Optionally an overrides map can be provided which
  should map spec names or paths (vectors of keywords) to no-arg
  generator-creating fns. These will be used instead of the generators at those
  names/paths. Note that parent generator (in the spec or overrides
  map) will supersede those of any subtrees...
#2017-12-2123:14mishathis way you can sort of dynamically inject generators built specifically for the current circumstances. But I yet to see (or spend time building myself) complex enough dependent generator "trees", so can't point to any caveats or best practices yet#2017-12-2123:15misha(e.g. imagine ETL pipeline: you generate random input file, generate ETL config based on generated input file, and then property-test extracted result against input)#2017-12-2123:18mishamore detailed your specs become ā€“ more branchy and fragile (I guess?) your generators become. And managing that is noticeable overhead in and of itself. Throw in proper shrinking capabilities support, and random seed honoring, and it is suddenly a full time job opieop#2017-12-2123:59cflemingJust to double check my understanding here - if Iā€™m wanting to use spec for macro grammars in a library which also needs to support clients on older versions of Clojure, I can just have the specs and the fdefs in a separate namespace which is loaded dynamically somehow if Clojure 1.9+ is detected, is that right?#2017-12-2123:59cflemingOr perhaps users of the library can just require that namespace if theyā€™re using Clojure 1.9.#2017-12-2200:03bfabryseems to be exactly what clojure.java.jdbc does#2017-12-2200:03cflemingThanks, Iā€™ll check it out.#2017-12-2200:03cflemingin fact, the specs could even be in their own library I imagine.#2017-12-2200:04bfabrywell if it works for core šŸ™‚#2017-12-2200:04cflemingYeah, but core is special šŸ™‚#2017-12-2200:06cflemingOk, great - looks like that will work fine - thanks!#2017-12-2200:54bbloomwhatā€™s the idiom for negating a spec? ie if i want to greedly parse excluding something#2017-12-2200:55bbloomjust use #(not ā€¦) ?#2017-12-2200:58gfrederickscomplement?#2017-12-2200:58bbloomyeah or that#2017-12-2200:59bbloomjust wondering if there was any negative match regexes or anything like that#2017-12-2201:05bbloomi guess iā€™m struggling with a non-greedy (reluctant?) s/* pattern#2017-12-2201:08andy.fingerhutHas anyone attempted to write code that determines the 'difference' between two specs yet? That this is possible to do for "vanilla" regular expressions was, I recall, mentioned as a motivating factor for spec to be include regex matching. (I realize, of course, that spec generalizes vanilla regex's in a way, with extra side conditions via and/or, such that it might not always be possible to apply those regex diff algorithms in those cases).#2017-12-2201:08andy.fingerhuts/be include/be based on/#2017-12-2201:11Alex Miller (Clojure team)I donā€™t think itā€™s going to be efficient to do so without leveraging the guts of the regex spec impl#2017-12-2202:48cflemingIs it considered idiomatic to name things that I donā€™t care about capturing :_?#2017-12-2209:46misha@cfleming I've seen it used. I prefer :_actual-name-of-thing-you-ignore both for keywords and symbols (documentations is hard to come by these days)#2017-12-2209:51cfleming@misha Thanks. This is for constant symbols and keywords in macro grammars, I had things like (s/cat <whatever> :from #{:from} <whatever>) which seems a bit redundant.#2017-12-2209:51cfleming(i.e. when capturing a constant :from)#2017-12-2210:03mishaof course there are things which are impractical to document (in any way)#2017-12-2216:00trissis it possible to look up the spec for a function?#2017-12-2216:18arohner(s/form (s/spec 'foo/bar))#2017-12-2216:22athosYou probably mean s/get-spec rather than s/spec ?#2017-12-2221:15curlyfry@triss clojure.repl/doc will show you any specs for a function#2017-12-2222:36pablorehello, is there any way to define a spec for a sorted collection?#2017-12-2311:34stathissiderisI gave a talk on Spec and what you can do with it for the Athens Clojure Meetup, itā€™s in Greek with English captions: https://www.youtube.com/watch?v=T1qpIaB6_vM#2017-12-2320:02bbrinck@stathissideris your talk was excellent. I particularly liked the descriptions and diagrams of more advanced generative-testing techniques. Good stuff.#2017-12-2320:06stathissideris@bbrinck thanks a lot, Iā€™m glad you enjoyed it! :)#2017-12-2408:08misha@pablore yes, expensive one would be something like
(s/def ::sorted-coll
  (s/and
    (s/coll-of any?)
    #(= % (sort %))))
#2017-12-2408:11mishabasically, write a predicate function, which will answer your "is it sorted?" question with true or false.#2017-12-2408:11rauh#(apply <= %) is probably faster.#2017-12-2408:11misha(the one above changes collection type, and will not work most of the time)#2017-12-2408:17mishayeah, ~400times faster opieop the point is ā€“ basic spec is just a predicate function#2017-12-2414:27Alex Miller (Clojure team)just use sorted?#2017-12-2414:27Alex Miller (Clojure team)if you are checking whether itā€™s a sorted collection or not#2017-12-2416:28tayloris there a predicate for checking if something is a lazy seq#2017-12-2416:37taylordisregard, I really just want to convert all sequences to vectors#2017-12-2523:53flyboarderHey guys, trying to figure out how I can serialize a clojure spec exception. java.io.NotSerializableException: clojure.spec.alpha$regex_spec_impl$reify__2436#2017-12-2607:45gklijs@flyboarder The Class java.lang.Exception extends Throwable implements Serializable, to get it to work you probably need to create your own exception which either extend from the java exception, or implement Serializable, thatā€™s how far I can help you with my java background, never did anything yet in clojure for exception handling, so I donā€™t know how itā€™s different from java.#2017-12-2609:11stathissideris@flyboarder looks like the serialization breaks for the spec itself which I believe is included in the ex-data of the ex-info, under the key :spec#2017-12-2609:11stathissideriscould you dissoc it as a quick test and try serializing again?#2017-12-2619:58flyboarder@stathissideris that seems to work, I need to remove the entire namespaced key :clojure.spec.alpha/spec is there a way to future proof the removal for when the namespace changes?#2017-12-2713:23Alex Miller (Clojure team)Alias clojure.spec.alpha to s in your namespace definition, then use ::s/spec. That way, you will just need to change the namespace in one spot.#2017-12-2718:54flyboarder@U064X3EF3 thanks!#2017-12-2619:59flyboarder@gklijs I needed to wrap the exception in my own ex-info while removing the non-serializable parts, thanks!#2017-12-2704:40shdzzlI'm trying to work out what is happening with the following snippet, I think I must be misunderstanding how s/cat works:#2017-12-2704:43shdzzlIt makes no sense to me that it would be matching the first vector element on ::bound, so I'm not sure how that's the spec it's failing.#2017-12-2704:43flyboarder@shdzzl I had issues wrapping my head around s/cat also, it kinda explodes the argument, is the best way I can describe it#2017-12-2704:47shdzzlI'm not sure I follow, it explodes which argument? I visualize it as kind of zipping predicates to a sequence, so my s/cat example should match any seq of length three and check the first element against ::needs, the second against ::bound and the third against ::body. What am I missing?#2017-12-2704:48flyboardertry (s/and vector? (s/cat .....))#2017-12-2704:49shdzzlSame result.#2017-12-2704:50flyboarderLoading up a repl, one sec#2017-12-2704:50shdzzlThanks.#2017-12-2705:10flyboarder@shdzzl well to start with your :needs spec fails#2017-12-2705:11flyboarderthats the real issue#2017-12-2705:11flyboarder(s/def ::needs (s/coll-of symbol?))#2017-12-2705:12flyboarder
boot.user=> (s/explain (s/cat :needs ::needs :bound ::bound :body ::body) [['a 'b] {} '(+ a b)])
Success!
#2017-12-2705:15shdzzlUhuh, that does fix it. ::needs wasn't failing when tested on it's own. Is there some flattening or something going on? I'm going to have read over s/* again. Thanks a ton.#2017-12-2705:16flyboarder@shdzzl thats what s/cat is doing (s/cat :needs (s/* symbol?)) is different than when you move that into itā€™s own spec#2017-12-2705:17shdzzlOoohhh! When written like that it's more obvious.#2017-12-2705:18flyboarderrule of thumb for (s/cat) build it up one arg at a time#2017-12-2705:18flyboarderotherwise itā€™s hard to figure out where it fails#2017-12-2705:19shdzzlGood advice.#2017-12-2705:29shdzzlWhile I'm here, I had another question. Is it possible/advisable to spec constructors (`MyRecord.` for example)?#2017-12-2705:35shdzzlIt touches on it in the spec guide, but I was curious if someone had tried it with fdef or similar.#2017-12-2705:37johanatan
user> (time (stest/check `a-symbol))
"Elapsed time: 2.153437 msecs"
does anyone have an explanation for why the above completes almost instantly but my repl does hang during the 10s of seconds that it actually takes to check the symbol in question?
#2017-12-2705:40tbaldridge@johanatan what does that code print?#2017-12-2705:40johanatanwhich code? it just prints the elapsed time line, then hangs until the check actually completes#2017-12-2705:40johanatanthen i get my REPL prompt back to execute more lines#2017-12-2705:40tbaldridgeand what does it return when it completes?#2017-12-2705:42johanatanlet me check. didn't wait for it to complete before because i was trying to debug why a loop/recur was spinning constantly instead of waiting for check to complete#2017-12-2705:45johanatan@tbaldridge it returned the typical map result from check#2017-12-2705:46johanatan
({:spec #object[clojure.spec.alpha$fspec_impl$reify__2451 0x6d92492f "
#2017-12-2705:48tbaldridgeinteresting#2017-12-2705:48tbaldridgeSo one last thing to try:#2017-12-2705:49tbaldridge(do (time (stest/check a-symbol)) nil)`#2017-12-2705:49tbaldridgebleh you get the point, wrap it in a do, nil. that will remove any questions about REPL printing taking a long time#2017-12-2705:49tbaldridgewhich is normally what it is in cases like this#2017-12-2705:50johanatanhmm, that actually returned immediately like before but also gave me back control at the REPL prompt (i'm assuming the check is happening in the bkg and i do have a process called main doing a lot of CPU work in my procmon)#2017-12-2705:51johanatan[i think main is the cider repl server if i'm not mistaken-- last time i killed it, cider immediately printed out a bunch of stuff indicating unhappiness lol]#2017-12-2706:04johanatanah#2017-12-2706:04johanatanfound it#2017-12-2706:04johanatancheck returns a lazyseq#2017-12-2706:04johanatanso wrapping it in doall seems to have done the trick#2017-12-2706:05johanatan[i suppose the result wasn't being dropped on the floor previously because the repl itself was waiting for the value to be returned so that it could print it]#2017-12-2706:19johanatanthanks for the help!#2017-12-2706:19flyboarder@shdzzl sorry I havnā€™t tried that one#2017-12-2706:21shdzzlNo problem, I'm going to look into it when I get the time.#2017-12-2814:57romniis there a way to spec a map using a coll of types/keywords (instead of a single type/keyword as with multi-spec)? the idea is that all specs corresponding to matched types are merged and used to validate the map#2017-12-2815:02romniwhat i have is { ::types [:first :second] } and what i'm looking for is a way to create something like a multi-spec which dispatches on the values in the coll corresponding to ::types and which will build a spec like (s/merge ::first ::second)#2017-12-2913:27souenzzo(defmulti types ::types) (defmethod types :default [{::keys[types]}]) (eval (apply s/merge types)))` Some lke this?#2017-12-2815:12ikitommiHi. I have a vector of s/keys specs and I would like to merge them, without eval. As s/merge is a macro, I guess I should use merge-spec-impl?#2017-12-2819:53ikitommiIā€™m trying to combine stuff with s/merge. Is this intentional or a bug?
(s/explain any? {:x 1, :y 2})
; Success!

(s/explain (s/merge any?) {:x 1, :y 2})
; val: {:x 1, :y 2} fails predicate: any?
#2017-12-2819:55ikitommiok, this works thou:
(s/explain (s/merge (s/spec any?)) {:x 1, :y 2})
; Success!
#2017-12-2820:04ikitommiBut as the s/spec is macro too, with my arbitrary vector-of-specs input, I would have to call s/spec-impl instead to avoid eval and figure out the forms somehow. Any news on the new functional core for clojure.spec?#2017-12-2913:31souenzzoI think that this is a bug @ikitommi#2017-12-2914:34ikitommithanks @souenzzo, wrote CLJ-2302. As a workaround, Iā€™m forcing all the incoming spec to be Specs (or names of Specs):
(defn merge-specs [specs]
  (when-let [non-specs (seq (remove #(or (s/spec? %) (s/get-spec %)) specs))]
    (throw
      (ex-info
        (str "Not all specs satisfy the Spec protocol: " non-specs)
        {:specs specs
         :non-specs non-specs})))
  (s/merge-spec-impl (vec specs) (vec specs) nil))
#2018-12-3110:09ikitommiHi. Is there or is someone working with a spec error reporter that could hint on misspelled keys or non-defined (directly on s/keys) map keys?#2018-12-3118:16taylorsomething like this? https://gist.github.com/stuarthalloway/f4c4297d344651c99827769e1c3d34e9#2018-12-3110:11ikitommiI have a s/keys spec for configuration with all the keys optional. Normal s/keys is not helping much here.#2018-12-3110:15ikitommispec-tools has the utilities for pulling out all the keys & merges and could built on that, but would rather use something ready for this.#2018-12-3111:21dominicm@ikitommi figwheel has one#2018-12-3111:21dominicm@ikitommi strictly-specking-standalone#2018-01-0115:34ikitommiThanks @dominicm for the pointer. Looks great, seems to lean on custom strict-keys Spec to catch the typos. Coudnā€™t get it working as it uses really old version of spec, but will poke around to see if I could get it working. Different artifact name btw in Clojars and in Github.#2018-01-0119:12drcodeHi everyone- I have a clojure DSL that comes in two variants that are about 95% the same and I want to write a single spec that validates both variants- What is the best way of doing this? Basically what I want to do is write about a hundred clojure.spec/def declarations that are the same for both variants, but a few deeply nested items will need idiosyncratic logic, looking something like:
(s/def :deeply/nested-item
     (if variant1? 
          variant-1-specification
          variant-2-specification))
I know I can solve this through some nasty macrology (that maintains two separate namespaces of specs for the two variants, so that the idiosyncratic differences can be "spliced in") but usually it is a bad idea to lean heavily on complex macros... Thanks for any tips!
#2018-01-0119:43gfredericksaaron brooks was just talking to me on twitter about this sort of thing#2018-01-0119:43gfrederickshttps://twitter.com/0x1B/status/946499269279846400#2018-01-0119:44gfrederickspaging @abrooks#2018-01-0120:45drcode@gfredericks yep, that's my situation...#2018-01-0120:49drcodeyes, I think I'll just use @abrooks' cheat until "spec specs" are available- Thanks for the pointer!#2018-01-0215:37pabloreHow would you spec this function?
(defn reducer
  ([] {})
  ([state action]
    (...)))
Where state and action are specced
#2018-01-0215:38taylor
(defn such-arity
  ([] "nullary")
  ([one] "unary")
  ([one two & many] "one two many"))

(s/fdef such-arity
        :args (s/alt :nullary (s/cat)
                     :unary (s/cat :one any?)
                     :variadic (s/cat :one any?
                                      :two any?
                                      :many (s/* any?))))
#2018-01-0215:43pablorethanks#2018-01-0215:46pabloreI thought that could solve my validation problem with my function, but it still failing :c#2018-01-0215:46pablore
Couldn't satisfy such-that predicate after 100 tries. {}
#2018-01-0216:00taylorthat could be due to one of your specs using an s/and and not being able to generate something that conforms to the whole spec#2018-01-0216:02taylor
(s/def ::foo-map (s/and map? #(:foo %)))
(gen/sample (s/gen ::foo-map))
ExceptionInfo Couldn't satisfy such-that predicate after 100 tries.  clojure.core/ex-info (core.clj:4739)
#2018-01-0216:03taylorfor example ā˜ļø that s/and is generating based off that first map? predicate, but the generator is unlikely to produce a map with :foo and so it gives up after so many tries#2018-01-0216:29pabloreCannot fix it for now. Will put it on hold until errors are more specific#2018-01-0216:29pablorethanks anyway for the info#2018-01-0402:34cflemingIs there a way to make the spec optional and star operators greedy?#2018-01-0402:34cflemingI have the following spec:
(s/cat :binding (s/? (s/cat :binding-var ::variable-name
                            :sep #{'<-}))
       :fact-type any?
       :destructured-fact (s/? ::destructured)
       :expressions (s/* any?))
#2018-01-0402:35cflemingThis is horribly ambiguous as written, due to the use of any? and repeated/optional elements.#2018-01-0402:35cflemingIā€™d like to make the first s/? greedy, so to say if itā€™s possible to match the :binding part then thatā€™s what should happen.#2018-01-0402:36cflemingThe only alternative I can think of is to do this, taking advantage of ordering precedence for alternates:
(s/or
  :with-binding (s/cat :binding-var ::variable-name
                       :sep #{'<-}
                       :fact-type any?
                       :destructured-fact (s/? ::destructured)
                       :expressions (s/* any?))
  :no-binding (s/cat :fact-type any?
                     :destructured-fact (s/? ::destructured)
                     :expressions (s/* any?)))
#2018-01-0402:36cflemingBut thatā€™s kind of ugly.#2018-01-0402:38cflemingIs there any defined precedence of optional elements inside an s/cat?
#2018-01-0404:11Alex Miller (Clojure team)In short, no. The way the regex derivative stuff works is kind of doing all the options in parallel#2018-01-0502:56cflemingFWIW other regex parallel implementations allow this even though they work in lockstep with no backtracking.#2018-01-0502:56cfleminge.g. NFAs or VM approaches#2018-01-0503:23cflemingI think it would be worth specifying in the doc if ops like ? and * are greedy or not - itā€™s not mentioned right now and is useful information in ambiguous cases.#2018-01-0404:18Alex Miller (Clojure team)I donā€™t know that I understand what youā€™re trying to spec well enough but the latter seems more readable to me#2018-01-0404:20Alex Miller (Clojure team)One thing Iā€™ve found is that specā€™ing a dsl is a pretty good way to judge how good your dsl is :)#2018-01-0406:17cflemingThis is for Clara Rules#2018-01-0406:17cflemingSo I canā€™t easily modify how it works.#2018-01-0413:00mikerod> One thing Iā€™ve found is that specā€™ing a dsl is a pretty good way to judge how good your dsl is :) Yeah, in the case of Clara, there is a bit too much ambiguity in this case Iā€™d say. Have to figure out a way to tighten it down a bit perhaps. Spec to the rescue.#2018-01-0416:34bbrinckI seem to recall that someone published a repository with some specs for core functions (beyond what is currently included in https://github.com/clojure/core.specs.alpha), but now I canā€™t find it. Anyone else remember this?#2018-01-0416:35bbrinckI realize that these specs may be incomplete or buggy, and thatā€™s OK for my use case. This is just for toying around with instrumentation.#2018-01-0416:37bbrinckOh, nevermind, my fourth google search finally found it https://github.com/leifp/spec-play#2018-01-0416:38andre.stylianosThere's also this one: https://github.com/tirkarthi/respec Don't know how it compares to the one you mentioned#2018-01-0416:39bbrinck@andre.stylianos Awesome, thanks!#2018-01-0419:21Alex Miller (Clojure team)the repos Iā€™ve looked at were both mostly wrong and not very good examples of specs, so ymmv#2018-01-0419:23bbrinck@alexmiller Good to know. Iā€™m not that concerned about accuracy for this case, but if they are not well-written specs, thatā€™s more of an issue, since Iā€™m using these as a temporary proxy for the real clojure.core specs#2018-01-0419:24bbrinckI should clarify: Iā€™m not worried about completeness in the sense that if, say, some functions are missing specs for certain less common arities, thatā€™s OK. If the specs are just buggy in general then that would cause an issue.#2018-01-0419:26Alex Miller (Clojure team)from what Iā€™ve looked at, the specs are wrong, as in will both allow bad things and reject good things#2018-01-0419:27Alex Miller (Clojure team)and most of them spec a bunch of things that instrument wonā€™t work with (inline functions and functions with primitive args), which is not bad, but also not useful#2018-01-0419:27bbrinckGotcha. Yeah, for my case, rejecting good things will cause issues. Iā€™ll take a look.#2018-01-0419:29bbrinck> bunch of things that instrument wonā€™t work with (inline functions and functions with primitive args) Iā€™m afraid I donā€™t understand. Can you expand on this a little?#2018-01-0419:35seancorfield@bbrinck see https://dev.clojure.org/jira/browse/CLJ-1941#2018-01-0419:39Alex Miller (Clojure team)instrument currently doesnā€™t work with: functions taking/returning primitives (fixable), inlines, multimethods (fixable), and protocols#2018-01-0419:39Alex Miller (Clojure team)so specā€™ing + will buy you nothing for example#2018-01-0419:40Alex Miller (Clojure team)but itā€™s not harmful and does get included in docs, which might be valuable in and of itself#2018-01-0420:16bbrinck@seancorfield @alexmiller Ah, thanks. I did not know about that. If this works, Iā€™ll be using instrumentation in CLJS, so presumably the primitive issue will be different#2018-01-0420:26Alex Miller (Clojure team)Oh yeah, Iā€™m talking Clojure#2018-01-0420:29bbrinckWell, I didnā€™t mention the CLJS part before šŸ™‚ and in any case, those specs are in CLJ, so I can see why you thought CLJ (Iā€™ll need to copy them into a cljs file if I want to use them).#2018-01-0422:05kennyWhy is this valid?
(s/def ::cat (s/cat :a keyword? :b any?))
(s/def ::foo (s/coll-of (s/spec ::cat)))
(s/valid? ::foo {:a "a"})
=> true
#2018-01-0422:10noprompt@kenny because
(coll? {})
;; => true
#2018-01-0422:10cfleming@kenny I think because a map is a coll. Presumably s/coll-of calls seq on the map to get at the elements, which in this case are entries. Then s/cat calls seq and gets the entry key and val.#2018-01-0422:10kennyShit, that always gets me.#2018-01-0422:10nopromptšŸ˜„#2018-01-0422:11kennyThanks guys.#2018-01-0500:34andy.fingerhutI am guessing that if some function A included in Clojure calls function B, and I instrument B, then because of direct linking within default Clojure JAR, A will still call un-instremented version of B? And building my own Clojure with direct linking disabled is probably best way to enable A to call instrumented version of B?#2018-01-0500:36Alex Miller (Clojure team)That is all correct#2018-01-0500:36andy.fingerhutMakes sense. Thx#2018-01-0500:37Alex Miller (Clojure team)There is a build property you can set in the Clojure build (we run the tests both with and without on CI)#2018-01-0500:37Alex Miller (Clojure team)-Ddirectlinking=false#2018-01-0500:52andy.fingerhutI must be doing it wrong. I've tried "mvn -Ddirectlinking=false clean package install", and even edited build.xml to change directlinking to false in there, but instrument'd B isn't causing A-to-B call to fail as I think it should. I'll keep experimenting to see where I'm going wrong.#2018-01-0500:58bronsa@andy.fingerhut beware of inlining maybe#2018-01-0500:59andy.fingerhutA is clojure.data/diff, or actually a function/protocol implementation that it calls, that calls B = clojure.set/union, where clojure.set/union is the function I'm trying to instrument.#2018-01-0501:01andy.fingerhutUgh. I think I forgot that build.xml only affect ant builds, not mvn builds.#2018-01-0501:11seancorfieldIt's -Dclojure.compiler.direct-linking=true#2018-01-0501:12seancorfield(or false) ^ @andy.fingerhut#2018-01-0501:13seancorfield(or at least that's what we have in our script for production processes -- and we assume it's false by default elsewhere)#2018-01-0501:16andy.fingerhutI believe that flag is what tells the Clojure compiler whether to direct link user Clojure programs, or not. It doesn't seem to affect whether Clojure core code is compiled with direct linking (i.e. it still seems to have direct linking between my A and B functions that are both within Clojure itself)#2018-01-0501:16andy.fingerhutI can avoid direct linking within Clojure itself by building it from source after changing true to false for directlinking inside of pom.xml#2018-01-0501:16andy.fingerhutI haven't found the right incantation to make that happen from the command line, without editing pom.xml#2018-01-0501:16seancorfieldAh, OK, misread what you were attempting then. Sorry.#2018-01-0501:18seancorfieldGiven that it's hardcoded in pom.xml https://github.com/clojure/clojure/blob/master/pom.xml#L37 I would not expect it could be overridden at the command-line (but my mvn-fu is weak).#2018-01-0501:21Alex Miller (Clojure team)You can override it (thatā€™s what the clojure matrix builds do). Maybe that flag is only for test compilation. Sorry, on my phone and havenā€™t looked at it in a couple years.#2018-01-0501:22andy.fingerhut@alexmiller No worries. Hand-editing pom.xml is doing the trick for me, and I don't need it automated for any personal reasons. I suspect you may be correct that the command line options might only be affecting compilation of Clojure tests.#2018-01-0501:23Alex Miller (Clojure team)Oh, it is just for testing - see the profile definitions in the pom#2018-01-0501:24Alex Miller (Clojure team)Activating the different profiles will change the flag#2018-01-0501:25Alex Miller (Clojure team)You change the compilation for Clojure itself in build.xml#2018-01-0501:25Alex Miller (Clojure team)In the compile-clojure target#2018-01-0501:25Alex Miller (Clojure team)There is a sysproperty there#2018-01-0501:25Alex Miller (Clojure team)I think youā€™ll have to hard code that change#2018-01-0501:26Alex Miller (Clojure team)Tis a tangled web#2018-01-0501:34andy.fingerhutIf I get a reproducible way to build Clojure in this way, is it of any use to add it to dev wiki ?#2018-01-0501:58Alex Miller (Clojure team)Sure, although Iā€™m not sure how wide the need is for it #2018-01-0501:59Alex Miller (Clojure team)Stu sometimes runs with just the src too, no aot at all#2018-01-0508:29romniis there any way to write a spec for a multimethod?#2018-01-0513:19Alex Miller (Clojure team)Currently, no. You can separate the dispatch function though and spec that if you like#2018-01-0515:02romnithanks, that's what i've been doing so far but i was wondering if spec contained some syntax sugar for this#2018-01-0509:28andy.fingerhut@alexmiller I understand the need is not very wide for building Clojure without direct linking, but I took a shot at adding a short section with instructions for how to do so at the end of this wiki page. Feel free to delete it if it is not useful enough, or edit otherwise to your heart's contents: https://dev.clojure.org/display/community/Developing+Patches#2018-01-0512:36mishahow do I both have
(s/and vector? (s/cat :a int? :b (s/* string?)))
and generator for free? this exact form gives up on generating vector? part
#2018-01-0512:38mishaI do have to use gen/let, etc. don't I?#2018-01-0512:41gfredericksideally the API would be such that you could specify that the default generator for (s/cat :a int? :b (s/* string?)) be transformed with (gen/fmap vec ___)#2018-01-0512:41gfredericksI don't think the spec API makes that easy though#2018-01-0512:44mishaso I need to extract s/cat as dedicated spec, and use it both in s/and spec definition and generator#2018-01-0512:45mishaor is there a "shorter" way?#2018-01-0513:18Alex Miller (Clojure team)cat + vector is something that is quite difficult to do atm while retaining conform, generator, form, etc and a known gap. There are various ways to get parts of those but itā€™s hard to get all of them. This is certainly something Iā€™ve run into and you can see it in some of the core dsl specs. Iā€™ve come to the conclusion that a built in s/vcat would solve all of the cases Iā€™ve run into but I think Rich is looking for a more composable solution still#2018-01-0513:30mishas/vcat is definitely an ok solution for an application code, but smells like "giving up" in a library opieop#2018-01-0513:34Alex Miller (Clojure team)Which is one reason this is still unresolved :)#2018-01-0513:43Alex Miller (Clojure team)There is overlap here between regex ops and coll-of which both work on colls. Ideally those features could be more easily composed#2018-01-0516:21mishaIs there an "easy" way to overwrite :min-count :max-count of the s/coll-of s/map-of specs' generators "inline"? e.g. for REPLing. Or will I have to provide brand new generators for those and use them in s/with-gen?#2018-01-0518:56Alex Miller (Clojure team)the generators will respect the min/max count you gave them#2018-01-0518:56Alex Miller (Clojure team)but you can do something like (s/with-gen (s/coll-of int? :max-count 100) #(s/gen (s/coll-of int? :max-count 3)))#2018-01-0519:30mishaI am asking, if there is shorter way than you wrote. Many colls-of I speced as a standalone specs, and it would be cool to reuse those with something like
(s/def ::my-coll-spec (s/coll-of int? :max-count 100))
(-> ::my-coll-spec
  (with-opts :min-count 5 :max-count 10)
  (s/exercise))
#2018-01-0519:33misha2 main use cases for this for me, are: - do not generate too many, because I want to eyeball it. - do not generate too few, like (s/exercise) tends to do for first few samples (or all, if you give it small sample size (s/exercise ::my-spec 2))#2018-01-0519:41Alex Miller (Clojure team)in short, no. you can give exercise a generator override map though#2018-01-0519:48mishaI remember that, but it is basically the same effort stuff-typing- and copypaste-error-wise for these 2 usecases. thank you#2018-01-0522:45seancorfieldWhen I have (s/conform (s/merge ::spec-1 ::spec-2) my-data) this only seems to conform values in the last merged spec, not the first one. If you swap the spec order, you get conformedvalues from ::spec-1 instead but not ::spec-2 -- is that by design or is it a bug in s/merge? /cc @alexmiller#2018-01-0522:47seancorfield(if I change s/merge to s/and I get conformed values from both specs)#2018-01-0522:51Alex Miller (Clojure team)By design s/merge does not flow conformed values #2018-01-0522:57seancorfieldOK. That was just surprising, given the docstring.#2018-01-0522:59seancorfieldSo I'm going to need (s/and (s/merge ::spec-1 ::spec-2) ::spec-1) for my particular use case (so that it still generates), right @alexmiller?#2018-01-0522:59seancorfield(I know that checks ::spec-1 twice)#2018-01-0523:02Alex Miller (Clojure team)The generator for merge will produce maps that validate for both specs#2018-01-0523:03Alex Miller (Clojure team)So you shouldnā€™t need the and here for generation#2018-01-0523:05seancorfieldRight, but I need the s/and to get conformance flowing through the specs.#2018-01-0523:05seancorfieldIf I want both conform-flow and generation, I need both the s/and and s/merge.#2018-01-0523:06seancorfieldThe s/merge on its own doesn't flow conformance (but will generate fine).#2018-01-0608:40ikitommi@U04V70XH6 would this do both? https://github.com/metosin/spec-tools/pull/91#2018-01-0608:42ikitommispec-tools is already doing the rogue coercion stuff, so woudn't make the lib any more bad in that sense.#2018-01-0523:06seancorfieldThe s/and on its own doesn't generate properly (but will flow conformance).#2018-01-0523:08seancorfieldThis is because we have specs that conform API parameter strings to long, double, date etc -- and I want to be able to combine them and get full conformed values across the combination and get generation across the combination šŸ™‚#2018-01-0523:08seancorfield(cue @alexmiller saying "Well, don't do that!" šŸ™‚ )#2018-01-0523:09Alex Miller (Clojure team)Yeah#2018-01-0523:13Alex Miller (Clojure team)Didnā€™t anyone ever tell you not to do coercion?#2018-01-0611:12mishais there anything to read at length about "don't do the coercion"? It makes sense here and there, but I'd like to see the bigger picture#2018-01-0614:38Alex Miller (Clojure team)If you build coercion into your spec, particularly lossy coercion, then you have made a decision for all consumers of your spec that they have no choice over#2018-01-0617:02gfredericksis the recommendation for coercion use-cases "do it by hand outside the spec'd zone"?#2018-01-0618:40mishaso objection is not about coercion, but about complecting it with specs? Or even about lossy (in any way) specs?#2018-01-0620:28ikitommiā€œthey (spec consumers) have no choise overā€ => wouldnā€™t CLJ-2116 fix just that?#2018-01-0622:57Alex Miller (Clojure team)@misha right. I have no problem with coercion as a concept. :)#2018-01-0623:23gfredericksI on the other hand#2018-01-0703:33seancorfieldAs someone who uses coercions in some specs, I wouldn't support CLJ-2116. It enshrines pluggable coercion into clojure.spec which is not the right approach IMO.#2018-01-0801:34aengelbergI'm trying to write a util that prints out a more friendly error message based on the data returned from s/explain-data. I'm noticing that :in contains misleading data for certain cases:
user=> (s/explain-data (s/map-of integer? integer?) {345 "a"})
{:clojure.spec.alpha/problems ({:in [345 1],
                                :path [1],
                                :pred clojure.core/integer?,
                                :val "a",
                                :via []}),
 :clojure.spec.alpha/spec #<
So for a map-of, it is giving me the key as well as 1 (to indicate the val of the map pair), and for a set, it is giving me an index into some arbitrary ordered version of that set. Neither of these would work if I tried to get-in on those paths. If these values were somehow prefaced with an indicator of what special case triggered them, I could work with that, but the fact that the data may or may not be misleading with no additional context makes the overall data unhelpful for writing a universal tool. Is that a bug? Or is there another way of introspecting the failed specs that I'm missing?
#2018-01-0801:58Alex Miller (Clojure team)on the first case, this is a side effect of treating maps as a sequence of tuples. there is at least one ticket about this particular one with a pending patch.#2018-01-0801:58Alex Miller (Clojure team)for the set case, I donā€™t think anything is logged but a ticket for that would be fine#2018-01-0809:21aengelbergThanks, I'll try to find time to write up a ticket. I'll definitely check out the other ticket you mentioned for map-of, since I'm curious what the proposed solution was, so I know what the expected behavior should be for sets.#2018-01-0816:07bbrinckI ended up writing a lot of code to disambiguate these cases in Expound. https://github.com/bhb/expound/blob/master/src/expound/paths.cljc . The ticket for map-of is https://dev.clojure.org/jira/browse/CLJ-2192 .#2018-01-0816:08bbrinckIIRC, https://github.com/athos/Pinpointer takes a different approach to the problem: it analyzed the specs directly to help understand the explain-data (I could be mis-stating something here, @U0508956F please correct me if Iā€™m wrong šŸ™‚ )#2018-01-0816:09bbrinck@U0567Q30W ticket is https://dev.clojure.org/jira/browse/CLJ-2192#2018-01-0819:36aengelberg@U08EKSQMS thanks! do you have any specific thoughts on the set issue as well? are you also handling that specially from expound?#2018-01-0819:57bbrinckI havenā€™t traced the execution too closely, but I suspect that the set case is handled by this code https://github.com/bhb/expound/blob/master/src/expound/paths.cljc#L134-L135#2018-01-0819:58bbrinckThis could have bugs though. It took awhile to figure out an approach that would work for most :in paths that I could find.#2018-01-0820:00bbrinckRe: sets, I would vote for any JIRA ticket that makes :in paths non-ambiguous, because I think this is one of the biggest hurdles with building pretty-printers for spec. If you make an issue for the set case, send me the link šŸ˜„#2018-01-0821:10cflemingThanks @U08EKSQMS, Iā€™ve been meaning to take a closer look at Expound actually#2018-01-0822:09bbrinckCool! Feedback is welcome and appreciated #2018-01-0810:38cfleming@aengelberg Iā€™d like to see that too if you find it.#2018-01-0812:02mishais it overkill?
(s/nonconforming (s/or :i pos-int? :z #{0}))
#2018-01-0812:04misha- zero? throws on strings, keywords, etc. - just (complement neg-int?) accepts everything but neg ints: strings, maps, etc.#2018-01-0812:11mishais this in any "under-the-hood" way lighter?
(s/and int? (complement neg-int?))
#2018-01-0813:34Alex Miller (Clojure team)What about nat-int?#2018-01-0814:54mishamissed it opieop#2018-01-0820:46dadairGiven the following data: {:a {:name "a"} :b {:name "b"} :c {:name "not-c"}}, how could I construct a spec to say "the key to a value-map must be the keywordized name within the value-map"? This spec would complain that (keyword "not-c") does not match :c#2018-01-0821:01Alex Miller (Clojure team)You can use any arbitrary predicate like #(= (keys %) (->> % vals (map :name) (map keyword))) to check validity#2018-01-0821:02Alex Miller (Clojure team)but itā€™s not going to give you that specific error#2018-01-0821:09dadairgreat thank you#2018-01-0907:11athosIn that case, I'd prefer something like (s/coll-of (fn [[k v]] (= k (keyword (:name v)))))#2018-01-0918:30dadaircheers! I like that too#2018-01-0912:24Charles FourdrignierHello, I would like to write a client for testing some servers' responses. Not for production purpose, but for coding events like coderetreats (inspired by https://github.com/rchatley/extreme_startup). I'm trying to implement this with Clojure (http-kit) and think to Clojure Spec as a validator. I'm a beginner with Clojure and Clojure Spec and I don't know if it's a "good" idea, absolutely ridiculous or in the "Clojure's spirit". I write two specs to demonstrate the concept. https://gist.github.com/Charlynux/35c754f3f213012397da8ae61b97cb1c I would appreciate any feedback about the idea, code...#2018-01-0915:12bbrinckI donā€™t see any problem with the idea of using spec to validate responses. Depending on how you intend to use the spec, there may be some ways you can improve it so you get better error messages when it fails.#2018-01-0915:27bbrinckFor instance, it might be useful to use a multi-spec here https://clojure.org/guides/spec#_multi_spec or maybe use s/or so you can figure out which case matched with conform. It depends on whether you want to just make sure the response is any type of valid response (i.e. 200 and 404 will both be OK, as long as well formed) or if you want to use spec to determine if a response was successful or not#2018-01-0915:44bbrinckIā€™ve added an alternate idea to your gist. Also, check out https://github.com/ring-clojure/ring-spec/blob/master/src/ring/core/spec.clj#L129-L149#2018-01-0916:07Charles FourdrignierThank a lot for your ideas.#2018-01-0912:42stijnwhat can be the reason that explain is successful on an invalid conform?#2018-01-0912:55souenzzowhen I get it, i restart my repl.#2018-01-0912:44mpenetwhat's the spec?#2018-01-0912:49stijni found the 'syntax-error' in the data, so i'm going to try to narrow it down#2018-01-0912:51mpeneton a side note seems like that one still breaks#2018-01-0912:51mpenet(fn [_] :clojure.spec.alpha/invalid)#2018-01-0912:51mpenetoh, maybe not then, it just broke in my repl, hmm#2018-01-0913:53madstapThat's why it works here @mpenet#2018-01-0915:59bhauman@alexmiller I heard Rich mention in his last talk that next version Spec would be more "programmable" or something like that. Is there anything more I can read about that? Or can I ask about specifics?#2018-01-0916:07mpenetmy guess: either it's something related to https://dev.clojure.org/jira/browse/CLJ-2112 with unforming that re-creates a new spec, or something entirely new#2018-01-0916:36bhauman@mpenet Thanks, for the pointer.#2018-01-0916:53Alex Miller (Clojure team)Nothing to read yet and not related to 2112#2018-01-1008:10mpenetWould it be considered ok to promote s/assert* to good for "public" use (atm the docstring says : "Do not call this directly, use 'assert'.")#2018-01-1008:10mpenetin cases where you want the check to always happen no matter the value of check-assert/compile-asserts#2018-01-1008:10mpenetor is there a better way? ping @alexmiller#2018-01-1008:13mpenetI guess if it would be used that way it could warrant a rename maybe#2018-01-1010:33Andreas LiljeqvistI seem to remember that I could generate with a provided example. Something like (gen/fill-in ::spec {::id "all generated will have this id, other values are generated"})#2018-01-1010:34Andreas LiljeqvistCan't remember what it was called though... anyone?#2018-01-1011:06misha@andreas862
(s/def ::id string?)
(s/def ::foo (s/keys :req [::id]))
(s/exercise ::foo 2 {::id #(s/gen #{"a" "b"})})
;=>
;([{::id "a"} {::id "a"}]
; [{::id "b"} {::id "b"}])
?
#2018-01-1011:16Andreas Liljeqvist@misha yes, thank you!#2018-01-1016:56misha
(defn gen
  "... Optionally an overrides map can be provided which
  should map spec names or paths (vectors of keywords) to no-arg
  generator-creating fns. These will be used instead of the generators at those
  names/paths. Note that parent generator (in the spec or overrides
  map) will supersede those of any subtrees. 
parent generator will supersede those of any subtrees part just bit me in the ass harold
#2018-01-1016:57mishabut it forces me to keep generators and specs separate, which ends up to be cleaner#2018-01-1021:39pabloreI have some function
(defn foo
  [a]
  {:pre [(s/valid? ::bar a)]}
...)
That asserts if the input conforms to a spec. It works well, but Iā€™m having trouble to debug when it fails, I want to know what input Iā€™m passing that doesnt conform and why.
#2018-01-1021:47pabloreI just get an AssertionError#2018-01-1022:18misha@pablore I use s/explain-data instead of s/valid. it returns nil if everything is ok, and a datastructure, if there are any errors.#2018-01-1022:19mishaSo you can use it in if or when.#2018-01-1022:21misharead through this too, may be it'll give you some ideas https://clojure.org/guides/spec#_using_spec_for_validation afaik, specs are replacement of :pre/`:post`-conditions#2018-01-1022:24pablores/explain works better than I expected! I know of s/fdef but Iā€™m writing a really complex reducing function with lot of if-cases.#2018-01-1022:28misha
(s/check-asserts true)
(s/assert string? :a)
;; clojure.lang.ExceptionInfo: Spec assertion failed
;;    val: :a fails predicate: :clojure.spec.alpha/unknown
note string? and predicate: :clojure.spec.alpha/unknown
#2018-01-1101:35andy.fingerhutTrying to understand the core specs for destructuring here: https://github.com/clojure/core.specs.alpha/blob/master/src/main/clojure/clojure/core/specs/alpha.clj#L21-L44#2018-01-1101:36andy.fingerhutDoes s/every on a map check all key/value pairs as 2-tuples (or at least any that it does check, since I know that s/every does sampling)#2018-01-1102:47Alex Miller (Clojure team)Yes - map entries are 2-tuples#2018-01-1102:47Alex Miller (Clojure team)I wrote a blog about the destructuring spec btw#2018-01-1102:48Alex Miller (Clojure team)http://blog.cognitect.com/blog/2017/1/3/spec-destructuring#2018-01-1103:15andy.fingerhutGlad you wrote that. It is helpful for understanding.#2018-01-1114:54pabloreAm I using s/conform wrong?
(s/def ::id int?)
(s/def ::some-map
  (s/keys :req [::id]))
(s/conform ::some-map {:id 1}) => s/invalid
#2018-01-1114:56ikitommitry :req-un#2018-01-1115:01pablorethat worked! thanks#2018-01-1115:03pablorebut conform is returning {:id 1} instead of {::id 1}.#2018-01-1115:14Alex Miller (Clojure team)s/keys conform doesnā€™t modify keys. because youā€™re conforming a map with unqualified keys, the conformed map will have unqualified keys#2018-01-1115:15Alex Miller (Clojure team)you could also use :req with data that looks like {::id 1}#2018-01-1115:42Andreas LiljeqvistI have a problem with multi-spec and overriding generators#2018-01-1115:42Andreas LiljeqvistExample code at https://pastebin.com/TW886mDV#2018-01-1116:31Alex Miller (Clojure team)can you state your problem in the form of a question?#2018-01-1116:40Andreas LiljeqvistWhen using an overriding generator for the key ::obj-type the generator seems to be ignored.#2018-01-1116:40Andreas LiljeqvistIf I do the same without multi-spec it works#2018-01-1116:44Andreas Liljeqvistupdated example with better comments https://pastebin.com/FSwMG32t#2018-01-1116:57Alex Miller (Clojure team)Isnā€™t the question really why the generator override doesnā€™t work in the first exercise?#2018-01-1117:03Andreas LiljeqvistSorry, running on low sleep - Yes#2018-01-1117:06Alex Miller (Clojure team)doesnā€™t seem like the generator overrides make their way to the per-method spec generators, not sure why#2018-01-1117:07Alex Miller (Clojure team)looking at the code, it certainly seems like things are done with that intent, not sure why thatā€™s not working, feel free to file a jira#2018-01-1117:09Andreas Liljeqvistok, thanks. Will file a report tomorrow#2018-01-1117:09Alex Miller (Clojure team)the multispec generator drives from the known dispatch method values, not from the method itself, so that may also play into it#2018-01-1122:32dadairanyone know why a call to s/exercise would throw FileNotFoundException Could not locate clojure/test/check/generators__init.class or clojure/test/check/generators.clj on classpath. clojure.lang.RT.load (RT.java:458)?#2018-01-1122:39benalbrechti ran into the same issue a while ago. the way i understand the code, the multi-spec generator generates a value for each dispatch method (where the override map probably is respected) but then it just chooses one of these values using gen/one-of: https://github.com/clojure/spec.alpha/blob/a65fb3aceec67d1096105cab707e6ad7e5f063af/src/main/clojure/clojure/spec/alpha.clj#L939#2018-01-1122:42benalbrechtdadair: you need to add [org.clojure/test.check "0.9.0"] to your dev dependencies#2018-01-1122:47seancorfield@dadair s/exercise will also rely on test.check for a spec that involves a function spec.#2018-01-1211:18Andreas Liljeqvist@alexmiller jira for multispec generator problem: https://dev.clojure.org/jira/browse/CLJ-2311#2018-01-1215:33stijnhow can you define or alter specs programmatically? I have my data model defined in an edn file and like to generate specs based on that#2018-01-1215:33stijn
(let [k :my.ns/foo
      values integer?]
  (s/def k values))
=> user/k
(s/describe :my.ns/foo)
Exception Unable to resolve spec: :my.ns/foo  clojure.spec.alpha/reg-resolve! (alpha.clj:69)
(s/describe 'user/k)
=> values
#2018-01-1215:33mpenetwith eval, or a macro#2018-01-1215:34stijnok#2018-01-1223:19kennyI'm confused why the first s/keys call does not work and the second one does:
(def my-keys [::a])
=> #'user/my-keys
(s/keys :req my-keys)
java.lang.IllegalArgumentException: Don't know how to create ISeq from: clojure.lang.Symbol
(s/keys :req (conj my-keys ::b))
=>
#object[clojure.spec.alpha$map_spec_impl$reify__1931 0x2d5c83e "
#2018-01-1223:25kennyAnd it's not that it just compiles. The conj is actually evaluated:
(s/valid? (s/keys :req (conj my-keys ::b))
          {::a "a"
           ::b "b"})
=> true
(s/valid? (s/keys :req (conj my-keys ::b))
          {::a "a"})
=> false
#2018-01-1223:58gklijsIt's because in the first one you suply a symbol, while in the second a list. Not really sure why though.#2018-01-1300:23kennyThe asymmetry does not make sense to me. Either both should throw or both be allowed.#2018-01-1302:11seancorfieldThe (s/keys :req (conj my-keys ::b)) is treated as (s/keys :req [conj my-keys ::b]) so it requires ::b but the other two elements are symbols so s/keys probably doesn't handle them as expected (it certainly does not evaluate them).#2018-01-1302:11seancorfieldHence, when you supply a map containing ::b it validates and when it doesn't contain ::b it is invalid.#2018-01-1302:13seancorfieldI expect that deep within the implementation of s/keys, the (unevaluated) symbols conj and my-keys are processed in such a way that they fail to be treated as keys at all. @kenny#2018-01-1302:14seancorfieldAh, looking at the source of s/keys it filters out anything that isn't a keyword:
req-keys (filterv keyword? (flatten req))
        req-un-specs (filterv keyword? (flatten req-un))
#2018-01-1302:16kennyThen why is this validated correctly?
(def my-keys [::a])
=> #'boot.user/my-keys
(s/def ::a int?)
=> :boot.user/a
(s/valid? (s/keys :req (conj my-keys ::b))
          {::a "a"
           ::b "b"})
=> false
#2018-01-1302:16kenny
(s/valid? (s/keys :req (conj my-keys ::b))
          {::a 1
           ::b "b"})
=> true
#2018-01-1302:16kennyOh, duh.#2018-01-1302:16kennyI always forget the implicit checking of keys.#2018-01-1302:17seancorfieldIt looks like s/keys tries to treat anything that isn't a keyword in the sequence supplied to :req or :req-un as some sort of predicate but I'm not quite sure I follow all the parts of the macro...#2018-01-1302:18kennyDoesn't Spec auto-check any keys in a map against any specs defined for those keys?#2018-01-1302:18kennyWhich would explain why
{::a "a"
 ::b "b"}
is invalid
#2018-01-1302:19seancorfieldAh, I see what it's doing with those symbols:
:pred-forms (quote
               [(clojure.core/fn [%] (clojure.core/map? %))
                (clojure.core/fn [%] clojure.core/conj)
                (clojure.core/fn [%] my-keys)
                (clojure.core/fn
                 [%]
                 (clojure.core/contains? % :user/b))]),
#2018-01-1302:19seancorfieldand when you evaluate clojure.core/conj and my-keys they evaluate to truthy values (not nil or false).#2018-01-1302:20seancorfield@kenny Did you read the bit where I said (conj my-keys ::b) is not evaluated?#2018-01-1302:20kennyYes#2018-01-1302:21seancorfieldIt treats that as the key ::b and two predicates (which are "true").#2018-01-1302:21seancorfieldSo your spec is equivalent to (s/keys :req [::b])#2018-01-1302:21kennyRighttt#2018-01-1302:21kennyIs that from a macroexpand?#2018-01-1302:21seancorfieldYes, the above :pred-forms snippet is#2018-01-1302:22kennyGotcha. As always, everything has a logical explanation šŸ™‚#2018-01-1302:22seancorfieldVery logical. Not always obvious šŸ™‚#2018-01-1302:22seancorfieldOK, gotta run...#2018-01-1302:23kennyThanks. Have a good night!#2018-01-1302:36seancorfield...back šŸ™‚#2018-01-1320:27martinklepsch
(s/def ::name string?)
(s/def ::ns string?)
(s/def ::doc (s/nilable string?))
(s/def ::src string?)
(s/def ::type {:var :fn :macro})
(s/def ::line int?)
(s/def ::column int?)

(s/def ::def
  (s/keys :req-un [::name ::ns ::doc ::src ::type
                   ::line ::column]))
Trying to sample this spec but getting StackOverflowError ā€” is there anything obvious Iā€™m doing wrong?
#2018-01-1320:37martinklepschNever mind, I had two specs named ::ns šŸ™ˆ#2018-01-1321:50conanI wrote a blog post about writing specs for URLs: http://conan.is/blogging/a-spec-for-urls-in-clojure.html#2018-01-1321:59misha@martinklepsch also
(s/def ::type {:var :fn :macro}) ;;->
(s/def ::type #{:var :fn :macro})
#2018-01-1502:02gfredericks
user> (s/valid? (sorted-set 1 2 3) "you'd think this would return false")
;; throws
ClassCastException java.base/java.lang.Long cannot be cast to clojure.lang.Keyword  clojure.lang.Keyword.compareTo (Keyword.java:114)
#2018-01-1502:50andy.fingerhutstack trace shows it is trying to call (s/regex? (sorted-set 1 2 3)), which raises that same exception.#2018-01-1503:02andy.fingerhutworkaround: (s/valid? (hash-set (sorted-set 1 2 3)) ...)#2018-01-1503:02gfredericksthis makes me think the answer will be "you can't use sorted sets as specs"#2018-01-1503:03andy.fingerhutI looked for a CLJ issue that I thought existed about making set lookup on sorted sets never return an exception because of exceptions thrown by the comparison function, but not sure if I am imagining it.#2018-01-1503:03gfrederickseven if it didn't fail at that point, a sorted set is not a general predicate šŸ˜•#2018-01-1503:04gfredericksmaking a change like that would be weird because what if you supply a buggy comparison function to sorted-set-by, for instance? it would just swallow them#2018-01-1503:04gfredericksit could throw them on insert I guess, but would swallow on lookup#2018-01-1503:11andy.fingerhutFound several related tickets on sorted sets, but not the one I was imagining. My imagination can be vivid.#2018-01-1509:36bronsa@andy.fingerhut yeah I know which ticket youā€™re talking about#2018-01-1509:38bronsait was https://dev.clojure.org/jira/browse/CLJ-1242 which got refocused just on equality, but originally was about equality and lookup#2018-01-1514:04conanIs there a way to get the keys back from a map spec? i.e. the function get-keys:
(s/def ::banana (s/keys :req [::length ::type] 
                        :opt [::colour]))

(get-keys ::banana)
=> {:req [::length ::type]
    :opt [::colour]}
I can do this by using (s/form ::banana) but that just gives me a sequence so seems very brittle
#2018-01-1515:14andre.stylianosI don't think there's a better way for now, there's an issue in Jira that seems likely to be what you want but it's still open. https://dev.clojure.org/jira/browse/CLJ-2112#2018-01-1515:31ikitommi@U053032QC forms have been stable, so (->> (s/form ::banana) rest (apply hash-map)) while waiting the specs for specs.#2018-01-1515:37conan@U485ZRA58 yes that does look like what I want. @U055NJ5CC yep, it certainly gets me exactly what i want, it just seems a bit wonky#2018-01-1517:00Alex Miller (Clojure team)there is likely to be more support for this in the next major update for spec#2018-01-1517:14souenzzo
(let [spec (s/keys :req [:foo/bar])
      _ (s/def ::req (s/coll-of keyword?))
      keys-spec (s/cat :op symbol?
                       :args (s/keys* :opt-un [::req]))
      {{:keys [req]} :args} (s/conform keys-spec (s/form spec))]
  req)
@U064X3EF3, you always say to "not use conform to coerce values". In this case, we are using conform to "parse" a struct. It's a valid use? read the topic
#2018-01-1517:19Alex Miller (Clojure team)I donā€™t think Iā€™ve said that. conform is designed to conform values so Iā€™m not even sure what you mean.#2018-01-1517:21Alex Miller (Clojure team)this seems like a pretty ugly workaround (the embedded s/def seems like an obvious place where you could have issues too)#2018-01-1517:24souenzzo(it's just to fit on let to easy copy'n'paste on repl) I think that you say do not use conform to conform values related to CLJ-2116#2018-01-1517:25Alex Miller (Clojure team)do you mean ā€œdo not use conformers to coerce valuesā€ which is a very different statement?#2018-01-1517:25Alex Miller (Clojure team)that seems wholly unrelated to your example if so#2018-01-1517:27souenzzoYEP. Sorry. I got lost with the words šŸ˜ž#2018-01-1517:30Alex Miller (Clojure team)I donā€™t think what youā€™re doing here is related to coercion#2018-01-1517:30Alex Miller (Clojure team)which is independent from whether itā€™s good :)#2018-01-1517:30Alex Miller (Clojure team)my real question is why you want to ask this question in the first place and whether you can arrange things to not ask it#2018-01-1520:32mishahttps://clojurians.slack.com/archives/C1B1BB2Q3/p1516035630000041?thread_ts=1516025043.000637&amp;cid=C1B1BB2Q3 half life 3 confirmed kappa#2018-01-1523:04Steve PetersonHi Spec-cers! Which Clojure libraries are using Spec as a protection layer in their APIs, i.e. to validate args? Are they instrumenting API fns ā€œby defaultā€ so input args are checked on each call?#2018-01-1600:06seancorfield@stevep I can tell you what we're doing in production with clojure.spec at World Singles as far as REST APIs are concerned: we have a spec for each handler (i.e., each API endpoint) and the general pattern is roughly:
(let [params (s/conform ::this-api-spec (:params req))]
  (if (s/invalid? params)
    (error-response (s/explain-data ::this-api-spec (:params req)))
    (normal-process params)))
#2018-01-1620:44arnaud_bos(how-)do you prevent un-spec-ed keys to propagate further down the normal-process?#2018-01-1622:15seancorfield@U1DLD0WNR In keeping with Clojure's open-for-extension approach, normal-process will just ignore keys it doesn't care about. Similarly, if I only spec certain keys, others don't get checked -- so the constraints are open.#2018-01-1600:08seancorfieldWe have that wrapped up in a macro to reduce boilerplate but that's the essence of it. The error-response function decodes the spec failure based on a set of heuristics and "expected" failure points so we can return documented error codes and localized messages to the client.#2018-01-1600:09seancorfieldWe only instrument functions in our dev/test cycle, not in release code. We also have specific "tests" that run clojure.spec.test/check on certain functions.#2018-01-1600:10seancorfieldThe argument against just relying on instrument to check arguments is that you want better error messages than spec failures for your clients!#2018-01-1610:31vikeriCan I modify to a spec after Iā€™ve defined it? My use case is to add the :fn argument for an fdef but only when I run my tests.#2018-01-1611:22conan@vikeri why can't you just add it anyway? instrument doesn't run the :fn and :ret bits by default anyway#2018-01-1611:22conani use orchestra.spec.test/instrument to do this#2018-01-1611:24vikeri@conan I also use orchestra and it runs the :fn by default so thatā€™s why I have that issue#2018-01-1611:25conanah, can you swap out which you use in different circumstances? so you can switch on/off the :fn execution#2018-01-1611:25conani think you can just call clojure.spec/fdef again fine, and it'll update the spec registry with whatever you give it#2018-01-1612:55Alex Miller (Clojure team)You can specify an alternate spec to use with instrument #2018-01-1719:30borkdudeI cannot pass a keyword as function to s/conformer, since it will try to resolve it as a spec. e.g.: (s/conformer (fn [parsed] (:foo parsed))) is needed right?#2018-01-1719:31borkdudeOther question: I vaguely remember something about not using spec as a string/text parser from this channel, but what exactly was the argument against it?#2018-01-1719:48seancorfieldThe main objection is that is complects parsing and validation -- and "forces" the parsing effect on all clients of the spec.#2018-01-1719:49borkdudeforces the parsing effect?#2018-01-1719:46borkdudeIf anyone wants to review my clojure.spec parser (for performance comparison with other parsers like Instaparse, Kern), please do: https://github.com/borkdude/aoc2017/blob/master/src/day16.clj#L234#2018-01-1719:51seancorfield(what we tend to do is have a spec that accepts both the target type as well as a string that could represent the target type ... if the parse/conversion fails, the result is ::s/invalid; if the parse/conversion succeeds or if the target type was provided, the validation is performed)#2018-01-1720:26donaldballDo you have an example you can share of this style?#2018-01-1721:13seancorfieldI've shared it before here but it'll take me a while to de-tangle the generic code from the proprietary code so don't hold your breath for me to get around to it (too busy this afternoon and I'll probably forget after that).#2018-01-1721:16seancorfieldI probably ought to just open source it at some point...#2018-01-1721:20donaldballšŸ™‚#2018-01-1720:59Alex Miller (Clojure team)my main objection to it is that the regex op capabilities of spec are designed to support the structure of data and only incidentally can be used as a way to describe the structure of strings. Inevitably, I suspect you will find either the capabilities or performance of ā€œspec as string parserā€ to be lacking and will be disappointed in our lack of interest in addressing those. There are very good tools already in existence for creating parsers on strings and I those are the right tool for the job.#2018-01-1721:15seancorfieldAnd to be clear, what we do at World Singles is not any sort of spec-as-parser -- we just have a few, very specific specs that accept a string or a target type and, for the string, do a simple conversion. Those target types are Long, Double, Boolean, and Date. We don't attempt to do anything else via spec -- this is just for simple form or query string data.#2018-01-1721:18borkdudeI was surprised specā€™s performance was even better than Instaparse (see link above). Handwritten parser is fastest, Kern in between.#2018-01-1721:19borkdudeBut yeah, I understand there is no interesting in optimizing spec for strings, although it does quite well#2018-01-1721:21donaldballI was also fairly delighted at how both expressive and fast spec was when validating strings as seqs of chars, at least for my silly use cases.#2018-01-1721:26aengelbergto be fair, Instaparse has enough processing overhead that parsing one or two characters at a time in a 47 KB string isn't the best (or most common) use of it šŸ™‚#2018-01-1721:28borkdude@aengelberg yeah, Iā€™ve discussed this example with you before in #instaparse I believe, itā€™s part of Advent of Code. Nothing conclusive, just very contextual.#2018-01-1722:41Alex Miller (Clojure team)I think the perf differences are going to be highly dependent on the complexity of the grammar and the size of the data so itā€™s hard to make predictions from a single example.#2018-01-1722:53rickmoynihanAlso instaparse is a far more general purpose parsing solution, supporting both left and right recursion; Iā€™d be surprised if that didnā€™t incur an overhead.#2018-01-1723:46dpsuttoni've written some specs and i put some calls to s/valid? in comment macros in the namespaces where they are. is there a good way to take this out of a comment macro and have them run at some convenient but non-production time?#2018-01-1723:47dpsuttonclojurescript is the target if there's a compiler option to prevent spec checks on advanced compilation#2018-01-1815:39kingmobIn Clojurescript, what are people doing if they want to remove spec from production builds? Is there a lib for it? Macros wrapping fdef in the same file? Or placing all spec stuff in a dev-only namespace?#2018-01-1815:48borkdude@kingmob just curious, but why would you want to remove them?#2018-01-1815:48kingmobDownload size
#2018-01-1815:49kingmobIf Iā€™m only validating/testing with spec, I donā€™t want it to take up sie in the resulting Js artifact#2018-01-1815:49borkduderight. putting them in a dev only ns is what I would do then#2018-01-1815:54kingmobThx @borkdude. Thing is, I prefer fdefs be defined by their function, so I was hoping someone had already handled this. We have some dev-time-only logging macros I could repurposeā€¦ though I suspect dev-only namespaces would still be tricky, assuming itā€™s even possible#2018-01-1815:56borkdude@kingmob this project has an example how dev only namespaces are possible (both lein and boot): https://github.com/borkdude/lein2boot#2018-01-1815:57borkdude@kingmob you could also try in #clojurescript#2018-01-1816:00dpsuttonyeah it would be nice if you could elide all spec related stuff under advanced compilation#2018-01-1816:01kingmob@borkdude Ahh, it looks like you factored out everything common into its own file and then use different :source-paths. We do that, too, but if, e.g., I added fdefs in animals.crud, theyā€™d still end up in prod builds#2018-01-1816:01kingmobWe could add it to the dev source path, but then itā€™s separate from the definitionā€¦which is fine, itā€™s not really that big a deal#2018-01-1816:01borkdudeyeah, not getting around that when putting in a different ns#2018-01-1816:02borkdudehave you tried what difference in size you get for the production js with and without? it may not even be worth it#2018-01-1816:02kingmob@dpsutton Yeah, it would be nice. Does spec use asserts under the hood so that the :elide-asserts option would remove it?#2018-01-1816:04kingmob@borkdude Yeah, I have actually, itā€™s substantial šŸ™‚ Lemme recompile and check with source-map-explorer again#2018-01-1816:04borkdudeI believe you šŸ™‚#2018-01-1816:09kingmob@borkdude OK, bare-bones spec is adding 39.84 kb to the output .js. Itā€™s not major, but this is just for spec itself, and a sprinkling of initial spec defs. Itā€™s not extensive at all, and since none of it is used at run-time, we might as well ditch it.#2018-01-1816:11borkdudeI see there is an issue about it: https://dev.clojure.org/jira/browse/CLJS-1701#2018-01-1818:05shaun-mahoodI've been thinking about building some company/app wide specs. Say I spec something as :com.myco.app.service/id - is there any way that I can apply that spec to some data where everything is named as :service/id?#2018-01-1818:28seancorfield(s/def :service/id :com.myco.app.service/id)?#2018-01-1818:28seancorfield(and then use that local "alias")#2018-01-1818:34shaun-mahoodOh I didn't think of that - thanks!#2018-01-1820:38schmeeis there something like this in spec?
(defn conform! [spec thing]
  (if (s/valid? spec thing)
    thing
    (throw (ex-info (s/explain-str spec thing)
                    (s/explain-data spec thing)))))
#2018-01-1820:40seancorfieldSounds like s/assert to me @schmee?#2018-01-1820:41seancorfieldAlso, I'd note that your function does not conform the value so it's a very misleading name...#2018-01-1820:41schmeesorry ā€™bout that, copy-paste errorā€¦#2018-01-1820:42schmeebut it sure does sound like s/assert, thanks šŸ™‚#2018-01-1820:42seancorfieldPerhaps
(defn conform! [spec thing]
  (let [v (s/conform spec thing)]
    (if (s/invalid? v)
      (throw (ex-info ...))
      v)))
#2018-01-1820:43seancorfield(whereas s/assert returns the original thing, not the conformed value)#2018-01-1820:44schmeeyeah, I copied my conform thing from another project where I needed the conformed value, now I just want to validate so s/assert is perfect šŸ‘#2018-01-1820:46seancorfieldJust bear in mind it can be turned on/off by command line options and by code (including 3rd party libraries!).#2018-01-1820:49schmeehmmā€¦ using assert for config validation sounds kinda sketchy then#2018-01-1820:50schmeeahh, putting it in :pre is what I want, forgot that it existed šŸ˜›#2018-01-1820:52schmeeno, that just gives an AssertionError, it doesnā€™t show you whats wrongā€¦.#2018-01-1820:53seancorfield(and asserts can be turned on/off by calling code via *assert*)#2018-01-1820:53uwoso, when s/keys fails on a missing required key, thereā€™s no path information in explain-data. Weā€™re attempting to dispatch on the result of explain-data, but the information we need about the missing key is in a :pred function. So is the only way to parse this data to parse the s expression used internally to describe required keys?#2018-01-1820:55seancorfield@uwo Yeah, that's ugly. We wrote a heuristic-based explain-data walker to figure that stuff out. It recognizes and picks apart the :pred value...#2018-01-1820:56uwoyikes. off the top of your head do you know if thereā€™s jira ticket along these lines.#2018-01-1820:57uwothanks, btw!#2018-01-1821:04seancorfieldOff the top of my head, nope, sorry.#2018-01-1821:05seancorfieldBut we needed to pick apart spec errors in general and map them back to our domain for reporting to end users... it just took us a while to flesh out the walker (it's... non-trivial).#2018-01-1821:52uwosame for us#2018-01-1821:35schmeehow do you go about spec-ing anonymous functions with side-effects? since conform et al actually calls the function itā€™s validating, I guess the answer is ā€œdonā€™t do it?ā€#2018-01-1821:40seancorfieldNot sure what situation you're talking about @schmee?#2018-01-1821:42schmeeI have a piece of code that takes an array of functions, and those functions should return a particular type of record#2018-01-1821:42schmeeI call s/valid? on the input to this code, and if I use fspec to check the functions in the array, it will actually call all those fns to ensure that they return the correct thing#2018-01-1821:43schmeebut those fns could grab values from a database, launch nuclear missiles or whatever, so Iā€™m thinking I should just spec it as (s/+ fn?) instead#2018-01-1821:44schmeeinstead of (s/+ (s/fspec :args (s/cat) :ret (partial instance? MyRecord)))#2018-01-1821:44seancorfieldYes, best not to use fspec for side-effecting functions-as-arguments because instrumentation uses generative testing on them.#2018-01-1821:44Alex Miller (Clojure team)Use ifn?, not fn?#2018-01-1821:45seancorfieldAn array of functions sounds more like (s/coll-of ifn?)#2018-01-1821:46schmeenoted on ifn?, but why s/coll-of instead of s/+?#2018-01-1821:48Alex Miller (Clojure team)+ is a regex op used for describing the internal structure of sequential collections (like an arg list). coll-of is for sequential collections. Either may be applicable here, depends on context.#2018-01-1821:48Alex Miller (Clojure team)In particular, what if anything is including this spec#2018-01-1821:49Alex Miller (Clojure team)If, for example, youā€™re including it as one arg in an args spec, using + would be bad as it would combine in the same sequential parent context#2018-01-1821:50schmeeI see, Iā€™m using it as part of a s/keys spec so that shouldnā€™t be a problem then šŸ™‚#2018-01-1821:51Alex Miller (Clojure team)Either is prob ok then but I would prob use coll-of#2018-01-1821:51Alex Miller (Clojure team)With a :min-count constraint#2018-01-1821:52schmeeIā€™ll go with that, thanks for the help šŸ‘#2018-01-1821:53schmeeIā€™m a bit torn about the fspec thing though, I think it would be much more pleasant to see (s/fspec :args (s/cat) :ret (partial instance? MyRecord) in a doc-string instead of ifn?, but I canā€™t due to generative testing#2018-01-1821:53schmeeI can just write it out in the docstring anyway of course šŸ™‚#2018-01-1821:54schmeeIā€™m still new to spec so Iā€™m kinda feeling it out, itā€™s very easy to overdo it and use it as a type system#2018-01-1822:00seancorfieldNote that you can still run clojure.spec.test.alpha/check on your function with the full fspec -- independent of what the function's actual fdef says. (right @alexmiller)#2018-01-1822:00seancorfieldOr you could instrument it with a different fspec too I believe?#2018-01-1822:01Alex Miller (Clojure team)yes#2018-01-1822:01seancorfieldI'd probably fdef it with a spec that is safe to instrument, and then check it with a more detailed fspec...#2018-01-1822:02seancorfield(because it might, potentially, get instrumented by code far, far away in your code base that might "forget" to override the spec for it)#2018-01-1823:55arohnerIs there a library for an s/keys-alike but keys are strings rather than keywords, yet?#2018-01-1902:23gfredericks(s/keys :req [::foo]) or (s/keys :req ["foo"])?#2018-01-1902:24gfredericksif the former, is it not syntactically possible to express all string keys? if the latter, how are the strings tied to the spec registry?#2018-01-1920:15pabloreDo specs replace deftype and reify? If not, how do I use them together?#2018-01-1920:43schmeeno. deftype is used for performance and polymorphism, reify is mostly used for Java interop, and spec is about specifying your data. so itā€™s three different use-cases really šŸ™‚#2018-01-2209:10witekI want to spec a command vector, given (s/def ::command-name keyword?). Now the command-vector needs to be a vector where the first element must be the ::command-name. All other elements are optional arguments. Any suggestions? Thank you.#2018-01-2213:32taylor
(s/def ::command-name keyword?)
(s/def ::command (s/cat :name ::command-name
                        :args (s/* any?)))
(s/conform ::command [:foo "bar"])
#2018-01-2213:33taylorif you really need the input to be a vector then
(s/def ::command
  (s/and
    vector?
    (s/cat :name ::command-name
           :args (s/* any?))))
#2018-01-2209:19jakobI have a datastructure that looks something like this:
(def my-invoice-address
  {
   :use-delivery-address false
   :street "asdf"
   :zip "123"})
If :use-delivery-address is true, :street and :zip should not be there. If :use-delivery-address is false, :street and :zip should be there I find it really difficult to write a spec for this. Anyone who have any ideas?
#2018-01-2209:25stathissideris@karl.jakob.lind I guess :use-delivery-address is :req, the others are :opt and then you wrap your keys with an s/and to enforce the rest of the logic that you described#2018-01-2209:31jakobWell, the others are required if :use-delivery-address is true. So it's not that simple#2018-01-2209:36stathissiderisyeah, but Iā€™m proposing that this is enforced by the extra predicate that you will pass to s/and#2018-01-2209:39jakobhm.. something like this?
(s/def ::invoice-address
  (s/keys
   :req-un [::use-delivery-address]
   :opt-un [::street ::zip]))
where do the s/and belong ?
#2018-01-2209:44stathissiderissomething like this (Iā€™m sure thereā€™s a more elegant way to do it):#2018-01-2209:45stathissideris
(s/def ::invoice-address
  (s/and
   (s/keys
    :req-un [::use-delivery-address]
    :opt-un [::street ::zip])
   (fn [{:keys [use-deliver-address street zip]}]
     (or (and (not use-deliver-address)
              (not street)
              (not zip))
         (and use-deliver-address
              street
              zip)))))
#2018-01-2209:46jakobThat's at least more elegant than my attempts šŸ™‚ Thanks! will use it!#2018-01-2209:51Jakub HolĆ½ (HolyJak)Perhaps use https://clojure.org/guides/spec#_multi_spec ?
(defmulti address-type :address/use-deliver-address)
(defmethod address-type true [_]
  (s/keys :req []))
(defmethod address-type false [_]
  (s/keys :req [ :address/street ...]))
@karl.jakob.lind
#2018-01-2209:58jakobgood suggestion @karl.jakob.lind. will read about that!#2018-01-2216:33carkhlet's say i want to write a spec for a string that should be splitted in two, imagine i have a split function that can be called with that string and returns a pair of strings... i have a ::check-whole-thing spec and also a ::check-part spec ...how would i write that ::check-whole-thing part ?#2018-01-2216:35carkhthere are numerous examples, but these all work on clojure data structures#2018-01-2216:36carkhi think what i'm asking here is how to transform the data being specced before speccing it further#2018-01-2217:04Alex Miller (Clojure team)have you read the spec guide? https://clojure.org/guides/spec If not, maybe take a spin through that first and then refine the question.#2018-01-2217:06carkhi did ... actually found a related question to which you responded on the forum ... the conformer function looks like it fits the use case#2018-01-2217:06Alex Miller (Clojure team)that does not seem like a use case where conformers should be used to me#2018-01-2217:07Alex Miller (Clojure team)Iā€™m confused by several aspects of your question so not sure how best to answer it#2018-01-2217:07carkhhttps://groups.google.com/forum/#!topic/clojure/9Bg_9P5o3h8#2018-01-2217:08carkhi'm conforming a string, i want to have it transformed into a vector of strings before further conforming#2018-01-2217:08carkhcould be any kind of transformation really#2018-01-2217:08Alex Miller (Clojure team)well, I would recommend not doing that#2018-01-2217:08carkhah !#2018-01-2217:08Alex Miller (Clojure team)do that in code explicitly#2018-01-2217:08Alex Miller (Clojure team)donā€™t use spec for it#2018-01-2217:08carkhbut semantically i'm really validating the string there#2018-01-2217:09carkhwhich is what spec is for isn't it ?#2018-01-2217:09Alex Miller (Clojure team)I think you are better off if you spec the inputs and outputs of the transformation#2018-01-2217:09Alex Miller (Clojure team)and explicitly invoke the transformation#2018-01-2217:10Alex Miller (Clojure team)and donā€™t use spec to actually do the transformation part#2018-01-2217:11carkhthe thing is that i'm not really interested in that transformation at all, i'm only validating it to pass the whole thing merilly along#2018-01-2217:14Alex Miller (Clojure team)if you have to do that transformation in able to pass it along, then it seems like you are interested in it after all#2018-01-2217:16carkhlet's imagine a use case for one of those html dsls ... they have this way of specifying class attributes like so [:div.app-container ...] ...that's not what i'm doing at all but that's a good example... are you saying that spec wouldn't fit to do the "destructuring" of that keyword ?#2018-01-2217:19carkhor maybe to destructure an udp packet or you know go inside a data structure and destructure it for me#2018-01-2217:20carkhthat last one was a bad example =)#2018-01-2217:23carkhwhat's the issue here ? the fact that one could not rebuild the data structure from the conformed result ?#2018-01-2217:23ghadito rephrase slightly, don't conflate spec/validation with transformation in one phase#2018-01-2217:23ghadiyou'll be perpetually unhappy#2018-01-2217:24ghadido your validation, do transformation, then do validation again if you want#2018-01-2217:24ghadithere are other gotcha with conform to look into#2018-01-2217:24ghadilike it forces conformance on all consumers of the spec#2018-01-2217:25ghadiwhich you don't want#2018-01-2217:25carkhok i need to think a little bit on this#2018-01-2217:25carkhthansk for your time the both of you#2018-01-2217:26ghadino problem. Lots of other validation libraries provide "mutative validation".... conflating transformation into its API#2018-01-2217:26ghadireally impedes reuse#2018-01-2217:27ghadiTypical example is 'validate this string into a UUID'#2018-01-2217:27carkhthat's a good example#2018-01-2217:28carkhbut now i first need to conform a big data structure, then walk it and validate parts of it#2018-01-2217:28carkhthat's a lot more code ><#2018-01-2217:28carkhanyways i'll think on it, thanks again !#2018-01-2220:58golinI'm trying to spec polymorphic maps where a particular key is used for dispatch. Is there a way to spec the dispatch key to specify the dispatch value?
(s/def ::foo
  (s/keys :req [:data/type :foo/stuff]))

(s/def ::bar
  (s/keys :req [:data/type :bar/stuff]))

e.g. ::foo's :data/type should have a value of :foo, and ::bar's :data/type have a value of :bar
#2018-01-2221:20schmee@scallions I think you are looking for multi-spec: https://clojure.org/guides/spec#_multi_spec#2018-01-2221:28golinThanks, I'll look into it. That retag argument looks interesting.#2018-01-2312:01stathissiderishereā€™s a tricky one: I have a data structure and I use conform with a regex spec to extract information from it. If the data structure is invalid in two places, I only get the first problem reported, because the regex stops. Is there any way to get the second problem too? (I suspect itā€™s a no)#2018-01-2313:04Alex Miller (Clojure team)No#2018-01-2313:05Alex Miller (Clojure team)To get a deeper level for something like this you would really need to write your own custom spec#2018-01-2313:43stathissideris@alexmiller ok, makes sense, thanks. Iā€™ll see if I can circumvent the problem somehow#2018-01-2315:54vikeriIs there any way of making sure that the specs I write point to other specs? I can write (s/def ::my-spec (s/keys [:not-a-spec/key]) And :not-a-spec/key doesnā€™t have to be a spec. Can I somehow enforce that it really is a spec that itā€™s pointing to?#2018-01-2317:04bronsahttps://gist.github.com/stuarthalloway/f4c4297d344651c99827769e1c3d34e9#2018-01-2411:44vikeri@U060FKQPN Thanks!#2018-01-2316:26vikeriAlso, if I could somehow avoid defining specs twice that would be fantastic#2018-01-2316:59stathissideriswhy do you have to define them twice?#2018-01-2410:10vikeri@stathissideris I donā€™t have to define them twice, but I want to avoid it happening#2018-01-2411:26misha@vikeri "them" who?#2018-01-2411:26mishado you have a specific example of what you are trying to avoid?#2018-01-2411:27vikeriI want this to throw an error/warning
(s/def ::hello int?)
(s/def ::hello string?)
#2018-01-2411:28mishathen you'll get en exception on the next ns reload in repl, would not want that, would you? )#2018-01-2411:29mishaotherwise, write a macro, which throws/warns if spec you are trying to define is already defined in specs registry#2018-01-2411:34misha
defmacro sdefonce [sym form]
  `(if (contains? (s/registry) ~sym)
    (throw (ex-info "NO!" {:sym ~sym :existing (s/form (get (s/registry) ~sym))}))
    (s/def ~sym ~form)))

(sdefonce ::foo string?)
=> :user.specs/foo
(sdefonce ::foo string?)
clojure.lang.ExceptionInfo: NO!
    data: {:existing clojure.core/string?, :sym :user.specs/foo}
#2018-01-2411:34misha@vikeri#2018-01-2411:36mishayou might check if form is the same as in registry, and it will not blow up on ns reload for unchanged specs, but still will on the ones you are iterating over (developing)#2018-01-2411:43vikeri@misha I only want to deny overrides when I run the tests to make sure I havenā€™t duplicated it in the code. (Loading some specs from an external lib). Yeah I was considering writing my own macro but then Iā€™d like to override the macro so that normally it uses the normal s/def but when I run my tests it throws the error. Iā€™ll probably go with my own macro actually.#2018-01-2411:45mishasounds like too custom use case for an out of the box solution#2018-01-2415:53pabloreIs there a way to use s/assert and s/explain when the assert fails?#2018-01-2416:23pabloreis it still good practice to use {:pre [(s/assert ā€¦)]} on functions?#2018-01-2416:24pablorefor some reason assert is failing but explain is giving me success#2018-01-2416:27schmee@pablore just keep in mind that asserts can be toggled off#2018-01-2416:27pabloreYes but thats not the idea, I need it to fail when I give it a bad input#2018-01-2416:28schmeehereā€™s a thing Iā€™ve found useful many times:
(defn validate [spec thing]
  (if (s/valid? spec thing)
    thing
    (throw (ex-info (s/explain-str spec thing)
                    (s/explain-data spec thing)))))
#2018-01-2416:28schmeeI use that instead of assert since it canā€™t be toggled off#2018-01-2416:28ghadihttps://github.com/Datomic/mbrainz-importer/blob/master/src/cognitect/xform/spec.clj#L13-L23#2018-01-2416:29ghadithere's another (similar) idea there#2018-01-2421:05mrchanceHi, I want to generate test data based on a swagger spec, but subject to some constraints. Do you think spec is a good tool for that? If yes, how would one best go about that? If not, what would be a better way to do it?#2018-01-2422:46kingmobQuick sanity check: in Clojurescript, is cljs.spec.test.alpha working for anyone else? I keep getting compilation errors about clojure.test.check.#2018-01-2422:47kingmob(Since itā€™s still alpha, I wasnā€™t sure)#2018-01-2423:04dadairIn some cases you need to explicitly import [org.clojure/test.check <ver>], at least on the clj side#2018-01-2423:10kingmob@dadair Thx. Is that based on https://dev.clojure.org/jira/browse/CLJS-1792#2018-01-2423:10kingmobIā€™ll try adding it#2018-01-2423:27kingmob@dadair That did it! Thanks#2018-01-2505:43ikitommi@mrchance check out https://github.com/immoh/swagger-spec#2018-01-2508:17mrchance@ikitommi Thanks, but I don't want to generate swagger, but the data consumed by the interfaces specified in a swagger file. It doesn't look like that is possible with swagger-spec?#2018-01-2509:38ikitommioh, the endpoint data. With vanilla swagger spec, there needs to be a JSON-Schema -> Spec converter, I donā€™t know if such exists yet. But if you describe your endpoints with spec, you get the data generation for free.#2018-01-2510:09mrchanceYes, already doing that, unfortunately some of the other services in my project aren't written in Clojure. If one were to write such a converter, how would one go about it? I invested an afternoon because I didn't really find anything too, but it seemed really hard and messy to generate specs at runtime, due to its reliance on global mutable state šŸ˜ž #2018-01-2510:10mrchanceMaybe <gasp> generate code </gasp> ?#2018-01-2510:12mrchance@ikitommi I saw that you had schema like maps in your spec tools, which could be handled like normal data. Would that be an option for this case?#2018-01-2510:19ikitommiSome more ā€œfunctional specsā€ might be coming to core, but while waiting, yes, there are some rogue versions at https://github.com/metosin/spec-tools/blob/master/src/spec_tools/data_spec.cljc#L8-L10 and the data-specs on top of those.#2018-01-2510:23ikitommimost/all of the core specs have functional versions already in the core, but currently not documented and the apis are bit hairy. Maybe those will be polished and published as part of the public api? The forms need to be created manually but for most core predicates, itā€™s easy to resolve those (and spec-tools does that already)#2018-01-2512:49mrchanceHm, I see. Thanks, I'll give it a go, and hope for the release of the functional api in core.#2018-01-2515:43mpenetis there something like s/cat but that doesn't do any labeling? need to match a subsequence within a sequence without too much ceremony#2018-01-2515:44mpenetit's a seq of chars (just playing), so nothing generic, I need equality matching (ex match (b a r) in (f o o b a r b a z))#2018-01-2515:45mpenetI know I can mix s/cat and sets of chars or predicates, but that's quite ugly#2018-01-2515:46mpenetI am also forced to use regex context (my question is part of a wider parsing thing)#2018-01-2515:46Alex Miller (Clojure team)if you donā€™t want labeling, just use s/valid?#2018-01-2515:47mpenetI dont need labeling per char, I do for the whole subseq (up one level)#2018-01-2515:47mpenetmaybe I didn't understand what you mean#2018-01-2515:49mpenetall the surounding value have a meaning that I need to retain also, I can't just check "does this seq contains this sub-seq"#2018-01-2516:01Alex Miller (Clojure team)you can use s/& to apply a custom predicate to a regex spec#2018-01-2516:02Alex Miller (Clojure team)which could be helpful here#2018-01-2516:04mpenetright, forgot about &#2018-01-2516:08mpenetthat's brilliant, I was messing with conformers but I think & might just do all this for me#2018-01-2521:26schmeeis it just me or do tools.namespace and instrument not get along?#2018-01-2522:05schmeeor does it not check things that are defed?#2018-01-2522:05schmeethat is, if I reload a namespace that has (def foo (some-instrumented-fn invalid-value)), does reloading the ns throw an exception?#2018-01-2522:08Alex Miller (Clojure team)Reloading is going to remove your instrumented vars#2018-01-2522:11schmeeok, so those checks should be done through tests instead?#2018-01-2522:35Alex Miller (Clojure team)what checks?#2018-01-2522:39schmeelet me change the question: whatā€™s the best way to check specs with clojure.test?#2018-01-2522:40Alex Miller (Clojure team)that seems pretty far from the original questions :)#2018-01-2522:40schmee(is (s/valid? ::my-spec (some-fn)) works, but it doesnā€™t give helpful output#2018-01-2522:40schmeeIā€™m not very familiar with either clojure.test or spec, so Iā€™m fumbling around a bit šŸ™‚#2018-01-2522:40Alex Miller (Clojure team)there used to be a pin here to an example, but maybe it has aged out#2018-01-2522:41Alex Miller (Clojure team)by ā€œcheck specsā€, Iā€™m assuming you mean to run clojure.spec.test.alpha/check on one or more specā€™ed functions#2018-01-2522:42schmeeactually, I want to check with s/valid?, but use s/explain instead of the usual error message provided by is#2018-01-2522:43Alex Miller (Clojure team)so s/valid? is not going to be as useful in checking specā€™ed functions as is calling check#2018-01-2522:44Alex Miller (Clojure team)unless ::my-spec here refers to a data spec#2018-01-2522:44Alex Miller (Clojure team)I canā€™t really tell#2018-01-2522:45schmee::my-spec in this case is a data spec, yes#2018-01-2522:45Alex Miller (Clojure team)ok#2018-01-2522:45schmeesorry if my questions are a bit confused, itā€™s hard to ask good questions when youā€™re not familiar with something#2018-01-2522:45Alex Miller (Clojure team)yep, no worries#2018-01-2522:46Alex Miller (Clojure team)I am similarly trying to re-interpret your questions :)#2018-01-2522:46Alex Miller (Clojure team)so you want to check if some data matches a spec and if not, return a message that is s/explain#2018-01-2522:46Alex Miller (Clojure team)one way would be to:#2018-01-2522:47Alex Miller (Clojure team)
(is (nil? (s/explain ::spec data)))
#2018-01-2522:48Alex Miller (Clojure team)although I guess you want the explain result as a value not as a print#2018-01-2522:49Alex Miller (Clojure team)(is (not= "Success!\n" (s/explain ::spec data))) would do that I think#2018-01-2522:49Alex Miller (Clojure team)thatā€™s quite ugly of course :)#2018-01-2522:49schmeeexplain-data did the trick!#2018-01-2522:51Alex Miller (Clojure team)yeah, can do same idea with explain-data#2018-01-2522:51Alex Miller (Clojure team)you can supply a final custom error message to is too#2018-01-2522:52Alex Miller (Clojure team)so something like: (is (valid? ::spec data) (s/explain-str ::spec data)) ?#2018-01-2522:54schmeeI went with this:
(defmacro spec-is [spec value]
  `(is (nil? (s/explain-data ~spec ~value))))
#2018-01-2601:26bbrinckA shameless plug, but you can also add a message to is. With expound you could do something like
(defmacro spec-is2 [spec value]
  `(is (s/valid? ~spec ~value)
       (expound/expound-str ~spec ~value)))
which will give you output like:
FAIL in (test2) (alpha_test.cljc:2152)
-- Spec failed --------------------

  1

should satisfy

  #{:c :b :a}



-------------------------
Detected 1 error

expected: (clojure.spec.alpha/valid? #{:c :b :a} 1)
  actual: (not (clojure.spec.alpha/valid? #{:c :b :a} 1))
#2018-01-2608:37schmeehaha, I donā€™t mind shameless plugs when they are great! šŸ˜„#2018-01-2608:37schmeethatā€™s way better, thanks for the tip šŸ™‚#2018-01-2522:54schmeethanks a ton for your help! šŸ˜„#2018-01-2522:54Alex Miller (Clojure team)np!#2018-01-2614:51trisshi all, how do I look up a functions spec at run-time? s/get-spec doesnā€™t seem to to work?#2018-01-2614:53Alex Miller (Clojure team)it should if you pass it a fully-qualified symbol#2018-01-2614:53trissah thankyou. just spotted it said that in the docsā€¦ an I get the fully qualified symbol for a function from that function?#2018-01-2614:54Alex Miller (Clojure team)you donā€™t need to get it from anywhere?#2018-01-2614:55Alex Miller (Clojure team)
user=> (defn foo [] nil)
#'user/foo
user=> (s/fdef foo :args (s/cat) :ret nil?)
user/foo
user=> (s/get-spec `foo)
#object[clojure.spec.alpha$fspec_impl$reify__2451 0x8c11eee "
#2018-01-2614:56Alex Miller (Clojure team)I used back-tick there - syntax quote will resolve symbols so thatā€™s an easy way, but I also could have done (s/get-spec 'user/foo)#2018-01-2614:56Alex Miller (Clojure team)note that what you get back is a function spec instance, but that spec supports keyword lookup of :args, :ret, and :fn#2018-01-2614:57Alex Miller (Clojure team)or you can use s/form on it directly
user=> (s/form (s/get-spec 'user/foo))
(clojure.spec.alpha/fspec :args (clojure.spec.alpha/cat) :ret clojure.core/nil? :fn nil)
#2018-01-2614:57trissah I see now. wonderful thank you Alex!#2018-01-2615:52don.dwoskeRich mentioned in a podcast (I think) that there was some sugar forthcoming which allowed a map to specify/lift the namespace for all the keys in the map (if they all had the same namespace) to the map level such that each individual key in the map could be printed / declared as un namespaced for cleanliness? Is there news on this, or a preview of what it might look like? I'm working on something similar and an example would be great to see. This was mentioned in the context of using spec, so that's why I'm putting it here vs. another channel.#2018-01-2615:55ghadiIt's syntax sugar -- already in 1.9#2018-01-2615:55bronsa#:foo{:bar 1} = {:foo/bar 1}#2018-01-2615:55ghadi
user=> *print-namespace-maps*
true
user=> {:foo/bar 42}
#:foo{:bar 42}
#2018-01-2615:55ghadijinx @bronsa#2018-01-2615:55ghadisame example even#2018-01-2615:56bronsathe universal example#2018-01-2615:57the2bearsSo this foo walks into a bar. Bartender says, "Hey buddy, what kind of example you trying to set?"#2018-01-2616:16bronsasimple_smile#2018-01-2616:16don.dwoskeAh, missed it.. thanks @ghadi and @bronsa#2018-01-2616:31don.dwoske...and it's been around for awhile now... I'll try not to feel dumb. https://groups.google.com/forum/#!topic/clojure-dev/8tHLCm8LpyU and https://dev.clojure.org/jira/browse/CLJ-1910#2018-01-2617:00trissok, so if I have a function `fā€™ thats been passed in to another function and I donā€™t know itā€™s fully qualified name is there a way to find it?#2018-01-2617:00bronsano#2018-01-2617:01bronsaif by function you mean var then yes#2018-01-2617:02trissah I do mean var I think - how do I look up a fully qualified name/spec for a function referenced by a variable?#2018-01-2617:02trissoh hang on `var?' just gave me a false...#2018-01-2617:09bronsamaybe if you give a code example of what youā€™re trying to do it might be clearer#2018-01-2617:11trissIā€™m trying to look up if a particular function has had a spec written for it.#2018-01-2617:11trissi.e does a function have a spec specified for it.#2018-01-2617:11ghadiif you have a fully-qualified symbol, you can do this#2018-01-2617:12ghadi(get (s/registry) 'my.ns/function)#2018-01-2617:12ghadiif you have a function, you cannot#2018-01-2617:12trissI wonā€™t have the fully qualified symbol when I come to do it - does this mean itā€™s a no go?#2018-01-2617:12ghadi"So what are you actually trying to do?"#2018-01-2617:12trissah ok - this is a shame - The REPL seems to be able to look up the fully-qualified symbol for function#2018-01-2617:13triss@gihadi - it is a bit convoluted - Iā€™m experimenting with agent based computing - and turning functions in to agents using only the information in there type signatures#2018-01-2617:14triss(well there specā€™d signatures)#2018-01-2617:16ghadican you turn on instrumentation prior a priori?#2018-01-2617:18trissnot what I want unfortuantely - Iā€™m doing some crazy stuff with the spec - I unpack info about each argument and tehn search a collection for it.#2018-01-2617:19trissthe only thing I canā€™t do is look up wether or a not a function has had a spec written for it (when I donā€™t have its fully wualified name)#2018-01-2617:26Alex Miller (Clojure team)If you donā€™t have the qualifier, then itā€™s ambiguous what youā€™re referring to #2018-01-2617:26Alex Miller (Clojure team)A function of the same name could be spec ed in many nses#2018-01-2617:27Alex Miller (Clojure team)That said, the registry is just a map with either qualified symbols or qualified keywords as keys - you could search for it #2018-01-2617:32trissok - thanks Alex. So is there a way to look up a functions source namespace/name?#2018-01-2617:33Alex Miller (Clojure team)Given what info?#2018-01-2617:33trissjust the functionā€¦#2018-01-2617:33trissthe REPL seems to know hwere it came from#2018-01-2617:33Alex Miller (Clojure team)Example?#2018-01-2617:34Alex Miller (Clojure team)A function instance doesnā€™t hold an explicit name for its source var#2018-01-2617:35triss
bb-simple.core> (defn function [a] a)
#'bb-simple.core/function
bb-simple.core> (def z function)
#'bb-simple.core/z
bb-simple.core> z
#function[bb-simple.core/function]
bb-simple.core> 
#2018-01-2617:35trissso I can store a function in z - the REPL tells me where I defined it#2018-01-2617:38trisswhen all I have is z is there a way to find that it refers to bb-simple.core/function?#2018-01-2617:38bronsanot reliably#2018-01-2617:39Alex Miller (Clojure team)thatā€™s not the default printer - you must be running with something addition in your repl#2018-01-2617:39trissah - itā€™s the CIDER repl in emacs#2018-01-2617:39Alex Miller (Clojure team)the function class name is a munged version of the original function name (assuming it was from a var, can be from other places too)#2018-01-2617:40bronsawhat thatā€™s doing is using demunge to get from the class name to an approximation of the original var#2018-01-2617:40bronsabut that is not relibale and can only be done if the function was compiled from a defn#2018-01-2617:40Alex Miller (Clojure team)you can demunge to recover the class name, but the munge operation is not a true function (mathematically) in that multiple inputs map to the same class name#2018-01-2617:40Alex Miller (Clojure team)thatā€™s where the unreliability comes from#2018-01-2617:41trissoh ok this all sounds a bit yuck. Think Iā€™ll steer clear for now. Itā€™s a lot of extra typing for me but such islife#2018-01-2617:41Alex Miller (Clojure team)(Compiler/demunge (.getName (class z)))#2018-01-2912:53roman01lašŸ‘‹ Given a naive example of a function spec, how to specify a custom generator such that it satisfies :fn spec?
(s/fdef sbtrct
        :args (s/cat :a int? :b int?)
        :ret int?
        :fn #(= (:ret %) (- (-> % :args :a) (-> % :args :b))))
#2018-01-2917:09conanwhat is it you want to generate exactly?#2018-01-2917:13conanare you trying to generate a 2-value vector containing ints that can be used with apply as the args for sbtrct?#2018-01-2917:14conangenerators generate values, so by providing a spec for each of your input args and for your return value, you've already provided generators (because spec provides a built-in generator for the int? predicate)#2018-01-2917:19roman01la@U053032QC I want to generate a sequence of args/ret values such that they satisfy relation defined in :fn spec#2018-01-2917:20conanah, ok:
(s/exercise-fn `sbtrct)
#2018-01-2917:21conanthat'll generate a bunch of input values for a and b (using the generators provided by their specs, i.e. int?), put them into your function, and return you a sequence of those input pairs with the calculated output#2018-01-2917:21conanif your function doesn't match your :fn spec, it'll blow up#2018-01-2917:22conanif you want to do a larger number of tests of your function, you can do generative testing using clojure.spec.test.alpha/check#2018-01-2917:22conanso
(stest/check `sbtrct`)
#2018-01-2917:23conanthat'll generate 1000 different inputs and check all the outputs for conformance with your :ret spec and :fn predicate#2018-01-2917:23conanneat, huh?#2018-01-2917:23roman01la> if you want to do a larger number of tests of your function.. thatā€™s exactly what I need, exercise-fn doesnā€™t seem to be able to generate huge sequences šŸ™‚#2018-01-2917:24conanyou can do
(s/exercise-fn `sbtrct 100000)
#2018-01-2917:24conanoh no, looks like that blows up#2018-01-2917:24roman01layeah Iā€™m getting integer overflow#2018-01-2917:25conandoesn't blow up so long as you don't print it#2018-01-2917:25roman01laoooooh, damn#2018-01-2917:26conanmight just be lazy though#2018-01-2917:26conanyeah it's lazy
#2018-01-2917:26conanso there are numbers in there that blow up#2018-01-2917:28roman01lahm, what can be done about this?#2018-01-2917:28conando you need the ints to be very large?#2018-01-2917:28roman01lano, just a lot of them#2018-01-2917:28conanif not, you could try
(s/def ::small-int (s/int-in 0 1000000))
(s/fdef sb
  :args (s/cat :a ::small-int :b ::small-int)
  :ret int?
  :fn #(= (:ret %) (- (-> % :args :a) (-> % :args :b))))
#2018-01-2917:29conanthat's specifying a range that the ints can be in, so you won't get huge numbers generated#2018-01-2917:29roman01laShould I use with-gen if I donā€™t want to modify the original spec?#2018-01-2917:30conanwith-gen is for when you need to create your own generator because the existing ones can't capture your set of values#2018-01-2917:31conani guess you'll have to look into what's actually causing the error#2018-01-2917:32conanthat approach above works for me though, with s/int-in as the spec for the inputs to your function#2018-01-2917:33conani used a different name for my function though so don't just copy/paste#2018-01-2917:34roman01laworks for me as well, thanks!#2018-01-2917:39conanno worries, docs are often a bit sparse around spec, but it is alpha, so /shrug#2018-01-2918:05roman01laI ended up with (s/int-in Integer/MIN_VALUE Integer/MAX_VALUE)#2018-01-2918:21johnDoes spec.test in any way supersede test.check? I couldn't find docs explaining the difference. There seems to be some overlap between the two.#2018-01-2918:39arrdemNot that I know of. spec.test leverages test.check extensively and seems designed to complement test.check by enabling you to derive many generators automatically rather than specify them by hand.#2018-01-2918:43Alex Miller (Clojure team)correct. test.check is useful for writing arbitrary property-based tests and creating different kinds of custom generators#2018-01-2918:44bronsaand spec.test uses test.check internally#2018-01-2919:08johnUnderstood. Thanks#2018-01-3020:11rafaelI'm trying to pass :gen overrides for clojure.spec.test.alpha/check, but I get the sense my understanding of the documentation.#2018-01-3020:12rafaelI expected the following to succeed:
(defn foo [k]
  (if (keyword? k) :k :nope))

(def my-kw? (partial instance? Keyword))

(s/fdef foo
        :args (s/cat :a-keyword my-kw?)
        :ret #{:k})

(spec.test/check `foo {:gen {`my-kw? (gen/return :kw)}})
#2018-01-3020:13rafaelI'm trying to pass in a generator for the my-kw? spec, but it fails with the cause "Unable to construct gen at: [:a-keyword] for: my-kw?"#2018-01-3020:15rafaelIs overriding generators on the call to check possible and I'm getting some detail wrong, or did I just misunderstand the docs for check and there is no way to override generators like that?#2018-01-3020:26Alex Miller (Clojure team)the values in that map should be 0-arg functions returning a generator#2018-01-3020:26Alex Miller (Clojure team)so change (gen/return :kw) to #(gen/return :kw)#2018-01-3020:28Alex Miller (Clojure team)although now that I back up a bit, I donā€™t think you can do what youā€™re trying to do with my-kw? either#2018-01-3020:28Alex Miller (Clojure team)you should instead register a named spec instead of my-kw?:#2018-01-3020:29Alex Miller (Clojure team)(s/def ::my-kw keyword?) and then use ::my-kw instead of my-kw? and as the key in the gen map#2018-01-3020:35Alex Miller (Clojure team)Putting it all together:
user=> (require '[clojure.spec.alpha :as s] [clojure.spec.test.alpha :as stest] [clojure.spec.gen.alpha :as gen])
nil
user=> (defn foo [k] (if (keyword? k) :k :nope))
#'user/foo
user=> (s/def ::my-kw keyword?)
:user/my-kw
user=> (s/fdef foo :args (s/cat :a-keyword ::my-kw) :ret #{:k})
user/foo
user=> (stest/check `foo {:gen {::my-kw #(gen/return :kw)}})
({:spec #object[clojure.spec.alpha$fspec_impl$reify__2451 0x6c25e6c4 "
#2018-01-3118:43rafaelThat works. I had tried the 0-arg generator function, but I was missing the fact that the spec ident had to be a keyword. Thank you!#2018-01-3022:06jqmtorhey all, I've been meddling with spec and I came across something I don't know how to represent. I want to create a spec for the s/keys spec itself and I have this:
(s/cat :spec-kind #{`s/keys}
         :keys      (s/+ (s/cat :mandatory+qualified? #{:req :opt :req-un :opt-un}
                                :keys                 (s/coll-of keyword? :kind vector?))))
the problem is that the keys :req :opt and so on can be repeated. is there any way to avoid this? (any other unrelated suggestion is also welcome)
#2018-01-3022:15Alex Miller (Clojure team)keys*#2018-01-3022:23jqmtorgreat, thanks! missed that when looking through the api#2018-01-3022:26eriktjacobsenI'm running into the same wall described at https://groups.google.com/d/msg/clojure/i8Rz-AnCoa8/OEg04fbKBwAJ where we'd like to resolve a symbol for a multi-fn in clojurescript but apparently cannot. is this being looked into? that thread died.#2018-01-3118:29uwo@stuarthalloway Iā€™d love to use this on a private project, though do I need to be worried about licensing? https://github.com/Datomic/mbrainz-importer/blob/5d2d90f9a35789824675a4cc86a9a433527cb41b/src/cognitect/xform/spec.clj#2018-01-3118:31uwosimple enough idea, but I did get it from you šŸ™‚#2018-01-3120:41Drew VerleeHas anyone seen any attempts to turn a postgres schema into a set of clojure specs so you can generate data like whats in the database? Or vise versa, fill the database with data? In the case where your using postgres or a relational db, would that be helpful?#2018-01-3121:50eggsyntax@drewverlee at one point I did the same thing with a Datomic schema. It was probably simpler because of the natural Datomic/Clojure match, but I'd definitely think you could do the same with a relational DB. You might want to think about what you'd want the specs to capture -- type is the obvious low-hanging fruit, but there might be other stuff. And you could probably capture the set of fields present in each table with s/keys. Possibly other stuff depending on your particular schema, but in my experience worth considering well in advance of taking on the conversion.#2018-01-3121:53Drew Verlee@eggsyntax iā€™m thinking either this sort of thing already exists (maybe in another language) or there are really good reasons why its not useful. But maybe clojure spec opens some new doors.#2018-02-0216:55stathissiderisit doesnā€™t take care of traversing the schema, but you can run your data through this to get specs out of them: https://github.com/stathissideris/spec-provider#2018-02-0216:55stathissideris(my project btw, so Iā€™m here for questions)#2018-01-3122:21Alex Miller (Clojure team)Iā€™m pretty sure someone has probably done this already for jdbc based on your databasemetadata, and maybe thatā€™s 90% of the work#2018-01-3122:23Alex Miller (Clojure team)donā€™t think Iā€™ve seen it specifically with spec though#2018-02-0108:22andy.fingerhutIn Rich Hickey's Spec-ulation talk, he mentions the idea of wishing to make specs include some notion of side effects, for functions that have them: https://github.com/jafingerhut/jafingerhut.github.com/blob/master/transcripts/2016-dec-rich-hickey-spec-ulation.txt#L563-L569#2018-02-0108:23andy.fingerhut"A function provides its return. If you gave me what I required, I will provide to you this result. And of course I would like to broaden this discussion to include services and procedures, and things like that. So if your thing is effectful, one of the things you provide is that effect. If you call this thing with these arguments, the thing will be in the database, or I will send an email for you, or some other thing."#2018-02-0108:24andy.fingerhutDoes anyone have any ideas on how such a thing might be described in a spec-like fashion?#2018-02-0111:20mattfordSo I've been using https://github.com/ring-clojure/ring-spec to validate http responses.#2018-02-0111:21mattfordI'd like to over-ride/extend the body part of the above spec to use spec's I've written that model the JSON body I get.#2018-02-0111:21mattfordIs that a thing?#2018-02-0111:22mattfordHow'd people go about that?#2018-02-0111:55borkdudeIā€™d write a different spec for the parsed body.#2018-02-0112:23misha@bbrinck quick expound question: is there a function, accepting vanilla spec/explain-data value and returns pretty error message as string?#2018-02-0112:31mishafound it! expound.alpha/printer-str https://github.com/bhb/expound/blob/master/src/expound/alpha.cljc#L479#2018-02-0115:15bbrinck@misha Yep, that should work. Iā€™ll add that to my notes about what to add to public API for the expound beta#2018-02-0119:01nwjsmithI'm having trouble getting a useful spec together for this map-entries function I've written:
(defn map-entries
  "Returns a map consisting of the result of applying f to the first entry of
  the map, followed by applying f to the second entry in the map, until the map
  is exhausted."
  [f m]
  (into {} (map f) m))

(s/fdef map-entries
  :args (s/cat :f (s/fspec :args (s/cat :entry (s/tuple any? any?)) :ret (s/tuple any? any?))
               :m (s/map-of any? any?))
  :ret (s/map-of any? any?)
  :fn #(<= (count (-> % :ret)) (count (-> % :args :m))))
#2018-02-0119:03nwjsmithIf I turn instrumentation on, and evaluate say (map/map-entries (fn [[k v]] [(name k) (dec v)]) {:a 1 :b 2 :c 3}), I get an instrumentation exception because spec tries to run (fn [[k v]] [(name k) (dec v)]) with a generated (s/tuple any? any?).#2018-02-0119:06nwjsmithIf I loosen the spec on the :f arg to just fn?, then instrumentation is okay, but I'll lose a the "free" property tests. I guess I can just supply my own generator for check...#2018-02-0119:08mishawhat would you gain, if you mock your f?#2018-02-0119:11nwjsmithWell if I had the following spec:
(s/fdef map-entries
  :args (s/cat :f fn?
               :m (s/map-of any? any?))
  :ret (s/map-of any? any?)
  :fn #(<= (count (-> % :ret)) (count (-> % :args :m))))
#2018-02-0119:11nwjsmithThen any function could pass instrumentation (not awesome).#2018-02-0119:12mishaI think you can at least spec f's args being a tuple, and stop there#2018-02-0119:12mattfordI have this map
{"took" 1,
 "timed_out" false,
 "terminated_early" false,
 "_shards" {"total" 1, "successful" 1, "failed" 0},
 "hits" {"total" 0, "max_score" nil, "hits" []}}
can I have spec conform the strings to keywords and deal with the overloaded "hits" key somehow?
#2018-02-0119:15mishaI think, strings to keywords conversion needs to happen separately. but later, you'll be able to use 2 :keys keyword specs with different namespaces#2018-02-0119:17nwjsmithMaybe I can spec the args differently here. What I'd like is the spec to be is
(s/cat :f (s/fspec :args (s/cat :entry (s/tuple <T: any?> <V: any?>))
                   :ret (s/tuple any? any?))
       :m (s/map-of <T: any?> <V: any?>))
#2018-02-0119:17nwjsmithugh, let me format that#2018-02-0119:19misha@mattford
(def m {"took" 1,
        "timed_out" false,
        "terminated_early" false,
        "_shards" {"total" 1, "successful" 1, "failed" 0},
        "hits" {"total" 0, "max_score" nil, "hits" []}})
(s/def :foo/hits vector?)
(s/def :bar/hits (s/keys :req-un [:foo/hits]))
(s/def :bar/m (s/keys :req-un [:bar/hits]))

(->> m
  (clojure.walk/keywordize-keys)
  (s/explain :bar/m))
Success!
=> nil
#2018-02-0119:21misha@nwjsmith try to drop :ret (s/tuple any? any?) for :f#2018-02-0119:22mattfordgreat ty!#2018-02-0119:22mishainto's spec might catch not-tuples from f#2018-02-0119:23mishait blows up opieop#2018-02-0119:28nwjsmith
(defn map-entries
  "Returns a map consisting of the result of applying f to the first entry of
  the map, followed by applying f to the second entry in the map, until the map
  is exhausted."
  [f m]
  (into {} (map f) m))

(s/fdef map-entries
  :args (s/cat :f (s/fspec :args (s/cat :entry (s/tuple any? any?)))
               :m (s/map-of any? any?))
  :ret (s/map-of any? any?)
  :fn #(<= (count (-> % :ret)) (count (-> % :args :m))))


(stest/instrument)

(map-entries (fn [[k v]] [(name k) (dec v)]) {:a 1 :b 2 :c 3})

ExceptionInfo Call to #'user/map-entries did not conform to spec:
In: [0] val: ([nil nil]) fails at: [:args :f] predicate: (apply fn)
  clojure.core/ex-info (core.clj:4617)
#2018-02-0119:30nwjsmithHow does instrumentation of function arguments work? Does the function get evaluated?#2018-02-0120:26Alex Miller (Clojure team)the args spec for the function argument is used to generate inputs. the function argument function is invoked with those examples and the ret spec is validated#2018-02-0120:29Alex Miller (Clojure team)also, you should pretty much never use fn? (if you go that route) - ifn? is almost always what you want#2018-02-0120:30Alex Miller (Clojure team)generic higher-order functions are inherently difficult to spec in useful ways. specs are great for saying concrete things about your data. The more generic your function, the harder it is to say something specific and meaningful.#2018-02-0121:59ghadinot sure if this is a bug, but I could use a second set of eyes:#2018-02-0122:02ghadiI'm asking spec to stub out the function TARGET while overriding the generator for ::bar, which is aliased to ::foo. The gen overriding doesn't work -- if I override ::foo (which ::bar aliases) then it works. Probably a non-minimal example if someone can help me reduce it#2018-02-0200:07Alex Miller (Clojure team)I think there is a known issue with swapping gens for aliased specs#2018-02-0200:07Alex Miller (Clojure team)On the phone so canā€™t easily search JIRA #2018-02-0202:35ghadihttps://dev.clojure.org/jira/browse/CLJ-2079 ^^#2018-02-0203:00Alex Miller (Clojure team)Thatā€™s the one#2018-02-0217:14dominicmIs there a convenience function for:
(s/cat :foo.bar.baz.bosh/title :foo.bar.baz.bosh/title
       :foo.bar.baz.bosh/description :foo.bar.baz.bosh/description)
#2018-02-0217:19stathissideris@dominicm catā€™s name arguments are non-qualified#2018-02-0217:20stathissideris
(s/cat :title :foo.bar.baz.bosh/title
       :description :foo.bar.baz.bosh/description)
#2018-02-0217:20stathissiderisand with aliases it can be shorter#2018-02-0217:21dominicm@stathissideris But I want to return a namespaced map šŸ˜„#2018-02-0217:21stathissideriscat is for sequences!#2018-02-0217:21stathissiderisnot maps#2018-02-0217:21stathissiderisuse s/keys instead#2018-02-0217:22dominicmI have a sequence of things, and I want to conveniently conform it into a namespaced map. I'm being lazy.#2018-02-0217:23stathissiderisoh!#2018-02-0217:23stathissiderisok makes sense#2018-02-0217:24dominicmI guess the manual way I go (hi ho!)#2018-02-0217:31stathissideriswould be a pretty trivial macroā€¦#2018-02-0219:42mishamacro all the things!#2018-02-0221:20gklijsDon't need a macro for that, it's pretty easy with some functions to convert data based on a spec to/from a vector.#2018-02-0309:51misha@alexmiller is there any estimate on where will this be addressed? It makes generating even slightly complex data structures noticeably painful. https://dev.clojure.org/jira/browse/CLJ-2079 (Generator overrides for spec aliases are not respected)#2018-02-0312:34benalbrecht@misha @ghadi you could resolve the spec manually before passing it to the generator map:
(defn resolve-spec [spec] 
(last (take-while keyword? (iterate s/get-spec spec))))
#2018-02-0313:49mishawhy? @benalbrecht#2018-02-0313:50mishado you mean "find all aliases manually and explicitly override those"?#2018-02-0313:50mishathat should work, yes#2018-02-0401:03richiardiandreaIs there a way to not report errors for an argument in a s/cat or spec only the second position of it? The first argument in my case is a JS connection object and I am not interested in seeing it when it does not validate because very verbose#2018-02-0401:04seancorfield@richiardiandrea If you don't care about it validating, why not spec it as any??#2018-02-0401:04richiardiandreaYes that is what I am doing, but it is included as part of the error message#2018-02-0401:05seancorfieldOh, I see. So you want a custom error reporter instead? Have you looked at Expound?#2018-02-0401:05richiardiandreaOh right, yes I am aware of expound. Probably I can have a custom reporter if not too tough too implement#2018-02-0401:09bbrinck@richiardiandrea With expound, you can customize the function that prints out your value https://github.com/bhb/expound#configuring-the-printer#2018-02-0401:10bbrinckSee the example under ā€œYou can even provide your own function to display the invalid value.ā€#2018-02-0401:13bbrinckAlthough by default, parts of your data that is valid wonā€™t be shown anyway with Expound#2018-02-0401:27bbrinck@richiardiandrea Hereā€™s an example that includes a custom printer that replaces the first arg (presumably one that is verbose) with a more succinct representation https://gist.github.com/bhb/5222914641bcdd08d07b7d930e388d89#2018-02-0404:38richiardiandreaThat is great thaaaanks, I am actually using expound#2018-02-0405:34bbrincknp#2018-02-0414:08alex-dixonHow do you write a spec for a map that has a namespaced keyword like :db/id?#2018-02-0414:21tayloruse a s/keys spec, and use its :req or :opt to specify the individual key specs
(s/def my-spec (s/keys :req [:db/id]))
#2018-02-0414:24taylorhttps://clojure.org/guides/spec#_entity_maps#2018-02-0415:01alex-dixonAwesome. Thanks. Thought I had to use ::db/id for some reason#2018-02-0422:45bbrinckComing soon in expound: optional human-readable error messages for predicates https://asciinema.org/a/161011#2018-02-0423:26stathissideris@bbrinck thatā€™s really good, but would it be possible to add an error message to an existing spec? Iā€™m thinking about the case where youā€™re trying to do this to a third-party spec whose code you donā€™t control#2018-02-0423:26stathissiderisit may be too much to ask, but I think it would be better to keep the libs off each otherā€™s toes#2018-02-0423:28bbrinckItā€™s an interesting idea ā€¦ certainly you could add a an error message for a spec directly (I can add a function for this).#2018-02-0423:29bbrinckIf you used expound/def on an existing function, youā€™d probably need to overwrite the actual spec though, so as not to break the way s/def works#2018-02-0423:29bbrinckin other words, calling expound/def would overwrite existing spec, just like s/def#2018-02-0423:30bbrinckAnother thing to watch out for is that this wonā€™t work unless the original spec author defined a spec for the predicate specifically#2018-02-0423:31bbrinckSo, for instance, you wonā€™t be able to add an error message for this instance of vector? https://github.com/clojure/core.specs.alpha/blob/master/src/main/clojure/clojure/core/specs/alpha.clj#L16 since itā€™s not a spec by itself#2018-02-0423:39stathissideris@bbrinck yeah, I would expect the parameter in that function to be the name of a spec, not a predicate#2018-02-0423:40stathissiderisI was looking at spec tools recently, and although I liked the functionality, I was a bit sceptical about that fact that I would have to write specs differently (using the ā€œspec recordā€ I think)#2018-02-0423:41stathissiderisso thatā€™s what struck me about your approach as well#2018-02-0423:41bbrinckYep, I could add something like (expound/add-message <qualified-keyword> <string>)#2018-02-0423:42stathissiderisIā€™d be very happy with that šŸ™‚ does that get stored in a separate registry for expound?#2018-02-0423:42bbrinckYes, thatā€™s correct#2018-02-0423:42bbrinck@stathissideris Yes, needing to write specs differently is a fair concern here. I donā€™t think thereā€™s a way to bolt this functionality onto existing specs, and even if we could, Iā€™m not sure Iā€™d want to#2018-02-0423:43bbrinckUnless clojure.spec supported this directly, of course ā€¦ šŸ™‚#2018-02-0423:43stathissideristhereā€™s been talk about spec metadata for quite a while now#2018-02-0423:43bbrinckBut for now, you can write your specs in either style#2018-02-0423:43stathissiderisā€¦but no official solution yet#2018-02-0423:44bbrinckand if the consumer is using spec, it works. if they are using expound, they will get the enhanced error messages#2018-02-0423:44stathissiderisyeah, I think it all composes a bit more nicely like that!#2018-02-0423:45bbrinck(for the record, Iā€™d be happy to see something compatible go into spec, and if it did, iā€™d use it in expound)#2018-02-0423:45bbrinckoh wait, I actually implemented this function - itā€™s register-message#2018-02-0423:46bbrinckMan, Iā€™ve already forgotten what I wrote yesterday šŸ™‚#2018-02-0423:46bbrinckI will add it to documentation#2018-02-0423:46stathissiderislooking forward to the new functionality, I have a direct use for it when you release it, so thanks for your work!#2018-02-0423:46stathissiderishaha, thatā€™s great, thatā€™s the style Iā€™m going to use#2018-02-0423:46bbrinckCool, I hope to release early this week#2018-02-0423:47bbrinckuntil then, itā€™s in 0.4.1-SNAPSHOT#2018-02-0423:47bbrinckbut, use at your own risk šŸ™‚#2018-02-0423:47stathissiderisnot in a hurry, but looking for ways to make the errors in my UI a bit more user-friendly#2018-02-0704:07bbrinckIā€™ve released Expound 0.5.0, which includes this feature#2018-02-0704:07bbrinckhttps://github.com/bhb/expound#error-messages-for-predicates#2018-02-0707:55stathissiderisgreat, thank you very much, Iā€™m going to give it a go in the next few days#2018-02-0913:58stathissiderisso, no register-message in the end?#2018-02-0509:57dominicmIs there a s/cat function which doesn't create a map? I essentially want a regex op which will consume exactly 2 predicates.#2018-02-0513:15Alex Miller (Clojure team)Use s/tuple#2018-02-0513:17dominicm@U064X3EF3 That isn't a regex op, so it doesn't work for this:
(s/conform
  (s/cat
    :foo int?
    :bar (s/tuple int? int?)
    :baz int?)
  [1 2 3 4])
#2018-02-0513:17dominicm
(s/conform
  (s/cat
    :foo int?
    :bar (s/cat :a int? :b int?)
    :baz int?)
  [1 2 3 4])
;; =>
;; {:foo 1, :bar {:a 2, :b 3}, :baz 4}
with the s/cat regex op
#2018-02-0513:21Alex Miller (Clojure team)s/+ then s/& with a size constraint?#2018-02-0513:23dominicm@U064X3EF3 This works!
(s/conform
  (s/cat
    :foo int?
    :bar (s/& (s/+ int?) #(= 2 (count %)))
    :baz int?)
  [1 2 3 4])
Rather cool.
#2018-02-0513:24dominicmI didn't really understand s/& when I read the docstring.#2018-02-0513:49Alex Miller (Clojure team)itā€™s like s/and but as a regex op#2018-02-0514:21dominicmThat's a good description. I prefer that #2018-02-0509:57dominicmMaybe my mistake is using something positional where I should be using a map#2018-02-0510:23mattforddo you have to conform it?#2018-02-0510:25mattfordI've instrumented and stubbed out a function that thanks to the magic of spec can now generate results. The result is generated from a multispec though, is there a way of limiting a multispec to one of the methods [so I can only return certain types of results]?#2018-02-0510:53acron@mattford add a predicate check for the dispatch type#2018-02-0510:54acron
(s/cat :foo (s/and ::my/thing is-correct-thing?))
#2018-02-0510:56acronwhere ::my/thing is your multispec#2018-02-0510:56acronthis has worked relatively successfully for me#2018-02-0510:57acronWhat would cause clojure.spec.test.alpha/spec-checking-fn/conform! to be applied to a function that has *not* been instrumented? šŸ¤”#2018-02-0510:59mattfordFeels a bit circular having a mutlispec generate many cases only then to filter it again on the same logic each defmethod has.#2018-02-0511:01mattfordworks though...#2018-02-0511:02acronWell, the real question could be why do we want just one result type generating?#2018-02-0511:05mattfordI instrumented my general get-event handler but I'm only interested in testing "visualisation" events in the next part of the tests..#2018-02-0516:06tcoupland@mattford check out the :gen option of instrument #2018-02-0604:01zalkyI'm assuming this is a bug?#2018-02-0605:20Alex Miller (Clojure team)I would expect it to fail in both, surprised it doesnā€™t in clojure #2018-02-0607:51bmaddyHmm, what am I missing here?
cljs.user> (require '[cljs.spec.alpha :as s])
nil
cljs.user> (require '[cljs.spec.gen.alpha :as gen])
nil
cljs.user> (gen/generate (s/gen int?))
#object[Error Error: Var clojure.test.check.generators/simple-type-printable does not exist, clojure.test.check.generators never required]
Is there somewhere I should be looking for an example of how to do this in cljs? I couldn't find anything on http://clojurescript.org or in the official spec guide.
#2018-02-0608:00bmaddyOh, and here's part of my :dependencies:
[org.clojure/clojure "1.9.0"]
[org.clojure/clojurescript "1.9.946"]
[org.clojure/test.check "0.9.0"]
#2018-02-0609:08bronsait's not throwing in clojure because asserts are disabled by default#2018-02-0609:08bronsa>>>Can be disabled at either compile time or runtime: If \compile-asserts\ is false at compile time, compiles to x. Defaults to value of 'clojure.spec.compile-asserts' system property, or true if not set. If (check-asserts?) is false at runtime, always returns x. Defaults to value of 'clojure.spec.check-asserts' system property, or false if not set. You can toggle check-asserts? with (check-asserts bool)#2018-02-0615:04zalky@alexmiller: are you sure? In my example, the ::t spec, (s/keys :req [:ns/y])) does not specify anything about the :ns/x key. But it seems to be applying it anyways, simply by virtue of it being defined globally. This would seem to defy the point of :req. Maybe I'm misunderstanding how keys was meant to be used, but I would have thought the clojure version to be correct.#2018-02-0615:05Alex Miller (Clojure team)s/keys is intended to check all registered keys in the map, regardless of whether they are listed in the spec#2018-02-0615:09zalkyah, thanks for clarifying, but then what is the point of :req? Shouldn't you only need :opt?#2018-02-0615:12Alex Miller (Clojure team)req defines the keys that are required (must be present)#2018-02-0615:12Alex Miller (Clojure team)req is about membership#2018-02-0615:13Alex Miller (Clojure team)so itā€™s really checking a property of the containing map#2018-02-0615:13zalkyhmm, but if all keys are just going to be globally applied whether they are in the spec or not, wouldn't that mean all keys a implicit req, unless specified by :opt?#2018-02-0615:13Alex Miller (Clojure team)validating values according to keys is about checking properties of attributes#2018-02-0615:13Alex Miller (Clojure team)no, it does not mean that#2018-02-0615:13Alex Miller (Clojure team)theyā€™re not required, but if they are present, they are checked#2018-02-0615:29zalkyHmm, ok, thanks for the clarification. Is the design assumption that entity attributes should conform the same properties in all contexts? You should never have an entity attribute conform one way in one context, and another way in another context?#2018-02-0615:35Alex Miller (Clojure team)correct#2018-02-0615:35Alex Miller (Clojure team)https://clojure.org/about/spec#2018-02-0618:08zalky@alexmiller: thanks for the responses, the doc was very helpful. My use case was that I was playing with a simple FSM that brought an entity map through one or more workflows. At first spec seemed to lend itself well to defining the transitions in the FSM. For example, an entity with any number of votes is a valid entity, but once it gets a certain number of votes, it transitions to another state in the FSM. Unfortunately having one valid global definition for an attribute seems to place significant restrictions on defining multiple validity states for an entity. Am I missing an easier way to do this with spec, or is it just that spec is not the right tool for this job?#2018-02-0618:40Alex Miller (Clojure team)I find itā€™s always useful to go back to: what is the truth about this attribute? if I see it in data, what does it mean?#2018-02-0618:40Alex Miller (Clojure team)if it has several possible kinds of values, you can use s/or to specify those alternatives - then youā€™re stating the truth#2018-02-0618:41Alex Miller (Clojure team)s/multi-spec can be used to choose different combinations of attributes at different points in the fsm#2018-02-0618:41Alex Miller (Clojure team)or perhaps you may want to reconsider whether itā€™s really the same attribute at every step#2018-02-0618:41Alex Miller (Clojure team)and in that case, maybe the attribute should change#2018-02-0618:45Alex Miller (Clojure team)to be clearer, maybe there is really more than one kind of attribute#2018-02-0618:46Alex Miller (Clojure team)or another option is donā€™t spec that attribute - youā€™re expecting it to change over time#2018-02-0618:46Alex Miller (Clojure team)you can still spec functions that operate on that attribute more precisely#2018-02-0619:22zalkyYeah, I was thinking the same thing about the attribute, there are probably times where it indicates that it is really two attributes. However this isn't always the case. The simple example I gave above with the voting is probably a case where splitting would unnecessarily complicating the data model because of an assumptions in the implementation. I think you're suggestion to not spec the attribute via the entity model, and just spec it separately using custom primitives (at least that is how interpreted your last sentence) is probably the one that makes the most sense . Unfortunately you lose some of the great higher order features of the entity model, like being able to define one transition as a merge of previous transitions and some new criteria. One final question: do you think that allowing local bindings in keys would break spec's entity model significantly? Something like:
(s/def ::entity
  (s/keys :req [::attr]
          :let [::attr #(<= 10 (count %))]))
#2018-02-0620:23Alex Miller (Clojure team)yes and you donā€™t need it#2018-02-0620:24Alex Miller (Clojure team)(s/and (s/keys :req [::attr]) #(<= 10 (count (::attr %))#2018-02-0621:12zalkyThanks alex for you time (and your contributions on spec!)#2018-02-0718:55Joe LaneIf the above snippet is OT and belongs somewhere else, Iā€™ll gladly take it there instead.#2018-02-0720:36bbrinck@lanejo01 Would it work to just remove the namespace from the keys in args? e.g. (into {} (map (fn [[k v]] [(keyword (name k)) v])) {:foo/bar 1 :foo/baz 2}) ;; => {:bar 1, :baz 2}#2018-02-0720:37bbrinckIf you didnā€™t want to remove the namespace from all keys, you could pass in a specific prefix to remove in this case, :myapp.entity.user#2018-02-0720:41Joe LaneInteresting, yeah I could do that to just keep the unqualified keys. Im wondering if iā€™m missing some design step though since from the jira ticket it was concluded that there wasnā€™t planned work for namespaced keys in records.#2018-02-0721:33Alex Miller (Clojure team)I didnā€™t read everything above, but have you tried using s/keys with :req-un to spec records#2018-02-0722:22Joe LaneAhh, I think thats the answer @alexmiller. I had read through that section of the spec guide probably 15 times last night but it didnā€™t click. This also pushes me away from desiring namespaced keywords in records. Thank you!#2018-02-0722:38seancorfield@alexmiller On that subject... given the general advice re: maps / records is usually "use maps, and only switch to records if you need extra performance", how does that sit with using namespaced maps -- since those cannot easily be switched to records?#2018-02-0722:38Alex Miller (Clojure team)it complicates it :)#2018-02-0722:38seancorfieldšŸ˜† but not helpful!#2018-02-0722:39Alex Miller (Clojure team)I mean if you have maps with namespaced keys, you can switch to records and change specs from :req to :req-un#2018-02-0722:39seancorfieldFair enough. The spread of namespaced maps does seem to reduce the attraction of using records further.#2018-02-0722:39Alex Miller (Clojure team)unfortunately, I agree. I seem to be in the minority, but I like records. :)#2018-02-0722:39seancorfield@alexmiller Yeah, but all of your code that manipulates those maps would need to change too -- since it will be using :foo/bar now and would need to change to :bar.#2018-02-0722:39Alex Miller (Clojure team)true#2018-02-0722:40Alex Miller (Clojure team)Iā€™ve had a few conversations with RH about this#2018-02-0722:40seancorfieldI'll be interested to hear what he decides in the end šŸ™‚#2018-02-0722:40Alex Miller (Clojure team)records actually do have a namespace context via the package of the defining record type#2018-02-0722:41Alex Miller (Clojure team)seems like there should be a good way to leverage that#2018-02-0722:41Alex Miller (Clojure team)like field a in record my.R could use spec :my/a if reqā€™ed in a s/keys#2018-02-0722:42Alex Miller (Clojure team)but that doesnā€™t really address the access difference#2018-02-0722:42Alex Miller (Clojure team)would also need to allow lookup by nsā€™ed key in the record, which maybe would be interesting#2018-02-0722:43seancorfieldPerhaps an option on defrecord to produce either plain keys or namespaced keys? I assume the / could be munged to something Java would accept?#2018-02-0722:43Alex Miller (Clojure team)meh#2018-02-0722:43seancorfieldHahaha šŸ™‚ I suspected that might be your response!#2018-02-0722:43Alex Miller (Clojure team)I think adding new access mechanisms is greatly preferable to adding options#2018-02-0722:44noisesmithitā€™s pretty ugly when munged
user=> (munge "x/y")
"x_SLASH_y"
#2018-02-0722:46Alex Miller (Clojure team)optional behavior flags either means youā€™re being lazy or havenā€™t thought about it long enough :)#2018-02-0722:48Alex Miller (Clojure team)probably better to improve how a record spec would link fields to attribute specs in a way that supports more kinds of code evolution#2018-02-0814:33justinbarclayIā€™m learning about cljs.spec, but when I try to run the code below in a figwheel repl it times out. Does anyone know why or how I can improve my spec?
(defn split-in-half
  "Splits a collection in half"
  [coll]
  (split-at (Math/round (/ (count coll) 2)) coll))

(spec/fdef split-in-half
           :args (spec/cat :col coll?)
           :ret  (spec/tuple coll? coll?))

(stest/abbrev-result (first (stest/check `split-in-half))
#2018-02-0816:28mpenetExpected, merge doesn't flow conformed values. This one comes up quite often#2018-02-0816:28mpenetThere s a (closed) jira issue about this#2018-02-0921:11justinleeso from the department of This Canā€™t Be Happening, I speced out a reagent component with fdef and instrumented it, and now it actually calls one of the callback-handlers that is passed to the component as part of the instrumentation process. is this expected/possible? or am i misunderstanding? in the code below, the on-cancel callback gets called 21 times if I leave the instrument call in the code
(spec/def ::name string?)
(spec/def ::class string?)
(spec/def ::type string?)
(spec/def ::*value #(instance? reagent.ratom/RAtom %))
(spec/def ::on-cancel (spec/nilable (spec/fspec :args (spec/cat))))
(spec/def ::focus? boolean?)
(spec/fdef input
 :args (spec/cat
        :props (spec/keys
                 :req-un [::type ::*value]
                 :opt-un [::class ::focus? ::on-cancel]))
 :ret any?)

(stest/instrument `input)
#2018-02-0921:14Alex Miller (Clojure team)fspec args are validated by genā€™ing from the :args spec and invoking the function, then checking the :ret spec#2018-02-0921:15Alex Miller (Clojure team)some people have found this to be surprising :)#2018-02-0921:16Alex Miller (Clojure team)probably the easiest ā€œfixā€ is to replace (spec/fspec :args (spec/cat)) here with ifn?#2018-02-0921:17justinleethe good news is iā€™m not crazy šŸ™‚#2018-02-0921:18justinleesometimes i suspect iā€™m using spec improperly. i really just want an assertion library to catch my typos#2018-02-0921:23justinleeso is it ever possible/safe to use fspec with a side-effecting function? i donā€™t see how it could ever work unless you passed it a pure function#2018-02-0921:24Alex Miller (Clojure team)I would not advise it#2018-02-0921:25Alex Miller (Clojure team)passing a side-effecting function that is#2018-02-0921:25Alex Miller (Clojure team)there are ways to (for example) swap out an alternate spec when you instrument - those are optional capabilities of instrument#2018-02-0921:26Alex Miller (Clojure team)but Iā€™m not sure youā€™re getting much value than bother out of having the fspec at that point#2018-02-0921:26mishaTIL about stubbing in s/instrument#2018-02-0921:26Alex Miller (Clojure team)well Iā€™m talking about replace, not stub, but yeah#2018-02-0921:27Alex Miller (Clojure team)stubbing is great for stubbing out a remote call in instrument#2018-02-0921:28mishaif I'd commit fdef + instrument-with-stub instead of actual implementation, I wonder how quickly my teammates will notice in production? kappa#2018-02-0921:32Alex Miller (Clojure team)oh, pretty quick Iā€™d guess#2018-02-0921:32Alex Miller (Clojure team)unless you wrote some really good generators#2018-02-0921:32mishanumber? troll#2018-02-1018:16arohnerIā€™m having trouble generating from a recursive spec, I keep hitting StackOverflowError. Are there any docs/guides on how to do that?#2018-02-1018:16arohnerIā€™m doing:
(s/def ::a a?)
(s/def ::b b?)
(s/def ::c (s/with-gen c? (gen/fmap #(make-c %) (s/or :a ::a :b ::b :c ::c)))
#2018-02-1018:19arohnerIā€™ve also tried t.c.gen/recursive-gen#2018-02-1018:25taylorhard to say because the example is missing several definitions but you could try (binding [s/*recursion-limit* 1] ...)#2018-02-1018:30arohnerThat helped, in that gen/sample occasionally finishes, but still stackoverflows some of the time. It surprised me that in the first 10 items, it produces nested cā€™s 5 deep, which I wouldnā€™t expect with a recursion limit of 1#2018-02-1018:34tayloryeah I think thereā€™s been a JIRA ticket w/some discussion about this, canā€™t find it at the moment#2018-02-1018:37arohnerhttps://dev.clojure.org/jira/browse/CLJ-1978#2018-02-1018:52taylor@arohner also this discussion https://clojurians-log.clojureverse.org/clojure-spec/2016-08-17.html#inst-2016-08-17T19:58:22.002442Z#2018-02-1019:08arohnerI think I can work around it using gen/frequency#2018-02-1019:09arohneri.e. (gen/frequency [[9 (s/or :a ::a :b ::b)] [1 (s/or :a ::a :b ::b :c ::c]]) #2018-02-1019:42arohnerfrequency helps, but doesnā€™t completely solve the problem#2018-02-1020:20andy.fingerhutA transcript of a 2-hour talk by Rich Hickey from Dec 2016 about clojure.spec is up now: https://github.com/matthiasn/talk-transcripts/blob/master/Hickey_Rich/ClojureSpec.md#2018-02-1020:20andy.fingerhutIt doesn't go deep into the details -- more of an intro, and big picture of ways the pieces of spec can be used.#2018-02-1200:42Vincent CantinThe URL in the sticky note of this channel seems broken : https://clojure.github.io/clojure/branch-master/clojure.spec-api.html#2018-02-1200:46Vincent CantinThe correct URL may be https://clojure.github.io/spec.alpha/ ?#2018-02-1201:29Alex Miller (Clojure team)Yes, that last url is the correct one. Earlier, spec was in clojure and the prior one was correct#2018-02-1201:55Vincent Cantinbeginner question: Can we use spec to validate the correlation between values at different places in a structure?#2018-02-1201:56tayloryes, are you thinking maps or another type of structure?#2018-02-1201:56Vincent CantinFor example, if I want to write a spec for [a b 1 5 ... 3 a b] where a and b could be anything.#2018-02-1201:59Vincent Cantin... supposing that the structure is complex enough so that I would need to use s/conform to match the location of a and b at different places.#2018-02-1202:00Vincent CantinI could use a classical function to check if they match at different places, but I would like to know if there is a better spec-ish way to do it. Something like declaratively defining the pattern.#2018-02-1202:03Vincent CantinFor example, something like: "For any a, b : [a b ... a b] is valid."#2018-02-1202:08taylorsomething like this?
(s/def ::my-coll
  (s/* (s/alt :a-b (s/cat :a #{'a} :b #{'b})
              :num number?)))
(s/conform ::my-coll ['a 'b 1 2 3 'a 'b])
#2018-02-1202:26taylorOr if you wanted to require the coll always begin and end with a b:
(s/def ::a-b (s/cat :a #{'a} :b #{'b}))
(s/def ::my-coll (s/cat :a-b-start ::a-b
                        :nums (s/* number?)
                        :a-b-end ::a-b))
#2018-02-1202:50Vincent CantinThank you. I will try it again later tonight.#2018-02-1214:31Jakub HolĆ½ (HolyJak)Is there a way to manually conform function arguments before I call (apply myfn args)? I.e. having
(spec/fdef myfn :args (spec/cat :userid ::userid))
how do I check that the args I have conform to that? Thanks! (I want to use this with expound. I have command functions invoked from a Slack bot and want to verify that the command the user typed has all the correct arguments and show a nice error if not)
#2018-02-1214:34mpenetmake a separate spec of the :args and use s/valid? on it ?#2018-02-1214:35mpenetyou can then write (s/fdef myfn :args ::foo-args ... )#2018-02-1214:55mpenetwondering: why wasn't check-asserts based on a dynamic var like *compile-asserts*? Right now it's a bit all or northing at runtime for assertion checking (for the one that are compiled at least)#2018-02-1214:57mpenetit's easy enough to work around it anyway#2018-02-1215:13Alex Miller (Clojure team)all or nothing is kind of the typical modes for assertions#2018-02-1215:14Alex Miller (Clojure team)you can also grab the :args spec of a specā€™ed function with
(-> `myfn s/get-spec :args)
#2018-02-1216:51jumarI'm wondering if I can do any better if I want to have two different specs for the same key... I have a generic auth-provider , e.g. #:auth-provider{:type "ldap" :config {:host "abc" ...}} This has a generic spec :
(s/def :auth-provider/config (s/map-of keyword? any?))
...
(s/def ::auth-provider-spec (s/keys :req [:auth-provider/type
:auth-provider/active?]
:opt [:auth-provider/id
:auth-provider/default-role
:auth-provider/config
:auth-provider/priority-order
:auth-provider/role-mapping]))
which is fine for some generic application code. However, in some scenarios I want to act differently based on the type of the auth-provider. E.g. for "ldap" I know that certain keys have to be present inside :auth-provider/config. I couldn't find a better solution than this:
(s/def ::ldap-provider-spec (s/and
::auth-specs/auth-provider-spec
;; this is less descriptive than using `s/keys` directly
;; but we cannot use `s/keys` because `:auth-provider/config` default
;; spec is already registered in auth-specs namespace
;; and you cannot have two different specs for the same namespaced key
#(s/valid? ::ldap-config (:auth-provider/config %))))

My problem is that since specs are global I cannot register two different specs for the same key (`auth-provider/config`). I also tried multi-spec but couldn't solved it either. My solution works but it's quite opaque - the error message just says that data fails ::ldap-config spec (not that e.g. :host key is missing in config map)
#2018-02-1218:04mishalook again at multi-spec, @jumar. it would be verbose solution, but sounds like it fits.#2018-02-1218:27jumar@U051HUZLD well, my problem was that I still couldn't define two different specs for :auth-provider/config key. Basically, based on the value of :type key I'd implement two multimethods - one for the :type value "ldap" and one for :default. But in both of them I'd need to use something like (s/keys :req [:auth-provider/config]) - and here I'm stuck. I don't know how to define two different specs for :auth-provider/config key.#2018-02-1218:12mishaalthough, it'd be much easier, if you'd put :auth-provider/type inside config itself#2018-02-1218:46mishaif you put type inside config itself - multispecs will be about config keys, net provider's ones#2018-02-1218:48mishaor you can just s/or the :provider/config, but it will not keep :provider/type and or's branches "in sync"#2018-02-1221:42roklenarcichow do you guys spec a map where keys are generic and values are constant for a type? To give you an example, clojure.xml will output something like {:tag :first_name, :attrs nil, :content ["John"]}. How would I spec that I want my ::first-name to have this structure? The keys function would require me to make a spec for ::tag but the value is constant for ::first-name and at the same time, different in another spec, I can't globally specify ::tag spec.#2018-02-1221:55seancorfield@roklenarcic I suspect you'll want a multi-spec for the various types of parsed maps you can get back from clojure.xml#2018-02-1221:55seancorfield(and the multi-spec would branch on the :tag key's value)#2018-02-1222:09roklenarcicI'll look into it.#2018-02-1222:12roklenarcicThere's something weird with the large-integer generator. As is, it never generates a number over a 100. Even if I use the large-integer* generator with options, the numbers are very low.
(gen/sample (gen/large-integer* {:min 1 :max 1000000}))
=> (2 2 1 4 1 1 9 1 2 15)
#2018-02-1222:20roklenarcicThe test.check generator clojure doc states:
(def ^{:added "0.9.0"} large-integer
  "Generates a platform-native integer from the full available range
  (in clj, 64-bit Longs, and in cljs, numbers between -(2^53 - 1) and
  (2^53 - 1)).
  Use large-integer* for more control."
  (large-integer* {}))
But in my experience it generates number in about a -100 +100 range instead of 2^64
#2018-02-1222:56taylortry generating a larger sample#2018-02-1222:56taylor(gen/sample (gen/large-integer* {:min 1 :max 1000000}) 1000)
#2018-02-1222:58roklenarcichm... I guess I expected that generated numbers were uniformly distributed and so it would be extremely likely that a number over a 1000 was generated in 10 tries#2018-02-1223:02gfredericks@roklenarcic the large-integer generator exhibits growth just like most of the other generators#2018-02-1223:03gfredericksmeaning it respects the size parameter#2018-02-1223:03gfredericksgen/sample with 1 arg gives you samples using sizes 0ā‡’9#2018-02-1223:08roklenarcicI see, thank you#2018-02-1223:13roklenarcicThe reason I was worried is that when I use integer generator to generate ids, I get a lot of non-unique small numbers (because of the small initial range), which makes it very likely for a sequence of generated entities to have non-unique IDs, which causes such-that to fail occasionally, when IDs are specced to be unique.#2018-02-1301:35gfredericksthere are generators for uniqueness, like gen/vector-distinct-by; and when it has trouble finding different IDs, it increases the size, which should be sufficient here#2018-02-1309:11Jakub HolĆ½ (HolyJak)Is there a way to derive a spec from a spec? We have a checkout flow that in each step adds something to an order and a spec for the final order. I would like to have a spec for each step of the checkout process (ie the final spec with some keys either removed or changed from required to optional). The problem is that order is not a flat map and may itself contain maps that grow across multiple steps. I want to avoid copy&paste&modify of the end spec. Any ideas? Thanks!#2018-02-1309:22gklijsYes, you can do things like
(def label (s/and (s/spec string?) #(> (count %) 2) #(< (count %) 40)))
(s/def ::nl-label label)
(s/def ::label (s/keys :req [::nl-label]))
not sure if it helps, but thatā€™s how I prevented a lot of copy-pasting
#2018-02-1309:31gklijsFor specs of maps there is also merge, which seems more use-full re-reading you questionhttps://clojuredocs.org/clojure.spec.alpha/merge#2018-02-1312:24stathissiderissounds like heā€™d need deep-merge (which doesnā€™t exist afaik)#2018-02-1313:30slipsetOh, and while on the topic of merge#2018-02-1313:30slipsethttps://dev.clojure.org/jira/browse/CLJ-1981#2018-02-1313:31slipsetI'm quite disappointed that this is closed as "working as intended" with a#2018-02-1313:31slipsetdocstring fix#2018-02-1313:31slipsethttps://github.com/clojure/clojure/commit/d920ada9fab7e9b8342d28d8295a600a814c1d8a#2018-02-1313:32slipsetMaybe I just need to understand the rationale behind why this is intentional behaviour?#2018-02-1313:53vikeriAny way of overriding the generators so that different specs depend on each other? My use case is testing a function where the arguments have internal relations, i.e. argument 1 contains references to argument 2. Randomly generated arguments will not have this relation naturally.#2018-02-1314:02Alex Miller (Clojure team)independently, no. but in a shared context like an args list, yes - you can use combinators like fmap or bind to achieve this#2018-02-1410:36vikeriI see!#2018-02-1417:45firstclassfunchey @alexmiller still a little fuzzy here. Trying to build a generator that produces a common ::id field as in
(defrecord Subscriber [name id])
(defrecord Subscribed [article id])

(s/def ::name string?)
(s/def ::article uuid?)
(s/def ::id uuid?)

(s/def ::Subscriber
  (s/keys :req-un [::name ::id]))

(s/def ::Subscribed
  (s/keys :req-un [::article ::id]))


(defn generate-subscribers
  "Mock function to generate subscribers"
  []
  (gen/sample (s/gen ::Subscriber)))


(defn generate-subscribed
  "Mock function to generate subscribed"
  []
  (gen/sample (s/gen ::Subscribed)))


#2018-02-1417:47Alex Miller (Clojure team)sorry, donā€™t have time to reply today but recommend: https://www.youtube.com/watch?v=WoFkhE92fqc as an intro#2018-02-1321:39misha@slipset if you will flow conformed values through merge's "components" - you will hit false-positive invalidation.#2018-02-1321:46mishacompare:
(s/def :foo/a (s/or :s string? :i int?))
(s/def :m/one (s/keys :req [:foo/a]))
(s/def :m/two (s/keys :req [:foo/a]))
(s/def :m/merged (s/merge :m/one :m/two))

;; not flowing conformed value
(let [m {:foo/a 1}]
  (and  ;; not accurate, but illustrates the idea
    (s/conform :m/one m)
    (s/conform :m/two m)))  ;; => #:foo{:a [:i 1]}

;; flowing conformed value
(let [m {:foo/a 1}]
  (->> m
    (s/conform :m/one)
    (s/conform :m/two))) ;; => :clojure.spec.alpha/invalid
#2018-02-1321:47misha(I hope @alexmiller will call me on my BS, if it's crazy talk)#2018-02-1408:27slipset@misha ha I don't quite think this is my use case. I might very well be misunderstanding here, but you're not using :m/merged at all#2018-02-1408:29slipsetAnd I guess what surprises me here is that the order of how the specs are passed to s/merge is important.#2018-02-1408:30slipsetI guess you could argue that since the order of how maps are passed to clojure.core/merge is important, it follows that the order of the specs passed to s/merge is also important, but I have a hard time accepting that šŸ™‚#2018-02-1408:40misha@slipset but you do accept clojure.core/merge, don't you? :)
(merge {:a 1} {:a 2})  ;;=> {:a 2}
(merge {:a 2} {:a 1})  ;;=> {:a 1}
#2018-02-1408:40slipsetabsolutely#2018-02-1408:41mishaif you s/merge s/keys with req/opt - order should not be an issue, but it might, if you merge req-un/opt-un#2018-02-1408:42slipsetWhich I find surprising, since this is not mentioned in the doc-string, and I don't see why there should be a difference.#2018-02-1408:43mishafirst s/keys will have :foo/a, seconds ā€“ :bar/a, which will be conformed differently#2018-02-1408:44mishathen, conformed values will be merged, and the last one will win#2018-02-1408:47misha
(s/def :foo/a (s/or :i int?))
(s/def :bar/a int?)
(s/def :foo/m (s/keys :req-un [:foo/a]))
(s/def :bar/m (s/keys :req-un [:bar/a]))

(s/conform (s/merge :foo/m :bar/m) {:a 1})  ;;=> {:a 1}
(s/conform (s/merge :bar/m :foo/m) {:a 1})  ;;=> {:a [:i 1]}
#2018-02-1410:24Jakub HolĆ½ (HolyJak)If only specs were data and I could use Spectre to transform them...#2018-02-1410:28mpenet@holyjak there are plans to improve that I think#2018-02-1410:28mpenetnothing specific was mentioned but they stated the intent a few times#2018-02-1410:34Jakub HolĆ½ (HolyJak)Crazy idea: perhaps I could only have a spec for the final order and, when validating for a non-final step of the checkout, I could just filter out exceptions for paths I know this step doesnt require yet. #2018-02-1410:39mpenetmy comment was about spec composition, not about the merge oddities#2018-02-1410:39mpenetfyi#2018-02-1410:39vikeriSo, if I run a spec with 100 tests and it fails, if I input the seed number it will only also fail if I run 100 tests? The seed doesnā€™t seem to be for the single test that fails since if I change num-tests to 1 it will usually pass. This decreases the usefulness of seed for functions that take large data structures and take some time to run.#2018-02-1413:32taylorseeding a RNG just makes it generate a certain sequence of values, it doesnā€™t mean itā€™s going to magically find the exact value in the sequence that triggered some behavior and then always generate that number first. I assume the same logic applies to test.check generators?#2018-02-1413:33tayloranyway, when the check fails it spits out a minimal failing case that you could use in a ā€œmanualā€ test#2018-02-1414:38Alex Miller (Clojure team) assuming you donā€™t use an external source of randomness in your generators , re-using the seed from a test should reproduce the identical generator values and the identical failure#2018-02-1414:40taylortrue, I think the disconnect is expecting the failing case to be re-genā€™d first when using the same seed, instead of at whatever position it naturally occurs w/given seed#2018-02-1414:42Alex Miller (Clojure team)right#2018-02-1413:47misha@holyjak specs are sort of data, like everything in clojure opieop #2018-02-1413:48mishaNow it just comes down to how much pain verbosity and macros you can tolerate#2018-02-1413:48mishas/form#2018-02-1415:53mishais there a builtin predicate for int in Integer/MIN_VALUE, Integer/MAX_VALUE range?#2018-02-1416:10madstap@misha I think that's what int? is#2018-02-1416:17mgrbyte(source int?) - is a check against type (Long/Integer/Short/Byte), not bounds.#2018-02-1416:17misha@madstap int? includes longs as well#2018-02-1416:24mishaguess I'll use (s/int-in 0 (inc Integer/MAX_VALUE))#2018-02-1416:26Alex Miller (Clojure team)isnā€™t that nat-int? ?#2018-02-1416:26Alex Miller (Clojure team)I guess nat-int? goes to Long/MAX_VALUE#2018-02-1416:30misha
(nat-int? Long/MAX_VALUE)        ;;=> true
(nat-int? (inc Long/MAX_VALUE))  ;;=> java.lang.ArithmeticException: integer overflow
#2018-02-1416:32Alex Miller (Clojure team)well the error is occurring on the inc there, not the nat-int?#2018-02-1416:32misha
Long/MAX_VALUE
=> 9223372036854775807
(nat-int? 9223372036854775808) ;;=> false
#2018-02-1416:32mishayes -_-'#2018-02-1416:34Alex Miller (Clojure team)if you really want to restrict to java Integer ranges, I think I would make custom predicates and custom specs that use those predicates + appropriate generators#2018-02-1416:35Alex Miller (Clojure team)but Clojure is not going to give you that as it does not intend to support them#2018-02-1416:38mishaI am going through kafka's config documentation and writing spec for it. Sometimes they use int, [0, ...], and sometimes int, [0, ... 2147483647] explicitly#2018-02-1417:09Alex Miller (Clojure team)ah#2018-02-1420:34firstclassfuncis there anyway to force say a large-integer generator to produce unique values?#2018-02-1420:36taylor@firstclassfunc this might apply#2018-02-1420:36gfredericksyeah you have to specify it at the higher level, where you know the scope of uniqueness#2018-02-1420:37gfredericksotherwise you'll have trouble e.g., during shrinking when all your IDs become 0#2018-02-1420:38firstclassfuncyea i just want to produce a set of integers numbered 1-100 without duplicates#2018-02-1420:39gfredericks(gen/shuffle (range 100))? (gen/set (gen/large-integer* {:min 1 :max 100}))?#2018-02-1420:46firstclassfuncthanks @gfredericks not quite the shape I need it yet but appreciate it!#2018-02-1420:47gfrederickssometimes another useful approach is to just remove duplicates#2018-02-1420:48firstclassfuncyea that would help I am basically trying to create a common index across records e.g.
(defrecord Subscriber [name id])
(defrecord Subscribed [article id])

(s/def ::name string?)
(s/def ::article uuid?)
(s/def ::id uuid?)

(s/def ::Subscriber
  (s/keys :req-un [::name ::id]))

(s/def ::Subscribed
  (s/keys :req-un [::article ::id]))


(defn generate-subscribers
  "Mock function to generate subscribers"
  []
  (gen/sample (s/gen ::Subscriber)))


(defn generate-subscribed
  "Mock function to generate subscribed"
  []
  (gen/sample (s/gen ::Subscribed)))
#2018-02-1420:49firstclassfuncI thought I would just narrow the scope of the generator for ::id to do it, but that has been more challenging.#2018-02-1420:54gfredericksif your IDs are UUIDs you shouldn't have any problems actually#2018-02-1420:59firstclassfuncthe problem is that I need the set of ids to be the same across each record to form a relationship#2018-02-1421:01gfredericksah yeah that kind of thing takes more effort#2018-02-1615:44vikeriHmm, I canā€™t generate positive numbers in clojurescript. (s/exercise pos?) doesnā€™t work. Any pointers?#2018-02-1616:20Alex Miller (Clojure team)Pos is not specific to a particular numeric type so does not have a mapped generator#2018-02-1616:20Alex Miller (Clojure team)pos-int? does#2018-02-1616:21Alex Miller (Clojure team)Or create a composite like (s/and double? pos?) where the initial type will gen#2018-02-1705:03bmaddyIs there a way to ensure two values are the same in a structure with spec? I didn't see anything in the spec docs (but maybe I missed it). I'd like to define a spec :account/by-id that specs data like this:
{-5 {:db/id -5 :account/title "customer five"}}
Here's what I've got so far:
(s/def :db/id (s/or :db/id int? :uuid uuid?))
(s/def :account/title string?)
(s/def ::account (s/keys :req [:db/id :account/title]))
(s/def :account/by-id (s/map-of :db/id ::account))
but gen/generate gives me stuff like this:
{7 {:db/id -5 :account/title "customer five"}}
#2018-02-1705:04bmaddyI'd like to ensure the :db/id is the same as the associated key in the outer map.#2018-02-1705:42seancorfield@bmaddy look at s/and to add a constraint#2018-02-1705:44seancorfield
(s/def :account/by-id (s/and (s/map-of :db/id ::account) db-id-matches))
where db-id-matches checks that the values match
#2018-02-1802:39bmaddyThanks @seancorfield, that's what I was missing!#2018-02-1712:41gfredericksthen you'll have to modify the generator too, to just copy the id from the key spot to the attribute spot#2018-02-1716:15roklenarcicI'm trying to add clojure.spec.test.alpha/check based testing to my clojure.test tests, but the integration doesn't seem trivial. Do I have to write my own transform of the check map to some sensible assert or is there a lib for that out already?#2018-02-1716:23gfredericksthat's asked a lot; I don't know if anybody's published anything. but in any case doing the integration manually should be just barely nontrivial#2018-02-1716:23roklenarcicI mean I can whip up something. I have to note that one thing that is very confusing is the :failure key in the return map#2018-02-1716:24roklenarcicit comes back set to false#2018-02-1716:24roklenarcicon a failed check#2018-02-1716:24roklenarcicthis will probably confuse a lot of people#2018-02-1716:30gfredericksyes there's a ticket about that#2018-02-1716:45roklenarcicThere seems to be a curious mix of namespaces in the returned maps as well: To get predicate that failed you need to get:
(-> (stest/check `fnsym)
  :clojure.spec.test.check/ret
  :result-data
  :clojure.test.check.properties/error
  (.getData)
  :clojure.spec.alpha/problems
  (get 0)
  :pred)
Goes from spec to test.check back to spec keys.
#2018-02-1716:46gfredericksyeah, I think spec is currently embedding data in an ExceptionInfo object because test.check didn't used to have a mechanism for adding any info to a failure#2018-02-1716:46gfredericksI thought it was being unwrapped somehow though#2018-02-1721:40mathpunkI'm trying to do spec-and-test driven development in ClojureScript, and specs are not resolving like I expect them to. This test (https://github.com/mathpunk/sherman/blob/master/test/sherman/grammar_test.cljs#L13) passes if I uncomment the specification above it, but fails as is, with the specification in another namespace (https://github.com/mathpunk/sherman/blob/master/src/sherman/grammar.cljs#L7)#2018-02-1723:00seancorfield@mathpunk Your test namespace does not require the namespace containing the spec -- how would it know about it?#2018-02-1723:12mathpunk@seancorfield thanks! Since the name of the spec has the namespace in it, I didn't realize the test namespace still needed to require it#2018-02-1723:12seancorfieldYou need to load the namespace for the s/def to be executed, otherwise the spec isn't defined -- the keyword is just a keyword.#2018-02-1723:13seancorfieldAlso, spec names (keywords) have no connection to code namespaces (except insofar as the :: alias resolution works).#2018-02-1723:19mathpunkwow! ok, I thought they were somehow being 'registered' in a way that needed to match the namespaces#2018-02-1723:23seancorfields/def and s/fdef are the functions that perform the registration and the spec name just needs to be as globally unique within your application as it needs to be in order to not clash with any other specs.#2018-02-1723:25seancorfieldNamespace-qualified keywords have been around for a long time before spec and have never been tied to code namespaces. For example, we use :ws.web/config as a key in Ring requests in our applications for our application configuration map -- but we don't have a ws.web namespace.#2018-02-1723:26seancorfieldYou can also set up aliases without needing code namespaces to match:
(alias 'm  (create-ns 'ws.domain.member))
(alias 'mu (create-ns 'ws.domain.member.update))
(alias 'mv (create-ns 'ws.domain.member.validation))
The first one is used for specs (`::m/username` for example), the other two are just used as part of unique keys in maps.
#2018-02-1723:29seancorfield(it just so happens we do have code namespaces matching those, but where we use the aliases, we don't need the functions or specs from those namespaces so we don't require the namespaces)#2018-02-1723:29seancorfield(hope that helps @mathpunk?)#2018-02-1723:33mathpunk@seancorfield Thank you, it does#2018-02-1801:30mathpunkI'm writing my first regular expression of specs. Both these tests should pass:#2018-02-1801:32mathpunkA :sherman.grammar/expanding-term should contain ONE or more :sherman.grammar/expanding-symbols, and ZERO or more :sherman.grammar/terminating-symbols#2018-02-1801:32mathpunkI thought this might be it:#2018-02-1801:33mathpunkAt least, I thought that would handle "This is #going# #to# #expand# maybe"#2018-02-1801:33mathpunkwhich isn't fully what I'm after but would be partway#2018-02-1801:33taylorhave you tried s/alt#2018-02-1801:33mathpunkI diiiid but not very well#2018-02-1801:35mathpunkCome to think of it I don't know that my input data is regular#2018-02-1801:36mathpunkI'm trying to express, "A string that, if you split it at the spaces, one or more of the elements of that collection would be an ::expanding-symbol"#2018-02-1801:37mathpunkMaybe I should write a function that tests that, and uses it as the predicate#2018-02-1801:38mathpunkI wanted to check here to see if there is a way to alt this intent#2018-02-1801:41taylorseems like s/cat works for your case unless Iā€™m missing something#2018-02-1801:43tayloroh wait, you want your inputs to be actual strings?#2018-02-1801:44noisesmiths/alt could spec the result of splitting the string at spaces#2018-02-1801:44taylorin that case, I think Iā€™d tokenize the string before specā€™ing it#2018-02-1801:53mathpunkGetting there. I'm incorrect about what s/+ means, though:#2018-02-1802:17mathpunkI've changed my terminology a little, but I'm still failing to reject invalid input:#2018-02-1802:21mathpunkI hypothesize that the right thing to do is to make functions do more work, and specs do less. E.g., much as noisesmith suggested when they suggested tokenizing first, I could have a predicate which detects well-formed tokens, and base a spec off of that#2018-02-1802:22mathpunkI'm still working out what responsibilities belong in spec and what belong in plain ol' clojure code šŸ™‚#2018-02-1803:34taylor@mathpunk another option using s/&:
(s/def ::valid-term
  (s/& (s/* (s/alt :terminating ::terminating-symbol
                   :expanding ::expanding-symbol))
       #(some (comp #{:expanding} first) %)))
(s/conform ::valid-term (tokenize "This #expands# also"))
=> [[:terminating "This"] [:expanding "#expands#"] [:terminating "also"]]
#2018-02-1803:46seancorfieldI'll channel Alex Miller and say clojure.spec is not intended for string parsing -- there are much better tools out that for that (e.g., instaparse).#2018-02-1803:47seancorfieldThe "regular expression" part of spec is intended for sequence processing.#2018-02-1803:48seancorfield^ @mathpunk#2018-02-1803:58mathpunkI've wondered exactly this!: whether instaparse was obsolete after spec#2018-02-1803:59aengelbergIt is not#2018-02-1803:59aengelbergCome join us#2018-02-1803:59aengelbergWe have cookies#2018-02-1803:59aengelbergAnd somewhat lackluster but existent respond rate to issues#2018-02-1804:16bmaddyI would expect clojure.spec.alpha/and to be commutative. Is anyone else seeing this behavior?
> *clojurescript-version*
"\"1.9.946\""
> (defn f [m] (println m) (-> m (get 1) :db/id (= 1)))
"#'invest-calc.state/f"
> (s/def :account/works (s/and f (s/map-of any? (s/keys))))
":account/works"
> (s/def :account/fails (s/and (s/map-of any? (s/keys)) f))
":account/fails"
> (s/conform :account/works {1 {:db/id 1}})
{1 #:db{:id 1}}
"{1 #:db{:id [:db/id 1]}}"
> (s/conform :account/fails {1 {:db/id 1}})
{1 #:db{:id [:db/id 1]}}
":cljs.spec.alpha/invalid"
#2018-02-1804:27Alex Miller (Clojure team)s/and flows conformed results through predicates#2018-02-1804:47bmaddyHeh, I guess I should read the docs closer. Is it expected that it would operate differently in normal Clojure?
invest-calc.state> *clojure-version*
{:major 1, :minor 9, :incremental 0, :qualifier nil}
invest-calc.state> (defn f [m] (println m) (-> m (get 1) :db/id (= 1)))
#'invest-calc.state/f
invest-calc.state> (s/def :account/works (s/and f (s/map-of any? (s/keys))))
:account/works
invest-calc.state> (s/def :account/fails (s/and (s/map-of any? (s/keys)) f))
:account/fails
invest-calc.state> (s/conform :account/works {1 {:db/id 1}})
{1 #:db{:id 1}}
{1 #:db{:id 1}}
invest-calc.state> (s/conform :account/fails {1 {:db/id 1}})
{1 #:db{:id 1}}
{1 #:db{:id 1}}
#2018-02-1901:15arrdemIs there a spec around for the fn form?#2018-02-1901:21arrdemHummmm I'm going about this wrong methinks šŸ˜›#2018-02-1901:27arrdemAh clojure.core/fn is a spec ID.#2018-02-1906:32ikitommithe new spec-tools (`0.6.1`) has a version of merge that retains the already conformed values:
(require '[clojure.spec.alpha :as s])
(require '[spec-tools.core :as st])

(s/def ::a (s/and (s/conformer str) string?))
(s/def ::b int?)

(s/def ::am (s/keys :opt-un [::a]))
(s/def ::bm (s/keys :opt-un [::b]))

(s/conform (s/merge ::am ::bm) {:a 1, :b 1})  ; {:a 1, :b 1}
(s/conform (s/merge ::bm ::am) {:a 1, :b 1})  ; {:a "1", :b 1}

(s/conform (st/merge ::am ::bm) {:a 1, :b 1}) ; {:a "1", :b 1}
(s/conform (st/merge ::bm ::am) {:a 1, :b 1}) ; {:a "1", :b 1}
#2018-02-1912:34igosuki@ikitommi Do you know if there is any way to conform two key aliases into one ? problematic example :
(s/def ::user
  (s/keys [(s/or ::name1 ::name2)])) 
Spec verifies for me already that itā€™s one of the other, subsequently, I donā€™t want to have to write (or (:name1 m) (:name2 m)) in all of my code Is there anyway to coerce these keys into another one ?
#2018-02-1912:37mpenetsyntax is wrong, I don't think you should use a ns on the or, it should read (or ::foo ::bar)#2018-02-1912:37mpenetafaik#2018-02-1912:37mpenet@igosuki I dont think you can no, you need to use eval for that, or a macro#2018-02-1912:38igosukiyeah sorry I wrote the ns by reflex, but actually am using just a plain or#2018-02-1912:41igosukiItā€™s annoying to have a declarative syntax for validation, and then have to write custom code for something that trivial thoughā€¦ something like (alias ::name1 ::name2) would be nice, equivalent :
import com.fasterxml.jackson.annotation.{JsonIgnoreProperties}
case class User(@JsonAlias(Array("name1", "name2") name))
#2018-02-1915:01mpenetit would really be handy to have specs for specs, I occasionally write (s/def foo ...) or (s/fdef ::foo ...) and it's far from obvious to trace when this happens#2018-02-1915:02mpenetyou get an Illegal arg exception in one case but the error is a bit cryptic:#2018-02-1915:02mpenet:type java.lang.IllegalArgumentException :message Invalid match arg: /cdn-cgi/l/email-protection#2018-02-1917:06EdI'm importing something from some json and would like to use the namespaced map in the rest of the program#2018-02-1918:32misha@igosuki (s/def ::my-alias (s/nonconforming (s/or :n1 ::name1 ::name2)))#2018-02-1918:33mishaunless I misunderstood the question#2018-02-1921:24misha@igosuki
(clojure.set/rename-keys {:name1 1 :bar 2} {:name1 :name :name2 :name})
=> {:bar 2, :name 1}
#2018-02-1921:26mishayou can do it pre- or post- spec, and your further convenience-while-working-with-data-structure is probably irrelevant to spec#2018-02-2009:31roklenarcicHm I have a spec that limits string length to be between 0 and 10 characters. As I ask generate to generate a larger sample size, it starts to throw that it failed to satisfy such-that. What's the best way to deal with that?#2018-02-2013:20gfredericksyou have to supply your own generator; (gen/map #(apply str %) (gen/vector gen/char-ascii {:max-elements 10})) or something like that#2018-02-2013:42roklenarcictest.chuck has a cap-size function that works great for this#2018-02-2014:03igosuki@misha faire enough, thanks for the heads up#2018-02-2017:37gfredericks@roklenarcic strictly speaking that's capping the size parameter, not the size of the collection. But they probably happen to coincide in this case.#2018-02-2101:02richiardiandrearegarding multi-arity functions, can I spec a 4-arity fn this way?
(s/fdef select-events-sqlmap
  :args (s/cat :domain :event/domain
               :entity :event/entity
               :key (s/alt :arity-2 nil? :arity-3-4 :event/key)
               :version (s/alt :arity-2-3 nil? :arity-4 :event/version)))
It seems not to work when I pass three args
#2018-02-2101:04richiardiandreaif there is a more idiomatic way I am open to it, this seems quite difficult to read#2018-02-2101:16Alex Miller (Clojure team)
(s/fdef select-events-sqlmap
  :args (s/cat :domain :event/domain
               :entity :event/entity
               :arity3-4 (s/? (s/cat :key :event/key :version (s/? :event/version)))))
#2018-02-2101:39richiardiandreaThanks a lot I will try this out!#2018-02-2117:38richiardiandreatried this, but it seems to fail when :version is nil, will play more with it#2018-02-2117:49Alex Miller (Clojure team)you can wrap (s/nilable <spec>) around a spec to also allow for nil#2018-02-2117:57richiardiandreaoh nice#2018-02-2118:03richiardiandreathis is what works:
(s/fdef select-events-sqlmap
  :args (s/cat :domain :event/domain
               :entity :event/entity
               :arity3-4 (s/? (s/cat :key (s/nilable :event/key)
                                     :version (s/? (s/nilable :event/version))))))
#2018-02-2101:16Alex Miller (Clojure team)is one way#2018-02-2101:17Alex Miller (Clojure team)you can also s/alt at the top for the 2, 3, and 4 cases if you want a single map with the same keys conformed out#2018-02-2117:35richiardiandreas/alt only works with keywords so does that mean that I need to break the spec in three?#2018-02-2117:49Alex Miller (Clojure team)s/alt works with any spec - keywords are used to tag the alternatives#2018-02-2108:52hawariCan I make a spec definition that depends on another spec like:
(spec/def ::start-time
  (spec/and string? valid-datetime?))
(spec/def ::end-time
  (spec/and string? valid-datetime?)) ;; need to validate that it's greater than start-time

(spec/def an-entity
  (spec/keys :req-un [::start-time ::end-time]))
#2018-02-2109:07moxaj@hawari.rahman17 you should do that validation in the ::an-entity spec#2018-02-2109:11hawariCan you suggest on how can that be achieved? My apologies, I've just recently started using spec, I don't fully grasp its concept yet. Do I use spec/and and pair the spec/keys with another predicate which validate that start-time is greater than end-time?#2018-02-2109:12hawariIf so, how can one destructure a map from within a predicate? Does it get treated as an argument to a spec? @moxaj#2018-02-2109:15moxaj(spec/and (spec/keys ..) (fn [{:keys [start-time end-time]}] ...))#2018-02-2109:15moxajyour predicate is just a regular function#2018-02-2109:16hawariAh great, I'll try it right away, thank you very much for your suggestion @moxaj!#2018-02-2113:55yogidevbearHi everyone. I'm a spec n00b so please have patience with my question here. Can/should I use spec for validating user input passed into my functions? For example, if I have a function that should accept positive integer values only and those value might need to be within a particular range, would spec work for this (and possibly what might the general structure of code look like)?#2018-02-2114:03taylorI wrote some examples that might help https://taylorwood.github.io/2017/10/15/fspec.html#2018-02-2114:04yogidevbearThanks Talyor :+1: I'll check that out now#2018-02-2114:52Alex Miller (Clojure team)also check out s/int-in for your range spec#2018-02-2114:02OlicalConsidering it makes a very very good parser for all sorts of data structures, that'd probably be a fine use case. Since the result is in itself more data, you could add more constraints in the future then build your error messages from that output.#2018-02-2114:02OlicalAlso, by user input do you mean an actual form on a page, or as an API?#2018-02-2114:03yogidevbearAt the moment, user input from a REPL.#2018-02-2114:04yogidevbearvia running lein repl and the calling the functions from the REPL#2018-02-2114:05OlicalAh, so you're talking runtime, not dev time, which is a big difference. At dev time you can just define your function specs and the user can decide to enable instrumentation (in my opinion). At runtime you'll be able to run your inputs through conform and respond with an informative error if it doesn't fit.#2018-02-2114:05OlicalAh, so you're talking runtime, not dev time, which is a big difference. At dev time you can just define your function specs and the user can decide to enable instrumentation (in my opinion). At runtime you'll be able to run your inputs through conform and respond with an informative error if it doesn't fit.#2018-02-2114:07yogidevbearCan you explain what you mean by >the user can decide to enable instrumentation#2018-02-2114:08OlicalSure! So, if you define specs for all of your functions with s/fdef, nothing actually happens at runtime. It's just sort of there, like documentation.#2018-02-2114:08OlicalThe user, or developer, can decide to instrument a namespace / all namespaces so that these fdef specs are checked.#2018-02-2114:09OlicalIn normal spec, this only means the arguments being passed to functions are checked, but you can use something like Orchestra to check the return values too.#2018-02-2114:10OlicalYou may find https://github.com/bhb/expound interesting, it makes spec errors human readable at a glance. And here's https://github.com/jeaye/orchestra#2018-02-2114:10yogidevbearCool, thanks Oliver šŸ™‚#2018-02-2114:11OlicalNo problem. I think you'll want to use https://clojuredocs.org/clojure.spec.alpha/conform on your data and a spec that checks your int is in range. For which you can use https://clojuredocs.org/clojure.spec.alpha/int-in-range_q#2018-02-2114:12yogidevbearPerfection#2018-02-2114:19Olical
user=> (s/conform #(s/int-in-range? 5 10 %) 2)
:clojure.spec.alpha/invalid
user=> (s/explain #(s/int-in-range? 5 10 %) 2)
val: 2 fails predicate: :clojure.spec.alpha/unknown
#2018-02-2114:21OlicalYou can use (s/def ::my-num #(s/int-in-range? 5 10 %)) then (s/explain ::my-num 2) gives you a more descriptive error too. Not sure how much spec you know though, so sorry if I'm explaining too much šŸ˜…#2018-02-2114:23yogidevbearAt present, I know about 2 hours worth šŸ™‚#2018-02-2114:23yogidevbearSo all the explanation is very welcome#2018-02-2114:27yogidevbearAnd if I'm using clojure 1.9.0, do I simply add this to my ns?
(ns my-ns.core
  (:require [clojure.spec.alpha :as s]))
#2018-02-2114:29OlicalIt should be present anyway, it's included as part of Clojure. No need to depend on it afaik#2018-02-2114:29OlicalBut yes, that require line is fine.#2018-02-2114:30OlicalAnd you can use cljs.spec.alpha in cljs#2018-02-2114:31taylor> Clojure 1.9 depends on this library and provides it to users of Clojure. Thus, the recommended way to use this library is to add a dependency on the latest version of Clojure 1.9, rather than including it directly#2018-02-2114:31yogidevbearYeah, it seemed to be throwing some error on my spec function without needing to depend so looks like 1.9.0 includes it by defaut#2018-02-2114:39yogidevbearI think I've got a spec working without throwing an error when I run lein repl. Excuse the trivial function example:
(defn my-fn
  "An example function. Takes two arguments and makes a vector from them"
  [a b]
  (into [] (list a b)))
(s/fdef my-fn
        :args (s/cat :a #(s/int-in-range? 1 11 %)
                     :b #(< 10 %)))
#2018-02-2114:40yogidevbearDoes that seem correct?#2018-02-2114:41OlicalLooks right to me, yep!#2018-02-2114:41OlicalYou can put the fdef before the defn if you want to btw#2018-02-2114:41OlicalAnd to get clojure to check this you have to use spec tools instrument or something like orchestra#2018-02-2114:42OlicalSo these are more of a developer tool thing, if you want to check things at runtime that you present to the user, you probably want s/conform and s/explain#2018-02-2114:42yogidevbearSo my next question then... if I run (my-fn 1 2) within the REPL, it works instead of throwing an error#2018-02-2114:43yogidevbearIs this where the spec tooling you're referring to would come into play?#2018-02-2114:43OlicalYep, this is where you have to instrument#2018-02-2114:43OlicalYou don't want spec checking EVERY function call in prod#2018-02-2114:43OlicalSo it's opt in#2018-02-2114:43yogidevbearAh, that makes a lot of sense šŸ™‚#2018-02-2114:45OlicalCheck out orchestra though, it has some really neat tooling https://github.com/jeaye/orchestra#defn-spec#2018-02-2115:52yogidevbear@U38J3881W any ideas why I might be getting Could not locate orchestra/core__init.class or orchestra/core.clj on classpath when running lein repl? I have [orchestra "0.2.0"] in my dependencies and (:require [orchestra.core :refer [defn-spec]]) in my ns declaration#2018-02-2115:55OlicalOoh, odd. Maybe it's a doc thing? I suppose it's not found in [orchestra.spec.test :as st]?#2018-02-2115:55OlicalAlso, maybe the docs are wrong in another way, try [orchestra "2017.11.12-1"], that one seems to be for newer versions of Clojure.#2018-02-2115:55yogidevbearHmm, not sure. They don't seem to include orchestra.spec.test in their defn-spec example on that readme#2018-02-2115:56yogidevbearIsn't that version the cljs version?#2018-02-2115:56Olical0.2.0 says it's for "1.9.0 >= Clojure < 1.9.0-alpha16"#2018-02-2115:56yogidevbearI'm on 1.9.0#2018-02-2115:56yogidevbearLet me try the other version and see#2018-02-2115:57OlicalYeah, I think you want the other one, it's for CLJS and CLJ šŸ™‚#2018-02-2115:57Olical0.2.0 seems to be legacy for an older style of spec.#2018-02-2115:58yogidevbearThanks, that seems to have fixed it#2018-02-2116:00OlicalNice! I hope it's all working well. Tried it with expound yet? Definitely makes the errors more palatable.#2018-02-2116:01yogidevbearNot yet#2018-02-2116:44viestihum#2018-02-2117:40bfabryfollowing on from hawari's example above, what's the best way to attach a generator to the compound spec that generates end-time by adding some random amount to start-time. assume the spec/keys call involves a lot of keys all the things that I'm coming up with have a lot of boilerplate, but I probably just don't understand test.check very well#2018-02-2117:43gfrederickswhat do you have so far?#2018-02-2117:56bfabrywell I think I just ran into some unrelated cljs macro problems#2018-02-2117:56bfabrybut ignoring those#2018-02-2117:57bfabry
cljs.user=> (s/def ::call' (s/keys :req-un [::direction ::duration ::agent_id ::group_id ::id ::start_time ::end_time]))
cljs.user=> (def call-gen #(gen/let [e (s/gen ::call')] (assoc e :end_time (+ (:start_time e) (rand-int 3600)))))
(s/def ::call (s/and (s/with-gen ::call' call-gen) #(> (:end_time %) (:start_time %))))
#2018-02-2118:47ghadihttps://dev.clojure.org/jira/browse/CLJ-2197 I just ran into this :(((#2018-02-2119:13johanatanhi, does spec have support for heterogeneous maps? in particular i want to validate a map like this: {:a-special-kwd [:some :elements] "string" ::some-structure "string2" ::some-structure ... }#2018-02-2119:52Alex Miller (Clojure team)http://blog.cognitect.com/blog/2017/1/3/spec-destructuring#2018-02-2119:52Alex Miller (Clojure team)hybrid map example#2018-02-2119:16johanatani suppose i could do: #(s/and (valid-special? %1) (s/valid? (s/map-of string? ::some-structure) (dissoc %1 :a-special-kwd))) ?#2018-02-2119:16johanatanis there a better way?#2018-02-2119:23johanatani suppose i was trying to avoid having lambdas which explicitly call valid? though (if there is a way)#2018-02-2119:35aaron51Hello! How do we check return values using fdef and stest/instrument? I see you can add the :ret key, but it doesn't seem to fail when it should. Also can stest/instrument check all fdef'd functions (instead of having to list them all)?#2018-02-2119:36taylorstest/instrument doesnā€™t check :ret specs, only :args AFAIK#2018-02-2119:36johanatanstest/check does however#2018-02-2119:36taylorand calling (stest/instrument) with no args should instrument everything thatā€™s been loaded#2018-02-2119:37taylorhttps://clojure.org/guides/spec#_combining_code_check_code_and_code_instrument_code#2018-02-2119:40aaron51Any way to check ret without auto generating tests with check? We're unit testing and trying to use specs on the return value#2018-02-2119:41taylorI think someone has written some utility code for doing that, canā€™t remember which lib itā€™s in#2018-02-2119:41taylorhttps://github.com/jeaye/orchestra#2018-02-2119:41taylorpretty sure thatā€™s it ā˜ļø#2018-02-2119:42aaron51We were looking for a way to continue using defn, instead of defn-spec, because defn-spec confuses Cursive#2018-02-2119:44taylorI donā€™t think youā€™re required to use defn-spec with orchestra#2018-02-2119:45taylori.e. just using orchestraā€™s instrument seems like it will do :ret checking on plainā€™ ol defns and s/fdefs (hopefully?)#2018-02-2119:47aaron51oh wow, that's exactly the answer I needed. We were fighting with Orchestra and Cursive for a while#2018-02-2119:47aaron51thanks!#2018-02-2119:48taylorFWIW (and it probably wouldnā€™t help you here) but Cursive has a feature where you can tell it to treat custom macros like the core/known macros, assuming the custom macro follows one of those known patterns#2018-02-2123:11aaron51oh, we tried that. It partially worked, but then jumping between tests and implementation broke. Seems like the feature we want but it isn't quite fully baked yet#2018-02-2123:11aaron51thanks again#2018-02-2119:39gfredericks@bfabry ignoring the unrelated issues with using rand-int, I think the boilerplate you have is necessary given the current API maybe a helpful thing would be a variant of s/and that lets you modify the default generator of the first argument#2018-02-2119:40bfabry@johanatan I think you're looking for s/keys#2018-02-2119:40johanatan@bfabry no, that won't work. the "strings" (string1, string2) are dynamic/unknown#2018-02-2119:40bfabry@gfredericks what are the issues with using rant-int and what should I be doing instead?#2018-02-2119:41bfabryoh sorry I didn't realise you meant heterogeneous keys as well#2018-02-2119:41johanatanyes, heterogeneous#2018-02-2119:48gfredericksusing your own randomness instead of a generator means you lose growth, reproducibility and shrinking; one way to address that is to add a temporary key to ::call' that maps to something that generates the sort of nonnegative integer you want, and then remove that key in ::call#2018-02-2119:53bfabrycan I just use duration (s/gen (s/int-in 1 3600)) in the gen/let and then use that?#2018-02-2119:55gfredericksoh yeah that's a good idea#2018-02-2119:56bfabryok cool, slowly understanding#2018-02-2120:00gfrederickscould be helpful: https://www.youtube.com/watch?v=F4VZPxLZUdA&amp;t=1625s#2018-02-2122:30bfabrythanks!#2018-02-2312:01otfromdumb question of the day: can you add a fsepc to a defmethod? (I remember reading somewhere you couldn't but I think I'm misremembering and google has been failing me)#2018-02-2312:47mpenetYou can't. you can spec the dispatch fn or/and create separate defns that will be called by the multimethod impl#2018-02-2313:27otfrom@mpenet thx!#2018-02-2313:28otfromI hadn't thought about specing the dispatch function but it is obvious now that you point it out#2018-02-2313:29mpenet@alexmiller pointed that out the other day, didn't think of it either before then šŸ™‚#2018-02-2313:29Alex Miller (Clojure team)That covers the args side at least#2018-02-2313:29otfromman, he should write a book#2018-02-2313:30otfromoh, sorry didn't see you there @alexmiller;-)#2018-02-2313:31otfrom@alexmiller I was a bit surprised yesterday and today w/instrumenting an fspec. It looks like it calls gen/check on the way in which I wasn't expecting (21 times if I'm reading correctly)#2018-02-2313:32otfromwe got caught out b/c we had a spec that was wider than the code that was being called#2018-02-2313:32Alex Miller (Clojure team)Yeah, you are not the first#2018-02-2313:32otfromšŸ˜„#2018-02-2319:16jjttjjIs leaning on spec's conform/unform to do translations to and from some outside api an intended and/or acceptable use case? IE if the api gives dates and integers as strings and i need to convert them to java dates and real ints when the come in, and convert them back to strings when they go back out to the api, and there are a lot of varied translations like this in a nested structure. I'm thinking I spec all the incoming string data with conformers, then additionally spec out all my own representations as normal. Is there a reason I'm missing not to do this?#2018-02-2319:58Alex Miller (Clojure team)This recent thread is probably a good starting point https://groups.google.com/d/msg/clojure/Tdb3ksDeVnU/LAdniiZNAgAJ#2018-02-2320:08tayloragree that cautioning against ā€œabusingā€ conformers should be documented#2018-02-2320:31seancorfieldI would have sent a PR for the site repo -- but conformer isn't even mentioned anywhere there. So I think it would have to be a docstring update for the function itself? @alexmiller thoughts?#2018-02-2321:03Alex Miller (Clojure team)Itā€™s intentionally not mentioned as anti documentation :)#2018-02-2321:04Alex Miller (Clojure team)but maybe it should be more explicitly unmentioned :)#2018-02-2320:02jjttjjthank you#2018-02-2408:54Peter WilkinsHi - I'm learning spec. I've googled and doc'ed but can't find a clue how to spec the sum of all numbers in a collection (a map) in this case
(s/def ::percentage (s/and double? #(>= % 0) #(<= % 100)))
(s/def ::scores (s/map-of keyword? ::percentage))

(s/valid? ::percentage  50.05)
(s/valid? ::scores {:a 70.49 :b 20.21 :c 9.29})
I'd like to confirm that the sum of ::scores = 100
#2018-02-2409:54robert-stuttaford#(= 100 (apply + (vals %))) surely?
#2018-02-2410:04yogidevbearDoes anyone here have an open source repo where they're using spec? I feel like I'm missing something fundamental about spec and I'm hoping that looking at a few good examples of it being used within a project might help solidify some of the core concepts for me.#2018-02-2410:24jannis@yogidevbear We wrote https://github.com/functionalfoundry/entitydb some time ago. Starting from workflo.entitydb.core/empty-db, most functions and input/output data have specs to allow random testing and integrity checking.#2018-02-2410:26jannisSomething less complex perhaps: specs for the Om Next query language: https://github.com/functionalfoundry/macros/blob/master/src/main/workflo/macros/specs/om_query.cljc#2018-02-2410:55Peter Wilkins@robert-stuttaford sorry I worded the question badly. I can't figure out how to compose a spec using map-of and
#(= 100 (apply + (vals %)))
.
(s/def ::scores (s/and (s/map-of keyword? ::percentage) #(= 100 (apply + (vals %)))))
doesn't work. Also, wouldn't we want to be the kind of community to support and encourage beginners, surely?
#2018-02-2412:48luskwater@poppetew
(defn approx=
  "Return a fn that checks that a value is approximately `n` (within `e`)"
  [n e]
  #(-> (apply + (vals %))
       (- n)
       (Math/abs)
       (< e)))

(s/def ::percentage (s/and double? #(>= % 0) #(<= % 100)))
(s/def ::scores
  (s/and (s/map-of keyword? ::percentage)
         (approx= 100 0.1)))
(s/def ::scores-restrictive
  (s/and (s/map-of keyword? ::percentage)
         (approx= 100 0.001)))

(s/valid? ::percentage  50.05)

(def the-map {:a 70.49 :b 20.21 :c 9.29})
(apply + (vals the-map))
(s/valid? ::scores the-map)

(s/valid? ::scores-restrictive the-map)
#2018-02-2412:52luskwaterActually, @poppetew, your (s/and ,,, #(= 100 (apply + (vals %)))) would work, if you were using integers. Thatā€™s why we needed the approx= fn up there.#2018-02-2412:53luskwaterYour e may vary, of course. šŸ™‚#2018-02-2413:00luskwater
(def the-map {:a 70.49, :b 20.21, :c 9.29})
=> #'cognitect.transcriptor.t_1/the-map

(apply + (vals the-map))
=> 99.98999999999998

(s/valid? :cognitect.transcriptor.t_1/scores the-map)
=> true

(s/valid? :cognitect.transcriptor.t_1/scores-restrictive the-map)
=> false
#2018-02-2417:03robert-stuttaford@poppetew i apologise if my rapidly typed response wasnā€™t suitably welcoming for you - i figured something would be better than nothing. i recognise now that it could have come across as condescending - not my intent! looks like @luskwater provided a far more considered response, which has taught me something!#2018-02-2417:03robert-stuttaford@poppetew i am incredibly supportive of beginners šŸ™‚#2018-02-2417:38luskwaterWouldn't have had an answer for @poppetew so quickly, had it not been for @robert-stuttaford and his initial suggestion. Collaboration across timezones takes a little time itself. #2018-02-2514:39Petrus TheronHaving a hard time with Spec performance on runaway regular expressions for a simple lexer/parser to lex a string into tokens before I parse it. I'm trying to parse a string into tokens so I can put it through a second "compiler" pass:
(s/def ::plexpr*
  (s/+
    (s/alt
      :whitespace (s/+ #{\space \tab \formfeed \backspace})
      :line-sep (s/+ #{\newline \return \formfeed})                   ;(s/+ line-separator?)
      :atom (s/+ (set "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890")))))

(s/def ::plexpr
  (s/and
    (s/conformer seq)
    ::plexpr*))

(s/conform ::plexpr "drive home fast") ;; => (fast), [[:atom [\d \r \i \v \e]] [[:whitespace [\space]] [:atom [\h \o \m \e]] [:whitespace [\space]] [:atom [\f \a \s \t]]]]
(s/conform ::plexpr "drive home fast but now it gets slow") ;; => extremely slow
I'm not using any optional s/? regexes, so I don't understand why this would cause any kind of exponential search
#2018-02-2518:59seancorfield@petrus clojure.spec is not intended for string parsing -- you know that, right?#2018-02-2518:59seancorfieldThe regexes are for "parsing" sequences, not strings. You probably want to look at Instaparse which is designed for parsing strings.#2018-02-2607:46Petrus TheronThanks - a string is a sequence, no? šŸ˜‰ I can switch to instaparse for tokenising, but I'm worried about the length of data sequences I'll be able to handle with Spec#2018-02-2617:54seancorfieldA string can be coerced to a sequence šŸ™‚ I saw @U0516PHE3 mention that your spec is an ambiguous grammar and that might be causing the exponential slow down. As for data sequences, I would expect most folks would use coll-of or every and the elements would be self-contained specs (so, no ambiguity).#2018-02-2619:54Petrus TheronHow can I make my spec unambiguous? If I split my string into a word tokens, can I still use spec and coll-of/every to parse the substrings? Or do I need an instaparse step before I send it to spec?#2018-02-2620:04seancorfieldRight now "aa" can parse as "atom (a)" "atom (a)" or "atom (aa)" so you could specify that the overall sequence is "atom" then a sequence of "whitespace" and "atom" perhaps? (assuming you trim leading/trailing whitespace before parsing?)#2018-02-2620:05seancorfieldIf you require whitespace separating the atoms, then "aa" can only parse as "atom (aa)"#2018-02-2620:07seancorfieldYou'll want to make sure whitespace and line endings don't overlap too (so there's no ambiguity there) and if you care about the difference have your overall spec clearly indicate line / sep / line structure and line as a spec indicate atom / whitespace / atom structure. Make sense?#2018-02-2519:02gfredericksit's still a surprising slowdown, I would say#2018-02-2519:02gfredericks(and I did say it, in fact)#2018-02-2519:08seancorfieldI'm kinda surprised even the 15-element string is handled "fast". That would be long for an argument list so I'd say it was outside the intended design space?#2018-02-2519:11gfredericksthe spec regexes are only meant for arglists?#2018-02-2519:11seancorfieldProcessing "drive" takes 3ms, "drive home" takes 72ms, "drive home fast" takes almost 1 second. That doesn't feel "fast" to me.#2018-02-2519:12gfredericksI coincidentally just wrote code that uses spec to parse the lines of a 17277-line file; it took 0.6s#2018-02-2519:13seancorfield@gfredericks Not just arglists but I don't think folks are spec'ing long sequences in data...? And data is more likely to get spec'd with coll-of or every I think?#2018-02-2519:13ghadiI wonder why that example appears to show exponential behavior. What are the bounds on parsing with derivatives?#2018-02-2519:13gfredericksI'm using regexes because I actually want the parsing capabilities, and I think this would be too hard with instaparse#2018-02-2519:13ghadiThere was an updated paper Matt Might released that improved performance.#2018-02-2519:14ghadiEven if that example above isn't a bug -- it's a valuable example for spec's test suite.#2018-02-2519:14ghadi(... but it's probably a bug)#2018-02-2519:16gfredericksyeah I can't imagine any good reason for something that simple to take a long time#2018-02-2519:16seancorfieldI got an OOM error trying to parse that long string šŸ‘€#2018-02-2519:16ghadiIs the performance still bad if you feed it the seq directly and remove the s/and & the conformer?#2018-02-2519:22seancorfieldAdding each new word seems to double the time it takes. I think there's a clue in how it conforms -- for one word you get a sequence of one element which is a sequence for the atom: (time (s/conform ::plexpr "one")) => [[:atom ["o" "n" "e"]]] (in just a ms or two), but (time (s/conform ::plexpr "one two")) => [[:atom ["o" "n" "e"]] [[:whitespace [" "]] [:atom ["t" "w" "o"]]]] -- note: two elements, with the second containing the whitespace and the next word. Adding more words still produces a two-element sequence with the second one containing all the extra elements.#2018-02-2519:34aengelbergYou could change the set predicates to functions that print some debug information. That would give you an idea of what things the spec engine is unnecessarily repeating.#2018-02-2519:38aengelbergI think that parser is ambiguous, because a string of "aa" could match [:atom ["a"]] [:atom ["a"]] or [:atom ["a" "a"]]. That could explain the exponential growth.#2018-02-2519:57gfredericksooh good point#2018-02-2523:59devnHow does one access a spec within the registry's req/req-un/opt/opt-un info?#2018-02-2523:59devnI'd like to peek at a spec's list of req/req-un/opt/opt-un#2018-02-2600:00devnare s/describe or s/form what I want to use here, or am I missing something obvious?#2018-02-2600:01devnI'd like to peek at req and opt to pass them into a pull query#2018-02-2600:30seancorfield@devn https://clojure.github.io/spec.alpha/clojure.spec.alpha-api.html#clojure.spec.alpha/registry ?#2018-02-2600:51devn@seancorfield that'll get me the spec objects by name, but not in a way that's easy to pluck out the req/opt#2018-02-2600:52devni have a feeling im just missing something obvious, maybe in how i define the :req to begin with. Is it possible for me to use a spec as the arg to :req I wonder#2018-02-2600:53seancorfieldYeah, right now you'd have to get the form for each spec and then walk them.#2018-02-2600:53seancorfieldSince the keys part could be embedded deep inside a spec...#2018-02-2600:53gfredericksdoesn't somebody mention a yet-to-exist spec spec every time this comes up?#2018-02-2600:54devnsomething loosely like:
(s/def ::the-keys [::a ::b])
(s/def ::the-map :req ::the-keys)
#2018-02-2600:55seancorfieldAlex says they're looking at ways to make it easier to build specs programmatically but it's not there yet.#2018-02-2600:55devnerr, well, when it's a set, I can at least do (s/form ::the-keys) and get back #{::a ::b}, it's just that I can't use it in the s/keys form#2018-02-2600:56seancorfieldclojure.spec is a wonderful concrete example of how macros don't compose šŸ‘€#2018-02-2601:02devnim not complaining. i think there's probably a solution im missing here, let me try to be a bit more clear on what i want to do. I have a spec which says that ::a and ::b are required. I need some value holder that can be used both in the spec definition: (s/keys :req ...) and a pull query (d/q '{:find [(pull ?blah ?the-qualified-keys) :in [$ ?the-qualified-keys]] ...} db the-qualified-keys)#2018-02-2601:05devnerr, i could relax that second part and say I'm not sure if that's possible. if it isn't, what kind of clever solutions are there? I suppose I could generate the spec and a corresponding def, for instance, so i don't have to manually keep modifications to either place consistent.#2018-02-2601:11seancorfieldWhat we've done at work is to define a spec with keys, build any more complex specs on top of that, then get the spec form itself, parse out the :req [...] part and pull the keys from that to use elsewhere. So the base spec was the "system of record" and we derived other stuff from it.#2018-02-2601:11seancorfieldIt's kind of icky but it works, given what we have available right now. It seems like the path of least resistance.#2018-02-2601:13devnmaybe (s/describe) is good enough for what i want to do here, the structure of it just felt a little wonky to me: (keys :req [:foo/a :foo/b]), so i need to do something like (apply hash-map (rest (s/describe ::the-map))) or whatever#2018-02-2601:19seancorfieldHmm, we use s/form... what is s/describe?#2018-02-2601:19devnroughly the same deal#2018-02-2601:19seancorfieldAh, an "abbreviated" form as data...#2018-02-2601:20devnform: (clojure.spec.alpha/keys :req [:foo/a :foo/b]) describe: (keys :req [:foo/a :foo/b])#2018-02-2601:21devni'm surprised it's not a map, but perhaps there are Reasonsā„¢ why that I don't understand#2018-02-2601:21gfredericksgetting a map isn't hard though: (apply hash-map (rest thing))#2018-02-2601:31devnfair point#2018-02-2601:31devnit gave me that "is this a hint that i shouldn't be using it like this?" feeling#2018-02-2601:37gfredericksprobably just a hint that it's not done yet#2018-02-2613:26mpenet@alexmiller any details/insight about how could spec meta data look like in the future?#2018-02-2613:39Alex Miller (Clojure team)No#2018-02-2613:26mpenetanother question while I am at it: was it ever considered to implement (s/def ::foo ::bar) via clojure.core/derive? I would personally find it quite interesting with that form, you can leverage ancestors/isa? etc#2018-02-2613:43Alex Miller (Clojure team)Not sure if Rich ever considered that.#2018-02-2613:44mpenetdoes that sound like something desirable in your opinion?#2018-02-2613:46Alex Miller (Clojure team)Donā€™t know. The example above I would consider aliasing, not a hierarchy. The notion of derivation with respect to s/keys (containership) is interesting though for additive aggregations.#2018-02-2613:46mpenetwe use that on a small lib to add metadata to specs, we then can just say spec ::x derives from another and add useful little utils on top of that that can optionally fetch all the metadata from the ancestors and merge them for instance.#2018-02-2613:46mpenetyes exactly why I am asking#2018-02-2620:00ghadiI filed https://dev.clojure.org/jira/browse/CLJ-2327 just to capture that slowdown#2018-02-2620:09Alex Miller (Clojure team)I think probably replacing the big set with a char int numeric check would probably help a lot#2018-02-2620:12ghadiI don't think so @alexmiller -- it's OOMing which means some extra retention#2018-02-2620:13ghadiI think we could remove the chars entirely and have a smaller example#2018-02-2620:23Alex Miller (Clojure team)if you replace the :atom branch with something like (defn atom-char? [c] (let [i (int c)] (or (<= 48 i 57) (<= 65 i 90) (<= 97 i 122)))), I think youā€™ll find it doesnā€™t have the issue#2018-02-2620:24ghaditrying it...#2018-02-2620:26ghadiit does not help at all @alexmiller#2018-02-2620:27Alex Miller (Clojure team)helped for me#2018-02-2620:27ghadidid you (s/+ atom-char?)#2018-02-2620:27ghadimine is still running#2018-02-2620:27Alex Miller (Clojure team)oh, maybe I just did atom-char?#2018-02-2620:27ghadipretty sure it's the s/alt and s/+ combo#2018-02-2620:28Alex Miller (Clojure team)not (s/+ atom-char?)#2018-02-2620:28Alex Miller (Clojure team)yeah, prob#2018-02-2712:33borkdudeI have written a macro to get rid of some boilerplate. Iā€™m using a spec to validate arguments passed to a generated function.
(s/merge ::util/job-args
           (s/keys :req-un
                   [::importers
                    ::database
                    ::rabbitmq
                    ::stats]))
But I still have to destructure the args, where something could go wrong if you make a typo.
(let [{:keys [report importers database rabbitmq stats]}
        args]
    )
Is there a good solution for this, e.g. key destructuring using spec?
#2018-02-2712:41gfredericksI'd probably use plumbing/safe-get#2018-02-2712:42borkdudeyeah, I thought about writing that before I wrote the specsā€¦ might be just as effective#2018-02-2712:42borkdudebut nice to have it in a lib#2018-02-2715:51mishawell, it has nothing to do with spec opieop#2018-02-2716:33borkdudebut it does what I want. šŸ˜„#2018-02-2716:37gfrederickstheoretically it overlaps with spec because the spec could tell you whether you're accessing a key that can be ex-spec-ted to be there#2018-02-2716:39stathissideris@borkdude Iā€™ve written this https://github.com/stathissideris/spectacles but it doesnā€™t cover destructuring#2018-02-2716:40stathissiderisit has get, get-in, assoc, assoc-in, update, update-in#2018-02-2716:41stathissiderisyou can also compose the lenses#2018-02-2716:45misha@gfredericks should it, say, throw, if you are asking for key not in spec?#2018-02-2716:45gfredericks@misha that sort of thing, yeah#2018-02-2716:46mishathen it'd be nice app-level let-speced macro#2018-02-2716:46stathissideristhatā€™s what spectacles does! (more or less)#2018-02-2716:47stathissiderisif someone wants to extend it with let-speced pull requests are very welcome#2018-02-2716:47mishaI think you are missing binding part of destructuring, @stathissideris#2018-02-2716:48mishathis is where the money is#2018-02-2716:48stathissiderisdefinitely, but there is quite a bit of code there to introspect the specs etc#2018-02-2716:49stathissiderisso itā€™s a good base to build on, thatā€™s what Iā€™m saying šŸ™‚#2018-02-2716:49mishaI think the most common and useful case would be specced-destructuring for maps (s/keys)#2018-02-2716:50mishabecause most of conformed things end up being maps anyway opieop#2018-02-2716:51stathissideristhe main reason I didnā€™t do it was the syntax, couldnā€™t come up with a nice way to express it#2018-02-2716:51mishaand seq-destructuring does not need any names from input value anyway#2018-02-2716:51stathissiderisbecause the spec name has to be mentioned#2018-02-2716:51mishatrue#2018-02-2716:52stathissiderisitā€™s already a bit awkward with get-in#2018-02-2716:52mishaI'd love it to have let-like form#2018-02-2716:54stathissideris@misha can you come with an example of what it would look like using it?#2018-02-2716:58mishanot yet, thinking of options. what I know, is: - binding has to be a core feature, to avoid repetition - should look "familiar", so plugins, like #cursive could understand it (most probably like let) - should be natural to use, because following does not really looks good:
(let [_ (spec-bind spec m [foo bar baz])]
  ...)
#2018-02-2716:59stathissiderishow about:
(lens/let [{::logging [::level ::destination ::type]} foo]
  (prn "level is" level)
  (prn "type is" type))
#2018-02-2716:59mishabut that is second best of my ideas (but I dont have top1 yet :) )#2018-02-2716:59stathissiderisinstead of :keys you give the name of the spec#2018-02-2716:59stathissideris(havenā€™t worked out the implications of the syntax yet)#2018-02-2717:00mishathis is better, although preserving all destructuring features would be neat#2018-02-2717:03misha(not saying you can't have both with this design, just need to think it all through)#2018-02-2717:03stathissiderisyeah, the ideal situation would be for spectacles to handle the ā€œspecialā€ parts and pass on anything not related to spec to clojure.core/let#2018-02-2717:04mishaup next: how to bind keys from the same map using different specs? 2 lines? or 2 specs in destructuring form?#2018-02-2717:05stathissideris
(lens/let [{::storage [::url ::port ::password]} foo
           {::logging [::level ::destination ::type]} foo]
  (prn "level is" level)
  (prn "type is" type))
#2018-02-2717:05mishaselect-keys should just work for this (passing the rest to core/let)#2018-02-2717:06stathissiderishmm, it will throw if you try to destructure a key that is not speced, but not when the map contains extra keys#2018-02-2717:07mishaa week ago we were discussing adding & to the map destructuring, so I spent some time looking at clojure.core/destructure#2018-02-2717:09borkdudeactually my problem was like this:
my-ns=> (s/def ::job-args (s/keys :req-un [::report ::scheduler]))
:my-ns/job-args
my-ns=> (lens/get {:scheduler 1} ::job-args :scheduler)
1
my-ns=> (lens/get {:foo 1} ::job-args :scheduler) ;;=> nil
So the input map should conform to the spec, which I can validate with e.g. s/assert, s/valid? etc.
#2018-02-2717:10borkdudebut you can also get the keyword wrong of course, this is what spectacles checks#2018-02-2717:10stathissideris@borkdude are you saying that the last call should throw instead of returning nil?#2018-02-2717:10mishaI think the value proposition here would be, "(say) throw, if trying to bind value under key not in spec"#2018-02-2717:11borkdudeboth things can go wrong: your input map or your selection in the map#2018-02-2717:12mishaif we are still on let-like macro topic: I'd just check if the key is supposed to be in map according to spec, because validation can be expensive, and I'd like to be able to separate the two#2018-02-2717:13mishaalthough it starts to sound like static types opieop#2018-02-2717:13borkdudesafe-get is the most effective in my case here: I just get an exception and prints out the keys that were in the map and the key I used. I can figure out which one was wrong#2018-02-2717:14borkdudeI guess you can mess up specs themselves too, nothing protects you from typosā€¦ typos all the way down#2018-02-2717:15mishafrom a broader perspective, destructuring and select-keys - are a bit orthogonal to each other, and both would need to be addressed individually#2018-02-2717:15stathissiderisāŒØ :thunder_cloud_and_rain:#2018-02-2717:15stathissideris@misha select-keys would be a good (and easy) addition to spectacles#2018-02-2717:16borkdudeI just discussed select-keys with someone else. After you use select keys you can still mess up selecting from the result of select keys#2018-02-2717:16stathissiderisnot if youā€™re using lens/get! šŸ™‚#2018-02-2717:16misha@borkdude this is why I want let :)#2018-02-2717:17mishaI much more often (destructure) bind keys, than select-keys and then bind#2018-02-2717:18stathissiderisIā€™ve raised this, https://github.com/stathissideris/spectacles/issues/6#2018-02-2717:18stathissideriswould be good to work out a good syntax for this#2018-02-2717:19mishaif collision with :my-ns/keys can be solved - it is the best syntax there could be, already#2018-02-2717:20mpenetPersonally I would just add a :spec key in the map destructuring form and limit it at that: {:as foo :keys [..] : spec ::foo}#2018-02-2717:20stathissiderisjust donā€™t ever call your specs ::keys šŸ˜‚#2018-02-2717:21mishaanother data point, i usually don't use keys in :keys, but symbols (let [{:my.ns/keys [foo bar]} m])#2018-02-2717:21stathissideris@mpenet that does sound a bit less collision prone, thank you!#2018-02-2717:21mpenetWould work for seq destructuring too#2018-02-2717:21mpenet[..... :as foo :spec ::foo]#2018-02-2717:21misha@mpenet you then miss on opportunity to destructure by multiple specs in the same map :)#2018-02-2717:22mpenetYes true, trade-offs #2018-02-2717:22stathissideris@mpenet Iā€™ve added your idea to the issue#2018-02-2717:22stathissideriswith attribution!#2018-02-2717:22mishayou probably don't need spec-destructuring for seqs#2018-02-2717:23mpenetProlly not#2018-02-2717:23mishaunless you are willing to support spec form walking for nested specs like s/or#2018-02-2717:23mpenetSpec walking is gross#2018-02-2717:23mishathen OMG you'd solve "damn, I forgot which keys I used in s/or for this"#2018-02-2717:23mpenetToo alpha to play this game imho#2018-02-2717:25mpenetNecessary evil for now#2018-02-2717:25misha2 years strong alpha kappa#2018-02-2717:25mpenetI am hoping we ll get something better for introspection/composition #2018-02-2717:26stathissiderisspectacles relies on looking at forms to figure out the keys#2018-02-2717:26stathissiderisbut yeah it feels risky#2018-02-2717:27mishaat this point I'd like to have compositional spec-walking#2018-02-2717:27mishait seems like everyone does it over and over again, for spectacles, for expound, etc.#2018-02-2717:29mpenetYup, spec-tools, speculate, etc been there done that too and trying to avoid doing it from now on;)#2018-02-2717:31stathissiderisspec-based seq destructuring could check that (1) the spec is in fact a coll-of, a cat or a tuple (2) for finite length specs, youā€™re not attempting to destructure more elements than what is defined in the spec#2018-02-2717:31stathissiderisnot sure how usefulā€¦#2018-02-2717:39stathissiderissorry Iā€™m not sure what that emoticon means!#2018-02-2717:39stathissideris@misha thanks for the comment#2018-02-2717:40mishaI went for "not really useful"#2018-02-2717:41stathissiderisyeah maybe not really worth it#2018-02-2717:56misha#2018-02-2717:58mishaoh wait, is this fixed in spec already, sweet#2018-02-2804:27bbrinckIs there a way to provide a seed to s/exercise or gen/sample? I have some code that uses these functions and Iā€™d like to test the result, so forcing a specific seed would keep my tests reliable.#2018-02-2804:27bbrinck(or, if there is another test.check function that can generate values based on a fix seed, I could use this)#2018-02-2810:27taylor@bbrinck looking at test.check there seems to be a way to do this by tweaking the sample-seq function that gen/sample wraps:
(defn sample-seq
  "Return a sequence of realized values from `generator`."
  ([generator] (sample-seq generator 200 nil))
  ([generator max-size] (sample-seq generator max-size nil))
  ([generator max-size seed]
   (core/let [r (if seed
                  (random/make-random seed)
                  (random/make-random))
              size-seq (gen/make-size-range-seq max-size)]
     (core/map #(rose/root (gen/call-gen generator %1 %2))
               (gen/lazy-random-states r)
               size-seq))))

(def my-gen (s/gen (s/map-of string? int?)))
(take 10 (sample-seq my-gen 200 0x42))
the only practical difference being the sample-seq in test.check doesnā€™t take seeds. sample is just calling this function with a default sample size. consider this answer extremely non-authoritative, but it does always generate the same samples for a given seed šŸ™‚
#2018-02-2811:49gfredericksalso gen/generate has a seed arg in the master version I believe#2018-02-2811:50gfredericksit wasn't there originally because sample is meant to be a dev-only function which raises the question of what you're doing with it that it needs to be tested šŸ™‚#2018-02-2812:40misha@gfredericks eyeballing data structure generated with the same seed is easier while developing spec for it. Presumably, only subset of data structure, related with spec change, would change on subsequent generate calls#2018-02-2812:42gfredericksyeah that makes sense; I think I misread what @bbrink said originally also, I had thought he must be calling sample in non-test code#2018-02-2812:43mishaI think you were not wrong :)#2018-02-2812:44mishabut exposing seed argument would benefit dev-time use case.#2018-02-2812:46gfredericksyep; for now gen/generate is pretty similar#2018-02-2813:15taylornice, I was only looking into the 0.9.0 code#2018-02-2815:11gfredericksI'm like two years late at making a new release#2018-02-2816:55mishaand a book!#2018-02-2816:59gfredericksand a guide#2018-02-2817:00misha#2018-02-2817:00gfrederickssecond edition of the book#2018-02-2817:00mishalambo license plate#2018-02-2817:00gfrederickscompany offering enterprise support#2018-02-2814:33bbrinck@taylor @gfredericks thanks for the ideas. Yes, my original question about was using s/exercise (with random seed) in non-test code, but then controlling the seed for tests (to make them reliable). Although, on further consideration, I guess it would be fine to control the seed in non-test code as well. The use case is, given a spec, to print out example data.#2018-02-2814:34bbrinckIn this case, the randomness of examples isnā€™t desirable#2018-02-2814:36taylorI also did this to auto-gen some API docs, but I was too lazy to try using a fixed seed#2018-02-2814:43gfredericksYeah, test.check generators are awkward in a few ways for the example use case#2018-02-2819:10jjttjjwhat if I want to spec something as being an ordered list of "params" or a map of the same params with names attached, like this:
(def s (s/or :arglist (s/cat :mylib/my-id (s/? :mylib/my-id)
                             :mylib/my-map :mylib/my-map
                             :x string?
                             :y boolean?
                             :z boolean?)
             :argmap (s/keys :req [:mylib/my-map]
                             :opt [:mylib/my-id]
                             :req-un [:mylib/x :mylib/y :mylib/z])))


(s/conform s {:mylib/my-map {:a "b"}, :x "asdf", :y true, :z false})
;;=> [:argmap {:mylib/my-map {:a "b"}, :x "asdf", :y true, :z false}]
(s/conform s [{:a "b"} "asdf" true, false])
;;=>[:arglist {:mylib/my-map {:a "b"}, :x "asdf", :y true, :z false}]
I have many different things that need to be spec'd like this. Is this worth trying to write some hacky function to make easier to type out? Or is this probably just a bad idea and I should stick to being explicit about it?
#2018-02-2819:15gfredericksthe :arglist version is unfortunate because it dictates the order of the keys#2018-02-2819:21jjttjjnot sure I get what you mean. the idea is i can figure out what i'm getting either via the order of the values, or by names being associated with the values#2018-02-2819:22gfredericksoh I'm sorry#2018-02-2819:22gfredericksI misread the first half as being like (f :a1 x :a2 y ...)#2018-02-2819:23gfredericksdisregard#2018-02-2819:23jjttjjno worries šŸ™‚#2018-02-2819:37Alex Miller (Clojure team)s/keys* can be used for your arglist I think#2018-02-2819:37Alex Miller (Clojure team)itā€™s like keys, but in any order, in a sequential structure#2018-02-2819:45gfredericksI don't think that kind of signature is even being used, that was my misunderstanding#2018-02-2820:00jjttjjyeah. it's more like "if x is a map, validate it with s/keys. if it's sequential, validate the conformed map after calling s/cat on it"#2018-02-2820:01mishais s/keys* considered being public api?#2018-02-2820:02jjttjjI believe it is, it's how you're supposed to validate optional keyword arguments i think#2018-02-2820:03jjttjjlike (json->clj x :keywordize true) or something. it's mentioned in the spec guide#2018-02-2820:12Alex Miller (Clojure team)yes, the * here is supposed to be evocative of s/*, not as a marker for something implementation-ish#2018-02-2820:13Alex Miller (Clojure team)you can still use s/keys* here in with a wrapper s/spec so you get a layer of nesting#2018-02-2820:15Alex Miller (Clojure team)
(def s (s/or :arglist (s/spec (s/keys* ...stuff...))
             :argmap (s/keys ...stuff...)))
#2018-02-2820:16Alex Miller (Clojure team)or maybe Iā€™m still not understanding the intent#2018-02-2820:17Alex Miller (Clojure team)I guess looking at your examples above, Iā€™m not#2018-02-2820:19Alex Miller (Clojure team)for the arglist, I think you can just (s/and (s/cat ...) (s/keys ...)) ? where the s/keys bit is the same spec as :argmap, presuming so, Iā€™d pull out the :argmap spec into itā€™s own thing to reuse it#2018-03-0101:41clumsyjediI must be missing something very simple about composing specs#2018-03-0101:41clumsyjedi
(s/def ::nums (s/alt :pos pos? :neg neg?))
(s/def ::int int?)
(s/def ::all (s/and ::nums ::int))
(s/valid? ::all 5)
;; => false
(s/explain ::all 5)
;; => val: [:pos 5] fails spec: :myns/int predicate: int?
#2018-03-0101:44clumsyjediso the ::nums spec uses s/conform and changes the value being validated. So ::int is trying to validate [:pos 5] and failing.#2018-03-0101:44clumsyjediHow do I avoid this though? I just want to chain validations together, each validating the original value I pass in#2018-03-0101:45gfredericksI feel like I ran into this recently but can't find any evidence of that#2018-03-0101:46gfredericksI think there's an s/unform or something like that that could be a workaround#2018-03-0101:46gfredericksbut that seems like it shouldn't be necessary#2018-03-0101:46gfredericksI know this has been talked about a lot, just don't remember any conclusions#2018-03-0102:02clumsyjedithanks @gfredericks - It seems like for my actual work scenario I can use s/merge instead of s/and.#2018-03-0102:03clumsyjediI'm curious what the recommended approach is for the example I pasted though#2018-03-0102:03clumsyjediI googled around for unform, didn't find much#2018-03-0103:17Alex Miller (Clojure team)Itā€™s best to start s/and with the pred that conforms to itself, particularly a type predicate like int? that will also generate#2018-03-0103:17Alex Miller (Clojure team)Youā€™re probably thinking of the undocumented s/nonconforming wrapper#2018-03-0103:18Alex Miller (Clojure team)Still on the fence about that one#2018-03-0103:19Alex Miller (Clojure team)However Iā€™d just do (s/and ::int ::nums) here#2018-03-0103:21Alex Miller (Clojure team)And s/or is better than s/alt here (although you wonā€™t see any difference in behavior until you combine ::nums with another regex spec#2018-03-0114:39mishaI use s/nonconforming quite often (not sure how to feel about this though)#2018-03-0115:57stathissiderishereā€™s a tricky situation: I have a very large spec to describe my very large config. The config contains values in different nested maps that need to be consistent with each other (some have to co-occur, some refer to each other so IDs need to be consistent etc), so in order to validate the config I have extra predicates on the top-level spec to check for consistency. That was great, until I tried generating a config, and of course it wasnā€™t valid (because itā€™s very unlikely to get consistency by chance). So I started writing overwriter functions that would be used via fmap in the generator in order to re-align the different parts of the randomly generated config. But it does feel a bit like an uphill struggle. Am I missing something?#2018-03-0116:15Alex Miller (Clojure team)generating large structures with semantic constraints is (inherently) hard#2018-03-0116:16Alex Miller (Clojure team)an alternate approach is to supply a custom generator at the point of use (with stest/check for example) that picks from example configs#2018-03-0116:16Alex Miller (Clojure team)by using a generator on a set of enumerated examples#2018-03-0116:34stathissideris@alexmiller thanks for the insight. I donā€™t like the idea of giving up the variability that I get from using an actual generator instead of example configs (which I assume youā€™re implying would he ā€œhandā€-written). Youā€™re right, itā€™s inherently hardā€¦ Iā€™ll see if I can refactor a bit to decomplect the constraints between distant parts of the structure, and also see if I can express the ones that cannot be avoided in a more fluent way.#2018-03-0116:45Alex Miller (Clojure team)the general approach if you are using custom generators is to first build a generator for a simpler model, then use gen/bind to create a generator that produces the appropriate structure#2018-03-0116:49stathissiderisIā€™ve done this before, but I think itā€™s not so applicable in this case, this one has cases where you get a bunch of IDs generated for part of the structure (say metrics collection), and then in another part (for example filter-metric) you need to refer to one of the existing metrics, so the overwriter just replaces the generated filter-metric with one from the IDs in the metrics collection#2018-03-0116:51stathissiderisyouā€™d think that a flag on one of the metrics (something like :filter?) would be betterā€¦ Iā€™m beginning to think the same šŸ™‚#2018-03-0117:11cap10morganI have a string I want to write a spec for, but it is composed of previously-spec'd string A, followed by a /, and then previously-spec'd string B. They are GitHub "user/repo" strings. If I already have a :github/user spec and a :github/repo spec, how would I write a spec for the "user/repo" strings that composes those first two specs?#2018-03-0117:13Alex Miller (Clojure team)spec is not great for doing composition of string specs like this - I suspect composing the patterns and creating a spec from that will probably end up better#2018-03-0117:13cap10morganšŸ‘‹ Hey Alex! OK, thanks. I figured it was either something like that or I was missing something really simple. I'll go down the route you suggest. Thanks!#2018-03-0117:14Alex Miller (Clojure team)itā€™s possible, I just donā€™t know that youā€™d be happy with the spec, the generated values, or the conformed values once you were done#2018-03-0117:14cap10morganI see#2018-03-0118:49misha@stathissideris generator is "just a function", so you can generate a set of ids upfront, and inject those in sub-generators. it will be verbose and might not be pretty, but sometimes you got to do what you got to do...#2018-03-0118:50misha(or change config format, if you control it)#2018-03-0118:50ghadithat's what I do ^#2018-03-0118:51ghadigenerate the stuff that you want to "align" first -- then pass them down#2018-03-0118:51mishaanother option, is to generate sub-structure and explicitly overwrite id. this saves you some code, and sub-generators customizations#2018-03-0118:53mishaofc, "thicker" dependency is ā€“ tighter your generators would be coupled, and "just finish it with assoc-in's" would not be enough#2018-03-0217:08jimbobI donā€™t really understand why Cloujre spec is any better than simple pre conditions for simple validation. example:
(defn mk-protobuf
  [{:keys [reward-value available event-uri event-name event-at schema revision
           hostname fake environment shopper-uri service-agent shopper-name reward-uri reward-name action-at]
    :or   {service-agent service-agent
           hostname      (hostname)
           environment   (str env)
           event-at      (c/to-epoch (t/now))
           schema        (str schema)}}]
   :pre  [(string? event-uri)
          (string? reward-uri)
          (string? shopper-uri)
          (string? reward-value)
          (instance? Boolean available)]
wonā€™t these preconditions be sufficient run time validation? what willl i get from spec that i cant get from these?
#2018-03-0217:09Alex Miller (Clojure team)docs, generated examples, automated generative testing#2018-03-0217:13jimbobi suppose so, but is not these pre conditions and or conditions documentation enough?#2018-03-0217:13jimboband much fewer lines of code#2018-03-0217:13bbrinck@ben.borders whatā€™s the error message if you pass in something incorrect? What if one of the args was not just a string, but, say, a vector of strings?#2018-03-0217:13jimbob(assuming we are not doing generative testing)#2018-03-0217:18jimbobits not terrible#2018-03-0217:18jimbobex:
Exception in thread "async-dispatch-12" java.lang.AssertionError: Assert failed: (integer? event-uri)
#2018-03-0217:18jimbobbut yes not as good as spec errors.#2018-03-0217:19bbrinckI think the different may be starker if you pass in nested data e.g. a vector of maps that must contain keys#2018-03-0217:19bbrinckbut for simpler predicates like string? (as in your example), the differences in error messages wonā€™t be as significant#2018-03-0217:20jimbobsure, but then couldnt i just use clojure.test/is ?#2018-03-0217:38nhaI donā€™t recall what exactly but this ended up causing trouble for me down the line (maybe it was nested is? I donā€™t remember now)#2018-03-0217:20jimboband get things like
;FAIL in 
#2018-03-0217:22bbrinckYes, although it becomes a little less clear for nested data e.g. a vector of strings#2018-03-0217:22bbrinckor vector of maps of specific keys#2018-03-0217:27jimbobi see#2018-03-0217:28jimbobthanks for your responses#2018-03-0217:29bbrinck@ben.borders np. hereā€™s an example of error message with is vs explain (using expound) https://gist.github.com/bhb/44d68a7ab878187b68d4fd48579d591b#2018-03-0217:29bbrinckI copied and pasted from my REPL session and omitted my mistakes, so hopefully I didnā€™t miss anything important šŸ˜‰#2018-03-0217:32jimbobso then to get the same desired runtime validation, would it be advised to place s/valid? in the :pre block of the function? or are there better ways to do this?#2018-03-0217:32jimbobiā€™ll probably end up exploring this.. it does indeed seem to be much better self documenting#2018-03-0217:33bbrinckthere are a few options. You can use fdef, or you could use a s/assert#2018-03-0217:34jimbobmm alright#2018-03-0217:34bbrinckfor s/assert, keep in mind youā€™ll need to run (s/check-asserts true)#2018-03-0217:35bbrinckyou could also use s/explain-str and raise an exception#2018-03-0217:35bbrinck(if the result is not literally "Success!\n")#2018-03-0217:43jimbobgotcha. and for speccing the example map above, would i have to create an s/def for every key in the map? for example some of the keys in the map should contain the same type and structure of data, so it would be nice to reuse something like
(s/def ::non-empty-string not-empty string?)
for multiple keys instead of
(s/def thing1 ::nonempty-string) (s/def thing2 ::Nonempty-string
#2018-03-0217:49Alex Miller (Clojure team)yes, you will need one for every key#2018-03-0217:49Alex Miller (Clojure team)that puts it in the spec registry which is used by s/keys#2018-03-0217:49jimbobok, gotcha thanks alex#2018-03-0217:50Alex Miller (Clojure team)if you have many repetitive such things, macros can help#2018-03-0217:50jimbobright, cool#2018-03-0217:50Alex Miller (Clojure team)maybe eventually weā€™ll provide something to def multiple things at a time#2018-03-0311:55pfeodrippeWoww, I saw the problem, s/and enters at the land of predicate instead of staying at the regex as said by the great @alexmiller at https://groups.google.com/forum/#!topic/clojure/S3UaS_4FRqY. Gonna use s/&#2018-03-0502:50seancorfield@mpcarolin You can use back tick you expand a symbol's name:
(-> (test/check `my-index-of) (test/summarize-results))
will probably work.
#2018-03-0502:52mpcarolin@seancorfield Hey, thanks! I did not know that existed. It will save me a lot of redundant typing.#2018-03-0506:15tianshuhow to write a spec allows only the specified keysļ¼Ÿ is there a convenient way?#2018-03-0506:17tianshuI want to use spec to constraint the schema of edn which used as a configuration file. I want useless keys are not allowed.#2018-03-0509:29stathissideris@doglooksgood there is no official way, you use s/and with s/keys and another predicate that constrains the keys to be in a specified set, but then you are listing the keys twice. Some people have written macros for this sort of thing#2018-03-0509:30stathissiderisbut spec is like that by design, probably the decision that has caused the most controversy about it
#2018-03-0509:31tianshuso I should write something like s/keys, okay#2018-03-0510:53taylorhereā€™s an example https://github.com/gfredericks/schpec/blob/b2d80cff29861925e7e3407ef3e5de25a90fa7cc/src/com/gfredericks/schpec.clj#L13#2018-03-0511:28tianshuThanks!#2018-03-0514:24Andreas Liljeqvistis https://github.com/clojure/spec.alpha the main repository for spec?#2018-03-0514:26Andreas LiljeqvistAsking because it is like 3 months since the last commit#2018-03-0514:31bronsayes#2018-03-0514:31bronsaclojure repos usually see activity in bursts not continuous#2018-03-0514:32bronsaapart from initial development#2018-03-0514:37Andreas Liljeqvistok, thanks. Looking to fix a bug#2018-03-0514:44Andreas LiljeqvistNow I just have to figure out how it is built, I expected a .deps.edn,project.clj, build.boot, makefile....#2018-03-0514:51Alex Miller (Clojure team)Itā€™s maven - you can test with mvn clean test etc#2018-03-0514:53Andreas Liljeqvistgot it to run with mvn clojure:compile#2018-03-0514:53Andreas Liljeqvistgoing with mvn clean test since I want to fix a problem#2018-03-0514:53Andreas Liljeqvistthansk#2018-03-0516:55nnboskoIs there a way to make a function return an error if parameters don't conform to their given spec?#2018-03-0517:37mpcarolinI think what youā€™re looking for is spec.test/instrument. Once you execute it, it will tell you if any :args specs are violated for a given function. #2018-03-0517:56mpcarolinE.g (stest/instrument ā€˜foo)#2018-03-0517:59mpcarolinBut if you want it at runtime without instrument, you can also use :pre condition for a function with s/valid? #2018-03-0516:57nnboskoThe function's given spec, anyways#2018-03-0517:17gfredericksdoes clojure 1.9 come with specs for clojure core functions somehow?#2018-03-0517:20taylorthere must be some coverage in there, hereā€™s a spec error for invalid let:
CompilerException clojure.lang.ExceptionInfo: Call to clojure.core/let did not conform to spec:
In: [0] val: () fails spec: :clojure.core.specs.alpha/bindings at: [:args :bindings :init-expr] predicate: any?,  Insufficient input
 #:clojure.spec.alpha{:problems [{:path [:args :bindings :init-expr], :reason "Insufficient input", :pred clojure.core/any?, :val (), :via [:clojure.core.specs.alpha/bindings :clojure.core.specs.alpha/bindings], :in [0]}], :spec #object[clojure.spec.alpha$regex_spec_impl$reify__2436 0x482c71e3 "
#2018-03-0517:23gfrederickswell that's a macro, which is a pretty different use case (in particular, it's checked at compilation; function specs would only get involved if a user intentionally instrumented them)#2018-03-0517:18gfredericks(or are there official specs somewhere else?)#2018-03-0517:29gfredericksI guess they'd probably be in core.specs.alpha if anywhere, and I see those are basically just a handful of macros#2018-03-0517:59bronsathatā€™s all there is available#2018-03-0519:22gfrederickssurely somebody somewhere is assembling a non-official collection of such things?#2018-03-0520:13bbrinckThe two I know of are https://github.com/leifp/spec-play and https://github.com/tirkarthi/respec . I have used neither but @alexmiller has pointed out errors in those specs#2018-03-0520:13bbrinckBut they might be sufficient for some perf testing#2018-03-0520:13gfredericks:+1: thanks#2018-03-0520:15Alex Miller (Clojure team)Theyā€™re not good#2018-03-0520:16bbrinckIf we had a large enough test suite across enough projects, could we infer ā€œgood enoughā€ specs for core functions with https://github.com/stathissideris/spec-provider ? šŸ¤”#2018-03-0520:17bbrinckI understand that for general usage, ā€œgood enoughā€ is not really a concept, or weā€™ll get confusing false positives and false negatives šŸ™‚#2018-03-0520:18bbrinckBut in my case, ā€œgood enoughā€ specs are useful so I can see if error messages can be improved#2018-03-0608:13stathissideris@U08EKSQMS spec inference for functions is still work in progress in spec-provider (but not too far off). I suspect it wouldnā€™t be able to give you anything very useful if you ran it on map though (which I think is very tricky to write a spec for anyway)#2018-03-0519:22gfredericksI wonder how much it would slow down a test to instrument all of clojure.core šŸ˜„#2018-03-0520:16Alex Miller (Clojure team)A lot :)#2018-03-0520:16gfredericksdo you think some aspect of this will be useful for some things at some point?#2018-03-0520:17Alex Miller (Clojure team)I hope so #2018-03-0520:18Alex Miller (Clojure team)I have worked with portions of core speced and itā€™s really useful at the repl#2018-03-0520:18gfredericksdo you think there's a reasonable place to draw the line that maximizes utility and speed?#2018-03-0520:18Alex Miller (Clojure team)I think itā€™s mostly not useful in a test suite (if the suite already runs)#2018-03-0520:19Alex Miller (Clojure team)I donā€™t feel ready to answer that yet#2018-03-0520:20gfredericks:+1:#2018-03-0519:25cap10morganI have a utility fn that can operate on any map, so its :args spec is just (s/cat :v (s/keys)). But when I do generative testing with that, I get a bunch of empty maps (which makes some sense I suppose). What's a good way to make that generate a bunch of random maps instead (ideally w/ an emphasis on namespaced keyword keys)?#2018-03-0519:32cap10morganShould I just use map? instead of s/keys?#2018-03-0519:34gfrederickssounds plausible I'm of the opinion that virtually all uses of maps fall into two nonoverlapping categories, roughly corresponding to s/keys and s/map-of#2018-03-0519:34gfredericksmap? seems to not commit to either#2018-03-0519:59cap10morganyeah, map? seems better#2018-03-0520:22bbrinck@alexmiller Are specs for the core library something that contributors could help with? In other words, are they blocked on dev/reviewer time, or are they more fundamentally blocked on any upcoming changes in spec?#2018-03-0520:24Alex Miller (Clojure team)They are really blocked on Rich review time. have written some of them but donā€™t have them public anywhere#2018-03-0520:24Alex Miller (Clojure team)They tend to raise many questions with issues of taste and judgement#2018-03-0520:25bbrinckI see, that indeed seems like it would require quite a bit of time to review#2018-03-0520:25bbrinckand change accordingly#2018-03-0520:26Alex Miller (Clojure team)Eric Normand actually was working on some of this at one point and I gave him some ideas of things to do with it#2018-03-0520:26bbrinckHe was working on official specs? Or on his own set?#2018-03-0520:28bbrinckI understand the issues of taste will take time to resolve, but to the extent that the official specs need testing, Iā€™m sure the community could provide feedback there#2018-03-0520:28Alex Miller (Clojure team)He wanted to help and so I told him what would be helpful #2018-03-0520:28bbrinckGotcha#2018-03-0520:29bbrinckPerhaps correctness is much easier than making idiomatic specs that conform/generate cleanly šŸ™‚#2018-03-0520:29gfredericksI doubt correctness is easy either#2018-03-0520:29seancorfieldAnd there's also the issue that some functions are extremely hard to spec correctly...#2018-03-0520:30bbrinckCorrect, although thatā€™s one place where the community could help - install specs, run tests, see what breaks#2018-03-0520:30Alex Miller (Clojure team)I donā€™t remember now what that was but I think I asked Eric to help in categorizing what was out there. I think he did but it was a while ago#2018-03-0520:31Alex Miller (Clojure team)My experience in testing core specs is that itā€™s extremely difficult to test more than one at a time#2018-03-0520:35bbrinckHm, thatā€™s a good point. Since we can see the list of isntrumentable symbols, a project could loop over them, instrument each one in isolation, and run tests#2018-03-0520:35bbrinckthat way, if there was a failure, it would be isolated from other specs.#2018-03-0520:36bbrinckagain, if it was useful to get broader validation of specs by running across a large variety of project#2018-03-0522:54ScotHi, is there a good way to spec anonymous functions? I am trying to use fspec, but I keep getting Cannot read property for_all_STAR_ of undefined#2018-03-0523:39seancorfield@scot-brown Do you have org.clojure/test.check as a dependency?#2018-03-0523:41seancorfieldIn order to check whether the function conforms to the spec, it is tested used a generative approach, so you need test.check in your project for that (`clojure.spec` will try to load it dynamically and you'll get some odd-looking failures if it isn't available)#2018-03-0611:20roklenarcicHow would I create a BigDecimal generator?#2018-03-0612:05tayloryou could use fmap to coerce some other number generator to BigDecimal but you wouldnā€™t get the full range of values (and you might not want that?)#2018-03-0614:18gfredericksyeah it's hard to give a good answer without knowing what kind of distribution you want; but (gen/let [factor gen/large-integer, log gen/int] (* factor (... 10 to the logth power ...))) wouldn't be bad#2018-03-0614:23gfredericksthere's probably a method on BigDecimal that would do the scaling a bit more naturally#2018-03-0700:07roklenarcicthanks#2018-03-0701:14taylorHaha I just read that the largest possible big decimal value can consume about 8GB RAM#2018-03-0701:33gfredericksI'm disappointed that there's a largest#2018-03-0619:02cap10morganIs there a good workaround for keeping NaN out of your spec-generated test data? Filtering at the top level doesn't keep them out of nested data structures, and they break all kinds of equality tests when they show up.#2018-03-0619:04seancorfield@cap10morgan I thought the double-in generator had options to suppress that?#2018-03-0619:04cap10morganI'm working with any?#2018-03-0619:05seancorfieldAh, then you'll get "anything" šŸ™‚#2018-03-0619:07cap10morganyep. there just doesn't seem to be a way to define any-but-nan. my code handles NaNs just fine, it's the specs that barf on them (since the equality tests in my predicates choke on them).#2018-03-0619:07cap10morganmakes spec'ing fns that can handle any value tricky#2018-03-0620:24cap10morganI have tested it locally and it seems to solve the problem.#2018-03-0621:09gfredericksthis is a sticky issue because merely removing NaN from everywhere isn't obviously the best thing, since NaN is an important edge case to test in a lot of situations#2018-03-0621:09gfredericksit's obviously hard to opt-out of at the moment#2018-03-0621:17cap10morganAgreed but the concept of removing it from the any? generator seemed at least tacitly endorsed by the core team.#2018-03-0621:18cap10morganIf they instead requested a way to toggle it, Iā€™d be happy to work on that too.#2018-03-0621:19noisesmithpedantically, if you accept any value, you better do the right thing if the value is NaN right?#2018-03-0621:21cap10morganYes. And the code does. Only the spec has a problem. And it is not easy to work around.#2018-03-0621:21cap10morgan(In my use case)#2018-03-0621:22cap10morganA NaN at any level of nesting blows up equality checks in predicates.#2018-03-0621:23dpsuttonand you said your code gracefully handles that, right?#2018-03-0621:23dpsutton> do the right thing if the value is NaN right?#2018-03-0621:25cap10morganYes, the code handles it just fine.#2018-03-0621:25cap10morganItā€™s not doing any equality checking.#2018-03-0621:30cap10morganI think my ideal solution might be a spec-specific equality function that handled NaN, but I don't know where the core team stands on that.#2018-03-0621:32Alex Miller (Clojure team)Just driving by here - would much rather prevent the NaN from occurring in the first place. Custom equality has never gone well imho :)#2018-03-0621:33cap10morgan@alexmiller That's fair. šŸ™‚ Does that patch ^^^ seem OK to attach to the JIRA ticket?#2018-03-0621:36Alex Miller (Clojure team)No time to look atm but itā€™s ok either way :)#2018-03-0621:36cap10morganOK, attaching it then.#2018-03-0623:06gfredericksthe downside to that is that the list of generators that gen/any is composed of is now in two places#2018-03-0623:08cap10morgan@gfredericks Not sure I follow. Where are the 2 places?#2018-03-0623:10gfredericksDidn't you paste & tweak that code from test.check?#2018-03-0623:13cap10morganyes, but it is the one and only code for the any? generator now. and test.check itself largely repeats that one-of form between the any and any-printable generators.#2018-03-0623:14gfredericksYes, I was pointing out that if test.check added anything to its gen/any, that wouldn't propogate to spec anymore#2018-03-0623:14cap10morganIf test.check defined the vector separately I definitely would have just re-used it though. Not sure there's a good way to re-use it as is.#2018-03-0623:14gfredericksProbably not#2018-03-0623:15gfredericksEven if test.check changed you couldn't take advantage of that without introducing a stricter version requirement between the two. I'm not sure if that's acceptable or not#2018-03-0623:18Alex Miller (Clojure team)Not#2018-03-0700:06gfredericksthat was my guess#2018-03-0715:36Andreas Liljeqvistis there any function like (complete ::somespec {::provided-value 2}) that will generate keys that aren't provided?#2018-03-0715:37Andreas LiljeqvistI could provide overrides like {::provided-value #(gen/return 2)}#2018-03-0716:33Andreas LiljeqvistI have written such a function, just interested to know if it exists somewhere in spec#2018-03-0717:11jimbobis there any way to have fdef functions be runtime validated?#2018-03-0717:11jimbobOr are these function specs only for documentation / testing and generation#2018-03-0717:11tayloryou can instrument the functions to check the :args specs#2018-03-0717:12taylorand thereā€™s another lib called Orchestra that gives you a version of instrument that checks :args and :ret (and maybe :fn?)#2018-03-0717:12taylor@ben.borders https://clojure.org/guides/spec#_instrumentation#2018-03-0717:13jimbobbrilliant. exactly what i was looking for, thanks a lot @taylor!#2018-03-0721:59aengelbergIs this a bug?
user=> (s/def ::a integer?)
:user/a
user=> (s/explain (s/merge (s/keys :req [::a]) (s/keys :req [::b])) {::a "a" ::b 1})
In: [:user/a] val: "a" fails spec: :user/a at: [:user/a] predicate: integer?
In: [:user/a] val: "a" fails spec: :user/a at: [:user/a] predicate: integer?
nil
#2018-03-0722:00aengelbergI would expect only one message to get spit out, not two. But since s/merge calls each of its sub-maps' validators, it gets the error message twice.#2018-03-0722:35misha@aengelberg each s/keys validates keys for presence, and every key of a known spec (even not mentioned ones) for conformance#2018-03-0722:37misha
(s/def ::a integer?)
(s/def ::m (s/keys :req [::b]))
(s/explain ::m {::a "a"})
In: [:user/a] val: "a" fails spec: :user/a at: [:user/a] predicate: integer?
val: #:user{:a "a"} fails spec: :user/m predicate: (contains? % :user/b)
=> nil
#2018-03-0722:38mishaand s/merge does not reduce issues from each subspec#2018-03-0811:29Andreas LiljeqvistIf I found a lot of problems with indentation in spec source, what is the best chance of doing a successful pull-request?#2018-03-0813:05souenzzohttps://github.com/clojure/core.specs.alpha/blob/master/CONTRIBUTING.md#2018-03-0814:03Andreas LiljeqvistThanks, actually I am registered as a contributor. More like if the changes would be accepted in any form. Perhaps they don't consider it as a problem#2018-03-0814:43tbaldridgeI doubt it, itā€™s really hard to make a good argument about indentation#2018-03-0814:43tbaldridgeGot an example though?#2018-03-0816:48Andreas Liljeqvisthttps://pastebin.com/6w3Rs1ai#2018-03-0816:48Andreas LiljeqvistLine 8 and 9 seems strange#2018-03-0816:55Andreas LiljeqvistPerhaps this also: https://github.com/clojure/spec.alpha/blob/739c1af56dae621aedf1bb282025a0d676eff713/src/main/clojure/clojure/spec/alpha.clj#L754#2018-03-0913:00tbaldridgeon the second one I don't see a problem at all#2018-03-0913:01tbaldridgeto be frank, indentation doesn't matter much in Clojure, so I doubt these sort of patches would ever make it through.#2018-03-0913:01tbaldridgeThe style guides that do exist for Clojure are more opinions than actual rules, so it's fairly common to see changes in indent styles across projects.#2018-03-0913:43Andreas LiljeqvistDisclaimer, I know I cannot possible "win" this#2018-03-0913:43Andreas LiljeqvistThe problem is that a lot of tools force indentation, parinfer for example. So diffs becomes problematic#2018-03-0913:44Andreas LiljeqvistThis is kind of a problem with the tools, but I figure that Clojure would be better of with a standard like pep8#2018-03-0913:46Andreas LiljeqvistMostly because indentation really doesn't matter that much. But a standard that enables linters and automatic formatters is valuable according to me#2018-03-0914:34Andreas Liljeqvisthttps://github.com/bbatsov/clojure-style-guide and usage of parinfer could eventually become a defacto standard#2018-03-0918:30souenzzoI highly recommend you to watch this video https://www.youtube.com/watch?v=1YCkOo5Y4Oo#2018-03-0811:35mpenetClose to none#2018-03-0912:47guyHi people, whats the best way to generate a not blank string?#2018-03-0912:48gfredericksyou can wrap the generator in gen/not-empty unless you meant something different by "blank"#2018-03-0912:48guyok thanks!#2018-03-0915:40Andreas LiljeqvistI have made a fix for CLJ-2311 - an issue that was preventing me from providing an override for the dispatch-fn(key) in a multimethod#2018-03-0915:40Andreas Liljeqvisthttps://dev.clojure.org/jira/browse/CLJ-2311#2018-03-0915:41Andreas Liljeqvistplease test it and leave comments#2018-03-0915:42Andreas Liljeqvistsource here: https://github.com/bonega/spec.alpha/tree/multi-spec-override#2018-03-0917:26gfredericksspec doesn't have anything analogous to schema completers, is that correct?#2018-03-0917:27gfrederickshas anybody used spec somehow to fill in incomplete test data?#2018-03-0921:25misha@gfredericks (merge (ffirst (s/exercise :foo/bar)) incomplete)? :)#2018-03-0922:22gfredericksthat works at one level, yes#2018-03-0922:22gfredericksschema does it recursively#2018-03-1002:33mpcarolinHi everybody. Iā€™ve been learning spec lately, and I think I have a decent grasp of its mechanics. What I donā€™t fully grasp is the real life effective use of spec. When should we be using spec? Should I be speccing as many of my functions and as much of my data as I can? Or just the borders? Or just what is publicly accessible?#2018-03-1003:32Alex Miller (Clojure team)there is no one right answer to that question#2018-03-1003:32Alex Miller (Clojure team)I donā€™t think you should or need to instrument everything#2018-03-1003:33Alex Miller (Clojure team)specā€™ing at the borders is particularly helpful, but any function with sufficiently complex inputs or outputs is worth considering#2018-03-1003:33Alex Miller (Clojure team)instrumentation is worth using at dev time, check is worth doing at test time, and validation is (maybe) worth doing at runtime#2018-03-1004:49mpcarolinThanks for the response. Iā€™ll keep that in mind!#2018-03-1021:14borkdudeweā€™re considering putting some (s/assert ...) at the beginnings of some functions, which can be compiled away with the right system property#2018-03-1021:15borkdudeinteresting thought on spec: https://twitter.com/pjstadig/status/972434922794373121#2018-03-1021:20Alex Miller (Clojure team)Well, disagree#2018-03-1108:37mishaboy, I hope that failing to read doc from db due to validation error about extra unspecified kv ā€“ is not "the clojure way"#2018-03-1109:06borkdudeI think his argument was more about representing specs in data rather than code#2018-03-1110:29ikitommiboth open & closed specs/schemas have valid use cases. Some comments from the GraphQL world: https://github.com/facebook/graphql/issues/127#2018-03-1111:12mishaThe defaults matter.#2018-03-1111:22borkdudeWe have a poor manā€™s graphQL in which we can express *: {:a {:filter/default {:b true}}} applied to {:a {:some-key {:b 1 :c 2}}} gives you {:a {:some-key {:b 1}}}#2018-03-1120:43borkdudeIdea: https://github.com/bhauman/rebel-readline/issues/135#2018-03-1120:44borkdudeAs of now itā€™s kind of hard to relate function arguments to the specs for them. Maybe this is what the above tweet was also about: creating a different naming scheme#2018-03-1214:08misha@borkdude what do you mean by ā€œrelateā€ exactly?#2018-03-1214:10mishaThe only coupling/relation you should have is order.#2018-03-1214:36oconnIā€™m working on writing a spec for a model that uses multi-spec. Is it possible to use unqualified keywords in a multi-spec? I found this thread https://www.mail-archive.com/clojure@googlegroups.com/msg99107.html and tried altering the example in the docs with no success.#2018-03-1214:43bbrinck> Is it possible to use unqualified keywords in a multi-spec? @oconn Can you explain this a little bit more? Iā€™m not sure I understand the question.#2018-03-1214:44taylor@oconn itā€™s possible, and the example in the multi-spec doc string uses an unqualified keyword#2018-03-1214:44taylorhttps://clojure.github.io/spec.alpha/clojure.spec.alpha-api.html#clojure.spec.alpha/multi-spec#2018-03-1214:46oconnOh nice, yeah not sure what Iā€™m doing wrong here.. So hereā€™s the spec#2018-03-1214:49taylorthat explain returns ā€œSuccess!ā€ for me#2018-03-1214:50taylorhaving trouble inferring what the variant-type "Text" spec is intending to do though#2018-03-1214:51oconnI stripped out a lot of other keys, there are other types as well.#2018-03-1214:52bbrinck@oconn Can you expand a bit on what is happening when you call explain and what you expect to happen?#2018-03-1214:55bbrinckI also see "Success!" printed, but Iā€™m guessing thatā€™s not what you expect (?)#2018-03-1214:56bbrinck(minor thing, but your defmethod appears to have a missing paren at the end)#2018-03-1214:56oconnYeah, not seeing that but I think itā€™s because I stripped out some keys before pasting here.#2018-03-1214:57oconnOk, now Iā€™m getting Success!. Sorry for the trouble#2018-03-1214:57bbrincknp, good luck!#2018-03-1219:23eggsyntaxI'm currently using gen/generate to make some sample data for myself, including fairly nested structures which use s/* at multiple levels. It'd be nice during dev to have the produced data be fairly short so it doesn't take up a whole repl screen. Wasn't there some way to tell it to keep those short? I vaguely thought there was. From the documentation it seems like it would be s/*recursion-limit*, but if I do
(binding [s/*recursion-limit* 1] (sgen/generate (s/gen ::my-spec)))
I still get large numbers of items on the s/* specs.
#2018-03-1219:26eggsyntaxOh, I see why it's not *recursion-limit*, I think; it's not actually recurring many layers deep, just creating a long sequence at each layer.#2018-03-1219:28eggsyntaxBut it'd be really nice to have an equivalent way to force shorter sequences. Am I missing something?#2018-03-1219:30mishaI'm afraid you are not missing anything, @eggsyntax#2018-03-1219:31eggsyntaxAh well. I guess I can walk it and take 3 or something.#2018-03-1219:31mishayou can provide overrides, which wrap original seq specs with just items count upper limit#2018-03-1219:31mishayeah ~#2018-03-1219:32eggsyntaxOh, so something like (s/with-gen (s/* ...) (take 3 ...))#2018-03-1219:32mishaI'd love to be proven wrong on this one, though : )#2018-03-1219:33eggsyntaxIt'd make a terrific addition to a future version of spec, to have s/*sequence-limit* alongside s/*recursion-limit*.#2018-03-1219:34mishaI doubt it is a good idea to have a single global count limit. I'd rather had a way to override :min-count/:max-count opts for maps and seqs specs#2018-03-1219:35eggsyntaxFrom my POV, that'd be nice too -- but for the case of just quickly generating a piece of data in the REPL, it'd be lovely to just bind a global limit.#2018-03-1219:37eggsyntaxHmm, test.check does have :min-elements and :max-elements on a number of its generators, I see....#2018-03-1219:37mishalike an extra count args here:
(with-gen* [_ gfn] (every-impl form pred opts gfn))
#2018-03-1219:37mishahttps://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/alpha.clj#L1318#2018-03-1219:38eggsyntaxThanks @misha šŸ™‚#2018-03-1220:59trisswhy are all the facilities for defining specs macros? Iā€™m tempted to use spec for pattern matching but the syntax is too heavyā€¦#2018-03-1221:29Alex Miller (Clojure team)theyā€™re macros to capture the form for explanations#2018-03-1221:29Alex Miller (Clojure team)that said, some of that is likely to change in the next rev#2018-03-1221:57pavani@alexmiller How can I validate a schema like this using Spec?
{:billing_contact
                          {
                           (schema/optional-key :street_address) schema/Str
                           (schema/optional-key :city)           schema/Str
                           (schema/optional-key :region)         schema/Str
                           (schema/optional-key :postal_code)    schema/Str}}
I have the following spec but if ::billing_contact has keys other than the optional keys, it considers the map to be valid. I just want to check if the map has either those specified keys in opt or no keys.
(s/def ::street_address string?)
(s/def ::city string?)
(s/def ::region string?)
(s/def ::postal_code string?)
(s/def ::billing_contact (s/keys :opt [::street_address ::city ::region ::postal_code]))
(s/def ::changes (s/keys :req [::billing_contact]))
#2018-03-1222:06Alex Miller (Clojure team)You can s/and with an every? predicate that checks every key is a known key set#2018-03-1222:16pavani@alexmiller Thank you for that. I am not sure how exactly I could use that here. Could you please exhibit that with my example?#2018-03-1303:19bbrinck@U7MMSSQKC Perhaps something like (s/def ::billing_contact (s/and (s/keys :opt [::street_address ::city ::region ::postal_code]) #(every? {::street_address ::city ::region ::postal_code} %))) ?#2018-03-1303:30bbrinckWhoops, I forgot the call to keys. Would this work: (s/def ::billing_contact (s/and (s/keys :opt [::street_address ::city ::region ::postal_code]) #(every? #{::city ::street_address ::region ::postal_code} (keys %))))#2018-03-1322:10pavaniThanks a lot @U08EKSQMS#2018-03-1303:36rahulr92Hi, can someone help me understand the difference between spec/and and spec/merge? The example of merge given in the Clojure Spec guide https://clojure.org/guides/spec (using :animal/common and :animal/dog) seems to work even if we use and instead of merge.#2018-03-1304:17seancorfield@rahulr92 s/merge will merge two s/keys specs (and, if I recall correctly, does not flow the conformed value); s/and is for combining a general spec with a general predicate (or spec) -- and it does flow the conformed value.#2018-03-1304:21seancorfieldAlso, if you look at the docstrings, the differences should be reasonably clear https://clojure.github.io/spec.alpha/clojure.spec.alpha-api.html#clojure.spec.alpha/merge#2018-03-1304:22seancorfieldhttps://clojure.github.io/spec.alpha/clojure.spec.alpha-api.html#clojure.spec.alpha/and -- that specifies that conformed values are propagated.#2018-03-1304:25seancorfieldIn the merge docs, you'll see another difference: it generates maps that conform to all the merged specs. With and, the first spec is used as the generator, and then the other specs are used as predicates to filter the generated values (which may not satisfy those predicates and so overall generation of values can fail).#2018-03-1304:25seancorfieldHope that helps @rahulr92?#2018-03-1304:40rahulr92@seancorfield Thanks for the help. I understood the difference in generated value. However I still do not fully understand the difference while using these for validation. Could you please elaborated on what you mean by the ā€˜flowā€™ of conformed value? > ā€™Unlike ā€˜andā€™, merge can generate maps satisfying the union of the predicates. Doesnā€™t and also perform union i.e. it passes validation only if all the specs in it pass?#2018-03-1304:52seancorfield@rahulr92 Here's an example showing a difference
boot.user=> (require '[clojure.spec.alpha :as s])
nil
boot.user=> (s/def ::a (s/or :i int? :s string?))
:boot.user/a
boot.user=> (s/def ::b (s/or :b boolean? :s string?))
:boot.user/b
boot.user=> (s/def ::merged (s/merge (s/keys :req [::a]) (s/keys :req [::b])))
:boot.user/merged
boot.user=> (s/def ::anded (s/and (s/keys :req [::a]) (s/keys :req [::b])))
:boot.user/anded
boot.user=> (s/conform ::merged {::a 1 ::b "x"})
#:boot.user{:a [:i 1], :b [:s "x"]}
boot.user=> (s/conform ::anded {::a 1 ::b "x"})
:clojure.spec.alpha/invalid
boot.user=>
#2018-03-1304:55seancorfieldIf you look at the result of conforming that map against (s/keys :req [::a]) you'll see why it fails with s/and but works with s/merge:
boot.user=> (s/conform (s/keys :req [::a]) {::a 1 ::b "x"})
#:boot.user{:a [:i 1], :b [:s "x"]}
boot.user=>
#2018-03-1304:58seancorfieldand finally, regarding generation:
boot.user=> (s/exercise ::merged)
([#:boot.user{:a 0, :b false} #:boot.user{:a [:i 0], :b [:b false]}] [#:boot.user{:a "", :b "N"} #:boot.user{:a [:s ""], :b [:s "N"]}] [#:boot.user{:a "u", :b "N"} #:boot.user{:a [:s "u"], :b [:s "N"]}] [#:boot.user{:a "", :b false} #:boot.user{:a [:s ""], :b [:b false]}] [#:boot.user{:a "8", :b true} #:boot.user{:a [:s "8"], :b [:b true]}] [#:boot.user{:a "u", :b "k"} #:boot.user{:a [:s "u"], :b [:s "k"]}] [#:boot.user{:a 2, :b "IzO"} #:boot.user{:a [:i 2], :b [:s "IzO"]}] [#:boot.user{:a 1, :b "8s0"} #:boot.user{:a [:i 1], :b [:s "8s0"]}] [#:boot.user{:a 0, :b "QsU1J"} #:boot.user{:a [:i 0], :b [:s "QsU1J"]}] [#:boot.user{:a 11, :b true} #:boot.user{:a [:i 11], :b [:b true]}])
boot.user=> (s/exercise ::anded)

clojure.lang.ExceptionInfo: Couldn't satisfy such-that predicate after 100 tries.
#2018-03-1304:58seancorfield(which is no surprise because the ::anded spec won't accept anything)#2018-03-1304:58seancorfieldDoes that help @rahulr92?#2018-03-1305:09rahulr92@seancorfield Great explanation. This really helps. Thanks a lot!#2018-03-1316:50tbaldridgeI'm not sure if anyone else has had this issue before, but I have a really strange spec problem that has shown up twice: I have a spec that defines a or:#2018-03-1316:51tbaldridge(s/def ::val (s/or :int integer? :symbol :simple-symbol?))#2018-03-1316:51tbaldridgeAnd then I put that inside something else: (s/def ::coll-of-stuff (s/coll-of ::val))#2018-03-1316:53tbaldridgeBut when I try to validate some data, explain fails and tells me: (s/explain ::coll-of-stuff '[x 1 2]) "value [:int 1] failed to conform to integer? or symbol?"#2018-03-1316:54tbaldridgeThe problem is that something is trying to validate the conformed value? What's strange is I can't seem to reproduce this in a smaller situation. But so far it's always been with s/or and every time it tries to validate the conformed value#2018-03-1316:57guyhow are you doing the symbols out of interest?#2018-03-1316:58tbaldridgedoing the symbols?#2018-03-1316:58guy(s/explain-data ::vals ['s 1])#2018-03-1316:58guylike that?#2018-03-1316:58tbaldridgethat was part of the example here, I'm actually using keywords#2018-03-1316:58guyThe symbols in your data you are trying i mean sorry#2018-03-1316:59guySorry iā€™m a little confused#2018-03-1316:59tbaldridgeSo here's the actual error message:
In: [:ast/select-attrs 0] val: [:attr :bar] fails spec: :ast/select-attr at: [:ast/select :ast/select-attrs :attr] predicate: simple-keyword?
In: [:ast/select-attrs 0] val: [:attr :bar] fails spec: :ast/select-attr at: [:ast/select :ast/select-attrs :child] predicate: map?
nil

#2018-03-1316:59tbaldridge(namespace names changed for security reasons#2018-03-1317:02tbaldridgeAnd the relevant specs:
(s/def ::select-attr (s/or
                       :attr simple-keyword?
                       :child (s/keys :req [::from ::select-attrs ::logical-attr]
                                      :opt [::where])))
(s/def ::select-attrs (s/coll-of ::select-attr))

#2018-03-1317:02tbaldridgeSo what I don't understand is why it's saying [:attr :bar] is failing to conform to simple-keyword?#2018-03-1317:02tbaldridgeIt's like or is trying to validate after it conforms the value or something.#2018-03-1317:03jgdaveyWouldnā€™t [:attr :bar] be a (s/coll-of simple-keyword?), rather than a simple-keyword??#2018-03-1317:05tbaldridgethis is interesting, at the top level, there's a multi-spec. If I remove that it seems to work fine.#2018-03-1317:06jgdaveyOh, I see, youā€™re saying that [:attr :bar] is what youā€™d get after conforming (s/conform ::select-attr :bar)#2018-03-1317:06tbaldridge@jgdavey that's what's strange, the data I'm passing in looks like this [:foo :bar]. Hence the path in the error being [:ast/select-attrs 0] so it gets into the select-attrs path, and then the first item doesn't validate because [:attr :bar] is not a simple-keyword.#2018-03-1317:06tbaldridgeright#2018-03-1317:07jgdaveyDoes using s/every instead of s/coll-of yield the same results?#2018-03-1317:08tbaldridgewow, yep, I changed my ::select-op spec to this:#2018-03-1317:08tbaldridge
(s/def ::select-op (s/and
                     (fn [x]
                       (println "VALIDATE " x)
                       x)
                     (s/keys :req [::select-attrs ::from ::where])))
#2018-03-1317:09tbaldridgeThat's the top level spec. If I run it through multi-spec the values it tries to conform are [:attr :bar]. If I run it without multi-spec, the values are :bar#2018-03-1317:09tbaldridgeSo multi-spec is doing something#2018-03-1317:15tbaldridgelet's see if I can get a minimal case now#2018-03-1317:27cap10morgan@alexmiller Does https://dev.clojure.org/jira/browse/CLJ-2054 need to be marked as "ready for testing?"#2018-03-1319:04Alex Miller (Clojure team)it is already marked thusly#2018-03-1320:25cap10morganah, great. thanks!#2018-03-1317:38tbaldridgewait...does:
(s/and ::foo
            ::bar)
pass the conformed value from ::foo to ::bar?
#2018-03-1317:38tbaldridgeguess so: " Returns a spec that returns the conformed value. Successive conformed values propagate through rest of predicates."#2018-03-1317:42tbaldridgeboom that's it, @jgdavey I used s/and and that flows conformed values through the predicates (rather mis-leading name). Replaced it with s/merge and it works fine#2018-03-1317:43jgdaveyInteresting. Thatā€™s probably a subtly worth calling out in docs or something.#2018-03-1318:02tbaldridgeyeah, funny thing is that the docs are fairly clear, both s/and and s/merge mention it. I wonder if the behavior there changed somewhat recently#2018-03-1318:05tbaldridgelol, nope, it's been like that since the start#2018-03-1318:24mpenetI guess both could have options to control how/if the flow of conformed values is done#2018-03-1318:26mpenetI tripped on the behavior of s/merge when I started with spec, so the other way around. I am not alone either#2018-03-1319:00tbaldridgeYeah, I think part of it is that names are a tad overloaded in English. Merge makes sense to me since I'd expect (s/merge ...) to and things together. But it seems like a better name for s/and would be s/flow or something like that.#2018-03-1319:01tbaldridgeMaybe we should rename s/merge to s/smoosh šŸ˜›#2018-03-1319:01Alex Miller (Clojure team)we considered having s/and (non-flowing) and s/and-> (flowing) at one point#2018-03-1319:01tbaldridges/& works the same way#2018-03-1319:01Alex Miller (Clojure team)unfortunately the current s/and does the latter :(#2018-03-1319:02tbaldridgebut in the end, I didn't read the docs, so I'll slap my own hand this time#2018-03-1319:02Alex Miller (Clojure team)yes, s/and and s/& flow. s/merge does not#2018-03-1319:02Alex Miller (Clojure team)s/merge is conceptually pushing the value through all specs ā€œin parallelā€#2018-03-1319:03Alex Miller (Clojure team)so flowing doesnā€™t make sense (except conforming returns the conformed result of the last spec which is hard to reconcile with that view)#2018-03-1319:06tbaldridgeSo what's the common use-case for flowing specs? I think I've only ever used them with s/conformer, which I tend to avoid these days#2018-03-1319:12Alex Miller (Clojure team)if you flow through a regex youā€™ll get structure which you can then easily apply predicates to#2018-03-1320:02bronsanaming vars & is not the best idea#2018-03-1320:15Alex Miller (Clojure team)it was originally called andre :)#2018-03-1320:19mishaflowing is handy when you have s/keys spec checking map's keys and vals independently, and then you apply predicate checking inter-kv-dependencies.#2018-03-1320:20mishaalthough, thinking about it a moment longer ā€“ this use case is sort of orthogonal to flowing :D#2018-03-1320:28tbaldridge'Is it pronounced "an-dray" or "and-re"?'#2018-03-1320:28tbaldridgeWould be the new "a-sosh" or "a-sock". šŸ˜„#2018-03-1320:49Alex Miller (Clojure team)the former#2018-03-1401:32gfredericksif it was big you could call it andre the giant macro#2018-03-1401:33gfredericks@bronsa what's wrong with &?#2018-03-1408:31bronsasyntax quote doesn't qualify it#2018-03-1411:56gfrederickswhy on earth would that be true?#2018-03-1412:12bronsabecause otherwise youā€™d have to
`(fn [a# ~'& b])
#2018-03-1412:24gfredericksaw dang I figured it somehow related to &env and &form; forgot about destructuring#2018-03-1412:25gfredericksso this only affects users who are refering s/&#2018-03-1412:25bronsayeah#2018-03-1412:25bronsaor macros defined in the same ns#2018-03-1412:25bronsabut tbh the same happens when naming a var def#2018-03-1412:25bronsawhich I also donā€™t like#2018-03-1412:26gfredericksthat's a different underlying cause, right?#2018-03-1412:26bronsano same reason#2018-03-1412:27gfredericksI guess I figured special formedness was checked in one place and &ness somewhere else; are there any other exceptions?#2018-03-1412:28bronsatheyā€™re all in one place#2018-03-1412:28bronsa
user=> (keys Compiler/specials)
(& monitor-exit case* try reify* finally loop* do letfn* if clojure.core/import* new deftype* let* fn* recur set! . var quote catch throw monitor-enter def)
#2018-03-1412:28bronsa& is considered a special form by the reader#2018-03-1412:33gfrederickswelp.
#2018-03-1413:47mikerodInsightful#2018-03-1422:28dpsuttoncan you spec cljs.core/+?#2018-03-1422:31dpsuttonand i realize that's a seemingly silly question but we've got js/big floating around which allows for arbitrary precision decimal arithmetic but the built in operators do not behave well. So i'd like to spec it and use it to help verify that no js/Big's are leaking into any arithmetic anywhere#2018-03-1500:45gfredericksI don't think you could instrument it (which I assume is the point) at runtime, since it's normally inlined#2018-03-1500:46gfredericksI'm curious how easily compilation could be modified to avoid the inlining does cljs spec do runtime instrumentation at all? I don't even know that#2018-03-1505:11didibusHow does one go about adding functionality to spec? I tried to extend s/keys so that it also did a contains check, basically making it a closed-key spec. I did so by wrapping s/keys in a separate macro which takes the same input then s/keys and returns an anded spec of keys and one which will validate that no extra keys are present. But composing macros like that isn't great. And so now I'm trying to extend it even more, so that my closed-keys macro can be used in a s/merge style. Anyways, bottom line is, the experiment has been tedious. So I was thinking, maybe instead of building on top of s/keys, I can just build a whole new spec, but how do you do that?#2018-03-1505:13didibusI also found it frustrated all the specs were macros, which some of them were plain old functions that took clojure data. Like s/keys could have taken a map, or a vector of vectors, instead of a custom macro DSL.#2018-03-1505:57ikitommi@didibus current version of spec doesnā€™t really support extensions. But you can reify clojure.spec.alpha/Spec and copy-paste current spec internals to make something similar.#2018-03-1506:00ikitommiI think s/keys could be rewritten as a function, as all keys need to be referenced via keyword. Something like: https://github.com/metosin/spec-tools/blob/master/src/spec_tools/data_spec.cljc#L39-L68.#2018-03-1506:01ikitommi.. actually the closed keys would be easy to implement on top of that via extra :closed? key or similar. hmm.#2018-03-1513:44guyIf you were writing an fdef for a simple function that takes a collection of strings (at least one) could you use (s/+ string?) instead of (s/coll-of string?) for the :args.#2018-03-1513:45guy
(s/fdef my-fn
  :args (s/cat :item (s/+ string?))
  
  or 
  
  :args (s/cat :item (s/coll-of string?))
  ..)
#2018-03-1515:59madstapThose are different though, (s/cat :item (s/spec (s/+ string?))) works the same as the latter.#2018-03-1516:16guy@U0J9LVB6G ah thanks!#2018-03-1516:22guy@U0J9LVB6G what was the difference out of interest? What does s/spec do to it?#2018-03-1516:23madstaps/spec makes it nested#2018-03-1516:23madstap
(s/conform (s/cat :item (s/+ string?)) ["a" "b"])

(s/conform (s/cat :item (s/spec (s/+ string?))) [["a" "b"]])

(s/conform (s/cat :item (s/coll-of string?)) [["a" "b"]])
#2018-03-1516:24guyahhhhh right got it thanks#2018-03-1514:07Alex Miller (Clojure team)absolutely#2018-03-1514:07Alex Miller (Clojure team)I would even say thatā€™s preferred (although you wonā€™t really see much practical difference)#2018-03-1514:14guyok great thanks#2018-03-1514:26bbrinck@guy keep in mind coll-of also has :min-count arg which might be useful here#2018-03-1514:26bbrinck(although more verbose than +)#2018-03-1514:26guyah yes thats right thank you!#2018-03-1514:33Alex Miller (Clojure team)generally when specā€™ing function signatures, youā€™re specā€™ing syntax, and spec regex ops are the best match so I usually try to stick to those in fdef args#2018-03-1514:33guyok so in general favour spec regex ops then?#2018-03-1514:40Alex Miller (Clojure team)when specā€™ing function args, favor regex op specs#2018-03-1514:41Alex Miller (Clojure team)when specā€™ing other kinds of data, I favor collection specs#2018-03-1514:42guyCan you give me an example of the latter sorry#2018-03-1514:42guyDo you mean if you were creating a spec for some sample data perhaps?#2018-03-1514:49Alex Miller (Clojure team)yeah#2018-03-1514:50guythanks for the help!#2018-03-1515:39borkdudeI have a spec:
(s/keys :req-un [::title
                   ::content]
          :opt-un [::icon
                   ::widget-class
                   ::settings
                   ::collapsed?
                   ::dropdowns
                   ::controls
                   ::preview?
                   ::tabs
                   ::collapse-opts
                   ::help])
Now I want to have a warning for myself when someone passes something that has more keys than I expected. Can I use my spec for this or do I have to duplicate the keys?
#2018-03-1515:40borkdudeIā€™m refactoring this component, so other arguments are suspicious#2018-03-1515:44seancorfield@borkdude You can extract the list of keys from the spec form itself, and write a predicate that checks those are the only keys. Stu Halloway posted an example on the mailing list a while back I think... Probably in a Gist somewhere...#2018-03-1515:46taylorhttps://github.com/gfredericks/schpec/blob/b2d80cff29861925e7e3407ef3e5de25a90fa7cc/src/com/gfredericks/schpec.clj#L13-L35 hereā€™s a macro to make closed keys specs and hereā€™s Stuā€™s gist which I think was more meant for finding ā€œforgottenā€ key specs https://gist.github.com/stuarthalloway/f4c4297d344651c99827769e1c3d34e9#2018-03-1515:47seancorfield@taylor Ah, yes. I misremembered. Stu's code was to find any keys in a spec that did not have definitions.#2018-03-1515:48borkdudeHey @taylor, great answers on Stackoverflow btw šŸ˜„#2018-03-1515:49seancorfieldBut this code from Stu's Gist is similar to what you'd need (if you don't want to go with shpec): https://gist.github.com/stuarthalloway/f4c4297d344651c99827769e1c3d34e9#file-missing_keys_specs-clj-L31-L33#2018-03-1515:50borkdude@taylor Oh thatā€™s great. I think I will adopt Schpec#2018-03-1516:05borkdudeI want to use schpec in clojurescript, but now I also have to bring in test.check.. ok#2018-03-1516:07borkdudeHmm, I already have thatā€¦ maybe I should upgrade. I get errors about clojure/spec/gen/alpha#2018-03-1516:09gfredericksWhat sort of errors?#2018-03-1516:10borkdude
ā€¢ public/js/app.js
WARNING: Use of undeclared Var com.gfredericks.schpec/limit-keys at line 53 src-cljs/dre/components/widget.cljs <-- this is my namespace
WARNING: No such namespace: clojure.spec.gen.alpha, could not locate clojure/spec/gen/alpha.cljs, clojure/spec/gen/alpha.cljc, or JavaScript source providing "clojure.spec.gen.alpha" at line 53 src-cljs/dre/components/widget.cljs
WARNING: Use of undeclared Var clojure.spec.gen.alpha/fmap at line 53 src-cljs/dre/components/widget.cljs
#2018-03-1516:11borkdudemy ns:
(ns dre.components.widget
  (:require
   [clojure.spec.alpha :as s]
   #_[com.gfredericks.schpec :refer-macros [excl-keys]]
   [dre.page-util :refer [open value-of in-iframe?]]
   [dre.react-util :refer [Collapse DropdownButton MenuItem]]
   [reagent.core :as r]
   [taoensso.timbre :refer-macros [info warn debug]])
  (:require-macros
   [com.gfredericks.schpec :refer [excl-keys]]))
#2018-03-1516:11borkdudeI first tried the one with #_ but then I get:
No such namespace: com.gfredericks.schpec, could not locate com/gfredericks/schpec.cljs, com/gfredericks/schpec.cljc, or JavaScript source providing ā€œcom.gfredericks.schpecā€ in file src-cljs/dre/components/widget.cljs
#2018-03-1516:19gfredericksDoes clojure.spec.gen.alpha exist in cljs?#2018-03-1516:19borkdudehm, it appears not no#2018-03-1516:20gfredericksInteresting.#2018-03-1516:20borkdudeah, cljs.spec.gen.alpha does exist#2018-03-1516:21borkdudehave you tried this in clojurescript?#2018-03-1516:23borkdudeprobably limit-keys should be in a cljc file?#2018-03-1516:23borkdudeand some other adaptations#2018-03-1516:24Alex Miller (Clojure team)cljs should rewrite the clojure to cljs one automatically, but you would need it in a cljc file#2018-03-1516:24Alex Miller (Clojure team)iirc#2018-03-1516:25borkdude@gfredericks you could also put everything in a .cljc file with the help of macrovichā€¦ just wrap every macro in macros/deftime#2018-03-1516:26gfredericks@borkdude I have not tried it in cljs; schpec hasn't gotten a lot of attention. I had the silly idea that I could just make a generic bucket to put things in and then people would put things in it and use it, but I think people prefer to make libraries with more specific purposes#2018-03-1516:26gfredericksbut I'm happy to incorporate any improvements you have, like making sure it works in cljs#2018-03-1516:28borkdudeIā€™ll make an attempt#2018-03-1516:30mgrbytehi, I'm wanting to make a model in spec of my data where the values for 2 keys in my map are dependant on another key's value. I've watched Stu's blog post about using gen/bind and gen/fmap; are there any other examples someone could point me to?#2018-03-1516:51anmonteiro@alexmiller weā€™re very confused about this behavior, wondering if it has come up before:
user> (s/conform #{11 false :foo} false)
:clojure.spec.alpha/invalid
#2018-03-1516:51anmonteirothe behavior seems to be in this condition: https://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/alpha.clj#L875-L877#2018-03-1516:51Alex Miller (Clojure team)you should never use logically false values in literal sets#2018-03-1516:52Alex Miller (Clojure team)(and this applies to more than spec - some is another case)#2018-03-1516:52anmonteiroyeah I guess that really messes things up#2018-03-1516:52Alex Miller (Clojure team)use false? as a predicate#2018-03-1516:52anmonteirothanks#2018-03-1516:52Alex Miller (Clojure team)or nil?#2018-03-1516:52Alex Miller (Clojure team)or s/nilable#2018-03-1516:53Alex Miller (Clojure team)this particular case does not typically come up in general use very often in my experience#2018-03-1516:54anmonteirowhat you said actually makes total sense, it just got me puzzled here for a minute#2018-03-1516:54anmonteiroI guess weā€™ll always need to return logical true for valid? to work#2018-03-1516:57Alex Miller (Clojure team)yes#2018-03-1517:03borkdude@gfredericks This is how far I got, but I still get errors: https://github.com/borkdude/schpec/blob/master/src/com/gfredericks/schpec.cljc#2018-03-1517:03borkdudeThe errors being:
WARNING: No such namespace: clojure.spec.gen.alpha, could not locate clojure/spec/gen/alpha.cljs, clojure/spec/gen/alpha.cljc, or JavaScript source providing "clojure.spec.gen.alpha" at line 54 src-cljs/dre/components/widget.cljs
WARNING: Use of undeclared Var clojure.spec.gen.alpha/fmap at line 54 src-cljs/dre/components/widget.cljs
when I call the macro excl-keys
#2018-03-1517:05borkdudeIā€™ll try to change the require into:
(:require [clojure.spec.alpha :as s]
            [clojure.spec.gen.alpha :as sg]
            [clojure.set :as set]
            #?(:clj [net.cgrand.macrovich :as macros]))
which I think should ā€œjust workā€ ā„¢ in cljs as well
#2018-03-1517:06borkdudehmm same errors#2018-03-1517:08borkdudeI give up for nowā€¦ short on time#2018-03-1517:40gfredericksMy apologies. cljc and macros can be a pretty gnarly combination#2018-03-1518:28borkdudeNo problem man. Iā€™m sure it could work with a small change.#2018-03-1518:28borkdudeI also ran into this issue with cljc, macros and clojurescript: https://dev.clojure.org/jira/browse/CLJS-2636#2018-03-1520:26pabloreI have a spec like this: (s/def (s/map-of ::id ::person) where ::id must be a property of ::person. Could anyone help me making the spec with s/with-gen so I can generate this map for testing?#2018-03-1520:29guy(s/def :entity.person/id string?) (s/def :entity/person (s/keys :req [:entity.person.id])#2018-03-1520:29guylike that do you mean?#2018-03-1520:29guywhat do you mean by property of person?#2018-03-1520:30pabloreyes#2018-03-1520:30pabloreperson is defined in terms of ::id#2018-03-1520:31pabloreso when I generate a map of ids and persons, the map of id-persons must have sense#2018-03-1520:41pabloreIndexed collections are structures I've come to make a lot of times doing clojure. Ie, I rather do (get-in state [:people id :name]) than have to find the vector index every time.#2018-03-1520:34bbrinck@seancorfield @borkdude keep in mind that any attempt to analyze specs wonā€™t work for multi-specs in CLJS, at least AFAICT#2018-03-1520:35bbrinck(which limits our ability to do things like look for undefined specs, or to automatically add checkers for missing values)#2018-03-1520:42Alex Miller (Clojure team)@pablore I donā€™t have time to give you a full answer, but I would start with a generator that creates a collection of ::person (which will have ::idā€™s in it). then gen/fmap over that generator to construct the map from the ids in the generated person entities#2018-03-1520:43Alex Miller (Clojure team)something like (gen/fmap (fn [ps] (zipmap (map ::id ps) ps)) #(s/gen (s/coll-of ::person)))#2018-03-1520:59pabloreThis almost worked! Just has to replace ::id with :id, because (s/keys) generates with keys with single colons#2018-03-1521:01pablore
(defn gen-indexed
  [spec idx]
  (gen/fmap (fn [xs] (zipmap (map idx xs) xs))
            (s/gen spec)))
#2018-03-1521:01pablorethis would be a more generic version#2018-03-1521:59dottedmagHow do I write a spec for [[1]]?#2018-03-1522:00dottedmagObviously my data is more complicated than that ā€” it's an number of vectors enclosed one in other. I'm trying to use spec for validation, so it's an external data type, not something I can change.#2018-03-1522:00Alex Miller (Clojure team)#{[[1]]}#2018-03-1522:00Alex Miller (Clojure team):)#2018-03-1522:01Alex Miller (Clojure team)seriously though, do you know how many levels of vectors?#2018-03-1522:01dottedmagTwo levels, it's a parsed CSV.#2018-03-1522:02dottedmagNot a "nice" uniform CSV, a complicated one. It's got a multiline header and a multiline footer, and empty strings delimiting data sections.#2018-03-1522:03Alex Miller (Clojure team)well, spec the inner levels, then spec the outer levels out of those :)#2018-03-1522:04Alex Miller (Clojure team)(s/def ::line (s/coll-of string? :kind vector?))#2018-03-1522:04dottedmagThat's what I tried. s/cat on outer level, then s/cat on inner level, and first predicate in inner level tried to match the whole line, not the first element of an array.#2018-03-1522:06dottedmagLet me shorten it a bit#2018-03-1522:09dottedmag
(s/def ::header-fields (partial = ["Account Number"]))
(s/def ::account-number (s/and string?
                               (partial re-matches #"^[0-9]{16}$")))
(s/def ::header-info (s/cat ::account-number ::account-number))

(s/def ::delimiter (partial = [""]))

(s/def ::header (s/cat ::header-fields ::header-fields
                       ::header-info ::header-info))

(s/def ::statement (s/cat ::header ::header
                          ::delimiter ::delimiter))
#2018-03-1522:10dottedmagThis ought to match [["Account Number"] ["1234123412341234"] [""]], but apparently the specs for sequences are flattened.#2018-03-1522:10dottedmagBecause
(s/explain ::statement data)
In: [1] val: ["1234123412341234"] fails spec: :bacc.foo/account-number at: [:bacc.foo/header :bacc.foo/header-info :bacc.foo/account-number] predicate: string?
#2018-03-1522:13dottedmagAh, if I add (s/and (s/coll-of string? :kind vector) ...) to the ::header-info then in succeeds.#2018-03-1522:13dottedmag@alexmiller Thanks#2018-03-1605:14ikitommiToyed more with spec coercion, with idea that coercion functions could be defined in spec meta-data so that the specs could remain self-contained . Currently requires spec-tools to work.
(require '[spec-tools.core :as st])
(require '[spec-tools.conform :as conform])

(def spec
  (st/spec
    {:spec string?
     :description "a string spec"
     ::conform/json #(str %2 "-json")
     ::conform/string #(str %2 "-string")}))

(st/conform spec "kikka")
; "kikka"

(st/conform spec "kikka" st/json-conforming)
; "kikka-json"

(st/conform spec "kikka" st/string-conforming)
; "kikka-string"
#2018-03-1618:42jjfineAnyone know why my function doesnt-work throws a java.lang.AssertionError:
(s/def ::bar (fn [x] true))
(defn return-true [_] true)

(defn doesnt-work [something & bar]
  {:pre [(s/assert ::bar bar)]}
  true)

(defn works [something & bar]
  {:pre [(return-true bar)]}
  true)

(deftest failing-test
  (is (works 1))
  (is (thrown? AssertionError (doesnt-work 1))))
This didn't seems spec-specific at first but this test shows I'm probably misunderstanding something about s/def
#2018-03-1618:47taylor@jjfine I think the problem is assert is returning its input if itā€™s valid, and bar is nil, and :pre is interpreting nil as an assertion failure#2018-03-1618:47taylortry {:pre [(s/valid? ::bar bar)]}#2018-03-1618:47jjfineoof#2018-03-1618:48jjfinethanks#2018-03-1718:36lgesslerhi all, I have an xml schema describing a data format I want to translate into a clojure spec. i should be able to translate it by hand, but i was wondering if anyone knew of an easier way? the closest thing I found takes the XSD and then turns it into some kind of clojure schema, but it's not a spec and it didn't seem to produce the right output for my XSD https://github.com/kolov/xelery#2018-03-1914:39mgrbyteI have 3 specs describing a possible "identity" for an entity. I then have an "identity" spec along the lines of (s/def ::identity (s/or :domain/id1 ::id1 :domain/id2 ::id2 :domain/id3 ::id3). one of the specs can't use the built-in generator, how would I go about providing a custom generator that works for s/or? I've managed to write some custom generators before for maps, s/and specs, but can't get my head around how to start writing one for this s/or case, and google hasn't yielded anything so far. Does anyone have any pointers pls?#2018-03-1914:48mgrbyteI'm guessing I need to figure out how to use a combination of gen/tuple and gen/one-of šŸ¤”#2018-03-1915:04bbrinck@mgrbyte Can you call with-gen inside the or?#2018-03-1915:04bbrinck
(require '[cljs.spec.alpha :as s])
  (require '[clojure.test.check])
  (s/def ::id1 string?)
  (s/def ::id2 pos-int?)
  (s/def ::identity (s/or :id1 ::id1 :id2 ::id2))

  (map first (s/exercise ::identity)) ;; => (1 2 "" 2 "" 2 "F91Q6p" "o6" "oet" "DE7")

  (s/def ::identity (s/or :id1 (s/with-gen ::id1 (fn [] (s/gen #{"foo" "bar"})))
                          :id2 ::id2))

  (map first (s/exercise ::identity)) ;; => (1 "bar" 2 2 2 15 31 "bar" "bar" 75)
#2018-03-1915:04bbrinckWould that work for your case?#2018-03-1916:16mgrbyte@bbrinck Trying to keep generators in tests only, since I'm using minter.strgen/test.check stuff to help make the gen for "::id1" . I've managed to get somewhere near what I need with (gen/one-of [gen1 gen2 gen3]), but as mentioned in the spec guide, need it's a good idea to generate s/conform like example for any s/or like specs: "For r specs that have a conformed value different than the original value (anything using s/or, s/cat, s/alt, etc) it can be useful to see a set of generated samples plus the result of conforming that sample data."#2018-03-1916:17mgrbyteSo I'll need to interleave the specs and generators. but think that will do it#2018-03-1916:31bbrinck@mgrbyte I havenā€™t tried it out, but I believe you can use (require '[clojure.spec.gen.alpha :as gen]) and that will be OK in dev and test#2018-03-1916:32bbrinck(it lazy loads test.check when generator is called, not defined, but presumably you wonā€™t be invoking generators outside of tests)#2018-03-1916:33bbrinckMight simplify your implementation, although I donā€™t know if you want to keep generators in test for dependency reasons, or for other reasons like attaching different generators at different tiems#2018-03-1916:37ikitommiHi. Did someone make a stab at spec bijections? Any lessons learned? Bumped into a actual need: need to uncoerce path-parameters in reverse-routing into string based either on the value type or on the defined spec.#2018-03-1916:38ikitommiWith spec, I guess we could put both the encode & decode functions into spec metadata, something like:
(st/spec
  {:spec keyword?
   :description "a bijecting keyword"
   ::encode/string #(name %2)
   ::decode/string #(keyword %2)})
#2018-03-1916:57gfredericksst/spec? does that exist?#2018-03-1916:58gfredericksI've been talking about bijections a lot; I was thinking of setting up a generic library for composing bijections, that's not spec-specific#2018-03-1916:58gfredericksif you have ideas about how to tie them with spec, that would be interesting#2018-03-1916:59ikitommist/spec is from spec-tools, until spec meta-data appears.#2018-03-1916:59gfredericksthat's a 3rd-party library?#2018-03-1916:59ikitommihttps://github.com/metosin/spec-tools#2018-03-1917:00ikitommidoes already the encode/coercion, could do the other way too.#2018-03-1917:01gfredericksmy starting point is assuming that the most common use for bijections would be where you have some internal canonical representation of something, and you want to specify how to transform it to/from other "contexts", which could be things like json, jdbc, cvs, etc.#2018-03-1917:01ikitommiGeneric library/foundation would be nice, as this is needed with Schema too.#2018-03-1917:02gfredericksso when deciding how to integrate that with spec, one question that would come up would be whether it's worth automating the process of transforming the specs themselves so that you get context-specific versons of the specs, or merely to automate the process of writing the functions that take values in each direction#2018-03-1917:02ikitommiyes, the name of the (`string` in ::encode/string) is the context name.#2018-03-1917:03gfredericksthe downside of actually generating specs for each context is that it's more complicated and probably more work for the programmer the upside is I think you can get more features, like validating closer to the edge, and giving validation errors based on that representation rather than the internal one#2018-03-1917:04ikitommiI tried the context-versions of spec, but coudnā€™t get it working.#2018-03-1917:04ikitommifailed with the qualified keys with s/keys.#2018-03-1917:05gfrederickss/keys is a big barrier; especially if you want to convert the keys to strings like in a json context#2018-03-1917:05ikitommi
(s/def ::id keyword?)
(s/keys :req [::id])
#2018-03-1917:06ikitommiif you have that, you canā€™t have a different ::id as the full key names are exposed outwards.#2018-03-1917:07ikitommiwhy would the validation be closer to the edge?#2018-03-1917:07gfredericksyeah you'd need a new namespace for each context probably#2018-03-1917:08ikitommiā€¦ or a different registry.#2018-03-1917:08gfrederickse.g., if I write a public JSON API and biject my internal spec to a JSON version of that spec, I can theoretically validate the user input using the json spec, before trying to transform/coerce the data to the internal representation#2018-03-1917:08gfredericksyou can also then provide validation errors to the user in a way that's specific to the json representation, rather than the less helpful internal representation#2018-03-1917:09gfrederickshere's what I came up with a few months ago; I did a proof of concept of integrating it with some code at work https://gist.github.com/gfredericks/358c39478f104281a6364f446f5b3c6b#2018-03-1917:09ikitommioh, true that. Would mean multiple walks on the same model, but in most cases, the perf impact would not matter.#2018-03-1917:11ikitomminice draft!#2018-03-1917:11ikitommiare there bijections in other languages?#2018-03-1917:12ikitommicould read some theory/reasoning to understand this.#2018-03-1917:28gfredericksI've not seen it, other than I've speculated you could typecheck them in a dependently typed language#2018-03-1917:28gfredericksthere is of course the relevant set theory stuff: https://en.wikipedia.org/wiki/Bijection,_injection_and_surjection#2018-03-1917:30gfredericksI've also wondered about separating the bijective parts from surjective parts (the times when you want to drop information, e.g., the order of keys in a query string)#2018-03-1917:30gfredericksif the surjections are formalized, you can use them to create generators of all the variant representations of something#2018-03-1918:22stathissiderissuper-interesting stuff, I think I should do some reading on this#2018-03-1918:30ikitommithanks for the pointer! In my test, the transforming functions take two arguments: the spec and the value. This allows the transformation to extract information from the spec to do the transformation.#2018-03-1918:31ikitommifor example, stripping out non-defined keys from keys-spec needs the set of keys defined:
(defn strip-extra-keys [{:keys [keys]} x]
  (if (and keys (map? x))
    (select-keys x keys)
    x))
#2018-03-1918:42gfredericksI'd always approached that by building the function at compile-time#2018-03-1918:42gfredericksI should also point out this old effort in case it hasn't been seen: https://github.com/gfredericks/schema-bijections#2018-03-1918:43gfredericksI used that for real code at my old job, and it worked pretty good but made the code difficult to read#2018-03-1919:09gfredericksValidating before converting isn't any extra walks compared to validating after converting I don't think#2018-03-1916:59Drew VerleeKnowing full well this doesnā€™t make sense. How would you write a spec for a collection of maps where each maps keys were 1) unique 2) represents a value themselves e.g [data, data, data, ā€¦] where data is something like {5 6}#2018-03-1917:04Alex Miller (Clojure team)well you would start with (s/coll-of (s/map-of int? int?))#2018-03-1917:05Alex Miller (Clojure team)each individual maps keys are (by definition) unique, but I assume you mean all of the keys over all the maps are unique#2018-03-1917:05Alex Miller (Clojure team)not sure what #2 means#2018-03-1917:06Alex Miller (Clojure team)for checking the uniqueness constraint, s/and another predicate that checks whatever property you need - something like #(apply distinct? (mapcat keys %))#2018-03-1917:22Drew Verlee> each individual maps keys are (by definition) unique, but I assume you mean all of the keys over all the maps are unique correct#2018-03-1918:38ghadiyou can't have duplicate keys in a map @drewverlee#2018-03-1918:39ghadioh I didn't see the collection of maps requirement. yeah, s/and is your friend here#2018-03-1918:52roklenarcicI'm trying to write my own Spec protocol implementation. Is there somewhere where Spec and Specize protocols' contracts are explained?#2018-03-1919:05Alex Miller (Clojure team)no, because they are likely to change#2018-03-1919:08misha@mgrbyte you can override generator just for ::id1 during exercise time
(s/def ::id1 string?)
(s/def ::id2 int?)
(s/def ::id (s/or :id1 ::id1 :id2 ::id2))
(map first (s/exercise ::id 10 {::id1 #(s/gen #{"foo"})}))
;;=>  (0 -1 1 "foo" 0 -7 "foo" -1 "foo" 0)
#2018-03-1919:09borkdudeI have:
(s/def ::key
  (s/and keyword?
         #(= "widget"
             (namespace %))))

(s/fdef widget
        :args
        (s/cat :opts
               (s/keys
                :req-un [::title
                         ::content]
                :opt-un [::key
                         ::icon
                         ::widget-class
                         ::settings
                         ::collapsed?
                         ::dropdowns
                         ::controls
                         ::preview?
                         ::tabs
                         ::collapse-opts
                         ::help])))
but ::key does not seem to get checked when itā€™s provided
#2018-03-1919:12taylorthis is working for me locally, w/a dummy, instrumented widget function#2018-03-1919:13taylorIn: [0 :key] val: :foo fails spec: : at: [:args :opts :key] predicate: (= "widget" (namespace %))#2018-03-1919:14borkdudehmm yeahā€¦ for me too when I isolate this#2018-03-1919:16borkdudeworks now!#2018-03-1919:16borkdudeI guess some reloading glitch#2018-03-1921:18cch1Is there an idiomatic way of expressing a spec for ā€œanythingā€ other than (constantly true)?#2018-03-1921:19cch1Iā€™m trying to define a spec for a value that must ultimately be usable as a key in an associative structure.#2018-03-1921:19cch1In Clojure, I think that is pretty much everything.#2018-03-1921:20bronsaany? is what you want#2018-03-1923:18camachomI'm trying to pass a vector to s/keys, but it keeps failing. Works great if I just write it out though. Something like this:
(def required
  [:foo/city :foo/country :foo/street])
(spec/def :foo/form (spec/keys :req required
                               :opt []))
#2018-03-1923:18camachomany ideas as to what i'm doing wrong?#2018-03-1923:28alexisvincentHi guys, got a bit of a modelling problem. How do you build up specs with context dependent information. E.g. Say youā€™re building a space for card details, say a map of the keys :card/number :card/expiry :card/name :card/cvv. Now we want this to be a nice composable spec called :card/details. But in some contexts :card/cvv is required and in others it isnt. So The :card/details spec is really a partial spec with requiredness of :card/cvv dependent of context. We might then say make two specā€™s :card/details and :card/full-details. But this isnā€™t really scalable. It means whenever we have branching up the composition tree we need to rename things for all cases, giving an exponential explosion. Is there any way to build ā€˜partialā€™ specs where specs higher up the composition tree provide more or overriding data for specs they depend on. Similar to dependent types.#2018-03-2000:11bbrinckThis feature still requires a lot of work, but soon(ish), Expound will be able to provide examples of valid clojure code https://gist.github.com/bhb/f637ef589ef3ac3d2ca5a883fafc2c12#2018-03-2000:12bbrinckHowever, this adds a dependency on test.check to use expound at dev time. Would you prefer to have this be a hard dependency OR would you prefer expound to disable this feature if you donā€™t have test.check available?#2018-03-2000:24seancorfield@mcama200 spec/keys is a macro so it takes literal code forms, not variables.#2018-03-2000:25seancorfieldWhat you can do instead is to use the literal vector in s/keys and then call s/form (I think?) on the spec itself and walk that to get the list of required keys back out of a spec definition.#2018-03-2000:35camachomawesome, thanks for the help!#2018-03-2000:29seancorfield@alexisvincent The short answer is: specs are currently pretty static so you have to have multiple specs -- or specify that :card/cvv is :opt in the base spec and then wrap it in another spec that uses s/and and a predicate that "requires" that key be present.#2018-03-2000:30seancorfield
(s/def :card/full-details (s/and :card/details #(contains? % :card/cvv)))
something like that (untested)
#2018-03-2009:12borkdudeI guess spec could eat its own dog food?
boot.user=> (s/fdef foo (s/cat :a1 string?))

     java.lang.IllegalArgumentException: No value supplied for key: (s/cat :a1 string?)
clojure.lang.Compiler$CompilerException: java.lang.IllegalArgumentException: No value supplied for key: (s/cat :a1 string?), compiling:(null:1:1)
#2018-03-2013:12Alex Miller (Clojure team)Actually it canā€™t, unless you enjoy infinite recursion#2018-03-2010:47mpenetbeen there done that: my personal favorite is (s/def foo any?) or (s/fdef ::foo :args (s/cat))#2018-03-2010:48mpenetboth silently fail, which makes it even better#2018-03-2010:57alexisvincent@seancorfield Thanks for the answer, the s/and is neat, and will help reduce branching factor, but I donā€™t think itā€™s a long term behaviour. I can use functions that return specs, but loose out on specā€™s repository. Do you know if dynamism will be added to named specs in the future, or if it even needs to be?#2018-03-2011:06borkdudeI have the current spec (simplified):
(s/fdef widget
        :args
        (s/cat :opts
               (s/keys
                :req-un [::title
                         ::content]
                :opt-un [
                         ::init-collapsed?
                         ::fixed?])))
How do I say init-collapsed? and fixed? are mutually exclusive?
#2018-03-2011:07mpenetbut they can be both absent as well?#2018-03-2011:07borkdudeyes#2018-03-2011:08borkdudeif you say the widget must be fixed, init-collapsed? can not be true, because the widget wonā€™t be visible at all that way#2018-03-2011:09borkdudeI could just emit a warning outside of spec, but maybe thereā€™s a nice solution#2018-03-2011:09borkdudeI also donā€™t want to introduce nesting in the arguments#2018-03-2011:09borkdudeI could maybe use s/conformer?#2018-03-2011:10mpenetI guess you need a separate pred in s/and since opt-un will not accept (or ...) etc#2018-03-2011:11borkdudeyes,
(fn [m]
  (not (and (:init-collapsed? m)
        (:fixed? m))))
works with s/and thanks
#2018-03-2011:14mpenetI imagine a multispec could do it too, but that's prolly too much complexity for such as simple check#2018-03-2014:44acronjava.util.concurrent.ExecutionException: clojure.lang.ExceptionInfo: Couldn't satisfy such-that predicate after 100 tries. {}#2018-03-2014:44acronMost depressing error ever.#2018-03-2014:50Alex Miller (Clojure team)agreed :)#2018-03-2014:51Alex Miller (Clojure team)unfortunately, the current linkage between spec and test.check prevents good error reporting on where this is happening. there are plans to make this better, but it requires changes in both projects.#2018-03-2015:26gfredericks@alexmiller I'm curious how what you just described relates to an earlier comment that spec can't enscricten its version requirements of test.check#2018-03-2015:26Alex Miller (Clojure team)enscricten = ?#2018-03-2015:28danielnealmake more scrict#2018-03-2015:29Alex Miller (Clojure team)and scrict = ?#2018-03-2015:33danielnealopposite of pecmissive#2018-03-2017:59misha@alexisvincent define all your fieldsā€™ specs, and then have several s/keys specs for different combinations of those fields. Then you might want to have the card spec, which is s/or around those s/keys#2018-03-2018:05mishaRemember, you can have keys with different namespaces in the same map/object, so donā€™t paint yourself into corner with unified namespace ā€œ:cardā€ when some attributes might be better named, like :amex.card/number, because itā€™ll have different spec and generator.#2018-03-2100:45gfredericks@alexmiller sorry, meaning that spec would, e.g., require test.check >= 0.10.0#2018-03-2100:45gfredericksyou made a comment a few weeks back that implied that wouldn't be acceptable#2018-03-2100:45alexisvincent@misha šŸ™‚ Thanks. Im just concerned about the name explosion when composing up the tree#2018-03-2100:53Alex Miller (Clojure team)@gfredericks I wouldnā€™t make a range version but we could certainly bump the requirement #2018-03-2100:53gfredericksthe difference between those two things is whether a max version is specified?#2018-03-2100:53Alex Miller (Clojure team)I just donā€™t like version ranges#2018-03-2100:54seancorfieldNo one likes version ranges šŸ™‚#2018-03-2100:54Alex Miller (Clojure team)The only people that like version ranges are people that havenā€™t used them :)#2018-03-2100:58gfredericks:+1: thanks#2018-03-2101:00kahunamooreI'm looking to do some visualization of specs and by default would think using the reader would be the most straight forward way to consume/read the specs. Is there a better way? A library perhaps that produces a higher level representation? Thoughts?#2018-03-2102:13andy.fingerhut@kahunamoore I have not used this tool, and not sure if it is what you are looking for, but it sounds at least related: https://github.com/jebberjeb/specviz#2018-03-2102:14andy.fingerhutLearned about it from Stuart Halloway's talk "REPL driven development": https://github.com/matthiasn/talk-transcripts/blob/master/Halloway_Stuart/REPLDrivenDevelopment.md#2018-03-2103:15seancorfieldOh, I didn't know there was a transcript of that! Cool!#2018-03-2111:24tayloryesterday I helped a coworker spec some financial calc functions, then we immediately found some bugs by checking them šŸ‘#2018-03-2111:26tayloraside: will the default generator for number? predicate generate all/most numeric types (like ratios)?#2018-03-2111:29taylordoesnā€™t seem like it generates ratios#2018-03-2111:49borkdudeWould it make sense if Spec could aid in producing human readable errors? https://gist.github.com/borkdude/89799a764a89af5c0be1a5745be58b4d#2018-03-2111:57ikitommi@borkdude related: https://dev.clojure.org/jira/browse/CLJ-2115#2018-03-2114:00hkjelsthe above obviously does not work#2018-03-2114:00hkjelsI could use this and digg into the meta, but Iā€™d prefer not to#2018-03-2114:05Alex Miller (Clojure team)what problem are you trying to solve?#2018-03-2114:14hkjelsI output form-controllers based on the type of a specification. So for simple spec definitions like (spec/def ::placeholder string?) I can output a textfield because I know itā€™s a string with (type the-def), but for definitions that use with-gen; type returns nil#2018-03-2114:24bbrinck@hkjels Can you talk more about how you determine the type in the simple case? Are you inspecting the spec itself, or an example generated value?#2018-03-2114:26bbrinckAlso, that generator above doesnā€™t generate valid values: (fn? nil) is false#2018-03-2114:27bbrinckIf you want a ā€œmaybe-fnā€, perhaps s/nilable would help?#2018-03-2114:27Alex Miller (Clojure team)heā€™s returning (constantly nil), which is a function#2018-03-2114:27bbrinckah#2018-03-2114:27bbrinckdoh#2018-03-2114:28Alex Miller (Clojure team)I feel like you should still be driving from the spec, not the generator#2018-03-2114:29hkjelsyeah, not sure how though#2018-03-2114:29Alex Miller (Clojure team)(you should also be using ifn?, not fn? btw)#2018-03-2114:29bbrinckAgreed, @hkjels if I were trying to generate forms based on the spec, Iā€™d look at introspecting the specs using s/form. Obviously itā€™s be hard with all possible specs#2018-03-2114:29bbrinckbut if you have a subset of forms in your app, you could reasonably inspect them and generate the cases that interest you#2018-03-2114:30Alex Miller (Clojure team)well in this case, s/form doesnā€™t currently work on predicates#2018-03-2114:30Alex Miller (Clojure team)(pending patch to fix that)#2018-03-2114:30hkjelsahh, nice#2018-03-2114:30bbrinckBut s/form returns something sensible for ::placeholder, yes?#2018-03-2114:30bbrinckAnd ::maybe-fn#2018-03-2114:31bbrinckYou could parse those (probably using specs!!) and then build cases based off that#2018-03-2114:31Alex Miller (Clojure team)you could use spec specs for that :) (CLJ-2112)#2018-03-2114:31bbrinckftw#2018-03-2114:32bbrinckšŸ™‚#2018-03-2114:34hkjelsšŸ™‚#2018-03-2114:34hkjelshere you see what Iā€™m trying to achieve for all kinds of elements#2018-03-2114:35hkjelsit currently works for buttons, since thereā€™s no atoms etc in use#2018-03-2114:36hkjelsOhh, shoot. I got to run#2018-03-2117:30kahunamoore@andy.fingerhut Thanks! I remember reading/hearing about it but Google failed me - too many spec tutorials, docs and other misses.#2018-03-2119:37hkjelsohh, that nonconforming is just some cruft left there from trial and error#2018-03-2123:43gfredericks@hkjels would (spec/* (s/alt string? ::hiccup)) work?#2018-03-2123:52ghadiNeed to label the branches @gfredericks ^#2018-03-2123:52gfredericksyeah that sounds truthful#2018-03-2207:21misha@hkjels just (s/def ::hiccup (s/tuple #{:div :span} string?))?#2018-03-2207:42andy.fingerhutSo Stuart Halloway gave a talk on Clojure spec at Strange Loop 2016 that I am transcribing now (mostly done), where he said this: 'So one of the things that is fun about developing with spec is: as soon as you have made a spec, and it can even be a very rudimentary one, you can then say "give me example data" or "give me example functions that do this".'#2018-03-2207:42andy.fingerhutIf you want to see it in context of the full text, you can search for that quote here: https://github.com/jafingerhut/talk-transcripts/blob/add-halloway-agility-robustness-clojure-spec-transcript/Halloway_Stuart/AgilityRobustnessClojureSpec.md#2018-03-2207:43andy.fingerhutThe part that I am wondering how to do is the part where he says "give me example functions that do this". Does anyone know if that is possible, and if so, has examples of doing that?#2018-03-2212:44Alex Miller (Clojure team)He was probably referring to the stub capabilities of instrument#2018-03-2211:44gfredericksfspecs can be generated I think#2018-03-2211:48taylor
(def my-fn (-> (s/fspec :args (s/cat :x int?) :ret string?)
               (s/gen)
               (gen/sample 1)
               (first)))
(my-fn 1)
=> "8VRbFJ59e2K2Ic79F8CX8HB1"
(my-fn 1)
=> "WSKN70JT5kZG"
#2018-03-2211:49tayloris it just making a function that ignores its inputs and using the :ret spec to generate return values?#2018-03-2211:50taylor> fspecs can generate functions that validate the arguments and fabricate a return value compliant with the :ret spec, ignoring the :fn spec if present.#2018-03-2211:51guyWell you could try doing (my-fn "string") and see if it complains#2018-03-2211:51guyBecause you are giving it an int right?#2018-03-2211:51taylorit does validate the inputs, I asked before I read the docstring šŸ™‚#2018-03-2211:51guyah#2018-03-2211:52guyšŸ‘#2018-03-2211:59gfredericksyes the return is unrelated to the input I think#2018-03-2211:59gfredericksI made a ticket many decades ago to discuss whether it should at least be a pure function#2018-03-2211:59gfredericksit's also currently independent of the test.check seed I think#2018-03-2212:00guyYou can use orchestra to make it related with the :fn i think?#2018-03-2212:03guy#2018-03-2212:21taylorI think that only applies to instrumentation#2018-03-2212:55danielcomptonI think this is a really dumb question, but how can I detect which branches were taken/captured in a s/cat?#2018-03-2212:56gfredericksdoesn't the conformed value tell you that?#2018-03-2212:57danielcomptonyeah I can see it in the data, I was just wondering if there was a function you could call to get back which parts matched
#2018-03-2212:58danielcomptonthe context is that I'm trying to parse defn forms and reassemble them with tracing wrapped around parts of the function#2018-03-2212:58danielcomptonand the function specs have quite nested data that is a little bit annoying to pull out#2018-03-2213:23danielcomptonSo I end up with this disgusting code:
(defmacro fn-traced
  [& definition]
  (let [conformed (s/conform ::ms/fn-args definition)
        name      (:name conformed)
        bs        (:bs conformed)
        arity-1?  (= (nth bs 0) :arity-1)
        args+body (nth bs 1)]
    (if arity-1?
      (if (= :body (nth (:body args+body) 0))
        (if name
          `(fn ~name ~(:args (:args args+body))
             
#2018-03-2213:24danielcomptonwas wondering if there's a pattern for doing this in a cleaner way#2018-03-2213:26mpenetsmall critique, seems like the if name bits could be done inside the (fn ... part to avoid repeating the whole fn expression#2018-03-2213:26mpenetboth times#2018-03-2213:26mpenetor I might be missing something#2018-03-2213:27danielcomptonbut if there is no name then I'll end up with (fn nil [] ...)#2018-03-2213:27mpenetoh right, I didn't pay attention#2018-03-2213:28danielcomptonAm I possibly approaching this whole thing in the wrong direction?#2018-03-2213:28danielcomptonFeels pretty dirty#2018-03-2213:44Andreas Liljeqvist@alexmiller anything missing for https://dev.clojure.org/jira/browse/CLJ-2311 ?#2018-03-2214:11danielcomptonI'll be able to clean this up a lot, but here's how it turned out:
(defmacro fn-traced
  [& definition]
  (let [conformed (s/conform ::ms/fn-args definition)
        name      (:name conformed)
        bs        (:bs conformed)
        arity-1?  (= (nth bs 0) :arity-1)
        args+body (nth bs 1)]
    (if arity-1?
      (if (= :body (nth (:body args+body) 0))
        (if name
          `(fn ~name ~(:args (:args args+body))
             
#2018-03-2214:16danielcompton@mpenet here's how to not repeat the name section twice:
`(fn 
#2018-03-2214:16danielcomptonWrap a single element in a collection and then unsplice it#2018-03-2214:16mpenetright#2018-03-2214:16danielcomptonšŸ˜ˆ#2018-03-2214:35Andreas Liljeqvistwhat would be the most idiomatic way of expressing xor for two keys in a map?#2018-03-2214:38Alex Miller (Clojure team)it is possible to actually define xor and use it in :req, but I do not guarantee that that will work forever#2018-03-2214:38Alex Miller (Clojure team)separately you can s/and around the map to enforce a predicate#2018-03-2214:50danielcomptonThat's a bit better:
(defn fn-body [args+body]
  (if (= :body (nth (:body args+body) 0))
    `(~(or (:args (:args args+body)) [])
       
#2018-03-2319:20bbrinckApologies if this doesnā€™t work in your case, but would this approach work? Weā€™ve used it successfully to insert timers around function bodies http://blog.klipse.tech/clojure/2016/10/10/defn-args.html#2018-03-2319:23bbrinckThe trick is to modify the conformed value and then call unform to go back to valid clojure code#2018-03-2319:42bbrinckEven if I deleted the docstrings, examples, and workaround for the spec bug, I think my code would longer than your solution, but hereā€™s an example of the approach described in that link https://gist.github.com/bhb/128bf97619e83541a8adda7094bc370d#2018-03-2320:56danielcomptonThanks, thatā€™s a very clever way to do it#2018-03-2215:56borkdudeBy mistake, but it surprised me:
(s/def :dropdowns/options (s/coll-of :dropdowns/options))
(s/valid? :dropdowns/options []) ;; true, ok, I can get that
(s/valid? :dropdowns/options [[]]) ;; true, uuh...
#2018-03-2215:58guyšŸ¤”#2018-03-2215:58Alex Miller (Clojure team)coll-of doesnā€™t limit cardinality by default#2018-03-2215:58Alex Miller (Clojure team)0 size is valid#2018-03-2215:58borkdudeyes, I get that#2018-03-2215:58Alex Miller (Clojure team)Use :count, :min-count etc if needed#2018-03-2215:59guywhat is your spec for :dropdown/option ? suprised it allowed [[]]#2018-03-2215:59borkdudeno, you see, itā€™s recursive#2018-03-2215:59guyoh right#2018-03-2215:59guySorry missed that#2018-03-2216:00borkdudeI missed that too, but then I was surprised about this behavior, because spec will allow arbitrary nesting of empty colls this way#2018-03-2216:00borkdudeI donā€™t know if set theory is happy with this#2018-03-2216:01borkdudethe way to justify: an empty coll is a valid :foo, a collection of empy collections is a collection of valid :foo, hence an arbitrarily nested empty coll is a valid :foo#2018-03-2216:02borkdudeso actually this is a spec for arbitrarily nested empty colls#2018-03-2216:05mpenetif you re thinking about monoid instances, it's perfectly valid imho#2018-03-2216:06borkdudewell if you think about types, a [a] isnā€™t the same as [[a]]#2018-03-2216:06mpenetyeah but the mempty of [a] is []#2018-03-2216:06mpenetbut yes, no type annotation to validate that [] is indeed potentially a coll of foo#2018-03-2216:06mpenetbut other than that it's ok#2018-03-2216:07borkdudeI guess#2018-03-2216:08borkdudeitā€™s an unexpected way to describe all possible sets of empty sets#2018-03-2216:08borkdude(s/valid? ::empty-nested [[] []]) is also true of course#2018-03-2216:08mpenetspec cannot go that far. same problem with (s/def ::first-name string?) (s/def ::last-name string?)#2018-03-2216:09mpenetusing ::first-name instead of ::last-name in an fdef args signature will be silent, in haskell it would scream at you#2018-03-2216:09mpenetsame shape, not necessary same meaning#2018-03-2216:10borkdudeyes, I guess that makes sense. itā€™s enabled things too#2018-03-2216:10mpenetno such thing as newtype or some kind of wrapper for validation#2018-03-2216:11mpenetyes, it's all trade-offs I guess#2018-03-2216:13borkdudeHow would you write this:
(s/def :dropdown/option (s/keys :req-un [:dropdown-option/id
                                         :dropdown-option/label]))
dropdown-option/id or dropdown/option-id . You can only namespace one level
#2018-03-2216:14guy>dropdown-option/id Says to me, thats the id of the dropdown option. >dropdown/option-id Says to me, thats the option-id of the dropdown#2018-03-2216:14mpenetI use a little helper and create a new namespace for it#2018-03-2216:14mpenethttps://github.com/mpenet/spex/blob/master/src/clj/qbits/spex.clj#L7-L10#2018-03-2216:15mpenet(rel-ns 'dropdown.option)#2018-03-2216:15mpenetthen ::dropdown.option/id#2018-03-2216:15mpenetThat's what I found to be the least horrible so far.#2018-03-2216:15borkdudeI could just use the dot anyway right? without creating a namespace?#2018-03-2216:16guyYeah#2018-03-2216:16mpenetyou can use a keyword with a dot yes#2018-03-2216:16borkdudeok, Iā€™ll do that then#2018-03-2221:50roman01laIs it possible that both s/conform and s/valid are failing, but s/explain returns Success!?#2018-03-2221:51roman01laNot doing anything fancy. Clojure 1.9.0#2018-03-2221:53roman01laok, nevermind, forgot to reload ns#2018-03-2223:52Alex Miller (Clojure team)any case of that is a bug#2018-03-2301:22pabloreAre there any differences in using spec in clojure and clojurescript? Theres a cljs.spec.alpha namespace#2018-03-2301:23pabloreCan I define specs in a .cljc file and use them in both clojure and clojurescript?#2018-03-2301:30seancorfieldI believe that ClojureScript is supposed to do an automatic mapping from clojure.* namespaces to cljs.* namespaces so I would expect you could just write .cljc files with a require of clojure.spec.alpha and it should just work...#2018-03-2301:30seancorfield(caveat: I don't do anything with cljs so I might be talking nonsense)#2018-03-2301:34benzapI believe that is correct, I just pull in clojure.spec.alpha, and it works in clj, cljs, and cljc#2018-03-2302:36Alex Miller (Clojure team)thatā€™s the idea#2018-03-2316:29borkdudeIs there a way to not have this conform in the two options but just pass through as it is?
(s/def :dropdown.option/id (s/or :s string?
                                 :k keyword?))
#2018-03-2316:29borkdudeso without the :s or :k#2018-03-2316:29taylorcould you s/and it with a conformer that discards the tag?#2018-03-2316:30borkdudeI am using s/and already I run into this in the conformer šŸ˜›#2018-03-2316:30borkdudebut I can just say (fn [id] (or (keyword? id) (string? id))) of course#2018-03-2316:32taylor
(s/def ::s-or-n
  (s/and (s/or :s string? :n number?)
         (s/conformer second)))
(s/conform ::s-or-n "hey")
=> "hey"
was what I was suggesting but not sure if thatā€™s a terrible idea or not
#2018-03-2316:37Alex Miller (Clojure team)You can wrap s/nonconforming around the or#2018-03-2316:37borkdudeawesome#2018-03-2316:38Alex Miller (Clojure team)Currently thatā€™s undocumented#2018-03-2316:38Alex Miller (Clojure team)I think we will ultimately either doc it or make a nonconforming or#2018-03-2316:38borkdudehmm, not in clojurescript yet probably?#2018-03-2316:38borkdudeoh it is#2018-03-2316:40borkdudeworks great, thanks#2018-03-2611:37borkdudeis it possible to instrument an anonymous function?#2018-03-2611:41mpenetfspec?#2018-03-2611:42borkdudehow? say I have a re-frame event:
(reg-event-fx ::foo
  (fn [...] ...))
#2018-03-2611:43borkdudeI could pull the fn out and name it, but that gets tedious#2018-03-2611:44mpenetI think the approach would be to spec reg-event-fx and have the second arg speced via fspec#2018-03-2611:44mpenetotherwise you can/need to add s/asserts if you want to have it "contained" at the anonymous function level#2018-03-2611:46borkdudeyou canā€™t do that because the function reg-event-fx receives is a different one every time. I want to be specific about the concrete function, not the function argument of reg-event-fx#2018-03-2611:46borkdudeI think for now itā€™s easiest to name it then#2018-03-2612:03borkdudehmm:
cljs.user=> (def param-keys [:query/query
       #_=>                  :query/source
       #_=>                  :vocab/filter
       #_=>                  :dict/guid
       #_=>                  :dict/labels?])
#'cljs.user/param-keys
cljs.user=> (s/def ::params
       #_=>   (s/keys :opt
       #_=>           param-keys))
              ^
param-keys is not ISeqable at line 2
#2018-03-2612:04borkdudeit only expects me to pass in literals? thatā€™s disappointing#2018-03-2612:04mpenetyou need to use eval to pass params-keys, s/keys is a macro#2018-03-2612:04mpenetyeah#2018-03-2612:05mpenetHopefully it something improved in the next spec iteration, it's a common problem#2018-03-2612:06mpenets/merge (or s/and) can sometimes help in these case too#2018-03-2612:07borkdudehow?#2018-03-2612:08mpenetto compose multiple sets of s/keys#2018-03-2612:09mpenethence the "sometimes" it does not always work#2018-03-2612:15borkdudeIs this a valid way of saying I want either :dict/guid or :query/query + these optional keys?
(s/def ::params
  (s/merge (s/or :guid (s/keys :req [:dict/guid])
                 :query (s/keys :req [:query/query]))
           (s/keys :opt
                   [:query/source
                    :vocab/filter
                    :dict/labels?])))
#2018-03-2612:19Alex Miller (Clojure team)
(s/def ::params
  (s/keys
    :req [(or :dict/guid :query/query)] 
    :opt [:query/source :vocab/filter :dict/labels?]))
#2018-03-2612:20borkduderegular or?#2018-03-2612:21borkdudebtw it seems to work#2018-03-2612:22borkdudethanks. is there also a way to get exclusive or?#2018-03-2612:22borkdudeand is this usage of or undocumented?#2018-03-2612:23borkdudeI could also handle this via an extra s/and + predicate#2018-03-2612:25borkdude
(s/def ::params
  (s/and
   (s/keys
    :req [(or :dict/guid :query/query)] 
    :opt [:query/source :vocab/filter :dict/labels?])
   (fn [m]
     (not (and (:dict/guid m)
               (:query/query m))))))
#2018-03-2612:33Alex Miller (Clojure team)if you implement xor as a function and use there, it will work. but I make no guarantees that that will continue to work in the future.#2018-03-2612:41borkdudethis is undocumented right#2018-03-2612:53Alex Miller (Clojure team)yes and not guaranteed to work in the future#2018-03-2614:15borkdudeAre there parts that we can rely on, since spec is still in alpha?#2018-03-2614:17borkdudeDonā€™t mind breakage btw if itā€™s for the better#2018-03-2614:19Alex Miller (Clojure team)generally, itā€™s better to rely on the documented usage than the undocumented usage#2018-03-2614:32borkdudeIā€™m emitting a warning in a s/and conformer function.
(fn [{:keys [opts]}]
           (if-let
               [unexpected
                (seq
                 (apply dissoc opts expected-keys))]
             (do (warn "Unexpected options" unexpected "in widget" (or (:id opts)
                                                                       (:title opts)))
                 false)
             true))
I see this message twice. Is this because spec does something like ā€œif not valid run it again for the explanationā€?
#2018-03-2614:35borkdudeThe error that spec returns itself is a bit unsatisfying here, thatā€™s why I want to print extra info#2018-03-2614:38Alex Miller (Clojure team)there are no guarantees on the number of times a conformer function might be run (in a regex with branches, it may be run many times). Iā€™m not sure why itā€™s run more than once in this particular case, but itā€™s possible that your explanation is correct.#2018-03-2614:38borkdudeok no problem at all, just wanted to know#2018-03-2614:39borkdudemaybe I could write this spec a little better, so spec provides what keys are unexpected though#2018-03-2614:39borkdudebut this goes against the default of spec where extra keys are just allowed#2018-03-2614:42borkdude
(clojure.test/is (empty? [1 2 3]))
FAIL in () (NO_SOURCE_FILE:37)
expected: (empty? [1 2 3])
  actual: (not (empty? [1 2 3]))
#2018-03-2707:31borkdudeI think it doesnā€™t know how to generate a ratom from a predicate.#2018-03-2708:20hkjelsThat springs another question though. How do I express that?#2018-03-2707:33borkdudeMaybe you should try making ::ratom with itā€™s own with-gen so that this with-gen also determines the contents of the ratom.#2018-03-2708:01hkjelsahh, ofcourse. Thank you @borkdude#2018-03-2708:20hkjelsfails with the same message#2018-03-2708:32borkdude@hkjels I made a small example in JVM Clojure which should be straightforward to port to cljs:
(s/def ::ratom
  (s/with-gen #(instance? clojure.lang.IAtom %)
    #(gen/fmap atom (gen/string-alphanumeric))))

(gen/sample (s/gen ::ratom))

;; (#atom["" 0x6ff87e8] #atom["" 0x710f246c] #atom["7" 0x43b11d93] #atom["N" 0x6fd3f101] #atom["k6" 0x15e9ebd4] #atom["xFdxB" 0x1dcd8141] #atom["" 0x56db5aee] #atom["J7" 0x26f0f2cd] #atom["67Pzk94" 0x7c1d0a8c] #atom["e" 0xdab4bc7])
#2018-03-2708:45hkjels(set? (deref %)) is what causes it to fail. Iā€™ll digg a little further#2018-03-2708:50borkdude@hkjels
(s/def ::set-ratom
  (s/with-gen (s/and #(instance? clojure.lang.IAtom %)
                     (fn [a]
                       (set? @a)))
    #(gen/fmap atom (gen/fmap set (gen/string-alphanumeric)))))
#2018-03-2708:59borkdude@hkjels You can check with gen/sample#2018-03-2709:01borkdude
(s/def ::id string?)
(s/def ::value string?)
(s/def ::label string?)

(s/def ::item (s/keys :req-un [::id ::value] :opt-un [::label]))

(s/def ::items
  (s/coll-of ::item :kind? set))

(take 2 (gen/sample (s/gen ::items))) ([{:id "", :value ""} {:id "", :value "", :label ""} {:id "", :value ""} {:id "", :value ""} {:id "", :value ""}] [{:id "", :value "", :label ""} {:id "", :value "6"} {:id "E", :value "2", :label ""} {:id "", :value "", :label "w"}])
Indeed, the generator doesnā€™t return sets
#2018-03-2709:02borkdudeDonā€™t know if this is a bug or by design. Alex probably knows#2018-03-2709:08borkdude@hkjels This is a workaround. Not pretty, but it works:
(s/def ::items
  (s/with-gen
    (s/coll-of ::item :kind? set)
    #(gen/fmap set (s/gen (s/coll-of ::item)))))
Iā€™m not sure if thereā€™s a better way as Iā€™ve not used this part of spec much yet.
#2018-03-2709:11hkjelshey, that works#2018-03-2709:11hkjelshmm, weird that (s/coll-of ::item :kind? set) does not return a set#2018-03-2709:11hkjelsthat must be a bug right?#2018-03-2709:12borkdudeDonā€™t know. Iā€™m sure Alex will reply later today#2018-03-2709:13hkjelsThank you!#2018-03-2709:59minimal@hkjels use a predicate like everything else: (s/coll-of ::item :kind set?)#2018-03-2710:00hkjelsahh, ok#2018-03-2710:00minimaland itā€™s :kind, not :kind?:#2018-03-2710:00hkjelsahh, that explains why it didnā€™t barf#2018-03-2710:00hkjelsšŸ™‚#2018-03-2710:01minimalyeah, would be nice if it told you#2018-03-2710:10borkdudeah I wasnā€™t awake enough to spot those typos šŸ™‚#2018-03-2714:39borkdudeI like it how spec lets you specify things gradually or as fine-grained as you want#2018-03-2719:34manutter51My google-fu is failing me: I want to write a spec for a function that takes an atom (actually a Reagent atom). Is there a way to do that?#2018-03-2720:20borkdude@manutter51 Maybe this helps: https://clojurians.slack.com/archives/C1B1BB2Q3/p1522140603000191#2018-03-2720:22borkdude@manutter51 replace the exact type with something from here: https://github.com/reagent-project/reagent/blob/master/src/reagent/ratom.cljs#L121#2018-03-2720:22borkdudebut IAtom will also do, but not specific to Reagent#2018-03-2720:24bbrinck
(s/def :test/ratom? #(instance? reagent.ratom/RAtom %))
(s/valid? :test/ratom? 1) ; false
(s/valid? :test/ratom? (r/atom nil)) ; true
#2018-03-2720:24manutter51Thanks much, Iā€™ll see if I can get that to work. The tricky part is this all has to work in CLJS, but I want to match both regular atoms and Reagent atoms so I can pass in test dummies#2018-03-2720:24borkdudethen IAtom seems the right abstraction#2018-03-2720:24bbrinckArguably, testing anything more than the class isnā€™t very useful, but YMMV#2018-03-2720:25bbrinck(I mean, testing the interface may be useful too, but the contents, maybe not so much)#2018-03-2720:25borkdudeyes, you can just as well create ratom dummies, so itā€™s a good question why you want to do that#2018-03-2720:26bbrinckThe advice I remember is that spec is mostly useful for testing values. Testing references is likely less useful#2018-03-2720:27bbrincksince testing the reference at any point in time doesnā€™t say much about what it will become at any point in the future#2018-03-2720:28bbrinckit might be more useful to spec the functions that mutate that reference, or, say, use spec in a validator#2018-03-2720:28manutter51Yeah, thatā€™s a good point, I could just use ratoms everywhere#2018-03-2720:28manutter51Thatā€™s probably the best solution ā€” tks much#2018-03-2802:56leontalbotHello! I am looking for a way to give free pass empty string values when provided as optional#2018-03-2802:56leontalbote.g.#2018-03-2802:57leontalbot
(s/def ::item (s/keys :req-un [::id] :opt-un [::url]))
#2018-03-2802:58leontalbotI could check for url, but if empty string, since it is in opt-un, let it be valid#2018-03-2802:58leontalbot(but if was in :req-un, then should apply full ::url spec)#2018-03-2802:59leontalbotis there a way to achieve that?#2018-03-2802:59leontalbotIt does work for nil but Iā€™d like it to work for ā€œā€ too#2018-03-2802:59leontalbotthanks!#2018-03-2803:14seancorfield@leontalbot I think what I'd do there is use :opt/url as the :opt-un key and define it as a spec like
(s/def :opt/url (s/or :url ::url :empty empty?))
which will allow it to be the full ::url string spec or nil or ""
#2018-03-2803:15seancorfield(or however you want empty values to be spec'd... maybe #{nil ""})#2018-03-2803:19leontalbotcool! Thanks @seancorfield. Is this good too to you:#2018-03-2803:19leontalbot
(defn valid-map?
  ā€œdissoc keys with empty vals to let optional keys pass, then validate...ā€
  [spec m]
  (s/valid? spec (apply dissoc m (for [[k v] m :when (#{ā€œā€ nil} v)] k))))
#2018-03-2803:20leontalbot(valid-map? ::my-spec-map my-map)#2018-03-2803:21leontalbotprobably less general#2018-03-2803:34seancorfield@leontalbot I wouldn't use that approach because now you have a custom function, not just a spec. You can't conform or explain with that approach.#2018-03-2803:35seancorfieldIt also wouldn't work for specs that have keys whose values can be nil or "" but are still required.#2018-03-2803:37seancorfieldFWIW, we have exactly this situation at work and the approach I suggested is basically what we do.#2018-03-2803:38leontalbotThank you so much @seancorfield!#2018-03-2803:40seancorfieldSpec is pretty flexible and powerful -- and the ability to use multiple qualified keywords for "similar" unqualified keys specs lets you deal with a lot of context.#2018-03-2803:41seancorfieldAnother aspect to consider is having different versions of specs at different "layers" in your application. For example, we have API-level specs (which deal with strings mostly) and we have domain-level specs (and we have a few DB-level specs as well).#2018-03-2803:41leontalbotmakes sense#2018-03-2803:41leontalbotand one last question:#2018-03-2803:42leontalbotwhy :opt/url and not :url/opt?#2018-03-2803:45leontalbot@seancorfield ^#2018-03-2803:45leontalbotthanks!#2018-03-2803:46leontalbotYeah I guess this is to map this hierarchy:#2018-03-2803:46leontalbot
(s/keys :req-un [::id] :opt-un [::url]))
#2018-03-2803:46leontalbotyeahā€¦#2018-03-2804:45seancorfieldYeah, if the unqualified key is :url then the options are :<something>/url#2018-03-2813:46leontalbotWanted to get end-user error message from spec. Useful for webform field validation.#2018-03-2813:46leontalbotFound Phrase library.#2018-03-2813:47leontalbotSeemed a bit overkill though. As I just wanted is attach an error text to a spec.#2018-03-2813:47leontalbotThen I saw https://github.com/metosin/spec-tools#2018-03-2813:48leontalbot
(s/explain (st/spec pos-int? {:reason ā€œpositiveā€}) -1)
; val: -1 fails predicate: pos-int?,  positive

(s/explain-data (st/spec pos-int? {:reason ā€œpositiveā€}) -1)
; #:clojure.spec.alpha{:problems [{:path [], :pred pos-int?, :val -1, :via [], :in [], :reason ā€œpositiveā€}]}
#2018-03-2813:49leontalbotMaybe use the :reason field accessible with explain?#2018-03-2813:49leontalbotWanted to know what you would do for form validation, thanks!#2018-03-2813:55guytheres https://github.com/bhb/expound as well which might be helpful#2018-03-2814:09leontalbotYes looked at this. Not sure if we can ā€œextractā€ only the error string#2018-03-2814:17leontalbothttps://github.com/bhb/expound/issues/77#2018-03-2814:37bbrinck@leontalbot Would this work?
(expound/def :user/name string? "should be a valid name")

(defn msg [spec val]
  (if (s/valid? spec val)
    nil
    (expound/error-message spec)))

(msg :user/name "John") ; => nil
(msg :user/name :John) ;  => "should be a valid name"
#2018-03-2814:42leontalbotHey! Thanks for answering @bbrinck! Fantastic! Thanks!#2018-03-2814:42bbrincknp, let me know if you have any other questions about expound#2018-03-2814:43leontalbotOk! Nice lib btw!#2018-03-2814:43bbrinckthanks! šŸ™‚#2018-03-2817:43borkdude@bbrinck Sometimes I get an error message from expound when it complains about not being able to render an error (exact error message I donā€™t have handy). Wouldnā€™t it be better in that case to print the vanilla spec error instead of only the expound error?#2018-03-2817:55bbrinck@borkdude Is this a bug in expound? Or a case where your spec has conformers, perhaps?#2018-03-2817:55borkdudethe latter#2018-03-2817:56bbrinckGotcha. Yeah, in that case, it might very well make sense to just print the default error. Let me think about that. Thanks for the idea!#2018-03-2817:57bbrinckAlthough if youā€™re using conformers, the vanilla spec error may not be very helpful either šŸ™‚#2018-03-2817:57bbrinckBut still better than nothing, for sure#2018-03-2817:57bbrinckhttps://github.com/bhb/expound/issues/78#2018-03-2817:58borkdudeUsually when I run into this, I turn off expound, re-run the code, inspect the error, fix it, and enable expound again.#2018-03-2817:58bbrinckAgreed, thatā€™s a pain. Iā€™ll fix it.#2018-03-2818:08seancorfield@bbrinck FWIW, that was why we tried and then stopped using Expound -- we have several conformers in our specs.#2018-03-2818:18bbrinck@seancorfield Would it work in your case to just fallback to s/explain?#2018-03-2818:19bbrinckDepending on how many conformers you have, I guess at some point you would rarely see an expound error#2018-03-2818:20bbrinckSo the fix @borkdude suggested is a good idea, but itā€™s mostly useful for projects that have a relatively small number of conformers compared to the total usage of spec#2018-03-2818:20bbrinck@seancorfield FWIW, apparently pinpointer works with conformed values#2018-03-2818:25seancorfield@bbrinck Well, the main cases where we wanted Expound's better messages were all conformer-specs, so we just fell back to explain ourselves šŸ™‚#2018-03-2818:25seancorfieldI'll take a look at pinpointer -- I hadn't heard of that.#2018-03-2820:40lilactownhas anyone used spec-tools to generate swagger objects?#2018-03-2820:40lilactownI'm having trouble with getting the response schema's to generate correctly#2018-03-2903:47ikitommi@lilactown one of the authors here and just integrated that to reitit routing library and worked ok. Porting Schema=>Swagger into the same model, which will cause small changes to spec-side to get unified tranformation model for both. How can I help?#2018-03-2903:52ikitommiStill need to backport support for json schema refs for recursive references, also will change the apis a bit. e.g. tranformations return both the transformed swagger schemas and the named schema definitions they refer to.#2018-03-2914:17lilactownI was trying to use the swagger-spec fn with an endpoint that had a response map like in the README:
:responses {200 {::swagger/spec ::user
                                                :description "Found it!"}
                                           404 {:description "Ohnoes."}}
#2018-03-2914:18lilactownhowever, this was getting converted to something like :x-spec-tools.swagger.core/spec :user/user#2018-03-2914:19lilactowninstead of expanding the ::user spec into the :schema key#2018-03-2918:23ikitommi@lilactown sorry, thread got buried. You should raise the ::swagger/responses one level up. So, ::swagger/responses {200 {:schema ::user}}#2018-03-2918:24ikitommishould add specs to the input so it would tell what happened.#2018-03-2918:41ikitommialso, I think there isn't ::swagger/spec defined, see https://github.com/metosin/spec-tools/blob/master/src/spec_tools/swagger/core.cljc#L103-L131#2018-03-2918:55lilactownyeah, I ended up using ::swagger/responses instead#2018-03-2918:55lilactownbut I was basing it off of this example in the README#2018-03-2918:55lilactownhttps://github.com/metosin/spec-tools#full-example#2018-03-2918:55lilactownwhich AFAICT does not work#2018-03-2920:39ikitommiOh, good catch! I was too blind to see that it's not correct. Could you write an issue, need to fix either the code or the sample in README.#2018-03-2916:38jimbobhow do you guys organize your specs? Are all of your specs in a top level spec directory? or do you also mix and match spec assertions and definitions in core namespaces#2018-03-2917:45taylorI usually fdef next to the function, but put other specs in their own namespaces#2018-03-2916:38jimbobany tips?#2018-03-2917:44borkdudeWhen I write an fdef I place it above the function#2018-03-2918:56seancorfield@ben.borders It depends. If I need code to work with pre-1.9 Clojure, I put all the specs in a separate namespace. Otherwise, if I'm spec'ing data I tend to put the specs in their own namespace (along with predicates and closely related helper functions), else if I'm spec'ing functions I tend to put those directly above the function they are for.#2018-03-2919:15borkdude@ben.borders Just moved the fdef + related specs to a widget/spec.cljs namespace because it became too large šŸ˜‰#2018-03-3006:41robert-stuttafordlikely a common question, and also likely not something spec is suited for, but iā€™m curious even so - i have a map with start and end dates. is there a spec pattern for declaring a relationship between those two values, i.e. one must be larger than the other? one structure that occurs is spec/fdefā€™s :ret, but iā€™m wondering if perhaps the spec api has something else like this?#2018-03-3009:55borkdude@robert-stuttaford you can use s/and + a conformer function#2018-03-3009:56borkdudeSomething like
(s/def ::dates
  (s/and
   (s/keys :req-un [::start ::end])
   (fn [{:keys [start end]}]
     (< start end))))
where < is the comparison function of your choice.
#2018-03-3010:08robert-stuttafordwonderful, thank you! that seems ridiculously simple, in hindsight. like, ā€˜how did i not see thisā€™ simple#2018-03-3010:10robert-stuttaford@borkdude i guess iā€™d have to write my own generator too then#2018-03-3010:11borkdudeDonā€™t know if the default generator generates enough samples where the conformer can strip away the invalid ones. Not as efficient as writing your own, but it could work#2018-03-3010:15borkdudeitā€™s about 50% chance for each sample#2018-03-3010:17robert-stuttafordright#2018-03-3010:22borkdudewhen you write an fdef in another namespace than the function, require the namespace where the function lives? or donā€™t and just fully qualify the symbol in fdef? hmm#2018-03-3010:23borkdudeRight now I have an init namespace that just requires them all, so no cyclic dependencies and I can just fully qualify in fdef#2018-03-3010:24robert-stuttafordwhatā€™s causing you to want to keep the fdef separate, @borkdude?#2018-03-3010:24borkdudebecause itā€™s more than 100 lines#2018-03-3010:26dominicmI feel like I remember there being some kind of way to add conformers to specs "locally"? Is that a correct memory?#2018-03-3010:26robert-stuttafordah šŸ™‚ Qualifies fn-sym with resolve, or using *ns* if no resolution found. seems to suggest that fully-qualified is fine, and probably better for discoverability#2018-03-3010:28borkdudeI have an interesting problem:
(s/def ::selection
  (s/nilable
   (s/and
    (s/keys :req-un [:selection/options
                     :selection/id]
            :opt-un [:selection/default-option
                     :selection/type])
    ;; default option must be one of the option ids
    (fn [dropdown]
      (if-let [opt (:default-option dropdown)]
        (contains?
         (set (map :id (:options dropdown)))
         opt)
        true)))))

(s/def ::dropdown ::selection)
I want the id in dropdown to be optionalā€¦ Maybe I should make an extra spec without the required id and then make the id in selection required with s/and?
#2018-03-3010:31borkdudeLike this:
(s/def ::selection
  (s/nilable
   (s/and ::selection*
          (s/keys :req-un [:selection/id]))))

(s/def ::selection*
  (s/nilable
   (s/and
    (s/keys :req-un [:selection/options]
            :opt-un [:selection/default-option
                     :selection/type])
    ;; default option must be one of the option ids
    (fn [dropdown]
      (if-let [opt (:default-option dropdown)]
        (contains?
         (set (map :id (:options dropdown)))
         opt)
        true)))))

(s/def ::dropdown ::selection*)
#2018-03-3010:34borkdudeSeems to work#2018-03-3011:51robert-stuttafordwhatā€™s the blessed method for checking whether a spec is registered? s/spec? seems to be for something else#2018-03-3011:52robert-stuttafordaha s/get-spec#2018-03-3011:53borkdude@robert-stuttaford brute force method: use a println in a conformer šŸ˜›#2018-03-3011:53borkdudeor just make an obvious mistake and if no exception, then no šŸ˜‰#2018-03-3011:54robert-stuttaford-grin- its for datomic attrs. the code using the spec canā€™t assume a spec is registered; it has to check first before it attempts to use it to validate.#2018-03-3011:54robert-stuttafordget-spec works#2018-03-3011:59robert-stuttafordgrr, iā€™m having to wrap my defmulti with a normal defn so that i can instrument it. otherwise defmethods defined after instrumentation fail#2018-03-3011:59borkdudeFunny that s/spec? doesnā€™t return a boolean#2018-03-3012:00borkdudedonā€™t all new fdefs after instrumentation fail?#2018-03-3012:00borkdudein the sense that they arenā€™t instrumented yet#2018-03-3013:01Alex Miller (Clojure team)Yes, although I wouldnā€™t call that a fail#2018-03-3013:05borkdudemore appropriate: not yet in effect#2018-03-3014:33gfredericksargument validation in libraries: going forward, should it be done entirely with s/fdef, meaning no validation happens unless the user instruments the functions?#2018-03-3014:35borkdudes/assert is also an option I guess which can be compiled away#2018-03-3014:35borkdudebut for arguments s/fdef is nicer#2018-03-3014:37gfredericksthe downside is that things are a lot more GIGO for users who don't think to instrument e.g., as a user, can I easily instrument all the spec'd functions in all my libraries without having to know which ones have specs? is that too much? wouldn't it be more efficient to instrument only the functions I'm calling directly?#2018-03-3014:38borkdudeyou can instrument all fdef-ed functions with stest/instrument?#2018-03-3014:38borkdudebut it has to be after you load their namespaces#2018-03-3014:38borkdudemaybe adding it to reloaded.repl/reset will be a common thing#2018-03-3014:39borkdudebut I get what you mean now. so you want to go only one level deep#2018-03-3014:39borkdudeno transitive fdef checking#2018-03-3014:40gfredericksthat'd be nice, since you probably have a large tree of libraries and don't want to slow down your dev by testing all the interactions between them#2018-03-3014:41borkdudeWhat overhead are we talking about? I donā€™t mind a couple of milliseconds more during dev#2018-03-3014:41gfrederickstotally depends on the libraries and what they're doing#2018-03-3014:42gfredericksin the extreme case, if specs get added to most of the clojure.core functions, instrumenting those will result in milliyears instead of milliseconds#2018-03-3014:45borkdudegood question#2018-03-3020:20hlshipHere's a question. I have a value that I want to ensure is a keyword, string, or symbol AND that it's string conforms to a particular regexp. Example here: https://github.com/walmartlabs/lacinia/blob/05940c7f7819fd88bc4e50c860b8d9854c3fa0b2/src/com/walmartlabs/lacinia/schema.clj#L306#2018-03-3020:21Alex Miller (Clojure team)thatā€™s not a question :)#2018-03-3020:22hlshipI'm working on it ...#2018-03-3020:22Alex Miller (Clojure team):)#2018-03-3020:21hlshipI've been down this path before, and what I've found is that the next term in the s/and gets the conformed value from the s/or, a tuple of (say), [:keyword :frob].#2018-03-3020:23hlshipThat's been fine so far EXCEPT as I'm switching to using Expound, the use of a conformer here is a problem:
(s/explain ::schema/enum-value "this-and-that")
             clojure.lang.ExceptionInfo: Cannot convert path. This can be caused by using conformers to transform values, which is not supported in Expound
clojure.lang.Compiler$CompilerException: clojure.lang.ExceptionInfo: Cannot convert path. This can be caused by using conformers to transform values, which is not supported in Expound {:form "this-and-that", :val [:string "this-and-that"], :in [], :in' []}, compiling:(/Users/hlship/workspaces/github/lacinia/src/com/walmartlabs/lacinia/expound.clj:50:3)
#2018-03-3020:23Alex Miller (Clojure team)this issue is actually discussed in the backchat#2018-03-3020:24hlshipRecently? Got a link?#2018-03-3020:24Alex Miller (Clojure team)2 days ago in this room - just scroll up till you see the expound stuff#2018-03-3020:29hlshipThat discussion wasn't helpful, if its the right one. I think they're hitting the same problem and want Expound to print it differently. I want to modify my spec to not trip over this scenario. s/nonconforming may work!#2018-03-3020:24hlshipSo my question is, how can I achieve the kind of spec I want in a way that avoids the use of a conformer in the middle.
#2018-03-3020:24Alex Miller (Clojure team)another option is to wrap s/nonconforming around s/or#2018-03-3020:25Alex Miller (Clojure team)then you get just the value without the tag#2018-03-3020:25Alex Miller (Clojure team)when you conform that is#2018-03-3020:25Alex Miller (Clojure team)currently thatā€™s an undocumented function but I think itā€™s likely we will either keep it or add a nonconforming variant of s/or#2018-03-3020:27seancorfieldAnd in the backchat, one suggestion was to look at pinpointer instead of Expound.#2018-03-3020:28bbrinck@hlship I will likely be adding a ā€œfallbackā€ feature to expound soonish where you will see the vanilla spec error in this case#2018-03-3020:29bbrinckNote that will help if you only occasionally use conformers, but not if you have lots of them. Definitely check out pinpointer šŸ™‚#2018-03-3020:29hlshipAgain, I'm quite willing to modify my spec to bypass this problem.#2018-03-3020:29bbrinckAh, sorry, I missed that in the thread. Yes, thatā€™d be the best approach! šŸ™‚#2018-03-3020:31bbrinckIā€™ve seen people run into other issues with conformers so itā€™s probably best to avoid e.g. https://groups.google.com/forum/#!searchin/clojure/conformer%7Csort:date/clojure/Tdb3ksDeVnU/uU0NT4x6AwAJ#2018-03-3020:33bbrinckFWIW, I tried to support conformers in expound but itā€™s really tricky if youā€™re just looking at the explain-data#2018-03-3020:34hlships/nonconforming looks to be just what I want:
(s/explain ::schema/enum-value "this-and-that")
-- Spec failed --------------------

  "this-and-that"

must be a valid GraphQL identifier: contain only letters, numbers, and underscores
BTW why are explicit messages not indented by Expound? Should I file an issue?
#2018-03-3020:35bbrinckCan you modify that to show what youā€™d prefer?#2018-03-3020:36bbrinck(Iā€™m always happy to get bug reports too šŸ™‚ if thatā€™s easier to discuss the options)#2018-03-3020:37bbrinckIā€™ll say this - off the top of my head, I think itā€™s working as I intended, but Iā€™m always interested in improving the layout of error messages if itā€™s not clear#2018-03-3020:40hlshipHere's a better example:
(s/explain ::schema/resolve {})
-- Spec failed --------------------

  {}

should satisfy

  fn?

or

implement the com.walmartlabs.lacina.resolve/FieldResolver protocol
The final line should be indented the same as the fn? line, don't you think?
;; is passed and should return.
(s/def ::resolve (s/or :function ::resolver-fn
                       :protocol ::resolver-type))
(s/def ::resolver-fn fn?)
(s/def ::resolver-type #(satisfies? resolve/FieldResolver %))
#2018-03-3020:52bbrinckYeah, when itā€™s part of the ā€œorā€ it looks weird. I guess I was thinking that since ā€œshouldā€ starts on left, custom messages would be the same.#2018-03-3021:05bbrinckThe reason itā€™s on the left is that it should be in the same spot as ā€œshouldā€ like so:
(s/def :example/temp #{:hot :cold})  
(expound/expound :example/temp 1)
;;-- Spec failed --------------------
;;
;;  1
;;
;;should be one of: :cold, :hot

(expound/def :example/name string? "should be a string")
(expound/expound :example/name 1)
;;-- Spec failed --------------------
;;
;;  1
;;
;;should be a string
#2018-03-3021:05bbrinckbut I agree that when itā€™s part of a long ā€œsatisfy..orā€ block, it looks bad. Iā€™ll make a bug#2018-03-3021:07bbrinck@hlship https://github.com/bhb/expound/issues/80 thanks for letting me know!#2018-03-3021:46hlshipAnother expound question (time for its own channel?) :
user=> (require [clojure.spec.alpha :as s])
nil
user=> (s/explain keyword? 3)
val: 3 fails predicate: :clojure.spec.alpha/unknown
nil
user=> (require '[expound.alpha :as expound])
nil
user=> (alter-var-root #'s/*explain-out* (constantly expound/printer))
#object[expound.alpha$printer 0x3c64e2a2 "
this works fine when I use set!, but not when I use alter-var-root!. Any ideas?
#2018-03-3021:50bbrinckIā€™ve always used set! myself, so Iā€™m not sure. Can you talk a little bit more about your use case and neither set! nor binding are a good fit here?#2018-03-3021:50hlshipWell, at application startup, might want to alter-var-root, so that any later threads will use the Expound printer, not the default one.#2018-03-3021:58bbrinckThanks for that. The short answer is I donā€™t know unfortunately - this seems to affect any printer, not just expound. Try: (alter-var-root #'s/*explain-out* (constantly (fn [ed] "Hello!"))). It may be a result of the way the REPL is set up - does it reproduce if you put the alter-var-root in the main of an application (as opposed to doing it in the REPL context)?#2018-03-3022:00hlshipI suspect it's because clojure.spec is AOTed. I hit something similar with redefining stuff in clojure.test and the hack was to do a RT/loadResourceScript: https://github.com/AvisoNovate/pretty/blob/db4e7677f74d8efb149db3e9ba5974fa9c84b6a0/src/io/aviso/repl.clj#L72#2018-03-3022:03bbrinckAh, gotcha. I certainly understand your use case and I suspect others may run into the same thing. If you figure it out, let me know and Iā€™ll update the docs. I have a note about using alter-var-root in non-REPL context, which IIRC, works fine, but if your scenario is at the REPL, then no dice#2018-03-3022:07hlshipbinding should work, as that will carry over into most started threads, including core.async threads.#2018-03-3023:32Alex Miller (Clojure team)I think the difference is that the repl binds explain-out so alter-var-root changes the root but the repl binding is the one being seen#2018-03-3023:33Alex Miller (Clojure team)So you have to set! at the repl#2018-03-3115:34gabrielehow can i write a spec to validate a map like this: {[:type1 "args"] {:key-type1 value} [:type2 "args"] {:key-type2 value} ...(type-n)}#2018-03-3115:35gabrielewhere the value is dependent on the first element of the list that composes the key#2018-03-3115:39borkdude@gabriele.carrettoni First of all I would try to get the source of that data in a more workable format#2018-03-3115:40borkdudeI usually try to go from dynamic keys to something more regular#2018-03-3115:41gabriele@borkdude i see, i wanted to use specs to do that#2018-03-3115:41borkdudeE.g.
(def data [{:type 1
            :args ā€œargsā€
            :value value}
           {:type 2
            :args ā€œargsā€
            :value value}])
#2018-03-3115:43borkdudeAnd then you can spec that after transforming the awkwardly shaped data and verify if your normalization worked#2018-03-3115:44gabrielethanks, i'll do that#2018-03-3115:47borkdude@gabriele.carrettoni E.g.:
(def data {[:type1 "args"] {:key-type1 1}})

(defn transform-data
  [data]
  (for [[k v] data]
    (let [[type-key args] k
          other-key (keyword (str "key-"
                                  (name type-key)))
          value (get v other-key)]
      {:type type-key
       :args args
       :value value})))

(transform-data data) ;;=> ({:type :type1, :args "args", :value 1})
#2018-04-0220:07avišŸ‘‹ hi all! Iā€™m having some trouble with a function spec with a :fn predicateā€¦ and my Web-searching has turned up nothing. Iā€™m using stest/check like so: (-> (stest/check 'fc4c.core/shrink) first stest/abbrev-result) wherein fc4c.core/shrink is my function that has an attached spec. I always get a failure, butā€¦ if I grab the value of :val from the result, and then manually pass it to my predicate, like so: (pred val) then I always get true. So am I missing something? I thought that if the predicate returns true, then the value conformsā€¦ Iā€™m confused! Thanks!#2018-04-0220:13seancorfield@aviflax We'll need to see a bit more code in order to help you...#2018-04-0220:15aviyeah? Iā€™m happy to share more code, but I think Iā€™m trying to just confirm an invariant about stest/check and function specsā€¦ that if the :fn predicate returns true then the ā€œtest caseā€ will be considered to have succeededā€¦ does that make sense? Or am I thinking about this wrong?#2018-04-0220:16taylorare you sure itā€™s failing due to the :fn spec? could it be failing on the :ret spec?#2018-04-0220:17aviIā€™m fairly sure but the output of check failures is somewhat overwhelming, even when abbreviated with abbrev-result ā€¦ Iā€™ll double-check right now#2018-04-0220:19aviHereā€™s a chunk of the output:
:failure
 {:clojure.spec.alpha/problems
  [{:path [:fn],
    :pred
    (clojure.core/fn
#2018-04-0220:19avithe function spec is here: https://github.com/FundingCircle/fc4c/blob/tests-yay/src/fc4c/core.clj#L76#2018-04-0220:25taylor#2018-04-0220:27aviThanks for the help! Iā€™m a little confusedā€¦ I donā€™t see that. If I dig the value of :ret out of the value of :val ā€” I see a map.#2018-04-0220:27aviSo it looks like shrink returned a map when check called it?#2018-04-0220:28tayloractually disregard, I copy/pasted your code poorly#2018-04-0220:28aviah ok no worries! happens to us all!#2018-04-0220:40taylorI was just able to successfully check the function when I replaced the :args and :ret spec:
:args (s/cat :in map?)
        :ret (s/nilable map?)
#2018-04-0220:40taylorsince the shrink function is more general/doesnā€™t need to know about your :diagram spec#2018-04-0220:41taylorbutā€¦ I did try running your :fn spec function against that sample and it does return true, so not sure whatā€™s up with that#2018-04-0220:45avithat makes senseā€¦ thanks for the help!#2018-04-0220:48seancorfield@aviflax I notice you're post-walking the tree -- are you aware of the bug in postwalk where MapEntry elements are not preserved?#2018-04-0220:48avino, I wasnā€™t aware of that#2018-04-0220:48aviI guess I can search the Web to find more info on that#2018-04-0220:49seancorfieldhttps://dev.clojure.org/jira/browse/CLJ-2031#2018-04-0220:49taylorthat doesnā€™t seem to be the big issue here#2018-04-0220:49aviBut Iā€™m not sure how that helps me understand the ā€œcontractā€ of fn predicatesā€¦ my predicate is returning true so I just donā€™t understand why a failure is being reported#2018-04-0220:50taylormaybe dumb question, but have you tried restarting your REPL if youā€™ve been working on this for a while?#2018-04-0220:50avinot dumb!#2018-04-0220:50aviyes, more than a few times#2018-04-0220:50avišŸ¤” maybe thereā€™s a problem with the name shrink ā€¦ maybe ā€¦ thatā€™s somehow confusing check since it internally does shrinking (seems like a long shot)#2018-04-0220:51seancorfieldWhy not test it with a much simpler :fn predicate that is (constantly true) and see if it still fails?#2018-04-0220:51avihm ok good idea Iā€™ll try that!#2018-04-0220:52seancorfieldIf that works (passes) then build the predicate up, piece by piece until you get a failure.#2018-04-0220:52taylorhis :fn spec predicate does return true if you manually call it with the map from the check problem output, which is odd#2018-04-0220:53aviyeah thatā€™s precisely what Iā€™m confused by#2018-04-0220:53taylorbut I agree w/trying to (re)build up from the simplest working case until it breaks#2018-04-0220:54avibut yeah, since (as I just learned) replacing the predicate with (constantly true) does lead to the check call succeedingā€¦ Iā€™ll try rebuilding the predicate to try to debug at what point find a small case wherein I can trigger the confusing behavior#2018-04-0220:54tayloranother thing to keep in mind is that it checks out fine if you donā€™t use his custom keys spec for in/out#2018-04-0220:55aviyeahā€¦ my input and output specs (and thus the generated args) are deeply nested collsā€¦ not sure why thatā€™d matter but maybe it does#2018-04-0221:15avianyone know whether/how to use stest/check and have it skip shrinking?#2018-04-0221:22aviah never mind, I can work with it šŸ˜‰#2018-04-0221:45avihmm well Iā€™ve been able to get the predicate to always work as expected by changing (= in-vals ret-vals) to (= (set in-vals) (set ret-vals)) ā€” it appears that the two colls had the same values but in different orders. Iā€™m confused as to why I wasnā€™t seeing this earlier. Iā€™m very unclear on what Iā€™m doing differently now šŸ˜ž ā€¦ PEBKAC ā€¦ sorry to have bothered everyone#2018-04-0221:57seancorfieldThat's cool that you've figured out the problem!#2018-04-0221:58avithanks!#2018-04-0221:58aviyes!#2018-04-0221:59avibut also a bit disorienting that I suddenly canā€™t reproduce my confusing tangentā€¦#2018-04-0222:00avimy only idea is that earlier I was copy-and-pasting the values as strings and then pasting them into the repl prompt, versus using e.g. (-> *1 :key :key) to get the in-memory values insteadā€¦ I donā€™t know why that would matter though.#2018-04-0423:30flyboarderhello everyone, is there a version of multi-spec that uses an outside piece of data to determine the shape of the data? Instead of the current multi-spec which uses a piece of the data to figure out the shape of the rest of it?#2018-04-0501:52Alex Miller (Clojure team)No#2018-04-0501:54Alex Miller (Clojure team)And generally that implies you are prob doing something you shouldnā€™t as you then have a spec that is impure.#2018-04-0614:39flyboarderCould you explain an impure spec? I am trying to generate a spec from some nested data#2018-04-0516:07Charles FourdrignierIf I want to specā€™ data from a CSV file, what would be the best strategy ? Should I describe it as a tuple ? Or should I convert it to a map then spec the map itself ? (I donā€™t find any article or blog post on this subject.)#2018-04-0516:32Alex Miller (Clojure team)a csv file is inherently a collection of collections of strings#2018-04-0516:32Alex Miller (Clojure team)and thus very uninteresting to spec and unlikely to tell you much#2018-04-0516:33Alex Miller (Clojure team)I suspect from an app perspective, it is far more interesting to convert to maps (coercing data if needed) and to spec the result, which is the data your app actually uses#2018-04-0516:41Charles FourdrignierThanks ! Tuples was my first intuition, but some reading changes my mind and your response finish to convince me.#2018-04-0520:53Drew VerleeI have some unqualified keywords that conceptually map to some specs i have created. Whats the suggested way to convert a unqualified keyword to a qualified one so i can exercise it (generate data) or conversely, use a generator on a unqualified keyword. I feel i could do this my pulling things apart and putting them back together with strings but im guessing im missing an obvious exisiting way to handle this:
(str *ns* "/" :unqalified)
#2018-04-0521:25manderson@drewverlee any reason that you couldn't just add namespaces to your unqualified keywords? If you do want to go back and forth, the best way I can think of is as follows (perhaps someone knows of something better):
(def +qualified+
  (keyword (-> *ns* ns-name str) (name :unqualified)))
=> #'user/+qualified+

+qualified+
=> :user/unqualified

(-> +qualified+ name keyword)
=> :unqualified
#2018-04-0613:09Drew Verleeno reason, just wonder if thats the right approach.#2018-04-0614:11mandersonSpec is opinionated about using namespaced keywords (see the rationale doc on this: https://clojure.org/about/spec#_global_namespaced_names_are_more_important), so in general, if you are using spec, I've found that it is best to follow this pattern.#2018-04-0613:01kardanAnyone who can point on where to read on how structure specs? I started to spec in com.example.user but then it was odd define the user spec using namespaced keys. Would really want to to type (s/valid? :: #::{:email ā€œ<mailto:/cdn-cgi/l/email-protection|/cdn-cgi/l/email-protection>ā€ :first-name ā€œBillā€ :last-name ā€œKarlsonā€}) . So Iā€™m guessing I should read up šŸ™‚#2018-04-0712:56leongrapenthin@kardan it has become commonplace to use unsegmented namespaces for domain specific entities like :user/email#2018-04-0712:57leongrapenthinBut I'd also love if someone wrote a state of the art summary because there is still lots of experimentation going on#2018-04-0712:59leongrapenthin@kardan consider that in your example you usually don't come up with a single user spec#2018-04-0713:00leongrapenthinin practice you end up creating different specs with keys in the usernamespace depending on what you need#2018-04-0713:01leongrapenthinfor example an api that changes a user may say all user keys are optional#2018-04-0713:01leongrapenthinan api that creates a user may have some required user namespaced keys#2018-04-0713:02leongrapenthinthis flexibility is easily missed when you are looking to create "the" ::user spec#2018-04-0713:04leongrapenthinthe great thing is that you can compose and mix the keys freely in the context you need them whereas in classical type systems you need inheritance or whatever to reuse them.#2018-04-0717:25ikitommiunsegmented namespaces are not safe, :user/id might mean different things in different domains. If you need to integrate the two domains, you'll have conflictings specs.#2018-04-0719:05seancorfield@ikitommi It all depends on the scope of the data. If you're writing an application, not a library, and you have data whose scope is just that application, and that data is used entirely internally in that application, then something like :user/email might be fine. If that needs to flow through other code for additional processing, then you might want a more unique prefix. If data is truly isolated to a single namespace for processing (and is entirely opaque to everything else), the ::email would make more sense. If the data does lie somewhere in between, a "reasonably" unique prefix should be sufficient. Using namespace aliases makes working with namespaced maps a lot less painful -- and you can decide whether those should be real (code-based) namespaces or just arbitrary namespace prefixes independent of the alias. /cc @kardan#2018-04-0719:31mathpunkI'm starting a wrapper for an interesting Java library, for which I have domain knowledge of the problem they're working on. I am interested in adding specs based on invariants that I know so as to make it easier to use this library. Advice on how to approach such a goal is most welcome.#2018-04-0914:39bbrinckPerhaps this post would be helpful? http://blog.cognitect.com/blog/2017/6/19/improving-on-types-specing-a-java-library#2018-04-1118:55mathpunk@U08EKSQMS This is exactly the kind of thing I was looking for, thank you!#2018-04-1118:58bbrincknp, good luck!#2018-04-0812:09leongrapenthin@seancorfield what do you think about keys like :git/sha used in ctd#2018-04-0812:10leongrapenthin@alexmiller would be interested how you decided that, too#2018-04-0812:20Alex Miller (Clojure team)Sorry, whatā€™s the question?#2018-04-0819:14seancorfield@leongrapenthin when Alex talked about that (on the list? in #tools-deps here?) he said that's a human written format so it needs to be clear but not too verbose. A while different set of considerations compared to code. #2018-04-0819:50leongrapenthin@alexmiller I'm interested in a rule when its ok to pick unqualified/unsegmented namespaces. "Lib no;app yes" seems what has emerged mostly, but in tools.deps you claim for example "git". When should a lib author choose to do this and how can he not be worried about conflicts?#2018-04-0819:51leongrapenthinOr should we say unqualifies nses in libs are reserved for core libs#2018-04-0819:52leongrapenthin@seancorfield thanks, that makes sense from a practical standpoint. I still couldn't do it in my libs without worrying about conflicts ))#2018-04-0901:04Oliver GeorgeI have an "on the fly spec generation in clojurescript" question.#2018-04-0901:05Oliver GeorgeI'd like to generate and register new specs but clojure-spec has a macro based api and clojurescript doesn't have eval.#2018-04-0901:06Oliver GeorgeCan someone point me at a sane approach please.#2018-04-0901:07Oliver GeorgeIt's possible a macro is the right way to do this but I get super confused about when macros evaluate in clojurescript.#2018-04-0901:08Oliver GeorgeMy spec generation looks at existing registered specs. That means my macro needs to consider the correct registry (presumably there's a CLJ one at compile time and a CLJS one at runtime).#2018-04-0901:08Oliver GeorgePretty sure I'm lots in the weeds.#2018-04-1006:17aaron51looking to auto-generate an HTML form from spec/schema for non-programmers (not API documentation, with validations). is there a simpler alternative to swagger out there, that works with compojure.api.sweet?#2018-04-1119:13donaldballWhatā€™s the least wrong way of tightening a spec in the registry in the context of another spec? For example, Iā€™m writing specs for config maps in general, and for config maps for production. The general spec for e.g. :config.mode might be #{:app :lib} but when it appears in my :config.prod spec, Iā€™d like to further constrain it to be simply #{:lib}. s/and with predicates is a perfectly adequate way to do this, but I seem to recall a better way.#2018-04-1119:47Alex Miller (Clojure team)s/and is what I would say#2018-04-1120:09donaldballIt makes generating values unlikely, but this is also a case where I really donā€™t care about that#2018-04-1212:49Andreas LiljeqvistHow would I spec a map like {:type akeyword :args dependsonkey}?#2018-04-1212:54Alex Miller (Clojure team)s/multi-spec is designed to handle cases where you choose the spec based on the data (here the :type)#2018-04-1212:54Alex Miller (Clojure team)So that may be a good match here#2018-04-1212:54Andreas LiljeqvistI can use a multimethod to spec the map depending on type (multimethod methodname :akeyword [m] (s/keys [:args]))#2018-04-1212:55Alex Miller (Clojure team)Yes, thatā€™s how s/multi-spec works#2018-04-1212:55Andreas LiljeqvistBut I can't see at the moment how I would specify that :args should have a different spec for that match#2018-04-1212:56Andreas LiljeqvistLike that it should accept string? if :type is :a, but int? if :type is :b#2018-04-1213:02Alex Miller (Clojure team)Is :args actually unnamespaced?#2018-04-1213:02Andreas Liljeqvistnah, everything should be namespaced#2018-04-1213:05Alex Miller (Clojure team)When unsure how to spec something, itā€™s best to always return to how to represent the truth of your actual data. In this case youā€™re saying that you have one attribute that can have a variety of different structures#2018-04-1213:06Alex Miller (Clojure team)So you need to capture that in the spec#2018-04-1213:06Alex Miller (Clojure team)The spec for that attribute is A or B or C#2018-04-1213:08Alex Miller (Clojure team)You then have a separate constraint that says that a particular value of :type should co-occur with a particular form of :args and you should capture that constraint as a separate predicate #2018-04-1213:09Alex Miller (Clojure team)So you would model the attribute with s/or, the map with s/keys, and the constraint by s/and-ing the map and the constraint#2018-04-1213:23Andreas LiljeqvistThank you. s/and-ing is always powerful, only problem is that we have to provide a custom-gen.#2018-04-1213:23Andreas LiljeqvistBut usually I have to write custom-gens anyway...#2018-04-1214:52guyWhat would you do for speccing [& args] ?#2018-04-1214:53Alex Miller (Clojure team)(s/cat :args (s/* any?))#2018-04-1214:53guyah perfect thanks#2018-04-1214:53Alex Miller (Clojure team)unless you have other knowledge about args#2018-04-1217:32kennyIs it possible to have a multi-spec dispatch on the first value of a vector and return a Spec for the rest of the vector? i.e. take [:my-vec 1 "2"]. The multi-spec would dispatch on :my-vec and each defmethod would return a spec for (vec (rest [:my-vec 1 "2"])) - [1 "2"].#2018-04-1217:36Alex Miller (Clojure team)no, but you could dispatch on the first value of a vector and return a spec for the whole vector#2018-04-1217:37kennyYeah... It's just the Spec for the first part of the vector is uninteresting -- it's always going to be any?. This makes the return value for the defmethods very repetitive.#2018-04-1217:39kenny
(defmethod event-vec :my-vec
  [_]
  (s/cat :x any? :a int? :b string?)
         ^^^^^^^
  )
That part will always be the same.
#2018-04-1217:45Alex Miller (Clojure team)if only there was a way to remove boilerplate syntaxā€¦.#2018-04-1217:45Alex Miller (Clojure team)oh wait, macros! :)#2018-04-1217:49kennyThat's also possible. The problem there is that once I move that to a macro, I need to move all functions that register a method for that spec to be macros.#2018-04-1217:54dadairWould x always be any? You could have x be say #{:my-vec} so the spec is more specific to the event spec you are returning?#2018-04-1217:54dadairMore specific for tests around that event#2018-04-1217:54kennyYes, always any?.#2018-04-1217:55dadairbut for that specific defmethod isntā€™ x :my-vec?#2018-04-1217:55kennyYes, but that's already guaranteed because the multimethod is called.#2018-04-1217:57kennyThe API consists of a lot of functions that look like this:
(defn reg-my-thing 
  [id spec other-stuff]
  (defmethod my-multimethod id
    [_]
    (s/cat :x any? :rest spec))
  ;; do other stuff
  )
In order to do what you're saying I'd need make most of the API macros. That isn't the end of the world but it does make the code base a lot messier to do what seems like such a simple operation.
#2018-04-1217:57Alex Miller (Clojure team)@kenny re ā€œThe problem there is that once I move that to a macro, I need to move all functions that register a method for that spec to be macros. ā€ - why?#2018-04-1217:57kennyBecause the above code will not work.#2018-04-1217:58kennycat needs a form, not a symbol.#2018-04-1217:58kenny
(defmacro reg-my-thing
  [id spec other-stuff]
  `(defmethod my-multimethod id
    [_]
    (s/cat :x any? :rest ~spec))
  ;; do other stuff
  )
#2018-04-1217:59Alex Miller (Clojure team)I donā€™t think that macro is correct, but it can be fixed#2018-04-1217:59kennyIt's not - more of psuedo code.#2018-04-1217:59kennyEssentially in order to program with spec, everything needs to be a macro.#2018-04-1218:01Alex Miller (Clojure team)this is a macro already, itā€™s just not the right macro#2018-04-1218:01kennyNot sure I understand what you mean.#2018-04-1218:06kennyMy API is defined a bunch of functions that are passed a spec for the (vec (rest [:my-vec 1 "2"])). The functions all do global registration sort of thing (akin to defmethod). Each of these functions needs to register a spec for the whole vector [:my-vec 1 "2"]. I could construct that spec at the macro level based on the spec they passed in, but that'd mean my whole API needs to be defined at the macro level.#2018-04-1218:07kenny... because this doesn't work šŸ™‚
(defn reg-my-thing 
  [id spec other-stuff]
  (defmethod my-multimethod id
    [_]
    (s/cat :x any? :rest spec))
                         ^^^^
  ;; do other stuff
  )
#2018-04-1218:15Alex Miller (Clojure team)something like this works:
(require '[clojure.spec.alpha :as s])

(defmulti v first)
(defmethod v :hi [_]
  (s/cat :o #{:hi} :p #{:there}))

(s/def ::v (s/multi-spec v (fn [val tag] val)))

(s/valid? ::v [:hi :there])

(defmacro defvspec
  [op tail-spec]
  `(defmethod v ~op [_#] (s/cat :op #{~op} :rest ~tail-spec)))

(defvspec :a (s/cat :x int?))

(s/valid? ::v [:a 100])
#2018-04-1218:15Alex Miller (Clojure team)another option is to register your specs with s/def and then refer to them by their keyword name#2018-04-1218:16kennyYes. Except that requires a breaking change to the API.#2018-04-1218:16Alex Miller (Clojure team)which gets you out of caring about the form#2018-04-1218:16kennyreg-my-thing is passed a Spec in its arguments.#2018-04-1218:16Alex Miller (Clojure team)if you have the spec instance, you can also have the macro invoke s/form to get back the form#2018-04-1218:17Alex Miller (Clojure team)in that case I donā€™t know that you even need a macro#2018-04-1218:18kenny
(defn reg-my-thing 
  [id spec other-stuff]
  (defmethod my-multimethod id
    [_]
    (s/cat :x any? :rest (s/form spec)))
                         ^^^^
  ;; do other stuff
  )
?
#2018-04-1218:18kennyWon't that create a mess of the error messages?#2018-04-1218:26Alex Miller (Clojure team)
(defn defvspec2
  [op tail-spec]
  (defmethod v op [_] (eval `(s/cat :op any? :rest ~(s/form tail-spec)))))
#2018-04-1218:28Alex Miller (Clojure team)
(s/valid? ::v [:b 10]) ;; true
(s/conform ::v [:b 10]) ;; {:op :b, :rest {:y 10}}
(s/explain ::v [:b nil]) 
;; In: [1] val: nil fails spec: :user/v at: [:b :rest :y] predicate: int?
#2018-04-1218:29kennyInteresting.#2018-04-1218:29Alex Miller (Clojure team)really the same thing youā€™re doing with a macro#2018-04-1218:29kennyThis is a CLJS project so I'm not sure about the eval usage.#2018-04-1218:30Alex Miller (Clojure team)oh sure, throw that in at the end :)#2018-04-1218:32kennyShould've mentioned that in the beginning šŸ˜¬ It's essentially adding Spec to re-frame, thus the reg-* API.#2018-04-1218:32Alex Miller (Clojure team)well then, I donā€™t know :)#2018-04-1218:33Alex Miller (Clojure team)I donā€™t understand the constraints in cljs as well. There are some changes coming to spec that will help with stuff like this too but Iā€™m not sure when or how they will play out in cljs.#2018-04-1218:34kennyI'm guessing everything will need to be done at the macro level. I think the only constraint is the lack of eval.#2018-04-1223:31mvIf I have a list of lists, is it possible to write a spec that enforces that no two sublists start with the same value?#2018-04-1223:35seancorfield@mv Sure, if you can write a predicate that tests for that, you can use that predicate in a spec (or even as a spec).#2018-04-1223:35mvSo just a standard function?#2018-04-1223:36seancorfieldYup.#2018-04-1223:36mvAnd Iā€™m assuming s/valid?? Iā€™m new to spec#2018-04-1223:37seancorfieldPresumably you already have a spec for "list of lists"?#2018-04-1223:38mvNot yet, but there is a spec being applied for the individual elements with s/*#2018-04-1223:39seancorfieldOK, so when you have your spec for list of lists, then you just s/and that spec with your predicate and that's your complete spec.#2018-04-1223:39mvCool#2018-04-1223:39seancorfield(s/def ::list-list-spec (s/and (s/coll-of (s/coll-of ::sublist-element-spec)) sublists-have-unique-prefix))#2018-04-1223:39seancorfield(or something like that)#2018-04-1223:40mvNeat!#2018-04-1223:41seancorfieldBear in mind you may not be able to generate data from that spec (you might, but generation may produce sublists with identical first elements quite often so the check on the generated data might fail).#2018-04-1223:41seancorfieldIf that's important, you'll need to write a custom generator.#2018-04-1223:41seancorfieldBut if you're just getting started with spec, you may not need that. Yet šŸ™‚#2018-04-1223:49mvYea I donā€™t think I need that yet, this is more to enforce a bug wonā€™t come back#2018-04-1322:00hmaurerIs it possible to use clojure multi-specs with derived keywords (through derive) and ensure that ā€œchild specsā€ are a superset of parent specs?#2018-04-1400:16hmaurerhttps://github.com/clojure/spec.alpha#2018-04-1400:16hmaureris clojure spec still in alpha?#2018-04-1400:25seancorfieldYes @hmaurer#2018-04-1400:25seancorfieldBut lots of people are using it heavily in production.#2018-04-1411:06hmaurer@seancorfield is it still in development? it seems ike there hasnā€™t been a single commit to the source code in 6 months?#2018-04-1411:22gklijs@hmaurer apparently Rich has some changes planned. There are some things which not yet work smoothly.#2018-04-1413:46matanlooks like docs are still pre-1.9 how do I use spec in 1.9? any good 1.9 getting started? https://clojure.org/guides/spec#2018-04-1413:47matando I really need to
(:require [clojure.spec.alpha :as s])
?
#2018-04-1413:57Alex Miller (Clojure team)yes#2018-04-1413:57Alex Miller (Clojure team)where do you see pre-1.9 docs?#2018-04-1413:58Alex Miller (Clojure team)docs are at https://clojure.github.io/spec.alpha/index.html#2018-04-1414:06matanhttps://clojure.org/guides/spec#2018-04-1414:11Alex Miller (Clojure team)why do think thatā€™s pre-1.9?#2018-04-1414:11Alex Miller (Clojure team)I wrote the guide btw#2018-04-1414:16matanokay, so the namespace remains alpha, got it šŸ™‚#2018-04-1414:17matangood guide BTW! šŸ™‚#2018-04-1414:18matanbut don't need to explicitly require it anymore with clojure 1.9 do we?#2018-04-1414:18Alex Miller (Clojure team)you do#2018-04-1414:18Alex Miller (Clojure team)require just loads the code into the clojure runtime, as you need to do with any namespace#2018-04-1414:20matanright!#2018-04-1414:20matanit's not part of core that's why#2018-04-1421:18seancorfield@hmaurer That says it's very stable in its 'alpha' state! šŸ™‚#2018-04-1421:20seancorfieldI gather that some changes to make specs easier to work with programmatically are on the roadmap @gklijs but it's been great for us to use as-is in our production code. We're very happy with it. And we were using it early on in the 1.9 prerelease cycle, before it became .alpha and was moved out of core (so, yeah, there were some code changes needed along the way -- but overall Clojure doesn't require many of those over the years).#2018-04-1422:54gklijsThat's fine and even if there are changes it will probably not change the main use. However I use it to store and retrieve data with the option to use a backward compatible spec to get the full data back, and I'm a bit scared changes might break it.#2018-04-1423:45mathpunkOh, also --- if I'm incorporating generative tests into a library, is the idea that I run stest/check inside a deftest?#2018-04-1501:03Alex Miller (Clojure team)as the doc string for s/fdef says: ā€ :args A regex spec for the function arguments as they were a list to be passed to apply - in this way, a single spec can handle functions with multiple aritiesā€#2018-04-1502:37seancorfield@mathpunk :args should always be an s/cat with each argument specified.#2018-04-1502:38seancorfieldIn your case
:args (s/cat :v :simplexity.simplex/natural-vertices)
if it has one argument, v
#2018-04-1502:39mathpunkI've been meditating on what @alexmiller said and, i just realized, i think i figured "well a simple string is a 'regex' that matches very few things so, one argument is just a spec for the argument"#2018-04-1502:39mathpunknow, i think i see it#2018-04-1502:39seancorfieldAlso (:args %) in your :fn spec will be the collection of argument values, so you'll likely want (:v (:args %)) -- to match the :v in the :args spec#2018-04-1502:40seancorfieldRead the Spec guide about spec'ing functions again to see how it uses s/cat#2018-04-1520:37mathpunkWhen I run something like "(test/check `simplex {:clojure.spec.test.alpha/opts {:num-tests 1}})" it looks like it's still running 1000 tests. šŸ¤”#2018-04-1521:07mathpunkMind you, that might not be my problem --- I think I'm tucking way too much into my spec for a simplex, when I should be instead specifying the functions that act on a simplex (i.e. the ones named in the Simplex protocol). I'm a little fuzzy on how to specify functions that should be true in /every/ implementation, though...#2018-04-1521:55mathpunkhmm, I'm also a little unclear on how to run a generative test on a function from outside of the namespace it's defined#2018-04-1522:18seancorfield@mathpunk Just specify the fully-qualified name of the function to test/check.#2018-04-1608:52hkjelsI have a lot of cases in my code where I destructure the input of a function using a spec and then extract the keys using a long vector thatā€™s basically the exact same as the spec itself. Is there a way to just use a spec to destructure keys?#2018-04-1608:52hkjelssomething like {:keys ::component/params}#2018-04-1608:57hkjelsin other languages you have like a splat or explode that would give you all of the keys as vars in the current scope. I get that it would be a bad idea, but with a spec your still explicit#2018-04-1615:05borkdude@hkjels This is exactly what I also asked for recently (some weeks back)#2018-04-1615:05borkdude@seancorfield wrote a macro but I wanted it to do without writing my own let macro#2018-04-1707:33hkjelsyeah, this seems like something that should be natively supported#2018-04-1615:25bbssI wrote a macro that uses the registry to return a vector with the keys, and use that in combination with C-c C-v C-w eval and replace in the editor. Of course there is some potential erosion as you'd need to maintain it in multiple places. But at least it's explicit.#2018-04-1618:31mathpunkI'm still working out where to put specs in my project. I'm coming from test-driven development so it seems natural to put fdefs in test namespaces: define what the function should do, separate from the implementation. But then again, does that mean that code that calls the function won't have loaded those specs and will be missing out on the benefits? I'm curious how people are organizing their projects.#2018-04-1621:47bbrinckI tend to put the fdef right before the defn. That provides the most documentation benefits IMO and also anyone who loads my code is guaranteed to get the specs if they want to instrument them#2018-04-1621:48bbrinckAs you noted, if you put them in test namespaces, then clients will need to load both namespaces in order to instrument, which I think would be surprising#2018-04-1621:48bbrinckYMMV#2018-04-1618:37leongrapenthin@mathpunk use :clojure.spec.test.check/opts#2018-04-1618:37leongrapenthinFor some reason they missed to alpha it#2018-04-1618:39mathpunkAh ha! Thanks so much#2018-04-1700:10d._.bI have a few transforms that are related in my application.
{:aH "abc"} ;;phase-1
{:aH :what-x-means} ;;phase-2
{:a-h :what-x-needs-to-be-for-an-external-app} ;;phase-3
What would be cool is if I could somehow link all of these specs together, and control generation. So, an initial spec for phase 1 would refer to phase 2, and phase 2's spec would refer to phase 3.
#2018-04-1700:16d._.bmaybe this is more of a controlled generation question?#2018-04-1700:17d._.bmaybe conform/unform is what im looking for here?#2018-04-1700:20d._.bwhere a named conformer would be the transform between two phases, so I could (s/def ::phase-1->phase-2 (s/conformer ...)) (s/conform ::phase-1->phase-2 {:aH "abc"})#2018-04-1700:55bbrinck@d._.b Hm, hard to say without seeing the code, but I suspect itā€™d be easier to just write transform functions with a transformation library and use the specs to validate each transform is working#2018-04-1700:57bbrinckGenerally speaking, using a conformer to transform data can cause issues https://groups.google.com/d/msg/clojure/Tdb3ksDeVnU/LAdniiZNAgAJ#2018-04-1701:27d._.b@bbrinck yes, this makes sense to me. it is fine for me to write the transform functions and validate the transform, but for some of these specs, the difference is very small. For instance, "person": {:name "bob"} => {:name (get {"bob" :fun-bob} "bob")} => {:nick-name "fun-bob"}. It would be nice if I could link the specs as an ordering, and then choose which phase I want to generate.#2018-04-1701:28d._.bfrom an initial state or spec#2018-04-1701:38bbrinck@d._.b Yep, I can see how this would be convenient. I canā€™t speak for spec maintainers, but from what Iā€™ve read so far, they have consistently indicated they want to stay away from transformation (and leave this to other libraries - both libraries that are spec-aware or those that are spec-agnostic)#2018-04-1701:39bbrinckI can certainly see the use, but this canā€™t be done in spec today and I doubt it will be added šŸ™‚ . https://github.com/nathanmarz/specter might be a good fit here#2018-04-1702:29andy.fingerhutNot sure if it is something targeted at what you are hoping to do, but late in the Google group discussion linked above is a mention of the project spec-coerce, which the author says "instead of making conforms that do coercion, it uses the spec and a separated registry to conform the value (similar to what spec does to find the generators for a given spec)". https://github.com/wilkerlucio/spec-coerce#2018-04-1704:17Alex Miller (Clojure team)+1 spec-coerce#2018-04-1705:22ikitommi@d._.b tested spec-driven data expansion some time ago, via conforming. Not sure if thatā€™s an good idea, but hereā€™s an example: https://www.metosin.fi/blog/clojure-spec-as-a-runtime-transformation-engine/#data-macros#2018-04-1705:43Alex Miller (Clojure team)itā€™s not a good idea#2018-04-1706:06ikitommiwould it be a good idea to add support for spec coercion in the core? or support for generic walking with coercion as one application for walking.#2018-04-1711:58Alex Miller (Clojure team)Generic walking yes, coercion maybe#2018-04-1712:41ikitommiI renamed CLJ-2251 to ā€œGeneric Spec Walking for clojure.specā€. Is something like that coming? Would you be interested in a demo / patch of that?#2018-04-1712:53Alex Miller (Clojure team)Itā€™s something weā€™ve talked about but not particularly looking for a patch. #2018-04-1706:19d._.bThanks for those recommendations. #2018-04-1706:27d._.bWhen an entity like ā€œfooā€ exists in numerous, ordered forms, it feels superfluous to name intermediate specs, and edges on ā€œtypesā€. I donā€™t want that extra, unnecessary complexity in my programs, so Iā€™m looking for a way to define a ā€œfooā€ as a succession of small transformations, where the phases (minor transforms) are named loosely, abstractly. One place to define all of the successions of miniature transforms is better than enshrining stronger, named entities. The names wind up feeling forced.#2018-04-1706:29d._.b:thing/upcased-name feels like extra nonsense if itā€™s a short-lived intermediate.#2018-04-1706:33d._.bCoalescing specs into logical, ordered groupings feels valuable if only to avoid the naming of tiny-delta intermediates.#2018-04-1707:11mathpunkI think I'm getting the hang of test/check and custom generators, but I'm getting this oddball response from one of my fdef's: https://github.com/mathpunk/simplexity/blob/complexes/src/simplexity/core.clj#L70#2018-04-1707:13mathpunkNB: Every simplex is a complex, but not every complex is a simplex.#2018-04-1714:39bbrinckThe problem seems to be with your :fn spec on dim. The reason the examples work e.g. (dim (complex [])) is that they donā€™t run the :fn code#2018-04-1714:39bbrinckI deleted the :fn spec on dim and now (test/check dim)` returns with no errors#2018-04-1714:41bbrinckIt looks like the issue with the :fn is that :ret does not contain the value, it contains the conformed value e.g. not -1, but rather [:empty -1]#2018-04-1714:52bbrinckI think this will avoid the exceptions, although the definition of dim still fails the :fn test in some cases:
(s/fdef dim
  :args (s/cat :complex ::complex)
  :ret (s/or :nonempty nat-int?
  :empty #(= -1 %))
   :fn #(and (number? (-> % :ret last))
                                (= (size (-> % :args :complex))
                                (+ (-> % :ret last) 1))))
#2018-04-1714:52bbrinckFWIW, if you want to test out your fn spec outside of check try https://github.com/jeaye/orchestra#2018-04-1717:34mathpunk@bbrinck Ah ha! I forgot about conformance.... aaaagain. And orchestra looks useful, much obliged#2018-04-1707:18mathpunkWhat's really baking my noodle is, there are a few things I can think of that might just be plain broke in the way I'm writing this, but, it would be broke with MUCH smaller examples than I seem to be getting as examples of failing results#2018-04-1720:09mathpunkI'm trying to define different specs for different implementations of a protocol, which I don't know if it's even possible. This code passes my example tests, but when I try and test/check any of the functions, I just get an empty sequence: https://github.com/mathpunk/simplexity/blob/complexes/src/simplexity/simplex.clj#2018-04-1720:12mathpunkThe problem I'm trying to solve is: A simplex is a special case of a complex. It would be nice to construct a simplex by going (simplex [0 1 2]) and a general complex by going (complex [[0 1 2] [2 3] [4 5 6]])#2018-04-1720:12mathpunkThat is, construct a simplex from a collection of vertices, and a complex from a collection of (maximal) simplexes#2018-04-1720:13mathpunkI'm open to other ways of accomplishing this goal#2018-04-1720:27bbrinckI havenā€™t tried it myself, but the advice Iā€™ve heard is that protocols should be internal implementation details and you should spec wrapper functions, which become the public-facing API.#2018-04-1720:27bbrinckI could be misremembering the advice šŸ˜‰#2018-04-1720:29bbrinckIn this case, youā€™d have a spec for the wrapper function. Iā€™m not sure that itā€™d need to be different though - if complex is the general case, could the work in terms of complex? Iā€™m not sure#2018-04-1720:30bbrinckYou could certainly create a generator that creates either simplex or complex values though, and use that#2018-04-1720:38bbrinckTry wrapping dim in another function e.g. (defn dim1 [x] (dim x)), then write spec for that. test/check works for me on dim1 (poorly named, of course šŸ™‚ )#2018-04-1720:28Alex Miller (Clojure team)You canā€™t spec protocol functions#2018-04-1721:07kennyWhy is the predicate function in the below error message :clojure.spec.alpha/unknown?
(s/assert nat-int? -1)
ExceptionInfo Spec assertion failed
val: -1 fails predicate: :clojure.spec.alpha/unknown
  clojure.core/ex-info (core.clj:4739)
#2018-04-1721:09kennyI need to wrap with s/spec?
(s/assert (s/spec nat-int?) -1)
ExceptionInfo Spec assertion failed
val: -1 fails predicate: nat-int?
  clojure.core/ex-info (core.clj:4739)
That seems asymmetrical with the rest of Spec's API -- you can pass a Spec kw, predicate, or Spec obj.
#2018-04-1721:13Alex Miller (Clojure team)itā€™s a bug - there is a pending patch for it
#2018-04-1721:13Alex Miller (Clojure team)wrapping with s/spec bypasses it#2018-04-1721:14Alex Miller (Clojure team)https://dev.clojure.org/jira/browse/CLJ-2068#2018-04-1802:22jumblemuddleIs it possible for me to have a spec: s/cat regex where latter parts of the regex use predicates that depend on the conform of prior parts?#2018-04-1802:29jumblemuddleI guess the alternative would be to have a second spec to run after the first one, so it can depend on the output of the first one.#2018-04-1816:54cap10morganIf you had some spec'd code where correctness was far more important than performance, a) would it make sense to turn on instrumentation of every fn at production runtime and b) if so, how would one go about doing that?#2018-04-1817:10seancorfield@cap10morgan instrument accepts a sequence of symbols to instrument as I recall. You'd probably want to just instrument the functions in specific namespaces (your own) to avoid turning on instrumentation for all libraries functions in your app.#2018-04-1817:11seancorfield(you can "instrument everything" by calling instrument with no arguments but you wouldn't know what functions got instrumented so you'd have no idea about performance impact)#2018-04-1817:12Alex Miller (Clojure team)you would know as it returns a collection of all symbols instrumented#2018-04-1817:12seancorfieldI meant, ahead of time.#2018-04-1817:12Alex Miller (Clojure team)you do have to ensure the specs have been loaded (by requiring them) before calling instrument#2018-04-1817:13seancorfieldBut, yeah, you could potentially get a very long list of instrumented functions that you weren't expecting šŸ™‚#2018-04-1919:09xiongtxIs there an easy way to modify an existing s/keys spec? I often find myself needing to take an existing spec and update a few keys. Itā€™d be nice if I could just replace a key instead of copying everything.#2018-04-1919:50seancorfieldMaybe s/merge would work for you @xiongtx? I'm not quite sure how it handles merging over existing keys tho'#2018-04-1919:51xiongtxI managed to get around it by redefining the specs that differ#2018-04-1923:04kennyIs it possible to instrument initial function calls? i.e. I have my dev namespace where I enable instrumentation:
(ns dev.user
  (:require
    [clojure.spec.test.alpha :as st]
    [my-project.util]))

(st/instrument)
But by the time I load my dev.user namespace, I have already ran some functions:
(ns my-project.util
  (:require
    [clojure.spec.alpha :as s]))

(defn reg-event
  [id]
  ;; do side effecting stuff
  nil)

(s/fdef reg-event
        :args (s/cat :id keyword?)
        :ret nil?)

(reg-event "my-event")
The call to reg-event is not instrumented at this point so I do not get a instrumentation error that my call to reg-event is incorrect.
#2018-04-2001:40dadairYou could just (st/instrument reg-event) after itā€™s fdef?#2018-04-2003:10kennyTrue but that doesn't scale well and requires special considerations in production.#2018-04-2019:12kennyI wrote a small library that addresses the above problem -- instrumenting initial/global function calls. It takes a slightly different approach than fdef. During development it immediately wraps function calls with assertions on a function's args and ret value. Thoughts on this idea? šŸ™‚ https://github.com/Provisdom/defn-spec#2018-04-2019:35franquitoNeat! Looks pretty similar to orchestra.#2018-04-2020:15kennyAh you're right. I did not realize orchestra did more than enable :ret checking for instrumentation. This approach is still different than orchestra's defn-spec. Orchestra's defn-spec also doesn't have great IDE support.#2018-04-2020:29kenny@U3UFFB420 Added an excerpt in the README addressing this: https://github.com/Provisdom/defn-spec#how-is-this-different-from-orchestra.#2018-04-2103:00franquito@U083D6HK9! I pretty much skimmed the README at work. Now I gave it a read and see more clearly the purpose of the lib (BTW, that's a well written README). My thoughts are: I liked it. You just don't care anymore about instrument.#2018-04-2105:03kennyThank you. And yeah, I came to a similar conclusion. #2018-04-2305:34steveb8nI have a problem with a recursive spec in a cljc file. itā€™s the ::tree (and ::node) spec here https://github.com/stevebuik/Stu/blob/master/src/cljc/viz/core.cljc#2018-04-2305:35steveb8nworks fine within this project but when invoked via an installed dep in another project, this spec fails to load#2018-04-2305:35steveb8nanyone seen anything like this before?#2018-04-2305:36steveb8nIā€™ve tried using (s/spec (s/* ::node)) but no change in behaviour#2018-04-2315:15thomassilly question.. but how do I spec a string which has to have a certain value?#2018-04-2315:15tayloryou can use a set as a predicate#2018-04-2315:15thomassomething like this: (s/def :mqtt/protocol-name "MQTT") but that doesn't quite work#2018-04-2315:15thomasat least not for the generators.#2018-04-2315:15taylor(s/def ::my-spec #{"MQTT"})#2018-04-2315:16thomas:+1:#2018-04-2315:17thomasthank you, that did the trick of course!#2018-04-2317:09kkruitIs there a way to compare the :pred response from s/problem to the s/def record?#2018-04-2317:10kkruiti'm trying (=(:pred (first (s/problem issue))) :test.spec/equal?)#2018-04-2317:12kkruitwhen i debug the value of pred is the value of the s/def#2018-04-2317:12kkruitbut i'm getting false#2018-04-2317:22kkruitsomething like that#2018-04-2317:23kkruitis there a way to compare functions like that?#2018-04-2317:45kkruitoh, nevermind, it's in :via, thanks šŸ˜…#2018-04-2319:05richiardiandreaI have never noticed this, but basically the output of clojure spec does not expand nested specs:
user=> (doc my-ns.core/test-fn)
-------------------------
my-ns.core/test-fn
([n])
Spec
  args: (cat :n :test-ns)
  ret: any?
nil
Is there a way to see what the :test-ns spec actually is?
#2018-04-2319:08Alex Miller (Clojure team)you can invoke doc on spec names too#2018-04-2319:08Alex Miller (Clojure team)I would expect that to be a qualified keyword above though#2018-04-2319:09Alex Miller (Clojure team)not sure if you altered the output#2018-04-2319:09Alex Miller (Clojure team)
user=> (doc :clojure.core.specs.alpha/args+body)
-------------------------
:clojure.core.specs.alpha/args+body
Spec
  (cat :args :clojure.core.specs.alpha/arg-list :body (alt :prepost+body (cat :prepost map? :body (+ any?)) :body (* any?)))
#2018-04-2319:09Alex Miller (Clojure team)for example#2018-04-2319:12richiardiandreado I need to namespace it? I have a global :test-ns only#2018-04-2319:13Alex Miller (Clojure team)spec names are expected to be namespaced#2018-04-2319:13Alex Miller (Clojure team)maybe weā€™re talking about different things#2018-04-2319:13richiardiandreaoh right#2018-04-2319:14Alex Miller (Clojure team)spec names should be qualified, as in :foo/bar#2018-04-2319:14richiardiandreano you are right my bad#2018-04-2319:18richiardiandreaperfect thanks it works fine šŸ˜‰#2018-04-2409:36ouvasamHi, With clojure.spec, Is there a way to get all the errors with s/and instead of having only the first one ? e.g.
``
(s/explain (s/and int? even? #(> % 1000)) 9)
;; val: 9 fails predicate: even?
`` Is it possible to also have the last predicate error ? (> % 1000) Many Thanks
#2018-04-2412:12Alex Miller (Clojure team)In this case, no. Because s/and flows values through if an earlier one fails it canā€™t run a later one#2018-04-2412:51ouvasamThanks. Is there another way to get all the errors ?#2018-04-2413:44Alex Miller (Clojure team)not without breaking apart the s/and into pieces and running each individually#2018-04-2417:23xiongtxIs there a way to express the idea that a spec is any subset of some provided set, and to have generator behave accordingly?#2018-04-2417:49misha@xiongtx something like:
(s/def :foo/item #{:a :b :c})
(s/def :foo/items (s/coll-of :foo/item :kind set?))
or just
(s/coll-of #{:a :b :c} :kind set?)
#2018-04-2417:56xiongtxšŸ‘#2018-04-2609:11borkdudeGary talked about an interesting issue recently. What if you instrument all and there are loads of fdefs for core functions. This will create a lot of validation overhead. Anyone experienced this becoming an issue?#2018-04-2611:54Alex Miller (Clojure team)Currently, no core functions are specā€™ed (just macros, which are done at compile time not runtime, although admittedly those are often pretty close together in repl dev). Having too much instrumented definitely can create a noticeable effect.#2018-04-2609:46Oliver GeorgeThere's definitely overhead. By rights it shouldn't matter so much at dev time which is the common use case for instrumenting.#2018-04-2612:36gfredericksit'll matter if specs get added to clojure core fns; I can easily imagine a 10000x slowdown depending on what you're doing#2018-04-2609:54Rose MolinaGood morning, I recently started using spec and I wanted to know the reason behind why s/and conforms values and propagates them. I hadn't read the spec guide (https://clojure.org/guides/spec#_spec_ing_functions) in full before I found the first example of this in our code base and I found it hard to read.#2018-04-2611:52Alex Miller (Clojure team)conforming tells you not just whether something validated (like valid? but how it validated - indicating which alternatives are taken and how value components were parsed). Flowing conformed values allows s/and to also flow conform info out to the user from internal components (which is also done in most other conforms)#2018-04-2613:03dominicmIs it possible to "redirect" a failure. For example, if a list of numbers must be sequential, this validation can only be performed on the "whole", but the failure is on a particular index.#2018-04-2613:06Alex Miller (Clojure team)not without implementing your own spec#2018-04-2614:20dominicm@alexmiller is that api private currently?#2018-04-2614:20Alex Miller (Clojure team)itā€™s public but undocumented and subject to violent change#2018-04-2614:21Alex Miller (Clojure team)as in, I expect it to change, and possibly even be completely replaced.#2018-04-2614:21dominicmI'll wait on that, knowing it isn't impossible is good.#2018-04-2614:21Alex Miller (Clojure team)things like spec-tools are using it extensively#2018-04-2614:21dominicmIt was one frustration I'd had, e.g. for password validation data.#2018-04-2614:21dominicmYeah, I wasn't touching it because I know it will break some day.#2018-04-2615:23ikitommithe Spec protocol will be replaced? Is it possible to comment or discuss on the upcoming changes, from 3rd party library maintainer perspective?#2018-04-2615:31Alex Miller (Clojure team)no, I donā€™t know yet what they will be#2018-04-2615:31Alex Miller (Clojure team)it may be replaced, it may not be replaced, it may change, I have no idea yet#2018-04-2615:32Alex Miller (Clojure team)this is the alpha part of spec.alpha#2018-04-2615:32Alex Miller (Clojure team)my advice right now would be - donā€™t use that#2018-04-2615:34Alex Miller (Clojure team)I expect changes to the main user api of spec to be minimal or at least well thought about#2018-04-2615:38ikitommiok. thanks. there is a alpha disclaimer too on spec-tools, but just checked, it has 13k downloads, so someone is using that already. hopefully there is a way to port the features to new spec apis - or remove stuff that is implemented by the new core directly.#2018-04-2615:41Alex Miller (Clojure team)I have not been shy about telling spec-tools and others what I said above#2018-04-2615:41Alex Miller (Clojure team)spec-tools is almost certain to be broken by future spec changes#2018-04-2615:42Alex Miller (Clojure team)I donā€™t agree with most of what that library is doing#2018-04-2921:27devurandomHi! I would like to spec a map where the values have to conform to some spec, but the fields can be arbitrary. How do I do that?#2018-04-2921:30Alex Miller (Clojure team)Use s/map-of and any?#2018-04-2922:07devurandomThanks, that works! This was the first time with Spec for me and the original code I modified was using cat, which was confusing me.#2018-04-2922:14devurandom@alexmiller s/cat also matches maps, is that correct? So I need to get the ordering right to make s/map-of match first / instead of s/cat?#2018-04-2922:15devurandomI came up with this, which appears to work (and passes tests), but I'll have to read more about spec to be sure: https://github.com/devurandom/venia/commit/31d0a15a28f4a20c9355f7d452f7b2b430f2ff16#diff-c1f178afedb93b23e615114ff149605d#2018-04-3006:13jimmyhi guys, how can I instrument spec in another namespace?#2018-04-3008:40gnl(st/instrument `namespace.something/function)#2018-04-3009:13jimmy@clojurians.net thanks for you help. I have tried it and it returns empty []. If I have the fdef in the same ns, and I run (st/instrument) it works#2018-04-3009:20gnlJust tested it to make sure I'm not missing something, but it works for me. Have you made sure that the external namespace the fdef is in is :required?#2018-04-3009:37jimmyyes it does. I feel like there is something missing as well, but I havenā€™t figured it out yet, it should work like your said.#2018-04-3009:44gnl@nxqd Can you post the head of your fdef, as in (s/fdef <name> ...)?#2018-04-3009:45guyI found if your fdef doesnā€™t work correctly you canā€™t instrument it#2018-04-3009:45guyif it gives that error saying it cant be satisfied after 100 tries or something, it wonā€™t be able to be instrumented#2018-04-3009:46jimmy@guy if I put this spec in the same ns, it works.#2018-04-3009:46guyinteresting#2018-04-3009:46jimmymy purpose is to reuse the same fdef button ns for different functions#2018-04-3009:46guyare you doing it across clj/cljs?#2018-04-3009:47jimmyyes, but for now, I test in in clj repl, so itā€™s only java for now.#2018-04-3009:47guykk#2018-04-3009:47gnlAm I being blind here or is there no (defn button ....)?#2018-04-3009:47guyi think he had ;;button fn#2018-04-3009:47jimmy@clojurians.net itā€™s defined in another namespace.#2018-04-3009:48gnlah#2018-04-3009:48guydont you have to import it?#2018-04-3009:48guysome-ns/button#2018-04-3009:48gnlwell then you have to (s/fdef some-ns/button)#2018-04-3009:48guy
(s/fdef some-ns/button
        :args (s/cat :opts (s/keys :opt-un [:button/type :button/state :button/size
                                            ::class])))
#2018-04-3009:48guyyeah#2018-04-3009:48guywhat gnl said šŸ™‚#2018-04-3009:48jimmyok I see#2018-04-3009:48jimmythanks šŸ™‚#2018-04-3009:49guyDid that fix it?#2018-04-3009:49jimmyit does. Thanks guys#2018-04-3009:49guyexcellent!#2018-04-3009:49guyit was all gnl really šŸ‘#2018-04-3009:49guyAlso thats a super common mistake to make so donā€™t worry about it#2018-04-3009:50guyI had the same issue before#2018-04-3009:50guyand spent an hour or two being confused#2018-04-3009:50guyšŸ˜„#2018-04-3009:50jimmyyeah, most of the time I just define it in the same ns šŸ˜„ so I didnā€™t notice this.#2018-04-3009:50guyšŸ˜„#2018-04-3009:50jimmythanks a lot guys#2018-04-3015:06deansherThis is a question about clojure.spec.alpha as an example code base, rather than about using it. In the following code, it seems to me as though alpha.clj is working very hard to avoid multimethods. Does anyone have insight into why?
(defprotocol Specize
  (specize* [_] [_ form]))

(extend-protocol Specize
  clojure.lang.Keyword
  (specize* ([k] (specize* (reg-resolve! k)))
            ([k _] (specize* (reg-resolve! k))))

  clojure.lang.Symbol
  (specize* ([s] (specize* (reg-resolve! s)))
            ([s _] (specize* (reg-resolve! s))))

  Object
  (specize* ([o] (spec-impl ::unknown o nil nil))
            ([o form] (spec-impl form o nil nil))))

(defn- specize
  ([s] (c/or (spec? s) (specize* s)))
  ([s form] (c/or (spec? s) (specize* s form))))
#2018-04-3015:29Alex Miller (Clojure team)protocols are faster#2018-04-3015:52deansherThanks, Alex. I wondered if it was as simple as that.#2018-04-3016:35ghadihttps://clojurians.slack.com/archives/C03S1KBA2/p1525106043000733#2018-04-3016:35ghadi@deansherthompson ^#2018-04-3016:35ghadiit's not just performance.#2018-04-3016:53Alex Miller (Clojure team)in this particular case, the protocol has one method, so thatā€™s not a concern#2018-04-3017:10deansherHmm, Iā€™m thinking of it quite differently. The Specize protocol is allowing keywords, symbols, and (to a degree) objects to participate in the multiple-method Spec protocol, instead of using multi-methods. For example:
(defn conform
  "Given a spec and a value, returns :clojure.spec.alpha/invalid 
	if value does not match spec, else the (possibly destructured) value."
  [spec x]
  (conform* (specize spec) x))

(defn unform
  "Given a spec and a value created by or compliant with a call to
  'conform' with the same spec, returns a value with all conform
  destructuring undone."
  [spec x]
  (unform* (specize spec) x))
#2018-04-3016:38jimbobis there a good spec function that will return the req-unā€™ed keys for a spec defined map?#2018-04-3016:40jimbobexample. i have a record blah
{:a a :c c :e e }
and a spec def
(s/def ::blah-fact
  (s/keys ::req-un [::a ::e]) 
whats the best way to call a function to get back
#{:a :e}
?
#2018-04-3016:42ghadi@ben.borders Here's one way:
user=> (s/def ::mymap (s/keys :req-un [::boo ::bar]))
:user/mymap
user=> (-> (apply hash-map (-> ::mymap s/form rest)) :req-un)
[:user/boo :user/bar]
#2018-04-3016:42ghadi
(s/form ::mymap) == '(clojure.spec.alpha/keys :req-un [:user/boo :user/bar])
#2018-04-3016:53eoliphantHi, Iā€™m working on a system of clojure based services, that passes messages based on clojure maps around. So Im thinking I can just create a common module of specs (or at least a common as they need to be), but I have one other potential wrinkle. We basically need to support ā€˜tenantsā€™. So I might have the same attribute, that means the same thing throughout the system (s/def :account/number ; like "A123") so far so good. But for another tenant maybe :account/numbers should start with ā€œBā€.. Is a good way to handle this ?#2018-04-3016:53Alex Miller (Clojure team)note that that wonā€™t work if :req-un contains and or or @ghadi @ben.borders#2018-04-3016:59jimbobcool that works. thanks!#2018-04-3021:35ghadiHas anyone experienced... "Overspeccing"?#2018-04-3021:52seancorfield@ghadi You mean, someone trying to spec "everything"? I've seen some folks (new to spec) go down that path...#2018-04-3021:54ghadiMaybe not as an intent, but definitely as an effect#2018-04-3021:54ghadie.g. spec the wire boundary, spec post-transformation, spec the output (of whatever)#2018-04-3021:54ghadithen you end up with hundreds of specs#2018-04-3021:55Alex Miller (Clojure team)itā€™s useful to spec core data structures, boundaries, functions amenable to property testing#2018-04-3021:56Alex Miller (Clojure team)but itā€™s totally ok to not spec stuff, or use any? or ifn? etc to under-spec things that are still in process or too annoying to spec#2018-05-0110:24dominicmThis possibly stems from not knowing when to not apply specs, causing an excess of inappropriate specs. It's easy to assume spec = types, ergo spec all the things [meme]#2018-05-0110:25dominicmI personally struggle at times to know when to, and not to apply certain tools that Clojure provides, and the entire ecosystem even less. e.g. there are still regulars queries about protocols vs multimethods, until recently I reached for protocols more often than I did multi-methods, for no real reason.#2018-05-0111:15mpenetspecs "could" be used everywhere in theory, but in practice gen gets in the way (even via instrumentation)#2018-05-0111:17mpenetthen if you want to s/check everything you're in for some waiting. not really usable in practice that way either. s/check requires to be more selective#2018-05-0111:25mpenetI personally wish instrumentation would never trigger gen, or at least that we would have the option to disable it without having to dumb down the spec (ex: not having to resort to ifn? for predicates)#2018-05-0116:39seancorfieldWe tend to spec arguments to APIs, some data structures around the DB (where we can extract the lists of fields to save/update from the specs), core domain data structures, and a handful of functions that interact with those, either across module boundaries, or for certain "critical" domain functions. We try hard not to overuse them because we want to avoid brittleness.#2018-05-0117:59Drew VerleeWhats the ideal way to spec a nested structure? e.g
{:x {:y {:a "hi" :b "bye"}}}
All the examples in the docs are fairly flat data structures. Also most examples assume the key is a key and not a unique value. Which in most cases, it should be, in this case i was modeling a graph and iā€™m not sure its reasonable/easy to do it that way. In a related vein. I wonder how easy it is to do a depth first search with datalolg and datascript. Which would be another way to go about doing this.
#2018-05-0118:37Drew VerleeYea. After some more thought, its not the nesting thats the issue. Its the fact that my keys are unique in this example. Iā€™ll have to think about if i want to lose some terseness in order to label things more.#2018-05-0119:03tbaldridgegraphs are going to be hard to spec completely, you'll need a "any?" in a few places.#2018-05-0119:03tbaldridgespec will consume stack as it traverses the graph, so if you manage to create a really deep graph, or a cyclic graph, you'll have problems validating it#2018-05-0214:28trissso how do I stop stest/check from generating NaNs all the time?#2018-05-0214:29trissthe spec that fails is: (s/and number? #(<= 0 % 100))#2018-05-0214:33Alex Miller (Clojure team)do you really need number? ? or is something narrower like int? appropriate?#2018-05-0214:34Alex Miller (Clojure team)it is a known issue right now that NaNs cause issues for certain generators and something we intend to fix#2018-05-0214:35trissah ok. Iā€™ll see if int-in works out for me. thanks.#2018-05-0214:37Alex Miller (Clojure team)#(not (Double/isNaN %)) pred may be useful to filter too#2018-05-0214:37guySounds silly but does the order matter inside the s/and#2018-05-0214:37guyin the NaN example#2018-05-0214:37guywould you have to have #(not (Double/isNaN %)) second?#2018-05-0214:38guythen #(<= 0 % 100) last?#2018-05-0215:07Alex Miller (Clojure team)yes, order matters as the first spec generates, then each subsequent one filters, so the order you suggest is probably good#2018-05-0215:11guyok thanks!#2018-05-0214:44eoliphanthi I have , I guess a ā€˜phlisophicalā€™ lol question about spec. Im working on a system that uses datomic, etc. So, Iā€™d gone though and created a bunch of specs, now Iā€™m building out datomic schemas, using my specs to name my attributes, etc. So of course, some of this isnā€™t especially DRY. Iā€™m just wondering about creating my own (or using umlaut, etc) lightweight DSL that in turn genā€™s datomic schemas, specs, etc. Is this theā€™rightā€™ way or is starting from specs ā€˜betterā€™, etc. Just wondering#2018-05-0820:09kenny@eoliphant We wrote a library to do this called Spectomic https://github.com/Provisdom/spectomic and have been using it successfully for about a year now.#2018-05-0215:08Alex Miller (Clojure team)some people have done this to various degrees#2018-05-0215:08Alex Miller (Clojure team)hard for me to say if they found that to be a win in the long run or not#2018-05-0215:40robert-stuttaford@eoliphant Datomic schema is not the same as a spec. they do similar things - link behaviour to a name, but they are not the same thing (they create different behaviour). conflate them only if you are confident that in doing so, users of your new combined API understand the consequences of this clearly. semantically, they are different. i find i prefer a little extra typing, and having decoupled systems that use very well documented APIs.#2018-05-0215:41robert-stuttafordmy hobby project https://github.com/robert-stuttaford/bridge keeps them separate, and i donā€™t think it actually costs anything to do that#2018-05-0215:43robert-stuttafordso i guess the question is, are you actually Repeating Yourself? šŸ™‚#2018-05-0215:50trissSo Iā€™ve got nice s/fdefā€™s for my functions. Is it possible to use those in a test?#2018-05-0215:50trissI know
(-> (stest/enumerate-namespace 'fca.slipnet.math)
      (stest/check))
#2018-05-0215:51robert-stuttaford@triss yes: https://github.com/robert-stuttaford/bridge/blob/master/test/bridge/test/util.clj#L8-L27#2018-05-0215:51trissbut when I run tests from CIDER it tells me there a no tests defined.#2018-05-0215:51robert-stuttafordoh, thatā€™s a different question šŸ™‚#2018-05-0215:51trissšŸ™‚#2018-05-0215:55trissSo stest/check returns a :clojure.test.check.clojure-test/trial how do I tell clojure.test to look out for it?#2018-05-0216:25eoliphantyeah thatā€™s kind of the way I was leaning @robert-stuttaford#2018-05-0216:45javiHi everyone, new to the channel. Does anyone know of any articles on best practices /process to spec a pre-existing codebase? I know spec, but so far haven't used it much and I have a "not-small" cljs codebase that i want to start spec-ing, but not sure where/how to start...#2018-05-0216:46guylike this you mean? https://clojure.org/guides/spec#2018-05-0216:46guyoh sorry right larger codebases sorry no#2018-05-0216:49javithanks!!#2018-05-0216:45Alex Miller (Clojure team)start by writing specs :)#2018-05-0216:45Alex Miller (Clojure team)if you have a map that is used a lot, write specs for the attributes#2018-05-0216:46Alex Miller (Clojure team)then write an s/fdef for a function that takes that map#2018-05-0216:46Alex Miller (Clojure team)if you want to know where to start, pick the most common/stable data you have#2018-05-0216:46Alex Miller (Clojure team)work outward from there#2018-05-0216:47Alex Miller (Clojure team)or pick the thing you trust the least and add constraints around it by making specs and using stest/instrument#2018-05-0216:47Alex Miller (Clojure team)if you get an instrument failure, youā€™ve learned something (either your data does things you didnā€™t anticipate, or something is wrong)#2018-05-0216:48Alex Miller (Clojure team)if the former, fix your spec. if the latter, fix your code.#2018-05-0216:48Alex Miller (Clojure team)if you have data transformation functions, write specs for the inputs and outputs and use stest/check#2018-05-0216:48Alex Miller (Clojure team)these are ideas, not a plan#2018-05-0216:48javi> or pick the thing you trust the least and add constraints around it by making specs and using stest/instrument ha ha, good one šŸ™‚ i think i ll start there. thanks!#2018-05-0216:52robert-stuttafordone very useful practice is to spec anything before you make changes. we have code from clojure 1.6 days that we do this to, and it really helps deal with stuff like regression testing, safe refactoring, etc#2018-05-0220:18djtangoHey folks, probably a noob question but am really struggling to find an answer via search (probably because of my lack of terminology): If I define a spec using a non auto-resolved keyword (s/def :my/spec any?) how do I use it if I've required that namespace in a separate namespace? I can't get a call to (s/valid? :my/spec 1) to resolve nor does (s/valid? :: 1)#2018-05-0220:42Alex Miller (Clojure team)the first should work#2018-05-0220:43Alex Miller (Clojure team)the latter is not valid#2018-05-0220:43Alex Miller (Clojure team)not a valid keyword that is#2018-05-0220:45Alex Miller (Clojure team)
user=> (ns foo (:require [clojure.spec.alpha :as s]))
nil
foo=> (s/def :foo/thing int?)
:foo/thing
foo=> (ns bar (:require [clojure.spec.alpha :as s] foo))
nil
bar=> (s/valid? :foo/thing 10)
true
#2018-05-0222:20djtangoorz it works now. I think I must have had something else wrong in the file - thanks @alexmiller#2018-05-0222:39Alex Miller (Clojure team)Cool#2018-05-0307:56pyrLet me post my twitter question here as well šŸ™‚ > Clojure library writers, what's your position on spec asserts? Leave it to library consumers to set, leaving it to calling code to> choose whether to assert or not, or enforcing it from the library to ensure malformed payloads always throw?#2018-05-0311:45joost-diepenmaatdo you mean using spec.alpha/assert ?#2018-05-0311:48joost-diepenmaatprobably wouldnā€™t use that in a library, Iā€™d rely on instrument during dev/test. if youā€™ve got data coming in that must be validated (say user input or stuff coming in over a network) then you do an unconditional (when-not (s/valid ..) (throw ā€¦))#2018-05-0311:48joost-diepenmaatnote that assert can be turned off by changing *compile-asserts*#2018-05-0311:48joost-diepenmaatand you wouldnā€™t want that for stuf that must be checked.#2018-05-0400:52jstewWhat do I use if I just want to ensure that a fn returns a string or something?
(s/fdef ::string-fn
        :args (s/cat)
        :ret string?)
(s/explain ::string-fn #(identity "string"))
#2018-05-0400:52jstewIt wants me to add test.check as a dependency#2018-05-0401:27Alex Miller (Clojure team)I think youā€™re mixing syntax here and thatā€™s leading you down a path trying to gen anonymous functions with check#2018-05-0401:28Alex Miller (Clojure team)(s/def ::string-fn (s/fspec :args (s/cat) ret string?)) would be better for this#2018-05-0401:29Alex Miller (Clojure team)When function specs are validated they use the generator for the anonymous function to invoke the function being validated and verify the outputs. Using that generator requires test.check#2018-05-0410:38jstewAh hah. That makes sense. Thank you, Alex.#2018-05-0522:56madstapWhat does it mean to call s/fdef with a keyword instead of a symbol?#2018-05-0600:23Alex Miller (Clojure team)You shouldnā€™t do that#2018-05-0600:23Alex Miller (Clojure team)But itā€™s similar to calling s/def with an s/fspec#2018-05-0622:18gfrederickshttps://github.com/clojure/spec.alpha/blob/a65fb3aceec67d1096105cab707e6ad7e5f063af/src/main/clojure/clojure/spec/test/alpha.clj#L286 There's a thing here where spec is relying, for no clear reason, on what I'd like to say is a quasi-buggy implementation detail of test.check that I'd like to change. So I'm wondering if actually changing that is realistic, given the alphas and the whatnot. It would mean that the next release of test.check is not compatible with the current releases of spec.alpha. And it's hard to think of how to keep that from being a confusing pain to users who upgrade in the "wrong" order. šŸ˜•#2018-05-0711:45Alex Miller (Clojure team)I would greatly prefer this to change in a growth compatible way (new fn if necessary)#2018-05-0711:49gfredericksthere are enough moving parts that I hadn't thought about something like that, but I might be able to make it work; thanks#2018-05-0702:09alexandergunnarsonSo, reached what seems to me to be a bug: (gen/sample (s/gen (s/and (s/? string?) string?))) -> ([""] [""] ["4"] [""] ["Ii5"] ["fye"] ["x4oa"] ["6i"] ["2LH7cR"] ["Y"]) But it seems that this should fail just as (s/and number? nil?). [""], for instance, is not a string?.#2018-05-0711:40Alex Miller (Clojure team)s/? matches collections but conforms to a value and s/and flows conformed values so I think this is the expected behavior#2018-05-0711:43Alex Miller (Clojure team)Using s/? as a top level regex op often leads to confusing behavior - this is tricky. Usually it gets nested in s/cat or something#2018-05-0713:27alexandergunnarsonThanks! Yeah I found that it works with s/exercise so that makes sense at least#2018-05-0713:35alex-dixonShould I expect spec validation errors to be thrown if I call a function that has a spec registered with s/fdef in CLJS?#2018-05-0713:38alex-dixon
(defn foo [a b] a)
=> #'elp.components.dialogs/foo
(s/fdef foo :args (s/cat :attributes map? 
                         :children vector?))
=> elp.components.dialogs/foo
(foo 1 2)
=> 1
I have check-asserts onā€¦working with specs defined with s/def and {:pre (s/valid? ā€¦.
#2018-05-0713:38Alex Miller (Clojure team)have you instrumented it?#2018-05-0713:39Alex Miller (Clojure team)with instrument?#2018-05-0713:40alex-dixonI was confused about that as well. An example I saw used s/instrument from the spec namespaceā€¦.maybe it moved? Showing as undeclared var#2018-05-0713:40Alex Miller (Clojure team)it moved long long ago#2018-05-0713:41Alex Miller (Clojure team)itā€™s in the clojure.spec.test.alpha namespace (in clojure) and parallel in cljs#2018-05-0713:41Alex Miller (Clojure team)although I think you can just require that same ns in cljs and it will automatically load the right one#2018-05-0713:41Alex Miller (Clojure team)https://clojure.org/guides/spec may be of help#2018-05-0713:47alex-dixonThanks. Sorryā€¦lazy this morning#2018-05-0713:51alex-dixonIā€™m on cljs 1.9.946 (unfortunately). Tried googling and canā€™t discern whether spec.test.alpha is a separate library#2018-05-0713:52alex-dixonQuestion after that would be how to instrument all functions thereā€™s a fdef for#2018-05-0713:56Alex Miller (Clojure team)clojure.spec.test.alpha is a namespace and is included in clojurescript#2018-05-0713:56Alex Miller (Clojure team)(well the cljs equivalent)#2018-05-0713:58Alex Miller (Clojure team)https://cljs.github.io/api/cljs.spec.test.alpha/#instrument#2018-05-0713:59Alex Miller (Clojure team)several arities - no args = instrument all registered fdefs, or with a single symbol or with a collection of symbols#2018-05-0713:59Alex Miller (Clojure team)instrument is covered in the guide above and should work the same way in clojure and clojurescript#2018-05-0714:03alex-dixonThanks @alexmiller#2018-05-0714:04Alex Miller (Clojure team)oh, one caveat is that you will likely need to include org.clojure/test.check 0.9.0 as a dep#2018-05-0714:06alex-dixonAh hah. Thank you šŸ™‚#2018-05-0714:07Alex Miller (Clojure team)
ClojureScript 1.10.238
cljs.user=> (defn foo [a b] a)
#'cljs.user/foo
cljs.user=> (require '[clojure.spec.test.alpha :as stest])

cljs.user=> (require '[clojure.spec.alpha :as s])

cljs.user=> (s/fdef foo :args (s/cat :attributes map?
                         :children vector?))
cljs.user/foo
cljs.user=> (stest/instrument `foo)
[cljs.user/foo]
cljs.user=> (foo 1 2)
#error {:message "Call to #'cljs.user/foo did not conform to spec:\nIn: [0] val: 1 fails at: [:args :attributes] predicate: map?\n:cljs.spec.alpha/spec  #object[cljs.spec.alpha.t_cljs$spec$alpha1385]\n:cljs.spec.alpha/value  (1 2)\n:cljs.spec.alpha/args  (1 2)\n:cljs.spec.alpha/failure  :instrument\n", :data #:cljs.spec.alpha{:problems [{:path [:args :attributes], :pred cljs.core/map?, :val 1, :via [], :in [0]}], :spec #object[cljs.spec.alpha.t_cljs$spec$alpha1385], :value (1 2), :args (1 2), :failure :instrument}}
Error: Call to #'cljs.user/foo did not conform to spec:
In: [0] val: 1 fails at: [:args :attributes] predicate: map?
:cljs.spec.alpha/spec  #object[cljs.spec.alpha.t_cljs$spec$alpha1385]
:cljs.spec.alpha/value  (1 2)
:cljs.spec.alpha/args  (1 2)
:cljs.spec.alpha/failure  :instrument
#2018-05-0714:17alex-dixonThanks @alexmiller. Dev setup for cljs with spec is now:
(defn dev-setup []
  (when config/debug?
    (stest/instrument)
    (s/check-asserts true)
    (set! s/*explain-out* expound/printer)
    (println "dev")))
#2018-05-0714:43pfeodrippeHi, guys, we'are having problems spec'ing protocols, we're implementing the protocols with records, so instead of wrap manually the protocol methods with functions, we're defining a defrecord* macro of the kind
defmacro defrecord*
  [name fields pname & opts+body]
  (let [[{:keys [method-parser],
          :or   {method-parser rest-symbol}}
         body]
        (parse-opts opts+body)]
    `(do (defrecord ~name
           ~fields
           ~pname
           
#2018-05-0714:44pfeodrippeIs it a good approach for now?#2018-05-0715:01Alex Miller (Clojure team)I think itā€™s a bad idea to wrap protocol functions (as you then destroy many of the benefits of protocol functions)#2018-05-0716:45bronsawhy? that's the advice I've always seen given (even before spec was a thing) : use protocol methods for polymorphism and a wrapping function for validation + varargs + else#2018-05-0716:45bronsaunless you're talking about a different "wrapping"#2018-05-0716:49Alex Miller (Clojure team)I wouldnā€™t wrap it purely to use spec#2018-05-0716:50Alex Miller (Clojure team)there are some good reasons to wrap a call to a protocol but you donā€™t necessarily need one#2018-05-0715:02Alex Miller (Clojure team)rather, I would recommend just not specā€™ing protocol functions#2018-05-0716:35pfeodrippeOk @alexmiller, thanks#2018-05-0717:39dominicm@alexmiller I'm curious, why not spec protocol functions?#2018-05-0720:09hiredmanhttps://dev.clojure.org/jira/browse/CLJ-1941?focusedCommentId=43063&amp;page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-43063#2018-05-0718:10Alex Miller (Clojure team)you canā€™t#2018-05-0718:11Alex Miller (Clojure team)or rather you canā€™t for the purposes of instrument#2018-05-0718:11Alex Miller (Clojure team)I suppose you could spec them regardless for doc (and maybe check)? I havenā€™t tried it.#2018-05-0721:18dominicm@alexmiller I thought you were suggesting it would be a bad idea even if it could be done. I wasn't missing any reason it would be a bad idea šŸ˜Š#2018-05-0919:03plinsany suggestions where to find (libs maybe?) specs for strings contaning datetime in the ISO 8601 format#2018-05-0919:13ghadiuse java.time @plins#2018-05-0919:14plinsnot sure if i expressed my self properly, but id like to validate if a strings conforms to the iso, so (s/valid? ::iso-spec "2018-05-06T13:14Z") shoud evaluate to true#2018-05-0919:14ghadi#(try (java.time.OffsetDateTime/parse %) (catch java.time.DateTimeParseException e false))#2018-05-0919:15plinsok! thanks#2018-05-0919:15ghadii wouldn't pull in any of the clj date time libraries, they all have serious defects compared to java.time#2018-05-0921:07seancorfield@ghadi That's a bit of a sweeping statement. Could you provide a bit more specificity?#2018-05-0921:08seancorfield(we are using clojure.java-time which is a thin wrapper around Java Time that automates the chains of conversions between (Java) types)#2018-05-0921:09seancorfieldThe clj-time readme already tells people not to use it if they're on Java 8 or later -- and recommends Java Time instead, optionally using clojure.java-time as a wrapper.#2018-05-0921:11ghadiI wouldn't even use a wrapper#2018-05-0921:12seancorfieldFair enough. I find Java Time's raw classes to be a bit ugly if you're doing anything more than basic stuff. I find clojure.java-time's wrapper much more pleasant for more complex date/time arithmetic/manipulation.#2018-05-0921:21ghadiclojure.java-time pulls in clj-tuple and potemkin, yuck#2018-05-0921:21ghadi"white gloves" for interop; mostly field accessors#2018-05-0921:23ghadiso much error-prone wrapper code... looking under the covers#2018-05-0921:23ghadisorry I stand by my statement#2018-05-0921:24ghadishow me a clojure time library and I'll show you a simpler way to do it with java.time#2018-05-0921:24seancorfieldI don't see it pulling in potemkin. clj-tuple has no dependencies.#2018-05-0921:25seancorfieldMy teammate is also a fan of direct Java interop rather than wrappers, but some interop code can definitely be made cleaner with wrappers (in this particular context, as we've seen in code reviews during pull requests).#2018-05-0921:26mgIt doesnā€™t pull in potemkin, it copies the relevant code to java-time.potemkin.*#2018-05-0921:26seancorfieldYeah, and I'll take that little bit of potemkin code but I agree that I wouldn't want the whole thing pulled in...#2018-05-0921:27ghadithe dep issue is minor at best#2018-05-0921:27ghadithese libraries add nothing to your understanding of the primitives, and introduce new names for everything#2018-05-0921:27ghadinames were very carefully chosen in java.time#2018-05-1000:53gfrederickswhat are the issues with clj-time? just the parallel "adds no value over joda"?#2018-05-1000:53gfredericksI've been using it for years and haven't noticed anything#2018-05-1002:15ghadijoda itself had serious design issues that the authors rectified as it became java.time#2018-05-1002:15ghadithere are a bunch of articles about the api differences of joda vs java.time#2018-05-1002:29gfrederickscoolthx#2018-05-1002:31seancorfield@U0GN0S72R FWIW, the readme of clj-time points to Joda Time's recommendation to migrate to Java Time and that links to several articles I think.#2018-05-1002:31gfredericks:+1:#2018-05-0921:12seancorfield(esp. if you're unfortunately enough to have to convert back and forth with java.util.Date and/or any of the SQL date/time types!)#2018-05-1010:55trissis there a quick way to flush the slipnet library thing?
#2018-05-1017:07devnWhat's the rationale for having a spec registry?#2018-05-1017:18Alex Miller (Clojure team)https://clojure.org/about/spec#2018-05-1017:18devnThe Data spec registration and Function spec registration sections in https://clojure.org/about/spec get at it#2018-05-1017:18devnguess I was wondering if you'd add any additional color#2018-05-1017:19Alex Miller (Clojure team)not a lot to add beyond that#2018-05-1017:21Alex Miller (Clojure team)there was a desire to not keep shoving stuff into var meta#2018-05-1017:21devn@alexmiller I wondered "Vars themselves are a kind of registry, no?"#2018-05-1017:21Alex Miller (Clojure team)or vars#2018-05-1017:21devngotcha#2018-05-1017:22Alex Miller (Clojure team)with good names, we are not limited to a single ā€œvarā€ registry#2018-05-1017:22Alex Miller (Clojure team)if you use vars, you need namespaces to hold them#2018-05-1017:23Alex Miller (Clojure team)with a separate registry, itā€™s perfectly ok to make a spec named :a.b.c.d.e.f.g/foo without having a related namespace#2018-05-1017:24Alex Miller (Clojure team)Rich has some future ideas on managing names and aliases independently from namespaces#2018-05-1017:36devnthis pleases me greatly#2018-05-1017:37devnwe have a spec aliases macro for calling alias + create-ns that we use, and while it works, it doesn't have good mouth feel#2018-05-1017:38Alex Miller (Clojure team)yeah, thatā€™s a hack. we have aims to do better.#2018-05-1023:05Oliver GeorgeThis hurts most in clojurescript. Do you think it will be supported in the new approach?#2018-05-1100:23Alex Miller (Clojure team)Probably but I donā€™t know what it is yet :)#2018-05-1102:52Oliver GeorgeCan't say fairer than that. I look forward to specs becoming less verbose and quicker to write.#2018-05-1017:38devnthere's also the potential for the "oops, someone clobbered a spec name and didn't know about it" situation#2018-05-1119:12cap10morganAnyone know why the default generator for (s/map-of keyword? string?) would take 10+ seconds to generate 1,000 samples?#2018-05-1119:15ghadiit's generating large maps. set (s/map-of .... :gen-max 2) to constrain sizes#2018-05-1119:16cap10morganAh, cool. I hadn't looked for that option in the spec fn itself. Thanks @ghadi.#2018-05-1119:16ghadi(doc s/every) will show some others#2018-05-1122:28ambroiseI have this spec
(s/def ::foo (s/keys :req-un [::zipcode
                              ::state
                              ::bar
                              ::baz]
And Iā€™d like to force zipcode and state to be consistent (i have a zipcode->state function so if they are both present, they need to match) How can I write my ::zipcode and ::state specs?
#2018-05-1122:38Alex Miller (Clojure team)
(s/def ::foo' (s/and ::foo (fn [{:keys [state zipcode]}] (= state (zipcode->state zipcode)))
#2018-05-1122:39Alex Miller (Clojure team)wrap your ::foo in another spec that combines it with a custom predicate to ensure the zip code constraint#2018-05-1122:39ambroiseawesome, thanks!#2018-05-1212:25dottedmagI have a spec:
(s/def ::statement (s/cat ::header ::header
                          ::delimiter ::delimiter
                          ::txs-header ::txs-header
                          ::txs ::txs))
-- is it how it is supposed to look like, or am I missing some shorthand?
#2018-05-1212:35dottedmag::header, ::delimiter etc are declared using s/cat themselves, so I suppose s/tuple can't be used here.#2018-05-1212:37dottedmagMaybe there is a simpler way to spec this data format? The input is a CSV file parsed into a vector of vectors. Header is a number of lines, more-or-less fixed, but some are optional and may be omitted. Delimiter is a single fixed line. Txs headers is another set of more-or-less fixed lines, and txs is the rest of the document.#2018-05-1213:16Alex Miller (Clojure team)s/tuple might be better#2018-05-1213:17Alex Miller (Clojure team)For vector of fixed position fields#2018-05-1213:18dottedmagToo bad they are mostly, but not completely, fixed. Banks are, well, banks.#2018-05-1213:22ghadiI would turn the vec of vecs into a vec of maps, then spec with s/keys#2018-05-1213:29dottedmagParsing individual lines is not so bad ā€” they can be handled by s/tuple. Parsing the sequence of lines is worse.#2018-05-1213:29dottedmagI'll leave it with s/cat for now.#2018-05-1318:16andy.fingerhutI saw this message by you, Alex, on a Reddit discussion: https://www.reddit.com/r/Clojure/comments/8i3hh0/what_is_the_future_of_clojure_in_the_industry/dysixcn/#2018-05-1318:17andy.fingerhutWould adding the links you gave for Cognitect blog articles on spec, and the videos, to one or both of the spec rationale and spec guide articles be of interest?#2018-05-1318:24Alex Miller (Clojure team)Nah, most of it is also in the guide#2018-05-1318:24andy.fingerhutI have read about some operation in spec doing 'sampling' in its validation/checking of a value against the spec, for better performance (but also, obviously, less precise checking). Is there something in the spec documentation I can look for to remind me where this occurs?#2018-05-1318:25Alex Miller (Clojure team)s/every and s/every-kv#2018-05-1318:30andy.fingerhutThanks. I see those in the spec guide, and good to see the doc strings for s/every and s/every-kv mention that partial checking property.#2018-05-1405:46Oliver GeorgeBit of a thought bubble but how about using spec's generative features to do "type checking" like behaviour in the IDE: https://gist.github.com/olivergeorge/584b54fe0b1d4c6ce3c7a44ee8c29095#2018-05-1406:12gklijsShould be doable at least with cursive I think, it already has a check for having the correct arity.#2018-05-1413:46jmayaalvwhatā€™s the best way to get the keys descriptions for a s/keys :req) form? calling (s/describe) returns a (keys) but not sure how can it be used.#2018-05-1414:09jmayaalvgot it. managed to do what i wanted this way:
(s/def :clecto/person (s/keys :req [:clecto.person/first-name :clecto.person/last-name]))
(apply hash-map (rest (s/form :clecto/person)))
user> {:req [:clecto.person/first-name :clecto.person/last-name]}
#2018-05-1423:08jimbobHow can i check against a collection that only certain values of maps in the collection are unique? i.e. that the collection of maps is valid iff all maps in the collection have distinct values for at least one of two fields. ex:
[{:unique-field "abc" :a 1 {unique-field2 "abc2"} {:unique-field "abc" :a 3 {unique-field2 "abc2"}]
does not pass validation but
[{:unique-field "abc1" :a 1 {unique-field2 "abc2"} {:unique-field "abc" :a 3 {unique-field2 "abc2"}]
does
#2018-05-1504:21nenadalmyour datastructures are not valid (parenthesis do not match)#2018-05-1509:33guyI think you would write a spec that for those values would have to be unique#2018-05-1509:34guy(s/def :a/unique-field #{"unique 1" "unique2"})#2018-05-1509:34guyso then the spec for that map would include that field and that spec above and would only allow those values#2018-05-1509:35guyOr do you mean the maps are only valid when, regardless of the values, they have to all be unique?#2018-05-1515:43jimbobthe first one yes.#2018-05-1515:55jimbobso mostly just curious if there are ways to spec collections of things, but spec them in terms of the collection, not in terms of the individual items in the collection (so whether or not only certain fields for the items are unique, or that summing fields for all items is within a certain range, etc)#2018-05-1516:03ghadicustom predicates with s/and is the common mechanism for doing this#2018-05-1520:24lilactownso I'm trying to do some work where I validate a JSON blob against a swagger spec. I'm thinking about trying to translate the swagger spec into clojure.spec and do it that way#2018-05-1520:24lilactownbasically the opposite of what spec-tools does with it's swagger stuff#2018-05-1520:25lilactownhowever, it seems like there's an assumption that clojure.spec's are registered globally with a name and everything. I would be generating these specs dynamically, on the fly#2018-05-1520:25lilactownis this a good idea?#2018-05-1520:25hiredmanany predicate is a spec#2018-05-1520:26hiredmanso if you have a function that given a swagger spec and some data returns true or false, then you just partial that with the swagger spec and you have a spec#2018-05-1520:26hiredmanit isn't a very nice spec, it won't generate and you won't get vary granular error messages, but it is a spec#2018-05-1520:28lilactownokay. I was thinking of trying to use clojure.spec to do some of the heavy lifting around e.g. checking whether a map conforms to a set of expected values#2018-05-1520:29lilactownso like
x-allOf: [
{
type: "object",
properties: {
personId: {
type: "integer",
format: "int64"
},
familyMemberId: {
type: "integer",
format: "int64"
}
},
required: [
"personId",
"familyMemberId"
]
},
gets turned into
(s/def ::personId integer?)
(s/def ::familyMemberId integer?)
(s/def ::my-obj (s/keys :req-un [::personId ::familyMemberId]))
#2018-05-1520:30lilactownbut obvi this is made more difficult by the fact that this would need to be done for arbitrary number of swagger specs at runtime, so I would have to make sure I'm not clobbering things in the registry...#2018-05-1520:32guyout of interest why couldnā€™t you model the data in spec? would it really be dynamic?#2018-05-1520:33guyIā€™m probably using the wrong definition of dynamic, but i thought you would know what sort of data you would be getting? so you could potentially spec out the shape of the data?#2018-05-1520:36lilactownI'm writing some code that you give it a URL to a swagger.json, a URL path, example request and an example response#2018-05-1520:36lilactownand it validates that the URL path, request and response are valid based on the swagger.json#2018-05-1520:36guyohhh i see#2018-05-1520:37guyso you can give it basically anything, and then you use the swagger to create a spec#2018-05-1520:37lilactownthat's the idea. not sure it's a good one#2018-05-1520:37guyit sounds pretty cool šŸ™‚#2018-05-1520:37guyi see ur worries about the registry now#2018-05-1520:37guyi guess you could test it by creating multiple unique specs and seeing how many you could create#2018-05-1520:40lilactownthe whole thing seems gross. I wish you didn't have to register them šŸ™ƒ#2018-05-1520:45seancorfield@lilactown Would it be easier/better to have something that generated spec source code from a swagger URL? After all, you only want to generate the spec once per API end point, but you'll want to validate data against that spec repeatedly...#2018-05-1520:49hiredmanit seems like it would be a better idea to use some swagger validator to validate the swagger, for a swagger validation service#2018-05-1520:52hiredmanI think the direction you see in spec-utils (spec -> swagger) makes sense because the idea is, you can consolidate different ways of specifying data (database schemas, swagger, json schema, etc) as spec specs, and then generate (or using spec to make sure you data matches the contract) those other things from spec as needed. But it doesn't seem like turning swagger in to specs, if all you care about is validating arbitrary swagger specs against arbitrary data, makes a lot of sense#2018-05-1520:59lilactownyeah I guess I'm looking for a swagger validator, and I decided to try and build my own using clojure.spec? and that sounds like it's not a good idea#2018-05-1521:02lilactownI'm actually struggling to find such a thing that exists, which is why I'm trying to build one in the first place#2018-05-1521:03guyIf you are doing it for fun, why not give it ago and see how it goes#2018-05-1521:03guyYouā€™ll defo learn something#2018-05-1521:03lilactowneh it's not exactly for fun šŸ˜…#2018-05-1521:04lilactownit happens to be fun but also, sprint ends in a week and a half#2018-05-1521:04guyah well thats a different matter, if ur doing it for work, i might try and find a different solution#2018-05-1521:11hiredmangoogling java swagger validator and jvm swagger validator both seem to turn up good leads#2018-05-1521:26ikitommiHere's one implementation: https://github.com/metosin/ring-swagger/blob/master/src/ring/swagger/validator.clj#2018-05-1521:58lilactown@U055NJ5CC scjsv is exactly what i'm looking for! šŸ˜„ thanks!#2018-05-1609:54troglotitusing (s/def ::my-spec ..) - is not a hard requirement. Everytime your keys are not about namespaces in your application - i think itā€™s better to qualify by another namespace like (s/def :swagger.employee-schema/employee ..)#2018-05-1802:28vladis it possible to evaluate spec description?#2018-05-1802:28vlad
(def big-integer 1000)
(s/def ::small-integer {:spec (fn [n] (n < big-integer)) :description (str "Smaller than " big-integer)})
#2018-05-1802:28vlad
(s/explain-data ::small-integer 5000)
#2018-05-1802:28vladproduces#2018-05-1802:29vlad
#:clojure.spec.alpha{:problems [{:path [],
                                 :pred {:spec (fn [n] (n < big-integer)),
                                        :description (str "Smaller than " big-integer)},
                                 :val 5000,
                                 :via [:xxx.spec/small-integer],
                                 :in []}],
                     :spec :xxx.spec/small-integer,
                     :value 5000}
#2018-05-1802:29vladI woud expect Smaller than 1000 in the description instead#2018-05-1802:43vladthe following one works, but I want to see this in the explanation#2018-05-1802:43vlad
(-> (s/form ::small-integer) :description eval)
#2018-05-1803:02Alex Miller (Clojure team)this is not supported#2018-05-1805:27mpenetIt's easy to hack a (simple) meta registry to do that without abusing the internals of clj.spec.#2018-05-1805:28mpenetI did that and a few other horrors here if you re looking for inspiration https://github.com/mpenet/spex#2018-05-1805:28mpenetLook for spex/with-doc in the readme#2018-05-1807:52mpenetgist of it:
(-> (s/def ::foo any?)
    (spex/with-doc "That's a foo"))

(spex/doc ::foo) => "That's a foo"
#2018-05-1813:55sundarjthere's a coll-of but no corresponding one-of; i can go from a scalar spec to a collection spec, but i don't think i can go from a collection spec to an element spec. is that intentional? i'm wondering because i'm currently speccing a text editor, and it's going to have a collection of working documents as well an active document, which would naturally be one of the working documents#2018-05-1813:59mpenetseems like you just need a set#2018-05-1813:59mpenetor I am misunderstanding#2018-05-1814:01sundarji presently have this:
(s/def :world.sometimes.nota.editor/working-notes
  (s/coll-of :world.sometimes.nota.app/note
             :kind vector?
             :distinct true))

(s/def :world.sometimes.nota.editor/active-note
  :world.sometimes.nota.app/note)

(s/def :world.sometimes.nota.app/editor
  (s/keys :req [:world.sometimes.nota.editor/working-notes
                :world.sometimes.nota.editor/active-note]))
#2018-05-1814:01sundarjbut it feels wrong because active-note isn't just a random note, it's only ever going to be one of the working notes#2018-05-1814:05Alex Miller (Clojure team)this would be easier if the working-notes was a set (not sure why it isnā€™t) but you could s/and an additional check into editor that working-notes contains active-note#2018-05-1814:07sundarjit's not a set because the active-note is going to be pulled from the working-notes, and sets would be the wrong data structure for that, right? sets are just about membership, as far as i know#2018-05-1814:08sundarji hadn't thought about putting an s/and on editor, that makes a lot of sense. thanks šŸ™‚#2018-05-1814:17Alex Miller (Clojure team)youā€™re declaring in working-notes that its elements are distinct. thatā€™s the same guarantee sets give you but they can check membership in (near) constant time rather than via a linear search#2018-05-1814:20sundarjyeah, i know. i had the :kind as set? before, but then i realised you couldn't pull arbitrary members out of a set; there's going to be a UI that lists all the working notes, and then you click on one to open it. i think a vector is a better choice given that use-case. i suppose i could have a vector and a set, but then i'd have to keep them in sync#2018-05-1814:22Alex Miller (Clojure team)what do you mean by ā€œyou couldnā€™t pull arbitrary members out of a setā€ ? it seems like you can do exactly that#2018-05-1814:22Alex Miller (Clojure team)do you mean you need to retain ordering?#2018-05-1814:29sundarji meant that to pull an item out of a set you need the item itself; i thought you couldn't iterate over a set, but i realise i was mistaken. in any case, i think i do need to retain ordering#2018-05-1814:47Alex Miller (Clojure team)and thatā€™s a good reason to use a vector, not trying to necessarily change your mind, just probing#2018-05-1814:54sundarji understand; i appreciate the questions. it did make me more fully analyse why i wanted it to be a vector, so thanks šŸ™‚#2018-05-2015:28karol.adamiecin clojurescript/javascript when i get json on the wire with UUID as a string, how should i spec that? uuid? passes even with empty string, (but has a correct genereator). I want the spec to be used as a tool for dev time to quickly catch misbehaving APIā€™s. Also want to generate sample data for unit tests. (ie javascript harness reads a file generated by simple clojurescript spec gen and runs a suite of tests for each generated input. i could swap / regenerate entry fixtures as i see fitā€¦)#2018-05-2105:41nenadalmany predicate can be used to spec value: https://clojure.org/guides/spec#_predicates#2018-05-2105:42nenadalmYou can write custom generator if existing ones are not good enough for you: https://clojure.org/guides/spec#_custom_generators#2018-05-2213:07troglotitHey! I want to spec for five characters long numbers: I could use regexes #(re-matches #"\d{5}" %), but I as far as I can understand that could be done in spec. I looked up a bit, but what I found is only s/* s/+ s/? that kind of operators. So, is there that kind of spec? I know about s/cat - but that seem too overkill for when I just want to repeat my regex#2018-05-2213:09ghadiwhile they may work for characters of sequences, I wouldn't use s/* s/+ etc. for that#2018-05-2213:10ghadiif you're parsing strings, a normal regex is more appropriate#2018-05-2213:10troglotitOh, I found (s/every :count) - seems like what I wanted#2018-05-2213:11ghadilike i said, it's a bad idea#2018-05-2213:12ghadithe regex predicate you have up there is the better approach#2018-05-2213:13troglotitI want to have more composable regexes and spec seem suited for that, compared to regular regexes#2018-05-2213:16Alex Miller (Clojure team)spec is not designed to be a good string regex parser and is unlikely to perform or work well for that use case as actual regex or a parser#2018-05-2213:17Alex Miller (Clojure team)you might find something like https://github.com/cgrand/regex interesting#2018-05-2213:18Alex Miller (Clojure team)I want to say there is something else like this out there for composable regexes too#2018-05-2214:44bbrinck@troglotit The comment thread on this reddit post goes into some of the issues you may run into when using spec for parsing strings https://www.reddit.com/r/Clojure/comments/7vwpu4/parsing_with_clojurespec/#2018-05-2306:59flowthingI have something like this:
(spec/def ::groups
    (spec/every-kv ::id ::group))

  (spec/def ::students
    (spec/every-kv ::group/id (spec/every-kv ::id ::student)))
  
  (spec/def ::root
    (spec/keys :req [::groups ::students]))
Is there a way for me to specify that when I do (gen/generate (spec/gen ::root)) that every group ID for every student has an equivalent group ID under the ::groups key?
#2018-05-2307:00flowthingIn other words, I'd like every group ID in ::students to have a corresponding entry in ::groups. I know how to spec that, but I'm wondering what the best way to write a generator like that is.#2018-05-2307:01flowthingI could always just post-process the generated map, but for more complicated maps, that becomes a bit difficult.#2018-05-2307:26andy.fingerhutI've not used this before, but have heard that test.check has something called bind that lets you generate one value randomly, e.g. in your case perhaps a set of group IDs, and then use that set to generate other things, e.g. your ::groups map and also the ::students map, which could be generated to have exactly the same set of group IDs in both.#2018-05-2307:27andy.fingerhutNot sure if the test.check examples of bind on this doc page are enough to get you going or not: https://clojure.github.io/test.check/generator-examples.html#2018-05-2310:36gfredericks@flowthing you can generate the groups first, and then (using bind) generate students where their group-id is from (gen/elements (map :id groups))#2018-05-2310:47flowthingMany thanks for the tips! I'll give gen/bind a go.#2018-05-2310:47gfredericks@flowthing gen/let can do the same thing, and you may find it more intuitive#2018-05-2310:48flowthingYeah, I've tried gen/let actually, but I haven't yet come up with a clean solution. Nonetheless, it's good to know I'm heading in the right direction.#2018-05-2315:18lwhortonhey peoples, iā€™m not really sure the term/noun for what iā€™m trying to do with spec, so itā€™s hard to googleā€¦ I want to define a spec where an id field thatā€™s present in two places is guaranteed to be the same:
{"id" {:id "id" :other-stuff true}}
#2018-05-2315:18lwhortonIā€™m assuming I have to write a custom generator to enforce this?#2018-05-2315:24guyI think you would have to have a predicate that for that map, it checks that key and the :id value is the same. And i believe yes, you would have to have a custom generator for it too.#2018-05-2315:24guyLike i donā€™t know how you would do it with s/keys for example#2018-05-2315:25guybut saying that, there might be another way#2018-05-2315:25Alex Miller (Clojure team)yes and yes#2018-05-2315:26lwhortonyea iā€™m not sure how to handle it either, i might just scrap this gen test entirely.. like how do you s/map-of ::id ::some-spec with also ::some-spec (s/keys :req [::id]), then write a generator that builds out two(?) specs. maybe i have to merge the map-of?#2018-05-2315:26guyYou could probably use this. https://clojure.github.io/spec.alpha/clojure.spec.alpha-api.html#clojure.spec.alpha/spec To create a spec for it#2018-05-2315:27guy
(s/def ::some-spec (s/spec #{200 202 400}
                        :gen (fn [] (gen/return (gen/generate (s/gen #{200 400 500}))))))
like something like this maybe but catered for making your map
#2018-05-2315:27guywhere u can supply a spec or a predicate function which would check both id key and value#2018-05-2315:28lwhortonah, thatā€™s interesting#2018-05-2315:28guyyeah let me find an example soz#2018-05-2316:12rickmoynihanSo Iā€™ve written some specs in an existing project with a clojure.test test suite. Does anyone have any recommendations on incorporating generative tests using clojure.spec.test.alpha/check with clojure.test? Obviously I could do:
(t/is (:result (first (clojure.spec.test.alpha/check `foo))
But that will mask the errors.
#2018-05-2316:13rickmoynihanA while back I had a similar issue and tried hooking into the clojure.test assert macros but it was never very satisfactory#2018-05-2316:14rickmoynihanis there a lein test runner for fdefs?#2018-05-2316:15rickmoynihanactually just remembered Iā€™ve been here beforeā€¦ and have already implemented a lein alias in this project for running specs using instrument etcā€¦#2018-05-2316:19lwhortoni wrote a little helper for your first problem..
(defn passes?
  "A light helper function to evaluate a generative/property test so that
  clojure.test spits out the failing result (if any) for easier debugging."
  [spec]
  (let [res (spec)]
    (t/is (nil? (:shrunk res)))
    (t/is (true? (:result res)))))
#2018-05-2316:20lwhortoniā€™m not sure if thatā€™s too implementation-specific with checking shrunk, but it works for me and i didnt have to dig around docs for a long time#2018-05-2316:22rickmoynihan@lwhorton what are the arguments?#2018-05-2316:23rickmoynihanobviously itā€™s a functionā€¦ but I guess I should ask how do you use it?#2018-05-2316:23lwhortonah, itā€™s just like you would use testing or is/are:
(defspec my-spec my-prop-test)

(deftest some-test
  (testing "some generative test should pass"
    (passes? my-spec))))
#2018-05-2316:24lwhortonerm, rather ā˜ļø#2018-05-2316:25rickmoynihanwhat is defspec?#2018-05-2316:25lwhorton[clojure.test.check.clojure-test :refer-macros [defspec]]#2018-05-2316:26rickmoynihanahh ok ā€” thatā€™s not strictly spec though is it?#2018-05-2316:27rickmoynihanIsnā€™t that a property test defined with test.check? Rather than through s/fdef etc#2018-05-2316:29rickmoynihanI appreciate spec builds on test-check and that theyā€™re compatible to some degree; but will that pick up an fdef :fn spec etc#2018-05-2316:29lwhortonyea thereā€™s a few moving pieces, the spec itself is defined using clojure.test.check.properties and a generator that uses spec/gen#2018-05-2316:30lwhortonthis was pretty helpful for me: https://github.com/clojure/test.check#examples#2018-05-2316:30lwhortonparticularly:
(defspec first-element-is-min-after-sorting ;; the name of the test
         100 ;; the number of iterations for test.check to test
         (prop/for-all [v (gen/not-empty (gen/vector gen/int))]
           (= (apply min v)
              (first (sort v)))))
#2018-05-2316:31rickmoynihanyes Iā€™ve seen that before; but Iā€™m not sure itā€™s what I need ā€” e.g. if you had a spec defined like this from the clojure docs:
(s/fdef ranged-rand
  :args (s/and (s/cat :start int? :end int?)
               #(< (:start %) (:end %)))
  :ret int?
  :fn (s/and #(>= (:ret %) (-> % :args :start))
             #(< (:ret %) (-> % :args :end))))
#2018-05-2316:32lwhortonooph, iā€™m not sure man sorry šŸ˜• ā€¦ im just getting into the gen testing myself#2018-05-2316:33rickmoynihanthat spec is already defed so to speak; as itā€™s metadata is registered already with the system.#2018-05-2316:33rickmoynihan@lwhorton: šŸ™‚ no worries; itā€™s subtle stuff#2018-05-2316:34rickmoynihanI have used defspec beforeā€¦ and tbh it seems like itā€™s a better way to integrate with clojure.test right now#2018-05-2316:36rickmoynihan@lwhorton in the interests of sharing I have something like this that works as a different entry point i.e. run via a lein alias:#2018-05-2316:38rickmoynihanit uses st/instrument to enable all the fdef specs and run them through st/check#2018-05-2516:18kvltHey all, I'm seeing something I didn't expect. Could anyone explain this?
(distinct (remove coll?(gen/sample (s/gen any?) 1000)))
=> (nil)
Why are there no numbers, strings, ... etc?
#2018-05-2516:33Alex Miller (Clojure team)b/c any? doesnā€™t have a very good generator right now#2018-05-2516:34Alex Miller (Clojure team)there is a ticket for this#2018-05-2516:40kvltThanks Alex#2018-05-2517:32jplazaHi all! Iā€™m getting an assert exception when using and, or or any combination of both inside (s/keys)#2018-05-2517:33jplaza(s/def ::invoice :req-un [::sequence ::uuid ::properties (or ::live ::environment)]) is this a bug?#2018-05-2517:34jplazaThis is the exception I get:
Assert failed: all keys must be namespace-qualified keywords
#2018-05-2517:35nenadalm@jplaza were in the code are your s/keys? I don't see that.#2018-05-2517:37jplazaSorry, a typo while simplifying my code. This is the correct snippet
(s/def ::invoice (s/keys :req-un [::sequence ::uuid ::properties (or ::live ::environment)]))
#2018-05-2517:38nenadalmI don't think you tried to run the code above...#2018-05-2517:42jplazaYou are right that simple example is working.#2018-05-2517:42jplazaBut my actual code isnā€™t. It works if I remove the and and ors though#2018-05-2517:52jplazaFound the error, I was stupidly trying to use or in :opt-un vector :face_palm:#2018-05-2610:31karol.adamiec@jplaza can you share ::uuid spec? i have issues with that one, as uuid? of ā€œā€ is trueā€¦.#2018-05-2619:59jumarNot sure how uuid? can return true for empty string:
(uuid? "")
;;=> false

;; clojure.core/uuid
(defn uuid?
  "Return true if x is a java.util.UUID"
  {:added "1.9"}
  [x] (instance? java.util.UUID x))

#2018-05-2622:21karol.adamiecin clojurescript šŸ˜ž#2018-05-2615:37jplaza@karol.adamiec we are using compact uuids (https://github.com/tonsky/compact-uuids) so we are using a predicate#2018-05-2622:22karol.adamiecthanks#2018-05-2802:31alex-dixonIs there a way to access why a spec passed?#2018-05-2802:38gfredericksisn't that what conforming is?#2018-05-2803:02alex-dixonNot sure. The situation that prompted that question is a multi spec that matched on a particular :type value of a map and spec associated with that value. I can tell that it passes and get the original map back, just curious if thereā€™s a way to access explain information when it passes. May be interesting stuff that could be done with it but seems only to get generated upon failure#2018-05-2813:45Alex Miller (Clojure team)explain info is only being built up when using explain, so no?#2018-05-2901:36callumI'm finding myself having to duplicate spec definitions in order to override generators and control the number of generated values. e.g.
(s/def ::name string?)
(s/def ::a (s/coll-of ::name))
(s/def ::b (s/keys :req-un [::a]))
(s/def ::c (s/keys :req-un [::a
                            ::b]))

;; only generate a single element for ::a
(gen/generate (s/gen ::c {::a #(s/gen (s/coll-of ::name :min-count 1 :max-count 1))}))
#2018-05-2901:36callumIs there a way to avoid this duplication?#2018-05-2903:47bbss
(s/def ::select
  (s/cat :plan-name #{'select}
         :thing-to-select #{"SCV" "SupplyDepot"}
         :at-least (s/cat :at-least-kw #{:at-least}
                          :at-least-number (s/int-in 1 50))
         :and-do-form (s/? (s/cat :and-do-kw #{:and-do}
                                                   :additional-do ::select))))

(gen/sample (s/gen ::select) 1)
Why does this stackoverflow? I'd expect the recursive spec to hit a "0 arity branch" of the s/? before it runs out of stack
#2018-05-2904:01bbssI notice https://clojuredocs.org/clojure.spec.alpha/*recursion-limit* doesn't mention s/? so maybe it's not intended to be used recursively.#2018-05-2905:49bbssEnded up accepting I can't generate it, but still validate which will be enough for now.
(s/def ::select
  (s/cat :plan-name #{'select}
         :thing-to-select #{"SCV" "SupplyDepot"}
         :at-least (s/cat :at-least-kw #{:at-least}
                          :at-least-number (s/int-in 1 50))
         :and-do-form (s/with-gen (s/? (s/cat :and-do-kw #{:and-do}
                                              :additional-do (s/spec (s/and list?
                                                                            ::select))))
                        #(gen/return ()))))
#2018-05-2908:40cvic@bbss https://gist.github.com/bhauman/2dca87815dfd92b3ff596bdc1e56c964#file-compiler-options-schema-clj-L30#2018-05-2908:45bbss@victor.cleja you're saying a duplicate key is the issue?#2018-05-2918:03bhaumanjust released a spec library#2018-05-2918:03bhaumanhttps://twitter.com/bhauman/status/1001523240529936384#2018-05-3007:00raymcdermottIā€™m trying to find a nicer way to report simple errors based on clojureā€™s core specsā€¦.#2018-05-3007:00raymcdermottfor example if a user types in (defn foo 1)#2018-05-3007:00raymcdermottwe get a fail on the defn spec#2018-05-3007:01raymcdermotthow can I have the result as data rather than a string?#2018-05-3007:01raymcdermottis there a magic var I can set or some other compiler property?#2018-05-3012:36Alex Miller (Clojure team)In the case of a spec error, you should get an ExceptionInfo where ex-data has the explain data#2018-05-3012:37Alex Miller (Clojure team)Or you can hook s/*explain-out*#2018-05-3012:40jumar@raymcdermott do you read and eval input from user?#2018-05-3012:41raymcdermottyes#2018-05-3012:42jumarThan this could work (although not sure if that's an optimal way to do it)
(try (eval (read-string "(defn foo 1)")) (catch Exception e (ex-data (.getCause e))))
#2018-05-3012:43raymcdermottIā€™m reading the results of evaluation via the PREPL which gives back a string for the :val key#2018-05-3012:44jumarI'm not familiar with prepl so don't know then#2018-05-3012:46raymcdermottIā€™m trying to avoid having to run eval on every input#2018-05-3012:47raymcdermottbefore I pass it to the REPL#2018-05-3012:47raymcdermottbut yes, thatā€™s a legitimate option @jumar#2018-05-3012:50raymcdermott@alexmiller Iā€™m using the PREPL from 1.10 and it would be nice (IMHO) if there was a property for spec fails#2018-05-3013:41Alex Miller (Clojure team)Iā€™m a little confused by what youā€™re saying. In the case of an exception, I would expect prepl to be converting the Throwable to data with Throwable->map which captures ex-data in a :data attribute.#2018-05-3013:55Alex Miller (Clojure team)trying some things in the repl, I understand what youā€™re saying I think, although I donā€™t understand why#2018-05-3013:57Alex Miller (Clojure team)oh, Iā€™m testing with io-prepl and itā€™s happening due to the default valf of pr-str - is that what youā€™re using @raymcdermott?#2018-05-3013:59Alex Miller (Clojure team)if you use a valf of identity, then youā€™ll get back throwables as data#2018-05-3014:07Alex Miller (Clojure team)
Clojure 1.10.0-master-SNAPSHOT
user=> (require '[clojure.core.server :as ccs])
nil
user=> (ccs/io-prepl :valf identity)
(defn foo 1)
{:tag :ret, 
 :val {:cause "Call to clojure.core/defn did not conform to spec:\nIn: [1] val: 1 fails spec: :clojure.core.specs.alpha/arg-list ...", 
       :via [{:type clojure.lang.Compiler$CompilerException, 
              :message "clojure.lang.ExceptionInfo: Call to clojure.core/defn did not conform...", 
              :at [clojure.lang.Compiler checkSpecs "Compiler.java" 6891]} 
             {:type clojure.lang.ExceptionInfo, 
              :message "Call to clojure.core/defn did not conform ...", 
#_here        :data #:clojure.spec.alpha{:problems (
                {:path [:args :bs :arity-1 :args], 
								 :pred clojure.core/vector?, 
								 :val 1, 
								 :via ...
`
#2018-05-3014:19raymcdermottthanks @alexmiller Iā€™ll give it a shot later#2018-05-3021:01raymcdermott@alexmiller I could be doing it wrong but Iā€™m using io-prepl as part of a socket-server like thisā€¦#2018-05-3021:03raymcdermottI then have a web server to intermediate#2018-05-3021:04raymcdermottAnd this takes request from a REPL client in the browser#2018-05-3021:04raymcdermottwhich send over strings that are then read and sent to the remote prepl#2018-05-3021:06raymcdermottRead failclojure.lang.LispReader$ReaderException: java.lang.RuntimeException: No reader function for tag object 1#2018-05-3021:08raymcdermottis this an abuse? Is there another option to read that is more compatible?#2018-05-3021:13raymcdermottIā€™ll keep fiddling but ideas welcome šŸ™‚#2018-05-3021:18raymcdermottRead fail: clojure.lang.EdnReader$ReaderException: java.lang.RuntimeException: No reader function for tag object#2018-05-3021:25Alex Miller (Clojure team)I havenā€™t put all these pieces together recently and donā€™t have time to look at it today. you might look at wrapping clj-server/remote-prepl for the client side?#2018-05-3021:26Alex Miller (Clojure team)and since stuff is mostly char stream based, you might just want the defaults in io-prepl but to read on the remote-prepl valf side, not sure#2018-05-3021:46raymcdermottthanks @alexmiller#2018-05-3111:21jumar@raymcdermott I've tried a limited version of your code and this seems to work (when tested with nc on command line):
(defn configured-prepl
  []
  (clj-server/io-prepl :valf identity))

(defn shared-prepl
  [opts]
  (let [socket-opts {:port          5555
                     :server-daemon false                   ; Keep the app running
                     :accept        `configured-prepl}]
    (let [server (clj-server/start-server (merge socket-opts opts))]
      (println "listening on port" (.getLocalPort ^ServerSocket server)))))
Run server:
(clj-server/remote-prepl "localhost" 5555)
Later on command line:
nc localhost 5555
(+ 1 1)
{:tag :ret, :val 2, :ns "user", :ms 0, :form "(+ 1 1)"}
(defn foo 1)
{:tag :ret, :val {:cause "Call to clojure.core/defn did not conform to spec:\nIn: [1] val: 1 fails spec: :clojure.core.specs.alpha/arg-list at: [:args :bs :arity-1 :args] predicate: vector?\nIn: [1] val: 1 fails spec: :clojure.core.specs.alpha/args+body at: [:args :bs :arity-n :bodies] predicate: (cat :args :clojure.core.specs.alpha/arg-list :body (alt :prepost+body (cat :prepost map? :body (+ any?)) :body (* any?)))\n", :via [{:type clojure.lang.Compiler$CompilerException, ...
For client written in Clojure I guess you should try remote-prepl as Alex suggested, but your problem seems to be related to the fact that read is not able to recognize #object in the response returned by remote repl. The error looks is caused by sth. like this:
(pr-str (Object.))
#object[java.lang.Object 0x66a64b49 "
The #object in response comes from spec failure (partial output, see the last line)
{:tag :ret, :val {:cause "Call to clojure.core/defn did not conform to spec:\nIn: [1] val: 1 
fails spec: :clojure.core.specs.alpha/arg-list at: [:args :bs :arity-1 :args] predicate: vector?\nIn: [1] 
val: 1 fails spec: :clojure.core.specs.alpha/args+body at: [:args :bs :arity-n :bodies] 
predicate: (cat :args :clojure.core.specs.alpha/arg-list :body (alt :prepost+body
 (cat :prepost map? :body (+ any?)) :body (* any?)))\n", :via [{:type clojure.lang.Compiler$CompilerException, 
:message "clojure.lang.ExceptionInfo: Call to clojure.core/defn did not conform to spec:\nIn: [1] 
val: 1 fails spec: :clojure.core.specs.alpha/arg-list at: [:args :bs :arity-1 :args] predicate: vector?\nIn: [1] 
val: 1 fails spec: :clojure.core.specs.alpha/args+body at: [:args :bs :arity-n :bodies] predicate: (cat :args :clojure.core.specs.alpha/arg-list 
:body (alt :prepost+body (cat :prepost map? :body (+ any?)) :body (* any?)))\n #:clojure.spec.alpha{:problems
 ({:path [:args :bs :arity-1 :args], :pred clojure.core/vector?, :val 1, :via [:clojure.core.specs.alpha/defn-args :clojure.core.specs.alpha/args+body
 :clojure.core.specs.alpha/arg-list :clojure.core.specs.alpha/arg-list], :in [1]}
 {:path [:args :bs :arity-n :bodies], :pred (clojure.spec.alpha/cat :args :clojure.core.specs.alpha/arg-list 
:body (clojure.spec.alpha/alt :prepost+body (clojure.spec.alpha/cat :prepost clojure.core/map? 
:body (clojure.spec.alpha/+ clojure.core/any?)) :body (clojure.spec.alpha/* clojure.core/any?))), :val 1,
 :via [:clojure.core.specs.alpha/defn-args :clojure.core.specs.alpha/args+body :clojure.core.specs.alpha/args+body], :in [1]}), 
:spec #object[clojure.spec.alpha$regex_spec_impl$reify__2436 0x233319e7 \"
I'm not sure if this can be resolved somehow...
#2018-05-3111:21jumar@raymcdermott I've tried a limited version of your code and this seems to work (when tested with nc on command line):
(defn configured-prepl
  []
  (clj-server/io-prepl :valf identity))

(defn shared-prepl
  [opts]
  (let [socket-opts {:port          5555
                     :server-daemon false                   ; Keep the app running
                     :accept        `configured-prepl}]
    (let [server (clj-server/start-server (merge socket-opts opts))]
      (println "listening on port" (.getLocalPort ^ServerSocket server)))))
Run server:
(clj-server/remote-prepl "localhost" 5555)
Later on command line:
nc localhost 5555
(+ 1 1)
{:tag :ret, :val 2, :ns "user", :ms 0, :form "(+ 1 1)"}
(defn foo 1)
{:tag :ret, :val {:cause "Call to clojure.core/defn did not conform to spec:\nIn: [1] val: 1 fails spec: :clojure.core.specs.alpha/arg-list at: [:args :bs :arity-1 :args] predicate: vector?\nIn: [1] val: 1 fails spec: :clojure.core.specs.alpha/args+body at: [:args :bs :arity-n :bodies] predicate: (cat :args :clojure.core.specs.alpha/arg-list :body (alt :prepost+body (cat :prepost map? :body (+ any?)) :body (* any?)))\n", :via [{:type clojure.lang.Compiler$CompilerException, ...
For client written in Clojure I guess you should try remote-prepl as Alex suggested, but your problem seems to be related to the fact that read is not able to recognize #object in the response returned by remote repl. The error looks is caused by sth. like this:
(pr-str (Object.))
#object[java.lang.Object 0x66a64b49 "
The #object in response comes from spec failure (partial output, see the last line)
{:tag :ret, :val {:cause "Call to clojure.core/defn did not conform to spec:\nIn: [1] val: 1 
fails spec: :clojure.core.specs.alpha/arg-list at: [:args :bs :arity-1 :args] predicate: vector?\nIn: [1] 
val: 1 fails spec: :clojure.core.specs.alpha/args+body at: [:args :bs :arity-n :bodies] 
predicate: (cat :args :clojure.core.specs.alpha/arg-list :body (alt :prepost+body
 (cat :prepost map? :body (+ any?)) :body (* any?)))\n", :via [{:type clojure.lang.Compiler$CompilerException, 
:message "clojure.lang.ExceptionInfo: Call to clojure.core/defn did not conform to spec:\nIn: [1] 
val: 1 fails spec: :clojure.core.specs.alpha/arg-list at: [:args :bs :arity-1 :args] predicate: vector?\nIn: [1] 
val: 1 fails spec: :clojure.core.specs.alpha/args+body at: [:args :bs :arity-n :bodies] predicate: (cat :args :clojure.core.specs.alpha/arg-list 
:body (alt :prepost+body (cat :prepost map? :body (+ any?)) :body (* any?)))\n #:clojure.spec.alpha{:problems
 ({:path [:args :bs :arity-1 :args], :pred clojure.core/vector?, :val 1, :via [:clojure.core.specs.alpha/defn-args :clojure.core.specs.alpha/args+body
 :clojure.core.specs.alpha/arg-list :clojure.core.specs.alpha/arg-list], :in [1]}
 {:path [:args :bs :arity-n :bodies], :pred (clojure.spec.alpha/cat :args :clojure.core.specs.alpha/arg-list 
:body (clojure.spec.alpha/alt :prepost+body (clojure.spec.alpha/cat :prepost clojure.core/map? 
:body (clojure.spec.alpha/+ clojure.core/any?)) :body (clojure.spec.alpha/* clojure.core/any?))), :val 1,
 :via [:clojure.core.specs.alpha/defn-args :clojure.core.specs.alpha/args+body :clojure.core.specs.alpha/args+body], :in [1]}), 
:spec #object[clojure.spec.alpha$regex_spec_impl$reify__2436 0x233319e7 \"
I'm not sure if this can be resolved somehow...
#2018-05-3111:23jumarThat last bit I tested in separate repl:
(require '[clojure.core.server :as ccs])

user=> (ccs/remote-prepl "localhost" 5555 *in* #(prn %))
(+ 1 1)
{:tag :ret, :val 2, :ns "user", :ms 0, :form "(+ 1 1)"}

(defn foo 1)
Exception in thread "clojure.core.server/remote-prepl" clojure.lang.LispReader$ReaderException: java.lang.RuntimeException: No reader function for tag object
        at clojure.lang.LispReader.read(LispReader.java:304)
...
#2018-05-3111:23raymcdermottyeah, he mentioned the valf option#2018-05-3111:24raymcdermottthanks for testing it out @jumar#2018-05-3111:24raymcdermottin theory with valf set to identity we should get a :data property and we can then read that directly#2018-05-3111:25raymcdermottso Iā€™m going to play with a remote-prepl as you suggest#2018-05-3111:26raymcdermotttbh Iā€™m struggling on what to give it as a Reader and an out-fn ā€¦. so Iā€™m in šŸ¤” mode at the moment#2018-05-3111:26raymcdermottif you have any ideas ā€¦ I am in the market šŸ™‚#2018-05-3111:51jumar@raymcdermott Couldn't find anything better than leveraging *default-data-reader-fn* somehow:
user=> (require '[clojure.core.server :as ccs])
nil
user=> (alter-var-root #'*default-data-reader-fn* (fn [_] (fn [tag value] value)))
#object[user$eval3$fn__138$fn__139 0xfd8294b "
#2018-05-3112:35raymcdermottI didnā€™t know about that var, so thanks ā€¦.#2018-05-3112:35raymcdermottI really must look at all of those ear-muff things one day if Iā€™m going to live down here šŸ™‚#2018-05-3113:00Alex Miller (Clojure team)This is interesting and probably ultimately needs a change in spec reporting. Rather than send the spec instance, it should be sending the spec name or form where you are seeing the object. For now, I think youā€™ll need to intercept either on the server or client side and either remove or transform it yourself#2018-05-3122:29raymcdermottdo you consider this as a spec issue rather than a PREPL issue? Or a bit of both?#2018-05-3122:51Alex Miller (Clojure team)spec for the reporting#2018-05-3122:51Alex Miller (Clojure team)Maybe prepl although it gives you the hook to address it#2018-05-3113:01Alex Miller (Clojure team)I think hooking the default data reader to fall back to using tagged-literal is a pretty good dodge that covers many potential issues like this#2018-05-3113:31Andreas LiljeqvistHow would I alias a namespace that isn't required? Circular dependency is a problem if I require it#2018-05-3113:33Andreas LiljeqvistI want to keep a separate namespace/file for my spec but use another namespace#2018-05-3113:34Andreas LiljeqvistLike my-namespace with functions and my-namespace.spec with the spec#2018-05-3113:35Andreas LiljeqvistBut using :t/id instead of having to write :my-namespace/id#2018-05-3113:42guySo do you mean like. Inside of my-namespace you have some spec like (s/def :t/id) instead of (s/def ::id) or (s/def :my-namespace/id)#2018-05-3113:42guyAs far as i understood you have to require the namespace the spec is in, to use it in a different namespace. But i could be wrong#2018-05-3113:44guySo you might want to just move your spec somewhere else to get around the circular dependencies? I ended up just giving certain specs fake namespaces to and not using ::id. Instead something like :user/id#2018-05-3113:45Andreas Liljeqvistin my-namespace.spec I have a (s/def ::id nat-int?) . Resulting in :my-namespace.spec/id#2018-05-3113:46Andreas LiljeqvistI would like the spec to remain in my-namespace.spec but be registered to :my-namespace/id#2018-05-3113:46guytry this instead (s/def :my-namespace/id nat-int?)#2018-05-3113:47Andreas LiljeqvistThat will work, but it feels wrong when I have a lot of defs#2018-05-3113:47guyYou will still need to require my-namespace.spec in a different ns if you want to use the spec contained#2018-05-3113:47Andreas Liljeqvistyeah#2018-05-3113:47guybut then you can refer to it like :my-namespace/id instead#2018-05-3113:48guyI think my counter argument to "but it feels wrong when I have a lot of defs" would be its a bit redundant to have .spec in the spec name#2018-05-3113:49guy:my-namespace.spec/id vs :my-namespace/id#2018-05-3113:49guylike you know they are both specs#2018-05-3113:49guyso is .spec really needed?#2018-05-3113:49guy(this is just my opinion though, so someone else feel free to correct me etc)#2018-05-3113:49Andreas LiljeqvistOnly reason for the .spec is that I want to have a separate namespace for the specs vs the functions#2018-05-3113:50guyYeah i had user and user.spec too, because it felt nicer to separate them#2018-05-3113:50guyyeah exactly the same thought process#2018-05-3113:50guyBut i ended up moving away from :: notation to something/id instead#2018-05-3113:51Andreas LiljeqvistA bit worried about keyword collisions when doing it that way, but everything you say makes sense#2018-05-3113:52guyI think thats more a case of you having a consistent naming convention really#2018-05-3113:52guy>keyword collisions#2018-05-3113:52guyi mean#2018-05-3113:52Andreas LiljeqvistI would have prefered to do something like (alias 't 'my-namespace)#2018-05-3113:53guyThe troubles i was having were similarly named fields#2018-05-3113:53guyuser/name vs something else /name#2018-05-3113:53guyso if u had a (s/def ::name) that was a string and one that was something else#2018-05-3113:53guyit would cause headaches#2018-05-3113:53guyif that makes sense?#2018-05-3113:54Andreas LiljeqvistYeah, but I might be using a third party library that also implement :user/name or something#2018-05-3113:54Andreas LiljeqvistBut that is a problem for future me šŸ™‚ Thanks a lot for the help#2018-05-3113:54guysure but then you can prefix to to Andreas.user/name or Projectname.user/name etc šŸ‘#2018-05-3113:54guynps! Good luck!#2018-05-3113:54guyHopefully i was helpful haha#2018-05-3114:00OlicalAfternoon. I was wondering if anyone else uses sets with their (s/keys...) calls?#2018-05-3114:01OlicalIn the docs it mentions vectors, but wouldn't a set of keys make more sense, like: (s/keys :req-un #{::a ::b})?
#2018-05-3114:01Alex Miller (Clojure team)conceptually, yes#2018-05-3114:01OlicalI ask because I've seen a weird issue where UUIDs are not being coerced by spec-tools if I use a set šŸ˜…#2018-05-3114:05OlicalYou can reproduce it with
(s/def ::a (st/spec uuid?))
(s/def ::b (s/keys :req-un #{::a}))
(st/decode ::b {:a "4c6e852b-e8a3-4686-8916-4e345be53731"} st/json-transformer)
;; => {:a "4c6e852b-e8a3-4686-8916-4e345be53731"}
where st is spec-tools.core
#2018-05-3114:06OlicalIf you replace the set with a vector it works fine and coerces the UUID string into an actual UUID. I was thinking "maybe this is an issue with spec-tools", but if you're supposed to use a vector anyway this is technically okay, just a confusing gotcha.#2018-05-3114:23ikitommiyes, the s/keys is parsed here: https://github.com/metosin/spec-tools/blob/master/src/spec_tools/parse.cljc#L112-L117. Hasn't been tested with sets. PR welcome ;)#2018-05-3114:27OlicalSo @U050V1N74 and I have spotted it šŸ˜„#2018-05-3114:27OlicalIt's because clojure.core/flatten on a set empties the set!#2018-05-3114:28OlicalTIL for the both of us.#2018-05-3114:28Olical(flatten #{1 2 3}) gives you ()#2018-05-3114:29Olical"because sets aren't sequential, they're only seqable"#2018-05-3114:30OlicalAny idea why the call to flatten is required here though? https://github.com/metosin/spec-tools/blob/93ca24167131a2e312fed1da79beb25632090fda/src/spec_tools/impl.cljc#L53#2018-05-3114:30Olical(probably this one actually https://github.com/metosin/spec-tools/blob/93ca24167131a2e312fed1da79beb25632090fda/src/spec_tools/impl.cljc#L64)#2018-05-3114:59ikitommiI think it's for supporting the nested`or`s? See the tests https://github.com/metosin/spec-tools/blob/master/test/cljc/spec_tools/parse_test.cljc#2018-05-3114:59ikitommi.. and ands#2018-05-3115:02jarohenTIL ors and ands in s/keys are a thing, too#2018-05-3114:13Andreas LiljeqvistSeems like https://dev.clojure.org/jira/browse/CLJ-2123 is related to my question about aliases#2018-06-0113:15bostonaholicIs this the proper way to spec a multiple arity function? with :args (s/or ...)
(defn foo
  ([a]    ...)
  ([a b]) ...)

(s/fdef foo
  :args (s/or :arity-one (s/cat :a int?)
              :arity-two (s/cat :a int? :b int?))
  :ret int?)
#2018-06-0113:25Alex Miller (Clojure team)Iā€™d use s/alt there instead of s/or as itā€™s a regex op, but thatā€™s one option#2018-06-0113:25Alex Miller (Clojure team)Another is to push the optionality into the cat with s/?#2018-06-0113:26bostonaholics/alt is better, thanks#2018-06-0113:29bostonaholicwhen I run my suite with lein test and I get the generator seed ā€œUsing generator seed: -1186346283ā€, how do I pass that seed into lein test again?#2018-06-0113:39Alex Miller (Clojure team)I donā€™t know of any way to do that from that level#2018-06-0113:39Alex Miller (Clojure team)stest/check takes an option map where you can pass in a seed#2018-06-0113:40Alex Miller (Clojure team)you might be able to do it through some test.check dynvar or something, not sure#2018-06-0116:16kapilIs there an idiomatic way to associate a doc-string with a spec?#2018-06-0116:21kapilI am trying to generate documentation from spec for non-clojure developers to read. Something like this
(s/def ::id string? "Id of resource X. Ex. resource_1")
(s/def ::num integer? "Number of results for pagination")

(s/def ::result (s/keys :req-un
                        [::id ::num]))

(gen-doc ::result) =>

{:id "resource_1" ;; "Id of resource X. Ex. resource_1"
 :num 10 ;;"Number of results for pagination"
 }
#2018-06-0116:38Alex Miller (Clojure team)no, not yet but we intend to add that#2018-06-0118:37kapil@U064X3EF3 Is there any workaround I can do for now? I tried using with-meta but it doesn't with no success since keywords are not Objects. I could however maintain a registry of documentation.#2018-06-0118:43Alex Miller (Clojure team)there are some external libs that are providing this, but itā€™s hard to recommend those as what they are doing is likely to break#2018-06-0118:43Alex Miller (Clojure team)you could build an external map of spec keyword to doc string#2018-06-0118:45kapil
(defonce doc-registry (atom {}))

(defmacro def-w-doc
  [k spec-form doc]
  `(s/def ~'k ~spec-form)
  (swap! doc-registry assoc k doc))

(def-w-doc ::id string? "Id of resource X. Ex. resource_1")
#2018-06-0118:47kapilThis is what I am using now. This is least intrusive change I could think. Later on when clojure.spec adds support for documenting specs I can just modify def-w-doc.#2018-06-0118:59Alex Miller (Clojure team)sounds good#2018-06-0116:16noisesmithfor the seed, one workflow would be to keep your randomized check, and also do another with a specific seed that creates a known rare corner case#2018-06-0116:18bostonaholicSEED=<seed> lein test is how to do it#2018-06-0116:18noisesmithoh that works? interesting#2018-06-0116:19noisesmithI like to have regression tests for things that have been known to fail in the past though#2018-06-0116:19noisesmith(I guess it's better to do that by hard coding the data that had been generated in an example based test)#2018-06-0116:25bostonaholicyup, setting the SEED as an env var works#2018-06-0213:47gfredericksFWIW, I have never heard of this and have no idea what's going on#2018-06-0213:47gfredericksmy suspicion is it's not a test.check seed, it's something else#2018-06-0118:21rapskalianI'm curious what others' tastes are regarding namespaced keys in spec definitions? As an example, let's say we're modeling GPS coordinates. Personally, I've been doing something like this:
(s/def :coordinate/latitude double?)
(s/def :coordinate/longitude double?)
(s/def ::coordinate (s/keys :req [:coordinate/latitude
                                  :coordinate/longitude]))
In which lat and long are both "nested" under the :coordinate namespace, and then the coordinate map itself is defined inside of some kind of :my-project.specs top-level namespace (using the ::coordinate sugar). Does this make sense, or are there other conventions that have been established that would be clearer?
#2018-06-0118:42seancorfield@cjsauer That's pretty close to my approach. Much depends on how unique you need the names to be and how your code might be used.#2018-06-0118:43seancorfieldIn a library, you'd really want all the names to be globally unique. In an application, you can use less unique names.#2018-06-0118:44Alex Miller (Clojure team)they should be sufficiently unique :)#2018-06-0120:58rapskalian@seancorfield good to hear. This is for an application, so I'll trade some uniqueness for readability šŸ™‚#2018-06-0219:19cvicIf I add
[tentacles.core :as c]
lein repl throws
Exception in thread "main" clojure.lang.ExceptionInfo: Call to clojure.core/defn- did not conform to spec:
In: [0] val: clj-tuple/conj-tuple fails spec: :clojure.core.specs.alpha/defn-args at: [:args :name] predicate: simple-symbol?
 #:clojure.spec.alpha{:problems [{:path [:args :name], :pred clojure.core/simple-symbol?, :val clj-tuple/conj-tuple, :via [:clojure.core.specs.alpha/defn-args :clojure.core.specs.alpha/defn-args], :in [0]}], :spec #object[clojure.spec.alpha$regex_spec_impl$reify__2436 0x162e29a1 "
No idea why...
#2018-06-0219:29cvicOr... some dependency that has a malformed defn- form.#2018-06-0219:29cvicTime to do some digging#2018-06-0219:35cvicAh, nvm. I'll just use [irresponsible/tentacles "0.6.2"]#2018-06-0222:22Alex Miller (Clojure team)The line in error is clj_tuple.clj line 556#2018-06-0222:22Alex Miller (Clojure team)Wherever that is#2018-06-0318:16cvicSometimes upgrading a lib is the shortest way of solving it#2018-06-0318:20cvichttps://github.com/ztellman/clj-tuple/blob/master/src/clj_tuple.clj Yep. Also old.#2018-06-0504:33Vincent CantinHi. I plan to try to use spec for auto-completion. Has it been done before?#2018-06-0504:41Vincent CantinAre the specs easy to parse /navigate programmatically?#2018-06-0506:48ikitommi@U8MJBRSR5 lot of parsers in 3rd party libs, using s/form. As a sample, one parser and a spec visitor in spec-tools (https://github.com/metosin/spec-tools).#2018-06-0506:48ikitommiAlso a issue to have a spec walker in the core: https://dev.clojure.org/jira/browse/CLJ-2251#2018-06-0506:49Vincent Cantinthank you#2018-06-0505:20Vincent Cantinā€¦ I think that s/explain-data does a lot already.#2018-06-0510:16Vincent CantinIs there some kind of s/seq-of operator, or a way to build it? The idea is to act as a bridge to spec the chars in a string from the string itself: (s/conform (s/seq-of char?) "abc")#2018-06-0510:35hkjelsI donā€™t have a running repl atm, but I think you can use (coll-of char? :kind string?)#2018-06-0510:41Vincent CantinIt works, thanks. Need to add :into [] to keep the chars order at conform time.#2018-06-0511:02Vincent CantinI still have trouble afterward ā€¦ I would like to run some regex spec (e.g. s/cat) on the content of a string but without having to apply s/conform on the seq of that string. I donā€™t know if it is possible.#2018-06-0514:22bbrinckFWIW, the general advice Iā€™ve seen is that spec is not a great fit for parsing strings. Certainly people have done it by turning the string into a sequence with seq, but often people run into further issues. More detail in comments here: https://www.reddit.com/r/Clojure/comments/7vwpu4/parsing_with_clojurespec/#2018-06-0514:25bbrinckYMMV of course#2018-06-0602:43Vincent CantinTL;DR : https://clojuredocs.org/clojure.spec.alpha/conformer ā€¦with a nice example under the documentation.#2018-06-0602:47bbrinckA good discussion of pitfalls with conformers and how to work around them: https://groups.google.com/d/msg/clojure/Tdb3ksDeVnU/LAdniiZNAgAJ#2018-06-0602:48bbrinckā€œwe recommend that you not use conformers for coercion. Conformers were added primarily as a tool for building custom composite spec typesā€#2018-06-0603:00bbrinckI know people have had success with using spec for parsing strings, just wanted to make you aware of possible issues down the road šŸ™‚#2018-06-0603:00bbrinckif you run into them, it may be worth looking at something like https://github.com/cgrand/regex#2018-06-0603:04Vincent CantinAh, Christophe again :-)#2018-06-0600:08noisesmithis there a way to get the individual spec keywords mapped to by an s/or call?#2018-06-0600:11noisesmithlooks like describe gets me most of the way if there's nothing better#2018-06-0611:37Alex Miller (Clojure team)s/form will be more complete than s/describe#2018-06-0621:24arohnerIs there a way to get conformed output to be different for non-regex specs? I have a bunch of strings I need to parse, and Iā€™d like to validate a parsed string against the spec, and returned the parsed values, if they conform#2018-06-0621:44Alex Miller (Clojure team)in short, no#2018-06-0621:46Alex Miller (Clojure team)you would need a custom spec impl for that currently and I wouldnā€™t recommend that as the guts are likely to change#2018-06-0621:46Alex Miller (Clojure team)in the future, maybe#2018-06-0622:54dpsutton
(s/valid? :fhir-type/PaymentReconciliation
            examples/pay-rec)
false
com.breezeehr.specs.fhir.PaymentReconciliation> 
(s/explain :fhir-type/PaymentReconciliation
            examples/pay-rec)
Success!
nil
anyone familiar with something like this?
#2018-06-0623:48seancorfield@dpsutton I have seen some occasional bugs reported against spec that seem to have that behavior -- I guess you'll need to dig into that :fhir-type/PaymentReconciliation spec and see what it's doing?#2018-06-0623:49dpsuttonIt kinda sorted itself out but no idea what the transient state was. Just wondering if it was known bug or if we were way off the beaten trail#2018-06-0623:52seancorfieldAh, a Heisenspec? šŸ™‚#2018-06-0623:52dpsuttonI'm guessing it's more simple and just something silly we are doing. But you never know#2018-06-0700:25Alex Miller (Clojure team)Generally anything like this should be considered a bug (if itā€™s reproducible). One thing that could possibly lead you down this path at the repl is changing a spec but not redefining a spec that used it. Specs are compiled at definition time so you can get a mixture of things that way.#2018-06-0702:08dpsuttonOh. I made a poor man's version of spec using defs and thought you got around that with the central database of specs and always getting a "fresh" copy. I'll make sure to have up to date definitions. Thanks#2018-06-0702:49Vincent CantinWhat is the most idiomatic way to apply s/cat, knowing that it is a macro?#2018-06-0712:33Alex Miller (Clojure team)A future way to do this will be to write a spec for s/cat, write conform data, and s/unform back to a spec.#2018-06-0712:34Alex Miller (Clojure team)See CLJ-2112 for some early work on that#2018-06-0702:50noisesmithto apply a macro, you need another macro (and to do it at runtime to something that is not a literal in your source file or created when loading the file you need to be compiling new code, eg. eval)#2018-06-0703:06Vincent CantinI will use a macro then. Do you know why it was made as a macro? (most importantly, why only as a macro)#2018-06-0703:07Vincent CantinThat would have been good to have the macro and the functions and let the users choose between them.#2018-06-1419:56johanatanCompletely agree. The prevalence of macros in spec is my number one complaint with it.#2018-06-0703:07Vincent Cantins/cat and sm/cat#2018-06-0703:17Vincent Cantinā€œmacro contagionā€ ā€¦ it reminds me of the ā€œconst hellā€ in c++#2018-06-0703:42seancorfield@vincent.cantin spec was designed for humans to write, so macros are fine. There is work in the pipeline to open up programmatic access to spec, which will mean functions.#2018-06-1419:54johanatanMacros are not fine. They prevent higher-order code and partial application (to name a couple of issues). And of course they result in 'macro proliferation' as well.#2018-06-0703:43seancorfieldSo, for now, you might be going through a lot of (macro) pain for nothing, depending on how quickly you need the thing you're trying to build...#2018-06-0703:45Vincent CantinThe task I gave myself is probably making me a complainer by design: I am trying to do the most I can do with Spec.#2018-06-0703:46Vincent CantinYou are right, it is not related to the normal use case.#2018-06-0703:46seancorfieldIncluding lots of things it wasn't designed for, I'm sure šŸ™‚#2018-06-0703:47seancorfieldIt's interesting... over time, the Clojure/core folks have produced a number of things that in the rush of new excitement around a feature, lots of people almost immediately try to do stuff with each new feature that's outside the design goals šŸ™‚#2018-06-0705:40Vincent Cantin@U04V70XH6 I was doing that: https://github.com/green-coder/TIS-100-Spec My problem with the cat being a macro was for the implementation of my token spec.#2018-06-0705:42Vincent CantinI wanted to see if it was possible to do it that way. Well ā€¦ it is possible, but not convenient.#2018-06-0705:44seancorfieldAnd how is performance? šŸ‘€#2018-06-0705:44Vincent Cantineh eh .. I donā€™t know, it does not matter here.#2018-06-0705:46Vincent CantinI wished that cat could accept nil instead of a token for specs which we do not need to know about when we use conform.#2018-06-0705:46Vincent Cantinin regex terms, it would mean a ā€œnon-capturingā€ expression.#2018-06-0705:52seancorfieldI'm wondering how many times folks have to say "spec is not a parser"... ? :rolling_on_the_floor_laughing:#2018-06-0705:53Vincent CantinšŸ˜„ that means that the community would like to use it in that way too.#2018-06-0705:54seancorfieldIf you get the grammar right and each spec generates then you have the possibility of generating random but syntactically valid TIS-100 programs... that you could then run and see what they do! šŸ™‚#2018-06-0705:54Vincent CantinYou start to see some of my goals#2018-06-0712:05dottedmagHowever spec is definitely sold as a parser, maybe unintentionally. "Check your inputs and, oh here is the parse tree".#2018-06-0712:05dottedmagIt's not a best string parser out there, that's for sure.#2018-06-0703:48seancorfieldI think it says a lot about how the features have been such great building blocks over the years, as well as how laser-focused many of the features have been.#2018-06-0703:49seancorfieldI don't know if you've done much with clj and deps.edn yet? That seems to be attracting the same "let's see what crazy things we can do with this" experimentation.#2018-06-1419:57johanatanI don't consider wanting to apply or partially apply a macro to be a "crazy experimentation". It's very much just a thing that advanced [meta-]programmers want to do to keep their code concise.#2018-06-0716:58favilaI can't find a ticket for this, but I can't believe it wasn't noticed before#2018-06-0716:59favilas/every or s/coll-of will interpret :kind as a predicate only (not spec) during s/valid? s/conform etc#2018-06-0716:59favilabut during explain, it does interpret as a spec#2018-06-0717:00favilaI think it is because the macros generate a cpred unconditionally interpreting kind as a pred#2018-06-0717:00favilaexample of how this can go wrong:#2018-06-0717:02favila
(s/def ::vector vector?)
=> :user/vector
(s/valid? (s/coll-of any? :kind ::vector) [])
=> false
(s/explain (s/coll-of any? :kind ::vector) [])
Success!
=> nil
#2018-06-0717:03favilaHere, this is even clearer:#2018-06-0717:03favila
(s/explain (s/coll-of any? :kind (s/spec vector?)) [])
Success!
=> nil
(s/valid? (s/coll-of any? :kind (s/spec vector?)) [])
ClassCastException clojure.spec.alpha$spec_impl$reify__1987 cannot be cast to clojure.lang.IFn  user/eval3301/fn--3303 (form-init7886713966146839296.clj:1)
#2018-06-0717:03dpsuttonin checking against the spec, it will call (::vector []) which returns nil and fails the spec. In explaining, it is aware that [] satisfies the ::vector spec#2018-06-0717:04favila
(macroexpand '(s/coll-of any? :kind (s/spec vector?)))
=>
(clojure.spec.alpha/every-impl
 (quote any?)
 any?
 {:clojure.spec.alpha/describe (quote
                                (clojure.spec.alpha/coll-of
                                 clojure.core/any?
                                 :kind
                                 (clojure.spec.alpha/spec clojure.core/vector?))),
  :clojure.spec.alpha/conform-all true,
  :clojure.spec.alpha/cpred (fn*
                             [G__3317]
                             (clojure.core/and ((s/spec vector?) G__3317))),
  :kind (s/spec vector?),
  :clojure.spec.alpha/kind-form (quote
                                 (clojure.spec.alpha/spec clojure.core/vector?))}
 nil)
#2018-06-0717:04favilaNote that :cpred is generated but nonsense#2018-06-0717:05faviladocs say#2018-06-0717:05favila> :kind - a pred/spec that the collection type must satisfy, e.g. vector? (default nil) Note that if :kind is specified and :into is not, this pred must generate in order for every to generate.#2018-06-0717:06favilanote "pred/spec"#2018-06-0717:12favilaok, found the ticket, nm#2018-06-0717:12favilahttps://dev.clojure.org/jira/browse/CLJ-2111#2018-06-0717:51bmaddyI'm trying to write a spec for a function that takes in another function. I have the spec for the function passed in, but am getting "Couldn't satisfy such-that predicate after 100 tries.". Is that because I need to write a generator for functions that are arguments?#2018-06-0718:15noisesmithsuch-that acts as a filter, it's saying it wasn't finding anything that the filter accepted#2018-06-0718:27Alex Miller (Clojure team)how is it specā€™ed?#2018-06-0720:03bmaddy(sorry, was at lunch) Here's how I have it spec'd:
(s/def ::ruleset-id keyword?)
(s/def ::rule-name keyword?)
(s/fdef ::fetching-fn
        :args ::ruleset-id
        :ret (s/coll-of ::rule-name))
(s/fdef rules-for-rulesets
        :args (s/cat :fetching-fn ::fetching-fn
                     :ruleset-ids (s/coll-of ::ruleset-id))
        :ret (s/map-of (s/and set? (s/coll-of ::ruleset-id))
                       (s/and set? (s/coll-of ::rule-name)))
        ;; Ensure each rule only appears in the values only once so we
        ;; don't run them multiple times
        :fn #(->> % :ret vals (reduce concat) frequencies vals (every? (partial = 1))))
#2018-06-0720:33taylorit might be the s/ands where the first predicate is set?#2018-06-0720:34taylorcould you try changing that (s/and set? (s/coll-of ::the-spec)) to (s/coll-of ::the-spec :kind set?) @bmaddy#2018-06-0720:35taylor(because s/and makes a generator from whatever the first predicate is, and set? is unlikely to produce values that will satisfy ::ruleset-id)#2018-06-0720:39tayloroh, another potential issue: ::fetching-fn's :args spec should probably be (s/cat :whatever ::ruleset-id) instead of just ::ruleset-id?#2018-06-0720:39bmaddytrying it...#2018-06-0720:43bmaddyOh geez, I think it's the :args (s/cat ... thing. picard-facepalm#2018-06-0720:43tayloryeah I just realized the s/and issue probably has nothing to do with the such-that issue#2018-06-0720:44bmaddy
((gen/generate (s/gen ::fetching-fn)) :foo)
[:K5.SY.if_!_._F6U/_
 :p1Je.*-/*
 :LbrF_/ZP]
That's so cool. šŸ˜„
#2018-06-0720:45tayloryeah FYI the fn generators (I think) just generate functions that'll return values generated from their :ret spec#2018-06-0720:46bmaddyYeah, that's what I would expect. It's really cool that spec does that though. šŸ˜„#2018-06-0720:49bmaddyThanks for the help @taylor (and others who looked at it!)#2018-06-0721:49hlshipI'm having a bit of trouble getting just the right bit of output for one of my specs. This is for expound 0.7.0. Context: https://github.com/walmartlabs/lacinia/blob/a09684af4f2cabf23eb3d315bba0adad66787b57/src/com/walmartlabs/lacinia/schema.clj#L275 https://github.com/walmartlabs/lacinia/blob/a09684af4f2cabf23eb3d315bba0adad66787b57/src/com/walmartlabs/lacinia/expound.clj#2018-06-0721:50hlshipI've made slight changes:
(s/def ::type (s/or :base-type ::type-name
                    :wrapped-type ::wrapped-type))
(s/def ::wrapped-type (s/cat :modifier ::wrapped-type-modifier
                             :type ::type))
(s/def ::wrapped-type-modifier #{'list 'non-null})
and
(defmsg ::schema/wrapped-type-modifier "a wrapped type: '(list type) or '(non-null type)")
#2018-06-0721:50hlshipBut I get:
(binding [s/*explain-out* expound/printer]
  (s/explain ::schema/field {:type '(something String)}))
-- Spec failed --------------------

  {:type (something ...)}
          ^^^^^^^^^

should be one of: (quote list), (quote non-null)

-- Relevant specs -------

:com.walmartlabs.lacinia.schema/wrapped-type-modifier:
  #{'non-null 'list}
:com.walmartlabs.lacinia.schema/wrapped-type:
  (clojure.spec.alpha/cat
   :modifier
   :com.walmartlabs.lacinia.schema/wrapped-type-modifier
   :type
   :com.walmartlabs.lacinia.schema/type)
:com.walmartlabs.lacinia.schema/type:
  (clojure.spec.alpha/or
   :base-type
   :com.walmartlabs.lacinia.schema/type-name
   :wrapped-type
   :com.walmartlabs.lacinia.schema/wrapped-type)
:com.walmartlabs.lacinia.schema/field:
  (clojure.spec.alpha/keys
   :req-un
   [:com.walmartlabs.lacinia.schema/type]
   :opt-un
   [:com.walmartlabs.lacinia.schema/description
    :com.walmartlabs.lacinia.schema/resolve
    :com.walmartlabs.lacinia.schema/args
    :com.walmartlabs.lacinia.schema/deprecated])

-------------------------
Detected 1 error
=> nil
#2018-06-0721:51hlshipI'd expect it to say:
should be a wrapped type: '(list type) or '(non-null type)
#2018-06-0721:58bbrinck@hlship Ah, yes, Iā€™ve run into this as well. The problem is that defmsg is narrowly applied to predicates, not any type of spec.#2018-06-0721:59hlship
Associates the spec named `k` with `error-message`.
doesn't make that clear. Hm.
#2018-06-0721:59bbrinckAgreed, and I donā€™t think itā€™s a good design#2018-06-0721:59bbrinckItā€™s a reflection of the original problem defmsg solved: trying to provide more ā€œhuman-friendlyā€ messages instead of string? int?#2018-06-0722:00bbrinckbut thatā€™s an arbitrary restriction, and I think it should be generalized to any spec, so you can add messages across the board#2018-06-0722:00hlshipI'm trying to think of a work-around.#2018-06-0722:01bbrinckI havenā€™t tried it, but what happens if you replace #{'list 'non-null} with (fn [x] (contains? #{'list 'non-null} x)?#2018-06-0722:02bbrinck(I realize itā€™s a hack šŸ˜ž )#2018-06-0722:02hlshipBingo#2018-06-0722:03bbrinckIn any case, https://github.com/bhb/expound/issues/101#2018-06-0722:03hlshipPerhaps you could tweak the logic to treat a set as a predicate#2018-06-0722:03bbrinckEven more generally, I think I should make sure to respect a registered message regardless of the spec type#2018-06-0722:06bbrinckfrankly, I wasnā€™t sure anyone was really using Expoundā€™s capability to register messages, so I wasnā€™t sure if it was really important. Glad to know you are using it šŸ™‚#2018-06-0722:07bbrinckIā€™ve recently gone down a rabbit hole of trying to generate specs to test Expound but itā€™s taken up a lot more time than expected. Iā€™m going to pause that effort and switch over to bug fixing for a bit#2018-06-0722:11hlshipI'm very glad you implemented those messages, I think it can make a major difference. In general, I've gotten the rest of the team addicted to Expound ... we use a fair amount of clojure.spec, but previously, spec errors were treated as a boolean ā€” they were so hard to parse, we just looked at code changes to figure out how to make them go away. Now we have a proper guide to exactly what's wrong.#2018-06-0722:42bbrinckVery glad to hear it!#2018-06-0722:42bbrinck@hlship Iā€™d be interested in your feedback on another potential feature: message-fns https://github.com/bhb/expound/pull/96#2018-06-0722:44bbrinckIā€™m on the fence as to whether this would be useful enough to warrant inclusion#2018-06-0722:45hlshipWell, as a library developer, I want expound to be an optional dependency, so I don't think I could use this.#2018-06-0722:48bbrinckOh sorry, I was unclear in my description on this issue: it works like messages today in the sense that there is no effect if someone isnā€™t using expound.#2018-06-0722:48bbrinckThe only difference is that instead of registering a static string, you can register a function which will be called to generate the string#2018-06-0722:49bbrinckThis would let you, say, augment the existing expound error message, or use the problem to craft a custom message. But perhaps in your use case, the static strings are sufficient.#2018-06-0817:44martinklepschIs there a way to find out from what namespace/file a spec has been defined?#2018-06-0819:54seancorfieldI'd be very interested in hearing an answer to this too!#2018-06-0817:58bmaddyHere's a spec puzzle for someone. I'm trying to get a spec that defines something like this:
([-7806 "rBQUP"] :_T-7/G0X)
I thought I would need to nest calls to s/cat to do it, but those seem to flatten or something:
> (gen/generate (s/gen (s/cat :int int? :str string?)))
(-61568 "66420tRep6jHRW6q07x647hV6qE6q")
> (gen/generate (s/gen (s/cat :pair (s/cat :int int? :str string?) :keyword keyword?)))
(-7806 "rBQUP" :_T-7/G0X)
How do I get a collection of two items, where the first is a collection of one int and one string, and the second is a keyword?
#2018-06-0818:19taylorlike hiredman said below, this should work: (gen/generate (s/gen (s/cat :pair (s/spec (s/cat :int int? :str string?)) :keyword keyword?))) just to prevent the "flattening" of the nested regex specs#2018-06-1208:14djtangolate to the party, s/tuple isn't a regex spec so you can nest it as you like:
(s/valid? (s/tuple (s/tuple int? int?) keyword?) [[1 2] :a])
#2018-06-0818:09hiredmanwrap (s/spec ...) around the inner cat I think#2018-06-0818:14hiredmannested regex specs sort of collapse into a single regex spec (nested cats become subsequences within the outer sequence) so you have to insert something to tell spec where to stop collapsing them, and I think it is wrapping with s/spec, but it has been a while#2018-06-0819:37bmaddyThat worked! Thanks @hiredman and @taylor! šŸ˜„#2018-06-1207:36kurt-o-sysI'm using cljs-spec but I have issues with #objects...:
Call to #'... did not conform to spec:
In: [0 :data :from] val: #object[b 2018-06-11T00:00:00.000+02:00] fails spec: :week.data/from at: [:args :arg-0 :data :from] predicate: vector?
:cljs.spec.alpha/spec  #object[cljs.spec.alpha.t_cljs$spec$alpha40486]
:cljs.spec.alpha/value  ({:data {:from #object[b 2018-06-11T00:00:00.000+02:00]}})
:cljs.spec.alpha/args  ({:data {:from #object[b 2018-06-11T00:00:00.000+02:00]}})
:cljs.spec.alpha/failure  :instrument
so, what I try to spec is #object[b 2018-06-11T00:00:00.000+02:00].
#2018-06-1207:37kurt-o-syshow to spec that from-field?#2018-06-1208:18djtangohow robust do you want the spec? Does that date object contain a "type" field? If you want to check if the from field only is valid, you could pass it back into the date library to parse the string to check it's a valid time stamp?#2018-06-1208:24kurt-o-systhere is no type-field (apparently) - I'm using a js-library for datetime handling (need timezones, and there are not many options). so, checking if the from field is a valid datetime would do...#2018-06-1208:24djtangowhat's consuming the date object further downstream?#2018-06-1208:25djtangoIf it's internal to your project, Is it good enough to spec it to be valid only for the code you know is going to use it?#2018-06-1208:26kurt-o-sysother functions that do operations on it, like .plus etc...#2018-06-1208:26kurt-o-sysyeah, that's good enough...#2018-06-1208:27kurt-o-sysfor dev, I just want to be sure I get an kind of datetime object, on which I can perform .plus and .minus operations (or something like that).#2018-06-1208:27kurt-o-sysNow, I'm using any?, but that too generic šŸ™‚.#2018-06-1208:29djtangoah, fair. You could store your own representation of the date object as a map, then use the library only for doing manipulations (then translate it back to your date representation)#2018-06-1208:31kurt-o-sysright... could do. that will be quite a lot of translations - not sure about performance impact here. But I may just try to do a .plus-operation on it and check if it works, or something similar. That may do as well... will try. Thx!#2018-06-1208:35djtangogood luck#2018-06-1208:57kurt-o-sysworks fine, like it:
(s/def :luxon/plus #(try (.plus % #js {:days 1}) true (catch :default e false)))
(s/def :luxon/setZone #(try (.setZone % "Europe/Hamburg") true (catch :default e false)))
(s/def :luxon/startOf #(try (.startOf % "day") true (catch :default e false)))
#2018-06-1208:58kurt-o-sys(just testing if one can perform some operations, which should give a good enough indication that it's working)#2018-06-1208:24roklenarcicis there a way to feed existing explain data to expound?#2018-06-1211:57bbrinckYou can call the expound printer directly:
(let [d (s/explain-data (s/coll-of int?) [1 2 ""])]
  (expound/printer d))
or
(let [d (s/explain-data (s/coll-of int?) [1 2 ""])
       printer (expound/custom-printer {:show-valid-values? true})]
   (printer d))
#2018-06-1307:23roklenarcicthx#2018-06-1213:08mpenetcross posting here since it got no answer in #clojure#2018-06-1213:16Alex Miller (Clojure team)If you use s/def with a resolvable symbol, the symbol itself is the spec registry key (this is his function specs are stored). So, not a bug, and matches the docs afaict#2018-06-1213:21mpenetnot sure I follow, do you have an example?#2018-06-1213:30Alex Miller (Clojure team)maybe I misunderstand what you are asking about, itā€™s not clear to me what you think the bug is#2018-06-1213:31Alex Miller (Clojure team)the purpose of resolvable symbol k in that docstring is to support registering function specs named by their resolvable symbol. the example given is doing something other than that, which is not what it is for#2018-06-1213:33mpenetok:#2018-06-1213:33mpenet(def x string?) (s/valid? x "sdf")#2018-06-1213:33mpenetsomething like this then#2018-06-1213:33mpenetyeah that makes more sense#2018-06-1213:33Alex Miller (Clojure team)no, not that#2018-06-1213:34mpenetI still don't get it then šŸ˜›#2018-06-1213:34mpenetsince we're talking about the first arg to s/def it's a bit odd#2018-06-1213:34Alex Miller (Clojure team)x here should be a symbol referring to a var that is a function, to declare a function spec for x#2018-06-1213:36Alex Miller (Clojure team)(s/fdef clojure.core/symbol? :args (s/cat :x any?) :ret any?) => (s/def clojure.core/symbol? (s/fspec :args (s/cat :x any?) :ret any?))#2018-06-1213:37Alex Miller (Clojure team)the first is a shorthand for the second#2018-06-1213:37mpenetoh ok#2018-06-1213:37Alex Miller (Clojure team)like, literally s/fdef is a macro that does that#2018-06-1213:38Alex Miller (Clojure team)the registry has keys that are either keywords (pointing to data specs) or symbols (for function specs on the function on that symbol)#2018-06-1216:19kvltI have written a couple of fdef specs for my functions. What is the best way to test that in a test file? I'm looking at defspec and prop/for-all but the path forward doesn't seem obvious#2018-06-1216:21taylorYou can instrument the function to assert arguments conform to the :args spec on each call. You can also check functions to assert that it's returning valid values for randomly generated inputs.#2018-06-1216:22taylorthere's also exercise-fn#2018-06-1216:29kvlt@U3DAE8HMG: Thanks for the response. Are you suggesting something along the lines of:
(deftest some-test
  (every? true? (map #(get-in % [:clojure.spec.test.check/ret :result])
                     (stest/check `sut/my-test {::stc/opts {:num-tests 1}}))))
#2018-06-1216:32taylorthat should work, although you probably want more than 1 test, and I believe there's a helper function to make the stest/check result easier to examine#2018-06-1216:36kvltThe problem with this is that it won't give me any information about why it failed#2018-06-1218:10bbrinck@U08UTJ5PB maybe something like the following (untested)?:
(deftest some-test
  (let [results (stest/check `sut/my-test {::stc/opts {:num-tests 1}})]
  (is (every? true? (map #(get-in % [:clojure.spec.test.check/ret :result] results)
       (stest/summarize-results results))))
#2018-06-1218:10bbrinckor something like: https://gist.github.com/Risto-Stevcev/dc628109abd840c7553de1c5d7d55608#2018-06-1218:11bbrinckhttps://gist.github.com/jmglov/30571ede32d34208d77bebe51bb64f29#2018-06-1218:13bbrinckFWIW, expound can also format the check results. https://github.com/bhb/expound#printing-results-for-check . You could call explain-results-str for the 2nd arg to is#2018-06-1414:40rapskalian@U08UTJ5PB here is how I've done some generative testing in the past. Works very well: https://github.com/seesawlabs/digraph/blob/master/test/digraph/core_test.clj#2018-06-1216:20guymaybe https://clojure.org/guides/spec#_instrumentation_and_testing ?#2018-06-1216:32kvltI've been looking at that, it's still not clear to me how I run these in an automated test environment#2018-06-1217:20bortexzI have a message coming from an API in a format like '(a b c d e f), and Iā€™d like to propagate that message through a channel to my program with a proper format in a map, with keys for each of the properties that come in the list. I am using spec to conform the message, so I have something like (s/def msg (s/cat :a number? :b number?ā€¦)) and then when I call conform I do have a map like {:a a, :b b, :c c, ā€¦}, which is the format I want already (a map). Okay now I have a format I like to pass it on to the rest of the program, except Iā€™d prefer to have namespaced keys, but s/conform does not seem to put namespaced keys even if they are specified as keys or values in the s/cat. Is there any other way to make this happen? Is this a valid approach to ā€œtransformā€ messages from the outside world to the inside language? It is really helpful as it is, but I am not sure it is the preferred way to do it. What other way then? How do you usually do it? Thanks!#2018-06-1218:02taylors/conform + s/cat does seem to preserve qualified keyword tags for me:
(s/conform (s/cat :foo/bar int? :what/ever string?) [1 "!"])
=> {:foo/bar 1, :what/ever "!"}
#2018-06-1218:17bortexzOh, youā€™re right, it does for me too. I got confused because proto-repl prints it out like #:bitfinex-clj.spec.data{:ts 1, :open 2, :close 3, :high 4, :low 5, :volume 6}, but I have checked and they are namespaced.#2018-06-1218:27taylorya that's just the shorthand for when all the keys in the map share the same namespace#2018-06-1218:34bortexzAnother question: imagine I have all the namespaced keys already as predicates previously defined, so I can reuse them. Anyway to make this to not repeat each key? (s/def a (s/cat ::a ::a ::b ::b ā€¦))#2018-06-1218:35taylorthe only solution that comes to mind is writing a macro that outputs a s/cat like you want it, but before I did that I'd probably reconsider doing the transformations outside spec, maybe before/after conforming#2018-06-1218:37taylorthis seems like it might lend itself more naturally to a s/keys spec, and you could just handle the transformation before you conform (using zipmap or something)#2018-06-1218:40bortexzhmā€¦So the thing is the messages can come in different shapes and I use spec and s/or to tell me which type of message it is. But I probably can, instead of conform, to just check which message comes in without keeping the transformation, and then do the transformation outside. On the other hand, it seems so much related that it feels everything should be together (the same keys used by s/cat and zipmap, etcā€¦), also because conform already does the transformation I want#2018-06-1218:43bortexzAnyway, Iā€™ll have a rethink about the current structure I am using. Thanks @U3DAE8HMG šŸ™‚#2018-06-1218:48taylornp, here's a macro that I think does what you want:
(defmacro kat [& ks]
  (let [ks' (mapcat #(repeat 2 %) ks)]
    `(s/cat 
#2018-06-1312:06ackerleytngis there a way to write and use stest/instrument or stest/check on a function in a record?#2018-06-1312:07mpenetnope, if you have to do that you need to wrap it#2018-06-1312:07mpenet(which is not free)#2018-06-1312:08ackerleytngwhat do you mean by wrap it with another function?#2018-06-1312:09mpenet
(defprotocol IFoo
  (-foo [x]))

(def foo -foo)
(s/fdef foo :args (s/cat))
#2018-06-1312:10mpenetI don't think you can spec and instrument a protocol fun otherwise#2018-06-1312:40ackerleytngthanks!#2018-06-1411:40borkdudewhatā€™s the way out here, alter-var-root? https://github.com/bhb/expound/issues/19#2018-06-1412:01Alex Miller (Clojure team)set! will only work on a dynamic var if you are in the dynamic scope of a binding call#2018-06-1412:01Alex Miller (Clojure team)Repls like clojure.main establish this#2018-06-1412:02Alex Miller (Clojure team)So you either need to use binding or modify the root value with alter-var-root#2018-06-1412:09borkdudeclear. Is there also something like *print-length* but for strings?#2018-06-1412:11Alex Miller (Clojure team)No#2018-06-1412:14Alex Miller (Clojure team)You could override the print-method for String, although that might have some adverse consequences #2018-06-1412:15borkdudeprobably not a good idea šŸ™‚ the reason I ask is that a spec error (with expound) flooded my emacs buffer so bad, I had to forcequit itā€¦#2018-06-1413:04bbrinck@borkdude although it elides potentially useful information, you can configure expound to not print out ā€œfailing specā€ info, which would likely shorten the output considerably.#2018-06-1413:45alex-dixonIs there an idiomatic way to check if a spec exists?#2018-06-1413:46alex-dixoni.e. given a namespaced keyword how to check if a spec is registered for it#2018-06-1413:48taylor#2018-06-1413:51alex-dixonThanks @taylor#2018-06-1413:53alex-dixonAlso spec is on cljdoc https://cljdoc.xyz/d/org.clojure/spec.alpha/0.1.143#2018-06-1413:57alex-dixon@taylor Iā€™m getting a compiler exception for an unknown spec instead of a keyword#2018-06-1413:57alex-dixon@taylor Iā€™m getting a compiler exception for an unknown spec instead of a keyword#2018-06-1413:57taylorcan you paste the exception here?#2018-06-1413:58tayloroh sorry, I think I tested this with a predicate function and not a keyword#2018-06-1413:59alex-dixonnp#2018-06-1413:59taylors/form does throw an exception if you give it a ::keyword that has no registered spec#2018-06-1414:00alex-dixonOk. Maybe Iā€™ll just implement with try catch#2018-06-1414:04tayloroh there's s/get-spec#2018-06-1414:09taylor(def registered? (comp some? s/get-spec))#2018-06-1414:11alex-dixonOh dā€™oh#2018-06-1414:11alex-dixonPerfect. Thanks lol#2018-06-1414:11taylorI didn't know this existed either :man-shrugging: never had to look up a spec like that#2018-06-1414:17alex-dixon
(defn registered-spec? [x]
  (and (qualified-keyword? x)
       (some? (s/get-spec x))))
Yeah. Working on something where a macro may receive a namespaced keyword with an associated specā€¦so hopefully this all works out at compile time. Unfortunately have to do actual work now. Thanks a lot for your helpā€¦was going to hack something in but I feel way better with this
#2018-06-1420:03noisesmithgiven that macro expansion is a simplification step before making bytecode, you either end up with a much more complex compiler or taking the huge performance hit of having an interpreted language rather than a compiled one#2018-06-1420:04noisesmith(in order to have first class macros)#2018-06-1420:04noisesmithhttps://en.wikipedia.org/wiki/Fexpr#2018-06-1420:04johanatan@noisesmith i'm not arguing for first-class macros (although that would be nice). i'm arguing for not using second-class macros in libraries#2018-06-1420:04johanatanand forcing their proliferation onto users.#2018-06-1420:05noisesmithoh, I misinterpreted you then#2018-06-1420:06johanatan[sorry, yea, it wasn't clear from without the context]#2018-06-1420:27rapskalianCan I generate specs en masse? Something like:
(doseq [k my-keys]
  (s/def k (s/double-in :infinite? false :NaN? false :min 0)))
#2018-06-1420:39taylorit's registering the specs to the literal symbol k#2018-06-1420:40rapskalianThis was my gut feeling of what was going on. Any way to "force" it to evaluate the keyword? At this point I can see the value of writing them out by hand, but still curious...#2018-06-1420:40rapskalianSmells like a macro...#2018-06-1420:41donaldballI have done:
(doseq [[spec-kw field-spec desc] fields]
  (eval `(s/def ~spec-kw ~(build-internal-spec field-spec))))
#2018-06-1420:41donaldballI donā€™t even feel bad about it.#2018-06-1420:49tayloryeah you could use a macro to spit out a bunch of s/def's for your keywords:
(defmacro alias-kws [ks]
  (let [defs (map #(list 's/def % ::my-spec) ks)]
    `(do 
#2018-06-1421:17rapskalian@U04V4HWQ4 interesting...using eval does do exactly what I'm after. It's a bit of a hack, but it beats having to repeat these massive lists of keywords over and over again...#2018-06-1421:20donaldballIt would be nicer if spec had an explicit affordance for this use case, but I donā€™t really think itā€™s abusive to take advantage of the fact that clojure is a lisp šŸ˜› . I wouldnā€™t reach for this often, mind, but in my case I had a gigantor table of fields for a structured file and this lets me leverage an edn representation for spec validation as well as actually e.g. generating serializers#2018-06-1421:23rapskalianVery cool, I agree that there are definitely valid use cases. Not to mention that the prospect of generating specifications from concretions is extremely interesting in its own right šŸ¤” Appreciate the help (@taylor as well, thank you)#2018-06-1420:27rapskalianFor some reason I'm getting "unable to resolve spec"...#2018-06-1420:28johanatan@cjsauer most (all?) attempts to meta-program spec will fail-- it's apparently intended to be hand-written in the first-order only.#2018-06-1420:30rapskalianOoph...ok, thanks @johanatan#2018-06-1420:39rapskaliangenerates all specs directly in emacs buffer and calls it a day#2018-06-1420:39rapskalianI ā¤ļø Clojure#2018-06-1420:46seancorfield@johanatan as noted, spec is currently built for human written code/input. There are plans for a more programmatic API which will open up meta-programming. I think building the latter first would have led to a lot more breaking changes (since folks would have built code against the API). Make sense?#2018-06-1506:05ackerleytngwhat's the difference between gen/hash-map and just using gen/fmap (fn [a] {:a a}) ...?#2018-06-1514:33bbrinck@ackerleytng hash-map can take multiple key-value pairs
#2018-06-1514:34bbrinck(whereas in your example, the function you pass to fmap only takes a single value)#2018-06-1514:35bbrinckunder the hood, hash-map does use fmap with zipmap to combine these into a map, which is just a more general version of your code above: https://github.com/clojure/test.check/blob/729de024f245c07011a2cd2fcaad04bcd90a223d/src/main/clojure/clojure/test/check/generators.cljc#L645-L646#2018-06-1516:23ackerleytngi see, thanks!#2018-06-1518:53kennyIs there a way to only check the keys explicitly written in a Spec? i.e. No implicit validation of keys. I have a case where I need to validate partial pieces of a larger map.#2018-06-1518:58noisesmith@kenny when would spec implicitly validate your key?#2018-06-1518:59kennyAny map implicitly validates all keys against existing specs.#2018-06-1518:59noisesmithselect-keys before validating?#2018-06-1518:59kennyCould totally do that but then I need to write the set of keys twice.#2018-06-1617:23alex-dixonIf I have a spec that calls conform on data that may have been specced by other users, should I expect to receive their conformed results in my conformed results?#2018-06-1722:58taylorif their specs are loaded into the spec registry in your process, I think so#2018-06-1708:46misha@kenny you can get it fairly easily from s/form#2018-06-1714:45kenny@misha Thatā€™s what I ended up doing šŸ˜‰ https://github.com/ComputeSoftware/spec-keys#2018-06-1800:53metametadataHi. I've noticed that CLJS impl of explain prints more stuff than Clojure. Is it on purpose? CLJS:
=>  (s/explain ::x 23)
val: 23 fails spec: :cljs.user/x predicate: string?
:cljs.spec.alpha/spec  :cljs.user/x
:cljs.spec.alpha/value  23
Clojure:
=>  (s/explain ::x 23)
val: 23 fails spec: :cljs.user/x predicate: string?
#2018-06-1811:26zcljIf I would like instrument to check my specs, what would be the preferred way of spec:ing a multimethod? Can I use a multispec with the same dispatch fn or do I need an indirection fn with a spec in each defmethod?#2018-06-1811:48Alex Miller (Clojure team)You can make the defmulti dispatch function explicit and spec that#2018-06-1817:58zcljIn my case each defmethod gets called with a map and the map should have different contents for each method. As I read your answer it would be the same spec for all methods?#2018-06-1812:10misha@kenny I wonder if qualified symbols can be replaced like this:
(case form-sym
      (clojure.spec.alpha/keys
        cljs.spec.alpha/keys) ...
;=>

    (case form-sym
      `s/keys ...
#2018-06-1812:10mishagiven you already required [clojure.spec.alpha :as s]#2018-06-1812:11mishathen, getting rid of alpha would be just 1 line :require update#2018-06-1814:39valeraukoI recall that spec was inspired in part by RDF. Is there any recommended way for describing XML structures?#2018-06-1814:51Alex Miller (Clojure team)xml represented by what data structure?#2018-06-1814:53valeraukoI'm using clojure.data.xml, so whatever that uses internally.#2018-06-1814:53valeraukoIts sexp-as-element is the closest I've seen it get to simple data structures, but even that seems to be pretty complex to spec#2018-06-1814:54Alex Miller (Clojure team)seems pretty straightforward?#2018-06-1814:55valeraukoI'm afraid I don't follow#2018-06-1814:55Alex Miller (Clojure team)there are at least two formats there - the input and output of sexp-as-element. they both seem pretty straightforward to spec if you want to#2018-06-1814:56valeraukothe output's clojure.data.xml.Elements, right?#2018-06-1814:56Alex Miller (Clojure team)Element is a defrecord, so you can just spec it as a map with 3 known keys#2018-06-1814:57Alex Miller (Clojure team)the sexp form is just structured vectors and can be specā€™ed with s/cat etc#2018-06-1814:59Alex Miller (Clojure team)oh, there actually are specs already in data.xml#2018-06-1814:59valeraukoreally?#2018-06-1814:59Alex Miller (Clojure team)https://github.com/clojure/data.xml/blob/master/src/main/resources/clojure/data/xml/spec.cljc#2018-06-1814:59Alex Miller (Clojure team)thatā€™s for the element forms#2018-06-1815:02valeraukoit's not in the stable release yet though, is it?#2018-06-1815:03valeraukoi see you bumped its clojure dependency up to 1.9 when specs were introduced#2018-06-1815:04valeraukoas you pointed out i can just spec them as maps with :tag :attrs and :content#2018-06-1818:02lwhortonif I want to declare a with-gen ::spec generator-fn, generator fn needs to be a fn that returns a generator. how does one create a generator thatā€™s simply a function, without using fmap or any of the other test.check.generators?#2018-06-1818:02lwhortonfor example I just want a ā€œgeneratorā€ that invokes (random-uuid) where the fn itself is already random and doesnā€™t need any randomness from the generator scaffolding#2018-06-1818:05lwhortoni had some garbage like
(s/exercise ::p/id 10 {::p/id (fn [] (gen/fmap #(random-uuid) (s/gen (s/int-in 1 10))))})
but this is abusing fmap to just give me access to a fn I can call that will return a value thatā€™s wrapped in the proper generator
#2018-06-1818:08guymaybe (gen/return .. ) ?#2018-06-1818:08guy(gen/return (random-uuid)) maybe?#2018-06-1818:09guyhttps://clojure.github.io/test.check/generator-examples.html
(def int-or-nil (gen/one-of [gen/int (gen/return nil)]))
(gen/sample int-or-nil)
;; => (nil 0 -2 nil nil 3 nil nil 4 2)
#2018-06-1818:09guythats using gen/return to always return nil#2018-06-1818:09guyI'm not entirely sure its what you are looking for though šŸ˜ž#2018-06-1818:10lwhortonthat seems like the same idea as abusing fmap and an existing gen (int-in), but with different fns. it just feels like thereā€™s a built-in fn like gen/return for this use case, right?#2018-06-1821:13Alex Miller (Clojure team)using external sources of randomness prevents shrinking and repeatability so is discouraged#2018-06-1822:45gfredericksif you want to do the discouraged thing, fmap will work and return won't.#2018-06-1909:41mpenetyou can mix both to limit the awful (g/fmap rand-uuid (g/return nil)) I think#2018-06-1909:41mpenetbut it's still bad#2018-06-1914:36lwhortoni see. so if you arenā€™t using the generatorā€™s api test.check canā€™t work its magic?#2018-06-1917:08favilatest check needs to control all sources of entropy#2018-06-1917:08favila(if you want the other features of generators eg strinking and repetition to work)#2018-06-1917:08Alex Miller (Clojure team)@lwhorton no, test.check uses the same api. the issue here is using external sources of randomness#2018-06-1917:10favilagen/uuid already exists, is that not ok for you?#2018-06-1920:45lwhortonoh, i didnā€™t notice that. i will check it out. my real gotchya was that i have an instance? check against cljs.core/UUID and wasnā€™t sure how to make that happen randomly.#2018-06-1922:09gfredericksThe builtin uuid generator should only generate instances of that type#2018-06-2005:41Oliver GeorgeI'm wondering how to avoid a REPL gotcha related to instrumentation. Since stest/instrument wraps existing functions you effectively lose instrumentation by redefining a function with defn at the REPL.#2018-06-2005:42Oliver GeorgeRe-running stest/instrument fixes this but would be annoying to do manually every time.#2018-06-2005:43Oliver GeorgeHow do you avoid losing instrumentation at the REPL?#2018-06-2007:27sundarj@olivergeorge define a defn+instrument macro?#2018-06-2008:11Oliver GeorgeAn instrumenting version of defn would be perfect but ideally it'd be seamless. #2018-06-2012:15gfrederickswhat tooling are you using? I'm often editing the files and running the tools.namespace refresh function, which means I can either have the call to instrument at the bottom of a given namespace, or else use the callback hooks in refresh#2018-06-2012:16gfredericksrelated, what's the purpose of instrumentation? just for running tests, or for interactive development?#2018-06-2012:21Oliver GeorgeInteractive development. Just starting on a re-natal mobile companion app to a clojurescript re-frame app. #2018-06-2012:23Oliver GeorgeFigwheel has an "always reload" option for namespaces but a simple cursive repl is the case I was considering. #2018-06-2012:24gfredericksI had the idea of monkeypatching defn, but I dunno how to accomplish that exactly in cljs#2018-06-2012:29gfredericks
(alter-var-root #'defn
 (fn [orig]
  (fn [& args]
   (let [form (apply orig args)]
    `(do ~form (st/instrument '~(ffirst (drop 2 args))))))))
something atrocious like that
#2018-06-2013:23Oliver GeorgeNo immediate joy but I'll give it some more time later.#2018-06-2014:28gfredericksyou'd have to somehow get that to run in the cljs compiler process also I realized it's possible that defn isn't even a macro in cljs; it could be a special form#2018-06-2016:20noisesmithI assume that do was meant to be doto?#2018-06-2016:22gfredericksI don't think so#2018-06-2016:22gfredericksI mean it definitely wasn't, and I believe it's right that way#2018-06-2016:22gfredericksI think st/instrument takes a symbol, not a var#2018-06-2016:34noisesmithoh, so the intention is to no longer return the var but instead return whatever instrument returns?#2018-06-2016:35noisesmithI am probably weird for doing this, but I have a repl workflow where I count on being able to call load-file and then (*1) to call the last var in the file#2018-06-2016:42noisesmithanyway all I was getting at was that the original return value of defn was being dropped#2018-06-2016:59gfredericksOh good point. Need a let #2018-06-2012:29gfredericksas long as it only runs in dev mode then it's not the worst thing in the world šŸ™‚#2018-06-2012:58Oliver GeorgeThanks I'll give that a try#2018-06-2013:20Oliver GeorgeI might be able to do something similar via a custom command in cursive. Looks like I can send something like this to the repl with a hotkey (untested):
~top-level-form
(instrument ~current-var)
#2018-06-2013:22Oliver GeorgeAnother approach would be using a patched version of clojurescript for development. (Must try and see how hard that is now that deps.edn does github shas)#2018-06-2104:26alex-dixonDoes anyone know if thereā€™s a proposal to add line numbers to spec error messages?#2018-06-2112:28Alex Miller (Clojure team)There should be line numbers included now for macro function spec errors#2018-06-2112:30Alex Miller (Clojure team)I think there was a ticket filed recently, or at least it came up, for the same with non macro function spec errors#2018-06-2215:09mpenet@alexmiller It seems spec might get some love given some jira activity. I know there are already a lot of things queueing on that, but do you think there's a chance we might get anything to allow to get relations between specs (like list all aliases for a registered spec, list all specs ancestors depending if it's a s/merged of others etc etc)?#2018-06-2215:09mpenetit's quite easy to add, but just curious if this is on the radar at all#2018-06-2215:28Alex Miller (Clojure team)Deep walk is of interest but I donā€™t think we will look at that until after Richā€™s next batch of impl changes#2018-06-2218:22bmaddyhas there been any indication on what the next batch of impl changes might be focused on? Even if only a general description?#2018-06-2219:41Alex Miller (Clojure team)Better for data-oriented functional construction#2018-06-2219:42Alex Miller (Clojure team)Due to other priorities for Rich, I think that will slip out a bit but weā€™re going to try to do some bug fixing in near term#2018-06-2316:01ackerleytngwhat's the difference between s/* and coll?#2018-06-2316:01ackerleytnglike if i do (s/def ks coll?), is that the same as (s/def ks (s/* any?))?#2018-06-2316:22Alex Miller (Clojure team)s/* is a regex op and will compose with other regex ops to describe the internal structure of a sequential collection#2018-06-2316:22Alex Miller (Clojure team)If you donā€™t have any structure, Iā€™d use coll-of#2018-06-2404:59ackerleytngthanks!#2018-06-2411:00martinklepschI would like to find out where a spec has been defined (source file, namespace etc.) ā€” is this possible?#2018-06-2412:44ackerleytnghmm sorry not sure about that...#2018-06-2413:04Alex Miller (Clojure team)Some specs retain meta, but generally no. There is a ticket about this#2018-06-2413:20ackerleytngI'm trying to test a function where the inputs are related#2018-06-2413:20ackerleytngit's actually a bit like update-in#2018-06-2413:20ackerleytngwhere ks has to be related to m#2018-06-2413:22ackerleytngI wrote a generator to generate m, and i want the result of that generator to be inputs to a function that returns another generator for ks#2018-06-2413:22ackerleytngthose already mostly work, but how would i supply gen-overrides in pairs?#2018-06-2512:56ackerleytngI got it!#2018-06-2512:56ackerleytngi'm now generating all the arguments to the function as a single list#2018-06-2512:57ackerleytnganother question! is there any "state" that carries over between instances of generators?#2018-06-2512:58ackerleytngsay i define (def string-gen (gen/string))#2018-06-2512:59ackerleytngif i use that in two different overrides, say {::foo (fn [] string-gen)} and {::bar (fn [] string-gen)}#2018-06-2513:00ackerleytngwould the two overrides generate strings independently of each other?#2018-06-2513:06guyI think the generators have seeds? which might be different each time?#2018-06-2514:26ackerleytngah yes i guess so#2018-06-2514:26ackerleytngalong those lines - will a more complex generator retain any sort of state between two invocations?#2018-06-2514:32favilaa top-level generator instance and all its "sub" generator instances share a common source of randomness; it's not shared with other instances#2018-06-2514:32guyunless you pass it a seed or something? i think#2018-06-2514:32guyI canā€™t quite remember#2018-06-2514:32guylet me google it šŸ‘#2018-06-2514:32favilayeah but even then you're regenerating the same randomness independently#2018-06-2514:33favilait's not shared#2018-06-2514:33guyahhhh#2018-06-2514:33guyyes you are correct#2018-06-2514:33guyšŸ‘#2018-06-2514:33favilaso if your overrides are part of a larger generator (e.g. you are generating examples for a top-level map and it has these two keys you override) then the generators share the same source of randomness#2018-06-2514:34favilaif these are independent uses of these generators, they do not share#2018-06-2514:40ackerleytnghmm then why do the overrides have to be wrapped in a function?#2018-06-2514:41ackerleytngit has to be {::foo/bar (fn [] (gen/return inc))} instead of just {::foo/bar (gen/return inc)} right?#2018-06-2514:41ackerleytngdoes wrapping it in a function kind of force a new generator to be returned or something? to avoid state carrying over?#2018-06-2514:51favila@ackerleytng wrapping in a function is to allow lazy invocation of the entire generator system; It's possible to use spec without a test.check dependency (e.g. in production) if you don't use any of the generation features.#2018-06-2515:00ackerleytngah i see, thanks!#2018-06-2514:51favilaif you didn't do this, spec would always require test.check even if you only wanted to use e.g. s/valid? s/conform s/unform etc#2018-06-2514:52favilathis is also why clojure.spec.gen.alpha exists: it wraps all the test.check functions and generators so that test.check itself is never required unless actually used#2018-06-2518:29blancedo you usually define generator together with your spec when the only use case for the generator is writing generative tests? It's nice that you can combine them with with-gen, but on the other hand I don't know if it's right to define something in src that's only needed for testing#2018-06-2519:02noisesmith@blance consider that a generator for foo will be needed for another library that uses foo, and test files are not transitive like source files are#2018-06-2519:03noisesmithso you end up hacking and adding the generator test files to exported classpath, or rewriting the generator#2018-06-2519:03noisesmithof those options, a generator in the source file is the least hackish#2018-06-2519:04blancethanks! that makes me feel better writing generator in along side with the spec itself#2018-06-2602:16athos@blance I was working on a library for managing generators separately from spec definitions a couple of months ago. If youā€™re interested, take a look at it! https://github.com/athos/genman#2018-06-2602:23blancethis looks promising! i'll definitely give it a try next time I write spec#2018-06-2621:03justinleequestion about spec-tools.data-spec. it seems like with data-spec there are now two namespaces: the one in the spec registry and the symbol i define in my own code. this makes sense with the one example in the readme because it is a map. but what about for code like
(def hit-box-spec
  (ds/spec ::hit-box [number?]))
It seems like both ::hit-box and hit-box-spec are now symbols that refer to a spec. Iā€™m just getting a bit confused about how to use this library with non-map specs.
#2018-06-2621:09noisesmith::hit-box is a keyword, specs resolve via a central registry that uses keywords. hit-box-spec is a var, containing whatever ds/spec returns (not necessarily the same value stored under that key in the spec registry, but maybe?)#2018-06-2621:31justinleeyea. thatā€™s how i understand it. i think iā€™m just confused why ds/spec doesnā€™t rely on the fact that it creates a side effect in the spec registry. iā€™m not sure why iā€™d want to also have a var in my namespace that points to a value that is apparently interchangeable (at least it is when using s/valid?#2018-06-2621:44noisesmithbut it also points to the literal data,that ds/ functions can use to create other specs, right?#2018-06-2621:44noisesmithI'd assume that's the reason#2018-06-2621:45justinleeI donā€™t think so. In the readme, they use a different var to point to the data used to create the spec. So they might write the above as (def hit-box-spec (ds/spec ::hit-box hit-box))#2018-06-2621:46justinleebut it might have to do with these transformations, which Iā€™m not using and donā€™t understand. that would make sense#2018-06-2622:05justinleehm what it is doing is more complex than this. my statement above that ::hit-box and hit-box-spec are interchangeable for valid? is wrong#2018-06-2622:57ikitommi@lee.justin.m side-effects are bad, I think the data-specs could benefit from a local-keys variant that doesnā€™t need to register the keys. data-specs are anyway not in align to the Spec filosophy of reusable keys, so why not go further down the roadā€¦#2018-06-2622:58ikitommibut for the original:
(ds/spec ::hit-box [number?])
can be written:
(ds/spec
  {:name ::hit-box
   :spec [number?]})
ā€¦ and as there are no keys, the :name can be omitted:
(ds/spec
  {:spec [number?]})
#2018-06-2622:59ikitommito register that, you can say:
(s/def ::hit-box
  (ds/spec
    {:spec [number?]}))
#2018-06-2622:59justinlee@U055NJ5CC ah i see. i should be using the map syntax.#2018-06-2622:59justinleeas for whether side effects are bad, thatā€™s just the way spec works, right?#2018-06-2622:59justinleeyou have to register your spec with the registry#2018-06-2622:59ikitommiright.#2018-06-2623:00justinleethe thing iā€™m not getting right now is this:
(def banana-spec
  (ds/spec ::banana {:id integer?
                     :name string?}))
(st/registry #".*banana.*"))
=> (:seekeasy.app-state$banana/name :seekeasy.app-state$banana/id)
#2018-06-2623:00justinleei see that the two subspec are registered, but why not the main banana spec?#2018-06-2623:00ikitommiif you try to put a map somewhere in the data-spec and donā€™t provide a :name, it will fail-fast:
(ds/spec
  {:spec [[[[[[[{:a int?}]]]]]]]})
; CompilerException java.lang.AssertionError: Assert failed: spec must have a qualified name
; (qualified-keyword? n), compiling:(data_spec_test.cljc:380:3)
#2018-06-2623:01ikitommigood question, not sure, maybe it should?#2018-06-2623:01justinleei just donā€™t even know how it works šŸ™‚#2018-06-2623:04justinleebasically i was expecting an unmunged :seekeasy.app-state/banana given that i provided it that as the name of the spec. although i guess thatā€™s consistent with the readme, now that i think about it. the only thing that got registered were the subspecs.#2018-06-2623:06ikitommiif there would be the local-keys, there would be no registering of any specs. that in mind, having a function called spec doing registration of the top-level would be bit odd.#2018-06-2623:07ikitommithere could be ds/def for thatā€¦#2018-06-2623:07justinleeoh i see what you mean#2018-06-2623:07ikitommiā€¦ but (s/def ::name (ds/spec ..)) is the way to do it now.#2018-06-2623:07justinleebecause spec does (s/def ...) so we would expect it to cause a side effect#2018-06-2623:08justinleebut here itā€™s just a function so we need to def it ourselves. okay. this is coming together for me.#2018-06-2623:08justinleeI guess I thought there was a separate data structure for the specs. But they are just stored on normal vars?#2018-06-2623:09justinleethe existence of the st/registry made me think this#2018-06-2623:09ikitommiall Specs are just values, you can store them in a Var.#2018-06-2623:09ikitommi(s/valid? string? "1")#2018-06-2623:10ikitommi(s/valid? (ds/spec {:spec [int?]}) [1 2 3])#2018-06-2623:11ikitommi.. unless you register them and get a name than can be used in s/keys. I think itā€™s the only one that requires a spec to be registered?#2018-06-2623:11justinleeahhhhh okay#2018-06-2623:11justinleedamn that is confusing šŸ™‚#2018-06-2623:12ikitommi(s/valid? (s/coll-of int? :into []) [1 2 3])#2018-06-2623:13justinleebasically, the thing that confused me is that if you do (s/def ::something ...) that will show up in the registry even it if isnā€™t meant to be used as a key but if you do (def something (ds/spec ...)) it doesnā€™t show up. but both work. i hadnā€™t appreciated the fact that nothing cares about the registry except for s/keys#2018-06-2623:16justinleebtw, thanks for spec tools. i really really really prefer the self-documenting format of schema, and now i have my cake and eat it too#2018-06-2623:21ikitommithanks! itā€™s kind of a roque lib, I hope the final version of spec will make much of it redundant.#2018-06-2623:13ackerleytngwhat types of functions do you guys normally spec? do you spec utility functions?#2018-06-2623:19justinleethe answer youā€™ll hear most commonly is ā€œat data boundariesā€ or something like that. e.g. reading data from a file or network or moving from one chunk of code to another, rather than doing it wholesale on every internal function#2018-06-2623:35noisesmithI'd put it as "system boundaries" rather than "data boundaries" but yes, exactly that#2018-06-2623:35noisesmithwhere system boundaries are has a lot to do with your design, but if your system is designed it should have some :D#2018-06-2623:36ackerleytngthanks šŸ™‚#2018-06-2623:36ackerleytngwon't your system boundaries keep changing?#2018-06-2623:36ackerleytngas you compose functions with functions#2018-06-2623:37noisesmithif your system boundaries are changing those weren't your system boundaries#2018-06-2623:37ackerleytnganother question! how do you define an fspec where you don't care about the name of the argument, but you care about the type?#2018-06-2623:37ackerleytngah thanks that sounds right haha#2018-06-2623:38noisesmiththe idea is that it isn't a system if you don't define some limit or border, that's really the first step. What the boundaries are, and which things cross them, should be one of the first things defined, and often you'll want to define things so it changes relatively rarely#2018-06-2623:39noisesmith@ackerleytng for the names of things in specs, the reason to have the names is for the error messages you get without a match. Otherwise the output of a failure turns into a soup of data types that isn't very helpful.#2018-06-2623:39ackerleytngoh! so it doesn't actually match against the name of the actual parameter?#2018-06-2623:41noisesmithif it is the thing I'm thinking of it's a series of name / type for each arg right?#2018-06-2623:41noisesmiththe name is used in generating the message, it doesn't have to match the arglist or anything#2018-06-2804:44ackerleytngthanks!#2018-06-2811:02gnlIntroducing Ghostwheel ā€“ it makes the writing, reading, refactoring, testing, instrumentation and stubbing of function specs easy; introduces a concise syntax for inline fspec definitions; helps keep track of side effects; and makes general debugging almost fun, with full evaluation tracing of function I/O, local bindings and all threading macros. https://github.com/gnl/ghostwheel#2018-06-2813:40andre.stylianosThis looks really, really nice! I will for sure give it a try soon#2018-06-2813:43gnlGreat, report back when you do!#2018-06-2812:24ackerleytnghow do i spec a function that has a variable number of arguments?#2018-06-2812:25gnlTry (s/* ...) https://clojure.github.io/spec.alpha/clojure.spec.alpha-api.html#clojure.spec.alpha/*#2018-06-2812:26ackerleytngah i think i see where my mistake is#2018-06-2812:27ackerleytngi'm actually trying to spec a function that may have a variable number of arguments#2018-06-2812:27ackerleytngso I should have an or#2018-06-2812:45ackerleytngwonder if anyone can help me understand this#2018-06-2812:45ackerleytngHow would i write a spec such that all these conform?#2018-06-2812:45ackerleytng
(defn- zzz
  [i] i)

(defn- zzz-args
  [i args] i)

(defn- zzz-var-args
  [i & args] i)

(s/conform ::helpers/test zzz)
(s/conform ::helpers/test zzz-args)
(s/conform ::helpers/test zzz-var-args)
#2018-06-2812:45ackerleytngI've tried
(s/def ::helpers/test
  (s/fspec :args (s/or :a (s/cat :itineraries (s/coll-of string?)
                                 :args (s/* any?))
                       :b (s/cat :itineraries (s/coll-of string?)
                                 :args any?)
                       :c (s/cat :itineraries (s/coll-of string?)))))
#2018-06-2812:46ackerleytngbut only (s/conform ::helpers/test zzz-var-args) conforms#2018-06-2814:33favilaThey can't all conform: these are incompatible function signatures#2018-06-2814:33favilayou can write a spec such that the args for any of those signatures passes, but you can't disambiguate their meaning#2018-06-2814:35favilanor can you work backwards: generated sample args from this spec won't work for all functions. zzz only works with :c and some :a for example#2018-06-2814:35favilaif you give it something matching :b it will fail#2018-06-2815:54justinlee@clojurians.net whew! youā€™ve been busy! iā€™ve just switched over to using spec-tools for its data-spec features that allow me to specify nested data types using data. I was going to try to get orchestra installed too so I can use their defn-spec, but your library looks like it provides that functionality and more. I think can can mix and match spec-tools with either of these. any pitfalls you can think of?#2018-06-2816:20gnl@lee.justin.m I honestly don't know. I have no experience with spec-tools and it is my (very superficial) understanding that they rely on some internal spec workings/APIs which are likely to change and break as spec matures and moves out of alpha, I remember reading a comment from Alex Miller to that effect, if I'm not mistaken. That's one of the reasons it hasn't been near the top of my list of new stuff to check out. Feel free to correct me if I'm mistaken and spreading FUD here, anyone. FWIW I can say that Ghostwheel is only using public APIs and not doing anything hacky at all. And if you do end up using it - feedback would be much appreciated! I've done a moderate amount of dogfooding, but there's always all kinds of things that tend to pop up in the wild.#2018-06-2816:21gnlOh and btw, Ghostwheel also supports orchestra for instrumentation.#2018-06-2816:22justinlee@clojurians.net thatā€™s true about spec-tools. it says so right in the readme. (doesnā€™t bother me though. code can always be changed šŸ™‚ )#2018-06-2816:23noisesmith> code can always be changed tell that to python3#2018-06-2816:24justinleeiā€™ve never kept up with python#2018-06-2816:25justinleeand at any rate, spec tools is a small library. if they break it, iā€™ll just not upgrade or iā€™ll rewrite my types. itā€™s not mission critical#2018-06-2816:25noisesmithpython3 was introduced in 2008. most of the community still uses python2 because they use libraries that are incompatible with version 3#2018-06-2816:26justinleeoh. well thatā€™s not exactly comparable. thatā€™s changing out the whole language šŸ™‚#2018-06-2816:27noisesmithif you are relying on the internals of spec, you need to change your code with updates to spec, and if you aren't careful your users need to change code too - the point here is the way that compatibility issues can fragment a community and get things stuck, and spec marks things as internal to avoid this kind of scenario#2018-06-2816:31justinleewell spec should have given me a usable tool instead of cramming its way of doing things down my throat. by releasing this thing as an official library schema lost its mindshare so iā€™m kind of forced to use spec and iā€™m doing my best. iā€™m working on a product so there are no consumers of my code.#2018-06-2816:23justinleeI basically just want to spec out the args inline using a macro. I think Iā€™ll try yours out because maybe I can get tracing too. I never could get debux to work with shadow#2018-06-2816:24gnlAnother thing that comes to mind regarding orchestra's defn-spec is that if I'm not mistaken it's not quite straightforward to do multi-arity functions with varying return specs between the different arities. With Ghostwheel it's just like writing single-arity functions and it automatically does the right thing.#2018-06-2816:25gnlAnd last but not least - migration to and from ghostwheel is easy, because you are not interspersing specs between the args, etc, you are basically just adding a single vector to the standard defn syntax#2018-06-2816:26gnlRegarding debux - I got it to work, sorta, and was considering using that, but I much prefer the clairvoyant+re-frame-tracer output, so I took those and pimped them a bit instead.#2018-06-2816:29justinlee@clojurians.net does all of this stuff print back to the repl? it seems like shadow-cljs defaults to the browser console. iā€™m just trying to figure out how it will work with all of this color output#2018-06-2816:29gnlIt prints to the browser console, js/console.log basically.#2018-06-2817:00metametadataThere's also this nice little alternative to orchestra - https://github.com/Provisdom/defn-spec. I like that it's friendly to Cursive IDE syntax highlighting.#2018-06-2817:11gnl@metametadata So is Ghostwheel! Cursive-friendly that is.#2018-06-3013:25kurt-o-sysif I have a map:
(def m #::{:item1 #::{:name         "whatever"
                      :description  "..."
                      :ref []}
           :item2 #::{:name "another one"
                      :description "...?"
                      :ref [:item1]})
can I write a spec so that all items in ::ref are keys of the containing map?
(s/def ::ref (coll-of ...??...))
or
(s/def ::ref (key-set ...??...))
#2018-06-3015:44ackerleytnghmm yes#2018-06-3015:44ackerleytngi think you can spec it as a predicate#2018-06-3015:44ackerleytngso#2018-06-3015:44ackerleytnglet me try it out on my side first#2018-06-3015:50ackerleytngsomething like that
(def m1 {:a {:b 1
             :ref [:a]}})

(def m2 {:a {:b 1
             :ref [:c]}})

(defn ref-contains-keys-of-containing-map [m]
  (let [ks (keys m)]
    (= (:ref (first (vals m))) ks)))

(s/def ::m (s/and map?
                  ref-contains-keys-of-containing-map))

(s/conform ::m m1)
(s/conform ::m m2)
#2018-06-3015:51ackerleytngm1 conforms, m2 doesn't#2018-06-3016:02kurt-o-sysright, thx... will try.#2018-06-3016:35kurt-o-sysoh right, but will see if I can make that more dynamic in some way... I'm fetching the map from outside. well... will see šŸ˜›#2018-06-3016:40kurt-o-sys@clojurians.net I like ghostweel - just trying it now - but it doesn't validate the return value of >defn. I seem to remember clojure.spec itself doesn't either, but orchestra does so... Any plans to add this to ghostweel as well (validation of return values of functions)?#2018-06-3016:43justinlee@kurt-o-sys have you configured to do so? thereā€™s a separate config option to turn that on#2018-06-3017:25andre.stylianos
;; Spec-instrument functions on namespace reload.
 ::instrument      false

 ;; Spec-instrument functions on namespace reload using orchestra,
 ;; which spec-checks the output in addition to the input.
 ::outstrument     false
#2018-07-0118:20kurt-o-sysI didn't... - I didn't know it was an option, but I do now šŸ™‚.#2018-07-0119:43andre.stylianosšŸ˜œ Glad to help!#2018-06-3017:25andre.stylianosfrom the README#2018-06-3018:25gnl@kurt-o-sys It's already in there! The option is ::g/outstrument.#2018-06-3018:28gnlDidn't read the other replies before I answered. By the way there's a bug in the current stable version, where ::g/instrument and ::g/outstrument require ::g/check to be enabled to have any effect. This doesn't really make any sense and is fixed in 0.2.2-SNAPSHOT, soon to be released as a stable 0.2.2 once some Figwheel issues have been cleared up.#2018-06-3018:28gnlDidn't read the other replies before I answered. By the way there's a bug in the current stable version, where ::g/instrument and ::g/outstrument require ::g/check to be enabled to have any effect. This doesn't really make any sense and is fixed in 0.2.2-SNAPSHOT, soon to be released as a stable 0.2.2 once some Figwheel issues have been cleared up.#2018-07-0118:22kurt-o-sysnice, thx! I did figure it out concerning that ::g/check issue, not about the ::g/oustrument šŸ˜›. Thanks a lot. Just one more question: how would you recommend instrumenting only during dev, but not for a prod build? (Just wondering what you consider as best practice)#2018-07-0118:30kurt-o-sysalso, I must be doing something wrong:
(>defn target
       "target function to be minified"
       {::g/instrument  true
        ::g/outstrument true}
       [in]
       [any? => double?]
       "wrong")
This should fail, right? (return value should be of type double, but it's a string). However, it doesn't fail at all. Adding the config to the ns doesn't help either:
(ns test-gw.core
  #:ghostwheel.core{:instrument  true
                    :outstrument true}
  (:require [clojure.spec.alpha :as s]
            [ghostwheel.core :as g
             :refer [>defn >defn- >fdef ? => | <-]])
 ...)
#2018-07-0118:33gnlDon't use ::g/instrument and ::g/outstrument together, only one will be used, in this case ::g/instrument. What you want here is only ::g/outstrument.#2018-07-0118:34gnlAnd regarding not instrumenting in production ā€“ just make sure that ghostwheel isn't enabled in your prod build config#2018-07-0118:35gnlNo way to do that in Clojure at the moment, so just don't do instrumentation there in production. šŸ™‚#2018-07-0118:46kurt-o-sysright, perfect :+1: :+1:#2018-06-3021:38gnlSpeaking of which ā€“ 0.2.2 is out: https://github.com/gnl/ghostwheel/blob/master/CHANGELOG.md#2018-07-0101:22codonnellDoes anyone know of an efficient way to generate a multi-spec value with a particular tag? (ie. without doing a such-that and hoping to get lucky)#2018-07-0106:09ackerleytng@codonnell what do you mean by a multi-spec value? have an example?#2018-07-0106:59ackerleytngIf i run a stest/check and hit an error, how do i inspect the value that was generated? my return value is a function. I fspecced it, and one of the inputs to that function, generated during stest/check is causing the spec to fail#2018-07-0112:39codonnell@ackerleytng
(s/def ::type #{::a ::b})
(s/def ::a-value pos-int?)
(s/def ::b-value keyword?)

(defmulti mymap-type ::type)
(defmethod mymap-type ::a [_]
  (s/keys :req [::type ::a-value]))
(defmethod mymap-type ::b [_]
  (s/keys :req [::type ::b-value]))
(s/def ::mymap (s/multi-spec mymap-type ::type))
;; How to generate a mymap value with tag ::a here without using gen/such-that
#2018-07-0113:18ackerleytngsomething like this?
(s/conform ::mymap {::type ::a
                    ::a-value 1})
(s/conform ::mymap {::type ::b
                    ::b-value :foo})

(map (partial s/explain-data ::mymap)
     (gen/sample (tgen/let [type_ (gen/elements [::a ::b])
                            a-value tgen/pos-int
                            b-value (gen/keyword)]
                   (conj {::type type_} 
                         (if (= type_ ::a)
                           {::a-value (inc a-value)}
                           {::b-value b-value})))))
#2018-07-0113:19ackerleytngtgen is clojure.test.check.generators#2018-07-0113:19ackerleytng[clojure.spec.gen.alpha :as gen]#2018-07-0113:20ackerleytngpos-int generates 0 sometimes, hence the inc#2018-07-0113:21ackerleytngthis talk was super helpful for me https://www.youtube.com/watch?v=F4VZPxLZUdA#2018-07-0113:24gfrederickss-pos-int doesn't generate zeros, fyi#2018-07-0113:25codonnellI'd prefer not to build up the value manually like that, since the actual spec I'm working with is much more complex than the toy example I posted above.#2018-07-0113:26codonnellI'll definitely check out that talk, thanks.#2018-07-0113:44ackerleytng@gfredericks thanks for giving that talk!! I loved the pictures too#2018-07-0113:45ackerleytng@codonnell hmm...#2018-07-0122:19Andreas Liljeqvist@codonnell (gen/sample (s/gen ::mymap {::type #(gen/return ::a)}))#2018-07-0122:20Andreas LiljeqvistIt would have been, but there is a bug with providing a generator for the dispatch key - See my bug report and patch https://dev.clojure.org/jira/browse/CLJ-2311#2018-07-0122:31codonnellthanks @andreas862, that is exactly what I was looking for. Hopefully your patch is accepted!#2018-07-0200:13ackerleytngs/gen allows you to override a specific generator?#2018-07-0200:13ackerleytngI see, thanks!#2018-07-0209:24otfrommorning#2018-07-0209:26otfromI have a map where one of the namespaced keys ::foo for example, in the whole program can be one of 3 values #{"foo" "bar" "baz"}, but in one part of the program can only be #{"foo" "bar"} because I've filtered out "baz". Is there a way of expressing this in spec? I've been looking around and I haven't found anything (might be morning brane and solved by more coffee, but I'm not sure)#2018-07-0711:53rickmoynihanInterestingā€¦ I think this is awkward because itā€™s expressing something thatā€™s brittle and contrary to the growth ideals of spec? i.e. saying ā€œthis set must have only these keysā€ is analogous to saying ā€œthis map must have these keys and no othersā€. The solution with maps in this situation is often to use select-keys in your function, so the function is robust to inputs with arbitrary other keys as it will ignore them. Could the solution in your situation be to intersect the set-argument with #{"foo" "bar"} so anything else is ignored?#2018-07-0815:59otfromhmm.. I hadn't thought about the growth ideals properly when thinking about this. It makes me think that the envelope should be very permissive and that I should create new maps for the payload with new specs going through the rest of the system. I think that is why I keep asking these questions as spec affords some forms and punishes others (for good reasons such as growth), but it can be difficult to see it all the time and design accordingly.#2018-07-0907:58rickmoynihanYes, if you build specs where you find yourself trying to spec ā€œand nothing elseā€ youā€™ll struggle. Obviously I donā€™t really know what youā€™re doing, but your new ideas seem to make perfect sense from a general architectural perspective i.e. the transport layer of an architecture typically shouldnā€™t know about specifics of the application/content layer.#2018-07-0908:58otfromI often find if I'm struggling against one of the affordances of something in clojure it is because I'm not thinking about it very well. I think this might be one of those cases. Thx!#2018-07-0914:17carocad@U0525KG62 I have had that case as well though not for part of the program but rather for part of an api response. My approach was to split the set into the two possible solutions and then use s/or and s/merge to express the parts that are common and differentiate the other ones#2018-07-0914:18carocadthat might sound too abstract so here is an example: https://github.com/hiposfer/kamal/blob/master/src/hiposfer/kamal/specs/directions.clj#L55#2018-07-0210:07andre.stylianosDepends on what you want out of it I think, and what those things mean in your code. You can make ::filtered-foo which is #{"foo" "bar"} and ::foo is now s/or of ::filtered-foo or #{"baz"}.#2018-07-0210:09andre.stylianosJust avoid having ::foo itself mean two different things depending on context, as that breaks specs being globally consistent#2018-07-0213:03otfromhmm... makes me want to use unqualified keys for things like that then#2018-07-0213:16otfromI basically have a lot of "type fields" and often filter down to particular types in certain parts of the system and would like to constrain things in those sections (esp things like generative testing)#2018-07-0213:16otfromand different affordances drive different design styles in maps#2018-07-0213:38andre.stylianosYou could also leave ::foo as being #{"foo" "bar" "baz"} and in specific places just say that ::foo.subset is (s/and #(not= % "baz") ::foo)#2018-07-0214:21otfromyeah, it just means that the key has to change going through so any code that does work on the supertype wouldn't be pointing at the right key.#2018-07-0214:22otfromit just makes an envelope and payload style tricky to do (as payload might have loads of types)#2018-07-0214:37otfromso this is tricky to model in spec then
{::id <some uuid>
 ::timestamp <some timestamp>
 ::payload <a number of different things that are maps that vary depending on domain>}
#2018-07-0711:55rickmoynihanisnā€™t this the use case for a multi-spec on ::payload?#2018-07-0214:39otfromif you want to be able to constrain ::payload later as you are only dealing with one of the domains.#2018-07-0214:40otfromI suppose you could leave ::payload as just a map#2018-07-0214:40otfromand deal with the payload maps separately#2018-07-0214:45andre.stylianosWell, you can either have ::domain-1/payload, ::domain-2/payload and so on, or you could spec ::payload with s/or, conform it and check if the conformed value is the branch you expected#2018-07-0214:47andre.stylianosdo keep in mind that I'm no clojure.spec expert, just brainstorming a bit :man-shrugging:#2018-07-0214:58otfromor have a non-namespaced :payload and define it as :req-un as needed#2018-07-0214:59seancorfieldWell, you can use namespaced variants of :xxx/payload and still define it as :req-un -- that way you can see which version you're working with -- and still just use :payload in the map.#2018-07-0215:03otfromI'm not sure I understand you @seancorfield. Does that give me the ability to say "this bit of code here only works with this domain of what might go in payload rather than all the domains"#2018-07-0215:14seancorfieldYou can have multiple specs with the same set of (unqualified) keys -- but each spec can use different definitions for those keys by using different qualified versions of a key.#2018-07-0215:15seancorfield(s/keys :req-un [:foo/bar]) and (s/keys :req-un [:quux/bar]) -- the actual map has :bar in both cases but the specs are distinct.
#2018-07-0215:25otfromcool. That was what I thought. That you could basically redefine what the *un*qualified keys meant where you wanted, but that you couldn't do that with qualified keys in a map. I didn't know about the :req-un sugar you had their tho (or had forgotten)#2018-07-0215:32guyI mean you still have to s/def the keys right#2018-07-0215:33guy(s/def :quux/bar string?) and (s/def :foo/bar pos-int?) for example#2018-07-0215:33guyu canā€™t just use :req-un to redefine them#2018-07-0215:33guyiā€™m pretty sure#2018-07-0215:33otfromyeah, but I can do that per domain :domain1/bar :domain2/bar#2018-07-0215:33otfrometc#2018-07-0215:33guyye šŸ‘#2018-07-0215:34guy>That you could basically redefine what the *un*qualified keys meant where you wanted, Just wasnā€™t sure what u meant here#2018-07-0215:34otfromwhich is exactly what I want to do here#2018-07-0215:34guycool cool#2018-07-0215:34guyšŸ‘#2018-07-0215:34otfromfeels like a bit of a cheat, but it does work with what spec affords#2018-07-0215:34otfromand I'm all about the affordances#2018-07-0215:34guyitā€™s in the spec guide if it makes you feel better šŸ˜„#2018-07-0215:34otfromis it?#2018-07-0215:34guyya i believe so#2018-07-0215:35guyhttps://clojure.org/guides/spec#_entity_maps#2018-07-0215:35guyscroll a bit down#2018-07-0215:35guy
Much existing Clojure code does not use maps with namespaced keys and so keys can also specify :req-un and :opt-un for required and optional unqualified keys. These variants specify namespaced keys used to find their specification, but the map only checks for the unqualified version of the keys.
#2018-07-0215:39guy:+1:#2018-07-0215:41otfromthx#2018-07-0318:28fominokHi there! I suppose this is a right place for questions and I have one: is it possible to have references with spec generators? Like this: I have an author spec and a book spec, book has an author_id, can I generate one author and n books with proper id?#2018-07-0318:31noisesmithI think gen/fmap could do that#2018-07-0320:00taylor@fominok you could also use test.check's let macro like this:
(s/def ::author-id uuid?)
(s/def ::name string?)
(s/def ::author (s/keys :req-un [::author-id ::name]))
(s/def ::book (s/keys :req-un [::author-id ::name]))
(s/def ::books (s/coll-of ::book))

(gen/sample
 (gen/let [author (s/gen ::author)
           books  (s/gen ::books)]
   {:author author
    :books  (map #(assoc % :author-id (:author-id author))
                 books)}))
which is generating a map of author + books from their individual specs, and just associng the author's ID over each book
#2018-07-0320:10fominokThank you, @taylor #2018-07-0415:13ackerleytnggenerative tests with specs take really long to run. what are some tips to try and reduce the testing time?#2018-07-0415:37taylorThis can depend on your specs. Recursive specs can be especially costly. Can you give an example thatā€™s taking a while to run?#2018-07-0423:22ackerleytngit's not a recursive spec though#2018-07-0417:02seancorfield@ackerleytng I don't feel generative tests should be run automatically as part of your (fast) unit test suite. They can/should be run separately or as part of CI for example. Because generative testing can definitely take a while.#2018-07-0417:24dominicmRich has mentioned a number of times that generative tests should only run when the relevant code changes. I'm sure he'd be very happy if someone built that for him.#2018-07-0418:13Alex Miller (Clojure team)I have built the hard parts of it, just needs a lot of productization#2018-07-0421:11dominicmDo you mean to suggest that it's going to be a product that's sold or branded? Or just that it's rough? If it's rough, is it in such a state that the community might learn much from it by seeing it and seeing where they can take it?#2018-07-0421:30Alex Miller (Clojure team)It will be a contrib library. Itā€™s not at a state where Iā€™m ready to publish it yet, both for design and impl#2018-07-0421:32Alex Miller (Clojure team)We may end up sharing some code with codeq 2 also, which is similarly unfinished. Just a lot of stuff to shake out and none of us have time to work on either atm#2018-07-0420:29gnl@dominicm Shameless plug ā€“ https://github.com/gnl/ghostwheel runs gen-tests per namespace on hot-reload which is effectively just that (or close to it), with the option to make re-rendering dependent on successful test completion.#2018-07-0421:11dominicmI hadn't appreciated that's why it did those things. Very impressive! #2018-07-0421:28Alex Miller (Clojure team)Thatā€™s great but what Iā€™ve been working on is code analysis at the function dependency level#2018-07-0421:44gnlFunction-level granularity would be even better for this of course ā€“ looking forward to it. Is this going to be a core thing or a third-party library?#2018-07-0421:45gnl@dominicm It's really just a fancy (run-tests) but it gets the job done. šŸ™‚#2018-07-0421:47gnlThe important thing is that if you're working on layout/view stuff where quick hot-reloading is most valuable, it doesn't run any expensive event handler gen-tests, etc.#2018-07-0421:49gnl@U064X3EF3 Oh, I just saw the other thread where you answered that, nevermind.#2018-07-0423:23ackerleytng@seancorfield Thanks! I'm writing my own custom generators, could that contribute to slower testing?#2018-07-0423:24ackerleytngeven with overrides like {::foo (fn [] (gen/return <something pre-generated>} does not improve testing speeds much. why would this be?#2018-07-0423:24ackerleytnggen/return probably doesn't run the generator at all, right?#2018-07-0423:43gfrederickswell it is a generator, but everything it does is trivial, so it by itself should never make anything slow#2018-07-0616:50twsIā€™d like to try and do this without custom generatorsā€¦ any ideas? I want to spec out a string of digits, with each having certain restrictions. e.g. each digit is (s/int-in 2 10) but Iā€™d like 3 in a row catted into a string. using regexes like #(re-matches #"[2-9]{3}" %) requires me to make a custom generator I believe. Struggling with the syntax.#2018-07-0616:51twsI can get an s/tuple easy enough, but not sure how to jam that into a string inside the s/def#2018-07-0617:06twsso I have
(s/def ::digit (s/int-in 2 10))
(s/def ::code (s/tuple ::digit ::digit ::digit))
but instead of a tuple/vector I want them jammed into a string
#2018-07-0617:08noisesmithI can't answer this question, but as an aside, spec is explicitly not for parsing#2018-07-0617:15twsItā€™s not to parse itā€™s to validate data. (And generate it)#2018-07-0617:19noisesmithwhy not use a regex as your spec validator, and write a custom generator?#2018-07-0617:20guyYou can give a spec a predicate and a generator#2018-07-0617:20twsI could. Just wanted to know if there was a cool way to do without #2018-07-0617:20guyso just make a custom predicate#2018-07-0617:20guylet me find an example 2 secs#2018-07-0617:20guyhttps://clojure.github.io/spec.alpha/clojure.spec.alpha-api.html#clojure.spec.alpha/spec#2018-07-0617:21guyso you can have say#2018-07-0617:21guy(s/def ::digit my-pred-fn my-gen)#2018-07-0617:21guy(defn my-pred-fn [x] your logic)#2018-07-0617:22guyif that makes sense?#2018-07-0617:22guybut you need a generator that would then fulfil that predicate#2018-07-0617:22twsYes thanks. Iā€™m familiar with custom generators. It just seems like Iā€™m codifying the same rule in two places (pred and gen) which I would like to avoid if I could. #2018-07-0617:23twsBut Iā€™ll just move ahead with a custom gen. #2018-07-0617:24guy@tws oh sorry just read your first line facepalm#2018-07-0617:24guy>Iā€™d like to try and do this without custom generators#2018-07-0617:24twsCommunication is hard!#2018-07-0617:25guyhaha nah i just need more coffee so i can read better šŸ˜ž#2018-07-0713:43ackerleytngwhy does this
(ns foo.bar
  (:require [clojure.spec.test.check]))
error out with
Could not locate clojure/spec/test/check__init.class or
   clojure/spec/test/check.clj on classpath.
#2018-07-0713:45gfredericksI don't think that's a real ns#2018-07-0713:47ackerleytnghmm there's no file associated with it#2018-07-0713:47ackerleytng
(ns clojure.spec.test.alpha
  (:refer-clojure :exclude [test])
  (:require
   [clojure.pprint :as pp]
   [clojure.spec.alpha :as s]
   [clojure.spec.gen.alpha :as gen]
   [clojure.string :as str]))

(in-ns 'clojure.spec.test.check)
(in-ns 'clojure.spec.test.alpha)
(alias 'stc 'clojure.spec.test.check)
#2018-07-0713:47ackerleytngi guess you're right, thanks!#2018-07-0713:47gfredericks:+1:#2018-07-0714:06ackerleytnghow do i use the default-reporter-fn from test.check (https://github.com/clojure/test.check/blob/master/src/main/clojure/clojure/test/check/clojure_test.cljc) with a specific deftest?#2018-07-0714:07gfredericksWhat's your entry point? The stest/check function?#2018-07-0714:09ackerleytngyup#2018-07-0714:09ackerleytngi have a deftest which calls stest/check#2018-07-0714:11gfredericksDoesn't it have an arg for arbitrary extra test.check args?#2018-07-0714:12gfredericksKeep in mind the reporter-fn stuff is only in 0.10.*#2018-07-0714:20ackerleytngit does? i'll look that up#2018-07-0804:52ackerleytngah actually 0.10.0-alpha3 improves testing error messages, it's exactly what i wanted!#2018-07-0811:52gfredericksI'm curious which improvement you're referring to#2018-07-1401:31ackerleytng
(deftest generative-tests
  (doseq [test-output (stest/check function-under-test {:gen gen-overrides
                                                        ::stc/opts {:num-tests 10}})]
    (testing (-> test-output :sym name)
      (is (true? (-> test-output ::stc/ret :result))))))
#2018-07-1401:32ackerleytngI do this, so previously it was just something like expected: true and actual: false#2018-07-1401:32ackerleytngwhich is meaningless#2018-07-1401:32ackerleytngbut now it shows #error and dumps the error from spec#2018-07-1412:55gfrederickshmm#2018-07-0804:52ackerleytngalso, youre right, it should be :reporter-fn#2018-07-1113:27roklenarcicdoes calling instrument with no arguments instrument all symbols in current namespace or all instrumentable symbols in all the loaded namespaces?#2018-07-1113:28taylorall instrumentable vars in all loaded namespaces AFAIK#2018-07-1113:31Alex Miller (Clojure team)yes#2018-07-1210:09Matt ButlerIs there any way to use predicates inside the literal comparator to achieve something like this in spec. (s/valid? #{["foo" "bar" string?]} ["foo" "bar" "baz"])#2018-07-1212:10tayloryou could spec that with s/cat#2018-07-1212:44Alex Miller (Clojure team)Or s/tuple#2018-07-1214:10Matt Butlerso (s/tuple #{"foo"} #{"bar"} string?) ? but if I wanted to combine that with other fixed values where I know all 3 elements, I'd have to use an s/or like so? (s/or :set #{["a" "b" "c"]} :tuple (s/tuple #{"foo"} #{"bar"} string?))#2018-07-1214:10Matt ButlerThere is no way to next dynamic values inside the #{}#2018-07-1214:13Matt ButlerThanks btw šŸ™‚ jut wanted to check my understanding.#2018-07-1214:17taylorif I understand the use case, s/or seems reasonable to me#2018-07-1214:19Matt ButlerYeah I have a set of triple store values, most are known but for some subset the final key is freetext.#2018-07-1215:58akielIs there a way to override key-specs in test like it is possible for fn-specs with instrument?#2018-07-1216:00ghadisee the second argument to instrument, specifically :overrides :stubs :gen#2018-07-1216:03akielI donā€™t see :overrides in the doc string of instrument. I also donā€™t see a possibility to override key-specs.#2018-07-1216:04akielWith key-spec I mean (s/def ::a int?). I like to override ::a.#2018-07-1216:04ghadisorry, I goofed, that's not an actual config key#2018-07-1216:05ghadihttps://clojure.github.io/spec.alpha/clojure.spec.test.alpha-api.html#clojure.spec.test.alpha/instrument You cannot override key specs. You can only override which specs are bound to particular functions, and which generators are used for particular specs#2018-07-1216:05ghadiOverriding the key spec doesn't really make sense conceptually -- you'd be changing the contract but preserving the same global name#2018-07-1216:10akielI have a function which gets a map of values and I donā€™t like to generate the real values in test. So I would like to override the specs for the keys in the map. But if I think more about it, why should a supply a map to the function at all? So I can change the fn-spec to take a custom test value instead of a map which custom test values in it. That should work. Oh I also stub the function.#2018-07-1314:48roklenarcicHow can one spec maps with string keys?#2018-07-1314:51mpenet(s/map-of string? ....) if you want to use s/keys you will need to convert them first#2018-07-1421:26roklenarcicHm I thought this would work:
(alter-var-root #'s/*explain-out* (constantly exp/printer))
#2018-07-1609:31bbrinckIs this at the REPL? If so, youā€™ll want to use ā€˜set!ā€™ not ā€˜later-car-rootā€™#2018-07-1609:33bbrinckSorry, autocorrect on my phone makes talking about code difficult šŸ˜€#2018-07-1609:34bbrinckHereā€™s a gist that shows why alter-var-root doesnā€™t work as you might expect in this context https://gist.github.com/bhb/d82fcf0f80f555c28afa8db320be16c8#2018-07-1609:36bbrinckAnd hereā€™s a different gist with an example of how Iā€™d recommend setting up expound for instrumentation (if thatā€™s what you are looking to do) https://gist.github.com/bhb/649c46ac6dfa290fa6a62bb96fb66f62#2018-07-1609:38bbrinck@U66G3SGP5 Let me know if you encounter any problems with the above code#2018-07-1611:12roklenarcicI tried set! before, but it's been my understanding that it only sets var value for current thread#2018-07-1611:18bbrinckHm, Iā€™m not sure what happens in threads started within the REPL - I can check later. You might have to call ā€˜set!ā€™ in each thread. But in any case, alter-var-root! wonā€™t work in the REPL context since the var is bound by Clojure before your code is run, so I think ā€˜set!ā€™ is the only solution. #2018-07-1706:59athosA quick workaround for this is to try (alter-var-root #'s/*explain-out* (constantly exp/printer)) in your user.clj. Clojure runtime should automatically load user.clj (if it exists) before launching the REPL.#2018-07-1713:17bbrinck@U66G3SGP5 Are the threads being created within the REPL context? If so, it looks like set! will apply. See this gist. https://gist.github.com/bhb/910f718e2da57793bc0f5817f006f28a#2018-07-1713:18bbrinckIf threads are being created outside the REPL, then yes, I would try what @U0508956F suggested above.#2018-07-1713:19bbrinck@U7PBP4UVA Did you end up getting expound to work with multiple threads?#2018-07-1421:27roklenarcicto make all my stuff print spec errors with expound#2018-07-1711:47marktThink I asked in the wrong channel... too many channels https://clojurians.slack.com/archives/CB19ETU0N/p1531736138000105#2018-07-1711:53mpenetnot really#2018-07-1711:54mpenetmaybe some external lib does, but not spec itself afaik#2018-07-1712:14ikitommi@markt you were using schema-tools.core/select-schema? By design, Spec doesn't support closed keys. 3rd party?, there is at least spec-tools.core/select-spec for this#2018-07-1712:16marktAh thought that would be the case, it's a bit of a odd case we have. Cool thanks for the heads up, i'll have a look at spec-tools šŸ‘#2018-07-1712:37ikitommiI think itā€™s the odd case not to strip out extra data at the borders.#2018-07-1712:50marktInteresting, so you would prefer to have a function only return what the spec dictates?#2018-07-1714:14marktJust noticed your the author of the library! Thanks for writing the library šŸ˜„#2018-07-1715:26ikitomminot at function level, but at system boundaries: Reading JSON sent by a (potentially rogue) third party, or when writing to a (document) database. Not removing extra keys is a security risk, stripping keys manually is both error prone & extra work.#2018-07-1812:25marktabsolutely agree, and thanks for all the help#2018-07-1818:31gklijsIt's not that hard using specs to implement losing the rest. I rather have a bit of code than a dependency. That's the road I took, also added the option to have default values for keys, to keep specs backwards compatible.#2018-07-1912:35ackerleytngMay I trouble someone to explain conformer to me? Don't really understand the docs#2018-07-1913:10claudiuDid you check out https://lambdaisland.com/episodes/clojure-spec ?#2018-07-2413:12ackerleytngthanks! I meant conformeR not just conform#2018-07-2413:34claudiuahh šŸ™‚ Also trying to learn this stuff. Stuart used it in the fizzbuzz example, maybe that example helps https://gist.github.com/stuarthalloway/01a2b7233b1285a8b43dfc206ba0036e#2018-07-2016:38ghadihttps://twitter.com/smashthepast/status/1020347051589144578#2018-07-2101:24jkrasnayHi folks, I have a Re-frame app that is nicely checking my function invocations with spec but itā€™s not showing file names / line numbers.#2018-07-2101:25jkrasnay
Call to #' did not conform to spec:ā†µ
<filename missing>:<line number missing>ā†µ
ā†µ
-- Spec failed --------------------ā†µ
ā†µ
Function argumentsā†µ
ā†µ
  (... :foo ... ...)ā†µ
       ^^^^ā†µ
ā†µ
should satisfyā†µ
ā†µ
  string?ā†µ
ā†µ
#2018-07-2101:26jkrasnayIs this normal, or should I be able to get a file name and line number here?#2018-07-2102:31seancorfield@jkrasnay Are you using Expound there? That doesn't look like a raw clojure.spec error... Just wondering if it's a bug in the reporter itself...?#2018-07-2102:56jkrasnayYep, itā€™s expound. Iā€™ll try taking that out.#2018-07-2103:32jkrasnayLooks like Expound has that code there waiting for the day when CLJS will send caller info. For now itā€™s just teasing me.#2018-07-2104:18seancorfield@jkrasnay Good to know. I don't use cljs myself but I know there are all sorts of weird edge cases/differences from clj...#2018-07-2204:32alex-dixonIs it possible to define a spec with a custom generator but have it separate from the spec itself? Something like
(s/def ::my-spec (s/or :keyword keyword? :symbol symbol?))
(s/with-gen ::my-spec (gen/symbol))
Seems like it could be looking at the defn for s/with-gen but not working at the REPL atmā€¦
#2018-07-2204:41athosYou can replace spec generator impls with s/gen's overrides argument or via :gen option of instrument and check.#2018-07-2204:44athosA library of mine called Genman also does that well https://github.com/athos/genman If you're instested, give it a shot šŸ˜‰#2018-07-2205:19alex-dixonWorks!
(gen/generate
    (s/gen ::my-spec
           {::my-spec gen/symbol}))
#2018-07-2205:19alex-dixon@athos Youā€™re awesome. Thank you šŸ™‚#2018-07-2311:58roklenarcicI have a problem with clojure.java.jdbc.spec, namely when calling query fn which is instrumented with this spec:
(s/def ::result-set-fn (s/fspec :args (s/cat :rs (s/coll-of any?))
                                :ret  any?))
It calls my result set fn with a bunch of garbage
#2018-07-2311:58roklenarcicSo when callbacks have a fspec defined, they get called with random data if they are instrumented?#2018-07-2312:03gfredericksyep#2018-07-2312:03gfredericksthat's how the function is validated#2018-07-2312:03gfredericksit is a common source of surprise#2018-07-2312:29roklenarcicI guess i need to instrument just a specific ns#2018-07-2312:34mpenetthere are options also on instrument you can use to bypass this#2018-07-2312:34mpenetwell, to provide stubs at least#2018-07-2316:30fedregIs there a way (or another function) to have spec/explain return just the spec errors and not the original data as well?#2018-07-2316:37samueldev@fedreg why not just pull the errors out of the result of spec/explain? :shrug:#2018-07-2316:37samueldevor create your own fn explain-errors that does that and use it instead :shrug:#2018-07-2316:39fedreg@samueldev I was about to do just that but then decided to check if I had missed something that already existed. Guess thatā€™s a noā€¦ thx!#2018-07-2317:02seancorfield@roklenarcic Yeah, I may revisit some of those fspecs in clojure.java.jdbc.spec as I get more feedback from people. As @gfredericks says, the behavior you're seeing is what clojure.spec instrumentation is intended to do, but it can be surprising at first. And of course your specific :result-set-fn may well not accept any old collection -- but given that :row-fn can transform the result set data into, essentially, an arbitrary collection of any type of element, the :result-set-fn could potentially be called on any? types... We don't have parameterized types so we can't constrain the row function to map? -> T and then the result set function to coll-of T -> U...#2018-07-2319:38manutter51This surprised me
1. In a CLJS namespace, define a function with a name that matches a valid HTML tag (e.g. "table" or "br" or whatever)
2. Create an (s/fdef ...) spec for it.

Result: every function in that namespace now evaluates to nil.
#2018-07-2319:38manutter51Just a local fluke in my current project, or something that other people can reproduce?#2018-07-2319:39manutter51If the latter, known issue, or new?#2018-07-2407:49claudiuThere seem to be a lot of nice tools/libs like expound, ghostwheel, orchestra, phrase etc... Is there an official/unoficial list with most of these like tools.deps wiki šŸ™‚ ?#2018-07-2408:27cvichttps://www.clojure-toolbox.com was a good list, but it's not updated anymore. Same goes with https://github.com/mbuczko/awesome-clojure https://github.com/razum2um/awesome-clojure#2018-07-2408:28cvicMaybe https://crossclj.info#2018-07-2409:27claudiuCool, but still just partials in multiple sources šŸ˜ž Was banging my head with tools.deps stuff and noticed that there was a really nice page. seems to be pretty complete. Was hoping for something similar for spec. https://github.com/clojure/tools.deps.alpha/wiki/Tools#2018-07-2409:28cvicYeah. In an ideal world you would find all of these on https://clojure.org Work in progress (I hope).#2018-07-2415:02avi@U064X3EF3 I tried to add a Tools page to the wiki in this repo: https://github.com/clojure/spec.alpha but canā€™t navigate to the wikiā€¦ I just get redirected back to the home page of the repo. Never seen that before. Any idea whether itā€™s a repo setting problem or a bug in GitHub maybe?#2018-07-2415:25eggsyntaxhttps://cljdoc.xyz/ is trying to become the new center for documentation of clojure/script libraries. But doesn't especially help with finding good libs. I know https://www.clojure-toolbox.com/ was updated with at least one lib recently, so it's not abandoned.#2018-07-2419:25Alex Miller (Clojure team)Itā€™s a repo setting. Happy to change it, but Stu and I are both on vacation atm. Ping me again next week or send me an email or Iā€™ll forget.#2018-07-3120:43avi@U064X3EF3 ping! ā˜ļø#2018-08-0217:56Alex Miller (Clojure team)hey, fyi email is always better than dms as it can stick around in my inbox rather than disappear#2018-08-0217:57Alex Miller (Clojure team)I donā€™t have admin access to that repo#2018-08-0217:57Alex Miller (Clojure team)Stu does, but he is still out#2018-08-0217:57aviah ok cool#2018-08-0217:57aviwill do ā€” thanks!#2018-07-2416:42martinklepschIs there a reason spec doesnā€™t have itā€™s own project in JIRA?#2018-07-2416:42martinklepschI assume itā€™s mostly historical?#2018-07-2419:24Alex Miller (Clojure team)Because spec and the core specs are tied into Clojure, we manage them in the context of Clojure even though itā€™s a different repo. That was an intentional choice. It may not always remain so.#2018-07-2419:34claudiuIn the next release will spec graduate from alpha ? #2018-07-2419:36Alex Miller (Clojure team)I hope so, no promises#2018-07-2512:47jumarLet's say I have a strange function that has two arities: 1-arity computes some results and returns them 2-arity uses 1-arity to compute the same results and save the output to file. How can I write :ret spec for this? I tried :fn but struggling with reusing an existing spec
:fn (s/or :results (s/and #(= 1 (-> % :args count))
                                  ::result-spec)
                  :ouput-in-file (s/and #(= 2 (-> % :args count))
                                        #(-> % :ret empty?))))

Here, the ::result-spec isn't properly used - I need to apply it only to the :ret key value, not the whole map passed to :fn.
#2018-07-2512:55gfredericksif it were me I'd have two different functions#2018-07-2512:57jumarThat was my idea too, but let's say it's a "convention" to do it this way. Can I still somehow reuse the ::result-spec or is there a better way to write :ret spec for this function?#2018-07-2520:34hiredman(s/def ::ret ::result-spec) (s/and ... (s/keys :req-un [::ret]))#2018-07-2708:12jumarNice, thanks! I guess the only problem would be if I had more than one such function in the same ns (conflicting ::ret specs) but that's not an issue in my case.#2018-07-2618:40EthanHello, I'm working on creating specs and I would like to use ::bindings from the core.specs.alpha in the alpha.clj file in my code. I was wondering how I would go about doing that without all copying the code to my file.#2018-07-2618:54Alex Miller (Clojure team)Just refer to the full spec keyword#2018-07-2618:54Alex Miller (Clojure team):clojure.core.specs.alpha/bindings#2018-07-2619:00Ethanok, thank you#2018-07-2713:21burbmaCan anyone provide insight as to whatā€™s going on here?#2018-07-2713:42jumar
(type (resolve 'string?))
;;=> clojure.lang.Var
user> (type string?)
;;=> clojure.core$string_QMARK___5132
#2018-07-2713:44guyIs resolve a spec function?#2018-07-2713:44guyšŸ¤”#2018-07-2713:44guyIā€™ve not seen it before#2018-07-2713:47burbmaresolve isnā€™t a spec function itā€™s in core. Suppose I started with "string?", how can I get to the clojure.core$string_QMARK_...? (resolve (symbol "string?)) wonā€™t do it as weā€™ve seen.#2018-07-2713:48guyreally? ill check it out thanks!#2018-07-2713:49guyohh right got ya#2018-07-2713:49guyneat#2018-07-2713:49guyTIL;!#2018-07-2713:50guyif u want to see what it does, you can generally just try s/exercise and see what it produces#2018-07-2713:51burbmas/exercise gives the same error for the (resolve (symbol "string?")) version of the spec.#2018-07-2713:56guyYeah i mean just for (s/exercise string?)#2018-07-2713:56guysorry#2018-07-2713:57burbmaAh, no worries. Thanks for the pointer.#2018-07-2714:01burbmaUsing (var-get (resolve (symbol "string?"))) does the trick.#2018-07-2716:39noisesmithor deref, or the shorthand @#2018-07-2715:49dspiteselfAny idea when we will get this fix https://dev.clojure.org/jira/browse/CLJ-2003 in? We are still running a patched version.#2018-07-2715:57Alex Miller (Clojure team)In queue to get screened#2018-07-2716:46dspiteselfThanks#2018-07-2718:03aviLast week at clojure/nyc I shared my experience, as a spec n00b, using spec to specify other peopleā€™s data structures ā€¦ hereā€™s the video if anyoneā€™s interested: https://www.youtube.com/watch?v=eqfSifXaXnw#2018-07-2718:04aviAnd for those in the Boston area Iā€™ll be sharing this again at the Boston Clojure Group on 9 August: https://www.meetup.com/Boston-Clojure-Group/events/tjztcpyxlbmb/#2018-07-2718:11fabraoHello all, how
(defn ^:private menor? [n t]
  (<= (count n) t))
(s/def ::limite-tamanho menor?)

(s/valid? ::limite-tamanho <what is in here?>)
#2018-07-2718:11fabrao?#2018-07-2718:13noisesmithI don't understand how your spec predicate could be a function of two args#2018-07-2718:13noisesmithperhaps (partial menor? some-n) ?#2018-07-2718:14fabraowell, itĀ“s for dynamic size#2018-07-2718:14noisesmithright, but how can you use that directly as a spec predicate? where would the other arg come from?#2018-07-2718:28fabrao@noisesmith IĀ“ll check many specs from vector, like fields in form -> [[:code "Error code" ::limite-tamanho 6] [:name "Error name" ::limite-tamanho 15]]#2018-07-2718:32noisesmithmaybe I'm missing some spec feature that would make this work, but I still don't understand how you'd make spec supply two args to your spec#2018-07-2718:32noisesmithand as far as I know you can't use a neighbor or parent data in the check for a key#2018-07-2718:33fabraoWell, IĀ“m thinking use like this so, [:codigo "CĆ³digo do vendedor" #(<= (count %) 6)]#2018-07-2718:34noisesmithwhich is equivalent to what I proposed with partial above, right?#2018-07-2718:34fabraoyes, thanks a lot#2018-07-2803:00burbmaHow can I get the forms to evaluate before going into the macro?#2018-07-2806:41valeraukonot sure if this is what you're looking for, but i have something like this
(def context-kw (keyword "ap.object" "@context"))
(eval `(s/def ~context-kw map?))
#2018-07-3017:33djtangoSo I am pretty sure this is absolutely not how spec was intended to be used, but has anyone gotten mileage from leaving stest/instrument turned on in production to use spec for runtime contract assertion? Even more so with jeaye/orchestra#2018-07-3017:33djtangoOne reason I can think of why this may not be such a great idea is that stest/instrument validates fspec using generated inputs#2018-07-3017:35djtangoare there any other reasons not to leave stest/instrument turned on?#2018-07-3017:48seancorfield@djtango The generative validation of fspec is one reason. Performance is another. You'll also get bare AssertionError exceptions instead of anything meaningful you might normally catch and handle (assuming you try to (catch Exception e ...))#2018-07-3017:59noisesmithyeah, my understanding is that AssertionError is for things you expect to throw during development - the vm even has a flag to ignore them#2018-07-3018:02djtangoI suppose, to give more context on my usecase, it is useful (for me) to declare some invariants about a function (e.g. a relationship between it's args and return value) but in a sufficiently large project, it's possible for inputs in runtime to fail that invariant and it's nicer when that failure comes at the point of the invariant failing rather than some arbitrary of steps later like a null pointer error#2018-07-3018:04djtangospec happens to be a great way for declaring properties about lots of things, and while generative testing is meant to give better guarantees about over your testing space, the associated complexity with them is a harder sell for the team#2018-07-3018:04djtangoIf that makes sense?#2018-07-3018:05djtangoComing from Racket, I'm kind of used to contracts always being on#2018-07-3018:24noisesmiththe pragmatic thing might just be overriding fspecs for the instrumenting#2018-07-3021:57nopromptitā€™d be great if there were a spec-impl that ā€œmergedā€ two s/ors or s/alts without having to reify the protocols (which is a dangerous game as i understand it).#2018-07-3022:01nopromptfor example, if you want to express that clojure.core/unquote-splicing forms are not permitted at the top level youā€™d have a union for your legal top level forms and then another union for your subforms in, say, a list that includes unquote-splicing forms in addition to what is legal at the top level.#2018-07-3022:02nopromptto fully leverage all of spec, including explanations, it seems like i have to spell this out entirely in both cases if i want to avoid an additional layer of rettags.#2018-07-3022:05seancorfield@noprompt Or wrap s/or in s/nonconforming I guess...?#2018-07-3022:07noprompt@seancorfield thatā€™s not quite the ticket. lemme demonstrate.#2018-07-3022:08noprompt
(s/def :clj.form/top-level
  (s/or :list :clj.form/list
        :number number?))

(s/def :clj.form/list
  (s/coll-of
   (s/or :a :clj.form/top-level
         :b (s/or :unquote-splicing unquote-splicing-form?))
   :kind list?))

(s/conform :clj.form/top-level '(1 
#2018-07-3022:09nopromptwhat i do not want is the :a and :b tags. i still want to retain the :number and :unquote-splicing tags.#2018-07-3022:09nopromptby using s/nonconforming i lose the tags. šŸ˜•#2018-07-3022:10nopromptessentially, i want to extend the :clj.form/top-level union without adding an additional layer of rettags.#2018-07-3022:10nopromptof course, i can do this by using s/with-gen, s/conformer, etc. but i lose out on s/explain-data.#2018-07-3022:12nopromptif there was an s/explainer thisā€™d be a done deal. šŸ™‚#2018-07-3022:13nopromptin short, i think what i want is an or-spec-impl or an alt-spec-impl that does not require tagging the options.#2018-07-3022:14nopromptwhen it conforms returns whatever the conformers itā€™s compose of returns.#2018-07-3022:15noprompti think there was previous discussion along same lines but for s/cat with an s/concat conformer..#2018-07-3022:29seancorfieldAlex has talked several times about the possibility of adding a specific non-conforming or variant so you'd have that sort of level of control...#2018-07-3022:32arohnerI have a big complicated s/keys, and Iā€™m getting "Couldn't satisfy such-that predicate after 100 tries.". How do I discover which predicate is failing?#2018-07-3022:58gfredericksif you're using a new enough test.check, the ex-data should have the generator and predicate attached, at least#2018-07-3118:15lilactownhow do I spec a function that takes only one argument?#2018-07-3118:27seancorfield(s/fdef my-func :args (s/cat :arg ::spec)) (to be more specific)#2018-07-3118:29seancorfield@lilactown Here's an example from our codebase at work
(s/fdef me
  :args (s/cat :access-token string?)
  :ret (s/nilable (s/keys :req-un [::id ::name])))
#2018-07-3120:48lilactownhow are people enabling instrumentation in development?#2018-07-3120:49lilactownI'm using CLJS. I'm thinking of doing something like:
(when js/goog.DEBUG
  (stest/instrument))
but I'm worried that clojure.spec.test.alpha is going to end up in my release bundle
#2018-07-3121:04justinleelet me know if you figure out the right solution šŸ™‚#2018-07-3121:55seancorfield(we use clojure.spec in production code for validation and conformance so I'm puzzled as to why you'd want to avoid it? doesn't cljs use tree-shaking anyway to remove unused code?)#2018-07-3121:57lilactownvalidating and conforming can be done without including clojure.spec.test#2018-07-3121:57lilactownright?#2018-07-3121:57lilactownand I'm not sure if tree-shaking is smart enough to completely eliminate the lib if it's not used in a release build. will have to test#2018-07-3121:57lilactownI have a second stupid question: is there an easy way to alias a spec?#2018-07-3121:58lilactowne.g. (s/def ::my-spec <point to ::other-spec>)#2018-07-3121:59hiredmanspec validates functions by generating example arguements and calling the function on the examples, which is not something you'll want to do in production#2018-07-3121:59hiredmanif you are strictly validating datastructures that should be fine#2018-07-3122:00lilactownright. I know I definitely don't want to instrument my fdefs in prod šŸ™‚#2018-07-3122:09avišŸ¤” I donā€™t think instrumentation involves generators ā€¦ not saying instrument is appropriate for runtime, just might be helpful to keep things clearā€¦#2018-07-3122:17noisesmithdoesn't it, for the case where the arg is specced to be a function, it would be checked by exercising it#2018-07-3122:18noisesmith(let me know if I misunderstand something here...)#2018-08-0113:51avi:man-shrugging: I dunno, I think I spoke beyond my level of understanding ā€” sorry#2018-08-0113:52aviyour example is fascinating!#2018-08-0113:53avithanks!#2018-07-3122:31noisesmith
user=> (require '[clojure.spec.alpha :as s])
nil
user=> (require '[clojure.spec.test.alpha :as stest])
nil
user=> (s/fdef bar :args (s/cat :f (s/fspec :args (s/cat :x int?) :ret int?)) :ret int?)
user/bar
user=> (defn bar [f] (f 0))
#'user/bar
user=> (stest/instrument `bar)
[user/bar]
user=> (bar (fn [x] (println "called with arg" x) x))
called with arg 0
called with arg 0
called with arg 0
called with arg 0
called with arg 1
called with arg 2
called with arg -1
called with arg 0
called with arg -2
called with arg -24
called with arg 3
called with arg -2
called with arg 9
called with arg -1
called with arg -757
called with arg -1
called with arg 3
called with arg 1963
called with arg 1165
called with arg -359
called with arg -3956
called with arg 0
0
#2018-07-3122:31noisesmithexample of the generation behavior with instrument#2018-08-0104:48tianshuHi, how can I write a spec for arguments like [name & opts], opts is a hash-map here, I already have a ::opts spec for a hash-map, what a spec looks like for this function?#2018-08-0104:56tianshuIt's likely I can use (s/cat :name ::name :opts (s/keys* :req-un [...]))#2018-08-0104:58tianshubut If I do (s/def ::opts (s/keys* ...)), I can't use ::opts here. It requires me to provide a vector#2018-08-0105:00tianshuoh, my mistake, if I use s/keys* it's ok, but can't use (s/and (s/keys* ...) (...)).#2018-08-0207:18athos(s/& (s/keys* ...) (...)) doesnā€™t work?#2018-08-0117:08arohnerIf I have a multi-spec, whatā€™s the best way to get an instance of a single dispatch value? Occasionally (-> (s/spec ::foo) (s/gen) (gen/such-that [f] (= :bar (:type f)) will ā€œcouldnā€™t satisfy such-thatā€#2018-08-0117:09arohnerbut if I call the multimethod directly, it can return nonsense values, because (defmethod :foo [_] (s/keys :req-un [::type])), the :type doesnā€™t necessarily match :foo#2018-08-0204:28andy.fingerhut@noisesmith Do you know why instrument of function foo causes a function arg of foo to be called additional times with arguments that the intrumented call is not itself calling? I can understand doing that if you were doing generative testing on foo, but don't see the rationale for why one would want to do that for instrument.#2018-08-0204:35noisesmithMy understanding is that's the only way spec has to validate a function argument#2018-08-0209:38mpenetIt could validate it at invoke time, just once. #2018-08-0209:39mpenetAt least in theory. A few people suggested that on jira among other places. I can imagine that being an option#2018-08-0204:49andy.fingerhutMakes sense. 20 times may be more than needed for that purpose, but probably tunable.#2018-08-0316:34kvltSo question: I have the following...
(s/def ::name string?)
(s/def ::age (s/and integer? pos? #(< % 100)))
(s/def ::occupation #{:engineer :mechanic :manager :clerk})
(s/def ::person (s/keys :req [::name ::age ::occupation]))

(s/fdef do-it
  :args (s/cat :people (s/coll-of ::person))
  :ret (s/coll-of ::age))

(defn do-it
  [people]
  (map ::age people))
Running this does not work:
(stest/check `do-it)
=>ExceptionInfo Couldn't satisfy such-that predicate after 100 tries.  clojure.core/ex-info (core.clj:4739)
However, I can exercise the fn:
(s/exercise-fn `do-it)
([([#:gentest.core{:name "", :age 1, :occupation :manager}]) (1)]
 [([#:gentest.core{:name "A", :age 15, :occupation :engineer}]) (15)]
 [([#:gentest.core{:name "", :age 3, :occupation :manager}]) (3)]
 [([#:gentest.core{:name "nA0", :age 3, :occupation :clerk}]) (3)]
 [([#:gentest.core{:name "1ZJO", :age 3, :occupation :engineer}]) (3)]
 [([#:gentest.core{:name "80v", :age 20, :occupation :mechanic}]) (20)]
 [([#:gentest.core{:name "M", :age 98, :occupation :engineer}]) (98)]
 [([#:gentest.core{:name "", :age 2, :occupation :engineer}]) (2)]
 [([#:gentest.core{:name "drn9G", :age 59, :occupation :mechanic}]) (59)]
 [([#:gentest.core{:name "2p", :age 72, :occupation :clerk}]) (72)])
Why is this?
#2018-08-0316:49favila(s/and integer? pos? #(< % 100)))#2018-08-0316:50favilathis value space is too large#2018-08-0316:50favilait can't create an efficient generator#2018-08-0319:24seancorfield@petr (s/def ::age (s/int-in 1 100)) should solve that.#2018-08-0319:31favilaExceptionInfo Couldn't satisfy such-that predicate after 100 tries. almost always means there's an s/and with values generated from earlier predicates could not satisfy later predicates#2018-08-0319:31favilavery often you need a custom generator
#2018-08-0319:32favilabtw it would be really nice to somehow have a generator associated with a predicate#2018-08-0319:33favilais there some trick to this? I end up manually using s/with-gen everywhere I use a predicate (and having to remember to do that)#2018-08-0405:03kvltHey guys, sorry for the slow reply. I had ended up with this:
(s/def ::age (s/with-gen (s/and integer? pos? #(< % 100)) (clojure.spec.gen.alpha/large-integer* {:min 0 :max 100})))
#2018-08-0405:40seancorfield@petr Did you see my suggestion of using s/int-in?#2018-08-0405:42seancorfield
user=> (require '[clojure.spec.alpha :as s])
nil
user=> (s/def ::age (s/int-in 1 100))
:user/age
user=> (s/exercise ::age)
([1 1] [1 1] [2 2] [2 2] [1 1] [6 6] [5 5] [1 1] [16 16] [7 7])
user=> (s/exercise ::age)
([1 1] [2 2] [1 1] [3 3] [8 8] [2 2] [2 2] [9 9] [2 2] [69 69])
user=>
#2018-08-0514:23kvltI did thanks!#2018-08-0406:45mishawhat is the practical purpose of s/keyā€™s :opt and :opt-un? - documentation(?) - include :opt keys during s/exercise - anything else? does s/conform conforms all known keys in a map, or just listed in :req/:opt/-un?#2018-08-0407:23valeraukoin my understanding it's that the presence of the key is optional, but if it's present, its shape has to fulfill its spec#2018-08-0407:48guy^#2018-08-0407:49guyThats what ive used it for before#2018-08-0408:20misha@vale s/keys checks against all the known key-specs, you dont have to list them in the s/keys. empty s/keys will test all the registered keys#2018-08-0408:21valeraukoare you sure about that?#2018-08-0408:21mishayes#2018-08-0408:22mishathis is one of the goals or benefits of having global specs registry and namespaced keywords#2018-08-0408:24misha
(s/def :foo/bar int?)
=> :foo/bar
(s/valid? (s/keys) {:foo/bar 1})
=> true
(s/valid? (s/keys) {:foo/bar "1"})
=> false
#2018-08-0408:26valeraukoi see... well it's still meaningful for the un ones where you have to be explicit#2018-08-0408:26valeraukootherwise the opt is probably just for what you said#2018-08-0408:27mishatrue#2018-08-0408:27mishaanything else?#2018-08-0408:37guy>you dont have to list them in the s/keys. I thought part of the reason u list them in s/keys is to show easily what the shape of the data is?#2018-08-0408:37guyusing (s/keys) like you did seems to be interesting, iā€™ve never seen that before#2018-08-0408:39misha@guy ā€œshowā€ is documentation + exercise I mentioned above#2018-08-0408:39guyoh right sorry#2018-08-0408:39guyšŸ‘#2018-08-0408:39guyyeah i think uve covered it imo#2018-08-0408:39mishaIā€™d not use empty s/keys, it is just an illustration#2018-08-0413:23ackerleytngdoes anyone have a good way of writing specs for ring middleware?#2018-08-0413:24ackerleytngfor every middleware function along the chain, the inputs and outputs change#2018-08-0413:24ackerleytngseems tedious to spec all the middleware functions#2018-08-0413:36gfredericksyou shouldn't have to fully describe the req/resp at each point though; you could describe just what's pertinent to that piece#2018-08-0413:39ackerleytngdo you mean like... if this middleware augments the request with a new key value pair#2018-08-0413:39ackerleytngthen perhaps the :args request should be just map?#2018-08-0413:40ackerleytngand the :ret is like a function with :args including just the new key-value pair?#2018-08-0413:40ackerleytngsort of like only writing spec for the change in this particular middleware?#2018-08-0414:00gfredericksyeah#2018-08-0414:01gfredericksyou could also write a :fn saying everything else gets passed through#2018-08-0423:01ackerleytngthanks!#2018-08-0500:23ackerleytngWhat's an example of a well or appropriately specced codebase?#2018-08-0500:26taylorI donā€™t think thereā€™s only one correct approach but hereā€™s a popular lib with specs https://github.com/clojure/java.jdbc/blob/master/src/main/clojure/clojure/java/jdbc/spec.clj#2018-08-0500:58seancorfieldNote: that deliberately puts all the specs in a separate namespace so that they are optional and clojure.java.jdbc can still be used with earlier versions of Clojure.#2018-08-0500:59seancorfieldAlso, if you instrument that library, there's a noticeable overhead in performance (running the test suite takes much, much longer) and that's partly due to the complexity of some of the specs. I have a ticket open to look into optimizing them.#2018-08-0501:03seancorfieldAnother point to consider: those specs focus on functions. Spec is amazing with data structures -- and that's mostly how we use it at work: to specify data structures (including API parameters) and to validate values against those specs (with some coercions). We also use specs for tests, but that's less of our focus.#2018-08-0511:53hmaurerSo you use specs to validate inputs from APIs?#2018-08-0600:20seancorfieldYes, we have specs for all our (public/external) APIs and then s/conform the data on entry to the API. If that is s/invalid? then we use s/explain-data to construct appropriate error codes and messages for the client/caller, otherwise we go forward with the conformed data.#2018-08-0600:23seancorfieldOur specs include some coercion, since API input data can be either strings or "values" -- so we conform strings to values (and pass values through). So true is acceptable where we expect a Boolean, but so is "true" and that is conformed to true. Same with long, double, and date values. We have custom generators that produce strings (mostly they just use the "expected" value spec as a generator and then call fmap str on the result).#2018-08-0610:48hmaurer@U04V70XH6 thanks for the detailed reply! considering that specs are arbitrary predicates, do you find it easy to construct human-readable error messages?#2018-08-0610:48hmaurer(I am aware of expound)#2018-08-0616:04seancorfieldIt took a while to develop heuristics for walking the explain data but, given our domain and the specific set of specs and predicates in use, it isn't too bad.#2018-08-0501:54ackerleytngi think i've a vague idea of what you mean by focus on functions, could you elaborate on what you mean by focus on functions vs focus on data structures?#2018-08-0501:54ackerleytngalso, whats the difference between s/nilable and s/??#2018-08-0504:26seancorfield@ackerleytng Sorry, I was off having dinner... s/nilable wraps a spec or predicate and produces a new spec that allows a value to be nil or conform to the wrapped spec.#2018-08-0504:27seancorfields/? is a regex spec (in an s/cat sequence) for something that is optional.#2018-08-0504:27seancorfieldSo for a data structure (or argument list) that may have two or three elements, the third one is optional and would be specified with s/?.#2018-08-0504:28seancorfieldfor an element that could be nil or a string, you would use (s/nilable string?) -- it's a value either way, it's just allowed to be nil or else conform to string?.#2018-08-0508:06ackerleytngThanks! So it's like... Elements can be nothing, nil or something. s/nilable allows the element to be either nil or something, but s/? allows the element to not be there at all.#2018-08-0504:28seancorfieldMake sense?#2018-08-0504:30seancorfieldAs for focus on function vs data, it's about whether you're spec'ing function arguments (and return values and the relationship between them), vs whether you're using s/def to spec entities that are part of a data structure (and the data structures themselves). Not sure how to explain it beyond that. Does that help @ackerleytng?#2018-08-0508:02ackerleytngBut without spec'ing functions, there's no way to trigger checks unless you manually call functions and use s/conformon the outputs right? So we're better off spec'ing the functions then running check in a deftest#2018-08-0508:04ackerleytngAs in, I thought whether or not you spec functions, you would have to spec the data going in and out of functions#2018-08-0508:04ackerleytngSince you kind of want to exercise the specs in some way, it makes sense to just spec the functions to run tests, does that make sense?#2018-08-0600:24seancorfieldWe call s/conform explicitly because we're using spec in production code. And we conform just the data (arguments) we want to, whereas instrument checks all of them -- so you have to write specs for all the arguments.#2018-08-0600:26seancorfieldSpec'ing functions and spec'ing data are very different approaches. Typically the former is for dev/test since the overhead of instrument can be high -- and it will use generative testing on fspec function arguments. The latter, however, is a great way to do data validation in production code.#2018-08-0823:47ackerleytngI see, thanks! So if i'm using spec in production, then s/conform is the way to go, and if i'm using spec for testing, then fspec is a good idea#2018-08-0520:20ikitommicould the Specs have a concise toString representation? example with expound:
-- Spec failed --------------------

  {:coercion ...,
   :middleware ...,
   :summary ...,
   :swagger ...,
   :parameters {:body ..., :header #object[clojure.spec.alpha$map_spec_impl$reify__1931 0x5ac022 "
#2018-08-0520:24ikitommiā€¦ or print-method implemented.#2018-08-0613:31ben.mumfordhi spec, i have two types i want to spec that have some fields in common. is it possible to derive specs from a common base spec (like a superclass in java)?#2018-08-0613:31ben.mumfordin schema i could merge the base schema with new stuff.#2018-08-0613:46ben.mumfordnvm sorted it using a combination of s/and s/keys#2018-08-0614:32favilas/merge is a thing also#2018-08-0616:44Alex Miller (Clojure team)s/merge is better for this#2018-08-0620:08hmaurer@U064X3EF3 @U09R86PA4 I was wondering a couple of months back if there would be a way to do this using derived keywords. i.e. with two specs :foo/a and :foo/b and the relation :foo/b derives :foo/a, enforce that :foo/b has to include :foo/aā€™s spec (or pass both, equivalently)#2018-08-0620:17favila(s/def :foo/b (s/and :foo/a EXTRA-STUFF))#2018-08-0620:18favilaif :foo/b and :foo/a are both map specs, s/merge is better because it can intelligently combine the key's generators#2018-08-0620:19favilas/and of two map specs likely cannot create a usable generator#2018-08-0711:43Alex Miller (Clojure team)There is no support for keyword hierarchies in spec, sorry#2018-08-0712:14hmaurer@U064X3EF3 out of curiosity, is that a design choice? or just a ā€œnot for nowā€ kind of thing?#2018-08-0712:34Alex Miller (Clojure team)Weā€™ve never talked about it#2018-08-0712:35Alex Miller (Clojure team)So, neither :)#2018-08-0712:01scaturrIs there a conventional way to describe a spec as an argument? Like if I have a function that takes a spec? I see s/spec? - but that doesnā€™t seem to work with something like (s/spec? ::my-spec)#2018-08-0712:36Alex Miller (Clojure team)There is no single predicate for it right now#2018-08-0718:57hmaurero/. Is there a reason why s/def does not accept an optional docstring?#2018-08-0719:00favilaProbably an accident of implementation; A spec is put in a global registry atom, which contains a hashmap from key/symbol to spec objects. There's no clear place to put metadata like there is with vars.#2018-08-0719:01hmaurer@U09R86PA4 isnā€™t it possible to attach metadata to the keyword representing the spec?#2018-08-0719:01favilayou cannot attach metadata to keywords#2018-08-0719:02hmaurer@U09R86PA4 ah šŸ˜•#2018-08-0719:02favila
(with-meta :mykw {:abc 123})
ClassCastException clojure.lang.Keyword cannot be cast to clojure.lang.IObj  clojure.core/with-meta--5142 (core.clj:217)
#2018-08-0719:02hmaureritā€™s really unfortunate; being able to attach doc to specs in this way would be really neat#2018-08-0719:02hmaurer(yep I just tried that :()#2018-08-0719:02favilathey could do something, but it wouldn't be the normal var thing#2018-08-0719:02favilathat's all#2018-08-0719:02hmaurerespecially since the doc function is already able to pull specs from the registry and show the predicate#2018-08-0719:03hmaurerit would be super nice if it could also show an extra description#2018-08-0719:03hmaureruseful in cases where the predicate doesnā€™t speak for itself#2018-08-0719:04hmaurerI guess the implementation doesnā€™t really matter; it could be an extra global hashmap from keywords to strings#2018-08-0718:57hmaurer@alexmiller I found this but it hasnā€™t moved in a while: https://dev.clojure.org/jira/browse/CLJ-1965#2018-08-0719:05Alex Miller (Clojure team)Itā€™s the most voted issue in jira and we are def on board with fixing it#2018-08-0719:06Alex Miller (Clojure team)But there are impl factors meaning it will be a while before itā€™s implemented#2018-08-0914:46trissIs there an easy way to flush the clojure spec library? Iā€™m refactoring and am never sure Iā€™ve got ll my specs right until I restart clojure.#2018-08-0914:50favilareset! the registry to empty then re-evaluate all namespaces with specs in them?#2018-08-0914:52trissah ok. Just reset an atomā€¦ but where is the registry atom?#2018-08-0915:35favila(reset! @#'clojure.spec.alpha/registry-ref {}) ?#2018-08-0915:36favilait's private so you do need to do some poking#2018-08-1000:07ackerleytngWhat's a good way to kind of namespace specs internal to a file? So the issue is that s/keys needs to have a spec named, say foo#2018-08-1000:09ackerleytngSo I would do ::foo. However, my function returns a modified map, hence the :ret also has to be something that has s/keys with ::foo#2018-08-1000:09ackerleytngAnd many functions in that file return modified maps#2018-08-1000:10ackerleytngSo I kind of want to namespace the spec for this function#2018-08-1000:11ackerleytngI'm now doing :function-name-pre/foo#2018-08-1000:11ackerleytngAnyone has a better solution?#2018-08-1000:12taylordo you need qualified keys in the map? or are you going to have multiple */foo keys?#2018-08-1000:12taylorif not, why not use unqualified keys?#2018-08-1000:13tayloror maybe I misunderstand the question#2018-08-1000:15ackerleytngThe s/keys already spec it as unqualified#2018-08-1000:15ackerleytngBecause of that#2018-08-1000:15ackerleytngThe actual key has to have a name that matches the key in the map that is specced#2018-08-1000:17taylorsorry, not sure I understand what the question is then#2018-08-1000:25ackerleytngThanks! Haha#2018-08-1000:26ackerleytngMaybe in short it's "how do you group specs for a function?"#2018-08-1000:37seancorfield@ackerleytng I'm not sure I understand the problem you think you have... why does :ret come into this and cause you a problem?#2018-08-1000:37seancorfieldPerhaps if you shared a concrete example?#2018-08-1000:38ackerleytngSure hmm I'll work one out and post it later#2018-08-1017:01bjais ::s/invalid part of the public api?#2018-08-1017:02seancorfieldI don't believe so but there's a function s/invalid? you can use to test for that value.#2018-08-1017:03seancorfield(that said, I've written conforming predicates that return either an updated value or ::s/invalid a few times so...)#2018-08-1017:18dadairI have a spec that checks that one of the keys is a core.async channel; Iā€™d like to write a custom generator for that specific key spec, but I canā€™t seem to determine how to write a generator that returns a channel; I feel like Iā€™m missing something obvious here#2018-08-1018:12taylorcan you use https://clojure.github.io/test.check/clojure.test.check.generators.html#var-return#2018-08-1018:18dadairHmm that would probably work! Thanks!#2018-08-1018:10bjathanks @seancorfield#2018-08-1020:17Alex Miller (Clojure team)yes, ::s/invalid is part of the public api @bja @seancorfield#2018-08-1021:04seancorfieldThanks for the clarification @alexmiller!#2018-08-1023:47bbrinckIā€™ve created a new channel for expound-related questions - #expound#2018-08-1103:41myguidingstarthis spec used to work in 0.1.x (s/explain (s/and map? (s/+ (s/or :pair (s/cat :key keyword? :value boolean?)))) {:foo true}) but with 0.2.168 it fails predicate: (or (nil? %) (sequential? %))#2018-08-1103:44myguidingstarI use that spec to conform some hash-map's key/value#2018-08-1103:47myguidingstarbut it looks like new clojure.spec use sequential? for s/+ therefore doesn't allow hash-maps#2018-08-1103:48myguidingstarwhat should I do?#2018-08-1103:52myguidingstarthat spec is actually used to describe a flexible dsl composed of vectors and hash-maps (the dsl itself is used as part of om.next queries)#2018-08-1103:53myguidingstarI can't think of any viable workaround#2018-08-1103:55myguidingstaralso, is there any reason clojure.spec use sequential? there instead of coll? which is almost identical except that it returns true for sets and maps?#2018-08-1104:54seancorfield@myguidingstar Why not just use (s/map-of keyword? boolean?)#2018-08-1104:58seancorfield(it took me a while to read your spec -- what does s/or even mean with just one named branch?)#2018-08-1105:05lilactownis it possible to have a recursive spec?#2018-08-1116:07taylorI wrote a post that covers this https://blog.taylorwood.io/2017/10/04/clojure-spec-boolean.html#2018-08-1105:28seancorfieldYes.#2018-08-1108:59myguidingstar@seancorfield yup, that s/or is for named branch#2018-08-1108:59myguidingstarI need to tag the key and value, hence the use of s/cat#2018-08-1109:00myguidingstarso s/map-of doesn't work for me#2018-08-1109:10myguidingstarthat spec's purpose is to used with s/conform, kinda like a dsl parser, not just to check validation#2018-08-1118:31seancorfield@myguidingstar I wonder if (s/and map? seq (s/+ ...)) would make it so what you need?#2018-08-1122:28beta1036I have specd a domain entity in my application and have to read instances encoded as JSON where the keys are without namespaces. What's the recommended way to validate and convert the JSON encoded instances to the internal representation?#2018-08-1313:11rapskalian@U4844LY6A I wrote a little library that uses spec to automatically qualify/unqualify domain entities. Given that JSON uses string keys, youā€™ll have to walk/keywordize-keys first on your map as this library currently only works with keyword keys. https://github.com/cjsauer/disqualified/blob/master/README.md#2018-08-1317:01beta1036You use qualify-map first and then conform the result to validate the input, right?#2018-08-1317:46rapskalianCorrect. qualify-map only translates unqualified keys into qualified keys. It doesn't do any conforming of its own. The source code is less than 50 lines in total if you'd like some inspiration for your own implementation: https://github.com/cjsauer/disqualified/blob/master/src/cjsauer/disqualified.cljc#L32-L47#2018-08-1320:46beta1036I've had a look. I'm afraid in order to extend it to my use case (embedded maps, collections, conjunctions, etc.) I'd have to re-implement a lot of spec. handling or this way seems to be particularly tricky.#2018-08-1203:53ackerleytng@seancorfield I have an example here: https://pastebin.com/Kuat3MPV . That's what i'm doing now, but is there a better way of "namespacing" specs?#2018-08-1204:40seancorfield@ackerleytng OK... and what's the question again? That seems OK to me on the face of it...#2018-08-1204:45ackerleytngah ok. I just thought there might be a better way of namespacing all of that#2018-08-1204:45ackerleytngthanks then!#2018-08-1215:36lilactownso I have this spec:
(s/def ::projection
  (s/* (s/alt :key keyword?
              :sym symbol?
              :projection ::projection)))
and when I try and run valid?, I get StackOverflowError clojure.spec.alpha/deriv (alpha.clj:1526)
#2018-08-1215:36lilactown(s/valid? ::projection [:a :b])#2018-08-1215:44taylorIā€™m not sure if youā€™re intending to validate nested collections, but you can do this:
(s/def ::projection
  (s/* (s/alt :key keyword?
              :sym symbol?
              :projection (s/spec ::projection))))
#2018-08-1216:53taylorI should mention this only compiles when ::projection has already been defined#2018-08-1215:44taylor
(s/valid? ::projection [:a :b 'c])
=> true
(s/valid? ::projection [:a :b 'c "x"])
=> false
#2018-08-1215:47lilactownšŸ˜„ okay awesome. yes, I am intending to validated nested collections#2018-08-1215:48taylor(s/valid? ::projection [:a :b [:c ['d]]]) => true#2018-08-1215:49lilactown:+1:#2018-08-1215:48lilactownwhatā€™s the difference between writing :projection ::projection and :projection (s/spec ::projection)?#2018-08-1215:50taylorby default all the regex-type specs ā€œflattenā€ themselves when nested#2018-08-1215:52taylorwrapping a regex-type spec in s/spec prevents that flattening, so you can use them to describe nested sequences of things#2018-08-1216:08lilactownI donā€™t understand what you mean by flattening#2018-08-1216:09lilactownor why something ā€œflatteningā€ would cause a stackoverflow#2018-08-1217:00taylorthe stack overflow is just because of how spec is implemented, if you have a certain form of nested/recursive regex specs it recurses infinitely trying to evaluate it#2018-08-1216:48taylor@lilactown FWIW you can also define your spec using s/or instead of s/alt, and then you donā€™t need the s/spec call to prevent the regex flattening:
(s/def ::projection
  (s/* (s/or :key keyword? :sym symbol? :projection ::projection)))
#2018-08-1216:49taylorby ā€œflatteningā€ I mean that when you nest the regex-type specs, by default they merge/compose/combine/maybe-a-better-word to describe a single sequence#2018-08-1216:51taylor
(s/conform
  (s/+ (s/alt :n number? :s string?))
  [1 "2" 3])
=> [[:n 1] [:s "2"] [:n 3]]
#2018-08-1216:55taylor
(s/conform
  (s/+ (s/alt :n (s/+ number?) :s (s/+ string?)))
  [1 "2" 3])
=> [[:n [1]] [:s ["2"]] [:n [3]]]
#2018-08-1216:57taylorbut what if you wanted to use regex specs to describe nested sequences? the ā€œflatteningā€ behavior wouldnā€™t allow it, so you could use s/spec to avoid it:
(s/conform
  (s/+ (s/alt :n (s/spec (s/+ number?)) :s (s/spec (s/+ string?))))
  [[1 2 3] ["1 2 3"]])
=> [[:n [1 2 3]] [:s ["1 2 3"]]]
#2018-08-1301:48lilactownI think I understand. thank you for the thorough explanation!#2018-08-1315:12trissso Iā€™ve got a few namespaces - each one responsible for managing the contents of particular type of map. Is there a convention in clojure for what to call your ā€œconstructerā€ method? Do you name it after the type of the type of object youā€™re going to create (already in the namespace?) or use a word like create or construct?#2018-08-1315:38guy(def job {:name "some job"}) like that you mean?#2018-08-1315:38guy(defn job [some-args] ...)#2018-08-1315:42lilactown@triss I think this is a better discussion for #clojure , but I try and elide create or make whenever possible#2018-08-1315:44trissah yes youā€™re probably right re: #clojureā€¦#2018-08-1315:45trissthe annoying thing is most of my constructers just do (s/conform ::definition args)#2018-08-1315:45trissbut thatā€™s way too much to typeā€¦#2018-08-1315:46lilactownI see what youā€™re saying though. If i had a particular namespace, like my-app.job and in there were all functions that work on the job entity type, then I would probably just have a create fn#2018-08-1315:46trissor is itā€¦ maybe I should just call (s/conform directly) in other namespaces?#2018-08-1315:46lilactownthat might be my OCaml bleeding through tho#2018-08-1315:47lilactownitā€™s probably better to hide the fact that all youā€™re doing is s/conform in case you ever want to do more?#2018-08-1315:48trissthat was my thinking I guess#2018-08-1315:48trissbut it is very rare I do more these daysā€¦#2018-08-1319:28mikerodIs it necessary to forward-declare specs that are referenced from other specs in a case such as mutually recursive specs? Contrived example:
(s/def ::a
  (s/map-of keyword? (s/or :b ::b :str string?))

(s/def ::b
  (s/map-of keyword? ::a))
#2018-08-1319:29taylornot in my experience, as long as they're both registered by the time you use them#2018-08-1319:29mikerodI believe, at least in some circumstances, that the forward-use of a spec isnā€™t resolved until later anyways, so the forward declaration isnā€™t needed (maybe Iā€™m wrong)#2018-08-1319:29mikerod@taylor that is what I thought I was seeing as well#2018-08-1319:29mikerodand looking at how the references are used in a various implā€™s I checked in clojure.spec.alpha it seemed like it was ok#2018-08-1319:30mikerodbut I havenā€™t seen any direct discussions on it really. Iā€™ve seen at least one posts in the wild that did something like
;; Forward-declaration, replaced later
(s/def ::b any?)

(s/def ::a
  (s/map-of keyword? (s/or :b ::b :str string?))

(s/def ::b
  (s/map-of keyword? ::a))
but that doesnā€™t seem necessary
#2018-08-1319:31tayloragree, seems unnecessary#2018-08-1319:32taylorthe keyword names are like "pointers" to the actual specs that get stored in the spec registry, so as long as the pointer points to a registered spec by the time it's used, I'd assume all's well#2018-08-1319:52mikerodyep#2018-08-1416:45trissis it just me or does spec cause me to end up with circular dependencies in namespaces? whatā€™s the workaround?#2018-08-1417:35guyinstead of using ::my-spec i create fake nsā€™s (s/def :guy-project/user (s/keys :req-un [:guy-project/name]))#2018-08-1417:35guyThen u can store those spec wherever u want#2018-08-1417:36guyspec/user.clj#2018-08-1417:36guyand import it where u need it#2018-08-1417:36guyif that makes sense? šŸ‘€#2018-08-1417:43manutter51Thatā€™s how I do it too. I think it makes for more fine-grained namespacing.#2018-08-1418:50EthanI am working on a project to rewrite error messages and I was wondering if anyone knows any libraries that are well specced so I can test my spec error rewriting on more than just my specced functions and the specs in clojure.spec.alpha#2018-08-1515:25bbrinckThe libraries I test against in expound are: ring/ring-spec and org.onyxplatform/onyx-spec. I also have a partial spec for specs that can generate some specs ā€¦ but itā€™s not quite reliable enough for me to depend on it. https://github.com/bhb/expound/blob/master/test/expound/alpha_test.cljc#L2877-L3124#2018-08-1515:26bbrinckFeel free to use that spec, or frankly you can use any tests in the test suite and update it for your library šŸ™‚#2018-08-1509:55djtangoI think clojure.java.jdbc has good spec coverage#2018-08-1509:55djtangohttps://github.com/clojure/java.jdbc/blob/master/src/main/clojure/clojure/java/jdbc/spec.clj#2018-08-1516:51Ethanthank you#2018-08-1520:06ericnormandHello!#2018-08-1520:06ericnormandCan someone tell me what :opt is for? How is validation affected by its presence?#2018-08-1520:07ericnormandI'm talking about :opt as an option to keys#2018-08-1520:10guyAs far as iā€™ve understood it, if you have say (s/keys :opt [::name]) If the key ::name is present it will check it against the spec.#2018-08-1520:11guySo iā€™ve used it in the past to say, If this thing is present then it has to look like this spec. Otherwise carry on.#2018-08-1520:12guySo if i just make up an example with some pseduo code#2018-08-1520:14guy
(s/def ::name string?)
(s/def ::type string?)

(s/def ::person (s/keys :opt [::name] :req [::type]))

(s/valid? ::person {::type "human})
=> probably true

(s/valid? ::person {::name 124 ::type "human"})
=> false because name is not a string
#2018-08-1520:14guyIā€™ve just written these by hand so donā€™t trust the code 1 for 1#2018-08-1520:16guyFor more reading i would recommend https://clojure.org/guides/spec#_entity_maps#2018-08-1520:16guy
This registers a ::person spec with the required keys ::first-name, ::last-name, and ::email, with optional key ::phone. The map spec never specifies the value spec for the attributes, only what attributes are required or optional.

When conformance is checked on a map, it does two things - checking that the required attributes are included, and checking that every registered key has a conforming value. We'll see later where optional attributes can be useful. Also note that ALL attributes are checked via keys, not just those listed in the :req and :opt keys. Thus a bare (s/keys) is valid and will check all attributes of a map without checking which keys are required or optional.
A rather large snippet
#2018-08-1520:16guyHope that helps!#2018-08-1522:02ericnormandHey @guy! Thanks!#2018-08-1522:02ericnormandHowever, if I do (s/keys), it still checks that ::name is valid.#2018-08-1522:03ericnormand
(s/def ::name string?)
(s/def ::person (s/keys))

(s/valid? ::person {}) ;=> true
(s/valid? ::person {::name "Eric"}) ;=> true
(s/valid? ::person {::name 15}) ;=> false
#2018-08-1522:04guyYeah but what does ::person check for#2018-08-1522:04guy:man-shrugging:#2018-08-1522:04guypart of why i use opt is documentation#2018-08-1522:04guyyour point is valid though and its been asked before#2018-08-1522:04ericnormandyes, documentation is good#2018-08-1522:04guylet me find it#2018-08-1522:04ericnormandthanks#2018-08-1522:05guyah bummer the historys gone#2018-08-1522:05guyAnother person asked something similar but a different way#2018-08-1522:05guysaying why use :opt at all when (s/keys) does the job#2018-08-1522:06guyI think for me its documentation + being explicit about what you want to check for#2018-08-1522:07ericnormandokay, thanks#2018-08-1522:07ericnormandthe spec guide on http://clojure.org hints that it'll explain later, but it never does#2018-08-1522:07ericnormandI thought it might be more than just docs#2018-08-1522:07mpenetAlso gen#2018-08-1522:07ericnormandoh!#2018-08-1522:08ericnormandgood call#2018-08-1522:08mpenetKind of important ;)#2018-08-1522:08guyI think when it comes to speccing out ur fn as well it might be awkward depending on how u use (s/keys)#2018-08-1522:09guyIā€™m unsure if you would really generate a spec like (s/def ::person (s/keys))#2018-08-1522:09ericnormandfor sure#2018-08-1522:09guyyou might just use (s/keys) as an anonymous spec inside a (s/fdef ..)#2018-08-1522:10guyiā€™ve probably got the terminology wrong#2018-08-1522:10guybut that would also lead to harder reading when it comes to spec messages when they have failed#2018-08-1522:10guybut yeah#2018-08-1522:11guyi would favour :opt over (s/keys)#2018-08-1522:11guyjust from a readability factor#2018-08-1522:11guybut im just one guy šŸ˜„#2018-08-1522:34lwhortontrollface#2018-08-1522:47gfredericksis there also a difference when the spec is not defined?#2018-08-1522:48gfredericksi.e., putting things in :opt forces you to define the specs you listed (i.e., prevents you from accidentally forgetting them)?#2018-08-1523:47ericnormand@gfredericks a little repling shows that opts does not check if the spec exists#2018-08-1615:15guyCan you give an example?#2018-08-1615:15guyLike do you mean just doing something like (s/keys :opt [::fake-spec]) ?#2018-08-1615:24dadairThat fact allows me to avoid circular spec dependencies; the s/keys doesnā€™t validate that the spec exists#2018-08-1618:59ericnormand@guy that's what I mean#2018-08-1600:37gfredericksoh weird#2018-08-1600:37gfredericksdoes :req?#2018-08-1602:20ericnormanddon't think so#2018-08-1602:20ericnormandthere's a pretty strong separation between checking keys and checking their values#2018-08-1602:20ericnormandall values are checked, even if they're not required#2018-08-1602:21ericnormandand all keys are allowed, even if they're not defined#2018-08-1612:03gfredericksyeah I guess that's consistent at least#2018-08-1618:59ericnormandyeah#2018-08-1618:59ericnormandI think that's why it's called "keys". It's just about the keys, and whether they exist.#2018-08-1619:34joost-diepenmaatthough you will get errors if you try to generate values for keys specs with unspecā€™d keys#2018-08-1620:13guy^ yeah thats why i think iā€™ve never found it to be a problem#2018-08-1620:14guyWhenever iā€™ve tried to exercise it for an fdef or whatever#2018-08-1620:14guyit would throw that 100 tries error etc#2018-08-1620:14guyiirc#2018-08-1622:04richiardiandreacan I stest/instrument a function that has been replaced with with-redefs ? would the spec still work on it?#2018-08-1701:50Alex Miller (Clojure team)I think both are working by swapping vars, so probably, but it might get weird. try it and see.#2018-08-1702:03richiardiandreaSeems like it is when the with-redefs happens before the instrument. But I've got mixed feelings about this idea šŸ˜„ #2018-08-1702:04richiardiandreaProbably better not to do that#2018-08-1702:04Alex Miller (Clojure team):)#2018-08-1710:08Oliver GeorgeIs this a common pattern: isolate a fn by stubbing everything else and running stest/check on it.
(defn check-all []
  (let [syms (stest/instrumentable-syms)]
    (doseq [sym syms]
      (let [stub-syms (disj syms sym)]
        (stest/instrument stub-syms {:stub stub-syms})
        (stest/summarize-results (stest/check sym))
        (stest/unstrument)))))
#2018-08-1712:10Alex Miller (Clojure team)First time Iā€™ve seen it, but interesting#2018-08-1801:28Oliver GeorgeI was thinking it would give you confidence that the inputs and outputs through the code match up. So essentially a cheap type checking substitute. The downside of that approach is that for full code coverage the generated data needs to trigger all branches of conditionalsā€¦ I think thatā€™s probably not something you can rely on.#2018-08-1717:21colinkahnLooks like it isn't an option, but curious if there's a way to specify an fspec for check like you can for excercise-fn#2018-08-1717:25colinkahnMy goal is to check a fn with an fspec whose :fn is specific to a more narrow generator.#2018-08-1718:00Alex Miller (Clojure team)you can do that with instrument first#2018-08-1718:01Alex Miller (Clojure team)with the :replace functionality#2018-08-1718:01Alex Miller (Clojure team)if I understand your goals that is#2018-08-1718:19colinkahnLooking at the docs for instrument I think :spec should do what I want, but trying the following still uses the registered fdef for files-reducer instead of the override fspec#2018-08-1718:19colinkahn
(st/instrument `files-reducer {:spec {`files-reducer (s/fspec :args (s/cat) :ret any? :fn (fn [_] false))}})
(expound/explain-results (st/check `files-reducer))
#2018-08-1904:32pablorenoob question here, how do I make a spec for a fixed length collection of another spec?#2018-08-1904:34pabloreIs this a good solution?
(s/def ::my-10-specs (s/and #(= 10 (count %)) (s/+ ::my-spec)))
#2018-08-1904:47pablorewell, using
(s/def ::my-10-specs (s/& (s/+ ::my-spec) #(= 10 (count %)))
fails after 100 tries
#2018-08-1904:47pablore(when generating)#2018-08-1906:54fedregHi all, Is there a way to see the data that is returned from a function that I call stest/check on? For example, I can see all the data generated for that functionā€™s inputs by passing in a reporter-fn to the opts but I canā€™t seem to find a way to access that data for the returns. Iā€™d love to pass a function in to format the output data as needed. thx for any pointers!#2018-08-1911:32gfredericksis there a coll-of?#2018-08-1913:39pabloreyes, but it doesnā€™t work well with s/cat.#2018-08-1913:44gfredericksoh so you want something that can go in a regex in particular? like (){12} from string regexes? I can imagine a macro that expands to an s/cat with 12 instances of the spec, but that'd be unfortunate. seems like something that could be done better built-in it does seem uncommon though; where did this come up for you?#2018-08-1911:32gfrederickssorry, that was for @pablore#2018-08-1911:34gfredericks@fedreg there might not be a supported way, but if you make a TCHECK jira ticket I'll make sure it's at least available to the reporter-fn
#2018-08-1913:30gfredericks@fedreg did it myself https://dev.clojure.org/jira/browse/TCHECK-152#2018-08-1913:47gfredericksActually, now that I look closer, I don't think this can be fixed in test.check -- it's spec that's hiding the return value#2018-08-1913:48gfredericksspec creates a test.check property that just returns true when the test passes#2018-08-1915:59fedreg@gfredericks Thanks for taking a look!! (and making the ticket!).. Ya, I saw that it was coming back ok from test.checkā€¦ Seems like there should be a way to see the resultsā€¦ I wonder if the :result is blocked by spec intentionally or if it is just an oversiteā€¦#2018-08-1916:00gfredericksit'd have to be wrapped if anything#2018-08-1916:00gfredericksspec is returning true, and test.check checks for truthiness; so to test functions that can return falsey you'd have to wrap with {:return ___} or something like that#2018-08-1916:01gfredericksdoes stest/check return that true also, or do you only see it in the reporter-fn? I think it's probably only the reporter-fn, in which case I don't see a big downside to spec doing that wrapping#2018-08-1916:02gfredericksit can't reasonably leak anywhere else, because the return value from a full quick-check run can't include the return value since there's n of them instead of 1 of them#2018-08-1916:30fedreg@gfredericks sorry, not sure which true you meanā€¦ The return data is evaled here but then just a true is returned if everything is ok. So, if all is good with the specs, just a true is returned. Details only returned if there is a spec validation#2018-08-1916:32gfredericksIf you replaced that line with {:return ret} I think you'd have what you need and nothing else would break#2018-08-1916:32gfredericksOr at least nothing that wouldn't be easily fixed#2018-08-1916:32gfredericksBut I'm not a spec lawyer so I dunno#2018-08-1916:39fedregYes... I'll file a ticket. Maybe I'm the only one who would find that useful.. thanks for taking a look regardless @gfredericks !#2018-08-1922:39jeayeWhat's going wrong here? It seems odd that args would be treated any differently than the s/valid? checks prior.
user=> (s/def ::any-args (s/cat :name keyword?))
:user/any-args
user=> (s/valid? ::any-args [:ok])
true
user=> (s/valid? ::any-args [1])
false
user=> (defn foo [aa])
#'user/foo
user=> (s/fdef foo :args (s/cat :aa ::any-args))
user/foo
user=> (dorun (clojure.spec.test.alpha/instrument))
nil
user=> (foo [:ok])

clojure.lang.ExceptionInfo: Call to #'user/foo did not conform to spec:
                            In: [0] val: [:ok] fails spec: :user/any-args at: [:args :aa :name] predicate: keyword?
#2018-08-2000:01Alex Miller (Clojure team)per the s/fdef doc string, ā€ :args A regex spec for the function arguments as they were a list to be passed to apply - in this way, a single spec can handle functions with multiple aritiesā€#2018-08-2000:03Alex Miller (Clojure team)or I guess maybe the confusing thing is that regex ops nested inside each other are combined to describe the same sequential collection#2018-08-2000:04Alex Miller (Clojure team)so you have an effective foo args spec of (s/cat :aa (s/cat :name keyword?)), however nested s/cats like this combine to form a spec for a single sequential collection (the args list)#2018-08-2000:04Alex Miller (Clojure team)what youā€™re wanting is that ::any-args described a new nested collection#2018-08-2000:05Alex Miller (Clojure team)you can do that with (s/fdef foo :args (s/cat :aa (s/spec ::any-args)))#2018-08-2000:06Alex Miller (Clojure team)the s/spec wrapper here forces a new (non-combined) collection level#2018-08-2001:43jeayeThanks, Alex. Got it.#2018-08-2001:45jeaye@alexmiller Is there any issue with using (s/spec ...), or unintended side effects? I'm thinking about defn-spec, within Orchestra, which provides fn specs inline. It seems like defn-spec should expand every arg spec to be wrapped in s/spec, based on your explanation.#2018-08-2117:01djtangoI've run into issues with regex flattening in arg-specs so I think you're right here#2018-08-2117:01djtango(w.r.t. using s/spec how you've described)#2018-08-2001:46jeayeSince none of them should ever flatten to affect the outer s/cat for the :args.#2018-08-2002:53Alex Miller (Clojure team)I canā€™t speak to what Orchestra or defn-spec does, sorry. There are possible use cases for wanting the flattening here, although I agree thatā€™s probably unusual.#2018-08-2021:49richiardiandreaWe were discussing a bit on #clojurescript about having the possibility to specify a custom printer (a la *explain-out*) for check results.#2018-08-2021:49richiardiandreaIs it something worth working on, or at least opening a JIRA about?#2018-08-2022:24Alex Miller (Clojure team)Sure#2018-08-2114:56Wilson Velez:+1:#2018-08-2218:59ikitommihelp most welcome with the cljs+spec issue: https://github.com/metosin/reitit/issues/127#2018-08-2309:13lmergeni'm struggling a bit on how to design my spec rules. specifically, i have a complex object that i want to spec, and this object can have several traits, depending upon where it is. so, for example, let's say i have an animal object with the following definition:
(s/def ::vaccinations (s/coll-of {:foo :bar})
 (s/def ::medical (s/keys :opt-un [::vaccinations])
(s/def ::animal (s/keys :req-un [::medical]))
so here we can see that an animal has a optional medical information about vaccinations. most of my functions just work with animals, and do not care whether some specific nested property it true. however, some other functions do care whether an animal is vaccinated. for now, i have this:
(s/def ::vaccinated-animal (s/and ::animal #(some? (get-in % [:medical :vaccinations])))
but this feels a bit hacky, and doesn't scale very well when you want to compose multiple properties into it. it also feels like i'm developing a poor man's trait system here, which is maybe the intention, maybe not regardless, what would be the proper way to organize this stuff be?
#2018-08-2309:14guyare you trying to do s/merge maybe?#2018-08-2309:15guyhttps://clojuredocs.org/clojure.spec.alpha/merge#2018-08-2309:17lmergeni'm not sure how to structure that in this specific case#2018-08-2309:18lmergenwould i s/merge the ::animal into an ::vaccinated-animal ? but how would i re-define a property of a nested map in this case (i.e., :req-un the ::vaccinations rather than :opt-un) ?#2018-08-2309:19guyCan you show me an example shape of the data?#2018-08-2309:19lmergenhmm ? the spec is up there ?#2018-08-2309:20guy(def animal {:medical [{:foo :bar}]})#2018-08-2309:20guylike that yeah?#2018-08-2309:20lmergenno#2018-08-2309:20lmergen(def animal {:medical {:vaccinations [:foo :bar]}})#2018-08-2309:20lmergenor#2018-08-2309:20lmergen(def animal {:medical {}})#2018-08-2309:21guyah yeah sorry#2018-08-2309:21lmergenso my problem is that i will be needing the s/merge nested traits#2018-08-2309:22guySo basically (s/def ::medical (s/keys :opt-un [::vaccinations]) or (s/def ::medical (s/keys :req-un [::vaccinations])#2018-08-2309:22lmergenyes#2018-08-2309:22guygot ya#2018-08-2309:23lmergenbut i cannot just rename ::medical to ::medical-with-vaccinations#2018-08-2309:23lmergenbecause then the key would change#2018-08-2309:23guyyou can do :some-ns/medical#2018-08-2309:23lmergenyeah i know, would that really be the solution here?#2018-08-2309:23lmergenit seems inappropriate#2018-08-2309:24guyI liked the use of that from http://conan.is/blogging/clojure-spec-tips.html#2018-08-2309:24guylet me get the bit#2018-08-2309:24guy
(s/def :company.department.employee/name string?)
(s/def :company.department.employee/payroll-number nat-int?)
(s/def :company.department/employee 
  (s/keys :req [:company.department.employee/name
                :company.department.employee/payroll-number]))
(s/def :company.department/employees (s/coll-of :company.department/employee))
(s/def :company.department/id uuid?)
(s/def :company/department (s/keys :req [:company.department/id
                                         :company.department/employees]))
#2018-08-2309:24guythis is sorta what i would do in ur case#2018-08-2309:24guyNaming conventions for nested maps#2018-08-2309:25guyIf you read that part#2018-08-2309:27guyThe reason i think the above is useful to you is it allows you to use nsā€™s to sorta describe what you are talking about#2018-08-2309:27guyWhich is ultimately vaccinated animals vs just animals#2018-08-2309:27lmergenright! this is useful#2018-08-2309:27lmergenok#2018-08-2309:27guyDoes that help at all?#2018-08-2309:27lmergenyes#2018-08-2309:27guyok great#2018-08-2309:27guyšŸ˜…#2018-08-2309:28lmergenso basically, additional namespaces#2018-08-2309:28guyits defo one answer#2018-08-2309:28guyI like it as it makes your specs predictable#2018-08-2309:59conanyes, this! no surprises!#2018-08-2311:21dominicmI don't suppose anyone has created a spec for "edn" have they?#2018-08-2311:49guyI wanna see it šŸ˜±#2018-08-2313:13dominicmI want to test aero better, there's gaps where certain data structures don't work. #2018-08-2313:13dominicmOne really hard part is getting it to generate #ref#2018-08-2314:40bbrinckIt depends on what you want to test, but on strategy for building generators like this is to some from a small set of valid keys OR random keys#2018-08-2314:42bbrinckOr am I misunderstanding what is difficult about generating refs?#2018-08-2314:56dominicmI want to generate valid nested paths, e.g. #ref [:adjfkf :bjkadf :foobar]#2018-08-2314:56dominicmwhich can have many levels of nesting#2018-08-2314:58bbrinckIs the goal of the generator to produce refs that will always resolve, or just be valid syntax (but maybe not resolve), or or a mix of resolved and unresolved refs?#2018-08-2314:59bbrincki.e. do you want all paths to be valid (assuming ā€œvalidā€ means ā€œit resolves to some actual path in the mapā€) or is it OK if some are invalid?#2018-08-2315:14dominicmA mix would be useful, but you can't spec that an exception happens can you? #2018-08-2315:14dominicmI think nil is the current behaviour, but I don't like that for other reasons. #2018-08-2315:16bbrinckThatā€™s true, you canā€™t spec an exception occurs, although you could generate EDN with a mix and then use a normal generative test (i.e. not using check directly) to say either an exception should occur or some other thing should happen#2018-08-2315:17bbrinckAnyway, if you want a mix, one thing you could try is to normally name keys from a small set in the generator e.g. {:a :b :c} and then generate refs of a usually small length from those same keys e.g. (s/coll-of #{:a :b :c} :kind vector?). Basically youā€™re just assuming youā€™ll get some number of ā€œcollisionsā€ but accepting some refs wonā€™t resolve#2018-08-2315:19bbrinckIf you want to ensure every path is going to resolve, then I think youā€™ll have to build a custom generator for the entire config. Iā€™d start by generating a set of paths that will be included in the map. Then, you can take those valid paths and write a function that fills in the values for each path, which may contain a ref to another valid path#2018-08-2316:50dominicmStarting with valid paths sounds like a great idea! #2018-08-2316:50dominicmEspecially as some things will be sets or vectors. #2018-08-2317:14bbrinckSuper hacky (and there are bugs), but hereā€™s a quick outline of how a generator might work. Hope it helps! šŸ˜„ https://gist.github.com/bhb/b4ca38670c9c20f3d2ed8623aca5ec11#2018-08-2317:16bbrinckIf the generator works a high percentage of the time, you could also just generate a lot of examples and filter out ones where the refs are invalid#2018-08-2317:48dominicmThat's amazing, thank you! #2018-08-2410:39mishahttps://clojurians.slack.com/archives/C1B1BB2Q3/p1535016259000100#2018-08-2410:39guyšŸ¤”#2018-08-2410:39mishabeware of this:
(def some-keys [:a/key :a/nother-key])
(s/def :a/spec (s/keys :req (var-get some-keys)))
(s/valid? :a/spec {:a/key 1 :a/nother-key 2}
=> true
#2018-08-2410:40mishatrue is because (s/keys :req (var-get some-keys)) is equivalent to empty (s/keys)#2018-08-2410:41misha(var-get some-keys) this does not work inside s/keys the way one would expect. (first of all, it should be (var-get #'some-keys))#2018-08-2411:07guy@conan this is good to know!#2018-08-2412:49conanah yes, thanks for pointing this out!#2018-08-2410:41mishaat least in clojure. mb in cljs it does#2018-08-2410:42misha
(s/exercise (s/keys :req (var-get some-keys)))
=> ([{} {}] [{} {}] [{} {}] [{} {}] [{} {}] [{} {}] [{} {}] [{} {}] [{} {}] [{} {}])
#2018-08-2410:42mishadespite the :req#2018-08-2410:43mishaand it does not even complain, that :a/key spec is not defined.#2018-08-2410:46mishanext: I did try to follow the similar strategy for nested maps:
(s/def :company.department/employees (s/coll-of :company.department/employee))
and it becomes hairy very-very soon with very little nested structures. and unless you will use -un (which defeats whole purpose of namespaces in just few calls down the call stack) - your get-ins and update-ins become unreadable and unbearable
#2018-08-2410:47mishainstead, I went with shorter namespaces, reflecting domain objects/classes, w/o including nestedness info into spec namespaces.#2018-08-2410:49mishaexample above does not look too terrible, but it is really a hello-world grade one, and is already whole line long.#2018-08-2411:07guyThanks @misha really good to know#2018-08-2412:52conanwe have complex nested structures and don't really encounter a problem, but maybe we have more complex but less nesting - going down our tree, we quickly hit refs to other top-level entities, so maybe that keeps it ok. I always prefer having predictability over terseness, i don't really see much value in the latter.#2018-08-2412:54conanin that example above for instance, it's likely i wouldn't chooes to model an employee as something in the company hierarchy. i'd be more likely to have a :conan/company containing :conan.company/departments, and each of those containing lots of :conan/employees (rather than continuing the nesting).#2018-08-2412:55conanor at the very least, if employees were in the company hierarchy, i'd look to break out of it and give each employee a :conan/person, i.e. a thing that exists independently of the company hierarchy but that can be attached.#2018-08-2412:59conanThe pattern where :my/a has a collection of :my.a/bs, each of which is a :my.a/b, which has a collection of :my.a.b/cs, each of which is a :my.a.b/c and so on works very well for us#2018-08-2413:00conanbut from a modelling perspective it's often useful to consider breaking that pattern when possible, so that a :my.a.b/c ends up with a collection of :my/zs#2018-08-2413:00conanit's tricky to read this stuff without a concrete domain in mind#2018-08-2413:16misha@conan I am not choosing terseness for the sake of terseness, I am suggesting to think twice before embedding "how one domain object is nested in another one maybe-only-in-just-one-out-of-many-existing-contexts" into spec namespaces#2018-08-2413:19mishafor example, if there is a code which manipulates some leaf of a deep nested structure outside of the nested structure context, does it really need to know that it is :foo.bar.baz.quux/fred and not just :quux/fred?#2018-08-2413:19conanabsolutely, gotta pick what works for you. everything we do is stored in datomic, so this is always our graphy structure#2018-08-2413:20mishaofc, the easy escape would be :*-uning all the keys. but that is just robbing yourself of power namespaces gave you#2018-08-2413:20conanin your example though, your function may well not need to know that - but as a developer, I do, and I don't like surprises#2018-08-2413:20conanyeah unqualified specs are just for data coming from external systems imho#2018-08-2413:21conani'm definitely not saying this is the "right" way to do it, not even the only way - it's just intended as a useful starting point if you're unsure. I'll update the post to make that more clear#2018-08-2413:21mishaI agree with what you are saying about knowledge, but only in very limited set of usecases#2018-08-2413:22mishaand I'd say, those use cases are solved by namespaces for functions foo.bar.baz/quux-fred-manipulator vs e.g. quux/fred-manipulator (dont read too much into it)#2018-08-2413:24conanreally appreciate the feedback though#2018-08-2413:24mishathe same way as you can have your own update fn, more specific than clojure.core/update one, in another ns#2018-08-2413:24mishawith similar semantics, but with more knowledge about thing it sh/would be applied to so you-as-developer-would-know#2018-08-2413:25conanyep, i can see that; we benefit a lot from transacting data generated from these specs into datomic, and it may colour my thinking more than i realise#2018-08-2413:26mishaoh, I tried keeping datomic db for such nested nss - was tough. and again - you are robbing yourself of a graph, replacing it with rigid tree#2018-08-2413:26conanit's really quite astonishing that i can often write specs for a domain, write schema from those specs, and the generators will produce data that transacts first time into datomic; i don't think i've ever once come close to that level with other databases#2018-08-2413:26conanwe have a very graphy structure, and this works well with it#2018-08-2413:27mishathat is true, that potential is awesome!#2018-08-2413:27conanessentially, anything that has isComponent ends up being another level of namespace hierarchy (i.e. :a.b.c/d), and anything that is not usually resets back to the top of the hierarchy (i.e. :e/f)#2018-08-2413:29mishait is very close to that, yes, I agree. although company.department/employee, we crucified above - does not fit that opieop#2018-08-2413:30conanyeah i'm terrible at examples, i'll think up a more evocative, less boring one over the weekend#2018-08-2413:31mishathat's fine, it is an example from linked article anyway#2018-08-2413:33mishaI have an example, contradicting deeply-nested isComponent rule: :geopoint/lat/long#2018-08-2414:36bjais there anything like the plumbing.core/defnk (which used Schema) for spec?#2018-08-2416:08snowellI have a function I'm trying to spec that takes 2 args, a map and a string that needs to be a key in that map. How do I reference the first arg in the spec for the second?#2018-08-2416:17taylor@snowell you could use s/and with s/cat to check the args:
(defn foo [x y] (get x y))
(s/fdef foo
        :args (s/and (s/cat :x map? :y string?)
                     (fn [{:keys [x y]}] (contains? x y))))
#2018-08-2416:18snowellYeah, I'd gotten there actually, but I was trying to get something that worked with generators, and apparently that's a lot more fickle#2018-08-2416:18snowellUnless it happens to randomly generate the right string(s)#2018-08-2416:18taylorhmm yeah because the OotB generators are created using just the first spec in the s/and#2018-08-2416:19taylorbut I don't think it'd be too hard to make a generator that will do this#2018-08-2416:19snowellIt's not the end of the world. Just playing with a new toy šŸ™‚#2018-08-2416:27taylor@snowell this might work:
(s/def ::foo-args
  (s/with-gen (s/cat :x map? :y string?)
              #(gen/fmap (fn [m] [m (rand-nth (keys m))])
                         (s/gen (s/map-of string? any?)))))
#2018-08-2416:27raymcdermottbasic question on s/keysā€¦#2018-08-2416:27raymcdermottgiven this#2018-08-2416:29raymcdermottcan I reuse the keys in q-min-n that I have defined in q?#2018-08-2416:30taylormaybe spec/merge is what you need @raymcdermott. (s/merge ::q (s/keys :req [::min-n]))#2018-08-2416:30raymcdermottah, nice#2018-08-2416:30raymcdermottIā€™ll give it a try - thanks @taylor#2018-08-2416:36snowell@taylor It generated something, though it looks weird. It's really OK though, thanks anyway!#2018-08-2705:19mishais there any downside in "conforming unqualified maps into qualified ones"? e.g.
(qualify
  (s/keys :req-un [:foo/bar])
  {:bar 1})
=>
{:foo/bar 1}
#2018-08-2709:40ikitommithe result doesnā€™t conform anymore to the spec.#2018-08-2711:51Alex Miller (Clojure team)thatā€™s often true regardless#2018-08-2712:09misha@alexmiller have you considered doing this (qualify keys) as part of s/keys or s/conform behavior?#2018-08-2712:18Alex Miller (Clojure team)I am confused by your question #2018-08-2712:31mishaopieop will such qualify be a part of clojure.spec someday? if "definitely not", why?#2018-08-2720:26eoliphanthi, quick question. what's the best way to handle getting 'context' into spec functions (or is this even a good idea?) For instance, I'm playing around with a little data-driven schema definition tool, and I want an attribute type to be either one of a set of primitive types, or another type defined in the schema. so I want to do something like (s/or ::attribute-type #{:type1 :type2} (fn [val] (is-val-in-list val external-list)). where external-list might have been setup right before a call to s/valid?, etc#2018-08-2721:38mishayou might keep context in an atom, and access it inside of your predicate#2018-08-2723:40eoliphantyeah i'd been thinking about that, but it feels weird lol. I'd really like to have the context scoped down to what's going on at that time, but I may give that a whirl to start#2018-08-2805:41mishaYou control what goes into that atom, and when. After all, global spec registry is an atom too.#2018-08-2817:30UlrichI do not understand this mismatch, as I see it. Do you know the reason why :gen takes a function and not a generator? (s/exercise boolean? 1 gen/boolean) ok but (s/exercise (s/spec boolean? :gen gen/boolean) 1) bad. So I write (s/exercise (s/spec boolean? :gen (constantly gen/boolean)) 1). Is there a better way?#2018-08-2817:39seancorfieldhttps://www.dropbox.com/s/plwu44v61adscap/Screenshot%202018-08-28%2010.39.11.png?dl=0 The first s/spec form works for me, using (constantly gen/boolean) does not...#2018-08-2818:11Alex Miller (Clojure team)Specs defer gens via functions to avoid loading clojure.spec.gen.alpha when not necessary. When you call exercise, it takes an overrides map which is a map of spec name to function returning generator.#2018-08-2818:13Alex Miller (Clojure team)(s/exercise boolean? 1 gen/boolean) is not ok, that last gen/boolean is just not being used (boolean? has a built-in generator)#2018-08-2818:15Alex Miller (Clojure team)I usually do something like (s/exercise (s/spec boolean? :gen #(gen/boolean)))#2018-08-2818:16Alex Miller (Clojure team)I suspect something is wacky in the macrology thatā€™s making the constantly version not work there#2018-08-2818:40seancorfield@alexmiller I'm curious as to why the unwrapped version worked in my REPL then...?#2018-08-2818:40Alex Miller (Clojure team)unwrapped version of?#2018-08-2818:41seancorfieldhttps://www.dropbox.com/s/3y3bm4hjvipthhp/Screenshot%202018-08-28%2011.40.39.png?dl=0#2018-08-2818:41seancorfieldSeems to work both with and without the #( ... ) around gen/boolean...#2018-08-2818:42Alex Miller (Clojure team)not sure - since this one has a built-in, that might be masking something too#2018-08-2818:52Alex Miller (Clojure team)oh, so gen/boolean (where gen is clojure.spec.gen.alpha) is actually already a thunk#2018-08-2819:01Alex Miller (Clojure team)so gen/boolean works as is and #(gen/boolean) is a function that invokes gen/boolean and returns the generator, so is exactly the same thing#2018-08-2819:02Alex Miller (Clojure team)but (constantly gen/boolean) is a function that returns gen/boolean not invoked, which is a function rather than a generator#2018-08-2819:03Alex Miller (Clojure team)so going way back to the top @ulrich.becker said ā€œ(s/exercise (s/spec boolean? :gen gen/boolean) 1) is badā€, but itā€™s not bad - thatā€™s exactly what you want here (the missing piece of information being that clojure.spec.gen.alpha already wraps gens in thunks#2018-08-2819:05Alex Miller (Clojure team)tumbleweed#2018-08-2819:38seancorfieldAh!! šŸ’” Now it all makes sense. That also explains why this did not work: (s/exercise (s/spec boolean? :gen (fn [] (println "Gen!") gen/boolean)) 1)#2018-08-2819:39seancorfield(I wanted to see how often the thunk was called and when I corrected it to call (gen/boolean) the answer was "just once", regardless of the exercise size)#2018-08-2920:30ChrisHas anyone done any research on 'learning' specs given a data set?#2018-08-2920:30taylorhttps://github.com/stathissideris/spec-provider#2018-08-2920:31ChrisWow, awesome! That was so fast I thought you were a bot!#2018-08-2920:32ChrisThanks!#2018-08-2920:37ghadiSee ambrose bonnaire-sargeants talk from last year's Conj#2018-08-3016:50Daniel HinesI'm writing a web app that writes to our CRM. We have a bunch of third party plugins installed on this CRM that I'm not allowed to touch. One function of these plugins is to validate incoming user-inputted data; however, somewhere in the validation, the plugins are occasionally hitting fatal exceptions, but they give no clues as to what it might be. The third party won't provide any documentation, and decompiling their code is expressly forbidden in their EULA. Through significant trial and error, I've narrowed down the cause to specific combinations of input values, which appear completely arbitrary to me. There are about 100 different inputs, some of which have hundreds of possible values, and the order the inputs seems to matter, making this very difficult to address by hand. Would it make sense to use spec in this situation to systematically explore combinations of data cause errors?#2018-08-3017:10jeayeI've hit an issue with instrumentation wherein the instrument macro generates and returns a vector of symbols of those fns instrumented which is too large for the JVM to appreciate. So, I see Method code too large! failing the compilation instead. This seems like something spec will need to address, in the long term, as more people end up spec'ing more fns. Right now, looks like this project is around 300 spec'd fns and the issue is arising. Both Clojure and ClojureScript seem to be subject to the issue.#2018-08-3017:14ghadican you give a repro for the case @jeaye?#2018-08-3017:28jeayeYup. I'll have one up in a bit.#2018-08-3017:55jeaye@ghadi https://github.com/jeaye/instrument-too-many#2018-08-3017:56jeayeA repro case in 20 lines or so. Couldn't get it going with Clojure, but it certainly happens with ClojureScript. When I look at the implementation of instrument for each, it'd seem both would be subject, but perhaps not.#2018-08-3017:56jeaye100% repro here though.#2018-08-3017:56seancorfield@d4hines That sounds very much like the scenario Stu Halloway was talking about in one of his presentations on spec...#2018-08-3017:57Daniel HinesWould that be this one? https://www.youtube.com/watch?v=VNTQ-M_uSo8#2018-08-3017:58seancorfieldI don't know. I don't think I attended SL in 2016. May have watched that online tho'. I thought it was a talk from one of the Conj's or West's...#2018-08-3017:58Daniel HinesOk, I'll look for it. Thanks!#2018-08-3019:46bbrinckThis blog post may be useful as well: http://blog.cognitect.com/blog/2017/6/19/improving-on-types-specing-a-java-library#2018-08-3019:48bbrinckKeep in mind that exercise will generate values that match the spec, but to get the power of shrinking (which is what you want, I expect), youā€™ll want to combine spec with test.check#2018-08-3019:53bbrinckI really like the test.chuck library for this, you can write something like the following (totally untested and written in slack, so beware šŸ™‚ )
(deftest find-bug
  (checking
     "all valid inputs don't cause exception"
     100
     [args (s/gen ::args-spec)]
     ;; replace 'string?' with whatever the actual expected return type is
     (is (string? (apply third-party-api/some-fn args)))))    
#2018-08-3018:07taylorIIRC there might be a talk he gave about writing specs for a Java charting library and finding a (JVM?) bug by "fuzzing" it with random spec checks?#2018-08-3018:19jeaye@ghadi Is that something which you can repro?#2018-08-3018:27dadairHas anyone encountered an open-source HL7-FHIR spec library? I have been creating one for work but Iā€™d be interested in pulling it out into an open-source project if others wanted to collaborate on it.#2018-08-3018:37ghadiwe would find that useful @dadair#2018-08-3018:39ghadi@jeaye haven't had a chance to try yet, but off-hand: try to get rid of the nested do https://github.com/jeaye/instrument-too-many/blob/master/src/instrument_too_many/macro.cljc#L8#2018-08-3018:39ghadii think the compiler special case's top-level do's but not nested ones#2018-08-3018:39jeayeThat macro's just making the fns though. That macro doesn't cause the error; the error is with instrumentation.#2018-08-3018:42jeayeIf you comment out the (stest/instrument) form, it compiles just fine.#2018-08-3020:42pabloreIs there anyway to decompose a spec definition? I want to use a s/keys spec to define a database table#2018-08-3020:49pabloreoh using s/describe#2018-08-3021:19Alex Miller (Clojure team)or s/form#2018-08-3109:51mishasanity check: how do you share specs between libraries/modules-of-app?#2018-08-3109:53UlrichHelp! I need my generated test data to have unique ids in various places (not uuids, though, just positive integers). I hacked a stateful generator (using an atom) for that which cannot be the right wayā€¦? How do I do stateful generators? exercise and sample seem to have no way to allow a stateā€¦#2018-08-3109:53Ulrich@misha require the namespace - itā€™s magic#2018-08-3109:54mishameaning: package it as a shared lib itself?#2018-08-3109:56Ulrichmaybe i misunderstood. If you want to use specs defined in one namespace, just require that namespace where you wish to use the spec. If it is in a lib the declaring namespace should be in the libā€™s jar ā€¦ Is that what you asked?#2018-08-3109:58mpenet@ulrich.becker you can do an ugly hack with gen/fmap with a closed around atom and a (gen/return nil)#2018-08-3109:58mpenetthere's probably a better way tho#2018-08-3109:58mpenetsomething like that: (gen/sample (let [a (atom 0)] (gen/fmap (fn [_] (swap! a inc)) (gen/return nil))))#2018-08-3109:59Ulrich
(gen/no-shrink (gen/fmap (fn [_] (swap! unique-id inc))
                                            (gen/return 0))
#2018-08-3109:59Ulrichlike this?#2018-08-3109:59codonnell@misha we share specs between a backend clojure repo and a frontend clojurescript repo at my work by putting the specs in a third repo that has the specs in cljc files. It works pretty well IMO.#2018-08-3109:59mpenet@ulrich.becker it's similar yeah#2018-08-3110:00mpenetbut messing with (gen/return nil) is usually a smell#2018-08-3110:00mpenetthis kind of breaks the purpose of using prop testing in the first place#2018-08-3110:02UlrichBut the need for unique ids is very real, somebody must have a better solution? (except using UUIDS)#2018-08-3110:02mpenetwhy uuids are bad?#2018-08-3110:03Ulrichnot bad just uglyā€¦#2018-08-3110:03mpenetyou can also use something like gen/vector-distinct of ints#2018-08-3110:03Ulrichnope vector/distinct does not work#2018-08-3110:03mpenetah?#2018-08-3110:21mpenetright, it would return a collection, not really useful#2018-08-3110:22mpenetI guess another alternative is to let the random part untouched and keep track of values used in one way or another, I guess again you can do that with an atom or something. Still feels a bit odd#2018-08-3110:23mpenetsomething like that? (let [bin (atom #{})] (gen/such-that (fn [x] (and (not (get @bin x)) (swap! bin conj x)) ) (gen/int)))#2018-08-3110:23mpenetbut that atom will grow, and that might hurt#2018-08-3110:25mpenetit can also fail to generate valid values after a while#2018-08-3110:26mpenetso yeah, (g/return nil) šŸ™‚#2018-08-3110:29mpenetjust for fun another dirty trick: (gen/fmap hash (gen/uuid)), but you would get neg values, so (gen/sample (gen/fmap #(bit-and (hash %) 0x7FFFFFFF) (gen/uuid))) would work (with potential collisions tho)#2018-08-3111:15misha@codonnell will sending specs as an edn make any sense?#2018-08-3122:56codonnellI'm not sure what you mean by this. Could you elaborate?#2018-09-0106:26mishafront-end sends "back-end, give me your specs" back-end responds with subset of spec registry as edn front-end reads, evals#2018-09-0106:27mishathis skips fron-end re-compilation if specs change#2018-09-0112:25codonnellI suppose you could do this if you need to update specs at runtime. I haven't personally run into a scenario where building a static registry at compile time was insufficient.#2018-08-3115:07lilactownspec-tools is super impressive#2018-08-3115:17lilactowngeneral question about spec: one of the things that keeps coming up for me, is that Iā€™d like a language to describe a schema or shape of data that I would need to translate to some other schema language (e.g. GraphQL, dynamoDB, etc.). As a spec neophyte, my first instinct/desire is to reach for clojure.spec as the schema language, and somehow convert the spec to the other schema language. But the more I think about it, the more it feels like spec is much better setup to be the tool to do the conversion, and itā€™s left up to me to create my own schema language in Clojure. Which one is correct?#2018-08-3115:23dadairI think there was some project that went from spec to various schema formats, but I canā€™t seem to recall it right now#2018-08-3115:24dadairOr maybe it was datomic schema to other schema#2018-08-3115:25lilactownI was just looking at spec-tools, which can convert specs to JSON Schema and Swagger Schemas#2018-08-3115:25lilactownwhich might work for my use case, but then I need to write JSON Schema => GraphQL or DDB or datomicā€¦#2018-09-0101:26spiedenhas anyone done anything with strictly validating some data against a spec? by this i mean not allowing anything other than whatā€™s specified. e.g. extra map keys, etc.#2018-09-0113:36bbrinckCheck out https://github.com/bhauman/spell-spec#2018-09-0102:34seancorfield@spieden Several libraries have facilities for that and it's fairly easy to do. It's just not necessarily a good thing to do.#2018-09-0106:32misha@spieden one example of such strict map validation would be s/map-of. if it specifies keywords-to-strings map - you can't have string key in there But, as Sean pointed out, it is kinda trivial to implement yourself with a particular shade of behavior you are looking for.#2018-09-0118:35jeayeI've a repro for a bug where ClojureScript's spec instrument fn generates too large a function for the JVM and compilation fails. This happens when too many fns are spec'd. https://github.com/jeaye/instrument-too-many Looking for some confirmation.#2018-09-0122:18andy.fingerhutIt appears specific to ClojureScript. When I tried a similar thing in Clojure/JVM, there was no problem I noticed.#2018-09-0123:14jeaye@andy.fingerhut Thanks. I've also tried it on Clojure JVM and I couldn't reproduce.#2018-09-0218:27bbloomwhat do people do when you have different keys for different specs, but want to handle them generically? do you just use an extra key to index back in to that map? for example, iā€™m trying to model a WebAssembly AST, which has ā€œmodulesā€ which are made up of ā€œsectionsā€ and each section has a vector of ā€œfieldsā€. but while fields have common structure across types of sections, iā€™d like to have ::funcs and ::types and ::imports instead of just ::fields ā€” my least bad idea is to add (s/def ::fields-key #{::funcs ::types ā€¦}) and then indirect through that#2018-09-0218:28bbloomthe two other more bad ideas are 1) just use the abstract name ::fields and rely on the specā€™s global name validation. This isnā€™t ideal because (A) it fails to spec the fact that fields in the funcs section must conform to ::func and (B) precludes unname-spaced keys, which is what iā€™m actually doing & would have to do a much bigger refactor for that#2018-09-0218:29bbloomand 2) just copy the fields to the other name like (let [{:keys [fields]} ast, ast (assoc ast ::funcs fields)]#2018-09-0218:29bbloomthis is really nice#2018-09-0218:29bbloombut it makes rewrites basically impossible#2018-09-0218:29bbloomb/c if you edit ::funcs, then ::fields is stale#2018-09-0218:30bbloomis this problem explanation clear? if not, why not? if so, what do you guys do about this sort of problem?#2018-09-0218:32jeaye> what do people do when you have different keys for different specs, but want to handle them generically? We'll define it somewhere common, like (s/def ::field blah?) and then alias it to match the expected keys where it's used like (s/def ::my-field ::common/field). > ... while fields have common structure across types of sections Depending on how common they are, you may define a (s/def ::base-field (s/keys :req [...])) and then do a (s/def ::field (s/merge ::common/base-field (s/keys :req [...]))) in the namespace tied to that specific field.#2018-09-0218:33bbloomyeah - so that much iā€™ve mastered. the tricky part is the collection of these things. for example:#2018-09-0218:34bbloom(s/def ::funcs-section (s/merge ::section (s/keys :req [::funcs]))#2018-09-0218:35bbloomgiven: (s/def ::section (s/keys :req [::fields]))#2018-09-0218:35bbloomnow some general code wants to visit all fields in all section#2018-09-0218:35bbloomhow do you define ::fields?#2018-09-0218:35bbloomassuming (s/def ::funcs (coll-of ::func) and (s/def ::func (s/merge ::field ...))#2018-09-0218:36bbloommy idea is to do: (s/def ::section (s/keys :req [::fields-key])) and then indirect through that key#2018-09-0218:36bbloombasically make the polymorphism explicit#2018-09-0218:37jeayeWhat about: (s/def ::funcs-section (s/merge (make-section-spec ::my-field-1 ::my-field-2) (s/keys :req [::funcs]))#2018-09-0218:37bbloomthe challenge isnā€™t making the spec - itā€™s using the data that has been specā€™d#2018-09-0218:38bbloomi want to be able to write (visit-section-fields some-section-of-any-kind)#2018-09-0218:38jeayeAh#2018-09-0218:38bbloomwhere visit effectively does something like: (update-in section [:fields] #(mapv f %))#2018-09-0218:39bbloommy ā€œleast bad ideaā€ from above is to do:#2018-09-0218:39jeayeYeah, that's beyond me. spec-tools allows for better introspection though. Might be of help.#2018-09-0218:39bbloom(update-in section [(:fields-key section)] #(mapv f %))#2018-09-0219:12misha@bbloom can you share a snippet of an ast? as json or whatever.#2018-09-0219:12bbloomthereā€™s a bunch of different designs iā€™m experimenting with, but it basically is something like:#2018-09-0219:14bbloom{:type-section {....}, :funcs-section {....}, :exports-section {....}} where each of those are something like {:env {$someid 0} :funcs [....#2018-09-0219:14mishawhile you are at it, how about multimethod for visit-section-fields?#2018-09-0219:14bbloom@malch the logic for visit-section-fields is trivially monomorphic if a section is defined as {:env ..., :fields [...]}#2018-09-0219:14mishaso you want same field name but different spec forms for it?#2018-09-0219:15bbloomhonestly, i donā€™t want different field names at all - iā€™m happy to have them all called :fields, but that makes the specs less strict#2018-09-0219:16mishawill you work with qualified keys in data itself? or unqualified?#2018-09-0219:17mishaif unqualified ā€“ you just need the same (name kw) for those different specs: :foo/bar :baz/bar#2018-09-0219:17bbloomi have no need for extensibility, so i was hoping to use unqualified keys to just avoid that entire syntactic headache#2018-09-0219:19misha:wasm.type/fields, :wasm.funcs/fields#2018-09-0219:20mishathere is also (s/keys :req [(or :foo/bar :baz/bar]), but I am not really sure what the behavior contract there#2018-09-0219:21bbloom> :wasm.type/fields, :wasm.funcs/fields i donā€™t understand what youā€™re trying to say with this#2018-09-0219:22mishaI think I did not understand your initial question and examples opieop#2018-09-0219:41bbloomok - new least-bad idea: donā€™t try to spec my whole AST - just spec pieces of it on an ad-hoc as needed basis ĀÆ\(惄)/ĀÆ take advantage of a-la-cart, i guess#2018-09-0219:41bbloomi need to write a validation pass anyway#2018-09-0219:42bbloommy experience with spec seems to be 1) it forces me to consider my designs & often improves them and 2) then doesnā€™t really add much actual value for checking at all#2018-09-0219:45mishahow do you intend to use spec for this?#2018-09-0219:46mishagen-testing? inbound data validation? destructuring?#2018-09-0220:09bbloomi was hoping to use it to validate the inputs/outputs of each compiler phase by just conforming the root node of the tree - but i need to hand-craft validation logic anyway, so iā€™ll use spec piece meal in there if it makes sense#2018-09-0416:40djtangoHas anyone come across issues with reader conditionals and spec in cljc?#2018-09-0416:41djtangoSay I want to include a spec from a clj-only library can I do this in cljc:
#?(:clj (s/def ::my-spec ::clj-only-lib/the-spec)
   :cljs (s/def ::my-spec any?))
#2018-09-0416:43djtangocurrently we are seeing this exception: Caused by: clojure.lang.ExceptionInfo: Invalid keyword: ::clj-time.s/local-date. {:type :reader-exception, :ex-kind :reader-error, :file "/x/spec.cljc", :line 16, :col 54}#2018-09-0416:47Alex Miller (Clojure team)Autoresolved keywords use the current env at read-time to resolve the ns#2018-09-0416:48Alex Miller (Clojure team)And this is reading, which happens regardless of the chosen branch#2018-09-0416:48Alex Miller (Clojure team)A workaround is to use the full keyword with namespace#2018-09-0416:49mishaharold#2018-09-0416:56djtangoAh so the reader for your cljs doesn't ignore the clj branches?#2018-09-0421:35Alex Miller (Clojure team)Itā€™s read before itā€™s ignored #2018-09-0416:57djtangoand so it gets to the auto-resolved keywords and if the clj-only ns isn't in the env at that point you get an exception#2018-09-0416:57djtangoHave I understood that right?#2018-09-0417:27seancorfieldYes, the expressions are read and auto-resolving keywords is part of that, prior to the conditional being applied. Then the result of reading is compiled.#2018-09-0417:28seancorfieldSo reader conditionals can only select between syntactically valid expressions that can be read in -- :some-ns/some-thing is read "as-is" but ::some-alias/some-thing has some-alias expanded as part of the reading process.#2018-09-0417:29seancorfield(I hope I explained that accurately!)#2018-09-0509:33djtango^ makes sense. Thanks for the explanations @seancorfield @alexmiller#2018-09-0512:45ikitommicross-posting here too: a sample app with clojure.spec & ring, with conforming-based coercion: https://github.com/metosin/reitit/blob/master/examples/ring-spec-swagger/src/example/server.clj#2018-09-0512:47ikitommiin the example, the ::x and ::y specs are coerced either from strings (the get-endpoint with query-parameters) or from json (the post-endpoint if client sends json), or just validated (the post-endpoint if client sends edn or transit).#2018-09-0513:17ChrisIs s/fspec the proper way to define a spec for a higher order function? When I declare something with s/fspec and then try to generate a value, it appears to be called many times (perhaps by conform?) is there a way to declare a higher order function without it being invoked with conform? Example: `(s/def ::post-download-fn (s/fspec :args (s/cat :size nat-int?) :ret any? :gen (fn [] (gen/return (fn post-dl-fn [size] (log/info (str "Post Download fn called! n: " size)))))))` (gen/generate (s/gen ::post-download-fn)) Gives a bunch of these before returning the single value generated:
Post Download fn called! n: 0
Post Download fn called! n: 1
Post Download fn called! n: 1
....
Am I missing something obvious?
#2018-09-0514:07ChrisI believe it to be instrumentation validating conformance - but it's odd that we can't turn it off in the case of a side effecting higher order function.#2018-09-0514:29taylor@chris547 there might be a better way, but there's a *fspec-iterations* dynamic binding you can use to sidestep that behavior:
(defn foo [f] (f 1))
(s/fdef foo :args (s/cat :f (s/fspec :args (s/tuple int?))))
(st/instrument `foo)
(binding [clojure.spec.alpha/*fspec-iterations* 0]
  (foo #(doto % prn inc)))
#2018-09-0514:30ChrisOh great, I wasn't aware of that binding - thanks @taylor!#2018-09-0514:50taylornp, I'm sure there's better advice/guidance on how to handle this for higher-order functions that take side-effecting functions, and I'm sure it's something that must be considered as clojure.core gets specs. One obvious workaround is to sacrifice specificity by just using e.g. ifn? as a predicate#2018-09-0519:36bastiHi, is it possible to nest specs, rather than creating for each tiny bit a new s/def ? So an inline version of e.g.
(s/def ::key1 string?)
(s/def ::my-map (s-keys [::key1]))
?
#2018-09-0519:37taylorfor s/keys specs, you need registered specs for the keywords. For other non-`s/keys` types of specs, you can nest spec definitions#2018-09-0519:38bastik thanks for the info - appreciated! how would that look like for that the spec above?#2018-09-0519:38taylor(s/def ::my-map (s/keys :req [::key1]))#2018-09-0519:39tayloror :req-un for unqualified keywords#2018-09-0519:41bastiOh I rather meant to define the spec for ::key1 in the ::my-map definition. I donā€™t want to reuse ::key1#2018-09-0519:42taylorI don't think that's possible#2018-09-0519:42taylornot with an s/keys spec, at least#2018-09-0519:36smothersDo you know if there is a way for a recursive spec to nest uniquely? So the same tag canā€™t repeat across a branch e.g. NO āŒ
[:paragraph
 [:paragraph "foo"]]
or āŒ
[:link
 [:link "foo"]]
or āŒ
[:italic
 [:bold
  [:italic "foo"]]]
but repeating siblings are fine āœ…
[:paragraph
 [:bold "foo"]
 [:bold" "bar"]]
Hereā€™s the spec Iā€™m currently using
(s/def ::attrs (s/map-of #{:url} string? :gen-max 2))
(s/def ::hiccup (s/or :string  string?
                      :element (s/cat :tag #{:paragraph :link :bold}
                                      :attrs (s/? ::attrs)
                                      :content (s/* ::hiccup))))
#2018-09-0519:38Alex Miller (Clojure team)Generally, I would say no#2018-09-0519:39Alex Miller (Clojure team)You can of course write any arbitrary predicate you want and include it#2018-09-0519:41smothersThanks. Iā€™ll give that a try#2018-09-0701:43bbrinckIn clojure.spec.alpha ā€œ0.2.176ā€, is there a new way to install a custom printer for macro-expansion errors?#2018-09-0701:44bbrinckIt seems to work differently than in ā€œ0.2.168ā€, unless Iā€™m missing something#2018-09-0701:44bbrinckhttps://gist.github.com/bhb/1948bfca870fbc8758b515a4d0acc5ff#2018-09-0701:47bbrinckThere may be some subtlety of https://dev.clojure.org/jira/browse/CLJ-2373 that Iā€™m not understanding#2018-09-0701:54seancorfieldIt's this patch https://dev.clojure.org/jira/secure/attachment/18391/clj-2373-spec-alpha-2.patch that removes the call to s/explain-out as part of simplifying the error messages.#2018-09-0701:55seancorfieldAnd specifically this part of the proposal: * stop printing data into message strings ExceptionInfo.toString should list keys, not entire map contents spec.alpha/macroexpand-check should stop including explain-out in message#2018-09-0701:59seancorfield@bbrinck Interesting. I can reproduce the behavior in a Boot repl but if I use clj I get different behavior:
(! 504)-> clj -Sdeps '{:deps {org.clojure/clojure {:mvn/version "1.10.0-alpha7"}}}'
src/user.clj was loaded
Clojure 1.10.0-alpha7
user=> (require '[clojure.spec.alpha :as s])
nil
user=> (set! s/*explain-out* (fn [ed] (println "failed")))
#object[user$eval5$fn__141 0x5b07730f "
#2018-09-0702:01bbrinck@seancorfield Good to know - I am still using Clojure 1.9 in my repro. Maybe itā€™s an issue with using new clojure.spec without new clojure alpha?#2018-09-0702:01seancorfieldCompared to alpha 6
(! 505)-> clj -Sdeps '{:deps {org.clojure/clojure {:mvn/version "1.10.0-alpha6"}}}'
src/user.clj was loaded
Clojure 1.10.0-alpha6
user=> (require '[clojure.spec.alpha :as s])
nil
user=> (set! s/*explain-out* (fn [ed] (println "failed")))
#object[user$eval5$fn__140 0x502f1f4c "
#2018-09-0702:02bbrinckThe announcement for spec.alpha 0.2.176 and core.specs.alpha 0.2.44 says ā€œthese libraries can be used now with Clojure 1.9.0ā€, so I expected that to work#2018-09-0702:03seancorfieldConfirmed, yes, the printing has changed:
(! 506)-> clj -Sdeps '{:deps {org.clojure/clojure {:mvn/version "1.9.0"} org.clojure/spec.alpha {:mvn/version "0.2.176"}}}'
src/user.clj was loaded
Clojure 1.9.0
user=> (require '[clojure.spec.alpha :as s])
nil
user=> (set! s/*explain-out* (fn [ed] (println "failed")))
#object[user$eval15$fn__150 0x590c73d3 "
#2018-09-0702:06bbrinck@seancorfield Thanks! Since it works in 1.10.0-alpha7, I may just add a note to Expound to suggest that people donā€™t upgrade to newest spec.alpha unless they also upgrade clojure. Or there may be a workaround#2018-09-0702:10Alex Miller (Clojure team)I am too tired to grok the issue - can you recap?#2018-09-0702:45seancorfield@alexmiller The change in 2373 that removes the explain-out call from several spec failures (and instead adds that call to the default REPL?) means that The details of the spec failure aren't run through a custom printer now in Clojure 1.9 or Boot REPL (and probably not in Leiningen's REPL -- I think both lein and boot use REPL-y?).#2018-09-0702:46seancorfield(it's the right choice but it means that tools that print explain-data in fancy ways now need to hook into the REPL instead of expecting to be invoking in the spec failures themselves)#2018-09-0703:12Alex Miller (Clojure team)Itā€™s not that they arenā€™t run through a custom printer though, itā€™s that theyā€™re not run through any printer, right?#2018-09-0703:13Alex Miller (Clojure team)Previously they were printed into the exception message and now theyā€™re not#2018-09-0703:15Alex Miller (Clojure team)The hooking is still exactly the same, its just that repls need to be better about how they print. The 1.10 clojure.main has a new function exposed to help with this#2018-09-0705:13seancorfield> Itā€™s not that they arenā€™t run through a custom printer though, itā€™s that theyā€™re not run through any printer, right? Yes, right. I could have worded that better šŸ™‚#2018-09-0705:15seancorfield> The hooking is still exactly the same, its just that repls need to be better about how they print. The 1.10 clojure.main has a new function exposed to help with this Boot and Leiningen have some work to do. Or REPL-y. šŸ™‚#2018-09-0714:08bbrinck@alexmiller I suspect Iā€™ll get questions about this for Expound, so Iā€™d like to put a notice for anyone who upgrades clojure.spec.alpha to ā€œ0.2.176ā€. What would you suggest? Should I recommend not upgrading to 0.2.176 unless they also upgrade to clojure 1.10.0-alpha7? Or until they upgrade to a newer version of Lein or Boot that changes the REPL printing?#2018-09-0714:10bbrinck> The 1.10 clojure.main has a new function exposed to help with this which function is this? Iā€™m toying around with a custom REPL built on top of rebel-readline#2018-09-0714:20Alex Miller (Clojure team)clojure.main/ex-str#2018-09-0714:21Alex Miller (Clojure team)thatā€™s the function that builds the string to print in the repl#2018-09-0714:22Alex Miller (Clojure team)for the note, I think the key is to just be clear about the combination of circumstances youā€™re warning about#2018-09-0714:27Alex Miller (Clojure team)as those circumstances will change with subsequent releases of clojure, spec.alpha, and the tools involved#2018-09-0716:18bbrinckthanks, iā€™ll add a warning to the readme#2018-09-0716:53JHHow would you spec this map?#2018-09-0717:13dadairWhat do you want the spec to express? Have you read the spec guide? https://clojure.org/guides/spec#2018-09-0717:21dadairā€œhow would you spec this mapā€ is very subjective; it depends on what you need. For example, a perfectly valid spec is (s/def ::map-thing map?), if that is a robust enough spec for your particular problem.#2018-09-0719:20JHThat makes sense, I just read through the guide and was able to figure it out. I wanted to spec that every key in the map was of certain type. Moreover that the :on-click was a function.#2018-09-0719:21JHThanks @dadair#2018-09-0802:32hoynkCan anyone point me to some GitHub code with a good use of spec?#2018-09-0802:32hoynkAlso, should I use a newer version of spec than the one that comes with 1.9.0?#2018-09-0802:34seancorfield@boccato If you're just getting started with spec, you might as well stick with 1.9 and the included version.#2018-09-0802:36hoynkThat is exactly my case, @seancorfield.#2018-09-0802:51seancorfieldThere are also some changes in the latest spec version that sort of assume some corresponding changes in 1.10 (and I think some of the REPL tools will need some changes to catch up as well).#2018-09-0901:04bastiI there a nested version of s/form ?#2018-09-0901:17bastito back up a little bit.. suppose I have a deeply nested map data structure, how can I inspect its structure without manually traversing all its composed (s/def)s ? I could generate an example, but that doesnā€™t tell me that a specific leaf is specd with a string? predicate e.g.#2018-09-0901:18bastiplus the example generation is also kinda limited because its predicated on the fact that every predicate I use has a generator#2018-09-0905:53bastiOther question, how do I spec a collection which has the sequence: [string? int? keyword?] .. Whereby int? keyword? can repeat arbitrary. So something like [sting? int? keyword? int? keyword? int? keyword?] would be valid?#2018-09-0905:56seancorfield@basti untested but I think (s/cat :s string? :rest (s/+ (s/cat int? keyword?)))#2018-09-0906:00seancorfieldYeah, (s/def ::x (s/cat :s string? :rest (s/+ (s/cat :i int? :k keyword?)))) should do it @basti#2018-09-0906:00bastithatā€™s awesome, thanks @seancorfield šŸ™‚#2018-09-0907:10mishawell, until skynet comes around, it's up to you to assess which spec is trivial to generate automatically, and which ones need your help, @basti#2018-09-0907:14mishawhat exactly do you mean by inspect? there is an s/conform, which destructures and assigns labels to most of the nodes. in which usecase it is not enough for you?#2018-09-0907:43bastibasically convert
(s/def ::name string?)
(s/def ::person (s/keys :req-un [::name]))
(s/def ::registration (s/keys :req-un [::person]))
to
(mapify ::registration)
=> {:person {:name 'cljs.core/string?}}
So that I see the whole structure at one glance. Meanwhile I wrote a small utility function that converts map specs to my desired structure - will write a blog post about that eventually šŸ™‚
#2018-09-0907:56ikitommi@basti at least spec-tools has a spec form parser & visitor, which can walk over all(?) specs. But I guess you already rolled your own.#2018-09-0907:57ikitommicoverage on the visitor: https://github.com/metosin/spec-tools/blob/master/test/cljc/spec_tools/visitor_all_test.cljc#2018-09-0908:03bastiawesome, yeah I hand-rolled my own visitor, but happy to switch to a library šŸ™‚ Iā€™ll have a look!#2018-09-1016:56snowellI have a function that takes 2 arguments, the second of which needs to conform to a spec:
(s/def ::MySpec (s/cat ::foo string? ::bar keyword? ::baz vector?))

(defn myFunc [one two])
(s/fdef myFunc :args (s/cat :one ::One :two ::MySpec))
#2018-09-1016:57snowellWhen running stest/check on this function, I get an ArityException because the (s/cat a (s/cat b c d)) is becoming 4 arguments#2018-09-1016:57snowellHow can I tell it that the second cat needs to be as one arg?#2018-09-1017:00favilasurround with s/spec#2018-09-1017:05snowellOK cool...that at least gets me a different failure#2018-09-1018:50borkdudeI listened to the REPL podcast today and the interviewee (Higginbotham) said there were benefits with putting specs in their own namespace. How exactly? For fdefs I find it useful to have them close to the function as documentation?#2018-09-1018:55seancorfield@borkdude It means your code can still be used without spec since the specs -- and those namespaces -- are optional.#2018-09-1018:56seancorfieldclojure.java.jdbc does that so it can still support back to Clojure 1.7.#2018-09-1018:56borkdudefor libraries thatā€™s nice yes#2018-09-1018:56seancorfieldAlso, most of our specs at work are data specs, not function specs -- and we find it easier to have a namespace for data specs. One place to go read them all (for a given data concern).#2018-09-1018:59borkdudeok#2018-09-1018:59misha+1 to data specs mostly#2018-09-1018:59borkdudedo you use instrumentations with these specs at all?#2018-09-1019:00borkdudee.g. in jdbc#2018-09-1019:02seancorfieldI instrument the java.jdbc lib for the tests within that lib -- which slows things down dramatically.#2018-09-1019:03seancorfieldAt work, we tend to instrument specific functions (that have fdef specs) only within their specific tests.#2018-09-1019:03seancorfieldWe use the data specs in production code (to validate/conform data).#2018-09-1019:03borkdudeI find it annoying to get the wrong keys. Whenever I encounter that I tend to put an fdef now and instrument it (only for dev)#2018-09-1019:03seancorfieldWe also use the data specs in tests -- for generative testing, or just for random example-based test data.#2018-09-1023:04Jon WalchIs there anyway to force a specific path with a call to s/gen? Hereā€™s a trivial example:
(s/def ::value (s/or :a map? :b int?)
(s/def ::message :req-un [::value])

(gen/generate (s/gen ::message))

I want the final line to always return with the ::value codepath of :a for the purpose of generating data for a test. Is this possible?
#2018-09-1023:05taylortake a look at the 2-arity version of s/gen https://clojuredocs.org/clojure.spec.alpha/gen, it takes an overrides map#2018-09-1023:06tayloryou might be able to override ::value there by providing a different generator#2018-09-1023:10Jon Walchperfect thanks#2018-09-1114:41slipsetSorry if this is a faq, but Iā€™d like to generate specs at runtime to use for validation eg:#2018-09-1114:42slipsetSay I receive something like:#2018-09-1114:42slipset
{:id 1, :model-id 4711, :foo 'bar}
#2018-09-1114:42slipsetIn order to know how to validate this thing, Iā€™d need to look up what the model is in a database, and then construct a spec based on that model.#2018-09-1114:44ghadieval#2018-09-1114:45lilactownunfortunately, clojure.spec is very unwieldy for this type of use case#2018-09-1114:45slipsetWhat Iā€™d be looking for is something like:#2018-09-1114:46slipset(s/valid? (model->spec (find-model (:model-id entity))) entity)#2018-09-1114:46ghadi@slipset (-> model-id (grab-from-database db) read-string eval)#2018-09-1114:46ghadithat'll give you a spec, if the form is saved in the db#2018-09-1114:46slipsetAnd with my understnading of spec, this is not possible, but Iā€™d like to flag it as a use case, just in case.#2018-09-1114:46ghadiand all of what it refers to is already loaded and available#2018-09-1114:47ghadiit's totally possible, you just have to dynamically load the spec#2018-09-1114:47lilactownI know that there seems to be some disinclination to adding more support this. itā€™s my most-desired feature#2018-09-1114:48ghadiit's not a feature you add to spec, you can already do it#2018-09-1114:48slipset@ghadi but my specs are not ā€œknownā€ or stored anywhere, and there might be a bunch of models (a bunch being in the thousands).#2018-09-1114:49slipsetWhile Iā€™m typing this, I guess I think that if specs were data, this would be feasible, right?#2018-09-1114:49lilactownthe ability to have ā€œanonymousā€ specs / specs-as-data would have a lot more utility as a general purpose way of describing data. but I think that it rubs against the desire of the core team to encourage people to use namespaced keys#2018-09-1114:50ghadiregardless of the mechanism, your specs have to be known somehow if you want to dynamically load them#2018-09-1114:51slipset@ghadi: in my db I would typically have someting like this (for a model)#2018-09-1114:52slipset
{:id 4711, :foo "integer", :bar "list-of-strings", :baz "string"}
#2018-09-1114:52slipsetand, while this might be fortunate, itā€™s how my world is.#2018-09-1114:53guyDo you have lots of models?#2018-09-1114:53slipsetyes.#2018-09-1114:53slipsetand theyā€™re user defined#2018-09-1114:54slipsetso my model->spec would return, for the model above, something like:#2018-09-1114:54slipset
(s/keys :req-un [::foo ::bar ::baz])
#2018-09-1114:54slipsetand there I see where the eval comes in šŸ™‚#2018-09-1114:54ghadiany validation system that stores stuff as data needs to convert list-of-strings into some predicate#2018-09-1114:54slipsetThanks, @ghadi šŸ™‚#2018-09-1114:55slipsetYeah, I see that, list-of-strings which is one of a few predefined types, would be mapped to (s/coll-of string?)#2018-09-1114:56slipsetThe main problem here, I guess is ::foo, ::bar, and ::baz needs to be defined.#2018-09-1114:57lilactownright. everything in clojure.spec must be named#2018-09-1114:58guyAre the types of your data always quite static?#2018-09-1114:58ghadinamed only if you refer to the spec by name#2018-09-1114:58ghadi(keys or def / fdef)#2018-09-1114:59ghadiyou can call (valid? (coll-of string?) ["a" "b"]) without naming the spec#2018-09-1115:00ghadiof course if you're checking a map, you'll want to handle that yourself#2018-09-1115:00ghadii.e. just because s/keys is available, you don't have to use it#2018-09-1115:01slipsetThe types are static#2018-09-1115:02slipsetbut the field names are not.#2018-09-1115:03lilactownšŸ¤” maybe using map-of instead of keys?#2018-09-1115:03lilactownthereā€™s also https://github.com/metosin/spec-tools which might be a more ergonomic solution#2018-09-1115:05ikitommiwith spec-tools, without macros, on top of the non-documented parts of clojure.spec:
(require '[spec-tools.data-spec :as ds])
(require '[clojure.spec.alpha :as s])

(let [model {:foo integer?
             :bar [string?]
             :baz string?}
      spec (ds/spec
             {:spec model
              :name ::spec})]
  (s/valid? spec {:foo 1
                  :bar ["kikka" "kukka"]
                  :baz "abba"}))
; true
#2018-09-1116:12borkdude@ikitommi thatā€™s cool. kind of like Schema notation for clojure.spec?#2018-09-1116:17borkdudefair reminder, go on šŸ™‚#2018-09-1116:17ikitommi@borkdude yes, it is. Quite nice for ad-hoc stuff like web apps, where one would use Schema#2018-09-1116:18ikitommihttps://github.com/metosin/reitit/blob/master/examples/ring-swagger/src/example/server.clj#2018-09-1116:20ikitommiIā€™m actually looking forward to seeing things break.#2018-09-1116:20ikitommiany plans on doing that @alexmiller?#2018-09-1116:21borkdudeeven the public parts of spec itself may break if itā€™s for the better, itā€™s alpha šŸ˜‰#2018-09-1116:33Alex Miller (Clojure team)Rich has been getting back to spec and right now I think itā€™s likely that weā€™ll try to get 1.10 out the door, then start working on spec in earnest#2018-09-1116:33borkdudecool#2018-09-1116:38borkdudewhat are the main features of 1.10? - I saw you were working on better error messages, very nice work#2018-09-1116:43Alex Miller (Clojure team)updated JVM/JDK requirements and compatibility, prepl (kind of alpha), error messages#2018-09-1116:45seancorfieldDoes that mean we'll get some new tooling from Cognitect based around the prepl?#2018-09-1116:46Alex Miller (Clojure team)unlikely for 1.10#2018-09-1116:47borkdudeprepl.alphaā€¦ what are we going to do with the prepl?#2018-09-1116:47Alex Miller (Clojure team)whatever you like :0#2018-09-1116:47Alex Miller (Clojure team)thereā€™s a bunch of things in work. rather than wait for them all to reach completion, I think weā€™ll just release 1.10 and keep working on them#2018-09-1116:48borkdudeanything to read on prepl?#2018-09-1116:48seancorfieldThe source code? šŸ™‚#2018-09-1116:48seancorfieldI had a quick play with it when it dropped... I can imagine some interesting tooling options based on it...#2018-09-1116:48Alex Miller (Clojure team)thereā€™s a small chance it might even be removed before 1.10 releases#2018-09-1116:48borkdudeI have this vague notion itā€™s something similar to unrepl, but thatā€™s all I know#2018-09-1116:49Alex Miller (Clojure team)it shares many goals with unrepl#2018-09-1116:49Alex Miller (Clojure team)both are data-oriented stream repls#2018-09-1116:50seancorfieldWe're thinking a bit about building some internal tooling on top of prepl. Automating some of the stuff we currently do today (via a REPL manually).#2018-09-1116:51seancorfield(we run a Socket Server REPL inside one of our production processes and currently use unrepl to talk to it over an ssh tunnel for certain things)#2018-09-1116:51borkdudewe run nREPL in production, but I only talk to it myself šŸ˜‰#2018-09-1116:52seancorfieldWe used to -- but I didn't like the stack of dependencies that nREPL brought in. Running just a Socket REPL is nicer. And of course we can start any process up with a Socket REPL without needing any code inside the process.#2018-09-1116:53borkdudeI did that once for a Scala project where I added clojure as a dependency šŸ˜‰#2018-09-1116:54borkdudeit wasnā€™t a serious project though, but it worked šŸ˜‰#2018-09-1116:56seancorfieldWe have a REPL running inside our legacy CFML app so I won't judge šŸ™‚#2018-09-1117:08slipset@ikitommi nice! Iā€™ll have a look at the data specs.#2018-09-1119:01arohners/keys supports and in :req, but not :opt, right? Is there any other way to express ā€œ:foo is optional, but if present, :bar must also be presentā€?#2018-09-1119:04mishas/multispec, s/or#2018-09-1119:06mishabtw, the exact behavior of or and and inside s/keys is still a mystery to me. is there something official to read about it, @alexmiller? or is it all unintended side effect, which should not be relied upon?#2018-09-1119:15seancorfield@arohner That's usually when I reach for s/and wrapped around s/keys and a predicate for the present/not-present relationships...#2018-09-1119:24arohner@seancorfield thanks. The combinations here get a little gnarly to multi-spec, so I guess Iā€™ll punt#2018-09-1119:25arohner@misha itā€™s documented in s/keys docstring:
The :req key vector supports 'and' and 'or' for key groups:
#2018-09-1215:08samedhiIs it possible to write a spec that generates a sequence where each element in the sequence has some relationship to the previous element(s) in the sequence?#2018-09-1215:09samedhiAs an example, like one that multiplies the last element of a sequence by 2 or 3 (random).#2018-09-1215:09samedhiso [1 2 4 8 24 48 144] and [1 3 9 18] would both be sequences it might generate.#2018-09-1215:10taylorare you only interested in generation, or do you want a spec to validate inputs too?#2018-09-1215:10samedhiI guess both#2018-09-1215:10samedhiI mean I would like to use test.check with this generated sequence, so I assume that means using both (I think).#2018-09-1215:13Alex Miller (Clojure team)the way I would handle this is to use gen/bind#2018-09-1215:13Alex Miller (Clojure team)generate a series of random multipliers (a sequence of random 2s and 3s)#2018-09-1215:14samedhiAh, I see where you are going.#2018-09-1215:14Alex Miller (Clojure team)then gen/bind to turn the sequences of multipliers into a sequence of values#2018-09-1215:14Alex Miller (Clojure team)or rather, I guess gen/fmap is sufficient here#2018-09-1215:14Alex Miller (Clojure team)itā€™s just apply a function to that sequence#2018-09-1215:16samedhiYeah, so what I was really hoping was to generate the initial (seed) state of my program as the first element in the sequence, and generate the rest of the sequence by constantly looking at the last element in the sequence, determining what actions can be taken at that state, and then picking one of those actions and applying it to said state.#2018-09-1215:16Alex Miller (Clojure team)yeah, same idea#2018-09-1215:17Alex Miller (Clojure team)start from a gen of a tuple feeding the fmap#2018-09-1215:17Alex Miller (Clojure team)(s/gen (s/tuple int? (s/coll-of #{2 3} :max-count 10)))#2018-09-1215:18Alex Miller (Clojure team)then gen/fmap a function over that that takes the first element in the tuple and repeatedly applies the multipliers in the second element to produce a sequence#2018-09-1215:18samedhiOk, the only wrinkle I see there is that only a subset of all of the actions are valid for some of the states.#2018-09-1215:19samedhiBut that is an idea.#2018-09-1215:19samedhiMaybe, I can just make it so that if the action taken is NOT a valid action for that state, it just returns the original state (identity).#2018-09-1215:19Alex Miller (Clojure team)if you need to randomly choose an action per state, you will need to move to gen/bind#2018-09-1215:20Alex Miller (Clojure team)but I think you could make that work too#2018-09-1215:20samedhiInteresting, interesting. I mean I tried to read through the "rose tree" thing for test.check generators. And... well, it looks like it is generating some sort of tree using something like a recursive algorithm. But, yeah, got lost there. šŸ™‚#2018-09-1215:20Alex Miller (Clojure team)in that case, Iā€™d probably gen the initial state and the number of actions to gen#2018-09-1215:21Alex Miller (Clojure team)yes, but you donā€™t need to worry about that - the key is just to capture all randomness via generators#2018-09-1215:21Alex Miller (Clojure team)test.check will do the rest#2018-09-1215:21samedhiOk, thank you for your help, I'll look at it some more this evening.#2018-09-1215:22Alex Miller (Clojure team)this is advanced generating, but itā€™s where things get interesting#2018-09-1215:22Alex Miller (Clojure team)a lot of interesting problems in simulation testing look like this#2018-09-1215:22samedhiReally, it is just the fantasy I have of letting spec generate every possible way that my program could run and being like "yep, that one thing encompasses all the test".#2018-09-1215:23samedhiYou still get a great deal of benefit out of using the normal fdef and test.check per function imo, but the idea of actually generating all states of the program would be really cool.#2018-09-1215:28gfredericks@samedhi @alexmiller this is exactly what the stateful-check library tries to automate, I believe. I haven't used it#2018-09-1215:33samedhi@gfredericks Awesome šŸ‘#2018-09-1215:34taylorwould there be a way to spec this kind of sequence without a custom predicate?#2018-09-1215:34roklenarcicIs there some way to specialize a key in s/keys? Like I have a (s/keys :req [.... ::type]) where type is #{"ORG", "USER"}, but now I'd like that same spec but where type is limited to `#{"ORG"}#2018-09-1215:35Alex Miller (Clojure team)@taylor do you mean custom generator?#2018-09-1215:36taylorno I mean if you wanted to conform/validate a sequence like that, asserting that each element is a multiple of the previous#2018-09-1215:36taylorI was trying to imagine how you might use the spec primitives to do that but I can't imagine how#2018-09-1215:37Alex Miller (Clojure team)not as a pred in the coll spec, but you could s/and with a function that verified that#2018-09-1215:37Alex Miller (Clojure team)or, maybe you could put that in :kind if you had a seq-of-mults? pred#2018-09-1215:35Alex Miller (Clojure team)@roklenarcic no, but you can s/and it with a more restrictive predicate#2018-09-1215:36roklenarcicIt tried to s/merge it with another keys spec which has key that is more restrictive, with different keyword ns and same name#2018-09-1215:37roklenarcicit kinda works, in that it will check both#2018-09-1215:38Alex Miller (Clojure team)there are pros/cons to those approaches wrt conform and gen#2018-09-1215:38roklenarcicah, haven't thought about that#2018-09-1215:39Alex Miller (Clojure team)I think s/and is the most direct way to say it: ā€œI have this map AND it must meet this extra constraintā€#2018-09-1215:39Alex Miller (Clojure team)s/multi-spec can also be handy for some cases like this but itā€™s a lot of weight to add if itā€™s only this#2018-09-1215:45roklenarcicI'll need to check out multi-spec#2018-09-1215:53roklenarcicUsing spec-tools's data-spec solves a couple problems, but sadly data-spec has a bug where they don't name their specs#2018-09-1216:02roklenarcichttps://github.com/metosin/spec-tools/pull/125#2018-09-1219:48misha@arohner well, that's embarrassing opieop (re: or in s/keys)#2018-09-1219:58taylorwhat's opieop all about#2018-09-1220:01mishaface expression#2018-09-1220:02taylorit's so small I can't make out his expression#2018-09-1220:03taylorI only see it used on this Slack and I feel like I'm missing... something#2018-09-1220:04mishait is from twitch .tv chats originally#2018-09-1222:23lilactownclojure.spec has the ability to transform data from one form into another, right?#2018-09-1222:24lilactownare there any ground-up guides around data transformation?#2018-09-1222:25seancorfieldThe basic rule of thumb is "don't do that"#2018-09-1222:26seancorfieldSpec isn't designed as a data transformation system and when folks ask about doing it, @alexmiller usually pops up and says that's not a good idea šŸ™‚#2018-09-1222:28lilactownbummer#2018-09-1222:29seancorfieldThat said, I think there are some use cases where limited coercion is acceptable. For example, we use spec for API / form input validation so data is mostly strings but we want strings, longs, doubles, Booleans, and sometimes dates out of it. I think that sort of spec is "OK" if you're careful (in particular, think about generation of data using such specs, as well as applicability of the spec elsewhere in your system -- just because you accept strings at the API/form level doesn't mean you want to accept strings elsewhere in your domain model).#2018-09-1222:29seancorfieldWhat sort of data transformation were you thinking about @lilactown?#2018-09-1222:31lilactownI'm creating an API for creating content schemas in a CMS#2018-09-1222:32lilactowni have a general purpose edn format I'd like to use, and then coerce it into the JSON representation for the particular CMS we're using#2018-09-1222:32Alex Miller (Clojure team)to be specific, transforming data use conformers in spec is imo not a good idea. using spec as a target for driving transformations is potentially a good idea. I think https://github.com/wilkerlucio/spec-coerce is pretty good.#2018-09-1222:50lilactownthanks! I'll take a look#2018-09-1302:46donaldballIā€™m working on a spec for an asynchronous message format and am curious about versioning. My instinct is to declare a versioned namespace for the message envelope - e.g. :my.message.v1/schema, declare all my envelope fields in that same namespace, and then just make a new one if and when I realize the need to make an incompatible change to the envelope. It occurred to my colleague that multi-spec was another way to accomplish this by including the version as a value, e.g. :my.message.schema/version 1 and using multi-spec to choose the correct spec. Are both of these viable options and is there anything to recommend one over the other?#2018-09-1303:06Alex Miller (Clojure team)Iā€™d recommend not making incompatible changes#2018-09-1303:06Alex Miller (Clojure team)then you donā€™t need versions :)#2018-09-1303:07Alex Miller (Clojure team)if the attribute needs to change, give the attribute a new name#2018-09-1303:08Alex Miller (Clojure team)if youā€™re talking about adding attributes, thatā€™s compatible. if you need to remove attributes from a map, then you should have a new name for the container, which I guess version is one kind of new name#2018-09-1314:29donaldballAlas, my foresight is sufficiently imperfect that I donā€™t believe I can develop the correct message format for all time. Incompatible changes I believe might be reasonably desired include (and are probably limited to?): adding and removed required keys.#2018-09-1314:33Alex Miller (Clojure team)well, Iā€™ll refer you to Richā€™s Spec-ulation keynote for the long form argument#2018-09-1314:35ghadiwhat is an example of adding keys that is a breaking change @donaldball?#2018-09-1314:35tayloradding new required keys would be breaking#2018-09-1314:36ghadiimportant word simple_smile#2018-09-1314:37taylorbut yeah I think the gist is "don't make breaking changes" which I generally take to mean "if you need to break something, make a new something, and leave the old something alone"#2018-09-1314:38taylorif you need to add a new required key to some map, it's no longer the same thing. Make a new map spec with a different name that requires the new key#2018-09-1314:40Alex Miller (Clojure team)^^#2018-09-1314:41ghadiif it's for a message on something like Kafka, the new name might be an entirely new Topic#2018-09-1314:44donaldballI agree thatā€™s very much the route spec wants you to go down and Iā€™m on board the train, but I am curious what makes the multi-spec idea a poor solution. It seems like it does constrain you forever to having a message be a map, and for the map to require the multi-spec keyword, but otherwise the map contents could vary as required by the versions.#2018-09-1315:17lilactownhow do I put a bound on how many times clojure.spec.test.alpha/check generates a test?#2018-09-1315:17lilactownthe documentation is very nebulous#2018-09-1315:22taylor@lilactown
(st/check `foo {:clojure.spec.test.check/opts {:num-tests 10}})
#2018-09-1315:24taylor>The opts map includes the following optional keys, where stc aliases clojure.spec.test.check >The ::stc/opts include :num-tests in addition to the keys documented by test.check.#2018-09-1315:24taylorit confused me at first too b/c test.check/quick-check takes the number of tests as its first argument#2018-09-1315:33lilactownyeah it was completely meaningless to me#2018-09-1315:34lilactownhm. even specifying {:num-tests 5}, my generative testing is taking minutes#2018-09-1315:42lilactownin fact, specifying 0 takes minutes? I'm not even sure this is working#2018-09-1315:48taylorthis works for me:
(defn foo [x] (str x))
(s/fdef foo :args (s/cat :x any?) :ret string?)
(st/check `foo {:clojure.spec.test.check/opts {:num-tests 1}})
(st/check `foo {:clojure.spec.test.check/opts {:num-tests 100}})
#2018-09-1315:49taylorand the :num-tests is accurately reflected in the check result#2018-09-1315:51lilactownI think some of my specs must be really expensive. is there a way for me to selectively narrow some of the child specs?#2018-09-1315:51lilactownI think some of my specs must be really expensive. is there a way for me to selectively narrow some of the child specs?#2018-09-1315:54taylorany recursive specs involved?#2018-09-1315:54lilactownnope#2018-09-1315:55taylorhmm hard to say w/o seeing all the relevant code#2018-09-1315:58Alex Miller (Clojure team)Expensive to check or to gen?#2018-09-1315:59lilactownI'm not sure šŸ˜ž#2018-09-1316:00lilactownI apologize, it's quite long. but I'm not confident what might be what's causing the slowness#2018-09-1316:11lilactowndefinitely seems to be in the generation of :torch/fields#2018-09-1316:13lilactownI'm going through and removing everything and slowly adding back in. adding back in
:torch.field/key
:torch.field/name
seems to slow it down, but not astronomically
#2018-09-1316:38Alex Miller (Clojure team)I meant what operation are you invoking when itā€™s slow?#2018-09-1316:39Alex Miller (Clojure team)Sounds like gen #2018-09-1316:40Alex Miller (Clojure team)If you have coll-of, itā€™s good to set :gen-max to something small like 3#2018-09-1316:51lilactownthat did help! thanks!#2018-09-1317:06Alex Miller (Clojure team)The default is 20 and that can get a little out of hand#2018-09-1316:27lilactownhow do I selectively narrow the values generated by stest/check?#2018-09-1316:28lilactowne.g. if my spec has a named child-spec somewhere that is defined as keyword?, but I don't really care about testing 1000s of keywords#2018-09-1316:29taylorI think you can pass in an override map in check opts :gen#2018-09-1316:29taylor>:gen map from spec names to generator overrides#2018-09-1316:30lilactownthanks!#2018-09-1415:28lilactownhow would I represent a constraint on two related fields? e.g.:
{:type :type/json
 :cardinality :cardinality/many} ;; invalid! :cardinality/many can only be used with :type/string and :type/ref
#2018-09-1415:30dominicm@lilactown use a predicate on the map?
#2018-09-1415:35lilactownright.... keep thinking of specs at the key level. thanks#2018-09-1417:01Spencer AppleHello, is it possible to include external data when using spec to validate data - or would a separate function be better? E.G. validating an api response, and using a key from the request (such as the length of the pagination to validate the number of values returned).#2018-09-1417:08dadairCould you define a spec that is the combination of the request + the response (i.e., the context), and then use a predicate to check the mentioned constraints?#2018-09-1417:10dadairsomething like the following?
(s/def ::request ,,)
(s/def ::response ,,)
(s/def ::context
  (s/and (s/keys :req [::request ::response])
         #(= (-> % ::request :pagination :length) (count (-> % ::response :values)))))
#2018-09-1417:13Alex Miller (Clojure team)I think the question is whether thatā€™s better than just writing the equivalent Clojure code#2018-09-1417:13Alex Miller (Clojure team)(I tend to think not in this case)#2018-09-1417:19Spencer Applehmm @dadair that is a really good example, and I think it would work. Thanks for writing that out. I was curious about if it made sense to include in the spec - and I kind of think the length of a vector is part of the data shapeā€¦ but perhaps it still doesnā€™t fit in the spec#2018-09-1417:20Spencer AppleI will eventually get to the data generation side of this spec, in which I will want to include the :pagination :length parameter from the request to generate the correct length of ::response ::values. At that point I might need to combine the ::request and ::response specs?#2018-09-1418:02Alex Miller (Clojure team)yes, if you want to generate one thing, they need to be in the same spec, although Iā€™m not sure why you need to generate responses (presumably your code is making those)#2018-09-1418:12Spencer AppleGotcha, and thanks for you responses! The request and response are to and from an external API. I want to generate responses to test our data munging flows. Since the data munging fns rely on the request parameters, the generated responses also have to conform (at least partially) to the requests that ā€œgenerated themā€.#2018-09-1419:46Alex Miller (Clojure team)gotcha, that makes sense#2018-09-1419:46Alex Miller (Clojure team)depending how much stuff there is, you might able to gen both independently, then just copy values that have to match from the request into the response or something like that#2018-09-1420:02Spencer Appleah interesting idea!#2018-09-1518:36richiardiandreawe have found that :: is a bit clunky for refactoring, is other people using the long :ns.foo.bar/quuz for specs?#2018-09-1518:39mpenetNot really. You change the name just in the ns declaration vs everywhere you use it. I guess it s quite personal #2018-09-1518:39richiardiandreawell you have to change the namespace and the alias in case it changes#2018-09-1518:40mpenetYes that's what I meant#2018-09-1518:40richiardiandreawhile with unqualified you don't have to do anything else, just copy them around#2018-09-1518:40richiardiandreaI mean fully qualified šŸ˜„#2018-09-1518:41richiardiandreafollowing along these lines, sometimes you need to make them unique...you cannot use :foo/bar or they would clash#2018-09-1518:42mpenetThe alias doesn't change really, just the thing it points to. #2018-09-1518:43richiardiandreathis though means you need to give names#2018-09-1518:43richiardiandreaand naming is hard and requirements change and all that#2018-09-1518:44richiardiandreaso we found fully qualified better for us - for now... šŸ˜„#2018-09-1518:44mpenetSame argument that's sometimes mentioned with clj.spec.alpha, just use aliases ex s/def and the day it changes to beta or whatever you just update the alias target#2018-09-1518:45mpenetIn the end it s just a search & replace with less/more hits, I guess both ways are fine#2018-09-1518:45richiardiandreawell my argument would be that you don't even need search and replace#2018-09-1518:45richiardiandreaat the price of spec verbosity#2018-09-1518:46mpenetYou d need to update every kw vs just ns declarations so yeah, there is some s&r :p#2018-09-1518:48richiardiandreamy question was something else though.. šŸ˜„#2018-09-1518:49richiardiandreakind of a feature request, but at this point I don't know if it makes sense šŸ˜„#2018-09-1518:49richiardiandreait would be great if you could have aliases of keywords that are not tight to a namespace#2018-09-1518:50Alex Miller (Clojure team)we do expect to have something like this, but not sure yet exactly what it will look like#2018-09-1518:50Alex Miller (Clojure team)didnā€™t want to couple it tightly to existing namespace/alias support#2018-09-1518:50richiardiandrealike (defkw :my-long.foo.bar :mfb)#2018-09-1518:50richiardiandreaoh cool ok#2018-09-1518:51richiardiandreais there a JIRA to upvote? I can open one#2018-09-1518:52richiardiandreaI'll check šŸ˜„#2018-09-1518:52mpenetWould be interesting if spec understood hierarchies, but I already mentioned that #2018-09-1518:53Alex Miller (Clojure team)there is one I think#2018-09-1518:54Alex Miller (Clojure team)https://dev.clojure.org/jira/browse/CLJ-2123#2018-09-1518:54richiardiandreaoh nice#2018-09-1518:56richiardiandreawill track it and write there if I have good ideas šŸ˜„#2018-09-1518:56richiardiandreathank you Alex#2018-09-1520:25lilactownyeah I ended up writing the fully qualified keyword for my specs lately and really am glad I did#2018-09-1520:25lilactownit made it really easy to e.g. move all of my specs to their own ns#2018-09-1523:27dadairDoes the thing change if it moves namespaces? Is it fundamentally different? If not, I think aliasing conflates location with name with regards to specs. I tend to prefer something like :transport/event because what it is doesnā€™t change based on where it is.#2018-09-1606:51ikitommiFrom reddit: > There are a couple areas (spec programmability and some map-related stuff) where we have some work to do.#2018-09-1606:56ikitommisounds good, despite doesn't say much about what kind of changes. As there are lot of libraries trying to extend spec for different use-cases, it would be great if the need of the libs could be taken into account. Personally would like to see some public spec walking protocols.#2018-09-1607:00ikitommiand not based on a parsing spec forms, that we can do outside, more like CLJ-2251.#2018-09-1607:02ikitommia library developer feature poll/discussion?#2018-09-1607:03misha@richiardiandrea you can create ns and give it an alias, it will not be tied to an actual file, but you will be able to use that alias to shorten keywords. it is not pretty though. clojure.core/create-ns, clojure.core/alias. I found, that having shorter spec names is less tedious and more readable (possible in app layer, where your system is pretty much the only system which sees them). In libraries you probably would want to use longer "truly" global long keywords for under the hood things.#2018-09-1615:50richiardiandreaThis is an interesting idea, but feels like a hack :) I will give it a try anyways, thank you!#2018-09-1616:01mishathis is sort of what ns does, but it looks alien and hacky, I agree with that.#2018-09-1818:30richiardiandreaunfortunately alias does not seem to work in cljs#2018-09-1818:36mishai'd cross-post this to #clojurescript, seems critical(?)#2018-09-1818:37richiardiandreadone that#2018-09-1608:49Ho0manHi, I am using spec for the first time. And it takes too long to generate my specs using (test.check/generate (spec/gen ,,, )). I had previously used test.check for similar data generation and did not had the problem. The spec if the configuration map having many nested spec/keys and spec/every. Limitin spec/every with :gen-max did not help. What am I doing wrong ?#2018-09-1608:50Ho0manHeres part of the spec ... the final configuration contains a numbeer of :ecosystem/service-instances#2018-09-1612:38Alex Miller (Clojure team)If you have a lot of s/every, they will gen up to 20 elements. Setting :gen-max in those to something like 3 would probably help#2018-09-1707:38Ho0manThanks, Alex. I did try that and it helped. My question is whether there is a fundamental difference in spec's data generation and that of test.check's ? And should I be concerned about using these specs in large generative tests as their size grows ? Thanks again#2018-09-1712:26Alex Miller (Clojure team)spec uses test.check generators so there is literally no difference. There will always be the need to tune the gens a little bit#2018-09-1618:00dadairWhatā€™s the preferred way of specing a spec? I have an internal lib function that takes a spec and I want the s/fdef to check that the arg is a spec. So far I have qualified-keyword? But thatā€™s a bit of a shallow spec? Tried spec? But it was returning false on s/defā€™d qualified keys.#2018-09-1618:02dadairThat spec arg is later used in internal machinery to validate some request routing#2018-09-1618:08seancorfield@dadair Remember that a predicate function could also be (treated as) a spec...#2018-09-1618:08dadairYeah, at this point I force using a ā€œdefinedā€ spec#2018-09-1618:09seancorfieldMaybe call s/form on it and succeed if you get truthy?#2018-09-1618:12dadairMaybe expect users to wrap in (s/spec ..) prior to passing argument, then internally spec against s/spec??#2018-09-1618:13dadairthat would also get around the limitation of removing the ability to use a raw predicate#2018-09-1618:14dadairAre there any downsides to that approach? Iā€™m not super familiar with the effects of (s/spec ..)#2018-09-1618:18seancorfieldThere's a get-spec function you can use with the qualified keyword (rather than s/form). You can wrap anything in s/spec: (s/spec 42)#2018-09-1618:19seancorfieldSo (s/spec? (s/spec 42)) is truthy -- so it won't tell you it's really a spec that is wrapped.#2018-09-1618:19dadairYeah maybe I need to do some code-based validation rather than relying on the fdef here#2018-09-1618:20dadairItā€™s not a critical check, Iā€™m just trying to add some guardrails for the other devs that arenā€™t super familiar with this part of the system#2018-09-1618:20seancorfieldI guess I'd take a step back and ask: Am I over-constraining this function?#2018-09-1618:21seancorfieldCan it accept a predicate and use it like a spec, for example?#2018-09-1618:22dadairYeah, thatā€™s what Iā€™m thinking too.#2018-09-1618:49Alex Miller (Clojure team)specs can be qualified-keyword?, set?, ifn?, or spec instances (thereā€™s a s/spec? for those)#2018-09-1620:28samedhiShould I be using (spec/fdef ...) for multimethods (does not seem to work)? Guess I can just wrap the defmulti invocation in a function and test that...#2018-09-1620:30samedhiErroneously thought that multi-spec was for multimethods, but it actually appears to be for dispatching data based on "kinds".#2018-09-1620:33samedhinm, silly ns error. spec/fdef does appear to work with defmulti#2018-09-1719:49spiedenis anyone aware of a way to strictly validate some data against a spec? e.g. fail on extraneous map keys, etc.#2018-09-1719:50taylorhere's one way https://github.com/gfredericks/schpec/blob/master/src/com/gfredericks/schpec.clj#L13#2018-09-1719:50taylorand I think spec-tools has another way, although the general advice is to not make map/keys specs "closed"#2018-09-1721:51bbrinckIf you just want to guard against typos, check out https://github.com/bhauman/spell-spec#2018-09-1721:51bbrinckbut yes, the general advice is not to make specs closed#2018-09-1722:27spiedenthanks!#2018-09-1722:27spiedenyes generally i leave them open#2018-09-1722:28spiedenin this case itā€™s closed, though, and the user can make all kinds of interesting typlos#2018-09-1722:29bbrinckYep, I think spell-spec is a good fit then#2018-09-1806:41misha@U0HM5MJ5V you also could do (s/and (s/keys ...) (s/map-of #{<allowed-keys-only>} any?)) to see whether this behavior is actually what you need. saves you custom macro or dependency.#2018-09-1816:46spieden@U051HUZLD great, thanks! didnā€™t realize map-of was strict#2018-09-1817:00mishawell, #{} is strict, but yeah#2018-09-2019:22spiedenah of course#2018-09-1806:22esp1Is there a way to have predicates return more detailed error messages? I have some complex predicates that operate on subtrees to do things like make sure all branches of the subtree are of the same type. Instead of just having the predicate return false at the root, I'd like to be able to have the predicate provide detailed path information about where within the subtree it detected an error. It would be great if I could have the predicate behave like the built-in spec functions so it could push this detailed information into the explain-data format. Is this possible to do in any supported way without reverse engineering spec internals?#2018-09-1806:45mishadecomposing your predicates in simpler ones, and recomposing them with spec's s/and, s/or and s/*?+ comes to mind. former will bring you closer to "can understand the exact issue from a predicate form w/o error messages", latter will give you exact paths in s/explain-data. @esp1#2018-09-1806:46mishathere is also https://github.com/bhb/expound#error-messages-for-predicates#2018-09-1807:46esp1Thanks for the reply @misha! I don't think decomposing the predicate will really work for me though; let me give some more detail about why. The structure I'm spec'ing is essentially a lisp-like expression syntax. For instance I want to validate that in the expression [ := :Foo/bar {:a 1, :b {:c "blah"} } ] that the arguments to the := comparison are both of the same type. The predicate to do this looks up the type schema I have associated with the :Foo/bar argument (which looks something like {:a :int, :b {:c :int} } and checks if the map argument matches this structure. In this example the type schema says that the value of :c should be an integer, but the value is actually a string, so validation should return an error. I'd like the error message to report the expected and actual types, along with the fact that the type mismatch error occurs in the second argument value at the path [:a :c]. If I just have a boolean predicate I'd just get a spec error indicating that a type mismatch error occurred at the root level. The ability to associate error messages with expound is nice - I hadn't realized you could do that. I suppose I could use a custom printer in expound that essentially walks the argument subtree again once a spec error occurs and prints out the error detail. This feels kinda hacky tho. I was hoping I could make the predicate such that it could provide detailed error info directly into the structure returned by s/explain-data.#2018-09-1812:47Timo Freibergcan i use a s/or spec but have conform only return the value instead of [:key val]?#2018-09-1813:10Timo Freibergnot sure how ugly this is, but i "solved" it by wrapping the s/or in an s/and and putting a (s/conformer second) at the end#2018-09-1813:11Timo Freibergso
(s/and
 (s/or :a #{"a"} :b ::b-spec)
 (s/conformer second))
#2018-09-1813:12taylor
(s/conform (s/nonconforming (s/or :i int? :s string?)) "foo")
=> "foo"
#2018-09-1813:13taylor@UA11M92KE s/nonconforming is what you want, I think#2018-09-1813:20Timo Freibergah, awesome! that makes it easier#2018-09-1813:29misha@esp1 well, sounds like you already have destructured spec for :Foo/bar, which should give you detailed paths to errors. You just need to figure out how to validate against it w/o enclosing it in an opaque predicate#2018-09-1813:49misha#2018-09-1813:54mishanumber of layers and complexity of this would depend on the diversity of the expressions you want to support.#2018-09-1813:56mishaway simpler version might be just
(defn validate-expr [expr]
  (s/assert (s/tuple keyword? any? any?) expr)
  (let [[op spec y] expr]
    (s/assert spec y)))
#2018-09-1813:57mishaI assume, this is not the case, hence extra layer of case or multimethod#2018-09-1818:56jrychterLooking for advice ā€” I've been looking to eliminate some verbosity in my specs and came up with this:
(defn sorted-set-of [spec]
  (s/and (s/coll-of spec)
         (s/conformer (partial apply sorted-set))))
Will this bite me in the future? (using a defn to define specs) Will it play well with generators once I get to those?
#2018-09-1819:00Alex Miller (Clojure team)You donā€™t need that - look into :into in coll-of#2018-09-1819:03Alex Miller (Clojure team)(s/coll-of int? :into (sorted-set)) does what you want#2018-09-1819:03Alex Miller (Clojure team)With whatever for int?#2018-09-1819:06jrychterOh, thank you, I didn't know about :into! Seems like it's exactly what I need. Trying it out.#2018-09-1819:07Alex Miller (Clojure team)Will work with gen automatically too#2018-09-1901:25richiardiandreawhat is a good intermediate value for quick-check's :max-size?#2018-09-1901:26richiardiandreaThe default kind of takes forever for me for a couple of generative tests#2018-09-1901:26richiardiandreaI am afraid 5, which is fast, is too small#2018-09-1901:26taylorFWIW I think the default is 200#2018-09-1901:28richiardiandreayep it is 200#2018-09-1901:41richiardiandreawow, I had to reduce :max-size to 10#2018-09-1901:44richiardiandreadon't even know what or why I am doing this frankly, any feedback is very welcome šŸ˜„#2018-09-1901:45taylora lot depends on what youā€™re generating and what youā€™re doing with the generated data. if itā€™s any thing non-trivial, itā€™s not unusual for generative tests to take a little while#2018-09-1901:45Alex Miller (Clojure team)the slow part is usually generating (too large) input values#2018-09-1901:46richiardiandreaI also added a s/coll-of :gen-max to 5#2018-09-1901:46Alex Miller (Clojure team)the most common thing I find is large collections (default max size per coll is 20) - usually I set :gen-max 3 in all of my s/coll-ofā€™s#2018-09-1901:46richiardiandreašŸ˜„#2018-09-1901:46Alex Miller (Clojure team)recursive / nested specs can also be a problem but less common than the above#2018-09-1901:47richiardiandreaI have some kind of nasty :fn checks:
(s/fdef generate-events
  :args (s/cat :options :my-ns/options)
  :fn (s/and #(count-matches-number? (-> % :args :options :number) (:ret %))
             #(data-nil-or-object? (:ret %))
             #(event-key-matches? (-> % :args :options) (:ret %))))
#2018-09-1901:49taylor:fn can also just be a simple predicate function, not sure if thatā€™d have much impact#2018-09-1901:50taylor
#(and (count-matches-number? (-> % :args :options :number) (:ret %))
      (data-nil-or-object? (:ret %)) ...)
#2018-09-1901:50richiardiandrearight, lemme try that#2018-09-1901:52richiardiandreathis seems to be definitely faster!#2018-09-1901:52richiardiandreathanks! didn't thing about that šŸ˜„#2018-09-1901:53taylorI imagine evaluating that s/and spec in each iteration is more expensive than a single predicate#2018-09-1901:54richiardiandreaI was able to get up to :max-size 50 with that simple trick, I guess reporting will be different but that's ok for now#2018-09-1901:48richiardiandreaso I think this is where it's all clogged...#2018-09-1901:48richiardiandreain ClojureScript too so I don't know, i might be many things#2018-09-1901:48richiardiandreabut thanks for the answer#2018-09-1901:49richiardiandreaif I go :max-size 20 it is taking more than I can wait for šŸ˜„ 10 is good#2018-09-1901:55richiardiandreathe trick in the tread above by @taylor makes things acceptable again:
:fn #(and (count-matches-number? (-> % :args :options :number) (:ret %))
           (data-nil-or-object? (:ret %))
           (event-key-matches? (-> % :args :options) (:ret %))))
#2018-09-1901:56richiardiandreainstead of s/and#2018-09-1905:36mishadid you try to rearrange s/and clauses?#2018-09-1905:39mishait seems to me, at this point you could benefit from custom generator(s)#2018-09-1907:11tianshu(s/valid? any? ::s/invalid) clojure.spec.alpha/invalid will always fail on spec vaild? so I can't write code like this:
(if-let [x x]
  x
  ::s/invalid)
because if-let use spec to validate the arguments.
#2018-09-1907:14mpenetyou can cheat your way around this:#2018-09-1907:14mpenet
(def si :clojure.spec.alpha/invalid)
(if-let [x si]
  :foo
  :bar)
#2018-09-1907:15mpenetit works also for your example#2018-09-1907:15mpenet(if-let [x true] si :bar)#2018-09-1907:15mpenetbut yeah that's unfortunate#2018-09-1907:16mpenetthere are a few jira issues about it already : https://dev.clojure.org/jira/browse/CLJ-1966#2018-09-1907:19tianshuyeah, if this is the only problem I think it's okay#2018-09-1907:22mpenetI only encountered this when writing custom conformers, which is something quite rare#2018-09-1907:23tianshuYes, I'm writing custom conformers...#2018-09-1921:22roklenarcichm I've got a head scratcher. I have two types of keys in a hashmap (keywords and strings). If key is string then spec A needs to be applied to value, if not then spec B is applied to value#2018-09-1921:23roklenarcicwhat's the best way to achieve this#2018-09-1921:23roklenarciccurrently I use seq to change map to sequence of pairs#2018-09-1921:23roklenarcicthen it's trivial to spec#2018-09-1921:44richiardiandrea@roklenarcic I do that with exclusive maps and I use s/or#2018-09-1921:45richiardiandreayou could also use multi specs#2018-09-1921:45richiardiandrea(if I understand the problem correctly)#2018-09-1922:16Alex Miller (Clojure team)btw, rather than seq to pairs, you can just spec as s/every of s/tuple on the map#2018-09-1922:17Alex Miller (Clojure team)thatā€™s a perfectly valid way. alternately, I think given just two choices, Iā€™d lean on s/or over s/multi-spec for this#2018-09-1922:18roklenarcicthx#2018-09-2007:07jaihindhreddyIs there a catenation which doesn't care about order? I have specs ::a, ::b, ::c and ::d. I want a catenation in which ::a and ::b are required, ::c and ::d are optional and order doesn't matter. Is this possible with spec?#2018-09-2012:01Alex Miller (Clojure team)Sounds like s/keys*#2018-09-2012:02Alex Miller (Clojure team)(s/keys* :req [::a ::b] :opt [::c ::d])#2018-09-2012:03Alex Miller (Clojure team)Does what you said#2018-09-2012:12djtango^ does keys work on non-maps?#2018-09-2012:54Alex Miller (Clojure team)s/keys* works on sequential collections#2018-09-2012:58Alex Miller (Clojure team)
user=> (require '[clojure.spec.alpha :as s])
nil
user=> (s/def ::a int?)
:user/a
user=> (s/def ::b string?)
:user/b
user=> (s/def ::c keyword?)
:user/c
user=> (s/def ::d symbol?)
:user/d
user=> (s/def ::s (s/keys* :req [::a ::b] :opt [::c ::d]))
:user/s
user=> (s/conform ::s [::b "abc" ::c :wow ::a 100])
#:user{:b "abc", :c :wow, :a 100}
user=> (pprint (take 5 (s/exercise ::s)))
([(:user/a 0 :user/b "" :user/d q :user/c :T)
  #:user{:a 0, :b "", :d q, :c :T}]
 [(:user/a -1 :user/b "" :user/d !D/- :user/c :oh)
  #:user{:a -1, :b "", :d !D/-, :c :oh}]
 [(:user/a 1 :user/b "kK") #:user{:a 1, :b "kK"}]
 [(:user/a -4 :user/b "3" :user/d Jw/R5 :user/c :?jH)
  #:user{:a -4, :b "3", :d Jw/R5, :c :?jH}]
 [(:user/a 0 :user/b "Y6" :user/d NS.+_.H/-)
  #:user{:a 0, :b "Y6", :d NS.+_.H/-}])
#2018-09-2017:14fbelinethere is a way to do exactly what keys* does but using quoted symbols instead of keywords? ex:
(s/conform ::foo [:source1 '-> :target1 'guard #() 'action #()])
Where the 'guard and the 'action maybe defined or not but the :source1 '-> :target1 should be required.
#2018-09-2017:16Alex Miller (Clojure team)no#2018-09-2017:17Alex Miller (Clojure team)keys* is capturing the common Clojure idiom of kwargs passed to a function (but those are always keywords)#2018-09-2017:17Alex Miller (Clojure team)I would probably use regex for that#2018-09-2017:19Alex Miller (Clojure team)(s/cat :s keyword? :_ #{'->} :t keyword? (s/* (s/alt :guard-pair (s/cat :_ #{'guard} :g any?) :action-pair (s/cat :_ #{'action} :a any?))))#2018-09-2017:21Alex Miller (Clojure team)or could be simpler if you want to spec less of the options (s/cat :s keyword? :_ #{'->} :t keyword? (s/* (s/cat :op symbol? :val any?)))#2018-09-2110:44jaihindhreddy@U064X3EF3 thank you sir! picard-facepalm I should have perused the clojure.spec.alpha docs.#2018-09-2414:05djtangooh wow didn't know that - but I suppose it makes sense for s/keys to work like that#2018-09-2007:13andy.fingerhutYou could probably do the s/or of all the possible orders, but that would be pretty tedious and error prone.#2018-09-2007:14andy.fingerhutWhy do you want a catenation if the order doesn't matter? Are these args to a function?#2018-09-2007:19jaihindhreddyTrying to implement serialization and deserialization of a protocol using conform and unform.#2018-09-2007:19jaihindhreddyTo see just how much of it can be pushed into spec? This is the last step.#2018-09-2007:22jaihindhreddyActual use case contains ~20 specs, so s/oring 20 factorial orders is out of the picture.#2018-09-2016:31lilactownI think I keep asking roughly the same question in different contexts but hoping someone can help me understand the best way to approach this: I need to programmatically create a spec based on external data (specifically, a GraphQL schema/query). Is there any good examples out there of how to approach this?#2018-09-2017:01taylorI think two current options depending on type of spec are either use spec's internals or eval, and I think spec-tools lib has some features for this#2018-09-2017:38favilaWe've done this with code-generation#2018-09-2017:39favilabuild up quoted lists of code then write it out to a clj file#2018-09-2017:39favila(pprinted)#2018-09-2017:40favilaif you have a deterministic order of evaluation (e.g. sorting the specs by name or topology or something) you will also see diffs clearly#2018-09-2115:00hjrnunesHi, I have a map for which there are several possible keys the values of which I'd like to use the same spec, for. So ideally I'd do something like
(s/def :key/spec (some spec...))
(s/def :key/spec2  :key/spec)

(s/def :map/spec (s/keys :opt-un [:key/spec :key/spec2])
But I don't seem to be able to do this (s/def :key/spec2 :key/spec). Ideas?
#2018-09-2115:01taylorI think I've done this same thing before and it worked#2018-09-2115:05hjrnuneshmm, then maybe it's something else. I'll double check, thanks#2018-09-2115:18Alex Miller (Clojure team)should work. you say ā€œdonā€™t seem to be able to do thisā€ - in what way?#2018-09-2115:57lilactownI'm real confused. I'm trying to spec a function, and this is the error I'm seeing in my tests now:
-- Spec failed --------------------

Function arguments

  :asur

should satisfy

  (cljs.spec.alpha/* any?)
#2018-09-2115:57lilactownI'm real confused. I'm trying to spec a function, and this is the error I'm seeing in my tests now:
-- Spec failed --------------------

Function arguments

  :asur

should satisfy

  (cljs.spec.alpha/* any?)
#2018-09-2116:00taylorI think it's complaining that :asur isn't a sequence#2018-09-2116:01lilactownthis is my fdef: (spec/fdef get-of :args (spec/* any?))#2018-09-2116:01tayloris get-of variadic?#2018-09-2116:01lilactownyes#2018-09-2116:01taylor
(s/conform (s/* any?) :asur)
=> :clojure.spec.alpha/invalid
#2018-09-2116:02lilactownI tried this in a CLJ REPL with a slightly stricter spec:
user=> (spec/fdef get-of
  #_=>   :args (spec/cat
  #_=>          :key keyword?
  #_=>          :path (spec/* keyword?)))
user/get-of
user=> (defn get-of [key & things] key)
#'user/get-of
user=> (require '[clojure.spec.test.alpha :as stest])
nil
user=> (stest/instrument)
[user/get-of]
user=> (get-of :asur)
:asur
user=> (get-of :asur :bsh)
:asur
#2018-09-2116:03taylor#2018-09-2116:03lilactownwell that exact spec fails the same way in my tests#2018-09-2116:05lilactown
-- Spec failed --------------------

Function arguments

  :asur

should satisfy

  (cljs.spec.alpha/cat
   :key
   keyword?
   :path
   (cljs.spec.alpha/* any?))
#2018-09-2116:09taylorhmm, not sure. this works for me:
(defn foo [& xs] (prn xs))
(s/fdef foo :args (s/* any?))
(st/instrument `foo)
(foo 1 2 3)
#2018-09-2116:09lilactownright. hence my WTF#2018-09-2116:11taylorin times like these, I restart REPL/computer/get back in bed#2018-09-2116:11lilactownhahaha#2018-09-2116:11lilactownI have restarted my REPL and everything. bed is tempting#2018-09-2116:11lilactownI wonder if there's something weird going on with CLJS instrumentation#2018-09-2115:58lilactownthe test output/stacktrace doesn't give the line # or file or anything. but I'm confused WHY the arguments wouldn't satisfy that spec??#2018-09-2116:21lilactownoh for crying out loud: https://dev.clojure.org/jira/browse/CLJS-2793#2018-09-2117:03Alex Miller (Clojure team)doh!#2018-09-2117:03Alex Miller (Clojure team)sorry, I was unaware of that one#2018-09-2117:00jrychterI'm writing up my thoughts on clojure.spec, having tried to use it for several months now in an application. I'm banging my head against multiple issues. So far it seems like most of them are related to data conversion: if I removed all conformers and :into from my specs and used separate explicit functions for data adjustments, I think my life with spec would be easier. But most writeups and presentations about spec tantalizingly mention conforming data as a major advantage...#2018-09-2117:01Alex Miller (Clojure team)conforming != coercion#2018-09-2117:01jrychterThat is something that I think I missed (I don't think I'm the only one).#2018-09-2117:02Alex Miller (Clojure team)conforming was always intended to be an advanced tool to use in writing spec implementations, not for data transformation#2018-09-2117:02Alex Miller (Clojure team)it is intentionally absent from the spec guide#2018-09-2117:02jrychterAnd yet there is :into (in the guide, too).#2018-09-2117:03Alex Miller (Clojure team)yes, and is useful for some use cases (+ for gen), but that doesnā€™t make it a generic transformation tool#2018-09-2117:03Alex Miller (Clojure team)https://github.com/wilkerlucio/spec-coerce is imo a pretty good approach to leveraging specā€™s for the purpose of coercion#2018-09-2117:13jrychterHmm. I think there is confusion around the issue (see for example https://stackoverflow.com/questions/45188850/is-use-of-clojure-spec-for-coercion-idiomatic). I think it's worth mentioning explicitly in the guide. I also found that there is not a lot of guides for spec. The "Spec Guide" is great, but I see it as more of a walktrough. I've been looking on tips on how to use spec in apps (e.g. do I use s/* or s/coll-of?). Most online presentations or tutorials are introductory material only.#2018-09-2117:14jrychter@alexmiller I have a file with my notes on spec (basically documenting the holes I've been falling into). I could E-mail it to you, would you like it? It would let you know what my (erroneous) thinking was.#2018-09-2117:17Alex Miller (Clojure team)the answers on that stackoverflow thread seem pretty straightforward and correct to me :)#2018-09-2117:17jrychterYes.#2018-09-2117:17Alex Miller (Clojure team)youā€™re welcome to email, but I am just about to enter the Strange Loop black hole and wonā€™t be looking at anything for Clojure for the next week and a half#2018-09-2117:18jrychterThat's fine. I don't want you to answer that E-mail anyway. I'm not looking for support. I just thought that if I was the author of the spec guide, I'd want to know what some misconceptions are in readers' minds.#2018-09-2117:19Alex Miller (Clojure team)happy to hear those - in fact, an issue on https://github.com/clojure/clojure-site would be just as good#2018-09-2117:19Alex Miller (Clojure team)at the moment, Iā€™m not looking to invest a lot of time in additional spec docs because we are starting to work on some spec changes that are likely to change some of whatever advice we would give#2018-09-2117:23jrychterAn issue it will be, then. Perhaps the holes I'm falling into will influence some of the thinking behind spec changes šŸ™‚#2018-09-2117:25Alex Miller (Clojure team)I do have a full day of spec materials Iā€™ve taught several times as a course now#2018-09-2117:25Alex Miller (Clojure team)given time, could be turned into some useful advice#2018-09-2117:33seancorfield@jrychter FWIW, we do use spec for some very limited coercion but, as noted in one of those SO answers, we also have two types of spec: "API (coercing) specs" and "domain specs" (non-coercing). All of our internal specs are non-coercing. And even in the API specs, we only do very limited coercion: we accept strings-that-can-be-converted-to-<T> and produce values of type T, for longs, doubles, Booleans, and date/time values. That's it.#2018-09-2117:33roklenarcicis it possible to create a generator from a specced function (i.e. one with a fdef)?#2018-09-2117:35seancorfieldWe've found that to be extremely convenient because if you have a form field that should be a long, it's going to come in as a string so you either have a spec that attempts coercion to long but still conforms to the string and then you need an actual coercion as well, or you have a layer of coercion first followed by specs for whatever successfully coerces -- and then you have two layers in your error handling which makes for more complicated code.#2018-09-2117:35jrychter@seancorfield Well, my coercions were very limited, too. Strings to keywords and collection types, basically (to keep a collection as a set or a sorted set). But even then I'm falling into traps. An example is that I though that s/valid? tells you if your data is valid according to the spec. It doesn't. It tells you if the data will be valid if you pass it through s/conform.#2018-09-2117:37seancorfieldRight, well the pattern we use is
(let [params (s/conform ::api-spec input-params)]
  (if (s/invalid? params)
    (respond-with-error (s/explain-data ::api-spec input-params))
    (happy-path params)))
#2018-09-2117:37jrychterI'd argue that parsing strings to longs is not really coercion, it's parsing (my form code does it).#2018-09-2117:39jrychter@seancorfield Yes, I discovered this the hard way. Problem is that if you spec your functions, then data that is not valid (but conformable) will reach your function code.#2018-09-2117:39seancorfieldWe don't spec many functions -- we mostly spec data structures.#2018-09-2117:40seancorfieldAnd, like I say, these are specs at the outer boundary of our system. Inside our system we only have non-coercing specs.#2018-09-2117:40jrychterI will try to remove all conformers (`s/conformer` and :into) from my specs and use explicit coercion. I'd love to be able to "attach" information about coercions to the specs, otherwise I have to manually define coercion functions and deal with nested coercions.#2018-09-2117:41seancorfieldAnd our use of spec is almost all explicit and part of our production code. So we don't rely on instrumentation of functions that way.#2018-09-2117:42jrychter@seancorfield That's how I see it, too, but I'm still considering instrumenting functions, and looking for possible uses of spec.#2018-09-2117:44jrychterSo, at a first glance, that spec-coerce library tries to do too much. I can see parsing from strings to doubles or integers. I'd rather have a way to attach custom coercion fns to specs and a way to walk a spec and apply all coercion fns that I specified.#2018-09-2117:45seancorfieldInstrument (and check) are purely part of testing for us -- as well as using exercise to produce random conforming data for use in some example-based tests.#2018-09-2117:45jrychterI think I will also remove all uses of s/coll-of ... :into and convert those into explicit (s/and (s/coll-of ::something) sorted? set?) or similar. That will let me use s/valid? to catch instances of data where something isn't a sorted set.#2018-09-2117:45seancorfieldAs an FYI, from our Clojure codebase:
Clojure build/config 48 files 2538 total loc
Clojure source 268 files 63902 total loc,
    3331 fns, 658 of which are private,
    403 vars, 42 macros, 71 atoms,
    478 specs, 19 function specs.
Clojure tests 149 files 20002 total loc,
    23 specs, 1 function specs.
#2018-09-2117:47jrychterI have ~45k lines of code in my app right now, so not nearly there, but close.#2018-09-2117:49Alex Miller (Clojure team)I think itā€™s useful to think of the purpose of conform as ā€œwhy does this value conform to the spec?ā€ and not ā€œtransform this value into some target valueā€#2018-09-2117:50jrychterI do think that s/valid? is not a good name, though.#2018-09-2117:51Alex Miller (Clojure team)becauseā€¦#2018-09-2117:52jrychterBecause it doesn't tell me if my data is valid.#2018-09-2117:52Alex Miller (Clojure team)sorry, donā€™t understand#2018-09-2117:53roklenarcic64k of lines of clojure is a lot of clojure šŸ™‚#2018-09-2117:55jrychter
(s/def ::stuff (s/coll-of int? :into #{}))
(s/valid? ::stuff [3 2 1])

(s/def ::other-stuff (s/and (s/conformer keyword) keyword?))
(s/valid? ::other-stuff "not-a-keyword-at-all")
#2018-09-2117:56jrychterBoth s/valid? will return true, even though the data, as supplied to s/valid?, will cause problems in functions that expect a set and a keyword, respectively.#2018-09-2117:58bbrinckFor the first spec, would :kind set? be more accurate?#2018-09-2118:00jrychterYes, I should remove all conformers and :into from my specs. My understanding right now is: s/valid? tells you if the data will be valid when conformed, so do not use s/valid? to check if the data is valid if you use conformers or :into anywhere in your specs.#2018-09-2118:00bbrinckYes, the examples seems like concrete cases where using s/conformer for coercion creates headaches#2018-09-2118:01jrychterHence my point about s/valid? not telling me if my data is valid.#2018-09-2118:01favilavalidity is only testable after conforming#2018-09-2118:01favilaconforming isn't something separate that exists outside of an abstract expression of the desired result shape#2018-09-2118:02favilait's a predicate that also transforms#2018-09-2118:04bbrinckCorrect, if you use conformer to do coercion (which is not recommended), the results of valid? are confusing. Also, some details of explain-data get confusing because the paths to data can change. This is one reason why coercion is not recommended, AIUI#2018-09-2118:04favilaconforming is super-handy though#2018-09-2118:05favilabut using it means you are indirectly specing the "input"#2018-09-2118:05favilathe input must be something-that-i-can-make-into-something-else#2018-09-2118:05favilamaybe think of conform as a normalization step#2018-09-2118:06favilarather than coersion or parsing#2018-09-2118:06favilathat said I have written parsers using conform#2018-09-2118:06Alex Miller (Clojure team)conform is not normalization or coercion or parsing#2018-09-2118:06favilawhat's left?#2018-09-2118:07Alex Miller (Clojure team)destructuring#2018-09-2118:07favila?#2018-09-2118:08Alex Miller (Clojure team)if the value matches the spec, it will destructure the input value to tell you which alternative was chosen in specs with alternates and which parts of the data matched which components of the spec#2018-09-2118:08jrychterThat's how I understand it now, too.#2018-09-2118:09favilaok, I'm talking about conformers#2018-09-2118:09Alex Miller (Clojure team)tags exist in s/or, s/alt, s/cat so that parts and choices can be labeled in the conformed (destructured values)#2018-09-2118:09favilanevermind we are talking past each other#2018-09-2118:09Alex Miller (Clojure team)conformers exist to build complex composite spec implementations from existing pieces#2018-09-2118:12favilawhat would be an example of that? s/keys* does not appear to use conformers#2018-09-2118:15Alex Miller (Clojure team)oh, I might have called out the wrong example (s/keys* uses s/&). might be remembering s/nilable (which actually has been rewritten since for performance)#2018-09-2118:17favilaah but the & predicate is a conformer#2018-09-2118:18favilait appears to be the only use in spec#2018-09-2118:19favilaso I think you are saying that the intended use of conformers is to act as glue to chain specs together#2018-09-2118:19favilait's not meant to make s/conform visible alternations#2018-09-2118:21Alex Miller (Clojure team)yes#2018-09-2118:09Alex Miller (Clojure team)like building s/keys* from s/keys#2018-09-2118:10Alex Miller (Clojure team)conform is not transformation, conformers are not coercion#2018-09-2118:11jrychterSo, I'm looking for suggestions: what is the simplest way to add my own coercion functions to certain specs and then walk a spec calling my functions? Metosin's spec-tools do this by wrapping specs, which I really do not like. Is that the only way?#2018-09-2118:11Alex Miller (Clojure team)https://github.com/wilkerlucio/spec-coerce#2018-09-2118:11Alex Miller (Clojure team)^^ is vastly preferable to spec-tools imo#2018-09-2118:12jrychter@alexmiller At a first glance, this seems to try to do too much. I don't want to parse strings to integers by default. I only want to walk the spec and call specific coercion functions that I provided. But perhaps I'm missing something, I'll look deeper.#2018-09-2118:13favila@jrychter spec really pushes you in the direction of using different keys#2018-09-2118:14favilai.e. if you care about both the "raw" data and the "cleaned-up" (slightly-parsed, whatever) data as independent specs, you need separate keys#2018-09-2118:14jrychterIt seems that with spec-coerce you have to use coerce-structure to walk a complex (nested) spec, duplicating your data structure in the call.#2018-09-2118:19jrychterBasically, I'm looking for something that would let me do this:
(s/def ::kw (s/and keyword? (s/coerce-fn keyword)))
(s/def ::nested (s/keys :req-un [::kw]))
(s/def ::data (s/keys ::req-un [::nested]))

(coerce ::data {:nested {:kw "x"}}) => {:nested {:kw :x}}
ā€¦without touching anything that doesn't have a coerce-fn defined.
#2018-09-2118:25favilayeah spec-coerce seems backward; I would want to opt-in to coercion, not have universal coersion predicates#2018-09-2118:26bbrinckSeems like a sensible suggestion to have the option to turn off the default coercions#2018-09-2118:26bbrinck(a suggestion for spec-coerce project, I mean)#2018-09-2118:27favilaI'll reiterate though that spec really seems to urge in the direction of more specs#2018-09-2118:27favilai.e. a separate "edge" spec vs an internal spec#2018-09-2118:28favilaso in your example "::kw as a string encoding a keyword" is different from "::kw as a kw"#2018-09-2118:28favilaand then rename keys everywhere#2018-09-2118:29favila(if using s/keys with namespaced maps)#2018-09-2118:36bbrinck@jrychter A challenging thing in your example above is that the coerce-fn is tied to something global (the qualified kw), but your data {:nested {:kw "x"}} is presumably figuring out that :kw means ::kw based on context?#2018-09-2118:36jrychterI need a way to walk a spec, which doesn't seem to exist. I thought about defining multimethods dispatching on spec keys, but that won't work for unqualified keys.#2018-09-2118:37jrychter@bbrinck I'm passing all information about the context as the first argument to coerce: the ::data spec knows everything.#2018-09-2118:37bbrinckright, but youā€™d have to walk the spec to know what spec :kw means#2018-09-2118:38bbrincklike you said above, that adds a requirement to walk the spec to know where you are in the spec#2018-09-2118:40jrychterYes, but I thought that is what spec does anyway (when, say, conforming).#2018-09-2118:41bbrinckcorrect, youā€™d need to duplicate this process. itā€™d be an interesting project, Iā€™d need to dig more into the spec impl to understand how viable itā€™d be to duplicate#2018-09-2118:42faviladoesn't spec-tools have a spec visitor?#2018-09-2118:42bbrinckOTOH, if you could assume that the name unqualified keyword always used the same coercion, the problem is simpler#2018-09-2118:43bbrinckwhich doesnā€™t seem entirely crazy. Perhaps naively, Iā€™d think that if you want to, say, convert :zip-code from string->int in one place, you probably want to do the same coercion elsewhere#2018-09-2118:43bbrinckmaybe thatā€™s oversimplifying the real world case šŸ˜‰#2018-09-2118:44jrychterNot necessarily, but I could stick with that for a while as a workaround. That would mean I could define a multimethod coercer and walk the structures myself. But then spec provides me very little value in the end.#2018-09-2118:45bbrinckHm, well spec would still let you validate the result of coercion (and conform to use destructuring)#2018-09-2118:45bbrinckhttps://gist.github.com/bhb/7fde08abc1f3f5d06b05481b4e200614#2018-09-2118:46favilaif you used the "option" keys in a consistent way, s/conform could perform the tagging for you (except for s/keys req-un and opt-un)#2018-09-2118:46bbrinckNote that if you donā€™t to have to call two different implementations of def for :kw, then you could just make a macro that does both e.g.#2018-09-2118:47bbrinckhttps://github.com/bhb/expound/blob/master/src/expound/alpha.cljc#L930-L940#2018-09-2118:47bbrinck@jrychter (an example of a def macro that calls the normal def but also registers a message. In your case, youā€™d be registering a spec + coercer)#2018-09-2118:51bbrinck@jrychter whoops, I messed up the example in that gist. Please reload, Iā€™ve updated. The example should have been: (coerce-structure {:nested {:kw "x"}}) ;; => {:nested {:kw :x}}#2018-09-2118:51jrychterThat is interesting. It's not as good as clojure.spec calling my coercions (clojure.spec would know exactly which function to call), but I guess it is a workaround.#2018-09-2119:01jrychter@bbrinck Thank you for that gist. I am trying to implement that right now to see how it goes.#2018-09-2119:03bbrinck@jrychter np. Good luck! Definitely a workaround, but hopefully covers the 80% case. Let me know how it goes šŸ™‚#2018-09-2119:11jrychter@alexmiller Can we hope that this use case (explicit coercion using user-registered coercion functions) will be considered in future spec work?#2018-09-2119:12Alex Miller (Clojure team)you can hope for anything you like :)#2018-09-2119:12Alex Miller (Clojure team)not something weā€™re working on currently#2018-09-2119:18jrychterLet's narrow it down a bit: at least a way to walk a spec, calling a function with the keyword and value at each point. That would still mean maintaining a separate registry, but that's fine.#2018-09-2119:27jrychter@bbrinck I'm discovering more limitations as I go ā€” for example, a certain spec might be redefined as s/nilable in some places under the same (when unqualified) keyword.#2018-09-2119:30bbrinck@jrychter Right, itā€™s probably worth making all coercing functions a little bit defensive i.e. if the pre-coercion type isnā€™t what you expect, just return the existing value#2018-09-2119:30bbrinckso, untested, but something like (if (string? x) (keyword x) x))#2018-09-2119:31bbrinckKeep in mind the role of coercion functions is not to validate the incoming data, but rather do a best-effort attempt to get the fields into a format that will be valid according the spec#2018-09-2119:32jrychterOf course. And the suggestion about being defensive is a very good one.#2018-09-2119:55jrychterAnd another limitation (as I'm going through my code): if a spec is redefined under a different name, coercion needs to be redefined for that new name, too. It won't carry over.#2018-09-2120:05bbrinckVery true, itā€™s per name, not per spec#2018-09-2120:06bbrinckYou could do some magic looking at s/form for a spec and then look up that spec ā€¦ but itā€™d get complicated šŸ˜‰#2018-09-2118:22josh.freckletonIs spec the right tool for validating data from JSON and db results (if so, any docs or blogs about it?), or should I stick with Schema?#2018-09-2118:24Alex Miller (Clojure team)spec is somewhat cumbersome right now for validating maps with unqualified keys (which tends to include cases like JSON)#2018-09-2118:24Alex Miller (Clojure team)the next round of spec changes will have some improvements in this area#2018-09-2118:26roklenarcicIs there some way to refer to args spec of a function?#2018-09-2118:29seancorfield@roklenarcic Can you provide more context? What problem are you trying to solve?#2018-09-2118:31roklenarcicI'm trying to generate arguments to a function that is specced#2018-09-2118:31roklenarcicbut without calling it#2018-09-2118:32roklenarcicso I have a function and a bunch of fdefs#2018-09-2118:32roklenarcicI mean one fdef for this particular one#2018-09-2118:32roklenarcicand I want to generate vectors of arguments#2018-09-2118:35Alex Miller (Clojure team)yes, if you get a function spec, it implements key lookup so you can grab :args, :ret, :fn out of it#2018-09-2118:36Alex Miller (Clojure team)(-> a-sym s/get-spec :args)#2018-09-2118:50roklenarciccool#2018-09-2118:51roklenarcicthat's what I was looking for šŸ™‚#2018-09-2119:23ikitommihave you @alexmiller used either of spec-coerce or spec-tools?#2018-09-2119:25jrychterNo, but I took a look at both. Both do something different, not quite what I want, and both are way too complex.#2018-09-2119:27ikitommiI feel both are hacks (sorry @wilkerlucio), because how spec is designed.#2018-09-2119:30ikitommiletā€™s look at an example:#2018-09-2119:30ikitommi
(require '[clojure.spec.alpha :as s])

(s/def :db/ident qualified-keyword?)
(s/def :db/valueType (s/and keyword? #{:uuid :string}))
(s/def :db/unique (s/and keyword? #{:identity :value}))
(s/def :db/cardinality (s/and keyword? #{:one :many}))
(s/def :db/doc string?)

(s/def :simple/field
  (s/cat
    :db/ident :db/ident
    :db/valueType :db/valueType
    :db/cardinality (s/? :db/cardinality)
    :db/unique (s/? :db/unique)
    :db/doc (s/? :db/doc)))

(s/def :simple/entity
  (s/+ (s/spec :simple/field)))

(def value
  [[:product/id :uuid :one :identity "id"]
   [:product/name :string "name"]])

(s/valid?
  :simple/entity
  value)
; true
#2018-09-2119:31ikitommiI think spec really shines here (the regex parts).#2018-09-2119:31ikitommibut same over json:#2018-09-2119:31ikitommi
(def json-value
  [["product/id" "uuid" "one" "identity" "id"]
   ["product/name" "string" "name"]])

(s/valid?
  :simple/entity
  json-value)
; false
#2018-09-2119:33ikitommiout of luck. To make a standalone transformer outside of spec, one would need to be able to parse the spec and basically rebuild the s/conform to understand what branches are being used.#2018-09-2119:36ikitommispec-tools uses the s/conform and makes the spec do all the heavy lifting. but sadly, the leaf specs need to be wrapped into ā€œconforming predicatesā€.#2018-09-2119:39ikitommispec-coerce reads the spec forms, which works nicely for most/simple forms.#2018-09-2119:47ikitommihopefully there will be a solution for this in the upcoming spec releases, I think it would be bad for the clojure / spec story not to support (or enable libs to fully support) coercion. one s/walkmethod on Specs for libs to use? protocol dispatch would be clean and much faster than parsing the forms or using dynamic binding like the current coercion libs do.#2018-09-2119:56jrychter@ikitommi It seems that that is exactly what I have been asking for in that thread (a way to walk a spec).#2018-09-2119:58Alex Miller (Clojure team)that is something we have talked about providing, but itā€™s a ways down the list#2018-09-2119:59jrychterIt would seem that this is something that people with large apps with complex data structures and JSON interop encounter all the time.#2018-09-2120:02seancorfieldWe haven't encountered that yet šŸ™‚#2018-09-2120:03seancorfield(but maybe we're deliberately keeping our data structures simple enough?)#2018-09-2120:03jrychter@seancorfield You seem to be using conformers for that, judging from the code you've shown?#2018-09-2120:04seancorfieldOnly right at the edge -- where I think it's acceptable.#2018-09-2120:04jrychterBut that proves my point. Coercion is needed.#2018-09-2120:04seancorfieldNot "needed". Convenient.#2018-09-2120:05seancorfieldAnd spec provides what we need already.#2018-09-2120:05jrychterI'd argue that it doesn't. Or my expectations are too high: I expected to be able to "spec" my data structures only once, and reuse the resulting structure for coercion. I can't do that right now.#2018-09-2120:06jrychterI can see how this could be outside the scope of spec, but this is why I'm asking (and @ikitommi seems to be, too) for a way to walk the spec data, so that we can extend it and reuse the definitions.#2018-09-2120:23ikitommi@jrychter there is CLJ-2251#2018-09-2120:13Alex Miller (Clojure team)does it seem weird to you to have a ā€œspecā€ that defines your data, but then require coercion that does not conform to that spec? (it seems weird to me)#2018-09-2120:19ikitommiweb is weird. we expect a valid edn data but get JSON instead ;)#2018-09-2120:23Alex Miller (Clojure team)it seems conceptually cleaner to me to separate the transformation of the crap that you get into the format you want from validating that transformed data#2018-09-2120:24Alex Miller (Clojure team)and I get that youā€™ve already defined the structure and it seems convenient to use that structure to learn about target expectations while doing the transformation#2018-09-2120:24jrychter@alexmiller In theory, yes. But in practice I'm dealing with data from a JSON database, nested data structures. What you are proposing means that I will be duplicating data structure definitions: once for spec, and again for coercion. Seems like a waste.#2018-09-2120:24Alex Miller (Clojure team)and so I think requests that deal with using specs as data in that way make sense#2018-09-2120:25Alex Miller (Clojure team)asking spec to do the work seems weird to me (but a good case for a lib on top)#2018-09-2120:25wilkerlucio@jrychter had you tried using coerce-structure from spec-coerce? because thats the indented feature for it, you pass it a structure and it will use the leaf attribute specs to do the coercion#2018-09-2120:26jrychter@alexmiller Actually, I do not want to involve "target expectations" in this. All I want is to reuse the structure from spec and be able to hang my own data (coercion functions in this case) on it.#2018-09-2120:26Alex Miller (Clojure team)well that would potentially be handled by meta support, which weā€™re in favor of (itā€™s just tricky to implement well)#2018-09-2120:27jrychter@wilkerlucio Yes. I am using a variant of that now, suggested by @bbrinck. But this has a number of drawbacks.#2018-09-2120:27Alex Miller (Clojure team)as a stop gap, itā€™s not hard to build a second registry, also keyed by spec name that had that info#2018-09-2120:27ikitommi@alexmiller agrer that it should be made on top. But we need help from spec to make this possible. E.g. the walk#2018-09-2120:27Alex Miller (Clojure team)yes#2018-09-2120:27jrychter@alexmiller That is exactly what I'm doing right now, and there are a number of limitations, especially with non-namespaced keys.#2018-09-2120:27Alex Miller (Clojure team)that may become clearer soon#2018-09-2120:27wilkerluciospec-coerce also supports a secondary registry to specify, what I would like from spec is to be easier to traverse the spec definitions, I have to do some things that I feel are not very stable to get that#2018-09-2120:28wilkerluciobut I guess specs on the specs will solve this#2018-09-2120:28Alex Miller (Clojure team)maybe#2018-09-2120:29Alex Miller (Clojure team)this is all helpful, in particular Iā€™m trying to clarify the problem statement so I can talk well to Rich about it#2018-09-2120:29Alex Miller (Clojure team)spec walking has come up in several contexts and is one potentially useful generic feature, meta is another#2018-09-2120:30jrychter@alexmiller It seems to me that you think much of this is unnecessary, because invalid data points to deficiencies in transport or db. That is true in general. But please consider that even using EDN you will not get sorted sets by reading. Something needs to coerce sets into sorted sets.#2018-09-2120:30Alex Miller (Clojure team)and better support for unnamespaced attributes#2018-09-2120:30Alex Miller (Clojure team)I didnā€™t say it was unnecessary. Iā€™ve built my share of apps and I get it.#2018-09-2120:31Alex Miller (Clojure team)I just donā€™t think that spec necessarily needs to be the thing providing ā€œcoercionā€#2018-09-2120:31jrychterOh, absolutely. It's just that it makes sense to re-use the resulting structure.#2018-09-2120:32Alex Miller (Clojure team)Iā€™m agreeing with you on that#2018-09-2120:33Alex Miller (Clojure team)using conformers for this stuff is what Rich calls the ā€œmeat grinderā€ approach - I see that turning the crank gets me from A to B#2018-09-2120:34ghadiPart of the challenge is that 99% of other libraries provide "coercion" of some sort (plumatic/schema and everything in Python) and there are Real Problems with that stuff. I think it has psychologically primed us to want to tangle it together... some caution is advisable and focus on the problems#2018-09-2120:34Alex Miller (Clojure team)but it misses that the grinder is not the point of it#2018-09-2120:35jrychterHere's a practical example from my screen right now:
(ns partsbox.build-quantity
  (:require [clojure.spec.alpha :as s]
            [partsbox.coerce :as c]))

;; A build quantity is an integer that is always greater than 0.
(s/def ::quantity pos-int?)

;; A non-empty sorted set of quantities used for build quantities and pricing quantities.
(s/def ::quantities (s/and (s/coll-of ::quantity :min-count 1) sorted? set?))
(c/defcoercion ::quantities (fn [data] (into (sorted-set) data)))

(def +default-quantities+ (s/conform ::quantities (c/coerce ::quantities [1 10 25 50 100 250 500 1000])))
This is all fine (a separate registry, another line of code for defining the coercion). I would just want to be able to write a general c/coerce that would walk the spec correctly. I can only have a deficient implementation right now.
#2018-09-2308:39dottedmagI'd like to instrument a function taking an argument spec'ed in another namespace. When I do (s/fdef my-func :args (s/cat :arg1 :alias/myspec)) it fails to resolve spec via alias. (s/fdef my-func :args (s/cat :arg1 :)) works. Is it intended?#2018-09-2311:36taylorfor ::alias/myspec to work I think you need to require/alias that namespace e.g. (require '[some.ns :as alias])#2018-09-2313:52dottedmag@U3DAE8HMG Yep, thanks. I was using wrong syntax (with a single :).#2018-09-2308:40dottedmagOh, :alias/myspec vs ::alias/myspec.#2018-09-2501:46bbrinckIā€™m confused about the behavior of explain for keys* specs#2018-09-2501:46bbrinckFor a cat spec, the in refers to the to the position of the bad data before conformance#2018-09-2501:47bbrinckbut for a keys* spec, it seems to refer to the position of the bad data after conformance.#2018-09-2501:47bbrinckWhy is it different? https://gist.github.com/bhb/a7ba32e6965bebecc94020f10751ec58#2018-09-2501:54taylorI wonder if it's because of this https://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/alpha.clj#L1787#2018-09-2501:55taylor>takes the same arguments as spec/keys and returns a regex op that matches sequences of key/values, converts them into a map, and conforms that map#2018-09-2515:29bbrinckAh, thanks! Although that explains the mechanism of why the two are different, Iā€™m still not sure on the reasoning šŸ˜‰#2018-09-2515:31bbrincki.e. why the difference in behavior is desirable#2018-09-2515:32taylorsame but that's out of my depth šŸ™‚ I only noticed the keys* impl. would seem to cause this behavior#2018-09-2515:33bbrinckYep, I appreciate the link šŸ™‚ It does indeed explain the difference. Thx!#2018-09-2515:33taylorI suppose it may be just to make the final conform step easier, when the input has already been conformed to a map :man-shrugging:#2018-09-2515:36bbrinckYeah, maybe! The downside is that when I want to locate the source of the bad data, the in value is inconsistent in itā€™s format. In particular, itā€™s hard to make a custom printer like Expound be able to locate the bad data#2018-09-2501:47bbrinckIs this a known issue? Or am I misunderstanding the intent here?#2018-09-2519:15roklenarcicHm if I define a spec, it requires that components are already defined. So if I have mutually recursive spec, do I just define one of the components at the top with bogus spec?#2018-09-2519:46taylorI've run into this issue before with a recursive spec, and I think it involved a call to s/spec inside the recursive spec definition#2018-09-2519:23donaldballIā€™m not sure what you mean by it requiring that components are already defined. Certainly something recursive like this works fine:
(s/def ::json
  (s/or :string string?
        :list ::json-list
        :map ::json-map))

(s/def ::json-list
  (s/coll-of ::json :kind vector?))

(s/def ::json-map
  (s/map-of ::json ::json))
#2018-09-2520:18nenadalmnote that in some cases it doesn't work in cljs. e.g.: https://github.com/metosin/reitit/issues/127#2018-09-2521:34richiardiandreaHello folks I am having a hard time reading this spec error, I think I need some help in understanding what is wrong:
#### [{:spec #object[cljs.spec.alpha.t_cljs$spec$alpha23581], :clojure.test.check/ret {:shrunk {:total-nodes-visited 0, :depth 0, :pass? false, :result #error {:message Call to #'lambda.response/error-map did not conform to spec:
val: "Invalid body" fails at: [:args] predicate: (cat :message string? :data (? (nilable map?)))
:cljs.spec.alpha/spec  #object[cljs.spec.alpha.t_cljs$spec$alpha23551]
:cljs.spec.alpha/value  "Invalid body"
:cljs.spec.alpha/args  "Invalid body"
:cljs.spec.alpha/failure  :instrument
, :data {:cljs.spec.alpha/problems [{:path [:args], :pred (cljs.spec.alpha/cat :message cljs.core/string? :data (cljs.spec.alpha/? (cljs.spec.alpha/nilable cljs.core/map?))), :val Invalid body, :via [], :in []}], :cljs.spec.alpha/spec #object[cljs.spec.alpha.t_cljs$spec$alpha23551], :cljs.spec.alpha/value Invalid body, :cljs.spec.alpha/args Invalid body, :cljs.spec.alpha/failure :instrument}}, :result-data {:clojure.test.check.properties/error #error {:message Call to #'lambda.response/error-map did not conform to spec:
#2018-09-2521:36taylorthat spec looks like it may be for a variadic function, so maybe this is related https://dev.clojure.org/jira/browse/CLJS-2793#2018-09-2522:24richiardiandreaLet me see if that's the issue#2018-09-2522:34richiardiandreaWell it seems like things are working with just a non-variadic function, however I am not sure my bug is related to the above#2018-09-2523:16richiardiandreaYep it is relate, I can reproduce it.#2018-09-2521:40guy
val: "Invalid body" fails at: [:args] predicate: (cat :message string? :data (? (nilable map?)))
#2018-09-2521:41guyto me it looks like you have a value "invalid body"#2018-09-2521:41guyand a predicate#2018-09-2521:41guy(cat :message string? :data (? (nilable map?)))#2018-09-2521:42guySo maybe your predicate / spec is wrong?#2018-09-2521:44guy
{:path [:args], :pred (cljs.spec.alpha/cat :message cljs.core/string? :data (cljs.spec.alpha/? (cljs.spec.alpha/nilable cljs.core/map?)))
#2018-09-2521:45guyThis is saying that the args (i guess ur function args) are being given the value Invalid body and failing your spec#2018-09-2521:48guyhttps://clojuredocs.org/clojure.spec.alpha/cat Tbh I donā€™t really understand how this works#2018-09-2521:49guy@richiardiandrea sorry iā€™m not much help haha#2018-09-2521:49richiardiandreayeah#2018-09-2521:49richiardiandreathat predicate is supposed to match one arguments as string and optionally a data field#2018-09-2521:49guywhat does ur (s/fdef ..) look like?#2018-09-2521:49guyah right#2018-09-2521:50richiardiandrea
(s/fdef error-map
  :args (s/cat :message string? :data (s/? (s/nilable map?)))
  :ret :tools.lambda.response/error-map)
#2018-09-2523:16taylorwhat's your defn error-map arg list look like?#2018-09-2523:37richiardiandreait was [message & [data]]#2018-09-2523:38taylorit could be related to that JIRA I mentioned above about CLJS, instrumentations, and variadic functions#2018-09-2523:56richiardiandreaYep left a comment there, I can reproduce it here...#2018-09-2607:21steveb8nquestion : Iā€™ve heard about context specific specs quite a few times i.e. same entity but different uses e.g. reading vs inserting a map from/to db. another might be reading entities in a list vs individually. I can guess how these specs might look but was hoping I could learn from looking at others. does anyone know of public codebases with examples of this technique?#2018-09-2612:11jaihindhreddy@steveb8n I would be interested to see this too. But what you describe feels like something spec is not meant to do. Spec specs data, not entities or identities. ::entity-while-reading, ::entity-while-inserting might be better as two different specs. Maybe metosin/spec-tools can do this.#2018-09-2612:14steveb8nagreed. I think each context is probably itā€™s own spec.#2018-09-2612:59favilaYeh spec doesnā€™t support this, not sure where you heard about that. If you find out Iā€™m interested#2018-09-2613:03favilaThere are two cases that hurt where I want this. One is dealing with datomic data. Because itā€™s a graph each ref key could be (in the widest possible case) an entity id, a string, a lookup ref, or a nested vector or map, again with unknown keys ( depends on what was pulled)#2018-09-2613:05favilaAt certain spots in the program I want to allow only a few of those possibilities. Eg On read I often want to guarantee that a certain set of sub-keys was pulled#2018-09-2613:06favilaThe other case is dealing with data where the same key is used but it may have a narrower spec depending on the ā€œkindā€ of map it is in#2018-09-2613:10favilaEg a key could have values 1 2 or 3, but that key must have value 1 when some other key has a certain value#2018-09-2613:11favilaMulti-spec and predicates can do this but itā€™s awkward and verbose and doesnā€™t gen well#2018-09-2613:11favilaMulti-spec and predicates can do this but itā€™s awkward and verbose and doesnā€™t gen well#2018-09-2616:59pvillegas12Interested about this, I would like to be able to express that a restriction on a composed spec. If spec1 has value :a within a set of alternatives, spec2 cannot have that value (both of spec1 and spec2 are keys of another spec)#2018-09-2620:55pvillegas12To do this, one can s/and an arbitrary function at the compound spec level to make this happen defined as the second operand (for gen support to be there)#2018-09-2621:17favilas/and of s/keys works very badly#2018-09-2621:17favilait's unlikely a useable generator will result#2018-09-2621:17favilas/and maps to gen/such-that#2018-09-2621:18favilathis is why s/merge exists#2018-09-2621:22pvillegas12In my case I donā€™t want to merge two sets of specs. I have a spec that is the composition of 5 specs (with s/keys) and I wanted two ensure that two of those did not have the same value#2018-09-2621:23pvillegas12This works with gen because the set of possibilities for those specs is 2 šŸ˜› (a really small set)#2018-09-2621:23favila5 s/keys specs?#2018-09-2621:23pvillegas12
(s/def ::integration (s/and 
                      (s/keys :req [:m/name
                                    :m/doc-type
                                    :m/origin
                                    :m/target
                                    :m/rules])
                      distinct-origin-target))
#2018-09-2621:24favilaoh that's not at all what I run in to#2018-09-2621:24pvillegas12what do you run into?#2018-09-2621:24pvillegas12(I can see how distinct-origin-target can bump gen if the resulting restriction is too big a set)#2018-09-2621:25favilasomething like if :m/doc-type is "x", :m/origin must be some narrower range of values that is normally allowed#2018-09-2621:25favilavs if :m/doc-type were "y" or "z"#2018-09-2621:26pvillegas12Right, if m/origin for example is restricted to a given string matching a regex it will destroy genā€™s ability to generate example data#2018-09-2621:27pvillegas12not sure how with-gen would work with that compound spec though#2018-09-2621:27pvillegas12(thatā€™s how I make sure gen continues to work)#2018-09-2621:39favilas/keys+ works like s/keys, except you can add a :conf map from spec->override#2018-09-2621:39favilathe spec is checked all the time#2018-09-2621:40favilabut the override is also checked, and is used for gen#2018-09-2621:44favilathis is really just contravariance/covariance at the end of the day. The trouble is spec doesn't have a way to structurally use the same map key and spec that key two different ways#2018-09-2613:13favilaI wrote a hacky keys+ macro to try and do this. It allows an inline redef of a keyā€™s spec+generator, but it will still conform both the ā€œnaturalā€ and the redef-ed spec of the key#2018-09-2617:03justinleeIn my reagent app on cljs, I use a spec validator on my appā€™s state atom. This works well, but is unfortunately a bit slow in some cases. Is there anything clever one can do to do validations in this way but skip parts of the state tree that havenā€™t changed? The answer might be just to use more than one atom.#2018-09-2617:07noisesmiththe validation function receives the two states as args, so at least hypothetically you could check only changed subtrees using cheap identity checks#2018-09-2617:08noisesmithI'd imagine a classic recursive tree walk, either short-circuiting if identity is true, or checking if identity was false, for each branch new/old#2018-09-2617:09noisesmiththen again you'd have to also walk subtrees of the spec to really make it work...#2018-09-2617:09noisesmithsounds messy#2018-09-2617:11dadairIs the state atom flat or deep? If its flat you could have a spec for each of the main subtrees, and only validate the main subtrees that have changed?#2018-09-2617:12noisesmiththat is probably the best plan#2018-09-2617:16justinleeyea thatā€™s probably the right approach. actually i already have specs for each of the main subtrees#2018-09-2617:19noisesmithand I bet using identical? instead of = performs better for checking each subtree for changes, but that should be straightforward to swap out and test#2018-09-2617:29justinleethat works beautifully. the obvious thing i didnā€™t realize is that the validator of course is called before the atom is changed so you can just compare it against the old atom#2018-09-2617:32pvillegas12I would like to be able to express that a restriction on a composed spec. If spec1 has value :a within a set of alternatives, spec2 cannot have that value (both of spec1 and spec2 are keys of another specā€™d map)#2018-09-2617:32pvillegas12Is there some way to achieve this?#2018-09-2617:36noisesmith@lee.justin.m one of the args to the validator function is the old state, one is the new state#2018-09-2617:40mishaassuming the state tree is a map, and if there are no cross-keys validations (if k1 is this, k2 should be that), you can top level diff new and old values, and just test it against empty (s/keys) @lee.justin.m#2018-09-2617:41mishaeg: old {:a 1 :b 2}, new {:a 1 :b 3 :c 4}, diff {:b 3 :c 4}, (s/valid? (s/keys) diff)#2018-09-2617:42mishamight save you some milliseconds, if state is huge, but diff is just few keys at a time#2018-09-2617:44justinlee@noisesmith I was using https://clojuredocs.org/clojure.core/set-validator! which says `validator-fn must be nil or a side-effect-free fn of one argument, which will be passed the intended new state on any state change`#2018-09-2617:44justinleeis there something else I could be using?#2018-09-2617:44justinlee@misha yea there arenā€™t that many top level keys and most of them wonā€™t change, so I think I can just test them with identical?#2018-09-2617:45mishathat's the validator, yes. I don't know about other ones either. I think @noisesmith meant usual ref-callback with [key ref old new] signature#2018-09-2617:47dadairthatā€™s using add-watch#2018-09-2617:48noisesmith@dadair yeah that's the one I was thinking of#2018-09-2617:48dadairit mostly depends on how you want to handle the spec failure#2018-09-2617:48noisesmithI got the two functionalities confused#2018-09-2617:55justinleethe only thing Iā€™d really like to do is have the validator throw an error synchronously so that i get a stack trace from the offending function, though this is straying from the channel topic#2018-09-2619:36justinleeare specs queryable once theyā€™ve been made? itā€™d be cool to ask for the spec corresponding to a key in a map dynamically#2018-09-2619:37dadairhttps://clojure.github.io/spec.alpha/clojure.spec.alpha-api.html#clojure.spec.alpha/get-spec#2018-09-2621:05Alex Miller (Clojure team)then s/form#2018-09-2621:21favila#2018-09-2622:23ChrisIs there a known issue with s/or and not working with s/keys?#2018-09-2622:25ChrisFor example, this works fine:
(s/conform (s/or :success (s/keys :req [::email/email-address])
                 :not-found nil?) {::email/email-address "
but if I have additional keys in the value, i get invalid:
(s/conform (s/or :success (s/keys :req [::email/email-address])
                 :not-found nil?) {::email/email-address "
#2018-09-2623:18seancorfield@chris547 If you call s/explain (instead of s/conform) in that second case, what does it say? I suspect you have a spec for ::email/creation-timestamp and your string is not valid for that.#2018-09-2623:18ChrisIt tells me that creation-timestamp doesn't match the :not-found predicate#2018-09-2623:19Chris
In: [:io.cloudrepo.signup.email/creation-timestamp] val: "2018-09-26T21:47:57.304Z" fails spec: :io.cloudrepo.signup.email/creation-timestamp at: [:success :io.cloudrepo.signup.email/creation-timestamp] predicate: :clojure.spec.alpha/unknown
val: #:io.cloudrepo.signup.email{:email-address "
#2018-09-2623:20ChrisIt's super weird#2018-09-2623:20Chrisbecause all it should care about is if ::email/email-address conforms, since that's the only key mentioned in the :success condition of the s/or#2018-09-2700:01seancorfieldYou're only reading the last part of the failure -- the first part says "2018-09-26T21:47:57.304Z" fails spec: :io.cloudrepo.signup.email/creation-timestamp#2018-09-2700:01seancorfieldSo you have an ::email/creation-timestamp spec and that string fails to conform to it.#2018-09-2700:02seancorfieldSince that fails, the whole :success part fails and the the whole hashmap fails the :not-found part -- the val in explain is the whole hash map, not just the creation timestamp string.#2018-09-2700:03seancorfields/keys will conform/validate any keys that are provided that have a corresponding spec, not just the listed :req keys.#2018-09-2700:03seancorfield^ @chris547#2018-09-2700:06ChrisAh, that makes sense now#2018-09-2700:06ChrisI didn't even check to see if the timestamp was okay because I didn't care about it#2018-09-2700:06Chrisbut I guess that's good because it will catch where you create bad data at the first possible point#2018-09-2700:07Chrisrather than when you explicitly check, right?#2018-09-2700:12seancorfieldYup. The assumption is that specs should apply wherever those uniquely-name keys appear.#2018-09-2700:12seancorfieldIf you're using unqualified keys, you won't run into that -- since only the :req-un and :opt-un keys will map to specs.#2018-09-2700:15ChrisOkay, thanks Sean - you're always super helpful!#2018-09-2722:02lilactownI have this spec:
(spec/cat :op #{:jdbc/insert}
            :table keyword?
            :row map?)
for a data structure like [:jdbc/insert :foo {:bar "baz"}]. how would I add an optional 4th element at the end?
#2018-09-2723:07seancorfield
:options (s/? ::opt-spec)
#2018-09-2723:08seancorfield(assuming you wanted to optionally pass in an options map of some sort @lilactown)#2018-09-2809:50manutter51Iā€™m trying to run (stest/instrument) in a clojurescript repl, but I canā€™t even get that far:
dev:cljs.user=> (require '[cljs.spec.test.alpha :as stest])
----  Could not Analyze    ----

  No such namespace: clojure.test.check, could not locate clojure/test/check.cljs, clojure/test/check.cljc, or JavaScript source providing "clojure.test.check" in file file:/Users/mark/.m2/repository/org/clojure/clojurescript/1.10.339/clojurescript-1.10.339.jar!/cljs/spec/test/alpha.cljs

----  Analysis Error  ----
#2018-09-2809:51manutter51
:dependencies [[org.clojure/clojure "1.9.0"]
               [org.clojure/clojurescript "1.10.339"]
               [org.clojure/spec.alpha "0.2.176"]
#2018-09-2809:52manutter51Am I missing a step somewhere?#2018-09-2809:55manutter51I just added the explicit dependency on spec.alpha 0.2.176 this morning, but it makes no difference, I get the same error either way.#2018-09-2809:56valeraukoi thought you needed to depend on clojure.test for generating etc#2018-09-2810:23manutter51You ought to be able to load the namespace though, right?#2018-09-2810:24manutter51Iā€™m spinning up a brand new REPL, and the first thing I type is the require, and it wonā€™t load the namespace.#2018-09-2810:28manutter51Hmm, something odd in lein deps :tree too:
[org.clojure/clojure "1.9.0"]
   [org.clojure/core.specs.alpha "0.1.24"]
ā€œspecsā€ plural? Whatā€™s that?
#2018-09-2811:10mgrbyteI think they are the specs that have been added for clojure language itself.#2018-09-2811:22manutter51Ah, that would make sense.#2018-09-2811:23manutter51So is anybody else having this problem? Canā€™t require the cljs.spec.test.alpha ns in a freshly-started REPL?#2018-09-2811:24manutter51It does seem suspicious that itā€™s looking for it specifically inside the clojurescript jar when I have a perfectly good clojure jar on the same classpath.#2018-09-2811:49athosProbably that can be fixed by adding test.check https://github.com/clojure/test.check to your dependencies.#2018-09-2822:51manutter51I didnā€™t get to it on my lunch break, but I just now added the dependency to test.check, and it works great! Thanks a ton! šŸ˜„#2018-09-2812:01manutter51Ok, thanks, I'll try that on my lunch break; got to start day job now.#2018-09-2812:25urzdsHi! I just tried to update to Clojure 1.9 and get weird errors when building my code: :cause Call to clojure.core/refer-clojure did not conform to spec: Sadly this error message is so convoluted that I cannot even figure out which file is causing this. Any idea what to look for?#2018-09-2812:26urzds(We do not use spec ourselves.)#2018-09-2813:01mpenetyou likely have malformed ns declaration(s)#2018-09-2813:01mpenetor one of your deps#2018-09-2813:07favilaUnfortunately the error will say it is coming from the requiring ns instead of the file it is in#2018-09-2813:08favilaThe ns form isnā€™t been evaluated yet (because it has an error) so the current ns ( as shown in the message) is still the requiring one#2018-09-2815:01urzds@mpenet, @favila: So the namespace that requires the broken namespace is the one mentioned in the LOAD FAILURE for <ns> line?#2018-09-2815:01favilathat's been my experience, yes#2018-09-2815:07urzdsThanks.#2018-09-2815:07urzdsAny idea what this spec problem report might be hinting at? https://gist.github.com/urzds/880cd51851a248230861b178c7d6b986#2018-09-2815:08urzdsAlso, in the backtrace it mentions a different file from the namespace it mentions in the LOAD FAILURE for ... line.#2018-09-2815:09urzdsSo my guess would be that the file it mentions in the backtrace actually contains something funny. But I have no idea what I am looking for...#2018-09-2815:14dadairLooks like it is complaining about the ā€œ:as coreā€ part#2018-09-2815:30seancorfieldYeah, the syntax is (:refer-clojure :exclude [...]) -- that :as core part is what is blowing up.#2018-09-2815:31seancorfield@urzds if you post a Gist of your dependencies and/or the full stacktrace, we can probably narrow it down for you.#2018-09-2815:32seancorfieldAlso @urzds check out this page and see if any of your dependencies are listed there https://dev.clojure.org/display/design/Errors+found+with+core+specs#2018-09-2815:35urzds@seancorfield Thanks!#2018-09-2815:38urzdsDifferent question (I am trying to stay on 1.8 and first make sure all my deps are still working): Is it possible to hot patch a function into clojure.core? Lacinia Pedestal is broken on Clojure 1.8, because it uses a function from 1.9: https://github.com/walmartlabs/lacinia-pedestal/issues/80#2018-09-2816:39seancorfield@urzds Is there a pressing reason to stay on 1.8?#2018-09-2816:41seancorfield(I think, yes, you can define your own clojure.core/pos-int? but you'd need to do it somehow before that Lacinia namespace was even loaded so...)#2018-09-2816:46urzds@seancorfield No pressing need. But before I try to upgrade to Clojure 1.9, I want to make sure everything still works after upgrading all my deps to versions that should in theory be capable of supporting 1.9.#2018-09-2816:52seancorfieldI suspect you'll be caught between a rock and a hard place there with anything that already uses clojure.spec since such code may assume a number of things present in 1.9 -- although I thought clojure-future-spec was supposed to patch those up as well?#2018-09-2816:56seancorfieldAh, I see... it expects you to (:require [clojure.future :refer :all]) into the spec-using namespace but doesn't add anything to clojure.core directly. So it only helps you use clojure.spec in your own code -- that's not going to help with any other code that already assumes 1.9+.#2018-09-2816:57urzdsAh, but that's nice. I am right now writing my own shim... šŸ˜„#2018-09-2816:58urzdsSo I'll just patch up lacinia-pedestal with clojure.future instead of my own shim.#2018-09-2816:59seancorfieldYeah, I think you'll have to fork lacinia-pedestal for now, patched with clojure.future, and then revert to the official version when you're ready to move to 1.9.#2018-09-2817:00seancorfieldBut, frankly, I would consider that a waste of effort and just go ahead with the 1.9 upgrade and do any necessary work there instead.#2018-09-2817:00urzdsYes, but they even claim to support Clojure 1.8, so it's actually a bugfix. I'll just send them a PR.#2018-09-2817:01seancorfieldBut on 1.8, doesn't it just not use ? That's how most of the libraries seem to handle 1.8 compatibility when they have specs?#2018-09-2817:01urzdsNot this one... šŸ˜ž#2018-09-2817:02seancorfieldSo it already assumes clojure-future-spec is being used? That's kinda odd...#2018-09-2817:02urzdsYes, that's what they suggest to use: https://lacinia.readthedocs.io/en/latest/clojure.html#2018-09-2817:03seancorfieldAh... and then clearly don't test against that setup šŸ™‚#2018-09-2817:05seancorfieldOverall tho', I still suspect you'd have a lot less pain trying Clojure 1.9 directly instead of all this working-around-spec on 1.8 šŸ™‚#2018-09-2817:07seancorfieldProbably a good idea to create an issue (or send them a PR) to add a :1.8 alias to project.clj with Clojure 1.8 and clojure-future-spec so it's easy to test with in the future (and they could auto-test against both 1.8 and 1.9 on CircleCI).#2018-09-2817:22urzds@seancorfield Thanks for all your help! I don't really know what you refer to with the last line, though.#2018-09-2817:26urzdsNow the spec violations vanished. That's weird...#2018-09-2817:28urzdsProbably it was good that I tinkered with the different dependencies for so long. Maybe cheshire or commons-codec or commons-io or org.clojure/tools.reader were causing it, because now I :exclude them from deps that request older versions and my code suddenly compiles...#2018-09-2817:56lilactownI have a map of var-quoted functions that I'm using as a lacinia resolver map. After add fspecs for all of them and turning on instrumentation, I'm getting this error:
com.fasterxml.jackson.core.JsonGenerationException: Cannot JSON encode object of class: class clojure.spec.alpha$regex_spec_impl$reify__2436: 
#2018-09-2913:02bbrinckItā€™s a bit esoteric, but I put together a quick guide on how to understand the :in path in spec https://gist.github.com/bhb/462c3ef97058d669a448aa85e7db5998 . Questions/feedback welcome!#2018-09-3012:10misha@bbrinck how is it even reproducible in case 4? https://gist.github.com/bhb/462c3ef97058d669a448aa85e7db5998#file-ex-txt-L47#2018-09-3012:11misha"it" = "same keys order in map"#2018-09-3013:59bbrinck@misha I may be misunderstanding your question, but the order of a map is consistent, (i.e. for the same map m, (first m) will always return the same thing, since maps are seqable?).#2018-09-3014:41jumblemuddleIs the only difference between using spec/valid? in the pre/post map of a function and spec/fdef that spec/fdef can be specified elsewhere?#2018-09-3014:56tayloryou have to instrument fdefā€™d functions, and that wonā€™t check the function return value like :post does
#2018-09-3014:57taylorpre/post checks will run by default, but instrumentation is opt-in#2018-09-3014:59jumblemuddleI'm noticing that instrument-all is no longer available. Is it common to just instrument a single function while testing it nowadays?#2018-09-3015:00taylorcalling instrument with no args will instrument everything thatā€™s been loaded#2018-09-3015:00jumblemuddleOh, nice šŸ™‚#2018-09-3015:00jumblemuddleThanks#2018-09-3015:01jumblemuddlefdef doesn't verify the output from the :ret parameter though?#2018-09-3015:01jumblemuddleAt least while using instrument?#2018-09-3015:01taylorright, instrumented functions will only assert the fdef :args spec#2018-09-3015:02jumblemuddleHuh, is there anything that actually uses the :ret then?#2018-09-3015:02taylorthereā€™s a lib called Orchestra with another version of instrument that adds return value checking#2018-09-3015:02jumblemuddlešŸ‘#2018-09-3015:02jumblemuddleThanks#2018-09-3015:02tayloryeah, if you check an fdef function it uses the :ret spec (and :fn if it exists)#2018-09-3015:03taylorcheck towards the bottom of the clojure.spec guide for check#2018-09-3015:03jumblemuddleAh, ok#2018-09-3015:03taylorI wrote some examples here too https://blog.taylorwood.io/2017/10/15/fspec.html#2018-09-3016:55andy.fingerhut@bbrinck I do not know your use case exactly, but as long as you are only relying on that for the same identical map m, not two different maps m1 and m2 where (= m1 m2). If you try to rely on (= (first m1) (first m2)) you will often be disappointed. But yes, you can rely on (= (seq m) (seq m)) being true.#2018-09-3017:02bbrinck@andy.fingerhut good point! Yes, this is for the identical map - the ā€œinā€ path returned by spec is only valid for the data you originally check, NOT other data that happens to be equal to the original data. #2018-10-0112:55jumblemuddleDuring development, does it makes sense to use fdef and instrument for all my functions or sprinkle my functions with assert and old enable assertions during development? Perhaps a combination of the two?#2018-10-0112:59jumblemuddleI guess the benefit of fdef is that it can be in a separate namespace; is that recommended?#2018-10-0113:09taylorI think this depends on personal preference and whether youā€™re writing something intended to run on Clojure 1.8. I tend to put fdef next to the function definitions, but keep other specs in spec-specific namespaces#2018-10-0113:10jumblemuddleHmm, ok. Thanks#2018-10-0113:11taylorthis lib keeps everything separate so it can be used w/pre-1.9 https://github.com/clojure/java.jdbc/blob/master/src/main/clojure/clojure/java/jdbc/spec.clj#2018-10-0113:12jumblemuddleAssuming pre-1.9 compatibility isn't a concern, do you think having them together is ideal?#2018-10-0113:12taylorpersonally I like to see the fdef right next to the function itself :man-shrugging:#2018-10-0113:13jumblemuddleThanks :+1::skin-tone-2: #2018-10-0114:50bbrinckYou may already know this ,but itā€™s perfectly fine to put the fdef definition above the function definition. I find this much more readable than putting it below the function.#2018-10-0114:50bbrinck(sometimes people donā€™t realize this is a valid way to organize their specs)#2018-10-0114:53jumblemuddleAh, interesting. :) #2018-10-0113:04jumblemuddleIt seems like that would defeat some of the namespace'd keyword designs of spec though.#2018-10-0115:08mike_ananev@alexmiller hi! We found strange flawing bug for clojure.spec lib in multithreaded environment. We use latest clojure.spec versions ("0.2.176" "0.2.44") for request validation. Every http request we translate to clojure map, then we call (s/valid? spec input-request-map) for input validation. We have 300 000 sample requests (saved in edn file) for testing (constant requests). If we use one thread on JMeter to send request then we always have result true for (s/valid? spec input-request-map) for every 300 000 constant requests. if we use 2 threads or more on JMeter to send requests then sometimes we have false result. If we save false map to a file and then try to validate it (s/valid? spec input-request-map) then we see that it is always true. If we validate twice in multithreaded env (or (s/valid? spec input-request-map) (s/valid? spec input-request-map) ) then it works (but logs tell us that one of s-exps inside or sometimes false). All map is standard clojure maps (immutable). The code (or (s/valid? spec input-request-map) (s/valid? spec input-request-map) ) works for us as workaround but we can't watch on it without crying. It's Clojure, but we met undefined behaviour like in mutable world.#2018-10-0116:34Alex Miller (Clojure team)Sounds unexpected to me. :) Instead of using valid?, could you try using s/explain so youā€™d get a print to the console when it goes wrong? How is JMeter getting the data to send? Is it possible you have two threads reading from the same reader and thus getting something invalid? If you could isolate this down to something reproducible, would be great to have a jira for it.#2018-10-0115:38mike_ananevAny advice?#2018-10-0117:04andy.fingerhut@djtango If you are using Leiningen, the command lein deps :tree can be useful. With tools.deps there is clj -Stree. Maybe you were already aware of those and looking for something more, though.#2018-10-0117:05djtangothanks for this - wasn't really aware of using lein deps or using it in this way#2018-10-0117:05djtangoat this point any and all suggestions are welcome!#2018-10-0117:31seancorfield"lein pedantic" is something to look into as it will flag the conflicts for you. Boot has a similar feature on its show task.#2018-10-0117:31seancorfieldI'm not sure if there's an equivalent yet for clj...#2018-10-0209:50djtangothank you!#2018-10-0119:14jumblemuddleSo, I added (clojure.spec.test.alpha/instrument) to the bottom of my core.cljs file, however I have a namespace alpha which defines a symbol (def global (beta/example ...)). The problem being that core depends on alpha which uses beta/example prior to it being instrumented. Do I have to add the instrument to every namespace to avoid this?#2018-10-0119:32taylorif you want that call to beta/example to get instrumented and you want to keep def global (instead of making it a function or wrapping it in a delay) then yeah I think you'd have to instrument it separately in this case#2018-10-0119:34jumblemuddleMakes sense šŸ‘ In this case, it was easy enough to not define the global, though I could see scenarios where that's not really a solution.#2018-10-0119:34jumblemuddleThanks#2018-10-0119:15jumblemuddleHow do I spec and instrument functions that will be used during initialization?#2018-10-0119:36jaihindhreddyHow would you spec standards based things. Country codes for example: https://en.wikipedia.org/wiki/ISO_3166-1_numeric#2018-10-0119:39jaihindhreddyThis is where I am so far. (s/def :iso/country-code (s/and s/string? #(= 3 (count %))))#2018-10-0119:40taylorI'd probably scrape the values, put them in a set, and use that set as the spec#2018-10-0119:41bbrinckIf you can get a big set of valid sample data and convert it into valid EDN, you could try using https://github.com/stathissideris/spec-provider to infer the spec#2018-10-0119:43jrychter@jaihindh.reddy in these kinds of cases I enumerate all the possibilities in a set and use that set in the spec.#2018-10-0119:43jaihindhreddy@taylor ISO charges money for downloads of data. But it is freely available on their website. Scraping that would be awkward to say the least.#2018-10-0119:44jrychtere.g. (s/and string? countries/all-country-codes) where all-country-codes is a set.#2018-10-0119:44taylorin your example all the ISO codes are on the Wiki page#2018-10-0119:44jrychterI generate from https://github.com/lukes/ISO-3166-Countries-with-Regional-Codes#2018-10-0119:44jrychterā€¦which also makes it somewhat maintainable in the longer term.#2018-10-0119:45taylorare you concerned with keeping the list fresh as time passes?#2018-10-0119:45jaihindhreddyIdeally I would love to do something like (into #{} (slurp url-for-all-country-codes))#2018-10-0119:45tayloryeah, that's a separate problem from clojure.spec#2018-10-0119:45jaihindhreddyCountry codes should be relatively stable. Dont mind manually updating it#2018-10-0120:00jrychterWell, manually is fine, but I would plan on doing it every once in a while. The list of countries is actually surprisingly unstable.#2018-10-0121:55jaihindhreddyCan I spec protocols and multimethods?#2018-10-0122:10seancorfieldSee https://dev.clojure.org/jira/browse/CLJ-2109#2018-10-0122:12seancorfieldI think you're OK with multimethods tho' (I saw a CLJS ticket that indicated s/instrument was fixed to work with multimethods, so I assume it works in CLJ too).#2018-10-0122:13jaihindhreddyOn that note, when should I use one or the other?#2018-10-0122:14jaihindhreddyCurrently I think protocols should be the default, and multimethod if you think you need the arbitrary dispatch.#2018-10-0122:22seancorfield@jaihindh.reddy I think this is still a good set of guidelines https://cemerick.com/2011/07/05/flowchart-for-choosing-the-right-clojure-type-definition-form/#2018-10-0313:50fabraoHello all, can I ask newbe question here? šŸ™‚ IĀ“m trying to figure out how to use spec in my projects.#2018-10-0313:51fabraoIs that in repl development process?#2018-10-0313:53fabraoLike I saw in this article https://blog.taylorwood.io/2017/10/15/fspec.html , so defn -> s/fdef -> stest/instrument -> stest/check ?#2018-10-0313:54fabraouse as replacement of unit test?#2018-10-0314:02taylorIMO they should supplement/augment tests, but not totally replace them#2018-10-0313:56fabraohow do you use in your workflow daily?#2018-10-0313:56taylorhey @fabrao, an example of how you might use it: have some defns you want to spec, write some specs/`fdef`s for those, then in a test namespace you might use check as part of a test#2018-10-0313:57taylorand the specs/`fdef`s can be in the same file as the functions or separate, up to you#2018-10-0313:57fabrao@taylor by the way, was you that wrote the artice?#2018-10-0313:59tayloryes, thanks for reading šŸ™‚#2018-10-0314:01fabraoIt was very clear about the propose of spec#2018-10-0314:01fabraoBut you use it in your testing workflow?#2018-10-0314:01tayloryes, I tend to use instrument more than check#2018-10-0314:02fabraobut, have you already keep it on in production scenario?#2018-10-0314:03fabraofor example to validade an csv file?#2018-10-0314:03taylorno, instrument has a performance penalty, and anywhere I need to use spec "all the time" then I use the explicit conform/`valid?` kind of functions#2018-10-0314:04taylors/explain-data etc.#2018-10-0314:04fabraohow do you use it in Exception scenarios?#2018-10-0314:05taylorlike if I wanted to throw an exception because of invalid input? Probably use ex-info and maybe put some of the s/explain-data into the info map#2018-10-0314:05tayloryou should also look into the expound library for spec#2018-10-0314:06taylorthat can help beautify spec's description of invalid inputs#2018-10-0314:07fabraoI got it, but you mix real code with spec codes?#2018-10-0314:07taylorin some cases yeah I might call s/conform or s/explain-data or s/assert from my "normal" code#2018-10-0314:08taylorfor always-on types of checks (asserts aside)#2018-10-0314:08fabraoin case of using valid?, you use it inside your functions?#2018-10-0314:09taylorprobably not as often as the others, because I'm usually also interested in how/why the input was invalid#2018-10-0314:10fabraoYes, I donĀ“t want to polute my code with spec[ing] stuffs#2018-10-0314:10fabraoI want to be non intrusive checking#2018-10-0314:11tayloryeah, overall I don't have many places in my projects where I'm explicitly interacting with spec in my app code/logic#2018-10-0314:12fabraoyes, thank you about your help, I was afraid beeing specing everthing šŸ™‚#2018-10-0314:13taylorno problem, one great thing about spec is you can use it as little as you like šŸ™‚#2018-10-0314:15fabraoYes, I saw many videos about it, but only shows something using valid? and nothing about the use like in your article#2018-10-0314:19fabraodo you think is it too intrusive doing this? https://gist.github.com/borkdude/8d69f326b3d363a0b34e7b949b039992#2018-10-0314:20fabraoin this way, itĀ“s better using typed-clojure than that šŸ™‚#2018-10-0314:23taylorI think that's fine (and it's cool that it's even possible) if that's your workflow, although personally I would probably just use regular defn and fdefs#2018-10-0314:24taylorit's really up to how you want to use spec and structure your code :man-shrugging: try some different approaches and see what you like#2018-10-0314:31fabraoYes, I understood, thanks for your time. IĀ“d appreciate all the subjects of your blog, all subject that I sometimes worked with#2018-10-0314:33fabraoIĀ“m using Instaparse in production already#2018-10-0314:33fabraothat you can build your own language#2018-10-0314:34taylorI've used Instaparse in a production project too, it's great#2018-10-0313:58tayloryou might choose to enable instrumentation for particular tests by calling instrument, or maybe in a dev profile so everything is instrumented at dev time#2018-10-0317:16seancorfield@fabrao The other thing to bear in mind with spec is that there are two related but separate types of spec: data specs and function specs. At work, we rely heavily on the former in production code, where we call s/conform (mostly) and s/valid? on those specs and some data. We use function specs to support testing, either with st/instrument while we're developing or as part of a "unit test", or with st/check mostly while we're developing but also in small, limited tests (generative testing can take a while so it's not really designed as part of your short, fast "unit test" cycle).#2018-10-0317:17seancorfieldWe've been using spec in production pretty much since the first alpha dropped, back in the early 1.9 days. We love it!#2018-10-0317:18taylorthere's a Clojure 1.8 backport of clojure.spec I've used in production haha#2018-10-0317:34seancorfieldThat appeared long after we were already on 1.9 and using spec in production tho' šŸ™‚#2018-10-0317:34seancorfieldWe're currently on 1.10 Alpha 8 in production.#2018-10-0317:26fabrao@seancorfield I saw that we can use s/conform to contruct maps from data in more easy way#2018-10-0317:28fabraoin that video explain lots of stuffs#2018-10-0317:28fabraohttps://www.youtube.com/watch?v=TD7VGSSZ3ng#2018-10-0317:32seancorfield@fabrao Be careful about coercions and conformers in spec tho' -- consider those "very advanced" usage until you're comfortable with the rest of it in your workflow.#2018-10-0317:33seancorfieldIf you bake a coercion into your spec, you are "forcing" that coercion on all clients of the spec -- and it may have consequences for generators / generative testing.#2018-10-0317:33seancorfieldBut, yeah, overall spec is awesome!#2018-10-0322:07ScotWhere's my error? What am I doing wrong?#2018-10-0322:08ScotI'm sure I am doing something stupid, but I can't find any answers#2018-10-0322:08noisesmithto verify an fdef I think you need to use instrument to explicitly turn on verification#2018-10-0322:09noisesmithalso beware of instrumenting functions that take function arguments - the spec is checked by passing various generated data to the function that was passed in#2018-10-0322:10ScotSo the only way to get runtime verification is via asserts?#2018-10-0322:10noisesmithinstrument turns on the validation of the function's spec, it's something you have to explicitly ask for#2018-10-0322:11ScotThanks#2018-10-0322:13noisesmithif you want to ensure that a specific arg is always checked, you can use s/valid? as a predicate, or s/conform if you want an error for non-matching data#2018-10-0408:10misha@scot-brown 1) https://clojure.github.io/spec.alpha/clojure.spec.alpha-api.html#clojure.spec.alpha/fdef
Note that :fn specs require the presence of :args and :ret specs to
conform values, and so :fn specs will be ignored if :args or :ret
are missing.
2) https://clojure.github.io/spec.alpha/clojure.spec.test.alpha-api.html#clojure.spec.test.alpha/instrument does not validate against :ret and :fn fdef's spec-components, see: https://groups.google.com/d/msg/clojure/JU6EmjtbRiQ/uND70kAFBgAJ The official way to test against :ret and :fn of s/fdef - is test.check. Folks who accept performance price also use https://github.com/jeaye/orchestra instead of vanilla stest/instrument
#2018-10-0416:48noisesmith@misha thanks for the clarification#2018-10-0416:48noisesmithso instrument only validates the arglist and not the return value#2018-10-0416:52Alex Miller (Clojure team)Yes- it instruments functions to ensure you are invoking them correctly#2018-10-0416:54trissIf I have a lot of functions all with the same spec how can I share it between them? Iā€™m guessing I can use an fspec with fdef somehow? I really want these specs to show up in the doc strings#2018-10-0416:55Alex Miller (Clojure team)Sure, name the common spec and use the name#2018-10-0416:56Alex Miller (Clojure team)fdef is really a wrapper around fspec then def#2018-10-0416:57trissoh nice. Thanks @alexmiller#2018-10-0416:57Alex Miller (Clojure team)So you can (s/def foo ::a) where ::a is an s/fspec#2018-10-0416:57Alex Miller (Clojure team)There were some bugs around this but I think those are all fixed now#2018-10-0417:48fabraowhat is the right use? [clojure.spec.alpha :as s] or [clojure.spec :as s]. I tried [clojure.spec :as s] but it seems no lib in this path#2018-10-0417:48fabrao?#2018-10-0417:51seancorfieldSpec is still alpha right now.#2018-10-0417:51seancorfield[clojure.spec.alpha :as s]#2018-10-0417:52favilaalways the namespace with "alpha". Clojure 1.9-alpha had a brief period where spec was in clojure core (not a separate lib). That is why you may see non-alpha namespaces sometimes.#2018-10-0417:52seancorfieldAll of the Spec namespaces end in .alpha -- until it stops being alpha.#2018-10-0417:52seancorfield@favila Ah, yeah, that must have been a fairly brief period šŸ™‚#2018-10-0417:53fabraoyes, I saw it https://www.youtube.com/watch?v=TD7VGSSZ3ng video#2018-10-0417:53fabraoThatĀ“s because I was confuse#2018-10-0417:54the2bearsThe great thing about REPL development is you can try these things very quickly and see which one works.#2018-10-0417:54seancorfieldThere's a comment on that video, dated a year ago, that says "Good introduction! But something changed in the meantime: starting from 1.9.0-alpha16 you have to include clojure.spec.alpha" @fabrao#2018-10-0417:54the2bearsThough Lein setup with the project.clj file can slow that down a bit.#2018-10-0417:55seancorfield@the2bears Aye, I always have a REPL open and I try stuff in it all the time.#2018-10-0417:55favilahttps://clojure.org/community/devchangelog clojure 1.9.0-alpha16 is when it split. So from May 24 2016 to april 26, 2017 "clojure.spec" was the correct namespace#2018-10-0417:55fabraoI left using lein for boot because of @seancorfield#2018-10-0417:56seancorfieldLonger than I realized -- thanks @favila for that detective work!#2018-10-0417:56seancorfieldBoot REPL is nice because you can easily add new dependencies without restarting the REPL!#2018-10-0417:56fabraothatĀ“s the main part I changed#2018-10-0417:57fabraoC-c C-e in set-env! and thatĀ“s it#2018-10-0417:57the2bears@seancorfield my problem is I lose track of my REPL's state from time to time šŸ™‚ . But I have 2 or 3 open at any one time, usually, main project then libraries used in it.#2018-10-0417:58fabraoIĀ“ll do cider-connect and start it from command line#2018-10-0417:59fabraoI tried to use Proto-REPL with Atom but I canĀ“t leave Emacs anymore#2018-10-0419:17fabraoIn this case, I got Assert failed. Is there any way to custom error message?
(defn stringer-bell
  "Prints a string and rings bell."
  [s]
  {:pre [(s/valid? (s/nilable string?) s)]}
  (println s "\007"))
#2018-10-0419:19taylormaybe relevant thread here about custom pre/post messages https://groups.google.com/forum/#!msg/clojure/xHrFyDcPS9A/gXiNY6pmAwAJ#2018-10-0419:19taylorone suggestion is to use clojure.test/is in the assertion#2018-10-0419:20taylor(or something similar of your own)#2018-10-0419:20fabraothanks again taylor, as you see I little lost about using it#2018-10-0419:20taylorhaha np!#2018-10-0419:21fabraoI tried
(if (not (s/valid? ::campos campo))
(throw (Exception. (str "Tipo do campo incorreto. VƔlidos = #{" (reduce #(str %1 %2 " ") "" campos) "}"))))
#2018-10-0419:21fabraobut is too much code, donĀ“t you think?#2018-10-0419:22taylorI think that reduce could be replaced with clojure.string/join maybe#2018-10-0419:23tayloralso, if you think you need this type of check a lot, you could easily define a function that takes a spec, an input, and maybe an error message or explain-data-to-string function; and then wherever you call it should be pretty terse?#2018-10-0419:24fabraoIĀ“m thinking use https://github.com/bhb/expound to be more explicity#2018-10-0419:24tayloralso check out https://github.com/bhb/expound if you haven't already, and I think there's a similar lib that's more geared towards user-facing error messages#2018-10-0419:24fabraošŸ™‚#2018-10-0420:32misha@fabrao offtopic: have a look at https://clojuredocs.org/clojure.core/when-not#2018-10-0420:32mishaand https://clojuredocs.org/clojure.core/ex-info#2018-10-0420:33fabraothanks#2018-10-0420:36mishaand (reduce #(str %1 %2 " ") "" campos) looks like (clojure.string/join " " campos)#2018-10-0420:37mishasuddenly, not too much code opieop#2018-10-0423:38agif you are writing a s/fdef foo but want to place it in a separate ns (not the same where foo is), should you require the ns where the function is in ns where the spec is? But what if you need to use specs say for validation (in the ns where s/fdef is implemented) how do you avoid circular dependency?#2018-10-0423:40agor should you move higher order specs to a separate ns, but keep s/fdef foo where foo is implemented?#2018-10-0423:41noisesmithisn't some of this simplified by the fact that the spec is looked up by a keyword? you can use a namespaced keyword before the namespace it nominally refers to exists#2018-10-0423:42noisesmithby the time the spec is actually checked, you need the ns that defined the spec for that keyword to be loaded, of course#2018-10-0502:04Alex Miller (Clojure team)you donā€™t have to require the namespace#2018-10-0502:05Alex Miller (Clojure team)fdef creates an entry in the spec registry keyed by the namespaced symbol, but you donā€™t need to load that namespace at that point#2018-10-0519:59aisamuWrapping a cat with an and is breaking my spec in interesting ways. What am I missing?
(s/def ::pair
  (s/cat :first (s/nilable number?)
         :second (s/nilable number?)))

(s/conform ::pair [1 2])
;; => {:first 1, :second 2}
(s/conform (s/? ::pair) [1 2])
;; => {:first 1, :second 2}

(s/def ::ordered-pair
  (s/and ::pair
         #(<= (:first %) (:second %))))

(s/conform ::ordered-pair [1 2])
;; => {:first 1, :second 2}
(s/conform (s/? ::ordered-pair) [1 2])
;; => :clojure.spec.alpha/invalid
(s/explain-str (s/? ::ordered-pair) [1 2])
;; => "In: [0] val: 1 fails spec: :apij.unit.specs.models.rate/pair predicate: (cat :first (nilable number?) :second (nilable number?))\n"
#2018-10-0520:02taylorthe s/? isn't "flattening" the regex spec (`s/cat`) because it's wrapped in s/and. I think those last tests would conform [[1 2]]#2018-10-0520:03taylorit should also conform []#2018-10-0520:04taylor(s/? ::ordered-pair) is becoming "match a sequence of zero-or-one ::ordered-pairs"#2018-10-0520:05taylorif you wanted a something-or-nothing spec you could try (s/nilable ::ordered-pair) too but I'm not sure of your use case#2018-10-0520:09aisamuYup, both cases conform correctly! I'm mostly trying to understand the non-flattening, though. Is there an alternative construct to use the ordering function without using s/and?#2018-10-0520:10aisamu(and thanks, @U3DAE8HMG!)#2018-10-0520:19taylorhmm what would multiple pair inputs look like for you? one big sequence like [1 2 3 4] or a sequence of pair seqs like [[1 2] [3 4]]#2018-10-0520:28Alex Miller (Clojure team)s/and converts a regex spec to a non-regex. s/& is a regex version#2018-10-0520:28Alex Miller (Clojure team)will still continue operating at the same ā€œlevelā€ of sequence#2018-10-0520:30taylorah yeah, you can just replace s/and with s/& in ::ordered-pair if you want to conform inputs like [1 2 3 4]#2018-10-0520:53aisamuOh,facepalm Not only it makes perfect sense, but it's pretty obvious in retrospect! Thanks @U064X3EF3 and @U3DAE8HMG!#2018-10-0800:49kingcodeWhat is the fastest way to tell whether one vector is a subsequence of another?#2018-10-0812:12roklenarcicProbably the same algo than the one for substring search.#2018-10-0812:19roklenarcicKnuth-Morris-Pratt or Boyer-Moore#2018-10-0800:50kingcodesorry..wrong channel (I was thinking of using specsā€™ coll regex)#2018-10-0805:18tianshuHi, how can I know if there is a spec on a given keyword or not? seems s/spec? only detect spec object.#2018-10-0805:31seancorfield(s/get-spec ::k) will return nil if no spec is registered (else will return the spec itself).#2018-10-0806:00tianshu@seancorfield thanks!#2018-10-0916:41jrychterI wish there was a book I could buy, titled "Data Modeling with clojure.spec".#2018-10-0916:44jrychterHere's an example of a practical question. Should I do (a) or (b)?
;; A:
(ns testing.spec
  (:require [testing.id :as id]))

(s/def ::id ::id/id)
(s/def ::data-structure (s/keys :req [::id]))

;; B:
(ns testing.spec
  (:require [testing.id :as id]))

(s/def ::data-structure (s/keys :req [::id/id]))
#2018-10-0916:45Alex Miller (Clojure team)You should do whichever one matches your data#2018-10-0916:46jrychterHence my wish for the book šŸ™‚#2018-10-0918:05mattlypersonally I would define the spec for the id in the same ns as the id#2018-10-0918:06mattlyunless the testing.spec ns is the only place you plan to refer to it#2018-10-0918:19jrychter:: is a spec for an UUID in a certain form, this gets used all over the place. Question is, do I alias it locally using ::id? In other words, who "owns" the ::id definition?#2018-10-0918:23jrychterAnd Alex's Zen Master answer is actually spot on. My current thinking is that I should use local ::id, since this is really a property of the data that I'm defining locally, and only coincidentally an UUID whose properties are defined elsewhere.#2018-10-0918:23Alex Miller (Clojure team)I think about it as two levels - itā€™s useful to have specs that capture common type/shapes/formats domain things - phone numbers, invoice numbers, etc. then there are entities that collect attributes and those may have their own names. Like you have a :foo.domain/phone and then you have :foo.company/phone#2018-10-0918:23Alex Miller (Clojure team)(and the latter would just be an alias for the former)#2018-10-0918:24Alex Miller (Clojure team)that adds weight (and there are some bugs around overriding generators for aliased specs) so it may or may not be worth it#2018-10-0918:24Alex Miller (Clojure team)phone number is probably something with extra constraints, but if you have something adequately represented as int? then I would not pull it out as a domain type as that doesnā€™t seem worth it#2018-10-0918:25Alex Miller (Clojure team)some day weā€™ll consider doing a 2nd edition of Clojure Applied and think about some of this stuff :)#2018-10-1013:57orestisIā€™d love a 2nd edition of Clojure Applied ā€” or perhaps a 2nd volume? šŸ˜„#2018-10-1014:02Alex Miller (Clojure team)based on my time availability, I donā€™t think itā€™s happening anytime soon#2018-10-1014:04orestisUnderstandable, of course. Thanks for the first edition in any case!#2018-10-0918:25jrychterRight. This is similar to what I've been thinking, too.#2018-10-0918:28jrychterA book would be very welcome. I would really like to learn about how people approach modeling data, preferably from people who actually build real systems. Spec seems very flexible, but I found that if you deviate from a certain path, you'll be swimming against the current (for example, if you insist on using unqualified keys).#2018-10-0918:30jrychterBTW, I solved my coercion problems for the time being by judiciously removing every s/conform from my code and writing a tiny piece of code that maintains registries of coercions (multiple registries, as it turns out: one for converting to the domain model, and one for converting to db form). This seems to work very well and apart from limitations of unqualified keys in specs solves the problem for me.#2018-10-1016:02ScottStackelhouseHi all, I donā€™t slack much. I have a case where I model data as a map, some keys required, some not. I already use one key as a ā€œtypeā€ or ā€œkindā€ indicator. The problem I have is I have a set of keys where all are optional, but at least one of them must be present. #2018-10-1016:02ScottStackelhouseI have not had luck figuring out how to spec that#2018-10-1016:02ScottStackelhouseAny thoughts or links to point me at?#2018-10-1016:04Alex Miller (Clojure team)use s/and to combine a predicate that checks that condition#2018-10-1016:05ScottStackelhouseMostly the problems I run ino with specā€™ing like that is it seems to imply/expect more hierarchy than o have#2018-10-1016:06ScottStackelhouseIā€™m sure itā€™s user error, but examples are hard to come by#2018-10-1016:06piotr-yuxuanSorry for disrupting your chat. Iā€™m wondering how to programmatically register a spec.
(defmacro draft-make-spec
  [spec-name spec-pred]
  (eval `(list 'spec/def ~spec-name ~spec-pred)))

(def spec-name ::g)
(def spec-pred int?)
(draft-make-spec spec-name spec-pred)
(spec/valid? ::g 4)
;; works correctly
;; => true

(let [spec-name ::h]
  (draft-make-spec spec-name int?)
  (spec/valid? ::h 4))
;; => throws in java.lang.InstantiationException
#2018-10-1016:07Alex Miller (Clojure team)@scottstack (s/and (s/keys :opt [::a ::b ::c]) #(some #{::a ::b ::c} (keys %)))#2018-10-1016:09Alex Miller (Clojure team)@piotr2b youā€™re doing too much there I think#2018-10-1016:09ScottStackelhouseThatā€™s interesting. I tried something similar with s/alt but didnā€™t quite get there. #2018-10-1016:10ScottStackelhouseThanks @alexmiller , Iā€™ll see where I get with that. It at least tells me Iā€™m heading in the right direction. #2018-10-1016:14piotr-yuxuan@alexmiller, at first I thought it would as easy as this:
(defmacro draft-make-spec
  [spec-name spec-pred]
  `(spec/def ~spec-name ~spec-pred))
but it doesnā€™t work:
(let [spec-name ::h]
  (draft-make-spec spec-name int?)
  (spec/valid? ::h 4))
;; => CompilerException java.lang.Exception: Unable to resolve spec: ::h
šŸ˜ž any tiny help from you would be greatly appreciated! šŸ’Ŗ
#2018-10-1016:16piotr-yuxuanI was wondering how I could write a small library to generate specs from avro schema, so I need to call spec/def in a let and I only know the spec name at runtime.#2018-10-1016:30ScottStackelhouse@piotr2b have you looked at the macro expansions of your macro and of s/def? I did a quick try and it didnā€™t come out how I expected. #2018-10-1016:31Alex Miller (Clojure team)Iā€™ve done some stuff like this elsewhere but canā€™t put my finger on it right now and I need to log off, sorry#2018-10-1016:36ScottStackelhouseI put the macro expansion of the s/def form into the let in place of the macro call, ie (s/def-impl ā€˜spec-name ā€˜int? int?) and it fails an assertion that ā€œkā€ is a keyword... #2018-10-1016:37ScottStackelhouseNomenclature fails me but it is like spec-name in your macro needs a double deref #2018-10-1016:53ScottStackelhouse@piotr2b If you take the expansion of s/def, and use that in your draft-make-spec macro, it works#2018-10-1016:53piotr-yuxuanwow, wait!#2018-10-1016:53piotr-yuxuanCould you show me that? šŸ˜„#2018-10-1016:54piotr-yuxuanItā€™s funny because weā€™ve swapped problems#2018-10-1016:54piotr-yuxuanI was working on solving yours#2018-10-1016:54ScottStackelhouseYeah, on my phone tho#2018-10-1016:54ScottStackelhouseHeh#2018-10-1016:54piotr-yuxuanHere is what Iā€™ve got so far:
(spec/def :entity/kind #{:entity/car :entity/person})
(spec/def :person/name string?)
(spec/def :person/age pos-int?)
(spec/def :car/model-name string?)
(spec/def :car/age pos-int?)

(spec/def ::scottstack-spec
  (and (spec/keys :opt-un [:entity/kind
                           :person/name
                           :person/age
                           :car/model-name
                           :car/age])
       #(condp = (:entity/kind %)
          :entity/car (every? (set (keys %)) #{:car/model-name :car/age})
          :entity/person (every? (set (keys %)) #{:person/name :person/age})
          false)))

(spec/valid? ::scottstack-spec {:entity/kind :entity/car
                                :person/name "scottstack"
                                :person/age 21})
;; => false

(spec/valid? ::scottstack-spec {:entity/kind :entity/person
                                :person/name "scottstack"
                                :person/age 21})
;; => true
#2018-10-1016:54piotr-yuxuanDoes it suit your need?#2018-10-1016:55ScottStackelhouseI will have to check#2018-10-1016:55piotr-yuxuanBasically Iā€™m a bit new with macros, so if you could write down the code in addition to your previous explanations, that would be wonderful šŸ™‚#2018-10-1016:55ScottStackelhouseLet me switch to something with a keyboard #2018-10-1016:55piotr-yuxuan^^#2018-10-1016:58piotr-yuxuan@alexmiller how sad! if you could find it back it would be awesome. Anyway, thanks for everything you do for this amazing language šŸ™‚#2018-10-1016:59piotr-yuxuan@scottstack, I fixed a typo, itā€™s better with :opt-un#2018-10-1017:00piotr-yuxuanAnd the last test case:
(spec/valid? ::scottstack-spec {:entity/kind :entity/person
                                :person/name "scottstack"})
;; => false
#2018-10-1017:02ScottStackelhouse(defmacro draft-make-spec [spec-name spec-pred] `(s/def-impl spec-name ā€˜spec-pred ~spec-pred)) (let [sname ::something] (draft-make-spec sname int?) (s/valid? ::something 4) #2018-10-1017:03ScottStackelhouseI am very good with macros either. I depend heavily on cider's macroexpansion #2018-10-1017:05ScottStackelhouseS/def would expand to (s/def-impl 'sname 'int? int?) #2018-10-1017:06ScottStackelhouseAnd that would fail an assertion that the first arg be a keyword#2018-10-1017:06ScottStackelhouseOh#2018-10-1017:06ScottStackelhouseI see I goofed up, maybe it still doesn't work#2018-10-1017:07ScottStackelhouseNevermind, I just confused myself momentarily... I think it is ok#2018-10-1019:00firstclassfuncAfternoon, is anyone aware of any projects to transform JSON Schema into Clojure Spec?#2018-10-1019:22lilactownI don't know of any projects for doing JSON Schema => clojure.spec#2018-10-1019:23lilactownspec-tools can go the other way: clojure.spec => JSON Schema#2018-10-1019:24firstclassfunc@lilactown Yea tks.. Do people think that would be valuable? There are lots of standards defined in terms of JSON Schema and would be beneficial to auto-generate specs so they can be consumes without the tedious nature of crafting data structures from scratch#2018-10-1019:53lilactownI think it could be pretty useful to help bootstrap some of the commonly needed specs, yeah šŸ˜„#2018-10-1021:57devnThis may be a test.check question more than a spec question, but here goes: I have a map named foo: {:a 1 :b nil :c 32} If a is present, then b should be nil, and c should be a pos-int?. {:a nil :b true :c nil} When a is nil, c must also be nil, and b must hold a value. I want to be able to generate examples of both types of maps.#2018-10-1021:57devnPerhaps I should be tagging them as :type :a and :type :b and using a multi-spec?#2018-10-1022:00gfredericksAs a pure t.c question it's easy - use gen/one-of; not sure about making that more spectomatic#2018-10-1022:00gfredericksmulti-spec sounds plausible but I am not a dentist#2018-10-1022:07devnThe way I explained it above is kind of crappy:
(s/def :my/a #{1 2 nil})
(s/def :my/b (s/or :has-b pos-int? :no-b nil?))
(s/def ::thing (s/keys :req-un [:my/a :my/b])
example:
(let [a-thing {:a 1 :b nil}
  (when (and (:a a-thing) (not (:b a-thing))) (println "it's a foo")
  (when (and (not (:a a-thing)) (:b a thing)) (println "it's a bar")))
I think I need some such-that magic maybe?
#2018-10-1022:10devnSo what I want is if I then ran (gen/sample (s/gen ::thing)), I'd never get back records which have both :a and :b populated, only one or the other.#2018-10-1022:12gfredericksHave you tried writing a map spec for each case and using s/or?#2018-10-1022:13devnno but that makes perfect sense#2018-10-1022:13devnthanks @gfredericks#2018-10-1022:19devnjust to be clear, what I think you're suggesting is this:
(s/def :myA/a #{1 2})
(s/def :myA/b nil?)

(s/def :myB/a nil?)
(s/def :myB/b pos-int?)

(s/def ::thing-a (s/keys :req-un [:myA/a :myA/b])
(s/def ::thing-b (s/keys :req-un [:myB/a :myB/b])
(s/def ::either-thing (s/or ::thing-a ::thing-b))
#2018-10-1022:25devnfollow on question is then perhaps, how to tune the distribution of thing-a and thing-b's generated#2018-10-1022:28bbrinckhavenā€™t tried it, but my guess in a custom generator (See s/with-gen) + https://clojure.github.io/test.check/clojure.test.check.generators.html#var-frequency#2018-10-1022:28devn
(gen/sample (gen/frequency [[2 (s/gen ::thing-a)]
                                      [2 (s/gen ::thing-b)]])
                      10)
perhaps?
#2018-10-1022:28bbrinckha#2018-10-1022:28bbrinckyeah#2018-10-1022:29devnthough you may be on to something with with-gen#2018-10-1022:30devnit would be nice to attach the frequency to the spec#2018-10-1022:30bbrinckuntested, but then you can do something like (s/def ::either-thing (s/with-gen (s/or ::thing-a ::thing-b) #(gen/frequency [[2 (s/gen ::thing-a)] [2 (s/gen ::thing-b)]]) ))#2018-10-1022:31devnthat doesn't seem to do it#2018-10-1022:32bbrinckhm, probably a bug in my code, but in general with-gen should override the default generator. remember the function needs to return a generator#2018-10-1022:33bbrinckIOW, the second arg canā€™t be a generator#2018-10-1022:34devnyes, which is indeed what your code does there#2018-10-1022:34devnmaybe gary's previous suggestion to use one-of is worth trying here#2018-10-1022:35bbrinckwhat doesnā€™t work with frequencies?#2018-10-1022:36devni tried changing the frequency from 2 to 0 for one of them, and it doesn't vary the samples#2018-10-1022:36bbrinckah#2018-10-1022:36devnthey all are ::thing-a-like things#2018-10-1022:36devnwhereas the thing I posted above does work#2018-10-1022:37bbrinckwell, if you put 0 for thing-b, then everything should be thing-a, yes?#2018-10-1022:37devnyes, but i tried it both ways, and the same result#2018-10-1022:37bbrinckah#2018-10-1022:39devntested again to make sure im not crazy#2018-10-1022:39devnExceptionInfo Couldn't satisfy such-that predicate after 100 tries.#2018-10-1022:40devnis what I wind up with when I try to make it return only one of the types, so perhaps something else is amiss here#2018-10-1022:44bbrinckhmmm, yes, iā€™m trying a simpler example and not seeing what i expect, so Iā€™m clearly missing something#2018-10-1022:46devn@bbrinck here's a thing: if i switch the order of the s/or, it works in one direction, but not the other
#2018-10-1022:49devnbah, ok, im an idiot#2018-10-1022:49devn(s/or ::thing-a ::thing-b) != (s/or :a ::thing-a :b ::thing-b)#2018-10-1022:49bbrinckhaha wow I missed that too#2018-10-1022:50bbrinckjust trying to figure out why my simple example wasnā€™t even conforming#2018-10-1022:51bbrinckgood catch#2018-10-1022:54bbrinckok, now with-gen + frequency is working again for my simple example.
(s/def ::name string?)
(s/def ::age int?)
(s/def ::either (s/with-gen (s/or :name ::name :age ::age) #(gen/frequency [[4 (s/gen ::name)] [1 (s/gen ::age)]])))
(s/exercise ::either)
#2018-10-1022:54bbrinckis your example working now?#2018-10-1022:54devnyeah, works good#2018-10-1022:54devnchef kisses fingers#2018-10-1022:55devnthanks for your help @bbrinck#2018-10-1022:55bbrincknp#2018-10-1022:56devn(my/or :name ::name 1 :age ::age 2) or somesuch#2018-10-1022:57bbrinckmaybe not easy, but easier if you define the syntax with a regexp spec and conform šŸ™‚#2018-10-1023:06devnok i spent 10min on it and the sugar ain't worth it šŸ˜„#2018-10-1023:18devn
(defmacro myor [& key-pred-freq-forms]
  `(s/with-gen (s/or 
or something, but :man-shrugging:
#2018-10-1116:35domkmHas anyone written a generator for keys that always chooses to generate optional entries until *recursion-limit* is reached?#2018-10-1118:09domkmIt seems like this may not be possible currently because whenever a generator-creating function is allowed (`with-gen`, (keys :gen (fn [] ...))) it is takes 0 args while the default generator-creating functions take 3 args (`[overrides path rmap]`). Why do custom generators not have access to the current length of recursion?#2018-10-1121:36bastiIā€™ve got a pretty specific question regarding clojure spec-toolsā€™s dataspecs - iā€™ve got those two data specs, how do I ā€œorā€ them together?
{:status (s/spec #{200})
   :success boolean?
   :body {:data [{:id string?}]}}
{:status (s/spec #{500})
   :success boolean?
   :body empty?}
#2018-10-1205:11ikitommi@basti there are two options: use ds/or or convert data-specs into normal clojure.specs and use s/or:
(ds/or
  {:200 {:status (s/spec #{200})
         :success boolean?
         :body {:data [{:id string?}]}}
   :500 {:status (s/spec #{500})
         :success boolean?
         :body empty?}})

(s/or :200 (ds/spec
             {:name ::200
              :spec {:status (s/spec #{200})
                     :success boolean?
                     :body {:data [{:id string?}]}}})
      :500 (ds/spec
             {:name ::500
              :spec {:status (s/spec #{500})
                     :success boolean?
                     :body empty?}}))
#2018-10-1215:13bastithat awesome thanks for your help --appreciated!#2018-10-1212:14Charles FourdrignierAfter solving Infix Calculator on 4Clojure (http://www.4clojure.com/problem/135), I would like to write some spec for the args.
(s/def ::infix-args
    (s/cat :a int? 
            :rest (s/* 
                    (s/cat :f #{+ - / *} :b int?))))
Do you see a better way to do that ?
#2018-10-1212:18taylorlooks fine to me#2018-10-1212:19taylorI suppose you could use number? instead of int?#2018-10-1212:22Charles FourdrignierOf course !#2018-10-1410:40vemvWondering if there's a precise definition of :in and :at within the context of clojure.spec failures:
...
             in: [:lines 1]
             at: [:lines]
...
#2018-10-1410:42vemv(this comes from CIDER so perhaps an extra key is being added)#2018-10-1417:05domkmIs there a recommended way of debugging built-in spec generators? I have a large graph spec in which all leaf nodes are clojure.core functions like any?, string?, boolean?, etc. and all collections are either sequential collections or maps with only optional keys, though some keys are recursive. It halts when generating data, eventually erroring (I think it runs out of memory).#2018-10-1418:27misha> though some keys are recursive
(s/def ::foo (s/or :my-non-recursive #{:a}  :my-recursive (s/coll-of ::foo)))

(binding [s/*recursion-limit* 2]
  (s/exercise ::foo))
;=> some data

(binding [s/*recursion-limit* 10]
  (s/exercise ::foo))
;=> OutOfMemoryError GC overhead limit exceeded  clojure.lang.PersistentVector$ChunkedSeq.next (PersistentVector.java:422)
@domkm
#2018-10-1418:27mishadefault is 4#2018-10-1418:32mishahowever, s/*recursion-limit* requires recursive specs to have a non-recursive branch available to be able to stop at the limit. but you will probably get StackOverflow much earlier, if you "explicitly" omit nonrecursive branch:
(s/def ::foo (s/coll-of ::foo))
(binding [s/*recursion-limit* 10]
  (s/exercise ::foo))
;=> CompilerException java.lang.StackOverflowError, compiling:(... .clj:691:1) 
#2018-10-1418:36domkm@misha Hmm, I hadn't considered that it would halt before reaching the s/*recursion-limit*. I assumed that since all keys are optional it would choose a non-recursive option. I'll try messing around with s/*recursion-limit* and see what happens. Thanks#2018-10-1418:37mishayou might get surprised how often limit=4 runs out of memory#2018-10-1418:38mishatry limit=1 and see how deep of a structure s/exercise returns, to at least somewhat calibrate your intuition#2018-10-1421:22alzaHi there, I have a small project to transform one xml format into another (call them fmt-a and fmt-b). I thought Clojure Spec might be useful to define the shape of the data at each stage of the transformation and check the transformation works correctly for expected inputs. Then I found spec-tools and it's transformers, so I thought I'd investigate that. Although I noted that although it supports JSON it doesn't seem to support XML yet. Anyway this was going to be my approach with spec-tools: 1. parse fmt-a-xml from xml file 2. transform fmt-a-xml-data (i.e. parsed {:tag :attributes :content structure}) to fmt-a (i.e. {:fmt-a-key some-val}) 3. transform fmt-a to fmt-b (i.e. {:fmt-b-key some-val}) 4. transform fmt-b to fmt-b-xml-data (i.e. {:tag :attributes :content structure}) 5. format fmt-b-xml-data to xml file Does this sound sensible? I'm not sure how to achieve all of the above with spec-tools, but I was going to start experimenting with step 3, the core data transformation. note: there's no xml schema available for fmt-a or fmt-b, if that matters.#2018-10-1605:59ikitommiOf coercion: tested the two approaches: conforming-based (spec-tools) and form-parsing based (spec-coerce). Both have big issues. This just canā€™t be resolved with the current spec architecture. One CLJ-2251 please šŸ™‚#2018-10-1606:04ikitommiwill most likely merge the two approaches into spec-tools, so the st/decode will work without wrapping of specs for many simple forms, fallbacking to conforming-based approach to support all specs (regex included)#2018-10-1606:04ikitommihttps://github.com/metosin/spec-tools/pull/139#2018-10-1616:40hmaurerHi! What would be the canonical way to force a map to have a :type key with a specific value with Spec? Would you have:
(s/def ::person (s/and map? #(= (:type  %) ::person) (s/keys ...))
#2018-10-1617:57taylorcould you use a spec for the :type key like (s/def ::type #{::person})#2018-10-1616:41hmaureror is there a better/neater way?#2018-10-1617:03jrychterDepends on what you want to do ā€” sometimes you want a multi-spec, where you dispatch on :type.#2018-10-1617:04jrychterAs a side note, I found that prefer to use boolean ::person? keys rather than a general :type. I'm converting my old code as I go.#2018-10-1617:25hmaurer@jrychter oh, why the boolean approach?#2018-10-1617:26hmaurerand under that approach, how would you suggest I handle a hierarchy like this: https://puu.sh/BM1sE/7aaf4c8e7b.png ?#2018-10-1617:26hmaurereach of these represent maps of a certain ā€œtypeā€ with certain properties#2018-10-1617:26jrychterLess complexity, basically. And I don't know, but I suspect that using a single data structure for all of these might not be the best approach.#2018-10-1617:27hmaurerhow so?#2018-10-1617:44Nolanhey everyone, i may be missing something obvious, but is there a concise way to use spec to perform what is essentially a select-keys according to a spec? e.g.
(def composite-thing { ... })
(s/def ::some-component ...)
(s/select-keys ::some-component composite-thing)
;; => map with keys of composite-thing relevant to ::some-component
#2018-10-1617:45Nolanor maybe read the relevant keys from a specā€™s definition at runtime?#2018-10-1617:53taylor
(s/def ::my-map
  (s/keys :req-un [::id]
          :opt-un [::parent-id ::children]))
(->> (s/get-spec ::my-map)
     (s/describe)      ;; get abbrev. form of original spec
     (rest)            ;; drop `keys` symbol
     (apply hash-map)) ;; put kwargs into map
=> {:req-un [:sandbox/id]
    :opt-un [:sandbox/parent-id :sandbox/children]}
#2018-10-1617:59seancorfieldI don't think you need (s/get-spec ::my-map) there, you can just do (s/describe ::my-map) (since s/describe calls s/form).#2018-10-1618:06ikitommiremember to walk over special and and ors.#2018-10-1619:33jrychterThis is something I'd really like to see. I know that Rich said that spec is not about limiting what you can do, but there are legitimate use cases where you want to enforce the set of keys and limit it only to what is in your spec (think for example API endpoints).#2018-10-1619:38seancorfield@jrychter It's easy enough (in most cases) to pull the set of possible keys out of the form for the spec and use that -- treating the spec as the "source of truth" and deriving key sets from it.#2018-10-1621:06trissIā€™m building my first really big interconnected system and I have to admit immutablity is driving me crazy!#2018-10-1621:07trissI just canā€™t used to passing paths around for my ā€˜objectsā€™ all the time.#2018-10-1621:07trissWhat tare the other functional approaches to this?#2018-10-1621:08trissIā€™m really missing OO/imperative systems being happy re: passing around references to objects all the time.#2018-10-1621:09the2bearsI don't miss that at all šŸ˜…#2018-10-1621:10trissI wish I didnā€™t!#2018-10-1621:10trissIā€™m passing around the paths in to a nested map all the timeā€¦#2018-10-1621:14the2bearsNot sure what you're trying to do, sounds like you have a big nested state, and what about the paths you're passing around?#2018-10-1621:15trissIā€™m porting an old AI project to Clojure for nowā€¦. the whole system seems thoroughly object orientated. Objects sending messages to others all over the place.#2018-10-1621:16trissIā€™m pretty sure Core-async would make things a bit easier since I could just pass the chanell around all the time.#2018-10-1621:16trissBut obviously this is a very different solution.#2018-10-1622:04jrychterIt sounds like this approach deserves a good refactoring. Core.async would be just swiping the dust under the carpet. Break things down into smaller functions that do one thing only, and are testable.#2018-10-1622:18the2bearsI was actually asking about your paths, that you're "passing around". I don't know what you mean by that. I can imagine a collection of functions that act upon something flowing through your program. They could be "wired" by a state machine, or a wiring of channels. Why do you think you might need to pass channels around? It's certainly possible, but it might also make sense to push data to a channel that triggers a flow through your functions and something comes out the other end, transformed. That's what I was asking about, with the "what you're trying to do" comment. A description of your data/logic flow in a bit of high-level detail rather than just "porting an old AI project" which doesn't tell much šŸ™‚#2018-10-1622:18the2bearsie. What does the old AI project do?#2018-10-1712:10Ho0manHi, Is there a way to make s/cat to generate vector instead of list (other than using s/with-gen) ?#2018-10-1712:12Alex Miller (Clojure team)No, and that is a known issue#2018-10-1714:05hmaurerIs there any way I can define a generator for a spec like this?
(s/def ::hello #(isa? % :im/base))
#2018-10-1714:05hmaurerI would want it to pick at random from (descendants :im/base)#2018-10-1714:06hmaurer(it has to pick from the descendants at the point when the generator is called, not at the point where the spec/generator is defined)#2018-10-1714:08hmaurernevermind, thatā€™s just a simple application of with-gen!#2018-10-1715:46strickinatoHi - Iā€™m pretty new to clojure, Iā€™m playing around with spec and ran into some trouble: Iā€™m trying to spec a Person that has a last-contacted key and Iā€™m using clj-time, such that I expect the field last-contacted to be a DateTimeZone. Iā€™m running into problems though and Iā€™m not sure how Iā€™m supposed to be using the spec from their library (or generally the best way to proceed. ### Thought 1: Iā€™d like to access the ::date-time spec defined in clj-time.spec since my understanding is specs are ā€œregistered globallyā€ Unfortunately, it appears I canā€™t access it. Trying to (s/exercise ::date-time) yields and ā€œUnable to resolve specā€ error and qualifying it like (s/exercise clj-time.spec/::date-time) gives a parse error Question: How do I use specs defined in libraries? ### Thought 2: If the above did work, how could I even use it with spec/keys. From what I can tell, the name of the spec has to match the name of the key. eg: (s/def ::person (s/keys :req [::name, ::date-time])) Question: Is there syntax for doing something like the following invented code: (s/def ::person (s/keys :req [::name, [:last-contacted ::date-time]])) where a key with a different name gets a spec run on it?#2018-10-1715:55dadair::key is syntactic sugar that expands to :the.current.ns/key#2018-10-1715:56dadairSo if you see the use of ::key in some libraryā€™s namespace ā€œcom.fooā€, you need to use :com.foo/key. You will also need to require com.foo so that the ns is evaluated and the specs are registered#2018-10-1715:48taylorre: #1 you just need a slightly different syntax: :clj-time.spec/date-time, or if you've aliased clj-time.spec e.g. [clj-time.spec :as ts] then you could do ::ts/date-time#2018-10-1715:56strickinatowhoa - very unexpected, but totally works!! Thank you!#2018-10-1715:49taylorre: #2 yes the name of the spec must match the name of the key, however you can alias the original spec to whatever name you like e.g. (s/def ::my-date-time :clj-time.spec/date-time)#2018-10-1715:50taylor(then use ::my-date-time in your keys spec)#2018-10-1715:57strickinatoThank you so much!! All answers I needed!#2018-10-1716:13Nolanwhere would be the best place to expand a bit on a question from yesterday? i have a somewhat longer scenario id like to get some input on, but dont want to spam the channel. is there an active mailing list for spec?#2018-10-1716:25taylorI wouldn't worry about spamming the channel :man-shrugging: There's clojureverse, stack overflow (code review section?), there's a google group/mailing list for Clojure but not sure if there's a spec-specific one#2018-10-1716:33Nolanmuch appreciated. still thinking on it a bit#2018-10-1718:12manutter51This seems like an easy one: how do I specify the :args for this fn when I do s/fdef?
(defn demo [opts & [label]]
  {:opts opts
   :label label})
#2018-10-1718:14manutter51Iā€™ve got this:
(s/def :key/a string?)
(s/def :key/b boolean?)
(s/def :key/c vector?)
(s/def :demo/label string?)

(s/fdef demo
        :args (s/cat :opts (s/keys :req [:key/a]
                                   :opt [:key/b :key/c])
                     :label (s/? :demo/label))
        :ret map?)
#2018-10-1718:14manutter51which seems to pass:
(s/explain (s/get-spec 'demo) '(demo {:key/a "foo"}))
;; ==> Success!
#2018-10-1718:15manutter51but when I instrument it and try to run it, it blows up.
(demo {:key/a "foo"})
;;  #error {:message "Call to #'cljs.user/demo did not conform to spec:\nIn: [0] val: ([:key/a \"foo\"]) fails at: [:args] predicate: (cat :opts (? :demo/opts) :label (? :demo/label)),  Extra input\n:cljs.spec.alpha/spec  #object[cljs.spec.alpha.t_cljs$spec$alpha38378]\n:cljs.spec.alpha/value  {:key/a \"foo\"}\n:cljs.spec.alpha/args  {:key/a \"foo\"}\n:cljs.spec.alpha/failure  :instrument\n", :data {:cljs.spec.alpha/problems [{:path [:args], :reason "Extra input", :pred (cljs.spec.alpha/cat :opts (cljs.spec.alpha/? :demo/opts) :label (cljs.spec.alpha/? :demo/label)), :val ([:key/a "foo"]), :via [], :in [0]}], :cljs.spec.alpha/spec #object[cljs.spec.alpha.t_cljs$spec$alpha38378], :cljs.spec.alpha/value {:key/a "foo"}, :cljs.spec.alpha/args {:key/a "foo"}, :cljs.spec.alpha/failure :instrument}}
;;  Error: Call to #'cljs.user/demo did not conform to spec:
;;  In: [0] val: ([:key/a "foo"]) fails at: [:args] predicate: (cat :opts (? :demo/opts) :label (? :demo/label)),  Extra input
;;  :cljs.spec.alpha/spec  #object[cljs.spec.alpha.t_cljs$spec$alpha38378]
;;  :cljs.spec.alpha/value  {:key/a "foo"}
;;  :cljs.spec.alpha/args  {:key/a "foo"}
;;  :cljs.spec.alpha/failure  :instrument
#2018-10-1718:19taylorhere's another take:
(s/fdef demo
        :args (s/cat :opts (s/keys :req [:key/a]
                                   :opt [:key/b :key/c])
                     :rest (s/? (s/cat :label :demo/label)))
        :ret map?)
#2018-10-1718:30manutter51Ok, I get
app:cljs.user=> (demo {:key/a "foo"})
#error {:message "Call to #'cljs.user/demo did not conform to spec:\nIn: [0] val: [:key/a \"foo\"] fails at: [:args :opts] predicate: map?\n:cljs.spec.alpha/spec  #object[cljs.spec.alpha.t_cljs$spec$alpha38378]\n:cljs.spec.alpha/value  {:key/a \"foo\"}\n:cljs.spec.alpha/args  {:key/a \"foo\"}\n:cljs.spec.alpha/failure  :instrument\n", :data {:cljs.spec.alpha/problems [{:path [:args :opts], :pred map?, :val [:key/a "foo"], :via [], :in [0]}], :cljs.spec.alpha/spec #object[cljs.spec.alpha.t_cljs$spec$alpha38378], :cljs.spec.alpha/value {:key/a "foo"}, :cljs.spec.alpha/args {:key/a "foo"}, :cljs.spec.alpha/failure :instrument}}
Error: Call to #'cljs.user/demo did not conform to spec:
In: [0] val: [:key/a "foo"] fails at: [:args :opts] predicate: map?
:cljs.spec.alpha/spec  #object[cljs.spec.alpha.t_cljs$spec$alpha38378]
:cljs.spec.alpha/value  {:key/a "foo"}
:cljs.spec.alpha/args  {:key/a "foo"}
:cljs.spec.alpha/failure  :instrument
#2018-10-1718:34manutter51wait, for some reason you just made me think of the caveat about nested specs, I wonder if thatā€™s itā€¦#2018-10-1718:54manutter51Nope, no joy in Specville#2018-10-1718:54manutter51I should mention this is CLJS-specific ā€” seems to work ok in CLJ#2018-10-1719:46taylorhmm yeah I tested in CLJ and it works, and I feel like I've seen a JIRA ticket for this issue or similar for CLJS#2018-10-1719:50Nolanfinally got around to posting a longer question on SO. hopefully it makes sense, but let me know how it could be improved for clarity and whatnot, or if im just missing basic concepts here. in any case its been a fun (s/)exercise https://stackoverflow.com/questions/52862471/using-clojure-spec-to-decompose-a-map#2018-10-1721:07Nolan@taylor epic response! really appreciate it. i had played with using a separate s/def for the keys, but i kept going down a path where i was trying to use multi-specs there, too, to no avail. i think ill ultimately use something like the get-spec-keys way. the eval isnā€™t something i had thought about but thats a cool one too. thanks again man!#2018-10-1817:45Nolanpretty amped about how this turned out. ended up dispatching using s/or, and it will work with arbitrarily merged multi-specs (as long as the s/keys are defined by the convention you suggested, which im ok with, since this is sort of a hack anyway). but its pretty sweet. it works pretty much just like select-keys i.e. (select-keys-spec m ::many-level-merged-spec)#2018-10-1816:19Alex Miller (Clojure team)ā€œdoesnā€™t workā€ == ?#2018-10-1816:20lwhortondisregard, being stupid again. šŸ˜ž#2018-10-1816:20lwhortoni was getting compiler errors and it was mostly unrelated (missing spec definition further up the chain)#2018-10-1816:21favila:foo/1 won't read; using a bare number as the name part in a keyword literal is a bad idea#2018-10-1816:21favilathat it works at all is an historical accident, and it wasn't fixed because it was noticed too late#2018-10-1816:22lwhortonis ::1 equally a bad idea?#2018-10-1816:22favilayes, so is :1#2018-10-1816:23favilathink "would this be a readable symbol if I chopped off the initial :s?"#2018-10-1816:24lwhortonwell, isnā€™t {:1 true} a valid map?#2018-10-1816:24favilait is#2018-10-1816:24favilaI'm just talking about literals#2018-10-1816:25lwhortonhm, i have to digest this because itā€™s not making sense. are you saying that ::1 ultimately just becomes my.namespace/1 and thus {1 true}?#2018-10-1816:25favilano#2018-10-1816:27favilahttps://dev.clojure.org/jira/browse/CLJ-1527#2018-10-1816:27favilahttps://dev.clojure.org/jira/browse/CLJ-1286#2018-10-1816:27lwhortonšŸ‘ thanks#2018-10-1816:27favilahttps://dev.clojure.org/jira/browse/CLJS-677#2018-10-1816:29favilaI am saying clojure does not have a strict formal grammar for keywords and symbols, only human descriptions and implementations; using numbers is a greyer, edgier case that risks you falling into implementation-specific behavior#2018-10-1816:30favilaI think this is the heart of this particular reader ambiguity: https://dev.clojure.org/jira/browse/CLJS-677?focusedCommentId=35025&amp;page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-35025#2018-10-1816:30favilabut that's just my opinion#2018-10-1816:30favilano core member has confirmed or clarified to my knowledge#2018-10-1816:31taylor
Clojure 1.9.0
user=> {::1 1}
#:user{:1 1}
user=> {:1 1}
{:1 1}
user=> #:user{:1 1}
#:user{:1 1}
user=> {:user/1 1}
RuntimeException Invalid token: :user/1  clojure.lang.Util.runtimeException (Util.java:221)
1
RuntimeException Unmatched delimiter: }  clojure.lang.Util.runtimeException (Util.java:221)
#2018-10-1816:32favilaoh also https://dev.clojure.org/jira/browse/CLJ-1252#2018-10-1816:32favilaI think this started it all#2018-10-1816:33Alex Miller (Clojure team)well Iā€™m a core member and I wrote that ticket and patch#2018-10-1816:33Alex Miller (Clojure team)the original intent was that keywords follow the same constraints as symbols#2018-10-1816:33Alex Miller (Clojure team)symbols cannot start with a leading digit#2018-10-1816:34Alex Miller (Clojure team)the regex for keywords is subtly incorrect and allows keywords to have a leading digit#2018-10-1816:34Alex Miller (Clojure team)by ā€œfixingā€ that we discovered that people were using such things (at the time java.jdbc made result set keywords like that for example)#2018-10-1816:35Alex Miller (Clojure team)there did not seem to be any good reason to break peopleā€™s existing working code so we rolled that back#2018-10-1816:36Alex Miller (Clojure team)so itā€™s possible to read and use :1 and ::1 and we have no plans to make that stop working#2018-10-1816:36favilait's deliberately not specifically allowed, however?#2018-10-1816:36favilaby specifications#2018-10-1816:36Alex Miller (Clojure team)I would say thatā€™s never been resolved#2018-10-1816:37Alex Miller (Clojure team)CLJ-1527 is the ticket to do so#2018-10-1816:37favilafrom https://clojure.org/reference/reader#_literals#2018-10-1816:38favila> Keywords - Keywords are like symbols, except: > They can and must begin with a colon, e.g. :fred. > They cannot contain '.' or name classes.#2018-10-1816:38Alex Miller (Clojure team)ā€œresolvingā€ means making that reference page and the code in sync#2018-10-1816:38Alex Miller (Clojure team)but the first question is what should be allowed#2018-10-1816:40Alex Miller (Clojure team)there are plenty of other open areas for characters that are allowed in the reader but not explicitly allowed in the reference docs#2018-10-1816:41favilatwo things: 1) "are like symbols" seems to cause confusion about whether what follows includes or does not include the initial colons. that was what my comment in cljs-667. 2) keywords can't have '.'? even in the ns part? (I know putting them in the name part is discouraged because that looks like class names). That's definitely wrong?#2018-10-1816:44Alex Miller (Clojure team)keywords can definitely have dots in the namespace part#2018-10-1816:45favilaany way to fix those docs short of a jira patch?#2018-10-1816:45favila"They cannot contain '.' after the first '/' or name classes" suggested edit#2018-10-1816:46Alex Miller (Clojure team)It says ā€œor in namespace namesā€ ?#2018-10-1816:46favila"keywords are like symbols, except: They cannot contain '.' or name classes." is what it says now#2018-10-1816:47Alex Miller (Clojure team)oh, sorry I was looking at the symbol part#2018-10-1816:48favila"are like symbols" is the handwavy part that I think causes most of the confusion#2018-10-1816:48Alex Miller (Clojure team)Iā€™ll look at an edit to that, but the intent again is to be similar to symbols and allow dots in namespaces#2018-10-1816:49Alex Miller (Clojure team)agreed - the question is whether to describe two things as similar and then indicate differences or to restate, obscuring the differences#2018-10-1816:50Alex Miller (Clojure team)I donā€™t feel that there is any ambiguity in intent that keyword namespaces should allow .#2018-10-1816:50favilaspecifically "Symbols begin with a non-numeric character" combined with "keywords are like symbols" causes one to wonder, "does a keyword begin with : or the char after that"?#2018-10-1816:50Alex Miller (Clojure team)well that is exactly the bug in the regex#2018-10-1816:50Alex Miller (Clojure team)but I would say after the colon#2018-10-1816:51favilathe intent of the spec being: "expand ::, chop off the first :, and you should have a valid symbol literal"#2018-10-1816:51Alex Miller (Clojure team)if you ask for the namespace and name parts of a keyword, you donā€™t see a colon. the colon is syntactical#2018-10-1816:52Alex Miller (Clojure team)I donā€™t think thatā€™s a semantically meaningful operation, so I wouldnā€™t describe it that way#2018-10-1816:53favilawhat I mean is the intuitive sense of a valid keyword was intended to be "does it look like a symbol if the colons that make it a keyword were removed"#2018-10-1816:53favilaor at least thats how I am interpreting what you are saying.#2018-10-1816:53Alex Miller (Clojure team)I understand what you mean. I just wouldnā€™t describe it that way#2018-10-1816:54favilaI think that's a little better than "keywords are like symbols," but still inappropriate for a capital-S "specification"#2018-10-1816:55favilabut I am glad I've discovered the authorial intent behind that phrase#2018-10-1816:55favilathank you#2018-10-1816:59Alex Miller (Clojure team)https://github.com/clojure/clojure-site/commit/5481163d24491ec2ebc5863bc2d0876d36aacc5a#2018-10-1817:01favilaThat :a/b.c reads is an accident of history/implementation?#2018-10-1817:10dominicmThat isn't supposed to work?! #2018-10-1817:18Alex Miller (Clojure team)Iā€™d say thatā€™s not supported in either symbols or keywords. whether it reads is a separate question#2018-10-1817:52borkdudeexpound question: I try to make a custom print function:
(alter-var-root #'s/*explain-out*
                  (expound/custom-printer
                   {:show-valid-values? false
                    :value-str-fn my-value-str
                    :print-specs? true}))
but I get:
Unhandled clojure.lang.ExceptionInfo
   Unknown data:
   

   {:data #function[clojure.spec.test.alpha/spec-checking-fn/fn--3026]}
what up?
#2018-10-1817:58borkdudehmm, no matter what options I give, custom-printer doesnā€™t work#2018-10-1817:59borkdudeah got it#2018-10-1817:59borkdudeconstantly#2018-10-1817:59bbrinckYep šŸ™‚#2018-10-1817:59borkdudethanks šŸ˜‰#2018-10-1818:00bbrinckha, I didnā€™t really do anything šŸ˜‰#2018-10-1818:00bbrinckJust keep in mind that alter-var-root will appear to not work if you are within a clojure REPL (youā€™ll want set! in that case)#2018-10-1818:05borkdudeit appears that expound is working, but I donā€™t see my function being used in the output:
(alter-var-root #'s/*explain-out*
                  (constantly (expound/custom-printer
                               {:show-valid-values? false
                                :value-str-fn my-value-str
                                :print-specs? true
                                }) #_expound/printer)
                  )
#2018-10-1818:06borkdudeoh wait, it does#2018-10-1818:06borkdudesorry šŸ™‚#2018-10-1818:06bbrincknp!#2018-10-1818:20borkdude@bbrinck in my custom function I want to say: if this is a com.stuartsierra component map, I want to print only the keys (else Iā€™m flooded with pages of information), else I want to do whatever is the default in expound#2018-10-1818:20borkdude@bbrinck can I do the else branch in a nice way?#2018-10-1818:31bbrinck@borkdude Hm, unfortunately itā€™s not super easy to do the else part right now, but if you create an issue Iā€™ll think more about how to make it nicer. Right now, I think the best you could do is to use the private function value-in-context or grab the value of the private var *value-str-fn* and call that. https://github.com/bhb/expound/blob/master/src/expound/alpha.cljc#L85#2018-10-1818:31bbrinckNot exactly pretty šŸ˜ž#2018-10-1818:32borkdudeWhatā€™s the best way to make clojure.spec.alpha not print the entire component system map. We are using (s/assert ::system system). Even when making a custom expound printing function, clojure.spec still barfs my entire console with output.#2018-10-1818:34bbrinckIā€™m not sure exactly what you mean. Can you provide an example of the output?#2018-10-1818:40borkdude@bbrinck https://gist.github.com/borkdude/fe495b05e157fd129628aa3a6e489b01#2018-10-1818:40bbrinckIā€™m curious to know which part of the output is showing the entire map - I would think if you provide a custom printer, you could in principle hide it entirely (or show as much as you like), but maybe itā€™s printing somewhere else?#2018-10-1818:41borkdudeuntil line 124 itā€™s ok, there I only print the keys of the system map. afterwards, still the whole system is printed.#2018-10-1818:44bbrinck@borkdude Ah, right, so this isnā€™t actually spec - the issue is that the program is printing out all the exception data from the assertion#2018-10-1818:44borkduderight#2018-10-1818:44bbrinckIs this running in boot? I wonder if the error printer is configurable? I can see why this is the default (you want to see the data for an assertion exception)#2018-10-1818:45bbrinckbut in the context of spec, it will put all of the detailed explain-data into the exception#2018-10-1818:45borkdudeyes, itā€™s boot#2018-10-1818:46bbrinckI can see why thatā€™s how boot works and I can see why spec adds a lot of info to the exception (might be useful), but when they work together, itā€™s definitely a lot of output.#2018-10-1818:47borkdudeI also tried this:
(try (rr/reset)
        (catch Exception e
          (throw (component/ex-without-components e))))
#2018-10-1818:47borkdudebut that doesnā€™t work, because the assert exception already has the entire system in it :-s#2018-10-1818:48bbrinckhm, not sure what you mean. Why canā€™t you alter the exception data in ex-without-components?#2018-10-1818:48bbrinckis it already in the string message?#2018-10-1818:49bbrinckif itā€™s just in the data, I presume you could just dissoc it#2018-10-1818:50bbrinckor just construct a new error that has less info, using the message from the exception#2018-10-1818:50borkdudehmm, I have to inspect the exception in the REPL to see what to dissoc, but good idea#2018-10-1818:51bbrinckIf that doesnā€™t work, the code for assert isnā€™t too long, might be possible to just use a custom one: https://github.com/clojure/spec.alpha/blob/f23ea614b3cb658cff0044a027cacdd76831edcf/src/main/clojure/clojure/spec/alpha.clj#L1959-L1968#2018-10-1818:51bbrinckbasically just omit line 1964 and 1965#2018-10-1818:52borkdudeah, right, thatā€™s easier šŸ™‚#2018-10-1819:40borkdude@bbrinck so what I wanted was actually not a custom :value-str-fn function, but a way to transform my value before itā€™s printed by expound. would it make sense to have a :transform (fn [v] ...) option in expound on the custom printer?#2018-10-1819:46bbrinckHmm, good question.#2018-10-1819:47bbrinckCan you talk more about why a transform fn would be preferable to customizing the way it prints?#2018-10-1819:48bbrinckMaybe we should move this to #expound since weā€™re talking about potential features#2018-10-2010:28slipsetThis is probably working as expected, but why?#2018-10-2010:28slipsethttps://gist.github.com/slipset/645b1a785a0d6e23097e03b837b6a18b#2018-10-2010:29slipsetSo, Iā€™ve created a spec for a function like#2018-10-2010:29slipset
(s/def ::reducing-fn (s/fspec :args (s/cat :accumulator any? :elem any?)))
#2018-10-2010:29slipsetThen, when I ask if a given fn is valid:#2018-10-2010:30slipset
(s/valid? ::reducing-fn (fn [x y]) ;; takes forever and ends with a out of heap memory
#2018-10-2010:31slipsetSo it seems (because test.check needs to be a dep for this to run) that spec is using generative testing to verify if the spec is valid.#2018-10-2010:31slipsetHmm, do I actually have a question? No, probably not, I just wanted to state that I was surprised by this, but itā€™s probably just the way it has to be.#2018-10-2010:33hmaurerIs there a spec which forces a certain value? same as #(= % some-value) but with a built-in generator, etc#2018-10-2010:43taylorYou can use a set with the value in it#2018-10-2010:59hmaurer@U3DAE8HMG how so?#2018-10-2010:59hmaurer@U3DAE8HMG oh yes sorry, nevermind#2018-10-2010:59hmaurerthanks!#2018-10-2014:18misha#{:foo}#2018-10-2014:18mishaslowpoke#2018-10-2020:03borkdudeSeems like some macros/special forms have different ways of catching/reporting insufficient input
user=> (let)
Syntax error macroexpanding clojure.core/let at (4:1).
() - failed: Insufficient input at: [:bindings] spec: :clojure.core.specs.alpha/bindings
user=> (defn)
Syntax error macroexpanding clojure.core/defn at (5:1).
() - failed: Insufficient input at: [:fn-name] spec: :clojure.core.specs.alpha/defn-args
user=> (fn)
Syntax error macroexpanding clojure.core/fn at (6:1).
() - failed: Insufficient input at: [:fn-tail]
user=> (def)
Syntax error compiling def at (7:1).
Too few arguments to def
user=> (if)
Syntax error compiling if at (1:1).
Too few arguments to if
#2018-10-2022:21gfredericksyour examples are distinguished by whether it's a special form or not#2018-10-2022:21gfredericksI get different results on version 1.10.0-alpha6#2018-10-2022:21gfredericksespecially interestingly, (fn*) compiles#2018-10-2022:22gfredericksit seems to produce a function that throws an arity exception on any call#2018-10-2121:23borkdude@gfredericks yikes, in RC1 as well. maybe a bug? EDIT: it was valid in 1.8.0 as well#2018-10-2121:25borkdudeitā€™s probably not meant as a public function anyway?#2018-10-2121:30gfredericksI hadn't considered that; I'm not sure#2018-10-2121:43borkdudenot listed here: https://clojure.github.io/clojure/clojure.core-api.html#2018-10-2122:00Alex Miller (Clojure team)Thatā€™s the compiler hook under fn#2018-10-2122:01Alex Miller (Clojure team)Certainly not the preferred entry point#2018-10-2122:06borkdudeWeā€™re writing some specs for clojure.core functions. I notice things can get a bit slower on big codebases with instrumenting all core functions. Is there a way to let spec know to not check when a function is called from clojure itself? so only external calls. I hoped direct linking would solve this, but I donā€™t think it is now#2018-10-2123:08Alex Miller (Clojure team)I would not expect calls between core functions to get checked#2018-10-2123:18andy.fingerhutI would be very surprised, if you are compiling Clojure with the default option of direct linking enabled, that calls between Clojure functions within Clojure are being checked, if you are using instrument. If you believe you are seeing otherwise, I would love to see an example of that.#2018-10-2123:20andy.fingerhutinstrument works by re-def'ing Vars. direct linking means that the current value of the Var is ignored, instead being embedded in the calling function at compile time.#2018-10-2123:22andy.fingerhutFYI, there is one function in clojure.data/diff's call tree that calls a function in clojure.set (I think clojure.set/difference) with non-set arguments. I have tried writing a spec for clojure.set/difference that causes errors if you call it with anything other than a set, and the only way I could make the error be caught is to disable direct linking when compiling Clojure.#2018-10-2123:23andy.fingerhutSome details here: https://dev.clojure.org/jira/browse/CLJ-2287#2018-10-2123:26andy.fingerhutAnd if you want to try compiling Clojure with direct linking disabled, and thus likely see instrumented runs get even slower, there are some instructions near the bottom of this page, with the heading "Building Clojure without direct linking": https://dev.clojure.org/display/community/Developing+Patches#2018-10-2206:51borkdudeThanks. Then my understanding of direct linking is correct and I need to do more research to see how to speed up things.#2018-10-2123:05hmaureris there a shortcut for a Spec s/or which only contain spec keywords and want to tag them with the same keyword? i.e. (s/or ::foo ::foo ::bar ::bar ...)#2018-10-2123:07Alex Miller (Clojure team)Not in spec, no#2018-10-2123:09hmaurerThanks @alexmiller. Other question: I read that specs are discouraged to be used as runtime because they can be slow. With that in mind, would it still make sense to use specs at runtime to branch on code based on the shape of data. i.e. check if data conforms to some spec to know if itā€™s of some ā€œkindā€, branch on that, etc#2018-10-2123:10hmaureruse it as duck typing for dynamic dispatch#2018-10-2123:11hmaureri.e. use s/valid? and s/conform pretty extensively in runtime code#2018-10-2123:11Alex Miller (Clojure team)Only you can answer whether itā€™s fast enough for you#2018-10-2123:11hmaurerItā€™s definitely fast enough for me; Iā€™m just wondering if it goes against some deeper philosophy that Iā€™m missing and could come bite me in the back down the line (beyond performance)#2018-10-2123:44hmaurerIā€™ve another question on Clojure spec for whomever is around: how do you avoid circular namespace dependencies? It seems preferable to put every spec in a namespace mapping to the namespace part of the spec keyword to be able to then require and alias that namespace wherever the spec is used. i.e.
(ns my-app.schema
  (:require [my-app.schema.person :as person]))

(s/def ::person (s/keys :req [::person/first-name]))
(ns my-app.schema.person)

(s/def ::first-name string?)
The simplest example of the issue Iā€™m running into is, then, what if I want to add a friends attribute on a person. i.e.
(ns my-app.schema.person
  (:require [my-app.schema :as schema])) ; oops, circular dependency!

(s/def ::first-name string?)
(s/def ::friends (s/coll-of ::schema/person))
#2018-10-2123:47hmaurerThe only solution I can think of it to still create the my-app.schema.person namespace but to move the attribute specs for person in the my-app.schema namespace, like this:
(ns my-app.schema)

(s/def :my-app.schema.person (s/keys :req [:my-app.schema.person/first-name :my-app.schema.person/friends]))

(s/def :my-app.schema.person/first-name string?)
(s/def :my-app.schema.person/friends (s/coll-of :my-app.schema/person))
#2018-10-2123:48hmaureractually I could still use the namespace alias here, but I would have to define the attribute specs in my-app.schema namespace and essentially just create an almost blank file for the my-app.schema.person namespace#2018-10-2123:54misha> It seems preferable to put every spec in a namespace mapping to the namespace part of the spec keyword false opieop actually, "it really depends..."#2018-10-2123:56mishaimagine random map flowing through your app code. Do you really want to look for :my-app.schema.person/first-name in it? or is something more readable would be better and enough? (and I bet :my-app is much longer, and you'll end up with keys violating PEP8's 80 chars length in no time)#2018-10-2200:00mishaalso, I feel like you split namespaces too much (with regards to spec, at least.)#2018-10-2200:01misha@hmaurer#2018-10-2200:03hmaurer@misha can you make it much shorter though? you could skip the my-app.schema part, sure#2018-10-2200:06mishathe spec kw length you have in example - is ok as is I think, but: I'd used that length for something more descriptive/convenient, so you would not feel the nee to alias it away, and avoid entire issue with dependencies.#2018-10-2200:08mishaand when you don't need to use aliases, you just need to make sure all specs are :required at the app entry points. no need to import those all over the place, as those are accessible from global registry, which you can get away initing just once.#2018-10-2200:11mishait is helpful to think about spec-kw namespaces as db table names, not as code namespaces those happen to be defined in. Otherwise specs would not survive first mild refactoring#2018-10-2200:13hmaurer@misha that makes sense; thanks a lot šŸ™‚#2018-10-2200:13hmaureralthough I was going to put those spec in a schema namespace, not next to code, so there isnā€™t a big difference there between that and the ā€œtable namesā€ approach youā€™re suggesting#2018-10-2200:13hmaurerbeyond the extra verbosity of the prefix#2018-10-2200:14mishayou receive map as rest response. it contains "...schema.../..." keys. what a waste of electricity opieop#2018-10-2200:15hmaureršŸ˜„#2018-10-2200:16mishathen, imagine, you now need to move them around in the code. ā€“ you either loose all your aliases, or break backward compatibility. ouch#2018-10-2200:17andy.fingerhutI mean, isn't one of the proposals of namespaces in keys that you don't rename them, unless you are planning to make a breaking change in a published API, and you should weigh consequences of that pretty seriously?#2018-10-2200:17hmaurerso I guess it would still be fine to namespace-qualify collection specs (i.e. (s/def ::person (s/keys ...), but you would avoid such long namespace qualification for key specs#2018-10-2200:17hmaurer?#2018-10-2200:17andy.fingerhutAnd gzip and things like Transit reduce the waste of electricity somewhat.#2018-10-2200:18mishathere is an ugly workaround, (create-ns 'foo.bar.baz) (alias 'fbb 'foo.bar.baz), but it is meh.#2018-10-2200:18hmaurerbecause collection specs are essentially a codebase-level notion, whereas keys may be sent over the network, stored, etc#2018-10-2200:18hmaurerI think?#2018-10-2200:18misha@andy.fingerhut imagine "...schema..." suffix in every sql column or table name#2018-10-2200:19andy.fingerhutMy mind isn't reeling yet, but I haven't lived and breathed it day in and out, either. Six of one, half dozen of another.#2018-10-2200:19mishaand yes, you should not rename those, if they are public outside the app code. therefore if you suddenly need to move definitions elsewhere - file ns and spec ns do not match anymore.#2018-10-2200:21misha@hmaurer nested maps is the collection spec over the wire right there.#2018-10-2200:22hmaurermmh yeah#2018-10-2200:22hmaurerbut the keyword does not appear on the wire#2018-10-2200:22mishaanyway. the best way to appreciate this - is to experience pain yourself kappa#2018-10-2200:24mishatry to model your app your way, and give it week or two#2018-10-2200:26mishawhy don't they appear?
{:my-app.schema/persons [{:my-app.schema.person/first-name "joe"}]}
#2018-10-2200:27hmaurerah yep nevermind, youā€™re right#2018-10-2200:27mishaalso, if you don't preserve ns for js clients - you are missing out as well: "my-app.schema.person/first-name" e.g. cheshire prints qualified keywords this way by default#2018-10-2200:27mishaand in js/python/java/xpath/etc. you can just obj["my-app.schema.person/first-name"]#2018-10-2200:28hmaurerso, i.e., in this case, would you name things as:
(ns my-app.schema)

(s/def :schema/person (s/keys :req [:person/first-name :person/friends]))

(s/def :person/first-name string?)
(s/def :person/friends (s/coll-of :schema/person))
?
#2018-10-2200:30mishawell, if you are confident you will not have external person - it might be ok. otherwise, prepending with a short project codename would not hurt. but I'd think twice before adding anything between codename and entity name#2018-10-2200:32mishaif you are in a corporate setting, you are probably required to add at least full org name instead or in addition to project name: :com.cognitect/person :com.cognitect.project/person#2018-10-2207:34borkdudeIs it possible to have clojure.spec not throw exceptions on fdef violations, but only print ā€œfdef error, line so and soā€ and then continue as always?#2018-10-2213:08Alex Miller (Clojure team)no#2018-10-2213:31hmaureris there a way to conform a value to a spec only one level deep?#2018-10-2214:04Alex Miller (Clojure team)no#2018-10-2222:50hmaurerthanks#2018-10-2222:45hmaurerhttps://github.com/funcool/cats/blob/master/src/cats/monad/either.cljc#L48 Is there anyway I can spec such a type (`Either left right`) on a per-function basis? So for example, I might want to add a spec to a function saying it returns either a string? or a number?, etc#2018-10-2222:53Alex Miller (Clojure team)Always good to remember that specs are not types. You can use s/or to spec an alternative but thatā€™s not going to be generic.#2018-10-2222:53hmaurer(I realise this sounds similar to what the s/or spec does, but I need to work with records here, as defined in cats)#2018-10-2222:53hmaurerah#2018-10-2222:53hmaureris there any way I can do something generic in this case @alexmiller?#2018-10-2222:54Alex Miller (Clojure team)You can treat records as maps with unqualified keys and use :req-un to specify specs on keys#2018-10-2222:55Alex Miller (Clojure team)But, not generically#2018-10-2222:56Alex Miller (Clojure team)For specific functions, you could define a :fn spec that checks a relationship between args and ret#2018-10-2222:56hmaurer@alexmiller but then Iā€™ll have to make qualified keys for every possible way there is to use the Either record. As far as I know there is no way to directly define an unqualified map spec, right? i.e. nothing like (s/foo :left string? :right number?) which would accept a map with keys :left and :right matching the specs
#2018-10-2223:12misha@hmaurer
(s/valid?
  (s/cat
    :left  (s/tuple #{:left} string?)
    :right (s/tuple #{:right} number?))
  {:left "a" :right 1})
=> true
kappa
#2018-10-2223:20Alex Miller (Clojure team)this s/cat assumes maps are ordered, which they are not. in fact, in latest spec, you will get false here.#2018-10-2223:20Alex Miller (Clojure team)s/cat can only be used with sequential? collections now#2018-10-2223:29misha@hmaurer there is also more legal way, where map-of will permit only 1 map-entry:
(s/or
  :left  (s/map-of #{:left} string?)
  :right (s/map-of #{:right} number?))
#2018-10-2223:15hmaurerO_o#2018-10-2223:17mishadon't do it#2018-10-2223:17misha
(s/def :cats/left (s/or :string string? :exception #(instance? Exception %)))
(s/def :cats/right any?)
(s/def :cats/monad (s/keys :req-un [(or :cats/left :cats/right)]))
(s/valid? :cats/monad {:left (ex-info "foo" {})})
(s/valid? :cats/monad {:left "a"})
(s/valid? :cats/monad {:right [:bar]})
    
=> true
=> true
=> true
#2018-10-2223:18mishayou then can (s/merge) with :more.specific/right#2018-10-2223:19mishabut I'd rewrite all using exceptions :D#2018-10-2223:19hmaurerRight, but then for every potential use of Either i need to define a keyword spec, i.e. :some.specific/right#2018-10-2223:20hmaurersame for left#2018-10-2223:20hmaurerinstead of being able to just specify it inline in the return spec of a function#2018-10-2223:20mishawell, the same for any return value, no?#2018-10-2223:21mishaor, if you will have specific spec for wrapped value anyway, you can unpack in custom predicate, and s/and with that#2018-10-2223:21mishabut then you'll lose generators for free opieop#2018-10-2223:21mishaconsider exceptions kappa#2018-10-2223:24mishaor, keep lifting separate from business-logic#2018-10-2223:30favila@hmaurer I had this problem continually, where I have a spec key but I need to narrow it further in specific contexts without changing the key name#2018-10-2223:30favilaI ended up making a terrible macro, keys+ to do it#2018-10-2223:31favila(s/def :either/either (s/keys ::req-un [:either/left :either/right])) is the basic notion of "either"#2018-10-2223:33favila(s/spec (s/merge :either/either (keys+ :req-un [:either/left :either/right] :conf {:either/left number? :either/right string?}))#2018-10-2223:34favilathat would be "this is an :either/either, but some of the keys use a different generator and predicate"#2018-10-2223:35favilahttps://gist.github.com/favila/ab03ba63e6854a449d64d509aae74618 if you are interested#2018-10-2223:35misha
(defmacro speceither [left-spec right-spec]
  `(s/or
     :left  (s/map-of #{:left} ~left-spec  :min-count 1)
     :right (s/map-of #{:right} ~right-spec  :min-count 1)))

(s/valid?
  (speceither string? (s/coll-of number?))
  {:right [1 2 3]})
=> true
conformed values are garbage though, but exercise works just fine
#2018-10-2223:42hmaurerthanks @favila! Iā€™ll read this up. Does this spec give nice errors / generators / conforming?#2018-10-2223:45favilaThe only thing you are allowed to do is use :conf to override the normal spec/predicate/generator for that key#2018-10-2223:45favilaā€œOverrideā€ is maybe too strong because the registered spec is checked too#2018-10-2223:57hmaurerAh right, great. Out of curiosity do you use Either / cats in your codebase @favila? Or did you write this for another use-case?#2018-10-2300:18favilaOther use case#2018-10-2300:18favilaBut itā€™s the same basic problem#2018-10-2300:19favilaThereā€™s a general structure and spec, but also more specific specs which must nevertheless maintain the structure#2018-10-2300:19favilaUsing predicates all the time became unbearable #2018-10-2313:39j0niHey folks, I'm having a tough time trying to figure out how multi-specs interact with unqualified keys - here's some example code which is a trivial version of what I'm trying to achieve:
(def mode? #{:dog :cat})
(s/def ::cat string?)
(s/def ::dog string?)
(s/def ::mode mode?)
(s/def ::dog-o (s/keys :req-un [::mode
                                ::dog]))
(s/def ::cat-o (s/keys :req-un [::mode
                                ::cat]))
(defmulti animode ::mode)
(defmethod animode :dog [_] ::dog-o)
(defmethod animode :cat [_] ::cat-o)
(s/def ::animal (s/multi-spec animode ::mode))
then some repl:
user> (s/explain ::animal {:cat "edward" :mode :cat})
val: {:cat "edward", :mode :cat} fails spec: :user/animal at: [nil] predicate: animode,  no method
=> nil
user> (s/explain ::animal {:cat "edward" ::mode :cat})
val: {:cat "edward", :user/mode :cat} fails spec: :user/cat-o at: [:cat] predicate: (contains? % :mode)
=> nil
any guidance as to how I square this apparent circle?
#2018-10-2314:01seancorfield@j0ni Your defmulti and s/multi-spec should have :mode, not ::mode. You want a function that operates on your data and returns the discriminant. That's the keyword :mode, not the spec ::mode.#2018-10-2314:02seancorfield
(def mode? #{:dog :cat})
(s/def ::cat string?)
(s/def ::dog string?)
(s/def ::mode mode?)
(s/def ::dog-o (s/keys :req-un [::mode
                                ::dog]))
(s/def ::cat-o (s/keys :req-un [::mode
                                ::cat]))
(defmulti animode :mode)
(defmethod animode :dog [_] ::dog-o)
(defmethod animode :cat [_] ::cat-o)
(s/def ::animal (s/multi-spec animode :mode))
then
user=>  (s/explain ::animal {:cat "edward" :mode :cat})
Success!
nil
#2018-10-2314:09j0niwow, thanks @seancorfield! I had been through this permutation, but apparently evaluating the ns is insufficient, I needed to reload the namespace completely#2018-10-2314:10seancorfieldA trick with defmulti is to put (def animode nil) above it and then it should work with just a re-eval of the ns.#2018-10-2314:10j0niooh nice šŸ™‚#2018-10-2314:10j0niwell, useful, if actually a little gross šŸ™‚ but I'll take it#2018-10-2315:41adamfreyI was surprised by this behavior in spec. Is there a reason that I shouldn't expect this to work?
(gen/generate (s/gen #{:works})) => :works
  (gen/generate (s/gen #{nil})) => clojure.lang.ExceptionInfo    Couldn't satisfy such-that predicate after 100 tries.
I know I can do (gen/generate (s/gen nil?)) to get around this, but I found this while writing some code for handling generic generator overrides
#2018-10-2315:48adamfreyI guess I can take generators out of the equation and get to the crux of the issue:
(s/valid? #{nil} nil) => false
(s/valid? #{false} false) => false
the spec just calls the set as a function and (#{nil} nil) => nil in clojure and spec is looking for the function truthiness. The only way to make this work would be to change the way spec deals with sets as specs where spec could call them with (contains? #{nil} nil) instead. But that sort of special casing might be undesirable.
#2018-10-2316:42bmaddyI'm trying to come up with a spec for nested associative structures:
(s/def ::nested-associative
  (s/or :coll (s/coll-of (s/or :branch ::nested-associative
                               :leaf string?)
                         :min-count 0
                         :max-count 1
                         :gen-max 1)
        :map (s/map-of keyword? (s/or :branch ::nested-associative
                                      :leaf string?)
                       :min-count 0
                       :max-count 1
                       :gen-max 1)))

;; in the repl:
> (gen/sample (s/gen ::nested-associative) 100)
 clojure.lang.ExceptionInfo: Unable to construct gen at: [:map 1 :branch :map 1 :branch :coll :branch :coll :branch] for: :user/nested-associative
Does anyone know why this would be crashing randomly? Alternatively, is there maybe a better way to do this?
#2018-10-2316:52bmaddyInteresting, it works if I just generate the s/coll-of form. It must be something to do with s/or.#2018-10-2401:57mishaI think it fails to create generator within s/*recursion-limit*#2018-10-2401:59misha
(dotimes [_ 100]
  (binding [s/*recursion-limit* 10]
    (count (gen/sample (s/gen ::nested-associative) 100))))
=> nil

(dotimes [_ 100]
  (binding [s/*recursion-limit* 1]
    (count (gen/sample (s/gen ::nested-associative) 100))))
=>
CompilerException clojure.lang.ExceptionInfo: Unable to construct gen at: [:coll :branch :map 1 :branch] for: :scratch/nested-associative #:clojure.spec.alpha{:path [:coll :branch :map 1 :branch], :form :scratch/nested-associative, :failure :no-gen}
#2018-10-2402:09mishatried 10000 times, limit=9 threw in a ~20 seconds, limit=10 ran for minutes, had to kill it opieop#2018-10-2402:14mishaso, you need to add non-recursive clause to s/or, so it can be chosen when recursion-limit is reached:
(s/def ::nested-associative
  (s/or
    :leaf string?
    :coll (s/coll-of ,,,)
    :map  (s/map-of ,,,)))
#2018-10-2402:14misha@bmaddy#2018-10-2402:26taylor@bmaddy Iā€™m able to get that to reliably generate with s/*recursion-limit* 1 by pulling the recursive s/or branch into its own spec:
(s/def ::branch
  (s/or :leaf string?
        :branch ::nested-associative))
(s/def ::nested-associative
  (s/or :coll (s/coll-of ::branch
                         :min-count 0
                         :max-count 1
                         :gen-max 1)
        :map (s/map-of keyword? ::branch
                       :min-count 0
                       :max-count 1
                       :gen-max 1)))
but Iā€™m not sure why that is, maybe itā€™s the way that recursive depth is tracked internally and the ā€œinlineā€ recursive s/or specs lose some context on recursion?
#2018-10-2402:27taylor
(dotimes [_ 1000]
  (binding [clojure.spec.alpha/*recursion-limit* 1]
    (dorun (gen/sample (s/gen ::nested-associative) 100))))
#2018-10-2403:03mishathis is the first thing I did, and it still failed for me numerous times, @taylor#2018-10-2403:53taylor:man-shrugging: works on my machine#2018-10-2404:16bmaddyOh, I get it now. I'd tried setting s/*recursion-limit* to a low number before to help avoid StackOverflow issues, but that's actually the problem here. When the recursion limit is hit, it must throw that error. Thanks @misha and @taylor!#2018-10-2415:40bastiIs there a strict validation mode in spec? Whereas if you provided more data then fail?#2018-10-2415:52dadairNo; closed specs are generally discouraged. You can do your own strictness using s/and, however.#2018-10-2419:31bbrinck@basti spell-spec gives you options for either keeping the map open (but reporting likely misspelled keys) or complete closed maps (not recommended) https://github.com/bhauman/spell-spec#2018-10-2500:23jrychterI really do not like this canned answer ("closed specs are discouraged"). It gets repeated a lot, and yet there are valid scenarios where you do want to restrict the data to exactly what is specified in the spec without manually duplicating lists of keys. Think about API endpoints, or anything that receives data from an untrusted source in general.#2018-10-2500:34seancorfieldIf you ignore the additional parameters, why do you care whether they are passed in?#2018-10-2500:36seancorfieldOne valid case I can see for closed specs is when you're about to store data into a database (and you'll get an error if you try to store columns/keys that do not exist in the table). At that point tho', you can extract the list of valid keys from the spec itself and use select-keys to get just the keys you care about. Again, you don't necessarily care that more data was present -- you just need to ensure you don't send it to JDBC.#2018-10-2500:37seancorfield(and of course you could use select-keys on the API input data too, if you wanted)#2018-10-2500:37seancorfieldThe reality is that most APIs out there silently ignore extra form and query parameters.#2018-10-2500:38seancorfield(which is all a long way of saying I think the canned answer is correct)#2018-10-2500:59taylorI agree there are cases were you need to restrict inputs but I think those are usually near the boundaries of apps, and spec doesnā€™t prevent this. Other than that I donā€™t think (in Clojure) your code should ā€œbreakā€ if passed a map with some data it doesnā€™t care about, and I think most projects have much more code on that side of the fence than the other#2018-10-2500:59jrychterIf your database is a document database (like mine) and stores your data pretty much in model form (like mine does), and your API serves your ClojureScript app which keeps the same data model, then you very much care about additional parameters. You can pass data pretty much intact between client code and the database, but you do need to make sure that you don't store any extra data that just happens to arrive over the network.#2018-10-2501:01jrychterThe manual select-keys solution seems obvious and results in error-prone code (been there done that). A more reasonable approach is to get the list of allowed keys from the spec.#2018-10-2503:30seancorfieldWhen I do select-keys I derive the list of keys to be selected directly from the spec.#2018-10-2503:41jrychterā€¦and that is exactly what I'll be doing.#2018-10-2501:02jrychterI'd also submit that "most APIs" describes "most traditional APIs". My APIs speak EDN and receive Clojure data structures which I verify using spec.#2018-10-2501:04csmpossibly s/conform should remove keys not in the spec? Iā€™m not sure of the implications of that, though.#2018-10-2501:05taylorthereā€™s no need to change spec, you can easily get this restrictive behavior if you need it#2018-10-2501:06bbrinck@jrychter Does https://github.com/bhauman/spell-spec solve your problem?#2018-10-2501:07bbrinckYou donā€™t need to duplicate keys w/ spell-spec#2018-10-2501:09csmmaybe I should just ask, though: why doesnā€™t conform remove keys not in a map spec?#2018-10-2501:09jrychterspell-spec solves a different problem and has a different goal. But I am dealing with the problem myself, I just wanted to point out that whenever people repeat what Rich said about specs intended to be open, there should be a footnote saying "except in cases where you do need to restrict your data, such as in APIs".#2018-10-2501:10bbrinckAFAICT, spell-spec (specifically strict-keys https://github.com/bhauman/spell-spec#spell-specalphastrict-keys ) does solve the problem you describe above. Can you talk more about why it doesnā€™t?#2018-10-2501:11bbrinck@csm AIUI, conform is useful for destructuring, so presumably there isnā€™t a need to remove keys.#2018-10-2501:12bbrinck(because you would usually know the keys you want to grab as part of destructuring)#2018-10-2501:12csmšŸ‘ Iā€™m just thinking out loud if there are approaches that would satisfy use cases like the above, but without making specs closed#2018-10-2501:13bbrinckAh, I see#2018-10-2501:14bbrinckremember that conform also changes the shape of data, so may not apply here if youā€™re trying to write the input to, say, a document DB#2018-10-2501:14bbrinck(I suppose you could conform/unform if it did remove unknown keys, but that gets a little strange šŸ™‚ )#2018-10-2501:15csmyeah, I thought about that; maybe a fn that does ā€œcheck this spec, then select-keysā€ would be an OK addition#2018-10-2501:15andy.fingerhutDoesn't the word "discouraged" in "closed specs are discouraged", vs. e.g. "closed specs are prohibited", convey to you that there might be occasional valid use cases for them?#2018-10-2501:15bbrinckanyway, I am curious why strict-keys from spell-spec doesnā€™t work here#2018-10-2501:17andy.fingerhutAnd sure, people can repeat things without knowing the reasons for them, nor the subtler points of when they are good ideas, and when they are not. That is a more general habit of some people to be second-handed in what they say, rather than understanding themselves.#2018-10-2501:17andy.fingerhutnot restricted to computing at all.#2018-10-2501:17bbrinckYeah, I think ā€œdiscouragedā€ is an accurate term, especially since Iā€™ve seen a few people jump to use closed specs by default. It turned out that open specs were a better fit.#2018-10-2501:17bbrinckBut yes, there are exceptions#2018-10-2501:18csmI, for one, wish spec was mature before I started using schema instead. Iā€™m not a fan of closed specs for APIs#2018-10-2501:25jrychterI do agree that open specs are a good idea.#2018-10-2501:27andy.fingerhut@jrychter I am not trying to jump down your throat. I am asking because I would actually like to learn more about your use case where you want to disallow unrecognized keys. You say you may get these extraneous keys over the network at some document database service, and want to detect this as an error?#2018-10-2501:28jrychter@andy.fingerhut my API receives EDN data, in model form. I want to pass this data to my model code (where extra keys might not matter), and often directly to a document database (RethinkDB in this case). I do not want to store extra keys, for obvious reasons.#2018-10-2501:29jrychterI know that if one uses an SQL database, there is code that needs to re-pack the data, but in my case there is often no such code.#2018-10-2501:31andy.fingerhutBear with me if I ask something foolish for lack of experience in these kinds of applications, but if the data model is extended in the future, a reason commonly given for open specs is that unless you can somehow arrange for both participants to be upgraded simultaneously (e.g. by shutting down the service while everyone upgrades), then you can't upgrade without lots of errors flying around during the partially updated state, can you?#2018-10-2501:32jrychterAn invariant in my case is that the spec describes the data model.#2018-10-2501:32andy.fingerhutIn all processes on all systems where the code is running, all of the time, even during updates to the data model?#2018-10-2501:33andy.fingerhutIf yes, then it sounds like partially updated systems are never an issue in your use case. That certainly simplifies implementing such things, if so.#2018-10-2501:36jrychterAt the moment I mostly use atomic "stop-the-world" migrations, but in general the assumption has been that the spec needs to accomodate partially updated systems. But again: I do agree that open specs are a good idea, I just think that people take Rich's "discouragement" a bit too far and see closed specs as "a bad idea". They are not a bad idea.#2018-10-2501:38andy.fingerhutIf open specs are a good idea, then there must be some situations where closed specs are a bad idea.#2018-10-2501:39andy.fingerhutIt sounds like Rich & team's experience in consulting is that they have seen multiple scenarios where people wanted to close their specs, and later regretted it.#2018-10-2501:40andy.fingerhute.g. the first occurrence of the word "consulting" in his talk on spec here: https://github.com/matthiasn/talk-transcripts/blob/master/Hickey_Rich/ClojureSpec.md#2018-10-2501:42andy.fingerhutI mean, I know people can read/hear that and think "Rich means that for 100% of all software everywhere", but I doubt that. But he is speaking in context of an industry where I think there is an over-tendency to make brittle inextensible systems.#2018-10-2501:42taylorspec isnā€™t particularly unusual in encouraging openness/forward-compatibility either, itā€™s a concern that has to be addressed in things like protobuf, thrift, etc. too#2018-10-2501:44andy.fingerhutI know I'm in the minority working at Cisco in embedded software that is far too much C code for my taste, but these systems are incredibly brittle. Changing an int to a long is often developer-weeks worth of effort, if it is in a message spread over 3 processes.#2018-10-2503:11the2bearsThis current discussion on closed specs sounds a little too much like using or relying on spec to clean your data. To be fair, though, I don't have any use cases for a closed spec myself and might be missing the real picture. Just seems better to clean the data with a different set of functions and tools.#2018-10-2503:14mishahow about not using tool-designed-not-to-solve-A to solve A?#2018-10-2503:15mishaif it is such a bummer, just implement closed keys with 1-2 macros yourself, or take one of many available in the libs already.#2018-10-2503:16mishaimplementing select-spec-keys is trivial, especially when you know what specifically you prefer to do if extra stuff present ā€“ ignore, warn, blow up, etc.#2018-10-2506:10orestisI implement this select-spec-keys just yesterday. Itā€™s very straightforward to write and I can deal with assumptions about namespaces, blow up or warm etc etc. #2018-10-2513:25jaihindhreddyHow can I add test.check to my deps.edn?#2018-10-2513:29jaihindhreddyBecause it doesn't have any dependencies, will this work?
{:deps
  {clojure/test.check {:git/url ""
                       :sha "f01e3d8ad3317e6dfbeee29332a6122b1a6c12bc"}}}
#2018-10-2513:30gfredericksYou can use it with the mvn/version just like clojure#2018-10-2513:33jaihindhreddy{org.clojure/test.check {:mvn/version "0.10.0-alpha3"}} then?#2018-10-2513:35gfredericksSounds right#2018-10-2513:38jaihindhreddyAh, so because test.check publishes artifacts (on maven) I don't need to worry about the fact that it doesn't have a deps.edn#2018-10-2513:41gfredericksRight I guess I should add an empty one#2018-10-2513:41gfredericksI'll do that soon#2018-10-2514:49Alex Miller (Clojure team)you can force the dep type without having a deps.edn#2018-10-2514:51Alex Miller (Clojure team)
{:deps
  {clojure/test.check {:git/url ""
                       :sha "f01e3d8ad3317e6dfbeee29332a6122b1a6c12bc"
                       :deps/manifest :deps}}}
#2018-10-2514:53Alex Miller (Clojure team)that overrides the manifest detection and forces it to be a deps.edn project, which is tolerant to a missing deps.edn#2018-10-2514:53Alex Miller (Clojure team)and just treats it as a project with no deps#2018-10-2516:15seancorfieldA useful trick -- is that documented in the CLI/tools.deps Reference? (specifically that it can be used to treat an arbitrary git-based project "as a project with no deps")#2018-10-2516:37Alex Miller (Clojure team)Not currently#2018-10-2600:38bbrinckIf youā€™ve run into ā€œCannot convert pathā€ errors with Expound (which happens if you use a spec with a conformer), please try out 0.7.2-SNAPSHOT and let me know what you think#2018-10-2602:02richiardiandreaWill try that out thanks a lot for the fix!#2018-10-2608:55borkdude
(s/fdef foo
    :args (s/cat :n number?)
    :ret string?)

  (defn foo [n]
    1)

  (s/conform (:ret (s/get-spec `foo)) 1) ;; :clojure.spec.alpha/invalid
  (s/assert (:ret (s/get-spec `foo)) 1) ;; 1 
#2018-10-2608:55borkdudewhy does the assert not throw there?#2018-10-2609:12mpenetyou prolly need to enable asserts#2018-10-2609:14mpenet(s/check-asserts true)#2018-10-2609:15mpenetby default they are compiled in but not enabled#2018-10-2609:18mpenetfor debugging (and because I am lazy) I sometimes use s/assert*, it bypasses the various flags#2018-10-2609:18mpenetbut that's hairy territory#2018-10-2611:27gfrederickswhy does stest/check take symbols instead of vars? it doesn't require code for you does it?#2018-10-2623:50asilvermanHi there clojurians, I am looking for some help regarding the use of clojure.spec.alpha. Basically I would like to verify that a map of key-value pairs i receive as an argument is a sub set of a well defined group. For example let the group of valid keys be :key1 :key2. I would like my spec to conform if I get a map with en empty amount of keys, or with either :key1 exclusively or :key2 exclusively or both :key1 and :key2 exclusively. Iā€™m confused about how to specify this#2018-10-2701:15misha@ariel.silverman what does "exclusively" mean here exactly? only those keys, and no others? are those qualified keys? do you have value specs too, or is it just keys?#2018-10-2916:13asilvermanThank you misha for the very thorough explanation. The code snipped was extremely helpful, as @U04V70XH6 mentioned, I also asked this question in the #beginners forum. The idea was to limit an API endpoint query parameters to a known set and error out when anything but a valid sub-set of them is passed down. Thank you again!#2018-10-2701:22misha#2018-10-2701:24mishaand a due disclaimer "clojure.spec is explicitly designed to discourage closed maps ("no keys except these")"#2018-10-2701:43seancorfield(it was answered in #beginners )#2018-10-2713:59souenzzo#datomic query returns a java.util.HashSet, that prints like a set but it's not a coll?. So I cant use s/coll-of on it. I changed my spec from (s/coll-of ::stuff) to seqable? Should spec have a s/seq-of that use seqable? or somethink like that?#2018-10-2715:52borkdudehttps://github.com/slipset/speculative#test-tools#2018-10-2716:12dangercoderIs running (spec/explain ::foo data-structure) in the beginning of a function bad practise?#2018-10-2716:16dangercoderI'm using spec/fdef right now and turning on instrument in dev-environment.#2018-10-2716:36borkdudeNothingā€™s bad, just depends what you want#2018-10-2716:51dangercoderTrue, I was thinking if there can be some kind of noticeable performance loss if having instrument on in production, but i'll have to measure and see later on.#2018-10-2717:22borkdudeyes, this definitely affects performance#2018-10-2717:48andy.fingerhutI don't have any measurements of a real application with instrument on vs. off, but I do have measurements for clojure.set functions with instrumentation on vs. off, for a simple spec that just checks whether the args satisfy set? or not. It is partway down the README on this page: https://github.com/jafingerhut/funjible#wait-isnt-this-what-clojurespec-is-for#2018-10-2717:49borkdudeI had enabled speculative that has some amount of core specs. My backend + frontend became significantly slower.#2018-10-2717:49andy.fingerhutinstrument in that case is significantly slower than the original functions when their run time is short. The percentage overhead obviously gets smaller as the run time of the instrumented function increases.#2018-10-2717:51borkdudeI think how may want to use ā€œinstrument allā€ is when I have some weird bug whose stack trace I find uninformative. Then instrument all, run again, find the problem, turn it off.#2018-10-2717:51dangercoder@borkdude sounds like a solid approach#2018-10-2717:52borkdudesome specs I do want on all the time in dev, especially at boundaries like DB insert queries#2018-10-2717:53borkdudeitā€™s a tweaking process I think#2018-10-2718:57seancorfieldAs another data point for the overhead of instrument: clojure.java.jdbc has its specs in a separate namespace and the time difference between running the tests with and without instrumentation -- just on the public API functions -- is huge. I would be extremely careful about turning on instrumentation in production.#2018-10-2719:00seancorfieldAnother thing to watch for: if you have a higher-order function instrumented, you'll invoke generative testing on the (function) argument being passed in -- which you almost certainly don't want in production (you probably wouldn't even want org.clojure/test.check included as a dependency in production?).#2018-10-2719:13dominicmWe've been using s/assert to have dev and production operate differently. #2018-10-2818:17domkmWhy is s/nonconforming not in the wiki? https://github.com/clojure/spec.alpha/blob/b2b5db433cad9f1ad56bd28f584136cef2bae73e/src/main/clojure/clojure/spec/alpha.clj#L1811#2018-10-2820:27Alex Miller (Clojure team)Undetermined whether it will remain in the api#2018-10-2820:55domkm@alexmiller I'm finding it to be extremely useful, for what it's worth#2018-10-2820:59Alex Miller (Clojure team)Are you using it for anything other than or?#2018-10-2821:00Alex Miller (Clojure team)I think itā€™s definitely useful, but wonder whether just having a nonconforming flag on or would solve the 95% case#2018-10-2821:11domkm@alexmiller It does look like I am only using it for or currently. I still think having general nonconforming is useful and shouldn't be removed. We don't know what third-party specs we might want to utilize without conforming.#2018-10-2821:12Alex Miller (Clojure team)So far, I donā€™t think Iā€™ve ever seen anyone use it for anything but s/or#2018-10-2821:00borkdudespeaking of undocumented APIs, is there a way to ask spec what functions are currently instrumented? not which functions are instrumentable?#2018-10-2821:01borkdudealternatively, asking wether a function is instrumented would also be fine#2018-10-2821:01Alex Miller (Clojure team)Ask the registry #2018-10-2821:02Alex Miller (Clojure team)Well I guess thatā€™s not telling you whatā€™s instrumented#2018-10-2821:02Alex Miller (Clojure team)Iā€™d look at the impl of unstrument#2018-10-2821:02borkdudeIā€™ll tell you the use case.#2018-10-2821:03borkdudehttps://github.com/slipset/speculative/blob/master/src/speculative/test.cljc#L28#2018-10-2821:03borkdudemaybe thereā€™s a better way to do it.#2018-10-2821:03borkdudeIā€™m now using the private atom: https://github.com/slipset/speculative/blob/master/src/speculative/test.cljc#L19#2018-10-2821:05borkdudethe with-unstrumented macro is problematic in clojurescript though, because part of instrumenting/unstrumenting happens at macro compilation time and it uses two separate atoms to bookkeep instrumented functionsā€¦#2018-10-2821:38borkdudeif (stest/instrument 'clojure.core/reduce) gave back [cljs.core/reduce] only when it wasnā€™t already instrumented, I could leverage that.#2018-10-2821:40borkdudebut of course I could call (s/unstrument 'clojure.core/reduce) first, to check if it was already instrumentedā€¦#2018-10-2821:40borkdudethis way I wouldnā€™t have to use something private#2018-10-2910:13borkdudefixed.#2018-10-2912:34borkdudehttps://twitter.com/stuarthalloway/status/1056881838927568896#2018-10-2916:33jaihindhreddyAlex Miller commented on a podcast about making specs more data oriented. Can't wait to see what's cookin' at Cognitect!#2018-10-2913:03domkmHas there been any thought/discussion about clojure.spec for directed graph validation? The problem we're running into is that some edges are required and that causes validation to never terminate due to cyclic validations. It would be interesting if there were a way to make required validation respect s/*recursion-limit*.#2018-10-3013:17borkdudeI have an interesting case where I call stest/check from the REPL (lein + cider nrepl) and it doesnā€™t finish. Anyone familiar with that one?#2018-10-3013:18borkdudeit does terminate when I run the tests from the command line#2018-10-3013:19Alex Miller (Clojure team)how do you know it wonā€™t finish? did you solve the halting problem!?#2018-10-3013:26borkdudeIā€™m a human, I use heuristics.#2018-10-3013:28borkdudeseems to work with clj repl#2018-10-3013:30borkdude
$ clj -A:test:repl
user=> (stest/check `str)
({:spec #object[clojure.spec.alpha$fspec_impl$reify__2524 0x3a94964 "
#2018-10-3013:33gfredericksHave you tried at least a few times from the command line? Tests that probabilistically take a Long Time can be misleading like that with just a few data points#2018-10-3015:05borkdude@gfredericks trying. hmm, now clj lingers after printing the result of stests:
$ clj -A:test -e "(require,'[speculative.core])" -e "(require,'[clojure.spec.test.alpha,:as,stest])" -e "(stest/check \`str)"
({:spec #object[clojure.spec.alpha$fspec_impl$reify__2524 0x3e14c16d "
#2018-10-3015:05borkdudeit just takes a long time before it returns to the cmd line. maybe garbage collecting?#2018-10-3015:06gfredericksis it exactly 60s?#2018-10-3015:09borkdudethis is the output with time:
$ time clj -A:test -e "(require,'[speculative.core])" -e "(require,'[clojure.spec.test.alpha,:as,stest])" -e "(stest/check \`str)"
({:spec #object[clojure.spec.alpha$fspec_impl$reify__2524 0x3e14c16d "
#2018-10-3015:13gfredericksI meant 60s following the end of the tests if so, then adding (shutdown-agents) after the stest/check call should help#2018-10-3015:17borkdudeRepro:
;; run with:

;; clj -Srepro -Sdeps '{:deps {org.clojure/clojure {:mvn/version "RELEASE"} org.clojure/test.check {:mvn/version "RELEASE"}}}' -i stest_time.clj


(ns stest-time
  (:require
   [clojure.test :as t]
   [clojure.spec.alpha :as s]
   [clojure.spec.test.alpha :as stest]))

(s/fdef clojure.core/str
  :args (s/* any?)
  :ret string?)

(stest/check `str)

(println "after stest")

(shutdown-agents)

(println "after shutdown agents")
Still takes 13s after printing the last message.
#2018-10-3015:18borkdudemaybe any? just generates loads of garbage?#2018-10-3015:18gfredericksthat sounds very strange#2018-10-3015:18borkdudeyou can try the scriptā€¦#2018-10-3015:18gfredericksI mean it does, but I don't think a JVM needs to GC on shutdown#2018-10-3015:18gfredericksyou just burn it all down and let the OS sort it out#2018-10-3015:19borkdudemaybe a System/exit helps#2018-10-3015:19borkdudeyeah it doesā€¦ but for a test namespace thatā€™s not an option maybe#2018-10-3015:19taylormaybe use jstack while it's "hung" to see what it's doing at the moment#2018-10-3015:20gfrederickskill -3 says there's an agent thread running generator stuff#2018-10-3015:21gfredericksso that code launches a future somehow#2018-10-3015:22gfredericksI think shutdown-agents might not be synchronous in its effect#2018-10-3015:22gfredericksso the fact that it waits longer when there's a future in the background that hasn't completed isn't surprising#2018-10-3015:22gfredericksbut why there's a future in this case I don't know#2018-10-3015:23gfredericks(I haven't used clojure.spec.test.alpha much)#2018-10-3015:26taylor
at clojure.spec.test.alpha$quick_check.invokeStatic(alpha.clj:310)
	at clojure.spec.test.alpha$quick_check.invoke(alpha.clj:303)
	at clojure.spec.test.alpha$check_1.invokeStatic(alpha.clj:336)
	at clojure.spec.test.alpha$check_1.invoke(alpha.clj:324)
	at clojure.spec.test.alpha$check$fn__3088.invoke(alpha.clj:412)
	at clojure.core$pmap$fn__8342$fn__8343.invoke(core.clj:6994)
šŸ¤”
#2018-10-3015:27Alex Miller (Clojure team)check uses pmap#2018-10-3015:27taylorhttps://github.com/clojure/spec.alpha/blob/f23ea614b3cb658cff0044a027cacdd76831edcf/src/main/clojure/clojure/spec/test/alpha.clj#L411#2018-10-3015:27gfredericksah well there you go then#2018-10-3015:28gfredericksgotta spend 15 seconds generating a few last giant piles of garbage in case somebody wants them#2018-10-3015:30gfredericks@borkdude the most recent alpha of test.check might be a bit better about this. I can't remember, and also don't know what version you get by specifying RELEASE#2018-10-3015:31taylorlooks like org/clojure/test.check/0.10.0-alpha3#2018-10-3015:31gfredericksoh; nevermind then#2018-10-3015:31gfredericksfor this kind of test it's probably best to use a smaller max-size for the gen/any#2018-10-3015:32borkdudeIā€™ll try that, thanks.#2018-10-3015:38gfredericksI wonder if gen/any should just have a smaller default size universally šŸ¤”#2018-10-3015:38borkdudeany snippet handy that I could re-use?#2018-10-3015:38gfredericksno, I'm not sure of the exact syntax for sizing control in spec#2018-10-3015:39borkdudek, Iā€™ll figure something out#2018-10-3015:39gfredericksif you don't mind affecting the whole generator, I think the stest/check call lets you pass max size#2018-10-3015:39borkdudeoh that would be nice#2018-10-3015:39gfredericksfor finer grained control you'd probably set an override specific to the any? spec#2018-11-0209:53borkdudeAny idea how to do this? I tried by using a :gen override for :clojure.core/any? but that doesnā€™t work, because any? is a builtin I think: https://github.com/clojure/spec.alpha/blob/31165fec69ff86129a1ada8b3f50864922dfc88a/src/main/clojure/clojure/spec/gen/alpha.clj#L118#2018-11-0212:58gfredericksI wouldn't have expected that, but assuming you're right and you can't override builtins, another option would be having your own (def ::any any?) that you can override#2018-11-0212:58gfredericksunfortunate if you have to do that, but should work at least#2018-10-3015:46borkdude@alexmiller if you want I can post that script in an issue if you think itā€™s something that should be improved over time#2018-10-3015:49Alex Miller (Clojure team)Hmm?#2018-10-3015:50borkdudehttps://clojurians.slack.com/archives/C1B1BB2Q3/p1540912650053300#2018-10-3015:51Alex Miller (Clojure team)I donā€™t understand what weā€™re talking about#2018-10-3015:51Alex Miller (Clojure team)check returns a lazy seq - you need to consume it#2018-10-3015:52Alex Miller (Clojure team)As noted in the doc string#2018-10-3015:52Alex Miller (Clojure team)So wrap a doall around the call to check#2018-10-3015:52borkdudeaah, wrapping it in a doall works#2018-10-3015:53borkdudethank you#2018-10-3015:53borkdudeissue solved (see above thread).#2018-10-3015:54borkdudedidnā€™t realize that (pun intended)#2018-10-3021:15andy.fingerhutI slightly envy your pun-creation skills.#2018-10-3018:00martinklepschI'm not sure if this is a bad idea or something but I'd love to always have stest/instrument switched on during development. Is there some way to keep functions instrumented after re-defining them in the REPL?#2018-10-3018:09taylorI see there's (defonce ^:private instrumented-vars (atom {})) in spec.alpha, but I don't see it publicly exposed. Something that came to mind was using that to re-instrument things that were previously instrumented?#2018-10-3020:39dadairIf you are using something like Component or Integrant you can wrap their system reset functions with calls to instrument#2018-10-3020:40dadairIā€™ve done that with our duct-based project at work#2018-10-3022:52martinklepschIā€™ve considered that but often Iā€™m just working in one namespace in the REPL and restarting the system seems like overkill in a way #2018-10-3022:53martinklepschI guess I should just bind stest/Instrument to a hotkey#2018-10-3018:45borkdude@martinklepsch I hook up stest/instrument with component. So when I restart the system, my new fns are instrumented.#2018-10-3023:10borkdudesomehow this also lingers after the test has executed, even with doall:
(ns spec-keys-test
  (:require
   [clojure.test :as t :refer [deftest testing is]]
   [clojure.spec.alpha :as s]
   [clojure.spec.test.alpha :as stest]))

(defn foo [n] n)

(s/fdef foo
  :args (s/cat :n number?)
  :ret number?)

(deftest repro-test
  (testing "output of check"
    (let [r (doall (stest/check `foo))
          c (count r)]
      (println "count" c)
      (is (= c (count (keep :spec r))))
      (is (= c (count (keep :sym r))))
      (is (= c (count (keep :clojure.spec.test.check/ret r)))))))

(t/run-tests)
#2018-10-3023:56andy.fingerhutI think shutdown-agents was mentioned before, but wasn't sure whether you had tried it. If your code calls future directly (or indirectly through one of several other functions), it can cause a 60-second pause at the end before the JVM exits: http://clojuredocs.org/clojure.core/future#2018-10-3108:18borkdudeRight, thatā€™s it. So doall + shutdown-agents it is.#2018-10-3023:56andy.fingerhute.g. pmap clojure.java.shell/sh#2018-10-3100:45misha@domkm I think you can use your own predicate in root graph spec (s/and asyclic? (s/keys ...)), which internally uses s/*recursion-limit*, but that would traverse graph twice#2018-10-3100:47domkm@U051HUZLD Thanks for the suggestion. I think I wasn't clear. The graph is cyclic. I just only want to validate to N levels and stop validating after that point.#2018-10-3100:52mishawell, I got nothing then, except writing your own validator, and forfeiting all the for-free generators, etc.#2018-10-3100:53mishaor, use 1 spec to generate, and custom walking predicate to validate#2018-10-3100:54mishapredicate can somewhat reuse specs for nodes and edges, for keys, but it is still not pretty#2018-10-3109:18magnusdkHas anyone found a nice workaround for this: https://dev.clojure.org/jira/browse/CLJ-2067? [spec] (s/def ::a ::b) throws unable to resolve error if ::b is not defined#2018-10-3109:19magnusdkWe hack-fixed it by defining this macro:
(defmacro def*
  [name spec]
  `(do (when (and (qualified-keyword? ~spec)
                  (not (#'s/maybe-spec ~spec)))
         (s/def ~spec
           #(throw (ex-info "Spec is declared, but not defined!"
                            {:spec-name  ~name
                             :spec-alias ~spec
                             :args       %&}))))
       (s/def ~name ~spec)))

(def* ::foo ::NaS)            ;;=> :my.spec/abc
(s/valid? ::foo "something")  ;;=> clojure.lang.ExceptionInfo: Spec is declared, but not defined!
(def* ::NaS string?)          ;;=> :my.spec/NaS
(s/valid? ::foo "something")  ;;=> true
And doing something similar with s/fdef, but this broke down when we started using orchestra for instrumentation.
#2018-10-3113:44domkmIf you can't rearrange the order of your s/defs, perhaps you can (s/def ::b any?) first and then redef it later?#2018-10-3115:27magnusdkedit: just some more context Using a similar macro as the one above we specced a functionā€™s :ret using a spec that hadnā€™t yet been defined, but we knew that it would be defined after all the namespaces had been loaded. However, when we started instrumenting the function (using orchestra) the spec was still throwing our Ā«declared, but not definedĀ» exception. It seems that the spec wasnā€™t overwritten, but describing the specs in the REPL displayed the expected results.#2018-10-3115:34magnusdkYour suggestion gets rid of the exceptions, but the spec is still not redefed šŸ¤” The function can now return anything because the any? spec still stands#2018-10-3117:32bmaddyThis seems like a silly question, but is there a standard way of checking specs in tests? I suspect it uses (-> (stest/enumerate-namespace 'user) stest/check) and looks something like this:
(deftest specs
  (...
    (-> (stest/enumerate-namespace 'user) stest/check)))
#2018-10-3117:53seancorfield@bmaddy Generative tests can be long-running so having them in "unit tests" that you run all the time is a bit of an anti-pattern.#2018-10-3117:57jaihindhreddyHow do you guys do unit testing? Just what percentage of tests are manually written example based ones?#2018-10-3118:13bmaddyWe mostly do classic unit tests. We have one file where we use clojure.test.check.clojure-test/defspec, but I think that was before spec came out. We're about 97% unit tests.#2018-10-3117:58bmaddyHmm, so do people just remember to run a stest/check function that's in a comment after the function every time they update it? That's what I've been doing so far, but I figured there was a better way.#2018-10-3118:10bbrinckI think check can be useful when writing the function/spec, but for tests, I tend to use test.chuck and write my properties in the test#2018-10-3118:11bbrinckthen I can use chuck/times to control the number of generative tests (low number during development, higher number on CI)#2018-10-3118:13bbrinckbut I donā€™t tend to write a lot of functions that have super interesting :fn specs, I just check :args and :ret specs in tests#2018-10-3118:13bbrinckMy tests will instrument my functions, then for any input I can just use s/gen to get the generator, use that with test.chuck#2018-10-3118:22jaihindhreddyI thought test.chuck was a typo. šŸ˜‚#2018-10-3118:23bbrinckheh, no, itā€™s a really useful lib for generative testing šŸ™‚#2018-10-3118:27seancorfieldThere's also clojure.test.check.clojure-test/defspec for property-based tests... https://github.com/clojure/test.check/blob/master/src/main/clojure/clojure/test/check/clojure_test.cljc#L74#2018-10-3118:27seancorfieldAnd, yeah, big +1 for test.chuck -- we love the regex string generator in it!#2018-10-3118:30seancorfield@bmaddy I think a good option is to have some scripts that run stest/check and summarize/assert the results are good, and then run those scripts directly as part of either periodic manual or full-suite testing (rather than automated/unit test level stuff).#2018-10-3118:32seancorfieldAt work we have a few generative tests that run in our "mainline" automated/unit tests, and then we have some more extensive generative/property tests that are in (comment ...) forms in various files that we currently run manually, from time-to-time.#2018-10-3118:33seancorfield("Rich Comment Forms" -- per Stu's Strange Loop talk)#2018-10-3119:43bmaddyThanks @seancorfield. I'd be interested to hear how others do this as well if they feel like sharing.#2018-10-3121:35borkdudeam I right that the stc namespace for options can only be aliased like:
#?(:clj (alias 'stc 'clojure.spec.test.check))
in cljc code for clojure? Putting it in the ns form didnā€™t compile.
#2018-10-3121:35borkdudeI think because it doesnā€™t exist as a file, but is created when loading clojure.spec.test.alpha#2018-10-3121:36gfredericksthat sounds plausible#2018-10-3121:36gfredericksI didn't know it did that#2018-10-3121:36borkdudeitā€™s a bit awkward. trying to think about this: https://dev.clojure.org/jira/browse/CLJS-2952#2018-10-3121:37borkdudeI think I had a nice solution, but now my ns looks like:
(ns spec-keys-test
  (:require
   [clojure.test :as t :refer [deftest testing is]]
   [clojure.spec.alpha :as s]
   [clojure.test.check]
   [clojure.spec.test.alpha :as stest]
   #?(:cljs [clojure.spec.test.check :as stc])
   ))

#?(:clj (alias 'stc 'clojure.spec.test.check))
#2018-10-3121:38borkdudemaybe changing the clojure version of the alias stc to clojure.test.check makes more sense than the in-ns thing in clojure.spec.test.alpha#2018-10-3122:02borkdudeAh I see why that isnā€™t the case, because clojure.test.check is lazily loaded (optional dep).#2018-10-3122:04borkdudeWould it be OK if clojure.spec.alpha created an empty ns in a file, instead of the in-ns? Then both requires for cljs and clj could look the same. This would be nice in .cljc.#2018-10-3122:23borkdudePosted an issue about this at JIRA#2018-11-0115:22borkdudeIs nil considered valid input for Clojure set functions? 0 arity returns empty set, but 1 arity with nil returns nil#2018-11-0115:22borkdudeIn other words should a spec for those functions account for nil input or throw#2018-11-0115:45seancorfield@borkdude Could you show some code? I'm not sure what you're asking.#2018-11-0115:50borkdude(set/union) vs (set/union nil), they differ in return type#2018-11-0115:55seancorfieldset/union is only defined for arguments that are sets tho', right?#2018-11-0115:56seancorfield(and nil is not a set -- so it's "garbage-in, garbage-out" here?)#2018-11-0115:58seancorfieldSo I guess the answer to your question is "No, a spec for those functions should not allow nil as input".#2018-11-0116:16borkdudeIf we can agree for union etc. that 1) :ret should be set? then we must either 2) reject nil inputs in the :args spec, or 3) assert that the implementation is wrong#2018-11-0116:18borkdudemaybe @alexmiller can say something on this#2018-11-0116:54Alex Miller (Clojure team)donā€™t know#2018-11-0116:55borkdudewould we find it useful for fdefs detect nil (as an instance of non-sets) inputs for set fns?#2018-11-0116:57borkdudeor can we say this is undefined territory and assume the simpler spec#2018-11-0116:57Alex Miller (Clojure team)I think right now I would treat both inputs and output as nilable
#2018-11-0116:57Alex Miller (Clojure team)as existing code may be relying on the behavior of those things#2018-11-0116:58borkdudeif we donā€™t allow nil, we will find out šŸ™‚#2018-11-0116:59Alex Miller (Clojure team)if your spec fails on working existing code, I think your spec is wrong#2018-11-0116:59borkdudetrue, but I mean, we can actually discover if people use this in the wild and then adapt the spec accordingly#2018-11-0117:00Alex Miller (Clojure team)well, disagree#2018-11-0117:01borkdudeyesā€¦?#2018-11-0117:01Alex Miller (Clojure team)I donā€™t know how else to say it#2018-11-0117:02Alex Miller (Clojure team)you asked for my opinion. My opinion is that you should spec the inputs and output as nilable.#2018-11-0117:03borkdudeyes, I wondered why you disagree with adapting the spec gradually when we discover that people actually use those fns like that#2018-11-0117:03borkdudebut maybe we can rightaway assume people do this#2018-11-0117:04Alex Miller (Clojure team)nils are often used interchangeably with empty collections. it seems unlikely to me that there is not some code relying on this either for input or output#2018-11-0117:04borkduderight.#2018-11-0117:08borkdudenext caseā€¦
(clojure.set/union 3)
#2018-11-0117:09borkdudethe identity functionā€¦.#2018-11-0117:09borkdudeI think itā€™s safe to assume people do not use the 1-arity as the identity function šŸ˜‰#2018-11-0117:09Alex Miller (Clojure team)I think you can figure that one out#2018-11-0117:11borkdudethanks for the replies#2018-11-0120:39danielcomptonIs there a recommended way of running stest/check in a deftest? I looked around and couldn't see anything official, just lots of different ways people did it#2018-11-0120:40borkdude@danielcompton if there is Iā€™d like to know, because Iā€™m doing that right nw#2018-11-0120:45danielcomptonhttps://stackoverflow.com/questions/40697841/howto-include-clojure-specd-functions-in-a-test-suite#2018-11-0120:48borkdude@danielcompton Iā€™m trying to write tests for clojure, cljs and self-hosted cljs all within one .cljc file. Guess what, all three environment expect and return different keys for the clojure.test.check opts/rets.#2018-11-0120:48danielcomptonOuch#2018-11-0120:53bmaddy@borkdude, @danielcompton There was a short discussion about that yesterday. People suggested using test.chuck (not a typo) or separating your property based tests out and run them separately.#2018-11-0120:53bmaddyThis is mainly because I'm curious, but does anyone now why the args passed to :fn in s/fdef are conformed values? I'd like to use the actual value in there, but dealing with the conformed value is tricky.#2018-11-0120:54borkdude@bmaddy does test.chuck support .cljc?#2018-11-0120:54bmaddyI've never used it, but the readme says this in the Acknowledgements section: @nberger for adapting to cljc format for ClojureScript, and general maintenance help#2018-11-0120:55andy.fingerhut@borkdude Fine detail nit on your earlier questions -- in the clojure.data/diff implementation there are calls at least clojure.set/union, and perhaps a couple of other clojure.set functions, with sequences as arguments (the return value of the clojure.core/keys function IIRC), so not a set and not nil. I believe the implementation of clojure.set/union for the ways they are called from always return correct results (i.e. duplicates in the return value are only harmful for performance, not correctness of the clojure.data/diff results). Not sure if that is considered a bug or something that an 'official' spec for clojure.set/union should allow. https://dev.clojure.org/jira/browse/CLJ-1087 I am an interested observer of such detailed questions, too, not a policy maker.#2018-11-0120:57borkdude@andy.fingerhut Please include these notes at https://github.com/slipset/speculative/issues/70#2018-11-0120:59andy.fingerhutOK, I've just added a comment to that issue with a copy/paste of what I just said above.#2018-11-0214:00borkdudeThis works:
(set/map-invert (java.util.HashMap. {:foo :bar}))
This also works:
(set/map-invert [[:a 1] [:b 2]])
Even this works:
(set/map-invert ["a1" "b2"])
{\1 \a, \2 \b}
Any reducible coll of nth-able pairs will do. How far should we go with the spec for set/map-invert?
#2018-11-0214:00borkdudeThe first case is used in Datomic#2018-11-0214:01borkdudeSee: https://github.com/slipset/speculative/issues/70#2018-11-0219:59borkdude@danielcompton I added some test helpers here: https://github.com/slipset/speculative/blob/master/src/speculative/test.cljc (`success?` and gentest, combined into this test util, also named gentest: https://github.com/slipset/speculative/blob/master/test/speculative/test_utils.cljc#L22)#2018-11-0221:08danielcomptonNice :+1: #2018-11-0415:09roklenarcicI'm writing a macro where I want to turn the passed in symbol/keyword into a spec. How do I do that?#2018-11-0415:09roklenarcicI'm writing a macro where I want to turn the passed in symbol/keyword into a spec. How do I do that?#2018-11-0415:11taylors/get-spec if youā€™re trying to look up an existing spec by symbol/keyword#2018-11-0415:12roklenarcicyeah but that doesn't work for predicates#2018-11-0415:12roklenarciclike someone passes in string?#2018-11-0415:12roklenarcics/get-spec of course says that no such spec has been registered#2018-11-0415:14borkdudeDoes creating a ::string spec work? @U08EKSQMS just made a PR to speculative for a lot of those#2018-11-0415:17taylorit sounds like you want to check if the symbol resolves to a function (predicate), and then make a spec out of that function?#2018-11-0415:17roklenarcicThe thing is that it's a library so I don't know what people will be passing in.#2018-11-0415:17roklenarcicYes something like that#2018-11-0415:18roklenarciccreating spec out of a symbol has the benefit of having that symbol show up in description of the spec failure#2018-11-0415:33taylorthis doesnā€™t really help, just an observation re: names/descriptions in failures:
(s/explain string? 0)
val: 0 fails predicate: :clojure.spec.alpha/unknown
=> nil
(s/explain (s/every string?) [0])
In: [0] val: 0 fails predicate: string?
=> nil
#2018-11-0415:37taylor
(s/explain (s/spec string?) 0)
val: 0 fails predicate: string?
=> nil
#2018-11-0415:40roklenarcicI see#2018-11-0415:42taylor
(defmacro ->spec [k]
  `(or (s/get-spec ~k) (s/spec ~k)))
(defn foo? [x] (= "foo" x))
=> #'?
(s/explain (->spec foo?) "foo")
Success!
=> nil
(s/explain (->spec foo?) "bar")
val: "bar" fails predicate: foo?
=> nil
(s/def ::foo #{"foo"})
=> :
(s/explain (->spec ::foo) "foo")
Success!
=> nil
(s/explain (->spec ::foo) "bar")
val: "bar" fails spec: : predicate: #{"foo"}
=> nil
#2018-11-0415:43taylormaybe something like that would work#2018-11-0415:43roklenarciccool I'll check it out#2018-11-0417:52mishawhy do you need it in the first place? to have named things in explain's out? @U66G3SGP5#2018-11-0418:30roklenarcicyes šŸ™‚#2018-11-0423:47didibusI'm confused, sounds like you just want to call (s/def). It takes a keyword or symbol and registers a spec for it.#2018-11-0611:38roklenarcicis there a way to spec map's key+value, but the key isn't specific#2018-11-0611:38roklenarciclike how do I say: map key is symbol or keyword, if key is symbol then value must be string but if key is keyword then value must be integer#2018-11-0611:39roklenarcicI tried s/coll-of :kind map? but that only conforms value#2018-11-0611:51guyIs this useful to you perhaps? https://clojure.github.io/spec.alpha/clojure.spec.alpha-api.html#clojure.spec.alpha/map-of#2018-11-0611:51guy@roklenarcic#2018-11-0612:07borkdude@roklenarcic can the map also have combinations of these key/val pairs?#2018-11-0612:08borkdudein that case I would make a spec for the map-entry#2018-11-0612:08Alex Miller (Clojure team)You can use s/every with an s/or of s/tuple for entry combinations#2018-11-0612:10borkdudeSelf-hosted cljs with spec in the browser: https://twitter.com/borkdude/status/1059758785613479937#2018-11-0612:12roklenarcicthank you alex, I'll try that#2018-11-0612:35roklenarcichm well every doesn't conform#2018-11-0612:37roklenarcichm... when using coll-of, if I specify kind as map? and if I don't specify kind, conforming works differently šŸ˜„#2018-11-0612:39borkdude@roklenarcic specify it as a seq of tuples?#2018-11-0612:40borkdudeso something like (coll-of ::my-tuple-spec)#2018-11-0612:40roklenarcictuple spec is no better than map-of spec... each item is independent#2018-11-0612:41borkdude::my-tuple-spec can spec independent things?#2018-11-0612:41roklenarcic(s/tuple pred1 pred2)#2018-11-0612:41roklenarcicyou can't link type of 1 and type of 2#2018-11-0612:42borkdude(s/or (s/tuple symbol? string?) (s/tuple s/keyword? int?))#2018-11-0612:42roklenarcicso s/coll-of that?#2018-11-0612:42borkdudeyes#2018-11-0612:44borkdudeand I guess :kind could optionally also be set to map?, itā€™s just an extra check#2018-11-0612:44roklenarcic
(s/conform (s/coll-of (s/or :s1 (s/tuple symbol? string?) :s2 (s/tuple keyword? int?))) '{a "A" :b 3})
=> {:s1 [a "A"], :s2 [:b 3]}
#2018-11-0612:44borkdudebut often a seq of map-entries can be used in places where maps are used#2018-11-0612:44roklenarcicthis is completely different to how I would expect things to go#2018-11-0612:44roklenarcicI would expect a collection of tuples#2018-11-0612:44borkdude@roklenarcic you mean the conformed value?#2018-11-0612:44roklenarcicyes#2018-11-0612:45roklenarcicthis looks completely wrong and unexpected to me#2018-11-0612:46borkdudeit isnā€™t wrong, itā€™s conformed. else you get ::s/invalid#2018-11-0612:46roklenarcic
(s/conform (s/coll-of (s/or :s1 (s/tuple symbol? string?) :s2 (s/tuple keyword? int?))) '{a "A" :b 3 c "B"})
=> {:s1 [c "B"], :s2 [:b 3]}
#2018-11-0612:46roklenarcicsee... now I lost a term#2018-11-0612:46roklenarcicinput map has 3 key-values#2018-11-0612:46roklenarcicresult has two items#2018-11-0612:46borkdudeah I see#2018-11-0612:47borkdudeso maybe every will help there?#2018-11-0612:48roklenarcicnah, I'll just switch back#2018-11-0612:48roklenarcicto :kind map?#2018-11-0612:48roklenarcicand take vals#2018-11-0612:48roklenarcicproblem solved#2018-11-0612:48borkdudeif I had time I would help you figure it out, but I have to go now#2018-11-0612:48roklenarcicdon't worry, I have a solution, I was just looking for something more elegant#2018-11-0612:50borkdudeyes, (s/conform (s/every (s/or :s1 (s/tuple symbol? string?) :s2 (s/tuple keyword? int?))) '{a "A" :b 3 c "B"}) works#2018-11-0612:51roklenarcicevery doesn't do any conforming at all FYI#2018-11-0612:51roklenarcicat least I think so#2018-11-0612:51borkdudeoh you want conforming#2018-11-0612:51borkdudesorry šŸ™‚#2018-11-0612:51roklenarcic(s/coll-of x :kind map?), conforms key+value into value#2018-11-0612:52roklenarcicso you just have to take (vals ) of value in the map then#2018-11-0612:52borkdude@roklenarcic
(s/conform (s/cat :map-entries (s/* (s/alt :s1 (s/tuple symbol? string?) :s2 (s/tuple keyword? int?)))) '{a "A" :b 3 c "B"})
#2018-11-0614:00Alex Miller (Clojure team)I donā€™t think this approach will work on latest spec - regex specs now only work on sequential collections (too confusingly dangerous to do sequential specā€™ing on unordered collections)#2018-11-0614:02Alex Miller (Clojure team)(s/conform (s/coll-of (s/or :s1 (s/tuple symbol? string?) :s2 (s/tuple keyword? int?))) '{a "A" :b 3 c "B"})#2018-11-0614:02Alex Miller (Clojure team)would be better#2018-11-0614:04borkdudethat was already proposed by me, but that didnā€™t work, because the result would be: {:s1 [c "B"], :s2 [:b 3]} and you lose map-entries#2018-11-0614:04borkdudeI guess you can call seq on the map first if it doesnā€™t work with the newest spec#2018-11-0614:07borkdudeevery also works, but that doesnā€™t conform and the OP wanted that#2018-11-0614:12Alex Miller (Clojure team)(s/conform (s/coll-of (s/or :s1 (s/tuple symbol? string?) :s2 (s/tuple keyword? int?)) :into []) '{a "A" :b 3 c "B"})#2018-11-0614:13Alex Miller (Clojure team)=> [[:s1 [a "A"]] [:s2 [:b 3]] [:s1 [c "B"]]]#2018-11-0614:13Alex Miller (Clojure team)
user=> (s/conform (s/coll-of (s/nonconforming (s/or :s1 (s/tuple symbol? string?) :s2 (s/tuple keyword? int?))) :into []) '{a "A" :b 3 c "B"})
[[a "A"] [:b 3] [c "B"]]
#2018-11-0614:14Alex Miller (Clojure team)if you want to hide the or tag#2018-11-0614:18borkdudeaah, :into [], ok#2018-11-0612:58roklenarcicwell that's clever šŸ™‚#2018-11-0613:45martinklepschI'm having some problems adding :args specs to a function, I think it kind of boils down to this:
(spec/def ::title string?)
(spec/def ::entry-x (spec/cat :title ::title))
(spec/valid? ::entry-x ["Title"])                     ; => true
(spec/valid? (spec/cat :entry ::entry-x) ["Title"])   ; => true
(spec/valid? (spec/cat :entry ::entry-x) [["Title"]]) ; => false
I would expect the additional cat to require additional wrapping but it appears that ::entrty-x and (spec/cat :entry ::entry-x) are identical for some reason.
#2018-11-0613:46borkdudeI guess you can nest s/cat?#2018-11-0613:48borkdude
(s/def ::foo (s/cat :a int? :bcat (s/cat :b int?)))
(s/conform ::foo [1 2]) ;; {:a 1, :bcat {:b 2}}
#2018-11-0613:55Alex Miller (Clojure team)regex ops nest to describe the same structured collection#2018-11-0613:55Alex Miller (Clojure team)if you need a new ā€œlevelā€, insert s/spec#2018-11-0613:55martinklepschInteresting I would have expected that spec to match [1 [2]] I guess#2018-11-0613:55Alex Miller (Clojure team)this is covered in the spec guide#2018-11-0613:56borkdudeit behaves the same as s/alt, s/*, etc. they consume the same seq ā€œlevelā€#2018-11-0613:58martinklepschOk, s/spec did the trick. Also found the section in the spec guide now#2018-11-0614:19borkdude@roklenarcic please watch the thread, Alex gave some advice there#2018-11-0614:28borkdudebtw, handy link if you quickly want to check examples posted here: http://app.klipse.tech/?cljs_in=(require%20%27[clojure.spec.alpha%20:as%20s])#2018-11-0614:36borkdudeis this feature in clojure 1.10 related to upcoming changes in clojure.spec maybe? https://twitter.com/timbaldridge/status/1059815352106795008#2018-11-0618:05Alex Miller (Clojure team)no, itā€™s a genericization of stuff from datafy#2018-11-0618:05Alex Miller (Clojure team)although datafy is potentially something we will do in spec, still tbd#2018-11-0618:12borkdude:+1:#2018-11-0614:46kirill.salykinwhat is instance based protocol polymorphism?#2018-11-0614:46mpenettype vs instance#2018-11-0614:46mpenetnumber vs 42#2018-11-0614:47mpenetin short you could have a custom impl of some protocol fn bound to a value (via metadata) instead of a type#2018-11-0614:47ghadiin clojurescript this is like specify!#2018-11-0614:47mpenetwell the number mention is not a good example šŸ™‚ no meta#2018-11-0614:48kirill.salykincant you dispatch now based on meta?#2018-11-0614:48kirill.salykinsorry for being annoying, but I am very curriuos about it šŸ™‚#2018-11-0614:48mpenet
(defprotocol Foo (foo [x])) (foo (with-meta [42] {`foo (fn [x] :boo)})) -> :boo
#2018-11-0614:49mpenethttps://github.com/clojure/clojure/blob/master/changes.md#22-protocol-extension-by-metadata#2018-11-0614:49kirill.salykini will have a look, thanks!#2018-11-0614:50mpenetsuper cool feature (not really related to spec tho, as far as we know)#2018-11-0614:56borkdudewell, I thought, maybe this way predicates could be bound to keywords and spec could use that#2018-11-0614:57mpenetkeywords do not support meta I believe#2018-11-0614:57borkdudeah#2018-11-0614:58bronsathey don't, this is ~ specify limited to IObjs (modulo the different method resolution rules around direct impl)#2018-11-0615:49jumarsatisfies? doesn't work with the new "instance-based polymorphism"?#2018-11-0615:49mpenetapparently not#2018-11-0615:50bronsano, I was looking at a related fix for CLJ-1814 but I think somebody should make a ticket just for this specific issue#2018-11-0615:51ghadi^^ yes, please.#2018-11-0615:53jumarOk, I can do it.#2018-11-0616:04jumarHere's the thing: https://dev.clojure.org/jira/browse/CLJ-2426 It's my first ticket so I hope it's clear.#2018-11-0616:07ghadiit is, thanks @U06BE1L6T#2018-11-0618:06Alex Miller (Clojure team)yep, thx#2018-11-0617:27eoliphantHi, I'm trying to get a feel for where to 'draw the line' on more dynamic behavior in specs. I've created an EDN based data description language for one of my projects. I have it spec'ed out pretty thoroughly, but I have a situation where I have say :data/type that could be one of a fixed set of types (:string :long, etc) or a type that's defined elsewhere in the larger data structure I'm validating. my code is already doing some additional validation based on core.logic etc once the definitions are loaded. So just wondering if it's better to defer this type of check to that point#2018-11-0618:20mathpunkIs there an equivalent of ns-unmap for a symbol you used spec/def on? (I suspect this question will reveal an error in my mental model of how symbols and specs relate)#2018-11-0618:21didibusJust s/def it to nil#2018-11-0618:21mathpunkšŸ†’ thanks!#2018-11-0618:21didibusIf I remember correctly#2018-11-0619:15Alex Miller (Clojure team)yes#2018-11-0713:38danielstocktonNot sure how to achieve something. If I want to check a map for :key, but define the spec with ::some-type-of-key (different spec name) in (s/keys :req-un [::key])?#2018-11-0713:39danielstocktonMore concretely, I have a :type key on multiple things, and the spec is slightly different for each.#2018-11-0713:39danielstocktonMore concretely, I have a :type key on multiple things, and the spec is slightly different for each.#2018-11-0713:46jumarYou should probably use namespace-qualified keys in that case (that is :req) You can also use a level of indirection and define specs like this:
(s/def :a/type int?)
(s/def ::a-type (s/keys :req-un [:a/type]))

(s/def :b/type string?)
(s/def ::b-type (s/keys :req-un [:b/type]))

(def int {:type 10})
(def b {:type "number"})

(s/valid? ::a-type a)
;; => true
(s/valid? ::a-type b)
;; => false

(s/valid? ::b-type b)
;; => true
(s/valid? ::b-type a)
;; => false

#2018-11-0713:51danielstocktonNamespaced keys works, thanks#2018-11-0719:35shaun-mahood@alexmiller: I'm doing my first really rigorous spec of something with a lot of data, and this morning I'm on my 4th or 5th time understanding why a certain design decision makes more sense than I realized with either smaller data or less complete specs. It's pretty great to be using something with so much depth of thought and experience behind it!#2018-11-0809:51danielstocktonCan anyone suggest how i might spec that a vector should contain maps in a particular sort order? (based on a key)#2018-11-0809:53danielstocktonIt might contain any number of maps, but they should be in a particular order based on a :type key.#2018-11-0810:37borkdude@danielcompton use this predicate?
(defn sorted-by? [coll k]
    (and (reduce (fn [prev m]
                   (if (pos? (compare (get prev k) (get m k)))
                     (reduced false)
                     m)) coll)
         true))
#2018-11-0811:08danielstocktonWrong @ šŸ˜‰#2018-11-0811:09borkdudeah sorry šŸ™‚#2018-11-0810:40borkdudeIā€™m not sure if predicates could give more information to spec to present a more helpful error than ::s/invalid#2018-11-0810:41danielstocktonYeah, i'll need a different comparator but the basic idea would work. Thanks!#2018-11-0810:42borkdude
(sorted-by? [{:type 2} {:type 3} {:type 3}] :type) ;; true
(sorted-by? [{:type 2} {:type 3} {:type 2}] :type) ;; false
#2018-11-0810:43danielstocktonType is a string and the order is quite specific. But I can easily work that part out, thanks.#2018-11-0810:44danielstocktonI'll use a higher order function like
(defn correctly-sorted? [sort-order k]
  (fn [coll] ...))
#2018-11-0810:59danielstockton
(defn correctly-sorted? [sort-order k]
  (fn [coll]
    (reduce
     (fn [prev m]
       (if (> (.indexOf sort-order (get prev k))
              (.indexOf sort-order (get m k)))
         (reduced false)
         m))
     coll)))
Does it matter if the returned value is simply truthy rather than true?
#2018-11-0811:00borkdudedepends what you want. itā€™s a convention to return booleans from ? functions.#2018-11-0811:01borkdudenote that youā€™re using > which doesnā€™t work for the case when you have equal elements with regards to k#2018-11-0811:01borkdudestylistically: correctly- is a bit redundant, itā€™s either sorted, or not#2018-11-0811:03danielstocktonThanks, you're right on all fronts..#2018-11-0811:04borkdudehaha, didnā€™t mean to be pedantic, but couldnā€™t help šŸ˜‰#2018-11-0811:04danielstocktonProblem is sorted? is a core function#2018-11-0811:04borkdudesorted-by? isnā€™t#2018-11-0811:05borkdudebut this is just naming, choose whatever you want šŸ˜‰#2018-11-0811:05danielstocktonsorted-accordingly? šŸ˜› sorted-by? is just fine#2018-11-0811:18Matt ButlerHi, I'm trying to spec a list of different maps, based on the docs I think I'm supposed to use a multi-spec, which i've done in the past without trouble. However I want to dispatch on more than one key/value, which I haven't seen an example of. Is this supported? I'm doing this, and valid? works correctly (returns true). But explain returns something unexpected so wanted to check im not doing something wrong
(defmulti foo
  (fn [x]
    [(:bar x) (:baz x)]))

(defmethod foo [1 2] [_]
  #{{:bar 1 :baz 2 :biff 3}})

(defmethod foo [:a :b] [_]
 #{{:bar :a :baz :b :biff :c}})

(s/explain (s/multi-spec foo :foo/type) {:bar :a :baz :b :biff :c}) => val: {:bar :a, :baz :b, :biff :c} fails at: [[:a :b]] predicate: foo
(s/valid? (s/multi-spec foo :foo/type) {:bar :a :baz :b :biff :c}) => true
I suspect its because I've just put something random :foo/type in the retag arg.
#2018-11-0812:36Matt ButlerSo it appears that multispec doesn't conform the set to a spec for you, which was causing the inconsistency. Returning (s/spec #{{:bar :a :baz :b :biff :c}}) from the methods, now keeps both explain and valid happy.#2018-11-0811:22Matt ButlerOn a slightly unrelated note, is it considered back practice to transform data before checking it against a spec? I have the same data but in 2 different formats and i'd prefer to not to spec each one individually. I think I can write some fairly simple code to transform 1 into the other and then check it against a single spec. [a b c] => {:foo a :bar b :baz c}#2018-11-0813:16Alex Miller (Clojure team)Doesnā€™t seem inherently bad to me#2018-11-0917:22Lyn HeadleyIs it the case that cljs.spec.gen.alpha does not support recursive-gen? I am seeing: ^--- Use of undeclared Var cljs.spec.gen.alpha/recursive-gen#2018-11-0920:50hiredmanrecursive-gen is a thing that test.check has, but I don't think spec exposes a wrapped version of it#2018-11-0920:51hiredman(in cljs or clj)#2018-11-1006:39tianshuI have a question, is it s/conformer not work on s/map-of? is it designed?#2018-11-1016:57schmeeIā€™m very confused by the behavior of instrument and fspec#2018-11-1016:57schmeelong story short: how do I turn off generative checking of fspec?#2018-11-1016:58schmee*fspec-iterations* doesnā€™t cut the mustard since it will try to create generators even though you set it to 0#2018-11-1016:59schmeemy workaround now is to copy-paste the spec in question and replace the fspecs with any? and pass that to the :spec arg to instrument, but that is not a viable solution#2018-11-1017:03schmeeis it an acceptable solution to never call validate-fn at all if *fspec-iterations* is zero or nil? https://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/alpha.clj#L1756#2018-11-1023:52didibusAre you saying a call to instrument forces the gen to be generated?#2018-11-1023:55schmeethat seems to be the case#2018-11-1023:57didibusHum, let me check if I have that behavior#2018-11-1100:04didibusI don't get that behavior#2018-11-1100:05didibus#2018-11-1100:06didibusThis works fine for example. Even though if you run:
(sgen/generate (s/gen `add))
you get ExceptionInfo Unable to construct gen at: [:a] for: (= (type %) java.lang.Long) clojure.core/ex-info (core.clj:4739)
#2018-11-1100:13Alex Miller (Clojure team)when an instrumented fdef with an fspec is checked, the :args spec of the fspec is generated and then the fspec function is invoked to verify it conforms to its spec#2018-11-1100:14Alex Miller (Clojure team)many people find this surprising and I expect we will ultimately change it#2018-11-1100:19schmeewhat do you think of my suggestion above as a temporary measure (to not call validate-fn when *fspec-iterations* is 0 or nil)?#2018-11-1100:20schmeealso, I appreciate that you are considering options for this šŸ™‚#2018-11-1100:23didibusOh I see, you mean if you spec a higher order function on an fdef. Interesting, I don't think I ever specced one before.#2018-11-1219:23asilverman#clojure-spec - hi all, could you help me understand why
(inst? (t/date-time 1986 10 14))
=> false
#2018-11-1219:24schmeeI guess t/ is clj-time/?#2018-11-1219:35asilvermanYes, t -> https://github.com/clj-time/clj-time#2018-11-1219:26ghadi@ariel.silverman (class (t/date-time 1986 10 14)) would be some third-party Joda time thing, which isn't in scope of the inst? predicate#2018-11-1219:38asilverman@ferossgp @schmee - fair enough. How do I sample a joda time thing then?#2018-11-1219:54asilverman#clojure-spec - I was hoping to ask for some help generating samples for clj-time/date-time since I am currently interested in testing my spec and using the spec as a generator for unit tests. Can anyone spare a couple minutes to help or point me out to some resource that can assist me in this goal?#2018-11-1220:02schmeesomething like this should get you going:
(ns foo
  (:require [clojure.spec :as s]
            [clojure.spec.gen :as gen]
            [clojure.test.check.generators :as tg]))
            [clj-time.coerce :as c]

(def epoch-2000-01-01 946684800)

(def epoch-2100-01-01 4102444800)

(def date-range
  (tg/large-integer* {:min epoch-2000-01-01 :max epoch-2100-01-01}))

(s/def ::timestamp
  (s/spec nat-int?
          :gen #(tg/fmap #(c/from-long %) (tg/large-integer* {:min epoch-2000-01-01 :max epoch-2100-01-01})))
#2018-11-1220:34asilverman@schmee Thank you! Actually I was looking into leveraging the specs specified here as part of my spec definition: https://github.com/clj-time/clj-time/blob/master/src/clj_time/spec.clj#2018-11-1220:35asilvermanI was able to require the namespace clj-time.spec and now I am trying to figure out how to do the dynamic binding of *period*#2018-11-1219:59schmee@ariel.silverman I have a one around for that, secā€¦#2018-11-1320:29DormoDo people tend to keep spec instrumentation on in testing environments? I don't mean automated testing, but environments for manual/internal testing before being deployed to production#2018-11-1320:30Dormoi guess "staging environments" would be a good term#2018-11-1321:56souenzzoI use just on automated test (lein test)#2018-11-1321:56souenzzoThe "develop" jar should behave the same as "prod" jar.#2018-11-1511:01stijnHow can I create a generator that either generates something of a type or nil? (e.g. (spec/gen (spec/inst-in start end)) or nil)#2018-11-1511:04borkdude@stijn the either part you can do with s/or or s/alt#2018-11-1511:04borkdudeor use nilable#2018-11-1511:05stijnoh, of course šŸ™‚#2018-11-1511:05stijnthanks!#2018-11-1514:43borkdudewhatā€™s the minimum clojure version in which spec can be used, 1.9.0?#2018-11-1514:51mpenet1.8 if you use clojure-future-spec#2018-11-1514:52mpenetprolly lower too, but we ran on 1.8 + clojure-future-spec smoothly for a while on some services#2018-11-1515:06borkdudeah yes, now that you remind me, we did too#2018-11-1516:19Wilson VelezHi all, Where can I start to learn spec?#2018-11-1516:19samedhihttps://clojure.org/guides/spec#2018-11-1516:19samedhihttps://clojure.github.io/spec.alpha/clojure.spec.gen.alpha-api.html#clojure.spec.gen.alpha/fmap#2018-11-1516:47eoliphanthi one of my devs is playing with spec, and ran into this issue when composing a couple
(s/def ::even-or-odd (s/or
                       :even even?
                       :odd odd?))

(s/def ::small-even-odd (s/and ::even-or-odd #(< % 100)))

(s/valid? ::small-even-odd 1)
; causes
=> ClassCastException clojure.lang.MapEntry cannot be cast to java.lang.Number   (Numbers.java:226)
It seems like the anon function is getting a KV pair instead of the value thatā€™s being checked.
#2018-11-1516:51borkdude@eoliphant whatā€™s the result of (s/conform ::even-or-odd 1)?#2018-11-1516:52eoliphantthatā€™s what I was thinking, but i didnā€™t realize that that would ā€˜carry forwardā€™ into the next spec#2018-11-1516:52borkdudemaybe try nonconforming around the spec#2018-11-1516:52borkdudehttps://clojuredocs.org/clojure.spec.alpha/nonconforming#2018-11-1516:57borkdudehttps://clojurians-log.clojureverse.org/clojure-spec/2017-08-12/1502573905.650871#2018-11-1516:57borkdudewonder what the current state of that is#2018-11-1516:59eoliphantthanks#2018-11-1517:00eoliphantyeah to your point itā€™s a tuple [:odd 1]#2018-11-1518:12jaihindhreddyWell, you could flip the order. (s/def ::small-even-odd (s/and #(< % 100) ::even-or-odd))#2018-11-1518:37Alex Miller (Clojure team)wonā€™t gen then#2018-11-1518:38Alex Miller (Clojure team)actually, I guess it wonā€™t gen regardless#2018-11-1518:38Alex Miller (Clojure team)you need an int? at the beginning or something#2018-11-1519:31devthif i have two functions in the same namespace that i want to spec with req-un with the same named arg e.g. match but different shapes, must I put those specs in other namespaces in order to specify the different shapes?#2018-11-1519:34devthsince the spec must be named match and I can't specify two specs with the same name, e.g.:
(s/def ::match string?)

...

(s/def ::match number?)
#2018-11-1519:35noisesmithif they are req-un why not just give them different namespaces?#2018-11-1519:36devthinstead of :: you mean?#2018-11-1519:36devthlike :fun1/match :fun2/match?#2018-11-1519:36noisesmithright - :: is a shorthand for the current namespace#2018-11-1519:37devthok. guess i wasn't sure if it's appropriate to abandon the :: convention since all my other specs in that namespace are using it šŸ¤”#2018-11-1519:37devthalmost seems like that makes them private/internal specs#2018-11-1519:38Alex Miller (Clojure team)itā€™s ok and fine to do that#2018-11-1519:38devthgot it. thanks :+1:#2018-11-1519:38Alex Miller (Clojure team)fyi, changes coming in this area for next version of spec that should make it less weird#2018-11-1519:38noisesmithI've never seen any assumption or convention that specs that don't use the current namespace are private#2018-11-1519:42borkdudeanother way to mark them private is to move them to an foo/impl.clj namespace šŸ™‚#2018-11-1519:42devthis that a java thing? šŸ™‚#2018-11-1519:43borkdudepretty general convention#2018-11-1519:43noisesmithit's a convention in many clojure projects, if the ns has impl as part of its path you know you aren't expected to rely on it as a consumer#2018-11-1519:43Alex Miller (Clojure team)core.async is a good example#2018-11-1519:43devthah, cool#2018-11-1519:44seancorfield@alexmiller Are you aware that clojure.spec-alpha2.test/check is "broken" at the moment?#2018-11-1519:45Alex Miller (Clojure team)yes#2018-11-1519:45Alex Miller (Clojure team)and I will fix it as soon as I finish getting the next Clojure released :)#2018-11-1519:45Alex Miller (Clojure team)beta that is#2018-11-1519:45seancorfield'kthx#2018-11-1519:46seancorfieldI tried to fix it locally but I haven't quiet gotten my head around the path through the code to figure out exactly what needs changing... yet...#2018-11-1519:47Alex Miller (Clojure team)I did not intentionally change anything around this, so itā€™s something I (unintentionally) broke, but havenā€™t looked yet#2018-11-1519:50seancorfieldI think it's because stest/check still, ultimately, calls s/spec (a macro) and this line in check-1 no longer behaves the same as before:
specd (s/spec spec)]
#2018-11-1519:51seancorfield(I just haven't quite figured out the correct invocation to replace that with)#2018-11-1520:05Alex Miller (Clojure team)prob a bug in the s/spec impl#2018-11-1521:20seancorfieldFYI, I tried switching our code over to clojure.spec-alpha2 because, why not? I ran into two speed bumps (so far)...#2018-11-1521:21seancorfield1. we reuse :clojure.core.specs.alpha/binding in one of our specs (I got around that by using eval to s/def that spec into the new spec's registry).#2018-11-1521:22seancorfield2. we use A spec that uses test.chuck fails with
Caused by: java.lang.IllegalArgumentException: No method in multimethod 'create-spec' for dispatch value: ws.domain.member/bounded-string
#2018-11-1521:31seancorfieldNot yet sure what causes that.#2018-11-1521:32seancorfield
(s/def ::email (s/with-gen (s/and (bounded-string 5 128)
                                  wstr/valid-email?)
                 (wgen/fn-string-from-regex wstr/email-regex)))
#2018-11-1521:32seancorfield
(defn bounded-string
  "Given min/max lengths, return a spec for a string within
  those lengths (inclusive).)"
  [from to]
  (s/and string? #(<= from (count %) to)))
#2018-11-1521:36seancorfieldI have a feeling this is related to the stest/check failure since it seems to be due to an assumption that a form like (a-fn ...) can be dispatched on the first argument, rather than evaluating it and dispatching on the result?#2018-11-1521:38Alex Miller (Clojure team)yeah, might be missing a case there#2018-11-1521:55seancorfieldIf it's any help, I tried this
(defn bounded-string
  "Given min/max lengths, return a spec for a string within
  those lengths (inclusive).)"
  [from to]
  (s/and string? #(<= from (count %) to)))
(def ^:private email-bounded-string? (bounded-string 5 128))
and it failed with
Exception in thread "main" Syntax error compiling at (ws/domain/member.clj:87:1).
Unable to resolve symbol: from in this context
so at this point I'll give up šŸ™‚
#2018-11-1613:51stijnis there a way to do something like this (s/def ::string-or-number (apply s/or [:string string? :number number?])) or is it macros to the rescue for e.g. creating a combined spec from a list of regexes?#2018-11-1613:55mpenetyou can do this with eval#2018-11-1613:56mpenet
(eval `(s/or 
#2018-11-1613:56mpenetetc#2018-11-1613:57stijn@mpenet thanks!#2018-11-1613:59borkdude@stijn apparently (s/def ::string-or-number (s/or # also works#2018-11-1614:00borkdude(ab)using reader conditionals#2018-11-1614:08Alex Miller (Clojure team)thatā€™s gross#2018-11-1614:08Alex Miller (Clojure team)this stuff is better in spec-alpha2#2018-11-1614:08Alex Miller (Clojure team)but donā€™t have time to talk about it#2018-11-1614:09mpenetyep hairy#2018-11-1614:09mpeneteager to see the new stuff#2018-11-1614:25mhuebertWhen using fdef, are the arguments validated by :args supposed to be affected by the destructuring forms present in the function argslist?
#2018-11-1614:26taylorno#2018-11-1614:26mhueberteg. in the following, the fact that I am using & [doc] to read the 2nd arg seems to affect how instrument works
(defn register-key
  [key & [doc]])

(s/def ::register-args (s/cat :key keyword?
                              :doc (s/? string?)))

(s/fdef register-key
        :args ::register-args)

(st/instrument)

(s/valid? ::register-args
          [:hello "there"])
;; => true

(register-key :hello "there")
;; => throws
#2018-11-1614:28taylorhere's an example I have for a similar variadic function:
(defn slarp [path & [blurp? glurf?]]
  (str path blurp? glurf?))
(s/fdef slarp
  :args (s/cat :path string?
               :rest (s/? (s/cat :blurp? int?
                                 :glurf? boolean?))))
(s/exercise-fn `slarp)
=>
([("") ""]
 [("" -1 false) "-1false"]
 [("e7" 0 true) "e70true"]
 [("du9") "du9"]
 [("K1" -6 false) "K1-6false"]
 [("op0" -3 false) "op0-3false"]
 [("2y" 2 false) "2y2false"]
 [("qaGWVK") "qaGWVK"]
 [("9h1Rh") "9h1Rh"]
 [("") ""])
#2018-11-1614:28mhuebertI think I found that same example in my searching#2018-11-1614:29taylorI think your example specs a fixed, 1- or 2-arity function instead of a variadic one that just destructures the first add'l arg#2018-11-1614:30taylorwhich is why it works when you rewrite the function with fixed 1- & 2-arity#2018-11-1614:33mhuebertinstrument also fails when I try to use it with your slarp example#2018-11-1614:33mhuebert
(defn slarp [path & [blurp? glurf?]]
  (str path blurp? glurf?))

(s/fdef slarp
        :args (s/cat :path string?
                     :rest (s/? (s/cat :blurp? int?
                                       :glurf? boolean?))))

(st/instrument)

(slarp "a" 1 true)
;; => throws
#2018-11-1614:34mhuebertI canā€™t say I understand why changing the destructuring form used by the function should affect how the :args spec works#2018-11-1614:35taylorthe difference is that one version is variadic and one isn't, not how you choose to destructure the variadic version#2018-11-1614:36taylorthe slarp example doesn't throw an exception (on my machine)#2018-11-1614:37mhuebertmaybe itā€™s a ClojureScript difference#2018-11-1614:37tayloryes, there is a difference there, I believes there's a JIRA for it#2018-11-1614:38taylorhttps://dev.clojure.org/jira/browse/CLJS-2793#2018-11-1614:38taylorlooks like it was recently fixed https://github.com/clojure/clojurescript/commit/4fb83eff87cc456600a3fd21c111e99a41c61285#2018-11-1614:41mhuebertah! ok#2018-11-1614:41taylorFWIW I'd go with the fixed-arity version regardless of CLJS spec bug :man-shrugging:#2018-11-1614:41mhuebertwhy?#2018-11-1614:41taylorit's less ambiguous about how many args the caller can pass#2018-11-1614:42mhuebertthe actual use case is more like (defn register-key [key & [doc options data]) where both doc and options are optional#2018-11-1614:43mhuebertiā€™d like to have them show up in argslist (for eldoc etc) but in smaller arities it isnā€™t known whether the 2nd option would be doc, options, or data#2018-11-1614:43mhuebertbut i just started writing in this way recently and wasnā€™t sure if it is a great idea#2018-11-1614:44taylorI might also consider kwargs-style variadic function in that case too#2018-11-1614:44taylor(defn register-key [key & {:keys [doc options data]})#2018-11-1614:44mhuebertthat would work.. i was hoping to keep it feeling more like def / defn#2018-11-1615:13mhuebertlooks like there is another bug with variable-arity#2018-11-1615:14mhuebert
(defn defx [key & [doc data]])

(s/def ::defx-args (s/cat :key keyword?
                          :doc (s/? string?)
                          :data map?))

(s/fdef defx
        :args ::defx-args)

(st/instrument)

(s/valid? ::defx-args [:a "" {}])
;; => true

(s/valid? ::defx-args [1 1 1])
;; => false

(defx :a "" {})
;; => not thrown (expected)

(defx 1 1 1)
;; => not thrown (but should throw)
#2018-11-1615:14mhuebertnow it only validates the length of the arg sequence but doesnā€™t validate the members#2018-11-1614:27mhuebertif I rewrite register-key in multi-arity form, the same spec passes, eg
(defn register-key
  ([key])
  ([key doc]))
#2018-11-1615:32pabloreHow do you spec xforms?#2018-11-1615:33pabloreie:
(def sort-dates
  (comp (map #(update % :date parse-date))
             (filter #(nil? (:date %)))
             (partial sort-by :date)))
#2018-11-1615:42borkdude@pablore for now we use ifn? in speculative, itā€™s a little (too) general thoughā€¦#2018-11-1615:44pabloreCan I still specify args and ret?#2018-11-1616:50jimbobdo i need to require the namespace of the specs im using?#2018-11-1616:50jimbobexample if in my ns i have :blah.spec/reward#2018-11-1616:50jimbobdo i need to require blah.spec?#2018-11-1616:50jimbobin the ns that calls it?#2018-11-1616:53borkdude@ben.borders no, but you need to call the ns in which the spec was defined to be able to use it#2018-11-1617:00jimbobthanks!#2018-11-1617:08jaihindhreddyI want to study the spec source and understand it. Should I study spec.alpha or spec-alpha2?#2018-11-1617:16seancorfield@jaihindh.reddy The only difference between those right now is the latter has the macro implementations refactored out as functions, that the macros then call.#2018-11-1617:17seancorfieldThere are several bugs in spec-alpha2 right now preventing it from actually being used but the code itself is fine to read for learning purposes... although I'm not quite sure what you'll really get from the source, as opposed to the guide and reference material...?#2018-11-1617:18seancorfield(like many parts of Clojure itself, the source is kinda gnarly and does not represent "typical" Clojure code nor, often, "best practice"... and that's fairly typical of a compiler and its runtime/standard library)#2018-11-1617:28jaihindhreddyI've recently started to read the Clojure source. Wanted to do that for spec too. Gotta say, it is pretty gnarly.#2018-11-1617:41jimbobstrange#2018-11-1617:41jimbobthis doesnt make sense to me#2018-11-1617:41jimbobgot a
java.lang.Exception: Unable to resolve spec: :facts.rewards.specs.reward-award/award
#2018-11-1617:42jimbobbut then i explicitly required the namespace#2018-11-1617:42jimbob
[facts.rewards.specs.reward-award]
#2018-11-1617:42jimboband now it works#2018-11-1617:42jimbobbut other specs do not require the spec namespace to be required?#2018-11-1617:42jimbobFWIW here is the spec:
(s/def ::awards (s/coll-of :facts.rewards.specs.reward-award/award))
#2018-11-1617:44schmeewhich specs are you referring to as ā€œother specsā€?#2018-11-1617:48jimbobthe other specs i have defined.. i dont want to enumerate all of them, but they dont require the namespace of the spec they are calling to be explitly required#2018-11-1617:48jimbobex:
(s/def ::qualification
  (s/and (s/keys :req-un [::action :facts.rewards.specs.qualification-conditions/conditions])
         #(qc/has-completeness-condition? %)))
#2018-11-1617:54schmeemaybe those namespaces are required somewhere else in the code?#2018-11-1617:55schmeespecs use a global registry, so as soon as a namespace is required in on place it will populate the registry with all its specs#2018-11-1617:58jimbobah so it needs to be required somewhere?#2018-11-1617:59jimbobotherwise its not populated in the registry#2018-11-1617:59jimbobso i must not be requiring the ns then in src code#2018-11-1707:02ikitommiIs there any high-level backlog or set of goals for the spec-alpha2? #2018-11-1722:47seancorfieldBased on the changes so far (compared to spec.alpha), it adds a function-based interface to allow programmatic construction of specs. Which people have asked for a lot.#2018-11-1909:45stijnhow do people practically deal with entity maps that have required keys in one situation, but don't in another situation?#2018-11-1909:48didibusMulti-spec#2018-11-1909:46stijnI understand you'll probably have to create a separate spec, but that might require duplicating the entire tree of references to other entities, no?#2018-11-1909:50borkdude@stijn best to come up with an example#2018-11-1910:17pyrHi, I'm trying to spec a map (over which I dont' have control) which uses an inconvenient idiom: for a number of members it accepts either a :membername or :memberid key, but at least one must be present. If there's a single member like this, spec/or is convenient. I naively started producing a number of spec/or specs (of the sort: (spec/def :member (spec/or :by-id (spec/keys :req-un [:member/memberid]) :by-name (spec/keys :req-un [:member/membername])))) but if I merge those with spec/merge or spec/and validation becomes impossible#2018-11-1910:19pyrLet's say that I have (spec/merge ::member1 ::member2) The input for validation of member2 seems to be the output of conform for member1, which becomes a tuple [:by-id val] instead of plain val. Beyond doing some sort of pre processing of the map, any idea of how to address this from spec?#2018-11-1910:23mpenetcan't you use (s/keys :req-un [(or ::membername ::memberid)]) ?#2018-11-1910:24pyror as in spec/or?#2018-11-1910:24pyrif so, I didn't try, let me try it#2018-11-1910:25mpenetas in or within keys#2018-11-1910:25mpenetit supports that#2018-11-1910:31mpenetit also supports and, kind of a cool feature#2018-11-1910:31mpenetfrom the docstring: (s/keys :req [::x ::y (or ::secret (and ::user ::pwd))] :opt [::z])#2018-11-1910:49pyr@mpenet thanks, never knew about that#2018-11-1910:49borkdude@mpenet wow, thatā€™s cool#2018-11-1910:50borkdudeI think Alex said this too before, but it was still an undocumented feature#2018-11-1910:54mpenetIt's in the docstring and the guide#2018-11-1910:54borkdudeah, that must have changed then or I misremember#2018-11-1910:54mpenetFrom day one i think no? #2018-11-1910:56pyr@mpenet here it fails on 'all keys must be ns qualified'#2018-11-1910:56borkdude@mpenet where in the guide is this?#2018-11-1912:51mpenethttps://clojure.org/about/spec#_overview#2018-11-1912:51mpenetthat was posted when spec was introduced#2018-11-1912:51borkdudeI see it now. Not the guide, but some post. Thanks!#2018-11-1912:52mpenetlinked there: http://blog.cognitect.com/blog/2016/5/23/introducing-clojurespec#2018-11-1912:52borkdude:+1:#2018-11-1910:56pyr(clojure 1.9.0)#2018-11-1910:58borkdudeI see the docstring, so thatā€™s definitely supported#2018-11-1910:59mpenetI had typos in my example, I am fairly sure it works with both req-un or req (not in front of laptop atm) #2018-11-1910:59pyrah got it#2018-11-1910:59pyrI left one in opt-*#2018-11-1911:00pyrperfect, thanks @mpenet šŸ™‚#2018-11-1912:52stijn@borkdude example would be: a Person entity in one case has required fields :person/name :person/birth-date :person/address, and Address has required fields :address/street :address/city, but in another case I want a person with required keys :person/social-security-number and :person/address, but the Address has no required fields. How do you model that? I can define a spec for Person type 1 and one for Person type 2, but both will have a required key :person/address, but the spec for :person/address is different in these two entity map specs.#2018-11-2715:15urzdsBeen discussing the exact same issue just now in #graphql: https://clojurians.slack.com/archives/C4DLZKR9T/p1543316717071700#2018-11-3014:07stijnhttps://www.youtube.com/watch?v=YR5WdGrpoug#2018-11-1912:56borkdudeare all these keys in the same map, or is address in its own map?#2018-11-1912:56stijnaddress in its own map#2018-11-1915:08borkdude@alexmiller I have ran code with a spec for assoc instrumented. it went from 3s to 15 minutes (approx.). I narrowed it down to this: https://gist.github.com/borkdude/54baa8504064779094d7dd35f10865d7#file-patch-clj-L64 itā€™s about 400x slower than running without instrumentation#2018-11-1915:17Alex Miller (Clojure team)well, assoc is used a lot#2018-11-1915:18Alex Miller (Clojure team)so, donā€™t do that?#2018-11-1915:18Alex Miller (Clojure team)Iā€™m not sure what youā€™re looking for from me#2018-11-1915:19borkdudeit seems to be expensive to validate specs in general: https://gist.github.com/borkdude/54baa8504064779094d7dd35f10865d7#file-patch-clj-L62#2018-11-1915:20Alex Miller (Clojure team)I think itā€™s not expensive in general, itā€™s expensive to have spec for things used by Clojure to do everything else#2018-11-1915:20Alex Miller (Clojure team)because the validation is then also checked during validation#2018-11-1915:21borkdudeto be clear, I specā€™ed a function like assoc, not assoc itself#2018-11-1915:21borkdudeso I made sure that it wouldnā€™t be used internally#2018-11-1915:21borkdude(def my-assoc assoc) and then specā€™ed my-assoc#2018-11-1915:23Alex Miller (Clojure team)given that youā€™re doing these a million times each, they seem pretty fast to me? youā€™re talking about microseconds per?#2018-11-1915:23Alex Miller (Clojure team)if you could provide a narrative rather than just a wall of data, that would help#2018-11-1915:24borkduderight. I instrumented an Advent of Code puzzle and the time went from 3s to 15 minutes. That seemed a little but much for instrumentation overhead.#2018-11-1915:24borkdudebut maybe thatā€™s just the way it is#2018-11-1915:24Alex Miller (Clojure team)well, it depends what itā€™s doing and what you specā€™ed#2018-11-1915:25Alex Miller (Clojure team)if you instrument something in a hot loopā€¦. then that will be slow#2018-11-1915:25Alex Miller (Clojure team)https://muhuk.github.io/validation-benchmark/#2018-11-1915:25Alex Miller (Clojure team)was an effort to compare different fws#2018-11-1915:25Alex Miller (Clojure team)at a micro level and spec did pretty well there#2018-11-2006:14didibus@borkdude 's benchmark is showing that instrument is much slower then valid? The benchmark you linked used valid? Have an idea why instrument would be slower then valid?#2018-11-2006:30borkdudeNot much slower. It does a little bit more.#2018-11-2006:49didibusRight, from the impl, it does seem it uses conform to validate. Even though it doesn't seem to do anything with the conformed value. I wonder why its not just using valid?#2018-11-2006:53didibusOh, valid also uses conform. Ok, well I'm learning a few things looking at the impl šŸ˜›#2018-11-1915:26rickmoynihanWhat are the big changes in spec-alpha2? Iā€™m assuming there are breaking changes, given the artifact has changed names.#2018-11-1915:27Alex Miller (Clojure team)at the moment, the public api is the same#2018-11-1915:27Alex Miller (Clojure team)there are some additions and likely breaking changes down the line#2018-11-1915:28rickmoynihanok so the repackaging is mostly precautionary? i.e. in case you were to push a breaking change?#2018-11-1915:29Alex Miller (Clojure team)it is currently a work in progress, weā€™ll have more to talk about it as we get to a release point#2018-11-1915:30rickmoynihanUnderstood.#2018-11-1915:26borkdudecool, thanks. yes, the code was a hot loop. I also tested with spec-alpha2 and saw about the same numbers.#2018-11-1915:27borkdudeis spec itself also compiled with direct linking?#2018-11-1915:28Alex Miller (Clojure team)no#2018-11-1915:28Alex Miller (Clojure team)itā€™s not compiled#2018-11-1915:28borkdudeah, that explains something#2018-11-1915:29Alex Miller (Clojure team)thereā€™s some trickiness in the circularity between clojure and spec.alpha#2018-11-1915:29Alex Miller (Clojure team)and compilation makes that trickier than itā€™s worth#2018-11-1915:29borkdudeno problem. thanks for the info#2018-11-2000:36lwhortonis it possible to express ā€œa map with either key A or key B, but not neitherā€?#2018-11-2000:38lwhorton(without unioning the value of the key under something like s/keys :req [:a/foo] where s/def :a/foo (s/or :b :a/bar :baz :a/baz) (since i want the keyā€™s value to be either :bar or :bar and not :foo)#2018-11-2000:41lwhortonoh, lol. :req [(or :bar :baz)])?#2018-11-2004:01favilaYes.#2018-11-2008:41roklenarcicIs there any difference between: (s/& (s/cat .... and (s/and (s/cat ...#2018-11-2012:38Alex Miller (Clojure team)& is a regex op that nests with other regex ops#2018-11-2008:47ChengI read some spec impl code, I see it using a global store for these specs. I think spec is more of meta data of data, why it cannot implemented via meta data? Make it a meta system?#2018-11-2009:01roklenarcicmetadata of what?#2018-11-2009:02roklenarcicWhen I define a spec for a keyword, where exactly would you attach the metadata?#2018-11-2009:05schmee@xfcjscn909 https://clojure.org/about/spec#_don_t_further_add_to_overload_the_reified_namespaces_of_clojure#2018-11-2009:06schmee@roklenarcic s/& is for spec regexes while s/and is for general predicates#2018-11-2009:08didibusWell, specs are not tied to a var, so meta wouldn't really make sense. They are tied to values. The keyword or symbol is just a name for the spec#2018-11-2009:09schmeefspecs are tied to vars#2018-11-2009:09didibusNo they're not#2018-11-2009:09schmeeor, it can be at least šŸ™‚#2018-11-2009:09didibusWell, you can relate them#2018-11-2009:12didibusThat's why check-fn takes a function and a spec seperate. Because you could use any fspec to spec to check any function#2018-11-2009:14schmeeyeah, I was thinking of s/fdef#2018-11-2009:15didibusYa, I mean, in practice, for functions, its convenient to use the same symbol to point to your function and its spec.#2018-11-2009:15didibusSo in that case, you could argue Specs could have just been meta on the function#2018-11-2009:15didibusOr its var#2018-11-2009:16didibusBut, when your speccing data, that wouldn't have made sense#2018-11-2009:16didibusSo that's why a registry is more logical#2018-11-2009:17didibusAlso, now you can define specs before you define functions.#2018-11-2009:43Cheng@schmee i am in china, now i cannot open that url, i will check that article later, thanks for the information. #2018-11-2009:43Chengi think spec is meta data in essence, whatever it currently implemented via meta mechanical or not. #2018-11-2009:45Chengi think the problem is keyword should support meta data as we are associating meta data to it. #2018-11-2012:41Alex Miller (Clojure team)Keywords are interned and reused so cant support metadata #2018-11-2009:46Chengi think now itā€™s a workaround to impl meta data in global store, is is really a good practice to do so? #2018-11-2009:49didibusIt depends what the meta is about. Like if you want to specify additional things about a Var then you can add that to the Var meta. Same for anything that supports meta.#2018-11-2009:49didibusBut Spec is meta about values#2018-11-2009:49didibusAnd values can not have meta attached to them directly. A value can be an int for example#2018-11-2009:50didibusSo for example, this is a spec #(> % 10)#2018-11-2009:51didibusIt doesn't need to be attached to anything to be one#2018-11-2009:52didibusSo say you want to give that spec a name, you can do s/def ::greater-then-ten #(> % 10)#2018-11-2009:52didibusYou did not add meta to the keyword ::greater-then-ten here. This spec is not metadata about the keyword.#2018-11-2009:53didibusIt is just a specification of values that you named ::greater-then-ten#2018-11-2009:53didibusNow you can do: s/valid? ::greater-then-ten 15) for example#2018-11-2009:55didibusSo I'm not sure you should think of Specs as meta data. It isn't really meta data in the normal sense. Like, yes, in this case, you can say that 15 conforms to the ::greater-then-ten spec. And so, ::greater-then-ten can be seen as metadata about the value 15. But again, meta about values is not really a concrete thing.#2018-11-2010:01Cheng@didibus thx for the explanation #2018-11-2010:06Chengin my opinion there are plenty of problems about current spec impl, for example: global store for spec, use keyword as spec name, mix keyword with spec, instrument is a state changer...#2018-11-2010:07Chengoverall, itā€™s not FP style#2018-11-2010:07Chengitā€™s OO style #2018-11-2010:08mpenetfor sure there are issues but it's not related to OO or fp at all#2018-11-2010:09mpenetvars are "global", shoving everything in them would be the same ultimately, you just move the bloat somewhere else. A separate registry is a good thing#2018-11-2010:10mpenetask youself: where is the info about the Types "stored" (if that even makes sense) in haskell: it's not a first class thing#2018-11-2010:10borkdudeeven in Haskell programs they use global mutable cells, IORef, MVar, TVar, etc.#2018-11-2010:12mpenetspec-alpha2 is supposed to fix quite a bit of the common complains, so I guess "wait and see"#2018-11-2010:20didibusā€¢ The global store has advantages too. It is simpler to manage a single global store, and namespaces should avoid clashes. ā€¢ You don't have to use keywords for spec names, you can also use symbols. But I think keywords was chosen as convention specifically to make it visually separate from Vars#2018-11-2010:21didibus* Instrument is used for testing only. And, it shouldn't be altering state. Think of it as aspect oriented programming if you know about that.#2018-11-2010:28Chenginstrument directly changed the function via byte code operate, i hope instrument can return a new function not directly change on existing function #2018-11-2010:29Chengi didnā€™t aware we can use symbol instead of keyword in a/def#2018-11-2010:30borkdudeinstrument doesnā€™t change the original function, it wraps it in a new function thatā€™s stored in the root of the var#2018-11-2010:30borkdudeon unstrument this is reverted#2018-11-2010:31Chengi donā€™t see itā€™s more easy to maintain a global store than separate vars #2018-11-2010:32borkdudean alternative could maybe have been via metadata on the var: instrumented? true but that would incur performance overhead when not using spec, because when calling a function via a var you would have to do this check#2018-11-2010:34didibusYa, instrument and unstrument also take a lock on the Var before changing it#2018-11-2010:35didibusBut it does mean you can not have an instrumented fn in one thread, and an uninstrumented one in another. Though that almost sounds like a positive in this case.#2018-11-2010:37borkdudeyou can have with-instrument-disabled per thread maybe, since itā€™s a dynamic var?#2018-11-2010:40didibusOh, that's true, I just saw the instrumented code checks for a dynamic var to decide to validate or not. So I guess you could in fact do that.#2018-11-2010:42didibusHum, that gives me an idea for when I alter-var-root things in my tests. I could instrument things to be "mockable", and then have a dynamic binding pointing to a mock, and this different mocks per thread and my tests would work in parallel.#2018-11-2010:38didibus@xfcjscn909 If you wanted non global stores, you'd have to first create a spec registry and put it somewhere in scope. And if it wasn't in scope anymore, it would stop working. And then as soon as you have more than one, you might forget which registry holds the spec you are looking for. All spec functions would take an additional arg for the registry to use. I mean, it would not be terrible, but I feel it would be a bit more inconvenient. And in practice, I never felt the need for separate registry. That's what namesapces are for on symbols and keywords.#2018-11-2010:39didibusIn effect, you would be using a Var as a namespace mechanism, instead of just using the existing namespace mechanism of keyword and symbols I feel.#2018-11-2010:39borkdudeClojure apps pretty much converged to ā€œone global atomā€ instead of ā€œmany smaller atomsā€, e.g. re-frame#2018-11-2010:39borkdudeitā€™s much easier to reason about#2018-11-2010:44didibusAlso I'd say, if you don't find that top level defs are a problem for defining Vars, in practice, Specs are defined in a similar fashion, so it'll be a similar experience.#2018-11-2010:53borkdudethe thing that would maybe problematic is if multiple libraries define specs for the same thing. e.g. lib1 defines a spec for clojure.core/merge and lib2 does too. currently you have no way to say: I want to exclude loading this fdef when requiring a namespace#2018-11-2010:54borkdudelast one wins#2018-11-2011:06didibusHum, right. I think I tend to use Spec much more around my data, and a lot less to spec functions. So I haven't faced these kind of issues very much.#2018-11-2011:08didibusNot yet at least#2018-11-2011:09borkdudeit hasnā€™t been a major problem, but it would be useful to me to see from which namespace a spec comes from and to conditionally instrument depending on the namespace where it came from.#2018-11-2011:11borkdudee.g. here I have some code only instrument/unstrument specs from speculative: https://github.com/slipset/speculative/blob/master/src/speculative/instrument.cljc but if the ā€œwhere did my spec come fromā€ feature existed, I didnā€™t have to#2018-11-2011:12didibusHum, interesting. fdef could probably capture that#2018-11-2011:13didibusProbably was not a feature they thought of#2018-11-2011:13borkdudeyeah and it could store it in the registry#2018-11-2011:15borkdudebut maybe in a future version of spec specs will have some form of metadata, so it could also happen there#2018-11-2011:16didibusYa, meta on spec would be great.#2018-11-2012:42Alex Miller (Clojure team)Itā€™s on the list#2018-11-2015:39martinklepschIs there a way to define a spec that matches the structure of a conformed value?#2018-11-2015:41martinklepschE.g. (s/cat :a string? :b string?) would conform to {:a "a" :b "b"}. Now I want to instrument a function so that it only takes values having :a and :b with values matching string?#2018-11-2015:47Alex Miller (Clojure team)automatically, no#2018-11-2016:00martinklepschok cool, thanks. Is the general expectation that once values are conformed they are no longer passed to functions that have spec'ed arguments? (I guess in most cases conformed values won't match the original spec)#2018-11-2016:02Alex Miller (Clojure team)correct#2018-11-2022:56DormoIs there a way to make a spec to check if a vector has either 1 or 2 values, where the first value is a keyword and the second value is "anything or nothing"? I was thinking this would work, but it does not: (s/def ::event (s/tuple keyword? (s/nilable any?)))#2018-11-2022:57Alex Miller (Clojure team)tuples are fixed size#2018-11-2022:58Alex Miller (Clojure team)(s/def ::event (s/cat :k keyword? :a (s/? any?)))#2018-11-2022:59DormoPerfect! Thanks!#2018-11-2023:00DormoThe explicit key names are actually a bonus as well.#2018-11-2023:01Alex Miller (Clojure team)the regex ops cat, alt, +, *, ?, etc are regex ops used to describe the structure of a sequential collection#2018-11-2300:29rgmis there a way to fspec that, say, 250+ functions all conform to the same shape? (Apologies if this is in the docs; my scanning and google-fu donā€™t quite know how to phrase my question quite right).#2018-11-2300:29rgmI guess I want something like (s/fdef-and-register-a-name ::function-shape ,,,)#2018-11-2300:30rgmand then loop over all my function names as (s/fdef-by-name my-var ::function-shape)#2018-11-2300:31rgmor maybe itā€™s just (map #(apply s/fdef % common-shape) ('my 'coll 'of 'function 'names))#2018-11-2300:31rgm(nvm, I think I just rolled my own)#2018-11-2300:32rgm(assuming common-shape is just '(:args ,,, :ret ,,, :fn ,,,) )#2018-11-2300:34rgm(argh, s/fdef is a macro not a fn)#2018-11-2301:58taylor@rgm maybe something like this would work:
(s/def ::fn-spec
  (s/fspec :args (s/cat :x number?)
           :ret number?))
(defn foo [x] (inc x))
(defn bar [x] (dec x))
(defmacro fdef-many [syms spec]
  `(do 
FWIW I think upcoming spec changes will make this easier
#2018-11-2302:10rgmOh neat, thanks. Iā€™ll give that a go.#2018-11-2309:37borkdudehttps://dev.clojure.org/jira/browse/CLJ-2443#2018-11-2523:45bbrinckFWIW, Expound wonā€™t work with spec instrumentation in CLJS 1.10.439 https://dev.clojure.org/jira/browse/CLJS-2913#2018-11-2700:49richiardiandreahello folks, it there a predicate for non-nil? I want to make sure that the key is there but it is not nil and I don't care about the value - can't use any? basically#2018-11-2700:50taylordoes some? work?#2018-11-2700:50richiardiandrearight šŸ˜„#2018-11-2700:51richiardiandreathank you @taylor forgot about that one šŸ™‚#2018-11-2700:53taylorjust checking it out locally b/c Iā€™ve never tried using some? for a spec:
(gen/sample (s/gen some?))
=> (() #{} . {} \8 {} {} [] [y5/S? false 5] :k-)
#2018-11-2700:55taylorand of course the built-in generators for any? and some? are very similar https://github.com/clojure/spec.alpha/blob/31165fec69ff86129a1ada8b3f50864922dfc88a/src/main/clojure/clojure/spec/gen/alpha.clj#L135-L136#2018-11-2715:25urzdsIs there a way to separate the spec of the structure of a data structure from the spec of the data, and allow different "models" to be used in different circumstances? I.e. have a :usergroup {::meetingpoint {...} ::users [{::address ... ::name ...} ...]} (with (s/def ::users (s/* ::user)) for the geo routing part of my application, but ::user {::email ... ::name ...} for the messaging part of my application, but let both parts share the non-structural specs of ::user's data? I.e. can I tell spec "validate this map with structural information X ... switch to a different component of my app ... now validate the same map with structural information Y", with both X and Y using the same leafs and the map being the same, using the same keywords as keys, just the structure being different? (Following up on @stijn's question from earlier and my own discussion with @orestis in #graphql.)#2018-11-2804:48valeraukoi'd split the specs#2018-11-2804:48valeraukoin the geo-relevant ns check with a ::geo-users and in the other with ::email-users#2018-11-2810:44misha@urzds and "structure" means "value of the keyword"? so you want shape (keys set) of the maps to be the same, but content (vals) be different? e.g. {::foo 1} and {::foo "str"}?#2018-11-2810:51mishaIf yes, and if you are limiting yourself to :: - just define everything (key specs and map specs) in 2 different namespaces. You'd have to define map specs twice (for your X and Y cases), since ::name will resolve to different keywords in each namespace: :my.ns.x/name and :may.ns.y/name, so your map specs would look similar, but resolve/expand into different ones, depending on namespace they are in: (s/keys :req [::name]) -> (s/keys :req [:my.ns.x/name]) and (s/keys :req [:my.ns.y/name]). I would not chase the goal to save few lines of text here, and just define it "twice". At least you'll keep "go to spec definition" feature or your IDE working.#2018-11-2812:17urzds@misha The other way round: The values should have the same requirements, but the structure / shape / keys required may be different.#2018-11-2812:19urzdsI always thought that if I (s/def ::foo ...), and validate a map that contains this exact fully qualified keyword, then spec would pick up the s/def automatically. And that this would stop working if the namespace part of the keyword would be different.#2018-11-2812:21urzdsThat's why I assumed that I can only define one map structure (at least using spec how I understood it), because eventually spec will validate the keys inside that map using the global definition for that key, instead of using the local structure that I would like to use in that specific place.#2018-11-2812:26urzdsOr will that start to work if I use :req-un / :opt-un instead of :req and :opt? I.e. can I define multiple structures / shapes using the -un variants, because the match in the map only uses the keyword name, but the structural / shape validation happens using the namespace+name? So a structure / shape that is defined using :req and :opt is enforced everywhere and defines the complete data structure, but if it is defined :req-un and :opt-un, other definitions might exist in other places that have different structural / shape requirements?#2018-11-2814:42favila@urzds Not sure I follow the convo here, but s/keys is about validating the presence and absence of keys on a map#2018-11-2814:44favilathe values of the keys are validated by the spec attached to that keyword#2018-11-2814:45favilas/keys does not require that every keyword mentioned have a spec#2018-11-2814:45favilaso you can require a key but not spec its value for e.g.#2018-11-2814:46favilaalso :req supports and and or for grouping requirements#2018-11-2814:46favila(and :req-un)#2018-11-2814:47urzdsGiven this data: {::foo {::bar 1 ::baz 2}}, how would you validate it two times using spec, such that for the first time {::foo {::bar ...}} is required and the 2nd time {::foo {::baz ...}} is required?#2018-11-2814:48favilayou cannot without a predicate#2018-11-2814:48favila::foo must always have the same spec#2018-11-2814:48urzdsBut in both cases, ::bar and ::baz should be int?, if they are present.#2018-11-2814:48favilawhat determines whether ::bar or ::baz is required?#2018-11-2814:49urzdsMe, depending on the context.#2018-11-2814:50mpenetspec of value of foo can be a multi-spec or a s/or#2018-11-2814:50favilayou either have to give up and get a least-common-denominator spec for ::foo (e.g. (s/keys :req [(or ::bar ::baz)]) or (s/keys :req [::bar ::baz])#2018-11-2814:51favilaor you have to assert your different requirements on the spec for the outer map using predicates that look inside ::foo#2018-11-2814:52favila(s/def ::foo-map-with-bar (s/and (s/keys :req [::foo]) #(-> % ::foo (contains? ::bar)))#2018-11-2814:53favilaspec has ironclad devotion to the principle that the keyword of an item in a map is its spec#2018-11-2814:53favilaso you can only spec ::foo once#2018-11-2814:54favilathere are a few ways to do this, but in the end you must do it in such a way that ::foo has only one spec#2018-11-2814:54favila(another possibility is making it a multispec and putting something on ::foo's map itself that determines if ::bar or ::baz is required)#2018-11-2814:55urzdsSo even if I spec (s/def ::foo-with-bar (s/keys :req-un [::bar] ::opt-un [::baz]) and (s/def ::foo-with-baz (s/keys :req-un [::baz] ::opt-un [::bar])) and use the appropriate one in the given context, that will not work?#2018-11-2814:55urzdsmulti-spec is if (s/def ... ...) has a 3rd argument?#2018-11-2814:56favilano multi-spec is a different concept#2018-11-2814:56faviladynamic dispatch to spec based on value#2018-11-2814:56urzdsDo you have a link to the library or docs on multi-spec?#2018-11-2814:57urzdsSorry, found it myself: https://clojure.org/guides/spec#_multi_spec#2018-11-2814:57favilaif you give your two foo uses different keywords your outer spec is just (s/keys :req [::foo-with-bar]) or (s/keys :req [::foo-with-baz]) and everything is fine#2018-11-2814:57favilathe problem is you want those two different specs, BUT ALSO as the value of a map whose key is ::foo#2018-11-2814:58favilaspec is steadfastly opposed to that#2018-11-2814:59favilahttps://clojure.org/about/spec#_map_specs_should_be_of_keysets_only#2018-11-2814:59favila^^ that is the principle stated formally#2018-11-2815:05urzdsThe ::foo-with-* examples were broken. Should look like this:
(ns common)
(s/def ::bar int?)
(s/def ::baz int?)

(ns with-bar :require [common])
(s/def ::map-with-bar (s/keys :req-un [::foo]))
(s/def ::foo (s/keys :req-un [::common/bar] ::opt-un [::common/baz]))
(s/valid? ::map-with-bar {:foo {:bar 1}})

(ns with-baz :require [common])
(s/def ::map-with-baz (s/keys :req-un [::foo]))
(s/def ::foo (s/keys :req-un [::common/baz] ::opt-un [::common/bar]))
(s/valid? ::map-with-baz {:foo {:baz 2}})
#2018-11-2815:06favilaunnamespacing the key is another possible workaround#2018-11-2815:06urzdsOK, seems they strongly disagree with me: > Defining specifications of every subset/union/intersection, and then redundantly stating the semantic of each key is both an antipattern and unworkable in the most dynamic cases.#2018-11-2815:07urzds> unnamespacing the key is another possible workaround I thought I read somewhere that specs are forbidden from having non-namespaced keys.#2018-11-2815:08favilano I mean the key in the value#2018-11-2815:08favilaI am saying exactly what you did is a possible workaround#2018-11-2815:08favilabut your maps can only look like {:foo ...}#2018-11-2815:08favilayou have deliberately introduced ambiguity about what the spec of :foo is#2018-11-2815:10favila(also you don't need to introduce ns forms when specing; maybe you are just doing to save typing but it isn't strictly necessary)#2018-11-2815:10urzdsHow else would you do it, without the ns forms?#2018-11-2815:10favila:with-baz/map-with-baz for eg#2018-11-2815:11favila:: is just sugar for "keyword in current namespace"#2018-11-2815:11favilait's not spec-specific#2018-11-2815:11favila::alias/sym is another thing that exists#2018-11-2815:12favilaif you (:require {my.long.ns :as n]), you can use ::n/x to mean :my.long.ns/x#2018-11-2815:13favilathese expansions occur at read-time#2018-11-2815:13favila(so very early)#2018-11-2815:16favilaanyway, to deal with this exact problem you have I wrote a spec called keys+ which is a compromise: in a particular map, a particular key can get an additional (not replacement) spec to validate against and to use for generators
#2018-11-2815:16favilaso spec ::foo with the widest spec#2018-11-2815:17favilaand contextually in certain s/keys specs, say it must conform to ::foo-with-bar or ::foo-with-baz also#2018-11-2815:17favilaI was dealing with data whose shape I could not control, and the alternative was renaming keys everywhere#2018-11-2815:17favila#2018-11-2815:18favilaso your example would look like (keys+ :req [::foo] :conf {::foo ::foo-with-bar})#2018-11-2815:19favilathe value in ::foo must validate against both ::foo and ::foo-with-bar#2018-11-2815:19favilaso ::foo is (s/keys :opt [::bar ::baz])#2018-11-2815:19favilabut you can see from the gnarliness of this code that spec really doesn't even want this to be allowed#2018-11-2815:20favilaI don't even necessarily recommend that you use this#2018-11-2815:36eoliphanthi iā€™m working on an fspec for a macro. The macro itself, takes an argument that will become a spec basically something like (s/def spec-key spec-pred)` in the macro body. Trying to come up with a valid spec for :spec-pred since it could be a function or say (s/or ā€¦ etc. etc.) . So far I have this (s/or :fn fn? :spec s/spec?)#2018-11-2815:42favilayou might need ifn? too (to catch stuff like #{1 2 3}#2018-11-2815:49Alex Miller (Clojure team)You might want to look at CLJ-2112#2018-11-2815:49Alex Miller (Clojure team)Specifically the patch in there#2018-11-2815:56urzds@favila Thanks!#2018-11-2816:01eoliphantthanks guys#2018-11-2818:25eoliphanthaving a peculiar issue . Iā€™m working on a spec for a variadic function, the ā€˜restā€™ should be members of a set. In my case the set members are maps, and the test always fails. replacing them with say keywords works fine.
(defmacro test-mac
  [one & many]
  `[~one 
#2018-11-2819:52taylorthe problem is spec can't see what the value of test-a is during expansion: https://clojure.org/guides/spec#_macros#2018-11-2819:53taylor(test-mac "a" {:a :a}) works, for example#2018-11-2915:12eoliphantah.. crap.. right.. any ideas on how to perhaps work around this?#2018-11-2915:13taylorcould maybe move some functionality out of the macro into a function (and spec that), or reconsider whether you need a macro at all#2018-11-2916:12eoliphantyeah itā€™s a bit of a chicken/egg problem. Iā€™m working on a little DSL to create data to define a domain, so want to be able to assert stuff about say :user/name including itā€™s spec. so I need to pass in the spec, as well as other meta data so needed to use macros#2018-11-2916:12eoliphantbut going to play around with it#2018-11-3013:59benzapSo Rich said they're releasing a new clojure.spec, is it going to be radically different, or backwards compatible?#2018-11-3014:00mpenethe mentioned being careful about backward compat in the talk. Then again it's an alpha, so breakage might very well happen#2018-11-3014:34Alex Miller (Clojure team)The plan is that it will be as backwards-compatible as possible#2018-11-3014:34Alex Miller (Clojure team)Things like s/keys may continue to exist but be deprecated for example#2018-11-3014:35gklijsFrom what I got, instead of having to split into required and optional argument sin the spec. it's moved to something different, and you can specify on the function witch fields you need/expect#2018-11-3014:36gklijsIt it already in spec alpha-2, or do we need to wait a bit?#2018-11-3014:45Alex Miller (Clojure team)No, itā€™s not available yet#2018-11-3014:47Alex Miller (Clojure team)We are doing some reorg of the impl in spec-alpha2 at the moment#2018-11-3014:47Alex Miller (Clojure team)And then we will be diving into the new stuff#2018-11-3014:49benzapjust finishing up the talk, so spec stuff will be baked into defn, or is that pseudo-code?#2018-11-3014:49benzapor rather spec/defn#2018-11-3014:52benzapman, i'm really excited for these changes#2018-11-3014:53benzapI was hesitant when using spec in my work's project, but I feel like it would really benefit from these changes, especially when defining specs between different data layers#2018-11-3015:02orestisJust a few days ago I suggested to someone to just use optional keys for his specs and validate presence of keys in some other way. (GraphQL context). Glad to hear thereā€™s going to be a better way. #2018-11-3016:02Alex Miller (Clojure team)@benzap itā€™s possible it will get integrated more into defn - currently TBD but Rich is thinking about it#2018-11-3016:03benzapInteresting! I can definitely see the tradeoffs on both sides#2018-11-3016:04Alex Miller (Clojure team)that of course wouldnā€™t happen till Clojure 1.11, if at all#2018-11-3016:05benzapThat's good to know, i'm thinking i'll use the current clojure.spec until then#2018-12-0206:23fossifoonice talk/announcement. really looking forward to it, especially the "Better programmatic manipulation of specs" part šŸ™‚#2018-12-0220:57djtangoDoes anyone know why this is crazy slow / never terminates?
(require '[clojure.spec.alpha :as s])
(require '[clojure.spec.test.alpha :as stest])
(defn my-reverse [c]
  (reverse c))
(s/def ::reverse-args (s/cat :c (s/coll-of any?)))
(s/def ::reverse-ret (s/coll-of any?))
(s/exercise ::reverse-args)
(s/exercise ::reverse-ret)
(s/def ::reverse-fn
  (fn [{:keys [args ret]}]
    (let [input-c (:c args)]
      (= input-c
         (my-reverse ret)))))
(s/fdef my-reverse
        :args ::reverse-args
        :ret ::reverse-ret
        :fn ::reverse-fn)
(stest/check `my-reverse)
#2018-12-0220:57djtango^ (you should be able to copy paste that straight into your REPL)#2018-12-0220:58djtangooddly - if I swap out (s/coll-of any?) for coll? I see the same behaviour but if I use (s/coll-of integer?) it terminates nearly instantly (and successfully)#2018-12-0220:59lilactownit looks like there are many cases to generate šŸ™‚#2018-12-0220:59djtangodid it work for you?#2018-12-0220:59jaihindhreddyany? generates pretty hairy stuff#2018-12-0220:59lilactownI didnā€™t try it, but I can tell that generating cases for (s/coll-of any?) is going to cause test.check to really go crazy#2018-12-0221:00djtangoah I see#2018-12-0221:00jaihindhreddyAnd is generally not a good fit for generative testing#2018-12-0221:00lilactownyou can specify an upper limit to generation#2018-12-0221:00djtangohave I unwittingly stumbled onto a bad example of how to use spec / gen testing?#2018-12-0221:01jaihindhreddyYou might wanna customize the generator so the tests will be quick without changing the semantics of validation that your spec gives#2018-12-0221:01djtangoreverse felt like a nice simple function for illustrating fn specs#2018-12-0221:02jaihindhreddyIt is, and I think you wanna keep any? in your spec but customize the generator to improve the testing#2018-12-0221:02lilactownclojure.spec/coll-of takes a :gen-max argument#2018-12-0221:03lilactownyou could limit it to like 5 or 100 or something that at least finishes šŸ˜›#2018-12-0221:07jaihindhreddyThat is an option too.#2018-12-0221:08jaihindhreddyThis is what I was talking about: (s/def ::reverse-ret (s/with-gen (s/coll-of any?) #(s/gen (s/coll-of int?))))#2018-12-0221:08jaihindhreddy@djtango ^#2018-12-0221:09djtangothanks @jaihindh.reddy#2018-12-0303:19kvltDoes anyone know of a tool where I can give it a datastructure, and it will generate a spec for me based upon that datastructure?#2018-12-0303:20taylorhttps://github.com/stathissideris/spec-provider#2018-12-0303:28kvltThats wonderful, thank you!#2018-12-0318:20aviI feel like Iā€™m a little rusty with spec, and getting tripped up by somethingā€¦ Can anyone see what Iā€™m doing wrong here?
(s/fdef probably-diagram-yaml?
  :args (s/cat :v (s/or :is-diagram     :ss/diagram-yaml-str
                        :is-not-diagram string?))
  :ret  boolean?)
When I test this function with stest/check Iā€™m seeing values generated via string? being validated against :ss/diagram-yaml-str ā€” is that whatā€™s supposed to happen? If soā€¦ why? (This is probably just PEBKAC)
#2018-12-0318:24seancorfieldWhat's the generator for :ss/diagram-yaml-str look like?#2018-12-0318:25seancorfield(I would expect that to also have string? in it based on its name)#2018-12-0318:29aviItā€™s a little crazy#2018-12-0318:30aviThe thing is, whenever I exercise that spec it succeeds just fine. Andā€¦ hmm, well, I hope that exercise does validate the generated values against the spec šŸ˜…#2018-12-0318:33aviGenerator:
#(gen/fmap
      (fn [diagram]
        (str (sometimes (str seyaml/default-front-matter doc-separator))
             (seyaml/stringify diagram)))
      (s/gen ::st/diagram))
and sometimes is:
(defmacro sometimes [body]
  `(when (< (rand) 0.5)
     ~body))
#2018-12-0318:38taylorFYI youā€™ll probably want to replace that sometimes with one of test.checkā€™s generators, otherwise itā€™s an unbound/uncontrolled source of randomness to test.check#2018-12-0318:38taylorhttps://github.com/clojure/test.check/blob/master/doc/generator-examples.md#an-integer-90-of-the-time-nil-10#2018-12-0318:39aviGreat tip ā€” thanks so much!#2018-12-0318:39taylormaybe something like (gen/frequency [[1 (gen/return "thing")] [1 (gen/return nil)]])#2018-12-0318:40aviah thatā€™d work? equal frequencies means equally likely?#2018-12-0318:40avi(I didnā€™t look at the docstring for frequency yet)#2018-12-0318:41taylorthatā€™s my untested assumption šŸ™‚ and there might be an even more appropriate function in test.check Iā€™m forgetting/donā€™t know about#2018-12-0318:41avibased on the link you provided, looks like I should use (gen/return nil) rather than just nil#2018-12-0318:41tayloroh yeah, sorryā€¦ my example had concrete values instead of generators, fixing#2018-12-0318:42avišŸ˜Ž#2018-12-0318:43taylorI suppose you could use gen/elements too, if your values are static/not-generated. I suppose elements given a two-element coll would choose between them equally?#2018-12-0318:44aviI dunno what the distribution is#2018-12-0318:44aviIIRC some test.check generators donā€™t use even distributions#2018-12-0318:48aviI wonder if this is on the right track:
#(gen/fmap
  (fn [diagram]
    (str
      (gen/frequency
        [[1 (gen/return nil)]
         [1 (gen/return (str seyaml/default-front-matter doc-separator))]])
      (seyaml/stringify diagram)))
  (s/gen ::st/diagram))))
#2018-12-0318:59taylorlooks pretty reasonable to me. I still wonder if elements would be better suited in this case b/c it'd definitely be more concise#2018-12-0318:59aviprobably#2018-12-0318:59avithanks for helping me improve my code!#2018-12-0318:35aviAh yes, peeked at the source of exercise and it calls conform on all generated values. (Itā€™s mentioned in the docstring too.)#2018-12-0318:57aviI think this is starting to dawn on meā€¦ I think maybe (Iā€™d like to think because Iā€™ve been rusty) Iā€™ve just been not quite grokking s/or ā€¦ when a value is conformed against an s/or spec it will quite naturally be conformed against each scenario in order. And after a value is generated via a generator, I guess thereā€™s no mechanism at that point to conform it straight off against its original scenario specā€¦ in other words, s/or isnā€™t like pattern matching, which I guess was for some reason how I was trying to use it.#2018-12-0319:03taylorI think it's pretty conceptually similar to pattern matching, but I'm not sure I follow this issue: >And after a value is generated via a generator, I guess thereā€™s no mechanism at that point to conform it straight off against its original scenario spec#2018-12-0319:06taylorBTW one way I think s/or differs from pattern matching like in ML languages is that s/or clauses don't have to be exhaustive by default. Don't have to be non-overlapping either#2018-12-0319:35avier, yeah, sorry. I think it was just my rusty thinking. ā€¢ An or spec is a spec, and (I suppose) most functions that work with specs will treat/use an or spec as an opaque spec, like any other spec. ā€¢ Accordingly, the workflows of conform and stest/check have no affordances for taking values generated from the or spec and influencing how those values are conformed against the or spec and its internal scenarios ā€¢ Accordingly, values generated by any given scenario of an or spec might be conformed against any other given scenario of an or spec, because thatā€™s how an or spec works ā€” values are conformed against its scenarios one by one in order ā€¢ The only reason I got caught up on this is because one of my specs would sometimes throw an exception when passed an invalid value#2018-12-0319:36aviI hope that might begin to approach making some kind of sense#2018-12-0319:37taylorYeah Iā€™ve been bitten using a predicate in an or that wasnā€™t nil safe. Order matters in some cases#2018-12-0319:38aviyes, exactly#2018-12-0319:38avimy spec calls a function that throws if passed an empty string#2018-12-0319:38avirelated to a regex search#2018-12-0318:57aviMaybe#2018-12-0318:57aviI feel like I need a couch to lie down on and someone with a pipe to nod and doodle on a notepad#2018-12-0318:57avisorry for the noise!#2018-12-0320:31prozzhmm, have a strange question: how to actually spec that something is a sentence? or write a generator for a good sentence? (let's say lorem ipsum language with limited number of words is enough)#2018-12-0320:33prozzjust realized im specing stuff as string? but would like my generators to be clever and generate usernames as random strings but description as some sort of lorem ipsum with spaces. if this question is too beginner like for this channel, let me know pls.#2018-12-0320:44andy.fingerhutIf you mean actual grammatically correct English, or some other natural language, then you need some kind of NLP library or a product of machine learning for that. If by "sentence" you mean something else, please clarify.#2018-12-0320:45noisesmithand spec explicitly isn't a string parsing library - anything for differentiating one string from another will be wrapped up and just be another function as far as spec is concerned#2018-12-0320:46andy.fingerhutSorry, you did mention "some sort of lorem ipsum with spaces", but still might need some clarification of exactly what kind of strings you want to include vs. exclude. A regular expression could be written over strings that matches most lorem ipsum like text, but rejects things that contain weird special characters.#2018-12-0320:47andy.fingerhutIf you did that, and then wanted to generate random examples for testing, you would likely only be able to do so by writing a custom string generator, because otherwise the default test.check string generator would, with very high probability, generate strings that do not satisfy the spec.#2018-12-0320:57gfrederickstest.chuck (different lib) has a regex -> generator thingamabob#2018-12-0320:58andy.fingerhutcool. I probably saw that mentioned before and forgot about it. Such a thingamabob sounds pretty non-trivial, unless you found a library that transforms regexes into state machines for you.#2018-12-0320:59gfredericksit was nontrivial#2018-12-0320:59gfredericksthe most nontrivial parts were punted on (esp. lookahead, lookbehind)#2018-12-0321:00andy.fingerhutMakes sense. Common regex libraries include so many features like that, that complicate analyzing them (and IMO, understanding what they do)#2018-12-0321:01gfredericksone big hard part was parsing the regexes correctly, and the other one was handling characters correctly#2018-12-0321:06andy.fingerhutI saw a quote similar to the following (paraphrasing from memory) in the early 1990s in someone's signature on a public Internet forum and loved it: "Ain't nothin's easy when you're doing it fer real! -- a gunnery sergeant in the U.S. Army"#2018-12-0321:08gfrederickswell some things are, but they're not very interesting to talk about#2018-12-0321:33prozzthanks for explanation, you all above confirmed what i thought. need to dive deeper into generators, check possibilities and decide what i really wanna do šŸ™‚ basically i want a fixtures generator for particular domain. hence need for sentences.#2018-12-0323:19prozzis there an existing generator for normal distribution from given range?#2018-12-0400:14Alex Miller (Clojure team)No#2018-12-0400:14Alex Miller (Clojure team)Well, not in spec#2018-12-0400:15Alex Miller (Clojure team)Maybe there is in test.check#2018-12-0400:15gfredericksNope#2018-12-0400:16gfredericksDon't have any continuous stat distributions#2018-12-0400:16gfredericksYou could make them using fmap from something we do have#2018-12-0400:16prozzyep, there is frequencies#2018-12-0400:17gfredericksUnclear what kind of shrinking and growth you'd want in many cases#2018-12-0402:49andy.fingerhutI think it is fairly straightforward to take a uniform random number generator for a float in the range [0, 1] and turn it into a generator for any distribution that has a function whose CDF you know how to evaluate. This page has what it claims is a very accurate approximation of the normal distribution CDF: https://web.archive.org/web/20151030215612/http://home.online.no/~pjacklam/notes/invnorm/#Pseudo_code_for_rational_approximation#2018-12-0402:51andy.fingerhutI haven't checked out their implementation but kixi stats lib claims to implement this and many other distributions: https://github.com/MastodonC/kixi.stats#2018-12-0416:52jaihindhreddyWill the decomplecting of schema/structure from selections/context be there in 1.11, or will it get in earlier?#2018-12-0416:55seancorfield@jaihindh.reddy The whole point of clojure.spec being developed and delivered as a separate library is that it is no longer tied to specific Clojure versions.#2018-12-0416:58jaihindhreddyWhile that is true, spec does come in the clojure jar if I'm not mistaken, hence my doubt.#2018-12-0417:10Alex Miller (Clojure team)no, it does not#2018-12-0417:10Alex Miller (Clojure team)itā€™s a dependency, which you can independently update#2018-12-0417:10Alex Miller (Clojure team)weā€™ve updated spec two or three times since 1.9 for example#2018-12-0417:11Alex Miller (Clojure team)I expect that the new work will be available before 1.11#2018-12-0417:11Alex Miller (Clojure team)but itā€™s likely to be in a different namespace so will be a little trickier to use - weā€™ve spent some time thinking through that, and probably will require some more#2018-12-0421:23urzdsIs there some valid-or-explain function that I can easily use in my function's {:pre [(valid-or-explain ::spec %)]} context?#2018-12-0421:23urzdsHaving s/valid? in there as suggested in the guide is nice, but if I cannot see what it is complaining about, it does not help much... šŸ˜ž#2018-12-0421:37seancorfields/assert#2018-12-0421:39taylorgotta be careful using s/assert in pre/post-conditions because if a nil input is valid, it'll return it (nil), causing the pre/post-condition to fail#2018-12-0421:39taylor(because nil ā‰ˆ false)#2018-12-0421:40taylorand also have to be mindful of s/*compile-asserts*#2018-12-0421:41taylor@urzds if you want this behavior in pre/post-conditions it should be very easy to write a small function that does it#2018-12-0421:41seancorfieldWell :pre / :post only take effect if *assert* is true...#2018-12-0421:42taylor*assert* is unrelated to s/*compile-asserts* AFAICT#2018-12-0421:43seancorfieldYes, but the same caveat essentially applies -- you don't get the check in all cases.#2018-12-0421:45seancorfieldYour point about nil is a good one -- I hadn't thought about that.#2018-12-0421:53taylorit was a learned lesson šŸ™‚#2018-12-0421:48seancorfieldIf you want the check always performed, write it as code in the body of the function. If s/assert works for you (not nil values, (s/check-asserts true) has been run), that's probably a reasonable approach -- otherwise you'll have to write your own variant of s/assert. Or perhaps write an fspec for the function and instrument it?#2018-12-0508:14jumarYou can also set it via system property clojure.spec.check-asserts#2018-12-0421:53urzdsThanks!#2018-12-0421:59urzdsWhat would be a good way to check whether a collection contains certain elements? I.e. I have [{:name "a"}{:name "b"}{:name "c"}] and I want to ensure that the collection always contains at least one element with :name "a" and one with :name "b".#2018-12-0422:00urzdsAdditionally, I would like to ensure that each :name only occurs once.#2018-12-0422:01tayloryou could use s/and with an extra predicate for those conditions#2018-12-0422:01urzds(Also note: The element's maps contain more keys than just :name, so I cannot use :distinct.)#2018-12-0422:03dchelimskyWould it be reasonable to change the structure to a map where name moves up to keys?#2018-12-0422:04dchelimskyI donā€™t assume it is reasonable. Just asking if it is.#2018-12-0422:04urzdsWas thinking about that initially, but I want to expose the data structure using GraphQL, and that can afaik only query for predefined keys.#2018-12-0422:05dchelimskyIā€™ve run into that recently :)#2018-12-0422:05taylormaybe something like this for your uniqueness constraint, and easy to add your other constraint
(s/def ::names
  (s/and (s/coll-of (s/keys :req-un [::name]))
         #(apply distinct? (map :name %))))
(s/explain ::names [{:name "a"} {:name "b"}])
(s/explain ::names [{:name "a"} {:name "a"}])
#2018-12-0422:06urzds@taylor: (s/and #(some (fn [x] (= (:name x) "a")) %) #(some (fn [x] (= (:name x) "b")) %))?#2018-12-0422:09taylor
(s/def ::names
  (s/and (s/coll-of (s/keys :req-un [::name]))
         #(let [names (map :name %)]
            (and (apply distinct? (map :name %))
                 (clojure.set/superset? (set names) #{"a" "b"})))))
here's another way, not necessarily any better though
#2018-12-0422:09urzdsAt least it becomes more clear what we're trying to do.#2018-12-0422:10taylorand you may prefer to break each condition into its own predicate like your first example#2018-12-0422:10tayloras opposed to packing multiple conditions into one predicate#2018-12-0422:11urzdsSo I can combine the ::specs instead of having the big inline function?#2018-12-0422:11tayloryep#2018-12-0422:12taylorthe explain output for invalid inputs will likely be easier to understand if the predicates are separated#2018-12-0422:14taylori.e. if they're all combined the explain output isn't going to tell you exactly which condition failed#2018-12-0422:16urzdsThanks!#2018-12-0422:17urzdsI guess there is no way to reuse names in both specs?#2018-12-0422:07urzdsLooks horribly complicated, I guess I'm missing the easy syntax...#2018-12-0422:49Alex Miller (Clojure team)coll-of has a :distinct option fyi#2018-12-0422:49Alex Miller (Clojure team)oh, I guess youā€™re a level off of that#2018-12-0514:27domkmA :distinct-by option would be helpful. For coll-of, I've rarely needed :distinct but often needed :distinct-by.#2018-12-0521:19lilactownsorry if this is a tired question, but is there any upcoming plans for supporting dynamic creation of specs other than eval-ing the macros? e.g. I want to stick a spec in a DB and use it to check some data later#2018-12-0521:36shaun-mahoodWould https://github.com/metosin/spec-tools#data-specs work for what you need?#2018-12-0521:40richhickey@lilactown work already in progress https://github.com/clojure/spec-alpha2#2018-12-0521:43lilactownthx! I'll wait for docs - it's not clear me to from the commits what it's taking shape as yet.#2018-12-0617:16borkdudewhat would be a good name for a lib with some testing utils around specs? mostly things like instrumenting and unstrumenting in the scope of a body kind of macros, cross platform. my gut feeling says that spec-test will be too confusing#2018-12-0617:22shaun-mahoodPretty sure checkulative should be a word#2018-12-0617:27borkdudehaha#2018-12-0621:36Marc O'Morainchickity check yo self#2018-12-0621:36Marc O'Morainhttps://www.youtube.com/watch?v=bueFTrwHFEs#2018-12-0621:37Marc O'MorainChickity-check your specs before you wreck your specs.#2018-12-0617:16taylorspec-too-the-speckoning#2018-12-0617:17borkdudethese functions will be in it: https://github.com/slipset/speculative/blob/master/doc/test.md#2018-12-0617:49Alex Miller (Clojure team)I actually wrote a with-instrument at one point, not sure if I put it in a ticket or not#2018-12-0618:00borkdudethis one also works with clojurescript#2018-12-0618:02Alex Miller (Clojure team)actually, there is a ticket, but not from me - https://dev.clojure.org/jira/browse/CLJ-2015#2018-12-0619:15lilactownI'm having a silly thought: clojure-spec-as-a-service. We have a few different clients in our system, some of which are Clojure(Script) and some aren't. For those that aren't, exposing common specs for our domain is a bit more troublesome then distributing a library. One solution might be to create a service that accepts (data, spec-name) and responds with true/`false` if it conforms. Could also do generation of fake data that conforms. Anyone done this before? I'm not sure if this is a good idea yet, architecturally#2018-12-0619:17noisesmith@lilactown I'd be tempted to copy the way kafka works with avro - the schema is versioned and serialized and there's a rest endpoint to look up the schema, then you can deserialize, cache and reuse it#2018-12-0619:18noisesmithand naturally you can use post to add a new spec or new version of a spec, with backward compat enforcement#2018-12-0619:21lilactown> look up the schema, then you can deserialize, cache and reuse it practically this means that I need to have a common language and engine for working with those schemas in all my clients#2018-12-0619:22lilactownI think the problem I have right now, is "I have all these specs; how do I reuse them across my JS web & mobile clients?"#2018-12-0619:23lilactowndo I build a language on top of spec and build an engine for each platform? Do I adopt something else other than clojure.spec that I more of my clients will understand?#2018-12-0619:24rapskalian@lilactown that's an interesting idea...would part of the spec endpoint also do format conversion? For example, would it be necessary to validate JSON data (convert it to EDN, run through spec/valid?, return result)?#2018-12-0619:24lilactownyes, I am thinking that JSON with be the format it primarily accepts#2018-12-0619:24lilactownso additional complexity#2018-12-0619:33rapskalianOne issue I can think of would be semantic parity between environments...JSON is sort of inherently less expressive then, say, EDN, so you'd have to be careful that they were validating "apples to apples" so to speak.#2018-12-0619:34noisesmithtransit could be useful for this - it's expressly designed for interchange (unlike EDN) and it has implementations for the bigger languages, and it can handle all the standard clojure data types#2018-12-0619:38rapskalianSpec reuse is definitely the right idea. It seems like there'd be quite a few "translation layers" no matter how you tackled it. For example, I've spoken with people who are using Datascript as a sort of "lingua franca" of domain modeling, and then generating specs from that representation. I wonder if "specs" could be generated for other runtimes in their respective native form.#2018-12-0619:38lilactowna lot of the data we'd be verifying would be JSON in the first place - e.g. we get a response from our GraphQL API, and then want to see if it conforms to certain rules: "Does this user data have <product> associated with them?" "Are they allowed to use <capability>?"#2018-12-0619:40lilactownbut there could be other context, unrelated to user data, of course#2018-12-0619:41noisesmithoh - also if you are doing permission checks there's actually an advantage to checking on the central server instead of telling the client how to validate#2018-12-0619:41noisesmith(eg. keeping changes in sync without pushing...)#2018-12-0619:43lilactownyeah permissions isn't primarily what it would be used for but I feel it could grow out of this service organically#2018-12-0622:34borkdudehttps://github.com/borkdude/respeced#2018-12-0700:21ro6I'm starting to try using ::keyword style keywords more, but running into a limitation and wondering if it's a misconception on my part. I tried using the namespace as a prefix on a longer keyword, eg ::more.prefix/name, but that's an "Invalid token". Is there something about the Clojure impl (eg around namespaces or interning?) that would break if that were allowed? Is there a "good idea" that's being enforced here to save me from a "bad idea"? I was hoping it would expand to :the.ns.more.prefix/name.#2018-12-0700:22seancorfield::foo/bar will expand the alias foo into the full namespace.#2018-12-0700:23seancorfield
user=> (alias 'foo.bar (create-ns 'long.ns.foo.bar))
nil
user=> ::foo.bar/quux
:long.ns.foo.bar/quux
user=> 
#2018-12-0700:23seancorfield(that's essentially the same as (require '[long.ns.foo.bar :as foo.bar])#2018-12-0700:24ro6right, at the usage site. I'm inside the defining namespace and conceptually wanting to "group" some things without creating a whole new ns#2018-12-0700:24seancorfield:: can only be used with an alias.#2018-12-0700:24seancorfield(or a bare keyword -- the "alias" is implicitly the current namespace)#2018-12-0700:24ro6right#2018-12-0700:27ro6I'm just wondering if there's a deeper "why" for that.#2018-12-0700:28seancorfieldI'm not sure what your question is...?#2018-12-0700:28noisesmith:: is a keyword reading feature that works with aliases#2018-12-0700:28seancorfield::more.prefix/name means "look up more.prefix as an alias and resolve it (to some namespace) and then construct a keyword with that namespace and /name"#2018-12-0700:28seancorfieldThat is the semantics of ::#2018-12-0700:31ro6I think what's going on in my mind is that as I imagine these ns-qualified keywords escaping my Clojure system, I want to narrow them to remove ambiguity. In other words, I'm trying to name things so they still make sense with fewer contextual assumptions.#2018-12-0700:32seancorfieldNot much outside Clojure is going to be able to accept qualified keywords... Or am I misunderstanding you?#2018-12-0700:34ro6No, I think you're right on.#2018-12-0700:35ro6I guess this line of thought is more about self-documenting keywords for human consumption than machine disambiguation.#2018-12-0700:35seancorfieldIf you want an alias, to avoid typing, you can always do this inside your the.ns namespace:
(alias 'more.prefix (create-ns 'the.ns.more.prefix))
... ::more.prefix/name ...
#2018-12-0700:37ro6interesting compromise. That would let me organize my code into files independently of how I'm organizing my names/concepts.#2018-12-0700:37seancorfield(there's talk of an easier way to create aliases being added to Clojure at some point but that's what we have right now)#2018-12-0700:37seancorfieldThere's no need for the "namespace" portion of a qualified keyword to be an actual namespace with code.#2018-12-0700:38seancorfieldWe have several qualified keywords in our code base that do not map to namespaces.#2018-12-0700:39ro6ok. Thanks for the info.#2018-12-0700:40ro6I also just realized that dot-separated segments after the / are possible too#2018-12-0700:46ro6I guess at runtime what separates keywords (ns-qualified or not) from strings is mostly faster equality checks due to interning.#2018-12-0700:46Alex Miller (Clojure team)But not recommended#2018-12-0700:46ro6haha, thanks.#2018-12-0707:34macDoes anyone know of a (stateful) generator of locally unique integers for use with spec?#2018-12-0716:27gfrederickswhat does "locally unique" mean?#2018-12-0716:27gfredericks@mac ^^#2018-12-0717:11ro6(guessing) Non-repeating relative to some local scope (eg the generator itself, the running process, ....)?#2018-12-0717:11ro6Not sure about the "stateful" part#2018-12-0719:09mac@gfredericks Just means that for the run of the test no two ints produced will be the same.#2018-12-0719:12gfredericks@mac depends what "for the run of the test" means in this context; is this the generative test that spec does with spec/check or whatever it is?#2018-12-0719:32gfredericksI'll probably just end up recommending https://www.youtube.com/watch?v=F4VZPxLZUdA#2018-12-0719:34taylormaybe (gen/vector-distinct gen/int) would be useful... somewhere#2018-12-0720:45mac@gfredericks Yes it is "the generative test that spec does with spec/check"#2018-12-0720:46mac@robert.mather.rmm Stateful because I cannot imagine being able to guarantee uniqueness without maintaining state.#2018-12-0720:47gfredericksthe standard way to do it would be to write your own generator at the lowest level that encompasses the whole relevant scope of uniqueness#2018-12-0720:47gfredericksyou could take the default generator and gen/fmap it to something that uniqueifies the ints#2018-12-0720:47gfredericksan alternative, if you don't care how big the ints are, is to just generate really big integers and not allow them to shrink#2018-12-0720:51gfredericks(you'd need to take care to make the generator something like uniform-distribution, so it doesn't start small either)#2018-12-0720:52mac@gfredericks thanks#2018-12-0720:56ro6I don't know if it can be done lazily/immutably, but if you're worried about performance and you know how many random+unique ints you need up front, there's a cool algorithm for that. #2018-12-1220:09domkmApologies if this has been asked previously but why are the Alpha1 and Alpha2 namespaces inconsistent? I would expect clojure.spec.alpha2 instead of clojure.spec-alpha2?#2018-12-1220:12Alex Miller (Clojure team)because we found having all the namespaces end in ā€œalphaā€ was annoying#2018-12-1220:43domkmI see#2018-12-1222:35seancorfield@alexmiller On that subject, should I open JIRA issues for the bugs I ran into with spec-alpha2 or isn't it time yet?#2018-12-1222:56Alex Miller (Clojure team)I would wait right now#2018-12-1222:56Alex Miller (Clojure team)I have another giant refactor pending#2018-12-1305:43nopromptThereā€™s also the greek letter beta. troll#2018-12-1305:44nopromptWhy should beta be reserved only for use-at-your-own-risk software I ask? šŸ˜„#2018-12-1310:29jumarLet's say I have following config spec:
(s/def ::my-config-spec
  (s/keys :req-un [::host
                   ::port
                   ]))

(s/explain-data ::my-config-spec {:host ""})
;; => #:clojure.spec.alpha{:problems (
;;    {:path [], :pred (clojure.core/fn [%] (clojure.core/contains? % :port)),
;;     :val {:host ""}, :via [:cacsremoteservice.config/my-config-spec], :in []}), :spec :cacsremoteservice.config/my-config-spec, :value {:host ""}}
I used explain-data and returned :path key to indicate which keys are in the config are invalid/missing. What I didn't realize is that when the key is missing then the path is empty (instead of pointing to the particular key). Is there a nice way how to get the name of a missing required key?
#2018-12-1313:41seancorfieldNot nice but you can pattern match :pred with destructuring since it has that particular shape and extract the missing field name from the contains? call. #2018-12-1313:56jumarThanks for the idea! I'm sure there's a better way but here's my quick solution:
(defn extract-key [spec-problem]
  (let [[_fn _args [_contains _map req-key]] (:pred spec-problem)]
    req-key))
  
(let [ed (s/explain-data ::my-config-spec {:host ""})]
  (->> ed :clojure.spec.alpha/problems
       (map extract-key)))
#2018-12-1313:42seancorfield(And it might change in a future version of spec)#2018-12-1323:42thedavidmeister@simon223#2018-12-1400:27aengelbergwhy doesn't clojure.spec.gen.alpha have recursive-gen?#2018-12-2020:21wilkerluciothis seems more a generation concern than a spec concern, so I think makes sense been part of test.check / test.chuck or something else#2018-12-1613:12ro6I'm surprised there aren't more examples of specs for common stuff like URLs using s/cat. Has anyone worked on something like that? I've seen http://conan.is/blogging/a-spec-for-urls-in-clojure.html and https://github.com/SparkFund/useful-specs, but they are using opaque url validators. I'm hoping for something where explain would actually be instructive.#2018-12-1707:07flyboarderHello everyone, im getting a strange error Unable to resolve spec: :clojure.core.specs.alpha/args+body anyone know whats up?#2018-12-1707:15flyboarder^I figured it out - renamed alpha spec#2018-12-1819:07Keith HarperThought the patter was interesting, so I figured Iā€™d share. Data points generated with:
(s/def ::latitude (s/double-in :min 32 :max 34 :NaN false :infinite? false))
(s/def ::longitude (s/double-in :min -84.9 :max -82 :NaN false :infinite? false))
#2018-12-1819:17rapskalianCool! Thanks for sharing. It looks like the double generator is a bit "quantum biased", for lack of a better word.#2018-12-1901:48bbrinckIf you previously couldnā€™t use Expound because your specs used a custom conformer, try Expound 0.7.2 - it may not be able to give a precise error, but at least it wonā€™t throw a ā€œCannot convert pathā€ error#2018-12-1919:56ro6Question about evolving specs: I realize the whole idea with specs is "don't change them in breaking ways", so I'm thinking about how to plan for that. If I define a spec with a single definition, then later realize there's another valid representation, so I want to change it to (s/or ....), can that be a non-breaking change to consumers?#2018-12-1919:58ro6I guess if the original definition was one of the options under the s/or it would still validate the same, but the conformed value would change right? I guess generally the conformed value is more likely to be depended-upon by internal code rather than API consumers...#2018-12-1920:15Alex Miller (Clojure team)adding additional valid things means old stuff should continue to be valid#2018-12-1920:15Alex Miller (Clojure team)I donā€™t know that we would expect forward-consistent stuff out of conform though#2018-12-2017:38jaihindhreddyWill there be a way to s/select into a subset of dispatch keys of a s/multispec?#2018-12-2017:44jaihindhreddySay I have a polymorphic map called ::payment-info which is polymorphic, like this: (s/def ::payment-info (s/multi-spec pi ::type)) where (s/def ::type #{::credit-card ::debit-card ::net-banking ::bitcoin}) Because cards are processed one way, and each of the other types another way, is s/selecting into ::credit-card and ::debit-card a good idea? Was just wondering about this, s/select isn't even a thing yet so these are potential future questions šŸ™‚#2018-12-2018:12Alex Miller (Clojure team)itā€™s not real enough yet that I canā€™t answer those questions#2018-12-2018:55ShaneLesterHow would I write a spec to say- in this map I expect to have X Y and Z keys (which have their own specs) and then I donā€™t care about the rest of the map? Does specifying those 3 in the :req of the keys function assume that I donā€™t care about the rest?#2018-12-2018:56tayloryeah the s/keys specs are "open" by default#2018-12-2018:57ShaneLesterOkay cool, thanks#2018-12-2020:53ShaneLesterSo I have these 3 specs hereā€¦ I am trying to say, hey there is going to be any number of inputs here in this input-type. But apparently something like {:inputs [:email {:value nil}]} doesnā€™t meet this spec? Itā€™s saying it should satisfy map? but it seems to me that that IS a map. Anyone have a clue what Iā€™m doing wrong here?#2018-12-2021:10taylor::inputs is expecting a collection of ::input-types which are maps/`s/keys` specs, but your input is a single map#2018-12-2021:11ShaneLesterohhh I see. thank you#2018-12-2021:12taylor[{:value nil}] would conform to your existing spec#2018-12-2021:13tayloryou may have some other issues judging from the vector [:email {:value nil}], with the spec as it is#2018-12-2021:14ShaneLesterRightā€¦ yeah I may be doing some things in not the best way šŸ™‚ time to re-evaluate#2018-12-2021:15taylorfeel free to post some other sample inputs in here and I could help write some sample specs for them#2018-12-2021:17ShaneLesterCool, I appreciate that#2018-12-2022:10agcan someone throw an example of recursive specā€¦ e.g. directory structure?#2018-12-2022:11agI want something like:
(s/def :folder/type #{"folder" "file"})
(s/def :folder/id (s/and string?  (partial re-matches #"\d*") #(< 8 (count %))))
(s/def :folder/name string?)

(s/def ::entries
  (s/coll-of ::folder))

(s/def ::folder
  (s/keys :req-un [:folder/type :folder/name ::entries]))
but obviously that doesnā€™t work
#2018-12-2022:11agI want something like:
(s/def :folder/type #{"folder" "file"})
(s/def :folder/id (s/and string?  (partial re-matches #"\d*") #(< 8 (count %))))
(s/def :folder/name string?)

(s/def ::entries
  (s/coll-of ::folder))

(s/def ::folder
  (s/keys :req-un [:folder/type :folder/name ::entries]))
but obviously that doesnā€™t work
#2018-12-2022:15taylorthis will work with (s/def ::entries (s/* ::folder))#2018-12-2022:16taylor
(binding [clojure.spec.alpha/*recursion-limit* 2]
  (gen/sample (s/gen ::folder)))
...
{:type "file",
  :name "8y4FOn",
  :entries ({:type "folder",
             :name "",
             :entries ({:type "folder", :name "x2RcQC", :entries []}
                       {:type "folder", :name "6on7", :entries []}
                       {:type "folder", :name "", :entries []}
                       {:type "folder", :name "aakQc3q", :entries []}
                       {:type "folder", :name "6g4748z", :entries []})}
            {:type "file",
             :name "1f",
             :entries ({:type "folder", :name "j52GS", :entries []} {:type "folder", :name "e223Z", :entries []})}
            {:type "file",
             :name "DUuXT6",
             :entries ({:type "folder", :name "t", :entries []}
                       {:type "folder", :name "720S2cy", :entries []}
                       {:type "file", :name "", :entries []}
                       {:type "folder", :name "", :entries []})}
            {:type "folder",
             :name "18EZK",
             :entries ({:type "folder", :name "27", :entries []} {:type "file", :name "Q", :entries []})}
            {:type "file",
             :name "2K8w",
             :entries ({:type "folder", :name "Pv", :entries []} {:type "file", :name "xl4346Q", :entries []})}
            {:type "folder",
             :name "j",
             :entries ({:type "file", :name "21C", :entries []}
                       {:type "file", :name "3", :entries []}
                       {:type "folder", :name "2", :entries []}
                       {:type "folder", :name "1dFvly", :entries []}
                       {:type "folder", :name "zczGQf", :entries []})})}
...
#2018-12-2022:16agright.. rightā€¦ I vaguely remembered, but I couldnā€™t figure it outā€¦ Thanks!#2018-12-2022:30agnow I wonder if there is a way to make
binding [clojure.spec.alpha/*recursion-limit* 2]
part of (s/with-gen?
#2018-12-2022:31agso it never generates more than 2 levels deep?#2018-12-2100:19agis it possible to run count of (s/def #{"a" "b" "c"}) ?#2018-12-2101:59tayloris that s/def missing a name?#2018-12-2101:59taylore.g. (s/def ::foo #{1 2 3})
#2018-12-2102:03tayloranyway if your spec is a literal set, you could do
(s/def ::foo #{1 2 3})
(s/form ::foo)
=> #{1 3 2}
#2018-12-2102:03tayloror you could define the set separately, and reference it from the s/def and wherever you need to count it#2018-12-2100:20aghow can I find out how many elements there?#2018-12-2101:57ro6Is there a way to abstract over whether a key in a map is namespaced or not? In a certain context, as long as the value conforms, I want to accept both with the same spec.#2018-12-2102:48Alex Miller (Clojure team)If youā€™re using s/keys, the :req-un and :opt-un will do that#2018-12-2117:08urzdsCan I spec record methods just like any ordinary function?#2018-12-2117:09urzdsThe guide does not really go into details on records, except that one can spec their fields/attributes: https://clojure.org/guides/spec#2018-12-2117:15Alex Miller (Clojure team)records donā€™t really have methods#2018-12-2117:15Alex Miller (Clojure team)they implement interfaces/protocols which define methods#2018-12-2117:15Alex Miller (Clojure team)currently, you canā€™t spec protocols or interface methods#2018-12-2117:16Alex Miller (Clojure team)and because of the calling implementation itā€™s not possible to instrument them (and probably not something weā€™re ever going to really do because of that)#2018-12-2117:16Alex Miller (Clojure team)but I wouldnā€™t completely rule it out#2018-12-2117:21urzds@alexmiller So I can spec neither the protocol for all its implementations, nor every record implementation individually?#2018-12-2117:23Alex Miller (Clojure team)no#2018-12-2117:23urzdsBut I could place the implementation in a function, which I could spec, and then from the record implementation just call that function? I.e. insert one step of indirection?#2018-12-2117:23Alex Miller (Clojure team)yes, but I wouldnā€™t do that just to be able to spec it#2018-12-2117:24urzdsWell, right now my implementations look like (s/assert ...args...) (s/assert ...body...), which is hardly better...#2018-12-2117:25Alex Miller (Clojure team)this is a case where the impl of protocols (designed to tap into the highly optimized java/jvm calling semantics) is at odds with the dynamic indirection possible in Clojure via vars#2018-12-2117:26Alex Miller (Clojure team)I guess maybe thereā€™s some possible future where newer Java capabilities like method handles could be used to implement this kind of thing#2018-12-2117:26Alex Miller (Clojure team)retaining most of the speed but also giving you the dev time instrumentation#2018-12-2117:33urzdsSo what do you do when during development you want to ensure that the maps returned from a method implementation of the record conform to a spec? And that all callers call it properly? Insert (s/assert arg-spec arg) (s/assert ...body...), like I did?#2018-12-2117:36urzdsI don't really care if the call slows down by an order of magnitude, if it helps me in debugging where that borked data came from.#2018-12-2117:37urzdsOr do you just split up the code in such small pieces that the method implementation in the record does not really do any significant work anymore, so you have enough functions that you can properly instrument?#2018-12-2117:38Alex Miller (Clojure team)I donā€™t think there is one canonical answer to that question, it depends on the code#2018-12-2117:39Alex Miller (Clojure team)splitting up code into smaller pieces is usually a good idea though#2018-12-2117:39urzdsThanks, I'll start with that then, maybe it already alleviates the problem.#2018-12-2122:06dominicmIs there any work yet on determining whether a spec is a subset or a superset of another? I think Rich has hinted at this a lot. #2018-12-2122:34Alex Miller (Clojure team)No#2018-12-2123:27ro6is anyone using (s/cat ...) to spec strings? I realize I could just delegate to a traditional string regex with (re-matches ...), but then I give up on high-fidelity explain results no?#2018-12-2215:39dustingetzHow do I spec inside a reagent reaction or other container type that isn't a seq#2018-12-2215:40dustingetzI just googled and found https://stackoverflow.com/questions/37972074/how-to-clojure-spec-a-reference-type-like-atom which says "Don't", okay i get it but that is not compatible with performant UI programming#2018-12-2215:41dustingetzHas this been debated before?#2018-12-2216:23didibusWell, you're not supposed to spec everything#2018-12-2216:25didibusThe same way you don't necessarily put comments on everything#2018-12-2216:26didibusYou need to ask, is this hard to understand, do people wonder what the shape and structure of this thing is, if so, a spec can be a great way to make that easier#2018-12-2216:28didibusOr, if you specifically want to perform generative tests on a particular function that would benefit from it#2018-12-2216:31didibusOr if you need to validate user input, or validate data you're about to persist, etc.#2018-12-2216:33didibusBut if its obvious what the things are, you don't need a spec.#2018-12-2216:34didibusNow to spec an atom, I mean, specs are just predicates. You can easilly do: #(s/int? @%)#2018-12-2216:35didibusNow you do have to make sure that the deref won't trigger unintended effects#2018-12-2216:35didibusLike in the case of a future or delay#2018-12-2216:37didibusAnd for atom, the other challenge is mutation. Your spec needs to either be the union of all possible valid values the atom will ever contain, throughout its many mutations#2018-12-2216:38didibusOr it has to be more generic, like validates that you expect the thing to be an atom and that's all#2018-12-2216:38didibusOr you need a way to know the particular context and time and what in that context and time the atom is supposed to contain#2018-12-2216:39didibusAnd spec relative to that#2018-12-2216:40didibusThe validator idea is good also. Validators were designed to validate the data being set on an atom. So you can leverage spec to make the validation. #2018-12-2216:40didibusSince spec is pretty much a DSL for easy validation#2018-12-2217:58dustingetzIt's the ratom which is the incidental complexity, it is spiritually just data. For better or worse, Reagent UIs pass things that are data as reactions and it is what it is. But I was unable to get it to work in a nontrivial case#2018-12-2223:33didibusCan you show an example? Also, the ratom isn't a normal atom, and its reactive nature has a different hook for when the data would be updated and need to be validated. At least from what I know, I actually havn't used reagent#2018-12-2221:41tedcushmanI have encountered errors when trying to use s/and with s/or as the first argument:
=> (s/explain
     (s/and (s/or :int int? :double double?)
            pos?)
     3)
ClassCastException clojure.lang.MapEntry cannot be cast to java.base/java.lang.Number  clojure.lang.Numbers.isPos (Numbers.java:96)
#2018-12-2221:43tedcushmanUsing s/or after the first argument appears to work though:
(s/explain
  (s/and pos?
         (s/or :int int? :double double?))
  3)
Success!
#2018-12-2221:44tedcushmanUnfortunately, if you are trying to write (s/and (s/or ...) (s/or ...)) there doesnā€™t seem to be a workaround.#2018-12-2221:58Alex Miller (Clojure team)s/and will flow the conformed result so the and will be receiving a value like [:int pos?]#2018-12-2221:59Alex Miller (Clojure team)prob the best option here is to (s/or :int (s/and int? pos?) :double (s/and double? pos?))#2018-12-2222:59tedcushmanok, now that I read the doc, I see the part about ā€œsuccessively conformed valuesā€.#2018-12-2718:33thomasHi All, I have been trying to write a spec recently and I can't quite describe what I want... I have a map where two sets of keys depend on each other like this:#2018-12-2718:35thomas
{:a "string"
   :b :value-b
   :c :value-c}
or
{:a "string"
  :b :value-not-b
  :c :value-not-c}
and the :b and :c values depend on each other.
#2018-12-2718:35manutter51( try three backticks to format that without emojis)#2018-12-2718:37thomasthank you @manutter51#2018-12-2718:37thomasso I either have to first version of the second one.#2018-12-2718:39thomasI assume I need something with an or and an and but how exactly?#2018-12-2718:41manutter51Iā€™m fairly shaky on spec still, but Iā€™m thinking maybe you could take advantage of namespaced vs. unnamespaced keys, let me see if I can type up what Iā€™m thinking#2018-12-2718:43thomasthank you... I don't get it either yet I'm 'fraid#2018-12-2718:45manutter51
(s/def :any/a int?)
(s/def :is-value/b #{1 2 3})
(s/def :is-not-value/b #{4 5 6})
(s/def :is-value/c #{1 2 3})
(s/def :is-not-value/c #{4 5 6})
(s/def :is-or-not/my-map
  (s/or :is (s/keys :req-un [:any/a :is-value/b :is-value/c])
        :is-not (s/keys :req-un [:any/a :is-not-value/b :is-not-value/c])))
#2018-12-2718:46manutter51Iā€™ve got a suspicion that wonā€™t work, because maps are specā€™ed by key, not by value, but a perverse part of my brain is saying maybe it would, because the keys are defined to take two mutually exclusive sets.#2018-12-2718:46thomaslet me give that a try @manutter51#2018-12-2718:48thomasI just did a few generates on that and that looks like it does work!#2018-12-2718:53manutter51Generates seem like theyā€™d be more likely to succeed, but Iā€™m not as confident about conforms with ā€œbadā€ data.#2018-12-2718:53manutter51maybe, though? :crossed_fingers:#2018-12-2718:55thomaswell... I assume if it generates the correct data, then the other way round should work as well.#2018-12-2718:56thomasbut let me try...#2018-12-2718:58thomasyes, validating works as well as expected.#2018-12-2718:59thomasThank you again @manutter51!!!#2018-12-2718:59manutter51Thatā€™s pretty wild, Iā€™ll have to remember that one myself.#2018-12-2718:59thomasa good trick to use the namespaces for that... I wouldn't have thought of that.#2018-12-2718:59thomasI suspect it is something that happens more often.#2018-12-2719:01taylorI think you could also combine your s/keys spec using s/and with another predicate, and do any "custom" assertions in the extra predicate#2018-12-2809:52borkdudeI tried to write a generator for arguments to assoc. Not sure if this is the best way to get overrides for the generators: https://github.com/slipset/speculative/blob/master/src/speculative/core.cljc#L66#2018-12-2821:38hmaurerHey! How can I define mutually recursive specs? I tried (s/def :my/spec nil) but it doesnā€™t seem to work#2018-12-2821:39taylorI think (in most cases?) it should work without you needing to register nil/dummy specs. Can you post a bigger example?#2018-12-2821:41hmaurer@U3DAE8HMG sure!
(s/def :elogic.negation/operand :elogic/formula)
(s/def :elogic/negation (s/keys :req [:elogic.negation/operand]))
(where a formula is later defined by referencing :elogic/negation)
#2018-12-2821:42hmaureractually sorry, I just realised in this specific case the formula definition can be hoisted above the negation definition, even thought it references it#2018-12-2822:41jaihindhreddyHere's a good example of mutually recursive specs by @U0FR82FU1 #2019-12-3018:13bedershello spec-fans, is there a way I can describe a map with spec that uses uuids as keys? It seems that this is outside the scope of spec and thus makes spec unsuitable for describing general purpose data schemas. Am I missing something here?#2019-12-3018:18mpenet(s/map-of uuid? any?)? Unless you mean mapping specific uuid values#2019-12-3018:30bedersThank you! That was what I was looking for !#2019-12-3022:38borkdudeI found the following workaround for the problem that (s/valid? any? ::s/invalid) is false: https://dev.clojure.org/jira/browse/CLJ-1966?focusedCommentId=51067&amp;page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-51067 Iā€™m not sure about the methods where I filled in nil, so feedback welcome#2019-12-3112:20hmaurerHow would you run https://clojure.github.io/spec.alpha/clojure.spec.test.alpha-api.html#clojure.spec.test.alpha/check as part of a deftest?#2019-12-3112:21hmaurer(I want to write a test that runs check on a function, passes if all cases pass, fails otherwise and reports the failing case)#2019-12-3112:31borkdude@hmaurer respeced has a thing for it: https://cljdoc.org/d/respeced/respeced/0.0.1/doc/readme#2019-12-3112:31borkdude@hmaurer respeced has a thing for it: https://cljdoc.org/d/respeced/respeced/0.0.1/doc/readme#2019-12-3115:11hmaurerI am getting Exception: java.util.concurrent.ExecutionException: Syntax error compiling at (clojure/test/check/clojure_test.cljc:95:1). with:
(deftest foo
  (is (successful? (stest/check `term->ast {}))))

any idea why?
#2019-01-0108:52borkdudeDonā€™t know. Can you paste the full exception somewhere?#2019-01-0109:31borkdude@hmaurer
Clojure 1.10.0
user=> (require '[respeced.test :refer [successful?]])
nil
user=> (require '[clojure.spec.test.alpha :as stest])
nil
user=> (require '[clojure.test :as t])
nil
user=> (t/deftest foo  (is (successful? (stest/check `term->ast {}))))
Syntax error compiling at (REPL:1:17).
Unable to resolve symbol: is in this context
user=> (t/deftest foo  (t/is (successful? (stest/check `term->ast {}))))
#'user/foo
user=>
#2019-01-0119:35hmaurer@borkdude the test definition works fine for me as well; itā€™s at the point when itā€™s run that the problem arises šŸ¤”#2019-01-0119:35hmaurer@borkdude actually weirdly enough calling it manually worksā€¦ but not as part of the test suite. SOmething must be fucked in my setup#2019-01-0119:40borkdude
(require '[respeced.test :refer [successful?]])
(require '[clojure.spec.test.alpha :as stest])
(require '[clojure.test :as t])
(t/deftest foo  (t/is (successful? (stest/check `term->ast {}))))
(require '[clojure.spec.alpha :as s])
(s/fdef term->ast :args (s/cat :x int?))
(defn term->ast [x] x)
(foo)
#2019-01-0119:42borkdudeif you donā€™t wrap the call to stest/check in successful? but in a doall, does it work?#2019-01-0119:43hmaurer@borkdude yep that works in the repl for me as well, but when running lein test it throws.. šŸ˜ž I donā€™t think the issue is with your library at all; I replaced the successful? function with a function that always returns false and got the same error#2019-01-0119:43borkdudeand if you remove the call to stest/check, do you still get the error?#2019-01-0119:44hmaurer@borkdude with doall, same error#2019-01-0119:44borkdudeit might be that you need to include the dependency clojure.test.check, but it depends on your error of course. I havenā€™t seen the full stacktrace#2019-01-0119:44hmaurer@borkdude nope, if I remove it I donā€™t get the error#2019-01-0119:45hmaurer@borkdude hereā€™s the full stack trace: https://gist.github.com/hmaurer/ab61d9f1e42cc961a5028fb847409e3a#2019-01-0119:45hmaurer(with lein test; I normally use kaocha)#2019-01-0119:46borkdudeit might be your spec? I see this in the error: Caused by: java.lang.ClassCastException: clojure.lang.AFunction$1 cannot be cast to clojure.lang.MultiFn#2019-01-0119:47hmaurer@borkdude well if I run stest/check in the REPL it works fine (it generates inputs, etc, and the output conforms to the :ret spec)#2019-01-0119:47hmaureršŸ¤”#2019-01-0119:48hmaurerhell, I even ran clojure.test/run-all-tests in the repl and it worked#2019-01-0119:48borkdudemaybe remove the target folder?#2019-01-0119:49borkdudedo you run the repl with the same dependencies and build system as the tests?#2019-01-0119:50hmaurer@borkdude removing the target folder didnā€™t fix it, and yes I do. lein repl and lein test respectively#2019-01-0119:51hmaurer@borkdude the dependencies are almost the same except for respeced and fulcrologic/fulcro-spec with have :scope "test" set on them#2019-01-0119:51borkdudeshould work. maybe make the smallest possible repro and put it in a separate repo#2019-01-0119:51borkdudeusing clojure.spec and not my lib#2019-01-0120:02hmaurer@borkdude ok I reproduced the error, pushing#2019-01-0120:03hmaurer@borkdude https://github.com/hmaurer/clojure-spec-check-bug#2019-01-0120:05borkdudeI get the same error when running lein test. Iā€™ll have a look#2019-01-0120:05hmaurer@borkdude Thank you šŸ™‚#2019-01-0120:11borkdudeI think you need to include :monkeypatch-clojure-test false in project.clj. then it works for me.#2019-01-0120:11borkdudeI have no idea how it works, but I found it in this issue: https://dev.clojure.org/jira/browse/TCHECK-113#2019-01-0120:13hmaurer@borkdude oh, interesting. I want to use kaocha though, so Iā€™ll have to figure out what their equivalent is. Thatā€™s still a solution though; thanks a lot šŸ™‚#2019-01-0120:16borkdudeIā€™m also using this: https://github.com/cognitect-labs/test-runner in some projects#2019-01-0120:17borkdudeoh but thatā€™s not for lein sorry#2019-01-0120:23hmaurer@borkdude switching to test.check v`0.10.0-alpha3` fixed it for me šŸ˜®#2019-01-0120:24hmaurer(as pointed out in the issue you linked)#2019-01-0120:48borkdudecool#2019-12-3112:31borkdude
(is (successful? (check `foo {} {:num-tests 10})))
#2019-12-3113:01hmaurer@borkdude fantastic, thank you!#2019-12-3113:02eoliphantdo specs support multimethods yet?#2019-12-3113:05hmaurer@borkdude out of curiosity, if the check fails, will this report the failing case as part of the failed test report?#2019-12-3113:07borkdude@hmaurer yes, you can see an example here: https://circleci.com/gh/borkdude/speculative/17#2019-12-3113:08borkdude(look at the script/test step)#2019-12-3123:31richiardiandreasay I have a password key in a map that is speced...and some other key does not validate, this will throw an exception...is there a way to tell spec NOT to include that sensitive key in the ex-data at all?#2019-01-0100:33seancorfield@richiardiandrea Isn't the concern there more than you need to not display ex-data values "as-is" to anyone?#2019-01-0100:34seancorfieldIt would be the same as any other ex-info you throw -- it could well include sensitive data in some context...#2019-01-0100:34richiardiandreathat's a tough one, because then my display thing needs to know about what is displaying#2019-01-0100:35richiardiandreaaka, I need to call dissoc on things, maybe it should#2019-01-0100:35seancorfieldI wouldn't expect production code to just display any spec failure as is -- those exceptions are intended for code, not humans, and certainly not end users.#2019-01-0100:36seancorfieldFWIW, our application error logging/reporting code has always contained logic to strip known, sensitive fields from any data logged or reported, long before spec was a thing...#2019-01-0100:38richiardiandreaok yeah maybe that's something I would need to do anyways#2019-01-0100:39richiardiandreaI wonder though if things change ...say the string for password changes now I need to change spec AND dissoc#2019-01-0102:12seancorfield@richiardiandrea That's a good reason to use a globally-unique qualified key name šŸ™‚#2019-01-0309:48y.khmelevskiiHi everyone! Just curious, does anybody know when clojure/clojurescript with spec/schema and spec/select will be released? I like this idea and want to try it šŸ™‚#2019-01-0313:34Alex Miller (Clojure team)We have not started developing it yet, so not soon :)#2019-01-0313:34Alex Miller (Clojure team)We have not started developing it yet, so not soon :)#2019-01-0320:18y.khmelevskiithank you for info šŸ™‚#2019-01-0316:25borkdudewhat is generally preferred in specs? (s/nilable (s/or ...) vs (s/or :nil nil? ā€¦) when enumerating alternatives#2019-01-0316:25Alex Miller (Clojure team)the first is perf optimized#2019-01-0316:25borkdudethanks#2019-01-0316:26borkdudeit might also be more robust, since some predicates can crash on nil#2019-01-0316:26Alex Miller (Clojure team)the generator for nilable is also intentionally unbalanced so it only produces nils 10% of the time#2019-01-0316:26Alex Miller (Clojure team)whereas the latter will be 1/n where n = # of options#2019-01-0316:28borkdudecan something similar be said about s/alt vs s/or in a regex?#2019-01-0316:29Alex Miller (Clojure team)not sure what you mean. do you mean nilable vs alt on nil?#2019-01-0316:29Alex Miller (Clojure team)if so, then thatā€™s a different situation as s/nilable is not a regex and wonā€™t compose in the same way as s/alt with other regex ops#2019-01-0316:29borkdudeno sorry, (s/cat :foo int? :x (s/alt ...)) vs s/or instead of alt#2019-01-0316:30borkdudeso independent of nil#2019-01-0316:30borkdudeIā€™m looking for an argument other than ā€œalt belongs in regexā€#2019-01-0316:30Alex Miller (Clojure team)s/alt is a regex op and composes with other regex ops to describe a single sequential structure#2019-01-0316:30Alex Miller (Clojure team)s/or is not a regex op#2019-01-0316:31Alex Miller (Clojure team)so acts as an independent value#2019-01-0316:31borkdudesometimes the spec for :x can be taken apart and then it only makes sense to do it with or to make it re-usable on a single value#2019-01-0316:31Alex Miller (Clojure team)it depends here on what ā€¦ is#2019-01-0316:32borkduderight, ā€¦ can contain other regex specs#2019-01-0316:32Alex Miller (Clojure team)in some cases, there is no perceptible difference#2019-01-0316:32borkdudeso, it depends then#2019-01-0316:32Alex Miller (Clojure team)there are cases where you might want s/alt and some where you might want s/or#2019-01-0316:33Alex Miller (Clojure team)if youā€™re describing alternatives of the sequential structure, then probably s/alt#2019-01-0316:33Alex Miller (Clojure team)if youā€™re describing alternate value sets that occur at a particular point in the structure, then probably s/or#2019-01-0316:34Alex Miller (Clojure team)usually the s/alt case will contain more regex ops inside the ā€¦ whereas the s/or will not#2019-01-0316:34borkdudeclear, thanks!#2019-01-0417:44dacopareWhat's the best way to integrate spec with clojure.test? I want to run my spectest/checks as part of the test suite.#2019-01-0418:33taylorI've used this pattern before, not sure if it's good or not:
(deftest foo-test
  (is (= 1 (-> (st/check `foo)
               (st/summarize-results)
               :check-passed))))
#2019-01-0418:34taylordoesn't give meaningful output on failure#2019-01-0418:20rapskalian@dacopare Clojure.test.check has the clojure.test.check.clojure-test/defspec macro for that purpose. https://github.com/clojure/test.check#2019-01-0418:29Alex Miller (Clojure team)thatā€™s not going to help you here#2019-01-0418:38borkdude@dacopare Iā€™m using respeced (a lib I wrote). Hereā€™s an example how I use it: https://github.com/borkdude/speculative/blob/master/test/speculative/core_test.cljc#2019-01-0419:29aviIā€™ve got a few primitive functions to glue specā€™s check and clojure.test together, along with expound; I originally copy-pastaā€™d them from a gist and then massaged mangled them over time: https://github.com/FundingCircle/fc4-framework/blob/master/tool/src/test_utils/fc4/test_utils.clj#2019-01-0419:30avi(Oh cool, that snippet includes the entire file if you expand it. Too bad it doesnā€™t include syntax highlighting.)#2019-01-0505:37dacopareThank you, @borkdude and @aviflax. I'll have a look at both.#2019-01-0517:29kuzmin_mIs it possible to relax requirements in s/keys? s/keys check input value with map? predicate. I try to spec datascript entity, but it implements only clojure.lang.Associative. https://github.com/tonsky/datascript/blob/master/src/datascript/impl/entity.cljc#L128 https://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/alpha.clj#L857#2019-01-0520:20mishaentities are lazy. validating/conforming with s/keys would imply realizing and walking the entity graph (friends of friends of friends ...)#2019-01-0604:21mathpunkThis doesn't look right at all. Am I using instrument incorrectly?#2019-01-0604:22lilactown@mathpunk :args needs to be, at the very least, a collection#2019-01-0604:22lilactownyou can think of it as, youā€™re verifying the [s] collection#2019-01-0604:24lilactownusually I use s/cat. e.g.:
(s/fdef friendlier
  :args (s/cat :s string?))
#2019-01-0604:24mathpunkohhhhh#2019-01-0604:24mathpunkok i think I see this#2019-01-0604:25lilactown(s/cat :s string?) checks if the first element in the sequence is a string#2019-01-0604:25mathpunkand we're verifying an args /vector/, which happens to have one element#2019-01-0604:26lilactownright#2019-01-0604:26lilactownitā€™s actually a sequence, I think. you can see itā€™s printed as ("World")#2019-01-0604:27mathpunkok this makes sense, thank you#2019-01-0604:28mathpunkand yeah it's right there but i guess i thought those were decorative parens šŸ™„#2019-01-0604:28lilactownyeah šŸ˜… trust me I went through the same confusion the first time I tried to use fdef#2019-01-0714:50urzdsAre logs of this channel available somewhere?#2019-01-0714:51kuzmin_mhttps://clojurians-log.clojureverse.org/clojure-spec#2019-01-0721:22kennyIs there a way to limit the size of the a map generated from (gen/generate (s/gen map?))?#2019-01-0721:25taylorgenerate can take another argument size e.g. (gen/generate (s/gen map?) 10)#2019-01-0721:25taylorI think the general range of sizes is 0-200 maybe?#2019-01-0721:26taylorlooks like it uses 30 by default#2019-01-0721:24Alex Miller (Clojure team)no, but you can use something like (s/map-of any? any? :gen-max 5)#2019-01-0721:35borkdudeis there a better way of making a transducer spec than ifn?? e.g. this works, but it shouldnā€™t:
(s/conform (:ret (s/get-spec `map)) [1 2 3])
[:transducer [1 2 3]]
#2019-01-0721:41Alex Miller (Clojure team)Iā€™ve looked at it, and generally Iā€™d say no#2019-01-0721:42Alex Miller (Clojure team)this is well outside the sweet spot for spec#2019-01-0721:52kennyI'm seeing strange behavior when running clojure.spec.test.alpha/check on an fdef'ed function. I have a prn as the first line in my function that prints out the size of the args. When the generative tests first start running, I get print statements out rapidly. After running for a few seconds, the print statements slow down to once every second. After 10-20s more, they speed up again. This goes on for about 10 mins until check returns. At first I thought it was due to test.check generating large maps so I changed the :args in my fdef to use (s/map-of any? any? :gen-max 5). Then I thought it was a JVM memory issue but (.maxMemory (Runtime/getRuntime)) says the JVM has over 7gb available. I haven't seen this behavior with spec check before. Any ideas on what could cause this?#2019-01-0721:58Alex Miller (Clojure team)sounds awfully like GC churn to me#2019-01-0721:59Alex Miller (Clojure team)do you have an fspec anywhere?#2019-01-0722:00kennyNot in this function.#2019-01-0722:01Alex Miller (Clojure team)you can add -verbose:gc to watch the gc#2019-01-0722:02Alex Miller (Clojure team)you could isolate just the gen on the args if you suspect that itā€™s it#2019-01-0722:04Alex Miller (Clojure team)something like
(gen/sample (s/gen (:args (s/get-spec `foo))) 1000)
#2019-01-0722:48kennyGenerating the args like you have there takes about 30s. I'll add the gc flag to see if it sheds any light.#2019-01-0722:50kennyAdded the gc flag. I get a message that looks like this [GC (Allocation Failure) 2477246K->1068161K(3133952K), 0.1575999 secs] printed out every second or so.#2019-01-0723:06kennyThis is what the output with the -verbose:gc flag looks like https://pastebin.com/raw/XCxMykgq. The numbers are the count on the two args this function takes.#2019-01-0723:08kennyUpdated to test.check 0.10.0-alpha3 and the problem goes away.#2019-01-0723:53lilactownI have a map like:
{::kind :bool
 ::default true}
that, given a different ::kind, might need a different predicate for ::default. E.g.:
{::kind :string
 ::default "foo"}
What's the best way to model this in spec? I thought multi-specs would work, but I'm still struggling with the fact that the name of the spec I pass into s/keys needs to match the key, which makes overloading ::default difficult
#2019-01-0723:57mattly@lilactown I've done this with s/or specs#2019-01-0723:57mattlyI can give you an example in our work slack#2019-01-0800:14lilactownAFAICT there's no way to do this without using :req-un šŸ˜ž#2019-01-0800:19favilahttps://gist.github.com/favila/ab03ba63e6854a449d64d509aae74618 is a hack I wrote a while ago that will add an additional thing to conform to for some of the named keys#2019-01-0800:19favila#2019-01-0800:20favilaspec is super duper opinionated on this point though#2019-01-0800:20favilahence the complexity of the hack#2019-01-0800:21favilause like (keys+ :req [::whatever] :conf {::whatever narrower-specish-thing})#2019-01-0800:23favila::whatever value will be asked to validate against both its own spec and narrower-specish-thing (so ::whatever should be the widest possible spec you could have for that key)#2019-01-0800:23favilabut narrower-specish-thing will be used for conforming and generators#2019-01-0800:25favilaThe idea is that contextually (in a specific map) a key may have a narrower spec than normal#2019-01-0800:26favilawhich for some reason happens to me all the time and made spec very painful#2019-01-0800:26favilathe alternative is s/or with more predicates, and with-gen to adjust the generator#2019-01-0800:26mattlyyou could also just forgo using s/keys and do it manually (spec/def :my-union/shape (fn [thing] (case (::kind thing) :bool (if (boolean? (::default thing)) true ::s/invalid) ::s/invalid)))#2019-01-0801:09lilactownhm. yeah, I think I settled on:
(defmulti parameter-kind ::kind)

(defmethod parameter-kind :bool [m]
  #(if (boolean? (::default %))
     true
     false))

(s/def ::parameters (s/map-of keyword?
                              (s/and
                               (s/keys :req [::kind])
                               (s/multi-spec parameter-kind ::kind))))
#2019-01-0803:26Alex Miller (Clojure team)s/multi-spec is kind of designed to use different specs based on data#2019-01-0803:27Alex Miller (Clojure team)youā€™d need to use it with s/keys and :req-un here though since you have the same attribute name with different specs apparently#2019-01-0814:28urzdsBack in December I asked whether it was possible to spec protocol methods, where the answer was "no" and "because of the implementation that is targetted at performance". I am wondering whether I could instead use multi-methods instead of protocols and methods and spec them. (My code should allow replacing the record with a map and the protocol methods with multi-methods.) Can I just use s/fdef on the multifn and spec :args and :ret that have to be valid for all implementations? An alternative would be to use pre/post conditions, but I cannot see anything resembling a pre-post-map (as is present for defn) in the docs for defmulti or defmethod.#2019-01-0814:37Alex Miller (Clojure team)Currently, I do not believe that works, but I think it could be made to work#2019-01-0814:54urzdsI just tried the following code and got no error, which I guess suggests that it does indeed not work:
(require '[clojure.spec.alpha :as s])
(defmulti testfn :type)
(defmethod testfn :atype [m] 1)
(s/fdef testfn :args (s/keys :req-un [::type ::does-not-exist]))
(testfn {:type :atype})
; => 1
What would be the path forward from here? Should I open an issue / feature request for https://github.com/clojure/spec.alpha ?
#2019-01-0815:34urzds@alexmiller ^^#2019-01-0815:51Alex Miller (Clojure team)we handle spec issues in the main CLJ jira system and I think there already is one for this#2019-01-0815:53Alex Miller (Clojure team)you didnā€™t call stest/instrument in the example above so that at least is a missing step#2019-01-0815:53Alex Miller (Clojure team)https://dev.clojure.org/jira/browse/CLJ-2450 is one issue#2019-01-0816:00urzdsYou're right, the following code at least throws an exception:
(require '[clojure.spec.alpha :as s])
(require '[clojure.spec.test.alpha :as stest])
(defmulti testfn :type)
(defmethod testfn :atype [m] 1)
(s/fdef testfn :args (s/keys :req-un [::type ::does-not-exist]))
(stest/instrument `testfn)
(testfn {:type :atype})
=> clojure.lang.ExceptionInfo: Call to #'user/testfn did not conform to spec.
But it also throws this exception for arguments that should conform:
(testfn {:type :atype :does-not-exist 1})
=> clojure.lang.ExceptionInfo: Call to #'user/testfn did not conform to spec.
Sadly there is no explanation why...
#2019-01-0816:06Alex Miller (Clojure team)I donā€™t actually see a CLJ issue for just ā€œmultimethods canā€™t be instrumentedā€ but would be ok to make one if you like! Like I said, I think this is something that is fixable.#2019-01-0816:06Alex Miller (Clojure team)actually I think your spec is wrong#2019-01-0816:07Alex Miller (Clojure team)youā€™re missing the top level args sequence#2019-01-0816:07Alex Miller (Clojure team)(s/fdef testfn :args (s/cat :m (s/keys :req-un [::type ::does-not-exist])))#2019-01-0816:08Alex Miller (Clojure team)that works for me#2019-01-0816:09urzdsOups#2019-01-0816:12urzdsCan specs be redefined? I just tried to execute (s/fdef testfn :args (s/cat :m (s/keys :req-un [::type ::does-not-exist]))) in the same REPL session, which appeared to be successful, but spec would still throw an exception even when I passed in the correct arguments. Only restarting the process fixed that.#2019-01-0816:15Alex Miller (Clojure team)you need to instrument again#2019-01-0816:16Alex Miller (Clojure team)or possibly unstrument / instrument (although I think either will work)#2019-01-0816:16urzdsyes, just calling stest/instrument again worked.#2019-01-0816:23borkdude@urzds if you use something like component, you can hook up re-instrumentation with the start/stop lifecycles#2019-01-0816:24urzds@borkdude Thanks!#2019-01-0816:28urzdsBTW, I also found the request for specs for protocol methods (my original question): https://dev.clojure.org/jira/browse/CLJ-2109#2019-01-0817:12urzds@alexmiller Could you please expand on the rationale behind this? https://dev.clojure.org/jira/browse/CLJ-2378?focusedCommentId=49601&amp;page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-49601 In https://groups.google.com/d/msg/clojure/RLQBFJ0vGG4/UGkYS7U_CQAJ you refer to the changelog, but that does not appear to include history data: https://github.com/clojure/spec.alpha/blob/master/CHANGES.md#2019-01-0817:16Alex Miller (Clojure team)So just repeating my comment in the jira, the idea behind instrument is to check whether a function has been correctly invoked#2019-01-0817:17Alex Miller (Clojure team)The idea behind check is to verify that a function produces proper outputs in response to valid inputs#2019-01-0817:31urzdsHm, I got that, but I would also like to see, during development, that a function is being invoked correctly and it responds correctly in live workloads. I.e. in my use-case I want more than to just check whether it was invoked correctly, but I also am not running the function in a test where I could use check. How is my use-case to be handled?#2019-01-0817:32urzdsWhat I did not understand is the technical necessity for instrument not to also check the return value. I assumed knowing why you decided against that might help me understand the context better.#2019-01-0817:42borkdude@urzds instrument can take a performance hit when you have a lot of fdefs. if you check the return values, you will often check those twice, since they are arguments for another function. so I think itā€™s a sane default#2019-01-0817:43borkdudethis may not the reason that core decided to do this, but I have come to appreciate it for this reason#2019-01-0817:44urzdsIn one of my cases these functions are GraphQL resolvers, invoked by Lacinia. So there really is nothing coming afterwards in my own code where I could check the return value... (And GraphQL schemas are not as expressive as Clojure Spec.)#2019-01-0817:45borkdude@urzds you can try to write a generative test for it.#2019-01-0817:45urzdsMaybe my understanding of Spec is wrong, though. I thought I should spec everything that goes into my system and everything that goes out of it, in order to ensure that it behaves nicely with others.#2019-01-0817:46Alex Miller (Clojure team)instrument has functionality to do stubs and mocking too, which can be used in combination with check#2019-01-0817:47borkdude@urzds some examples of generative tests of fdefs: https://github.com/borkdude/speculative/blob/master/test/speculative/core_test.cljc#2019-01-0817:47borkdudea ret and fn spec is really useful to check if your implementation is correct in combination with generative testing#2019-01-0817:49borkdude@urzds you can always plug in the ret spec checking manually, if you want. just separate out the spec and call valid?#2019-01-0817:49borkdudeor use :pre and :post#2019-01-0817:50urzdsHm, maybe I should find ways to split up those functions better in order to allow generative testing. Seems a bit difficult right now, because they rely on input from external services. So I just have unit tests for my internal functions, and wanted to rely on spec throwing exceptions when the interaction with the outside world shows signs of problems.#2019-01-0817:50borkdudeyou can also use spec/assert#2019-01-0817:50borkdudelots of options#2019-01-0817:51urzds:pre and :post do not work here, because the functions are actually protocol methods (could be changed to multi-methods, but those also do not support :pre and :post).#2019-01-0817:51borkdudespec/assert can also be elided with compile time options.#2019-01-0817:52urzdsspec/assert is what I am using right now. The whole body of the function is wrapped in spec/assert, which looks a bit ugly TBH and the commit introducing this will cause a large amount of code to be reformatted for indention, which is also undesirable.#2019-01-0817:53borkdudethen donā€™t wrap. you donā€™t have to. and when you elide the call, youā€™ll be left with an empty function#2019-01-0817:56urzdsHow would I not wrap and still check the return value, in the absence of :post?#2019-01-0818:40borkdude@urzds
(fn [args]
  (s/assert args-spec args)
  (let [res (calc args)]
    (s/assert ret-spec res)
    res))
#2019-01-0911:42urzdsAnother recommendation I received was something along these lines:
(defn wrap-fn-with-spec [f s-args s-reg]
  (fn [& args]
    (s/assert s-args args)
    (let [ret (apply f args)]
      (s/assert s-ret ret)
      ret)))
#2019-01-0911:43borkdudeyeah same idea. you could also do it with a macro to receive better line errors#2019-01-0911:45urzdsIs there a way to retrieve the specs attached to the symbol using s/fdef? That might make the above function integrate a bit better with the rest of spec.#2019-01-0911:46borkdudeyes, (s/get-spec foo)` and then call :args or :ret on it.#2019-01-0911:46borkdudebut another option is to spec the args and ret spec separately, so you can write a more specific generator for it#2019-01-0911:48urzdsSo like this?
(defn wrap-fn-with-spec [f]
 (let [s-fn (s/get-spec f)
       s-args (:args s-fn)
       s-ret (:ret s)]
   (fn [& args]
     (s/assert s-args args)
     (let [ret (apply f args)]
       (s/assert s-ret ret)
       ret)))
#2019-01-0911:48borkdudeIā€™m not sure if s/get-spec works with a function, I donā€™t think so. so youā€™ll need the symbol for the function too#2019-01-0911:49borkdudejust try it from the REPL and youā€™ll see.#2019-01-0911:49borkdudeah, it works with a var: https://clojuredocs.org/clojure.spec.alpha/get-spec#2019-01-0911:49borkdudeso youā€™re better off passing the var then#2019-01-0911:49urzds> but another option is to spec the args and ret spec separately, so you can write a more specific generator for it How do you mean? Doesn't s/fdef have the args and ret spec separately already?#2019-01-0911:50borkdudeyes, but they donā€™t have a specific name, so you cannot generate specific combinations of arguments. hereā€™s an example for assoc: https://github.com/borkdude/speculative/blob/master/test/speculative/core_test.cljc#L107#2019-01-0911:55borkdudeIā€™m not saying you have to, but it gives options#2019-01-0911:56urzdsThanks!#2019-01-0817:53borkdudegotta go#2019-01-0817:57urzdsGuess I gotta read more docs about this subject.#2019-01-0818:39dustingetzIs it possible to spec this: #<Track: [nil :fiddle/links 1234] ā€“ a tuple inside a deftype#2019-01-0819:39Alex Miller (Clojure team)deftypes do not expose their structure (other than the type)#2019-01-0819:39Alex Miller (Clojure team)whether you can spec it in some other ways depends on what interfaces/protocols you implement and which predicates you want to spec it with#2019-01-0819:40Alex Miller (Clojure team)so without more info, I would default to: no :)
#2019-01-0916:07ShaneLesterWhatā€™s the normal way I should be handling speccing functions that have optional arguments?#2019-01-0916:12Alex Miller (Clojure team)generally you should use regex ops (like s/cat) to spec args and ops like s/? to handle optional arguments within that#2019-01-0916:13ShaneLesterCool. Thanks šŸ™‚#2019-01-0920:17jstewIs there a way to match an exact number of items in a collection or sequence? Letā€™s say I want to spec a seq or collection to have 3 and only 3 items in it?#2019-01-0920:19jstewI suppose I could always assert that in the spec for the fn that uses the collection.#2019-01-0920:23tangrammerAs spec guide states šŸ™‚ ā€¦
(s/def ::point (s/tuple double? double? double?))
(s/conform ::point [1.5 2.5 -0.5])
=> [1.5 2.5 -0.5]

https://clojure.org/guides/spec#_collections
#2019-01-0920:23jstewtuple! Thank you.#2019-01-0920:35Alex Miller (Clojure team)s/tuple is good for heterogeneous fixed-size (ā€œslottedā€) vectors#2019-01-0920:35Alex Miller (Clojure team)s/coll-of with the :count option is good for homogenous (all items match the same pred) collections#2019-01-0920:36Alex Miller (Clojure team)like (s/coll-of double? :count 3)#2019-01-0920:36jstewThat fits this use case better. Thanks, Alex.#2019-01-0920:36Alex Miller (Clojure team)either will work and they conform and gen pretty similarly but I think one or the other usually has a better match on intent#2019-01-0921:07ShaneLesterIā€™m attemping to make a spec for something that can either be any number of vectors, or 0 or 1 strings. currently I have (spc/or :vectortype (spc/* vector?) :stringtype (spc/? string?)) but that doesnā€™t really seem to be working. (I know this is whacky but, using it to learn the nuances of spec#2019-01-0921:08ShaneLesterAnyone see if I am doing something obviously wrong there?#2019-01-0921:09manutter51Looks fair to me, what are you getting from that that makes you say itā€™s not working?#2019-01-0921:10ShaneLesterGetting a message in-console saying that the function call does not match the spec when the argument that that spec covers is just a string#2019-01-0921:13manutter51Function arguments need to be inside (spc/cat ...) maybe youā€™re missing that?#2019-01-0921:14ShaneLesterI guess this would have been more useful- here is the whole spec for the function currently (spc/fdef typog :args (spc/cat :props map? :children (spc/or :vect (spc/* vector?) :stringt (spc/? string?))))#2019-01-0921:14ShaneLesterso I do have cat. but not sure if im using it right#2019-01-0921:16manutter51Do you have the :ret key in your fdef too? Iā€™ve seen odd results when that was missing.#2019-01-0921:16ShaneLesterHmm I dont, ill try adding that#2019-01-0921:25manutter51Everything else looks good to me, with the caveat that Iā€™m still getting comfortable with spec myself, so I might be missing something.#2019-01-0921:30ShaneLesterI just threw a :ret any? in there, but same results sadly. Yeah. It seems to logically make sense to me, and to others Iā€™ve showed it to. Something strange going on perhapsā€¦#2019-01-0921:30ShaneLesterI appreciate your help šŸ™‚#2019-01-0921:43Alex Miller (Clojure team)the s/or is not a regex op so inserts a new ā€œlevelā€ of nesting#2019-01-0921:45Alex Miller (Clojure team)so that is always expecting 2 args with the first a map and the second either a collection of 0 or more vector or an empty collection or a collection containing one string#2019-01-0921:45Alex Miller (Clojure team)I donā€™t think thatā€™s what you want#2019-01-0921:46Alex Miller (Clojure team)I suspect changing the or to alt is probably closer, would at least match the string case you want#2019-01-0921:46Alex Miller (Clojure team)but depends what you mean by ā€œ0 or 1 stringsā€#2019-01-0921:47Alex Miller (Clojure team)if you mean ā€œno 2nd argā€ for ā€œ0 stringsā€, then that case is already covered by (* vector?), which can be 0 vectors#2019-01-0922:02ShaneLesterAhh it was alt! That is what I needed. I did gloss over that in the api but apparently that did not make enough sense for me. haha. Thank you alex.#2019-01-0922:19ShaneLesterAlso good point about the case of the arg not being there handled by the *, that is definitely true.#2019-01-1114:35mpingHi#2019-01-1114:35mpinghow can I create a spec for a map like
{"2018-01-01": {:value 10}, "2018-03-22: {:value 20}}
#2019-01-1114:35mpingmap where keys are dates#2019-01-1114:37Alex Miller (Clojure team)write a function that decides whether that is a valid string - that function is your predicate#2019-01-1114:39Alex Miller (Clojure team)
(s/def ::date valid-date?)
(s/def ::value int?)
(s/def ::data (s/keys :req-un [::value]))
(s/def ::m (map-of ::date ::data))
#2019-01-1114:52mpingtks Alex, pretty much what I had, problem must be somewhere else.#2019-01-1115:03ikitommi@mping you might want to add :conform-keys true to map-of#2019-01-1115:07Alex Miller (Clojure team)with a pred, the keys will will just conform to themselves, so thatā€™s just making it slower#2019-01-1115:23mping@alexmiller in what cases would conform-keys be needed?#2019-01-1115:23mpingcore specs?#2019-01-1115:26Alex Miller (Clojure team)cases where the keys need to be conformed#2019-01-1115:27Alex Miller (Clojure team):)#2019-01-1115:27Alex Miller (Clojure team)most maps have keys that are strings, numbers, keywords, uuids, etc - all of which conform to themselves, so there is no reason to conform the map keys#2019-01-1115:28Alex Miller (Clojure team)but if you had a map whose keys conformed to a value other than themselves (a regex spec, an s/or, a nested map, etc) then you would want to consider it#2019-01-1115:29Alex Miller (Clojure team)but even in that case, you have to think carefully to make sure the conformed values donā€™t create a case where two map keys have the same conformed value. if so, they will collide in the conformed map.#2019-01-1115:30mpingI see.#2019-01-1115:31mpingthanks guys#2019-01-1200:08blanceIs it idiomatic to use spec to validate data with remote service through http request?#2019-01-1200:09seancorfield@blance I think it's idiomatic to use spec for validating data in a lot of cases. Can you be a bit more specific?#2019-01-1200:10seancorfieldFor example, we use to spec to validate (and conform) input arguments to our REST APIs and in some other places either before or after interaction with an external system.#2019-01-1200:11blancesay I have a user id that needs to be an int, normally i would just validate it using int?. i'm just curious does it make sense to also do
(def ::user (s/and int? user-exist?)) 
#2019-01-1200:11blancewhere user-exist would have to call to an external service to check if user exists#2019-01-1200:16seancorfieldI'd be careful about using predicates that rely on outside systems for your specs. That feels wrong to me.#2019-01-1200:16lilactownif you try and use ::user with generators, are you going to be upset if it sends hundreds of thousands of requests to your external service?#2019-01-1200:17seancorfieldYup. I would say it's worth trying to stick with specs that can be used in generators.#2019-01-1200:17lilactownsometimes it's surprising which spec functions use generators too. e.g. instrumentation#2019-01-1200:19blancei haven't really start looking at generative testing yet#2019-01-1200:19blancei suppose http request can be mocked during testing?#2019-01-1200:25blancereading more on the doc,
A key design constraint of spec is that all specs are also designed to act as generators of sample data that conforms to the spec
sounds like I shouldn't be using any non trivial pred that can't easily generate good sample data
#2019-01-1212:37mishaYou can use non-trivial predicates (and, I think, it is one of the value props of spec), but if you end up using it for generative testing you will have to help that predicate with a custom generator to minimize "Couldn't satisfy such-that predicate after 100 tries" errors (https://clojure.org/guides/spec#_custom_generators) You can write predicates with side-effects, but as with usual code ā€“ try to isolate them from the pure ones, and know all the contexts where those might be used. So checking if user exists is perfectly fine, but probably should be done at a very specific points of the system, separate from checking hypothetical user's structure. Also: "not everything needs to be a spec", as well as "not everything needs to be specced". @blance#2019-01-1223:51blancethat make perfect sense, thanks!#2019-01-1220:38trevorI'm struggling on how to create a generator that returns a constant value#2019-01-1220:38trevormy example is UUID or constant id#2019-01-1220:39trevor(gen/one-of [(s/gen ::uuid) ...])#2019-01-1220:39trevorlet's say the constant is 0#2019-01-1220:45Alex Miller (Clojure team)(s/gen #{::uuid})#2019-01-1220:46trevorI was looking for gen/return#2019-01-1311:45borkdudeI have a spec of map-entry and seqable-of-map-entry. Sometimes I get this exception while generating values:
(require '[clojure.spec.gen.alpha :as gen])
(require '[clojure.spec.alpha :as s])
(s/def ::map-entry
  (s/with-gen map-entry?
    (fn []
      (gen/fmap first
                (s/gen (s/and map? seq))))))
(s/def ::seqable-of-map-entry
  (s/coll-of ::map-entry :kind seqable?))

(gen/sample (s/gen ::seqable-of-map-entry))
Error printing return value (ClassCastException) at clojure.core/conj (core.clj:82).
java.lang.String cannot be cast to clojure.lang.IPersistentCollection
#2019-01-1311:46borkdudeI wonder why it tries to call conj at all here#2019-01-1311:49borkdudeI can imagine it tries to build a seqable, so it starts with an empty string, and then it tries to conj map-entries to it:
(conj "" (first {:a 1}))
#2019-01-1311:51borkdudeif this is a bug, I would be happy to file it in JIRA#2019-01-1311:52borkdudefor now I can use this workaround:
(s/def ::seqable-of-map-entry
  (s/with-gen (s/coll-of ::map-entry :kind seqable?)
    (fn []
      (s/gen (s/coll-of ::map-entry :kind list?)))))
#2019-01-1312:01borkdudeI could also add :into [] but that would exclude lazy seqs#2019-01-1313:51Alex Miller (Clojure team)coll-of always gens a collection, never a lazy seq#2019-01-1313:59borkdudeYes, I meant, if I add the into, it would generate correctly but it would realize lazy seqs #2019-01-1312:03borkdudemaybe something like this? (doesnā€™t work yet)
(s/def ::seqable-of-map-entry
  (s/coll-of ::map-entry :kind (s/with-gen seqable?
                                 #(s/gen vector?))))
#2019-01-1312:25borkdudeso kind must be a predicate and cannot be a spec, but it must also generate. in other words, you can only use pre-defined predicates?#2019-01-1312:33borkdudethis seems to work:
(defn seqable-of
  "Prevents generating strings and therefore Exceptions during generation"
  [elt-spec]
  (s/with-gen (s/coll-of elt-spec :kind seqable?)
    #(s/gen (s/coll-of elt-spec :kind vector?))))
#2019-01-1313:52Alex Miller (Clojure team):kind seqable? does not make sense#2019-01-1313:52Alex Miller (Clojure team)coll-of always gens a collection#2019-01-1313:53Alex Miller (Clojure team)Itā€™s a collection spec#2019-01-1314:00borkdudeWhatā€™s the recommended way of specā€™ing a seqable of something then?#2019-01-1314:07borkdudeMaybe seqable and only checking the first value?#2019-01-1314:57borkdudeMaybe just coll-of would work, since
(coll? (seq {:a 1 :b -1 :c 1 :d -1}))
(coll? (filter (comp pos? val) {:a 1 :b -1 :c 1 :d -1}))
are both true
#2019-01-1315:04borkdudenope, it really should be a seqable, since (java.util.HashMap. {:a 1}) is also supposed to work.#2019-01-1315:08borkdudeevery might be the one I should use then#2019-01-1315:17borkdudethatā€™s it. every also only checks a maximum number of elts, so a lazy infinite seq would still be supported. thanks. :duck:#2019-01-1315:19borkdudeisnā€™t this a bit inconsistent, since nil puns as an empty sequence?
user=> (s/conform (s/every string?) '())
()
user=> (s/conform (s/every string?) nil)
:clojure.spec.alpha/invalid
user=> (s/conform (s/every string? :min-count 0) nil)
:clojure.spec.alpha/invalid
user=> (count nil)
0

#2019-01-1315:58borkdude(s/conform (s/every string? :kind seqable?) nil) works though, but then Iā€™m back into the same problem where I started:
user=> (gen/sample (s/gen (s/every string? :kind seqable?)))
Error printing return value (ClassCastException) at clojure.core/conj (core.clj:82).
java.lang.String cannot be cast to clojure.lang.IPersistentCollection
#2019-01-1315:59borkdudeso maybe (s/nilable (s/every ::map-entry)) is best then#2019-01-1316:12Alex Miller (Clojure team):kind seqable? is just not right here. every (like coll-of) is a spec for collections (not seqables). you are using a broader predicate for :kind than the spec is intended for.#2019-01-1316:15borkdudewhy is (s/nilable (s/every ::map-entry)) not the right fit for seqables? the implementation uses seq on the input and then checks every element up to a limit. so if this is not it, whatā€™s the alternative?#2019-01-1316:19borkdudeif you donā€™t specify the kind to every, whatā€™s the default?#2019-01-1316:42Alex Miller (Clojure team)Iā€™m saying seqable? does not make sense with coll-of/every because some seqables are not collections#2019-01-1316:43Alex Miller (Clojure team)The default is vector iirc#2019-01-1318:39borkdude@alexmiller would this be OK?
(defn seqable-of [spec]
  (s/with-gen (s/and seqable?
                     (s/or :empty empty?
                           :seq (s/and (s/conformer seq)
                                       (s/every spec))))
    #(s/gen (s/nilable (s/every spec :kind coll?)))))
#2019-01-1406:25devthi wonder why aren't specs first class things we can define literally and pass around, and why they need their own special registry when we already have namespaces and vars? šŸ¤” i'm playing with building up specs in an automated way (e.g. reducing a data structure into a spec representation). looks like i'm gonna have to s/def things along the way. isn't that some kind of PLOP? maybe this is something that will be improved in future version šŸ˜„#2019-01-1409:34mpenetThe latter it seems#2019-01-1409:36mpenetWhy another registry, I believe to not clutter vars/nses more and make their potential evolution separate?#2019-01-1415:33devththen why not have a registry for all atoms? and another for all fns? and on and on. šŸ˜‚#2019-01-1415:35mpenetThere are already issues with vars initialization at startup right now, I guess that's not to make it worse among other thing. Then it's a "private" thing, we never get exposed to the fact it's separate#2019-01-1415:36mpenet-> https://dev.clojure.org/display/design/Lazy+var+loading#2019-01-1415:36devthi see. šŸ˜ž#2019-01-1409:38mpenetWe ll see with alpha2 I guess. The posts from @alexmiller about the ongoing work on this are quite interesting #2019-01-1409:40mpenetthey don't reveal much about these 2 questions in particular, but soon enough it might#2019-01-1409:51borkdude@devth if you can give an example, we can see if it can be improved using the current version of spec?#2019-01-1416:20devthtried to explain the problem in detail at https://clojurians.slack.com/archives/C1B1BB2Q3/p1547482812312100#2019-01-1411:04misha@devth there is s/spec which gives you spec w/o being registered, which might be useful for you if you build specs dynamically, use, and throw away right away.#2019-01-1413:58Alex Miller (Clojure team)there are no plans to change the registry aspect of spec#2019-01-1413:59Alex Miller (Clojure team)all of the programmatic construction aspects will be different in spec 2#2019-01-1415:45flyboarder@alexmiller could you elaborate more on that? Will existing dynamic specs break?#2019-01-1415:56Alex Miller (Clojure team)hard to say yet#2019-01-1416:20devth#2019-01-1416:31rapskalianDoes there exist a flavor of s/merge for disjunctions, i.e. s/or? I have a base set of disjunctions (A or B) that Iā€™d like to be able to extend in certain contexts (base or C or D). #2019-01-1416:45borkdude@devth Isnā€™t this the flaw of current spec that Rich spoke about in his recent Clojure Conj talk?#2019-01-1416:45borkdude@devth Isnā€™t this the flaw of current spec that Rich spoke about in his recent Clojure Conj talk?#2019-01-1416:49devthi'm not sure i've seen his latest conj talk. i'll have to look it up#2019-01-1416:58Alex Miller (Clojure team)https://www.youtube.com/watch?v=YR5WdGrpoug#2019-01-1416:59devthoh, i have seen part of that one.#2019-01-1417:54devthstill curious about the place vs value oriented nature of spec šŸ¤” any thoughts?#2019-01-1417:54borkdudeI recommend watching the talk.#2019-01-1417:54devthk, i'm 30 min in. still watching šŸ™‚#2019-01-1417:55borkdudelet it sink in on the hammock šŸ˜‰#2019-01-1418:01devthtaking notes#2019-01-1500:54devthBig ideas: 1. separating out schema (shape) and selection (optionality). cool stuff! - it's still not "just data". why the departure? everything in Clojure is just data, and for good reason. the entire core lib is built around manipulating data. it's the best part of clj. - it's still place oriented, at least from what i gather so far šŸ˜ž 2. Better programmatic manipulation of specs. great! but again: why not make it data? then programmatic manipulation comes with. Imagine if the schema for:
{:weather {:weatherbitio {:key "xoxb" :default {:zip "98104"}}}
   :command {:prefix "!"}}
Was simply:
{:weather {:weatherbitio {:key string? :default {:zip string?}}}
   :command {:prefix string?}}
There's elegance in mirroring the data structure you're specifying, kinda like how Datomic Pull queries mirror the data you get get back. šŸ¤”
#2019-01-1416:46borkdudeor do you mean ā€œwhy do I have to gives names to each sub-spec, canā€™t you inline them?ā€ ?#2019-01-1416:49borkdudeit may be related to each other#2019-01-1416:49devthyeah, both i think. i want values#2019-01-1416:50borkdudefuture spec will allow you (I think!) to spec the whole schema and then define selections on them, so you donā€™t have to name each selection.#2019-01-1416:50borkdudeIā€™m not entirely sure if this maps to your issue with spec, but just my 2cts#2019-01-1416:50devthpretty sure that'd help!#2019-01-1416:51devthany word on timeline for the next version of spec?#2019-01-1416:54rapskalianWhenever Iā€™ve seen this asked, the short answer is ā€œwhen itā€™s readyā€ šŸ™‚#2019-01-1416:55devthhaha. fair#2019-01-1416:56Alex Miller (Clojure team)Iā€™m working on it every day#2019-01-1416:58Alex Miller (Clojure team)hard to say when the next point will be where itā€™s useful to look at but it wonā€™t be ā€œaā€ next version - itā€™s going to be a series of releases#2019-01-1417:00devthawesome#2019-01-1417:00devthso to be clear: currently there isn't necessarily a better way to achieve what i'm doing? i have to dynamically s/def a bunch of stuff?#2019-01-1417:01borkdude@devth there is a lib called spec-tools which may help you accomplish this, but I expect breaking changes when new spec comes out#2019-01-1417:01borkdudeI havenā€™t used it in anger myself#2019-01-1417:02devthlooks interesting, thanks.#2019-01-1417:02borkdude@devth https://github.com/metosin/spec-tools#transforming-nested-specs applies to your situation I think#2019-01-1417:03borkdudeyou may lose the ability to generate data this way, Iā€™m not sure#2019-01-1417:04borkdude@devth o wait, maybe it was this: https://github.com/metosin/spec-tools#data-specs (the README is so long ;))#2019-01-1417:04devth> Just data, no macros šŸ’Æ#2019-01-1417:04devthlong READMEs are nice šŸ™‚#2019-01-1417:06Alex Miller (Clojure team)most of things spec-tools is built on are going away, but there will be one or maybe even two alternative paths to this goal#2019-01-1417:06borkdudeIā€™m not sure if I agree. Do one thing well also has benefits šŸ™‚#2019-01-1417:07devthnot familiar enough with spec-tools to have an opinion on whether it should have been multiple libs. but long searchable READMEs with lots of examples are always nice.#2019-01-1417:07devthi think i'll play with spec-tools for now. if things change i can always port to the new way#2019-01-1417:08borkdudeglad I could help šŸ˜›#2019-01-1417:13rapskalian
(create-ns 'my.really.long.ns)
(alias 'mrln 'my.really.long.ns)

::mrln/attribute
Is this trick for shorter namespaced keywords likely to get me scolded by experienced clojure spec devs? Is it better to formally create the ns file even if itā€™s largely empty?
#2019-01-1417:14borkdude@cjsauer there is something like this in spec itself, so I think itā€™s ok#2019-01-1417:15borkdudehttps://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/test/alpha.clj#L19#2019-01-1417:15bronsasee also https://dev.clojure.org/jira/browse/CLJ-2123#2019-01-1417:15borkdude@cjsauer if youā€™re writing portable code, be aware that this may not work on CLJS#2019-01-1417:17borkdude@cjsauer I tried moving this to a proper ns, so CLJS could use the alias as a namespaced keyword as well, but that was rejected: https://dev.clojure.org/jira/browse/CLJ-2421#2019-01-1417:19rapskalianAh interesting, thank you. in-ns is a bit better. Which part of this does cljs not like? I am hoping to use these specs on both server and client.#2019-01-1417:19borkdude@cjsauer in-ns simply doesnā€™t work in CLJS.#2019-01-1417:20borkdudeit does, but only in the REPL.#2019-01-1417:20rapskalianBummerā€¦does the same go for create-ns?#2019-01-1417:20borkdudeyes#2019-01-1417:20borkdude(I think so, gonna check now)#2019-01-1417:23borkdude
(ns dude)

(in-ns 'foo)

(defn foo [x])

(in-ns 'dude)

(defn dude [x])

$ clj -Sdeps '{:deps {org.clojure/clojurescript {:mvn/version "1.10.439"} org.clojure/test.check {:mvn/version "RELEASE"}}}' -m cljs.main -t node -c dude
WARNING: dude is a single segment namespace at line 1 /private/tmp/repro/src/dude.cljs
WARNING: Use of undeclared Var dude/in-ns at line 3 /private/tmp/repro/src/dude.cljs
WARNING: Use of undeclared Var dude/in-ns at line 7 /private/tmp/repro/src/dude.cljs
#2019-01-1417:25rapskalianShootā€¦so then portable code will have to use the qualified keyword everywhere at the moment?#2019-01-1417:26borkdudethe fully qualified one yes#2019-01-1417:26borkdudedid that in CLJS: https://github.com/clojure/clojurescript/commit/731be5e0916ad8d619c302bfc9b985c4d10daa8d#2019-01-1417:28rapskalianI may actually fall back to formally creating the namespace file for now, even if its mostly empty. Keyword fatigue is realā€¦#2019-01-1417:28borkdudeitā€™s not a huge pain. you could make a function that creates the keyword if you want to save characters#2019-01-1417:29borkdude@cjsauer we also have that in the app Iā€™m working one, several almost empty ns-es for thisā€¦#2019-01-1417:30rapskalianOnce CLJ-2123 is patched, I imagine it should be quick work to refactor#2019-01-1417:30borkdudecool. hopefully CLJS gets this too#2019-01-1418:06rapskalianWhat is the philosophy behind specā€™ing attributes that are intended to be used as references to other entities? For example, I might have ::company/employees which models a one-to-many relationship. How would I spec this attribute? My intuition is to use something like (s/coll-of ::employee), but the issue here is that I now have to define some minimum key-set that is ::employee, which is a spec that I think is largely contextual. Further, there may be contexts where (s/coll-of (s/tuple #{::employee/uuid} ::employee/uuid)) are valid ::company/employees, e.g. datomic idents. Ultimately what I think Iā€™m wondering is: can reference attributes only be specā€™d within a given context, or at specific boundaries (e.g. fdef)?#2019-01-1419:17rapskalianI think my confusion stems from the fact that a :db.valueType/ref attribute can take on lots of different forms throughout an application, and so itā€™s difficult to spec. For example, it could be a raw entity id, a {:db/id 000} map, a [:db/id 000] ident, some kind of custom [:employee/uuid #uuid "000"] ident, or it could be a fully formed employee mapā€¦all these can values live under the same ref attribute at one time or another.#2019-01-1420:02favilaI think only spec-ing the pull form (ie what you would get from a pull)#2019-01-1420:02favilaThose other variants are all part of the Tx map DSL#2019-01-1420:03favilaThis just shifts the problem around though#2019-01-1420:03favilaI think the TX spec would only spec the coll not the keys individually #2019-01-1420:06rapskalian>I think only spec-ing the pull form (ie what you would get from a pull) Iā€™ve been playing with a similar strategy. This way you can still make good use of generators without the concept of a database getting involved.#2019-01-1420:12rapskalian>I think the TX spec would only spec the coll not the keys individually @favila by this do you mean something like (s/coll-of (s/keys))?#2019-01-1420:16favilaColl-of whatever a tx is#2019-01-1420:20favilaColl of either Vecs with entity ref (= 2-tuple, long, or keyword) first item (Tx fns and raw add/retract) or tx maps; which are maps with entity ref keys and Tx map or col-of-txmap or entity ref or non-ref value or coll of non-ref value)#2019-01-1420:21favilaSo pretty open...#2019-01-1420:28rapskalianIn my tx function fdefs Iā€™ve been specifying the :ret much more tightly. Iā€™m less interested in specā€™ing the :ret of any tx function, and more interested in specā€™ing this specific tx function.#2019-01-1420:35rapskalianIā€™ve been staring at the code more, and I think whatā€™s troubling is that when I want to specify the form of some ref attribute in general, it canā€™t really be doneā€¦the question of when is inescapable so to speak. Because if I specify the pull form for all of my ref attributes, now Iā€™ve bound the keyword :company/employees to one specific context: the fully realized one. I canā€™t use the name :company/employees in other contexts in which it might make sense, because spec has already registered the ā€œone true formā€. Does this make sense, or am I complicating the issue? šŸ¤”#2019-01-1420:37rapskalianFor example, a tx function might return a collection that includes a map with the :company/employees keyword in it, but it may not conform to the pull form. It may be a collection of idents instead, meaning it references pre-existing entities.#2019-01-1420:38rapskalianBecause Iā€™ve already specā€™d it to conform to the pull form, I canā€™t use that keyword again in a new contextā€¦#2019-01-1420:42lilactownAssuming it's a map using s/keys, you could make everything optional#2019-01-1420:43rapskalianI had the same idea, and it works okay. I think the downside is that it forces you to resort to custom generators pretty much everywhereā€¦#2019-01-1420:44lilactownBut this is the exact problem Rich stated in his last conj keynote Maybe Not#2019-01-1420:44lilactownThere's not a great solution for it in spec#2019-01-1420:46rapskalian>But this is the exact problem Rich stated in his last conj keynote Maybe Not I think so as well. Pretty much anywhere that a map has to be specified, this problem will arrive. The concept of ā€œentityā€ is still difficult to grasp in my opinion. ā€œEntityā€ doesnā€™t seem to become concrete until some data arrives at a boundary/function.#2019-01-1421:16favila@cjsauer you are exactly right about the problem#2019-01-1421:17favilaI don't think this is the same problem Rich has acknowledged#2019-01-1421:17favilaspec has a strong opinion that the keyword of a map is (essentially) describing the type of its contents#2019-01-1421:18favilabut there are cases where that is not true and the keyword is really a structure rather than a type tag#2019-01-1421:18favilaone is DSLs: e.g. tx map dsl, or even the pull expression dsl#2019-01-1421:19favilain those cases the key of the map is a structural tag, not a type tag#2019-01-1421:19favilaand foreign system's data routinely make their map entries contextual to the type of thing they are in#2019-01-1421:20favilaboth these use cases I have found are bad fits for spec; the only workaround is predicates#2019-01-1421:20favilaor more keys and lots of key renaming#2019-01-1421:21favila"structural tag" is wrong#2019-01-1421:21favilawhat I mean is they are like structurally-expressed arguments to some function call implied by the dsl#2019-01-1421:22favila{:my/attr [:db/id :db/ident]} is not expressing a :my/attr value, it's telling you to do something to a :my/attr value in some different context#2019-01-1421:23favilathey can't be interpreted apart#2019-01-1421:23favilaso it's the type of the entry itself that matters rather than the type of the value#2019-01-1421:24favilaso, maybe multi-spec on the key as the dispatch value is getting towards the right answer?#2019-01-1421:52rapskalian>I donā€™t think this is the same problem Rich has acknowledged @favila Iā€™ve just rewatched Maybe Not, and I think youā€™re right, itā€™s not the same issue. It is exactly the type/structural conflation that youā€™ve identified.#2019-01-1421:54favilaif anything, rich is moving even further away from keywords having any contextual meaning#2019-01-1421:54favilas/keys at least let you attach predicates to the whole#2019-01-1421:57rapskalianHe actually also mentions that :ret in function specs is starting to smell, and itā€™s in those :ret specs that I encountered this issue. So that may be tellingā€¦#2019-01-1421:58rapskalianIf I ease up on trying to ā€œnail everything downā€ with specs, as Rich puts it, it may alleviate some of these limitations#2019-01-1421:59borkdudeWhat did he say about ret specs again?#2019-01-1422:01rapskalianHe touches on it at around this point in the video: https://youtu.be/YR5WdGrpoug?t=3174#2019-01-1422:09borkdudewhat heā€™s saying there is that ret specs donā€™t tell you much, you almost always want fn specs. so he wants to refine fn specs (make them easier to use?)#2019-01-1422:30rapskalianTrueā€¦I think the root of it is definitely this ā€œcontextual keysā€ idea (keywords have different specs in different contexts). What might be cool is the ability to define a ā€œdisjunctive schemaā€:
(s/def ::widget (s/or* :A string? :B pos-int?))
and then use a function similar to Richā€™s proposed s/select function in order to mask the contextually relevant predicates, e.g. (s/select* ::widget [:A]) would mean ā€œin this context, only condition :A can match (anything other than a string would result in error)ā€œ.
#2019-01-1422:33rapskalianAlmost reminds me of tools.depsā€™ alias flag clojure -A:a:b:c#2019-01-1501:04favila#2019-01-1501:05favilaThis was something I wrote to tackle the specific case of having a more general spec (as a key in a map) that context (what kind of map it is in) would narrow#2019-01-1501:06favilae.g. :a could be 1 2 or 3, but when it's in a map with :map-type "foo" :a must be 1 or 2#2019-01-1501:08faviladealing with that case over and over with predicates was a hassle, so this macro (keys+) allowed adding an additional keyword->spec mapping for validity, conformance, and generator purposes#2019-01-1501:08favilabut the original "natural", "contextless" spec still had to validate#2019-01-1501:09favilaso it's not aimed at the DSL use case, this was to deal with data from a foreign system that had a kind of tagging/inheritance relationship. their fields and entites had a base definition that was overly broad, and could be narrowed without changing their structure#2019-01-1501:10favilathink xsd facets where the element names don't change#2019-01-1501:11favilathe purpose was to allow generic processors to understand everything, but particular producers/consumers to add additional constraints about what they would produce/consume#2019-01-1501:11favilae.g. a field is cardinality-many, but some processor guarantees they only ever put one value in that field#2019-01-1512:25misha@cjsauer you can s/or or s/alt lookup-refs spec with s/keys. Or avoid speccing maps with lookup-ref values (`{:my/attr [:db/id :db/ident]}`). Or, if you can isolate lookup-ref key-values from actual data key-values, spec lookup-refs with key-agnostic spec, something like (s/map-of qualified-keyword? lookup-ref-spec). I don't think having s/select would solve spec overload (data or lookup) in this case, because: https://clojurians.slack.com/archives/C1B1BB2Q3/p1547474311308100#2019-01-1512:27mishaAnother option is to have s/or + (s/map-of keyword? not-a-look-up-value?) where you expect already looked-up values.#2019-01-1516:43ShaneLesterSo I have [org.clojure/test.check "0.9.0"] in my dependencies, but when Iā€™m trying to use generators with spec Iā€™m still getting a clojure.test.check.generators never required errorā€¦. Anyone have an idea of what I should try? Is there a different one for cljs potentially?#2019-01-1516:52borkdudehave you tried requiring it manually?#2019-01-1516:52borkdude@shanelester55 are you on CLJS or CLJ and which version?#2019-01-1516:53ShaneLesterIā€™m on CLJS. Using shadow-cljs soā€¦ not positive which version. Whichever version it is currently using, which I imagine is the latest.#2019-01-1516:54ShaneLesterIs requiring it manually just specifiying it at the top of the file where it is used?#2019-01-1516:54borkdudeyou should be able to know the CLJS version in order to deal with known issues. can you try *clojurescript-version* in the REPL?#2019-01-1516:57borkdudethere were some changes around this recently, but they are only on master, not released yet, thatā€™s why Iā€™d like to know#2019-01-1516:57ShaneLesterSure thing. trying that nowā€¦ for some reason giving me trouble#2019-01-1516:58borkdudeif you donā€™t have a REPL, just print it the console from your app#2019-01-1516:59borkdudeor type cljs.core._STAR_clojurescript_version_STAR_ in the browser console#2019-01-1517:00ShaneLesterAh that worked easily enough. "1.10.439"#2019-01-1517:01borkdudeok, how do you use the generators. using clojure.spec.test.alpha?#2019-01-1517:02ShaneLesterCurrently using these#2019-01-1517:02borkdudeok, I imagine you want to call gen/sample to just see some generated data right?#2019-01-1517:03ShaneLestercorrect#2019-01-1517:03borkdudeyouā€™ll need a require to [clojure.test.check.generators] in your ns as well#2019-01-1517:04ShaneLesterAhhh that did it.#2019-01-1517:04ShaneLesterThank you very much#2019-01-1517:05borkdudeactually just [clojure.test.check] should work too#2019-01-1517:06borkdudesince it requires all the other needed namespaces#2019-01-1517:06ShaneLesterMakes sense. That indeed works as well.#2019-01-1517:07ShaneLesterThanks again šŸ™‚ Works well now#2019-01-1517:08borkdudeEnjoy šŸ™‚#2019-01-1517:29devthreposting my notes on spec-related content in the 'Maybe Not' talk (https://www.youtube.com/watch?v=YR5WdGrpoug) since the thread is buried. 1. separating out schema (shape) and selection (optionality). cool stuff! - it's still not "just data". why the departure? everything in Clojure is just data, and for good reason. the entire core lib is built around manipulating data. it's the best part of clj. - it's still place oriented, at least from what i gather so far 2. Better programmatic manipulation of specs. great! but again: why not make it data? then programmatic manipulation comes with. Imagine if the schema for:
{:weather {:weatherbitio {:key "xoxb" :default {:zip "98104"}}}
   :command {:prefix "!"}}
Was simply:
{:weather {:weatherbitio {:key string? :default {:zip string?}}}
   :command {:prefix string?}}
There's elegance in mirroring the data structure you're specifying, kinda like how Datomic Pull queries mirror the data you get get back.
#2019-01-1520:47schmeebecause this: https://clojure.org/about/spec#_decomplect_mapskeysvalues, https://clojure.org/about/spec#_sets_maps_are_about_membership_thats_it#2019-01-1520:47schmeethe answers to most questions about ā€œwhy doesnā€™t spec do Xā€ are on that page#2019-01-1520:49schmeeyou might be interested in this though: https://github.com/HealthSamurai/matcho#2019-01-1520:53jstaabI came here to ask a very similar question. I do agree with the two points you linked above, but it does seem very strange to use static place orientation to achieve it. I'd like to programmatically generate specs based on my own domain modeling dsl, rather than repeat my entity shapes multiple times, use spec as my domain modeling language, or write complex macros to do both at once. I've heard the objection of "why is it so hard to generate composed specs programmatically", and I still haven't heard a good answer.#2019-01-1520:55schmeetheyā€™re working on a new version of spec which is supposedly more data-oriented, there are some details about the development here, but nothing is released publicly yet: http://insideclojure.org/2019/01/11/journal/#2019-01-1520:55jstaabI've heard of that, I have high hopes šŸ™‚#2019-01-1606:19ikitommi@U066S8QGH spec-tools has a data-layer on top of specs like you described:
(require '[spec-tools.data-spec :as ds])
(require '[clojure.spec.alpha :as s])

(def weather-spec
  (ds/spec
    {:spec
     {:weather {:weatherbitio {:key string? :default {:zip string?}}}
      :command {:prefix string?}}
     :name ::weather}))

(s/valid? 
  weather-spec
  {:weather {:weatherbitio {:key "xoxb" :default {:zip "98104"}}}
   :command {:prefix "!"}})
; true
just functions & data.
#2019-01-1606:22ikitommithe forms are mapped with a multimethod, so string? gets a 'clojure.core/string? form. doesnā€™t help with anonymous functions.#2019-01-1615:01devthlooks good! i'm planning to use it#2019-01-1520:58dustingetzbut string? is code, not data#2019-01-1520:58dustingetzthe idea of predicates is core to spec, i dont see how spec could ever be jsut data#2019-01-1520:59devthdata for edges, predicates for leaf nodes#2019-01-1520:59Alex Miller (Clojure team)the symbol string? is data#2019-01-1520:59Alex Miller (Clojure team)the form (s/coll-of string?) is also data#2019-01-1520:59dustingetzthatā€™s thin#2019-01-1520:59Alex Miller (Clojure team)itā€™s not thin, this is the whole basis of lisp#2019-01-1521:00dustingetzhomoiconicity is about macros#2019-01-1521:01jstaabIf you call describe it does return a list of symbols at runtime, so spec isn't just confined to macros. Admittedly it could be hard to do anything with more complex predicates#2019-01-1521:02Alex Miller (Clojure team)thatā€™s just ā€¦ not true#2019-01-1521:03dustingetzfor string? to have meaning weā€™d have to syntax quote it for starters, and now weā€™re in the clojure reader, not the edn reader#2019-01-1521:03Alex Miller (Clojure team)and spec forms will be even more central in spec 2#2019-01-1521:04Alex Miller (Clojure team)string? has meaning in the context of a namespace#2019-01-1521:04dustingetzso now we hve (ns) forms and are very solidly in the clojure reader, not edn reader#2019-01-1521:05dustingetzthatā€™s how i udnerstand it anyway, always looking to deepen šŸ™‚#2019-01-1521:05Alex Miller (Clojure team)if you have fully qualified symbols, then the edn reader is sufficient#2019-01-1521:05Alex Miller (Clojure team)for example, s/form always give you that#2019-01-1521:06Alex Miller (Clojure team)we are also moving towards a world where spec forms are a data form, but may not be the only data form#2019-01-1521:06Alex Miller (Clojure team)so some ā€œmapā€ data form could also exist#2019-01-1521:07Alex Miller (Clojure team)but thatā€™s not more or less data than the list form (it may be easier to perform some kinds of transformations on)#2019-01-1521:08dustingetzyoā€™re saying (s/coll-of ...) is sugar for something deeper?#2019-01-1521:08Alex Miller (Clojure team)no#2019-01-1521:09Alex Miller (Clojure team)Iā€™m saying that (s/coll-of string?) is data. Some map syntax {:op coll-of :pred string?} (not a real syntax) is also data. both are sufficient to describe a spec.#2019-01-1521:09favilaI don't think this matters. the power and problem is the predicates. predicates can only be data if you know what all of them mean (independently of an implementation--a well-known list of predicates)#2019-01-1521:10Alex Miller (Clojure team)yes, that is essential in the design of spec which from day 1 was predicate based#2019-01-1521:11favilaand that is what people mean when they say "specs are not data"#2019-01-1521:11dustingetzi accept that a qualified symbol is equivalent to a qualified keyword ā€“ but not a closure or fn reference, only symbols#2019-01-1521:11favila> the idea of predicates is core to spec, i dont see how spec could ever be jsut data#2019-01-1521:11favilathat's I think what you mean @dustingetz?#2019-01-1521:11favilacertainly how I understood it#2019-01-1521:12favilain my own thinking about the "dataness" of spec#2019-01-1521:12dustingetzAs soon as someone puts #(and x y) as a spec pred, itā€™s all over, no logner data#2019-01-1521:12Alex Miller (Clojure team)well, thatā€™s false and we support that now#2019-01-1521:12dustingetzhow can you serialize that#2019-01-1521:12Alex Miller (Clojure team)(fn [&] ...)#2019-01-1521:12dustingetzits a reference#2019-01-1521:13dustingetzit could be jvm interop there
#2019-01-1521:13Alex Miller (Clojure team)or you mean x and y as closed over values?#2019-01-1521:13Alex Miller (Clojure team)if so, then agreed#2019-01-1521:13dustingetzright, it could be anything#2019-01-1521:13Alex Miller (Clojure team)although we have some thoughts on how to allow you to do that as well#2019-01-1521:13dustingetznow that has my interest piqued ā€¦#2019-01-1521:13Alex Miller (Clojure team)if x and y are vars youā€™re invoking#2019-01-1521:14Alex Miller (Clojure team)if itā€™s values, then youā€™re in the realm of custom spec creation (which also is now more codified and easier to do)#2019-01-1521:14Alex Miller (Clojure team)well, maybe not that much easier, but I think weā€™re ready to commit to what that means#2019-01-1521:16Alex Miller (Clojure team)I would still say that pred references are data if itā€™s possible to re-establish their meaning remotely (in the context of namespaces) and var serialization / hydration is something that weā€™ve been slowly working on for the last year or so. some of itā€™s in clojure already.#2019-01-1521:16dustingetzArenā€™t you going to run up against the halting problem
(= #(even? (inc (inc %)))
   #(even? (+ 2 %)))
#2019-01-1521:17Alex Miller (Clojure team)we may ultimately end up going beyond var quote to define something new, but weā€™re still just thinking about those parts#2019-01-1521:17Alex Miller (Clojure team)why do you need to do that?#2019-01-1521:17Alex Miller (Clojure team)weā€™re never comparing specs for equality#2019-01-1521:17dustingetzbecause you said itā€™s data, so i want to do data things, otherwise doesnā€™t that defeat the point#2019-01-1521:18Alex Miller (Clojure team)this is too theoretical. what problem are you trying to solve?#2019-01-1521:18dustingetz
(= `string? `string?)
is well defined since they are data and qualified
#2019-01-1521:18Alex Miller (Clojure team)why are you even doing that?#2019-01-1521:19dustingetzWell for example if specs are data then i want to store them efficiently in a database#2019-01-1521:19Alex Miller (Clojure team)youā€™re inventing problems that imho donā€™t exist#2019-01-1521:20dustingetzit is essential to hyperfiddle that datomic schema is actually data and stored in a database#2019-01-1521:20dustingetzwe match on :db.type/string, for example, to decide what widget to draw#2019-01-1521:22Alex Miller (Clojure team)well youā€™ll be glad weā€™ve scoped it to just data then - symbols, sets, and forms combining them#2019-01-1521:22dustingetzlol#2019-01-1521:22favilainline fns are no longer allowed?#2019-01-1521:22Alex Miller (Clojure team)inline fns are forms#2019-01-1521:22dustingetzwell, looking forward to seeing whatever you come up with#2019-01-1521:22favila....#2019-01-1521:23Alex Miller (Clojure team)work proceeding in https://github.com/clojure/spec-alpha2 although I have not pushed up in quite a while#2019-01-1521:23Alex Miller (Clojure team)debating how bad it would be to push wip stuff there, but maybe Iā€™ll just put it on a branch#2019-01-1521:24favilaspec is data if you have eval and a full clojure runtime and possibly other required nses got it#2019-01-1521:24dustingetzLike for example, imagine a big honking match, like this but huge, and it tested the specs too. If specs are data that should be possible and could be very interesting.#2019-01-1521:25Alex Miller (Clojure team)well specs are an open protocol, so no match is big enough to cover custom extensions#2019-01-1521:26dustingetzcould be multimethod, you get the idea#2019-01-1521:26Alex Miller (Clojure team)but sure#2019-01-1521:27Alex Miller (Clojure team)specs have a form, keyed by the op in first position (and thatā€™s how resolution occurs in the spec 2)#2019-01-1521:27mishawhat is the difference between form and data?#2019-01-1521:27Alex Miller (Clojure team)none#2019-01-1521:28faviladata can be understood without evaluation#2019-01-1521:28favilathat can be true for a well-defined subset of spec#2019-01-1521:28Alex Miller (Clojure team)as weā€™re talking forms above, I mean combinations of lists, symbols, keywords, and sets#2019-01-1521:29misha> data can be understood without evaluation well, it depends, because your brain is the evaluator which "understands", and knowing format is essentially :require in brain (if POV is a person)#2019-01-1521:29Alex Miller (Clojure team)it depends what you mean by ā€œunderstoodā€#2019-01-1521:32mishaarguably, string? can also be understood w/o evaluation, even w/o namespace. so yeah, it depends on definition of "understood"#2019-01-1521:33favilayes, most derivative uses of spec rely on some implementation-independent meaning for the predicate symbols#2019-01-1521:33favilabut you need to limit that set, or else have some way to mark useful traits about a predicate; otherwise you can't "understand" it without getting a clojure runtime to run it#2019-01-1521:33mishamore specific you (want to) get ā€“ more it'll depend on actual implementation behind the symbols#2019-01-1521:34favilathat's what I mean by a well defined subset#2019-01-1521:38mishait starts to sound like static types argument. There is value in being able to use anything clojure allows to construct a predicate. But if you want it to be understandable over the wire ā€“ use subset of what is allowed. Instead of having spec forbid e.g. java interop, because it would not be understandable in 0.0000001% cases#2019-01-1521:38Alex Miller (Clojure team)if you want use specs with the spec api calls (valid?, conform, explain, etc), then yes, you need implementations for all predicates mentioned in the specs#2019-01-1521:38Alex Miller (Clojure team)this has been explicit in the design of spec from the beginning#2019-01-1521:39Alex Miller (Clojure team)none of this is new or really any different#2019-01-1521:39favilano one is arguing it's not been explicit, or even that it is bad or wrong#2019-01-1521:39favilait's just annoying to hear that called "data"#2019-01-1521:39Alex Miller (Clojure team)thatā€™s baffling to me#2019-01-1521:40Alex Miller (Clojure team)as this is no different than the whole ā€œcode as dataā€ / macros thing that is part of Clojure, which presumably you are also familiar with#2019-01-1521:40mishawhat is "data" to you? @favila#2019-01-1521:40dustingetzdata makes possible a whole new universe of systems in 20 years that we barely even have the words to talk about today (thanks to Clojure) ā€“ what seems pedantic today is pretty important in hindsight, i think#2019-01-1521:41Alex Miller (Clojure team)just because parts of the data are linked to semantics, does not mean itā€™s not data#2019-01-1521:42dustingetz
[:find ?e :where [?e :dustingetz.reg/email]]
(d/q '[:find ?e :where [?e :dustingetz.reg/email]])
One of these is data, one of these is code (and code is data), but there is a huge difference in terms of what it is coupled to and what it means
#2019-01-1521:42dustingetzOne of these can be used to generate UI or compile to SQL, the other canā€™t#2019-01-1521:43mishaboth is edn data, no? vectors, lists, keywords, symbols#2019-01-1521:43Alex Miller (Clojure team)well this has been fun, but Iā€™m going to bow out here and go work on the code#2019-01-1521:43dustingetzThanks Alex#2019-01-1521:44lilactownI think that there is a valid desire to have specs be represented in a form that doesn't require a Clojure interpreter in order to use#2019-01-1521:45lilactownwe'd probably prefer to have a simpler DSL, using a subset of EDN, that would enable easier parsing and consuming#2019-01-1521:45mishafor example, how do you know what ?e is in 1st line? you need to provide "this is variable placeholder to be bound because datomic" context somehow somewhere. if you don't - this is exactly the same as 2nd line, but in a list, not vector#2019-01-1521:45dustingetzOnce it is coupled to Clojure, it is an order of magnitude harder to build a data driven system on top of it#2019-01-1521:46dustingetzXML is data too#2019-01-1521:46lilactownthe trick is to tease out, what is the minimum set of semantics required to express a spec?#2019-01-1521:47lilactownwe can also turn the knob on that by limiting the ease and power of specs#2019-01-1521:47lilactowne.g. right now, specs use predicates that you can easily inline. what if we disallowed that (either by convention or some other tricky mechanism)?#2019-01-1521:47lilactownthat would make it easier to serialize, but potentially less powerful and harder to write#2019-01-1521:48lilactown"every predicate must have a qualified name"#2019-01-1521:48dustingetzOr just say ā€œhey this thing is programmable, have at itā€, thatā€™s fine too, but the halfway world is pretty weird and i think leads to fragility#2019-01-1521:49devthwonder if free monads and the interpreter pattern is useful to consider in this context. spec as a pure data AST.#2019-01-1521:49misha
(defn make-sql [data] data)
=> #'user/make-sql

(-> "[:find ?e :where [?e :dustingetz.reg/email]]"  (edn/read-string)  (make-sql))
=> [:find ?e :where [?e :dustingetz.reg/email]]

(-> "(d/q '[:find ?e :where [?e :dustingetz.reg/email]])"  (edn/read-string)  (nth 2)   (make-sql))
=> [:find ?e :where [?e :dustingetz.reg/email]]
#2019-01-1521:49lilactownor we could have a convention that says, any predicate must not close over any vars#2019-01-1521:49misha> One of these can be used to generate UI or compile to SQL, the other canā€™t see above#2019-01-1521:51dustingetz(d/q (->> [:find '?e] (concat [:where ['?e :dustingetz.reg/email]]))) i can play that game all day#2019-01-1521:51mishait has extra step, yes, but it does not make it not data#2019-01-1521:51mishawell, how do you know the 1st one can be converted to sql?#2019-01-1521:52mishaby looking at it first? you interpret while you look. your brain is repl#2019-01-1521:52mishabrain of someone w/o knowing what edn is will not be able to tell that 1st can be converted and second cant#2019-01-1521:52mishait's a repl w/o edn/reader required yet#2019-01-1521:53favilaI don't think data = edn#2019-01-1521:53mishayes, it is less convenient, and requires extra step. if this is the argument - I agree, there is no limit to convenience opieop#2019-01-1521:53favilaif data is just "anything you can quote" then what is code?#2019-01-1521:54mishathat's why I asked https://clojurians.slack.com/archives/C1B1BB2Q3/p1547588438496700#2019-01-1521:54favilawhen people say "this is data" they are saying it has a constellation of useful properties that are strictly less powerful than a programming language#2019-01-1521:55mishawhat does it mean? in this spec conversation context, and ... in general?#2019-01-1521:55favilae.g., being able to understand and interpret it multiple ways; draw inferences from it that are not stated explicitly; compare it to other things for equivalence; build different interpreters on top of it#2019-01-1521:56mishayou somehow need to parse/read/interpret data to "draw inferences"#2019-01-1521:56lilactownI'm having flashbacks to this conversation between Rich Hickey and Alan Kay a few years ago: https://news.ycombinator.com/item?id=11945722#2019-01-1521:56dustingetzLOL#2019-01-1521:56favilayes @misha, you do. You need to assign meaning to things without implementing them#2019-01-1521:56favilathis is possible for spec until you hit a predicate#2019-01-1521:57mishaotherwise how do you know what is in the string? how do you know that this is map: {:a 1}, etc.#2019-01-1521:57favilaif you have a well-known list of predicate symbols, then you can hard-code that meaning in#2019-01-1521:57mishayou have to have some context, it might be in your brain, but you need to communicate it to the system "which might not be using clojure, etc."#2019-01-1521:57favilathat's how e.g. https://github.com/arohner/spectrum can work#2019-01-1521:57favilabut once you have a custom predicate, it's meaning cannot be shared without sharing it as code#2019-01-1521:58lilactownI don't think it even needs to be well-known#2019-01-1521:58favilathere's no data language that can express a new predicate#2019-01-1521:58favilaspec has parts of its language that can express new specs#2019-01-1521:58dustingetzData is also secure, imagine a REST service that returned Clojure code#2019-01-1521:58lilactownyou could receive a spec, and then read all of the symbols from that spec to see what needs to be implemented#2019-01-1521:58favilae.g. s/or s/and etc#2019-01-1521:58favilabut below predicates, you cannot know anything unless you have independent knowledge of its meaning#2019-01-1521:59mishaI think, that "assigning meaning w/o implementation" is a very blurry line#2019-01-1521:59favilaeven executing it you can only know what it means for a specific value you give to it#2019-01-1521:59mishahardcode that meaning where?#2019-01-1521:59favilayou cannot make any universal inferences about the behavior of a predicate without knowing its meaning#2019-01-1521:59mishabecause that is just "global state", and you need to make sure system you are sending it to - is in the same context#2019-01-1522:00dustingetzDatomicā€™s :db.type/string is not ambiguous or blurry#2019-01-1522:00favila@misha having to send it to the same runtime environment to use it sounds an awful lot like a property of code rather than data#2019-01-1522:01mishashow it to java dev and it would be pretty ambiguous to him opieop#2019-01-1522:01dustingetzThe success of HTML over desktop gui toolkits is also because it is data#2019-01-1522:01dustingetzYou can ship HTML over the wire to arbitrary platform, and somehow they manage to interpret it usefully#2019-01-1522:02mishawell, then yours "well defined symbols we could just hardcode" just means "we have implementation :ruquired here in clojure and there in python, so dont worry about it"#2019-01-1522:02mishawhich smells like transit#2019-01-1522:02mishaotherwise, define "well defined" :)#2019-01-1522:02dustingetzYou can render arbitrary HTML fearlessly, you arenā€™t worried about getting hacked#2019-01-1522:02favila"well defined" just means we know the list ahead of time#2019-01-1522:03favilaI know what "clojure.core/string?" means#2019-01-1522:03favilaI don't know what "foo.core/blah?" means#2019-01-1522:03favilaIf I had the code I could possibly execute foo.core/blah?#2019-01-1522:03mishahtml just has very small set of tokens#2019-01-1522:03dustingetzYou do however know what github/markdown? is, you know who the maintainer is, you know where the docs are and where you might find useful code to operate with it#2019-01-1522:04favilamaybe another angle: spec can express things that are not in spec#2019-01-1522:04favilai.e. the predicates#2019-01-1522:05mishawell, html makes no sense to my mom, neither does english, so can I say it is not data then? ĀÆ\(惄)/ĀÆ#2019-01-1522:05dustingetzYet somehow I manage to pay my bills#2019-01-1522:05mishaher brain is a system, and it does not support html. mine does.#2019-01-1522:06misha@favila you know the list of clojure code ahead of time too. it is just way too large.#2019-01-1522:07mishathe same way we know the set of real numbers#2019-01-1522:07favilawill foo.core/blah? return true for a real number?#2019-01-1522:08mishaone difference would be - you cant eyeball it and say "yep, it's fine, we are not getting hacked"#2019-01-1522:08mishawill the next system interpret 1 as true or int?#2019-01-1522:08lilactownsometimes you have to reject invalid data if you don't know how to parse it, too#2019-01-1522:09favilaor you convert it to a less meaningful form, e.g. clojure tagged-value#2019-01-1522:09favilaor, a fully qualified predicate symbol#2019-01-1522:09favilabut then the power the spec gained by being able to use any predicate it wants is lost by any other system that wants to make sense of the spec#2019-01-1522:10favilaby making s/valid? easier something else has been made harder#2019-01-1522:10misha@dustingetz if stuff on github - is data, code is data in exactly same way, so we are on the square 1 again#2019-01-1522:11lilactownthis is going to sound :male-astronaut: but: it becomes just like any other part of our system, where we need to keep a single source of truth for the semantics of that data#2019-01-1522:11lilactowne.g. maybe in the future we'll be building "spec-services"#2019-01-1522:11lilactown(I'm thinking of building a "spec-service")#2019-01-1522:11favilathat just sounds hard to do given predicates#2019-01-1522:12dustingetz@lilactown What is a spec service?#2019-01-1522:12lilactownwell, because serializing the spec itself requires a lot of context, what I'm thinking of doing is just having, literally, spec-as-a-service#2019-01-1522:12dustingetzso like an xmlns#2019-01-1522:13lilactownGET valid/foo.core/blah => (s/valid? :foo.core/blah (:body req))#2019-01-1522:13dustingetzah cool#2019-01-1522:15lilactownfor instance, we have a lot of disparate services that need to validate similar sets of data according to the same rules#2019-01-1522:15lilactownand they're not all Clojure šŸ˜ž#2019-01-1522:17dustingetzYes, e.g. imagine internet protocols like HTTP had a formal spec written instead of English, in clojure.spec, and thus it becomes much easier to, well im not sure what i would want to think about it#2019-01-1522:24jaihindhreddyI was just about to point out hyperfiddle. Then I read your name šŸ™‚#2019-01-1522:25dustingetzLol, thanks#2019-01-1522:18dustingetzIt is a bit like HATEOAS#2019-01-1522:18dustingetzin fact it is HATEOAS#2019-01-1522:21dustingetzI retract that, but it is really interesting to imagine#2019-01-1522:21dustingetzIf spec is clojure code, it gets a little harder to start throwing them around on wires though, because how can you trust if it is safe to clojure.core/eval#2019-01-1522:22dustingetzBut there is still a trust chain, so its probably fine#2019-01-1522:22dustingetzBut youā€™re saying a service, so that bypasses the trust issue#2019-01-1522:22lilactownright. just like in clojure, you have a single, global spec registry#2019-01-1522:23lilactownyour system should have the same: a single, global registry of what your specs are#2019-01-1522:23favilayeah that's a neat idea#2019-01-1522:24favilapretty scary to run that server#2019-01-1522:24favilahopefully no one manages to get a bitcoin mining predicate on there#2019-01-1522:24favilahow to make it fast?#2019-01-1522:25favilathat's a lot of data to push over the wire#2019-01-1522:25favila(uploads of clojure data to s/valid?)#2019-01-1522:26dustingetzRight, the huge difference between Datomic schema and Clojure specs today, is that you can recklessly throw them around. For example itā€™s easier to deploy them into a validated healthcare enterprise environment#2019-01-1522:26dustingetzSlurp in some EDN updates, not that big a deal#2019-01-1522:26dustingetzSlurp in some clojure specs to pass to eval ā€“ yeah right#2019-01-1522:28dustingetzI can imagine a dialect of clojure/core that does not have infinite recursion, and no unsafe operators like !, that might be safe enough#2019-01-1522:30dustingetzYou can also make the requestor pay for compute#2019-01-1522:34lilactownyeah, easier to tag things with specs and send the data you want validated along with the names of specs you want it to be validated against#2019-01-1523:03dustingetzHere is a cool application of specs as data: [:select.ui (select-keys props [:value :class :style :on-change :read-only :on-click])] aka https://github.com/facebook/prop-types#2019-01-1523:43mishait funny how
// An object that could be one of many types
  optionalUnion: PropTypes.oneOfType([
    PropTypes.string,
    PropTypes.number,
    PropTypes.instanceOf(Message)
  ]), ...
is data, and
(d/q '[:find ?e :where [?e :dustingetz.reg/email]])
is not d
#2019-01-1613:44dustingetzAlex you convinced me that (s/coll-of ā€¦) is data and that predicate equality isnā€™t a practical objection#2019-01-1613:44dustingetzThanks for the deep dive yesterday#2019-01-1718:23rickmoynihanIs it possible to (s/merge (s/keys :req-un [:foo/foo]) (s/keys :req-un [:bar/foo])) such that the rightmost stomps on keys to the left? i.e. looking for a way to combine s/keys with merge semantics for merging and precedence#2019-01-1718:26Alex Miller (Clojure team)define ā€œstomps onā€#2019-01-1718:27Alex Miller (Clojure team)I guess replaces?#2019-01-1718:28Alex Miller (Clojure team)seems like you want some kind of operation on specs themselves rather than composite specs#2019-01-1809:44rickmoynihanYes, by stomps on I mean replaces. And yes Iā€™m looking for an operation on specs themselves. Essentially I want two different specs that share the majority of their keys, but have a different spec for one or two keys. If such a thing doesnā€™t exist I guess I can rework it so the commonanility is together in a shared spec, and merge that with the extra different keysā€¦ rather than trying to merge the replacement keys over the top.#2019-01-1809:46rickmoynihanI guess itā€™s just a little confusing that s/merge has different semantics WRT keys than merge - but I get why s/merge is like this. I guess Iā€™m after an s/merge-keys for merging s/keys specs.#2019-01-1809:46rickmoynihanor maybe s/keys-merge would be a better name.#2019-01-1813:24Alex Miller (Clojure team)Having different semantics for the same key is generally a smell, unless your override is a subset (rather than a different set). I think there may be some new options in spec 2 but Iā€™d say right now I would separate the non-different keys into one spec, then merge with different partial map specs as needed.#2019-01-1814:20rickmoynihanThanks, Iā€™d already done what you suggestā€¦ Itā€™s 3rd party data ā€” so I donā€™t get to define the keysā€¦ but yes the override is actually a subset. Also Iā€™m not sure if this a smell or not; but the superset spec really just exists to represent ā€œnoiseā€ that I donā€™t care about but have to ignore gracefullyā€¦ Iā€™m not using those specs as specs so much as Iā€™m using the spec API as a way to write generators of noise I need to ignore not barf on. e.g. I have a spec :raw/Body representing all message bodys I should just skip over and a spec :specific/Body for ones I care about. I could use s/or to model this but Body is nested in some other specs e.g. a Message and essentially I donā€™t want to have to redefine all the intermediates. I think this is essentially similar to the ā€œMaybe Notā€ problems.#2019-01-1814:48Alex Miller (Clojure team)well you can use s/and now to combine your generic spec and (when necessary) the more restrictive spec#2019-01-1814:48Alex Miller (Clojure team)so it doesnā€™t replace, but only the more precise thing will pass both#2019-01-1814:50rickmoynihanyeah I did consider that but how do you do so when the spec isnā€™t the root spec?#2019-01-1814:52Alex Miller (Clojure team)yeah, thatā€™s tricky - you need to rebuild everything back to the root#2019-01-1814:52Alex Miller (Clojure team)this is the kind of thing that will be easier in the spec 2 select stuff#2019-01-1814:52rickmoynihanyeah#2019-01-1814:53rickmoynihanthat was my feeling too#2019-01-1814:54rickmoynihanthanks for confirming my intentions were good, at least! šŸ™‚#2019-01-1814:55rickmoynihanFWIW I have rebuilt to the rootā€¦ and rearchitectured the commonality into another spec etcā€¦ so itā€™s not so bad this time šŸ™‚#2019-01-1723:20kennyCan you merge a map-of and keys?#2019-01-1723:23tayloryes#2019-01-1723:24taylorfor example, this wonā€™t conform if :bar value isnā€™t a string (or if any of the keys arenā€™t keywords, or if :foo is missing, etc.):
(s/conform
  (s/merge (s/keys :req-un [::foo])
           (s/map-of keyword? string?))
  {:foo "" :bar ""})
#2019-01-1809:51rickmoynihanThanks. You can do this - but this is not the effect I want. https://clojurians.slack.com/archives/C1B1BB2Q3/p1547804648581400?thread_ts=1547749697.578800&amp;cid=C1B1BB2Q3#2019-01-1813:11taylorAh, I thought Kennyā€™s question was independent from yours#2019-01-1813:13rickmoynihandunno it may have been šŸ™‚#2019-01-1809:15borkdudeTrying to spec partition-all, but I get an exception. Repro:
Clojure 1.10.0
user=> (require '[clojure.spec.alpha :as s])
nil
user=> (require '[clojure.spec.test.alpha :as stest])
nil
user=> (s/fdef clojure.core/partition-all :args (s/cat :n pos-int?))
clojure.core/partition-all
user=> (stest/instrument `partition-all)
[clojure.core/partition-all]
user=> (partition-all 1)
Execution error (ClassCastException) at user/eval142 (REPL:1).
clojure.spec.test.alpha$spec_checking_fn$fn__3026 cannot be cast to clojure.lang.IFn$LO
#2019-01-1809:17borkdudethat could be because of the type hint on the arity-1 of partition-all#2019-01-1809:21borkdudehttps://dev.clojure.org/jira/browse/CLJ-1941#2019-01-1810:24bronsayep, that's it#2019-01-1810:25borkdudethis is actually the first time I couldnā€™t instrument something on CLJ that I could on CLJS, normally itā€™s the other way around šŸ˜‰#2019-01-1810:25bronsaI guess instrument could look at the arglists and figure out if it needs to compile the wrapping fn to a primitive fn#2019-01-1812:35rickmoynihanHow do people manage the seperation of concerns around app code, specs and generators? Generators are essentially a testing concern, but you also use want them at dev time so you can inspect sample values etcā€¦ Part of my issue is that writing generators eats up quite a few LOCs, so Iā€™m tempted to put them in a different ns to the specs and app logicā€¦ I also feel like in clojure that they should be optionalā€¦ which means you donā€™t really want them in the require tree in the production app. Are there any patterns people have for arranging these? Currently I just them all in the same ns ā€” but at some point I worry itā€™ll be hard to see the wood for the trees.#2019-01-1812:35rickmoynihanThinking also are there any lazy-loading patterns I can easily apply to my own generators?#2019-01-1812:36rickmoynihanAlso you often want different generators depending on context#2019-01-1812:37rickmoynihanI know you can override them via st/check etc ā€” but not really sure how to organise all these bits#2019-01-1812:44borkdude@rickmoynihan clojure.spec.test/check takes overridable generators. I use those, if I donā€™t make a generator attached to the spec itself.#2019-01-1812:44borkdudeoh sorry, that was your last sentence#2019-01-1812:51rickmoynihanyeah#2019-01-1815:36thomasHi, I am trying to write a generator... but no luck... I am trying to follow the examples from the test.check page... but no luck (yet)#2019-01-1815:37thomasit can't find certain functions for instance.. while as far as I can work out they should be in the clojure.spec.gen.alpha namespace#2019-01-1815:53taylorYou may have to reference test check namespaces directly. Spec doesnā€™t expose all the stuff thatā€™s available#2019-01-1815:37thomasfrequency for instance.#2019-01-1815:38borkdude@thomas are you using CLJ or CLJS?#2019-01-1815:38thomasCLJ#2019-01-1815:39borkdudemaybe make a small repro of what youā€™re trying to do so we can take it from there?#2019-01-1815:41thomasok will do. thank you. (I can't do it now, but I'll try later tonight) Thank you @borkdude#2019-01-1816:20rickmoynihan@thomas you know about lazy loading right?#2019-01-1816:21rickmoynihanyou need to be aware that clojure.spec.gen.alpha generators are subtly different to test.check ones#2019-01-1816:23rickmoynihanI always forget the detailsā€¦ but IIRC generators from spec are wrapped in a 0 arg function to allow lazy loadingā€¦ so if youā€™re trying to use test.check generators you need to wrap them one more time in a fn callā€¦ or something like that. šŸ™‚#2019-01-1816:25borkdude@thomas hereā€™s an example that might be helpful in addition to what @rickmoynihan said: https://github.com/borkdude/speculative/blob/master/test/speculative/core_test.cljc#L107#2019-01-1816:30rickmoynihan@borkdude: respeced looks usefulā€¦ do you have any machinery to trigger st/check in a deftest and get a good error out from clojure.test when things go wrong?#2019-01-1816:31rickmoynihanIā€™ve written a few things like this in the past ā€” but the results have always been lackluster.#2019-01-1816:31borkdudeyes, Iā€™m doing that here: https://github.com/borkdude/speculative/blob/master/test/speculative/test_utils.cljc#L31 when a spec doesnā€™t conform, I see the thrown exception#2019-01-1816:32borkdudethe crucial part is (clojure.test/is (respeced.test/successful? stc-result#)), succesful? realizes the stc-result (itā€™s lazy) and checks if thereā€™s at least one#2019-01-1816:35rickmoynihanyeah Iā€™ve done things like that before ā€” but yours looks like it might give better outputā€¦#2019-01-1816:35borkdudeit works fine for me#2019-01-1816:36rickmoynihanIā€™m guessing clojure.test just prints the error as (is (not (successful? ,,,)))#2019-01-1816:36borkdudetry it in the REPL and youā€™ll see šŸ™‚#2019-01-1816:36borkdudeyes#2019-01-1816:36rickmoynihanYeah Iā€™m going to šŸ™‚#2019-01-1816:44rickmoynihan@borkdude: would you say a use case for rt/check-call is to check real sample data conforms to the spec then? i.e. as a way to check that the spec is valid, as much as the data/fdef?#2019-01-1816:54borkdude@rickmoynihan check-call is made for checking fdefs on example based tests#2019-01-1816:55borkdudee.g. to verify that your ret spec works given an example#2019-01-1817:00borkdudeone problem Iā€™d like to tackle is: your function accepts arity 1 and 2, but you specā€™ed only 2. thereā€™s no way to detect this with stest/check since it only generates 2 args.#2019-01-1817:01borkdudeso fdefs could be insuffient. would be nice if that could be automatically detected.#2019-01-1817:02rickmoynihanagreed#2019-01-1817:03borkdudeit would be nice if you could generate data to detected that your fdef missed a case. I guess you can#2019-01-1817:03rickmoynihan+1#2019-01-1817:04rickmoynihanrelated what I also find useful is a test of a s/conform against some example dataā€¦ so you know you specā€™d the right thing.#2019-01-1817:05rickmoynihanbut then you find that clojure.test/is is effectively testing a boolean s/valid? is insufficient as you want to see the spec failure not just (is (not (s/valid? ,,,)))#2019-01-1817:06borkdudethatā€™s why I made check I think#2019-01-1817:06borkdudethis library is aimed at fdef checking#2019-01-1817:06rickmoynihanyeah#2019-01-1817:07rickmoynihanbut I think thereā€™s an equivalent need for spec checking too#2019-01-1817:07rickmoynihanas you say check requires an fdef#2019-01-1817:12rickmoynihanI was writing things like (is (not= :clojure.spec.alpha/invalid (s/conform :spec/here {:sample :data})) but you need to be careful using :clojure.spec.alpha/invalid in these macros because it can cause compilation errors due to :clojure.spec.alpha/invalid being used to check the macro syntax#2019-01-1817:13rickmoynihanbut the reasoning there is that comparing (= ::s/invalid sample-result) gives you a meaningful error in clojure.test.#2019-01-1817:13borkdudehah, yeah, I had a similar issue when specā€™ing assoc, because then you cannot assoc the value :clojure.spec.alpha/invalid which Cursive does when you use the REPL šŸ˜›#2019-01-1817:13borkdudesolved that by writing my own ::any spec#2019-01-1817:13rickmoynihanyeah IIRC thereā€™s a ticket open on the issue somewhere#2019-01-1817:14borkdudeyeah. one of the proposals there is to use a singleton object to represent invalid inside spec and only to the outside use the keyword#2019-01-1817:14borkdudeI think that makes sense#2019-01-1817:16borkdudehttps://github.com/borkdude/speculative/blob/master/src/speculative/specs.cljc#L29#2019-01-1817:23Alex Miller (Clojure team)you can use s/valid? rather than checking ::s/invalid#2019-01-1817:24Alex Miller (Clojure team)or s/invalid?#2019-01-1817:24borkdudeI used s/invalid?#2019-01-1817:24Alex Miller (Clojure team)I donā€™t think a singleton object is a feasible solution#2019-01-1817:24Alex Miller (Clojure team)Iā€™m not even sure I agree itā€™s a problem#2019-01-1817:26borkdude@alexmiller Itā€™s a problem when you instrument a function that must be able to deal with ::s/invalid as one of the arguments. This is important in tooling such as Cursive where ::s/invalid is remembered as REPL state for example#2019-01-1817:27Alex Miller (Clojure team)there are a very small number of such functions and some workarounds even for those cases#2019-01-1817:27borkdudeyes, a workaround is writing your own ::any šŸ˜‰#2019-01-1817:28Alex Miller (Clojure team)I consider ā€œdo nothingā€ to be a leading contender alternative for that ticket#2019-01-1817:29borkdudehow many votes are representative of ā€œconsidered important enough by the communityā€ in general for a CLJ Jira issue?#2019-01-1817:30Alex Miller (Clojure team)votes are about attention, not answers#2019-01-1817:30Alex Miller (Clojure team)if youā€™re not specā€™ing core functions used by spec, how often is this an issue?#2019-01-1817:30Alex Miller (Clojure team)b/c thatā€™s when Iā€™ve seen people raise it#2019-01-1817:31borkdudeprobably tooling, but then it would only be an issue for the developer of that tooling#2019-01-1817:32Alex Miller (Clojure team)I donā€™t consider this a settled result (which is why the ticket is still open)#2019-01-1817:32Alex Miller (Clojure team)thatā€™s just my current thinking on it#2019-01-1817:34Alex Miller (Clojure team)changing topics, I just merged a giant refactor in spec-alpha2 in case anyone is interested#2019-01-1817:34Alex Miller (Clojure team)although since itā€™s a refactoring rather than new stuff, itā€™s not particularly exciting#2019-01-1817:34borkdudeI think this issue is more important than the any problem: https://dev.clojure.org/jira/browse/CLJ-2443#2019-01-1817:35borkdudethatā€™s not only about self-specā€™ing#2019-01-1817:35Alex Miller (Clojure team)agreed#2019-01-1817:35Alex Miller (Clojure team)I havenā€™t had time to look at it seriously but I am aware of it#2019-01-1817:36borkdudeabout spec-alpha2: cool! Iā€™ll try it out soon#2019-01-1817:37Alex Miller (Clojure team)there are some subtly important differences in the api for callers#2019-01-1817:39Alex Miller (Clojure team)I will try to write about those in my journal today rather than spew them here#2019-01-1819:19tylerWhat approach do folks use to handle the namespacing of nested keys. We are trying to spec check the interface to an external service and the payload is a nested map. Namespaced keywords would seem to imply having to create a namespace for each nesting which isnā€™t ideal. Alternatively, we could generate the specs but this seems to have its own problems. As far as I can tell this would imply having to compose macros which seems excessive. Additionally, the caller side would have to construct the long form of these keywords in order to pull out the necessary data since there would be no namespaces to alias with this approach. Is there another approach that Iā€™m missing? Its also possible that spec isnā€™t appropriate for this use case.#2019-01-1819:20borkdude@tyler a similar question was asked not too long ago here#2019-01-1819:22tylerIs it in the recent history? Iā€™m having trouble searching for it.#2019-01-1819:24tayloryou don't have to create "real" namespaces for the keys, you can qualify keys with arbitrary prefixes e.g. :fake-ns/foo#2019-01-1819:25manutter51We sometimes use "dotted-path" namespaces like :order.line-item/qty#2019-01-1819:25borkdude@tyler https://clojurians.slack.com/archives/C1B1BB2Q3/p1547482812312100#2019-01-1820:20tylerThank you#2019-01-1903:02Alex Miller (Clojure team)http://insideclojure.org/2019/01/18/journal/#2019-01-1903:02Alex Miller (Clojure team)spec 2 stuff#2019-01-1903:24seancorfield@alexmiller Want me to go back to my spec2 branch and start testing our code against this again? Happy to do so starting Monday if you think this should have parity with current spec?#2019-01-1904:27Alex Miller (Clojure team)Sure, go for it
#2019-01-1904:30seancorfieldThank you!#2019-01-1903:24seancorfield(originally posted in #announcements by accident!)#2019-01-1903:28madstapI see that s/keys is still part of the api in spec2. I'm a bit surprised given RH's talk on how he wants to separate that out into s/schema and s/select. Are these going to be added in addition to s/keys then or is s/keysgoing away in favor of them in the next iteration? If they'll all exist together, will it ever make sense to use s/keys over s/schema/`s/select`?#2019-01-1903:28seancorfield@madstap That's a bit unfair -- the refactor to spec2 started before Conj.#2019-01-1903:28seancorfieldThe shape/required stuff is further down the pike.#2019-01-1903:29seancorfieldThe work in spec2 is groundwork to prepare for that.#2019-01-1903:30madstapSure, my question is more about the direction spec will take. Is s/keys "living on borrowed time" so to speak?#2019-01-1903:33seancorfieldYeah, I'd say s/keys is living on borrowed time. Spec is still alpha so it's subject to API breakage.#2019-01-1903:34seancorfieldGiven the namespace changes, you can have both libraries loaded and both types of specs defined while you transition.#2019-01-1903:34seancorfieldThat's the important part of namespace changes.#2019-01-1903:37madstapYeah, just like RH talked about in his penultimate(?) keynote. Pretty cool to see how that approach works out in practice.#2019-01-1903:40seancorfieldThis is why next.jdbc will be a new namespace at least and a new artifact more likely šŸ™‚#2019-01-1904:23Alex Miller (Clojure team)Unknown yet on s/keys. We havenā€™t started any of the work yet for the stuff rich talked about at conj#2019-01-1904:25Alex Miller (Clojure team)Look, people complain when we work in a private repo, then drop something at the end. Weā€™re trying to do something different here which means you get to see all the steps along the way, so things will be changing over a probably months#2019-01-1904:26Alex Miller (Clojure team)Some of it will be experimental #2019-01-1904:27madstapI'm not complaining, I hope it didn't come across that way#2019-01-1904:27seancorfieldI love that you're dropping stuff we can test against.#2019-01-1904:27Alex Miller (Clojure team)Just saying, donā€™t expect it all at once#2019-01-1904:27seancorfieldI don't much care which way you're going with the API -- we'll follow, regardless, even if we have to keep changing code šŸ™‚#2019-01-1904:29seancorfield(we're living on the bleeding edge for a reason -- it gives us an edge!)#2019-01-1904:29madstapAnd this iteration solves one of the big pain points which were programmable specs#2019-01-1904:34madstapOne observation about that though: The way spec op macros are implemented now breaks clojure.walk/macroexpand-all, which will stack overflow if the form contains any spec macros.#2019-01-1904:38seancorfieldThat's good feedback for the development of spec2. Last time I tried our code against spec2, a lot of stuff broke. I'll be trying it again on Monday when I get back to work. Breakage is to be expected in prerelease work and if folks pitch in and try it, the core team get more feedback which is helpful!#2019-01-1908:54borkdudeI have this spec in spec1:
#?(:clj (s/def ::java-map
          (s/with-gen #(instance? java.util.Map %)
            (fn [] (gen/fmap #(java.util.HashMap. %)
                             (s/gen ::map))))))
On spec2 I get:
Caused by: java.lang.IllegalArgumentException: no conversion to symbol
	at clojure.core$symbol.invokeStatic(core.clj:596)
	at clojure.core$symbol.invoke(core.clj:589)
	at clojure.spec_alpha2$explicate_1$fn__904.invoke(spec_alpha2.clj:314)
I have to look into this more, I just briefly tried it
#2019-01-1914:19Alex Miller (Clojure team)The key thing is that specs have to start off as forms, not function objects. So here the anonymous function is getting evaluated too early and I suspect you need to quote the (fn ... ). That may be fixable though in spec since I turned with-gen into a macro. Iā€™ll take a look when Iā€™m at a computer.#2019-01-1920:08borkdudethanks#2019-01-2011:01borkdudeQuoting the fn didnā€™t help#2019-01-1910:03borkdudesame with:
#?(:clj (s/def ::char-sequence
          (s/with-gen
            #(instance? java.lang.CharSequence %)
            (fn []
              (gen/one-of (map #(gen/fmap %
                                          (s/gen ::string))
                               [#(StringBuffer. %)
                                #(StringBuilder. %)
                                #(java.nio.CharBuffer/wrap %)
                                #(String. %)]))))))
#2019-01-1910:06borkdudeThis spec breaks:
(defn seqable-of
  "every is not designed to deal with seqable?, this is a way around it"
  [spec]
  (s/with-gen (s/and seqable?
                     (s/or :empty empty?
                           :seq (s/and (s/conformer seq)
                                       (s/every spec))))
    #(s/gen (s/nilable (s/every spec :kind coll?)))))

(s/def ::seqable-of-map-entry (seqable-of ::map-entry))
with the message:
Caused by: java.lang.IllegalArgumentException: No method in multimethod 'create-spec' for dispatch value: speculative.specs/seqable-of
#2019-01-1910:20borkdudeI tried to fix it like this: https://github.com/borkdude/speculative/blob/spec-alpha2/src/speculative/specs.cljc#L86 But I get:
user=> (s/valid? ::ss/seqable-of-map-entry {:a 1 :b 2})
Execution error (IllegalArgumentException) at clojure.spec-alpha2/pred-impl (spec_alpha2.clj:132).
Iā€™ll leave it at this for now. Broken code is indicated with FIXMEā€™s here: https://github.com/borkdude/speculative/blob/spec-alpha2/src/speculative/specs.cljc
#2019-01-1918:36lambdamHello, I have a case where having the keys of a map spec forced to be namespaced qualified, leads to a case like this:
(defn do-this [args]
  ;; ...
  {:foo :bar
   :resource {:key-a 1}})

(s/def ::foo keyword?)
(s/def ::key-a int?)
(s/def ::resource (s/keys req-un [::key-a]))
(s/fdef do-this
  :ret (s/keys :req-un [::foo ::resource]))

(defn do-that [args]
  ;; ...
  {:foo :bar
   :resource {:key-b "plop"}})

(s/def ::key-b string?)
;; (s/def ::resource (s/keys req-un [::key-b])) <-- problem here: the spec is being redefined
(s/fdef do-this
  :ret (s/keys :req-un [::foo ::resource]))
Here I would like to have to functions in the same namespace that return two different "partial views" of a resource along with other information in a map. It seems that I can't due to the fact that two pieces of information are glued together: the unqualified key and the spec. I feel like I miss kind of a scoped spec declaration or a function like this : (s/key :resource <spec>) that I can embed in the declaration: (s/keys :req-un [(s/key :resource <spec>) ::foo]). But I feel also that I am maybe misusing the library. Has someone bumped into this problem already? Thanks
#2019-01-1919:58misha@dam since you use :req-un in all 3 s/keys, you can pick another namespace for second ::resource:
(s/def :baz/resource  (s/keys req-un [::key-a]))
(s/def :quux/resource (s/keys req-un [::key-b]))
(s/fdef do-this :ret (s/keys :req-un [::foo :baz/resource]))
(s/fdef do-that :ret (s/keys :req-un [::foo :quux/resource]))
#2019-01-2008:24lambdam@misha thanks for this solution. I did that in a way for something a bit different and the problem I bumped into was that you cannot require the namespace containing the specs in another namespace and use it with a shortcut. Let's say the previous code was in a namespace a.
(ns b
  (:require [myapp.lib.a :as a]))

(s/valid? ::a/foo <whatever>)

(s/valid? :quz/resource <whatever>) <-- I have to remember that this comes from namespace myapp.lib.a
and doing this
(ns b)

(s/valid? :quz/resource <whatever>)
the dependency graph of spec definitions is not right and the compiler will complain (it happened to me and I spent time renaming a lot of specs) Thus, even though I can do it, I tend to feel that it's a clumsy solution.
#2019-01-2013:01mishatying all specs' ns to file ns - is often not a good idea#2019-01-2008:50lambdamAnother question that comes to my mind, how can I validate a map whose keys are string (typically data from an HTML form). Doing this (s/def ::form-data (s/keys :req ["foo" "bar"])) is half of the spec. I'd like to qualify the content of the keys. Again I feel like missing a syntax like this: (s/def ::form-data (s/keys :req [(s/key "foo" string?) (s/key "bar" int?)])).#2019-01-2009:37lmergeni keep running into an issue with managing the complexity of my specs, and was wondering whether i'm "doing it all wrong". specifically, of quite a few specs, i need to have multiple variations of the same "schema", depending on the context. to give an example, a user should not need to have an ID before it is inserted into the database, but it does need an ID when it is. this is a simple example, but this pattern repeats. since ideally, i want my code to be able to declare whether it expects a a "stored user" or not, i currently compose these things as follows:
(s/def :user (s/keys ...))
(s/def :user/inserted (s/merge :user (s/keys ...))
on the surface this seemed like a good idea, but after a while of using this, i discovered downsides to this. most importantly: as soon as you have other specs that depend upon inserted users or not (e.g. a "company" vs "inserted company"), you repeat this pattern a lot, even when there is not necessarily a difference between a "company" and "inserted company". as an alternative to this, i was thinking of making the state an explicit property i can multi-spec on. this makes the approach more data-driven. but then i wouldn't be able to directly say, "i expect an inserted user" in the code through specs anymore. are there any alternatives to this ? i hope my question is making a bit of sense šŸ™‚
#2019-01-2009:38lmergenthis code is a good example: https://github.com/lagenorhynque/spec-examples/blob/master/specs/clj/spec_examples/geometry/specs.clj#L13#2019-01-2009:39lmergennow, code cannot just fdef anymore saying it expects a ::shape/cube or a ::shape/sphere, it can only fdef on ::shape#2019-01-2009:45lmergenthis is another discussion on the same topic => https://lispcast.com/reduce-complexity-with-variants/#2019-01-2016:52ericnormand@lmergen I think Rich Hickey has noticed that problem, too#2019-01-2016:52ericnormandhe talked about it at the conj and hinted at a big change to spec to make this easier#2019-01-2016:52lmergenyeah I saw that, sounds very promising!#2019-01-2016:52ericnormandbasically, you define a map spec separately from the selection#2019-01-2016:53ericnormandwhich lets you separate out the definition of the entity from the context#2019-01-2016:53lmergenyes so you declare what you need rather than what something is#2019-01-2016:54ericnormandawesome#2019-01-2016:54lmergenok thanks I kind of forgot about that#2019-01-2016:54ericnormandi have faced this same problem myself#2019-01-2016:54ericnormandproliferation of specs because, for instance, if you have all the CRUD operations, they each require slightly different data#2019-01-2016:54ericnormandthough you'd like to think of them as operating on the same entity#2019-01-2016:55borkdudeitā€™s the same problems that type systems without extensible records have#2019-01-2016:56lmergenyes #2019-01-2016:57lmergenso now spec becomes a category system :)#2019-01-2016:57borkdudewhatā€™s that#2019-01-2016:57lmergensorry I meant category as in category theory#2019-01-2016:58lmergenrather than describing types you describe traits#2019-01-2016:59lmergenI am fairly sure I have just mentioned a few forbidden words #2019-01-2016:59borkdudeā€” awkward silence ā€”#2019-01-2016:59borkdudešŸ˜‰#2019-01-2016:59lmergenhaha#2019-01-2017:03Alex Miller (Clojure team)I think itā€™s most useful to think of it as oriented around value sets, as described by predicates#2019-01-2017:04lmergenIā€™ll be watching your progress alex, just found the new spec repo#2019-01-2017:05Alex Miller (Clojure team)maps are composites of their attributes and the key is that semantics are derived from the attributes, not from the composite itself#2019-01-2017:05lmergenso they become self describing ?#2019-01-2017:07Alex Miller (Clojure team)not sure thatā€™s the important bit#2019-01-2017:08lmergentrue #2019-01-2017:10Alex Miller (Clojure team)the new idea from Richā€™s last keynote is that different functions can speak more precisely about which subset of the composite it cares about, and we can have better tools for describing what a function does with those inputs wrt to the output#2019-01-2201:21kennyIs there any way to get more info about why a spec generator is failing? This is the error message:
clojure.lang.ExceptionInfo: Couldn't satisfy such-that predicate after 100 tries. {:pred #object[clojure.spec.alpha$gensub$fn__1876 0x7230d43f "
Nothing in the stacktrace points to my code. I guess I could go through and individually test all my specs for which failed but in this case the spec is huge and that would be very time consuming.
#2019-01-2201:22gfredericks@kenny not an answer to your direct question, but that particular failure is usually caused by an s/and, where one of the non-initial predicates is not likely to happen by chance#2019-01-2201:22gfredericks(or all of them together, more accurately)#2019-01-2201:22kennyRight. There's a lot of those šŸ˜ž#2019-01-2201:24gfredericksit's a long outstanding issue between spec and test.check; test.check's alphas have a feature in such-that that would let spec report what spec is causing it#2019-01-2201:24gfredericksbut spec doesn't use that feature yet afaik#2019-01-2201:27seancorfieldIt doesn't help you now @kenny but when I'm writing specs, I pretty much always have a (comment ..) form with calls to s/exercise for each of my specs, and I make sure they generate as I'm writing them... for exactly this reason.#2019-01-2201:31kennyI have some decent guesses for the ones that fail but the feature @gfredericks mentioned would significantly improve this experience. Then you could get rid of all those comment blocks @seancorfieldšŸ˜‰#2019-01-2201:32kennyAny idea if there's a jira issue for this?#2019-01-2201:35seancorfieldI have a lot of Rich Comment Forms in my code šŸ™‚#2019-01-2201:36seancorfieldI find they're a good way to keep notes about my thinking as I develop, as well as containing expressions that produce test data and show usage of functions -- and specs šŸ™‚#2019-01-2213:12lmergenwhat would be the recommend way to do a multi-spec dispatch for some nested map variable ? e.g. let's say that i have this
{:entity/type :user
 :entity/data 
 {:user/id 1234}}
and i want to multi-spec the :entity/data based on the :entity/type ... this seems pretty tricky ?
#2019-01-2213:13lmergenbecause this is not possible, although it's something that i would want:
(s/def :entity/data (s/multi-spec entity-data :entity/type))
(s/def :entity (s/keys :req [:entity/type :entity/data]))
#2019-01-2213:16lmergennow i could repeat :entity/type as a property of :entity/data again, but that seems redundant#2019-01-2213:43jeroenvandijkIf you really have to, you can use multispec with the dispatch on type. Repeat this for all the different types:
(s/def :your/data any?)


(s/valid?
 (s/coll-of (s/or :type (s/tuple #{:entity/type} #{:user})
                  :data (s/tuple #{:entity/data}
                                 :your/data))
            :kind map?)
 {:entity/type :user
  :entity/data :foo})
#2019-01-2213:45jeroenvandijkIf you control the design of the data, but i guess you dont, there is probably a better way more in line with clojure.spec#2019-01-2214:31lmergenšŸ¤” not sure whether this is brilliant or evil#2019-01-2214:31lmergenit works though#2019-01-2214:34lmergenhow would you redesign the data ? lift the data inside :entity/data into the parent map so that multispec works ?#2019-01-2214:34lmergenhow would you redesign the data ? lift the data inside :entity/data into the parent map so that multispec works ?#2019-01-2221:39jeroenvandijkyeah sorry for giving the evil hack šŸ™‚ Changing the data will make it much cleaner. And using multispec for example gives you very nice specific validation errors e.g. when using it with expound#2019-01-2222:54lmergenyes thanks a lot, your comment was helpful!#2019-01-2223:23jeroenvandijkbtw, not using a namespaced keyword, so :data instead of :entity/data could also be used as a fix instead of lifting the data inside. You would use s/keys with :req-un instead of :req#2019-01-2214:55lmergenactually that makes a lot of sense #2019-01-2219:48mishaqualified keywords allow you to have a mix of "unrelated" keys in a flat map#2019-01-2220:26lmergenyes, it is obvious that I was doing this wrong #2019-01-2220:26lmergenmakes a lot of sense #2019-01-2312:02victorbHm, I'm a bit confused about the s/or. Trying to get spec to accept that a key can be either a-map or b-map, both of them are fine, so far got this:
(s/def :test/a-map (s/keys :req-un [::name]))
(s/def :test/b-map (s/keys :req-un [::title]))
(s/def :test/key (s/or :test/a-map :test/b-map))
(s/def :test/map (s/keys :req-un [:test/key]))

(comment
  (s/valid? :test/map {}) ;; false
  (s/valid? :test/map {:key {:name "hello"}}) ;; false / should be true
  (s/valid? :test/map {:key {:title "hello"}}) ;; true
  )
#2019-01-2312:03jeroenvandijkyou need to label it with a keyword, e.g.
(s/def :test/key (s/or :a :test/a-map :b :test/b-map))
#2019-01-2312:04victorb@jeroenvandijk ooh, so simple. Thanks a lot for the fast help! šŸ‘#2019-01-2312:05victorbugh, should have been obvious if I just read the docs slower, missed the key+pred pairs. I'll blame it on too little coffee#2019-01-2312:08jeroenvandijkI had the same once, no worries#2019-01-2312:09jeroenvandijkIt doesn't blow up either so hard to know you did something wrong (without coffee šŸ˜‰ )#2019-01-2406:26dacopareI'm trying to write a spec for a map that satisfies two keys specs.
(s/def ::a string?)
(s/def ::b string?)

(s/def ::a-map (s/keys :req [::a]))
(s/def ::b-map (s/keys :req [::b]))

;; Satisfies both specs
(s/def ::a-and-b (s/and ::a-map ::b-map))

;; Extend an existing spec
(s/def ::another string?)
(s/def ::a-extended
  (s/and ::a (s/keys :req [::another])))
But generation obviously fails. Is there a way to do this that produces a good generator? Perhaps using the and section in s/keys? (s/keys :req [::x ::y (or ::secret (and ::user ::pwd))] :opt [::z])
#2019-01-2406:27seancorfields/merge#2019-01-2406:27seancorfieldItā€™s designed to merge multiple s/keys specs.#2019-01-2406:28dacopareThank you very much @seancorfield, it seems to be the perfect tool for the job.#2019-01-2412:55victorbHow can I spec a map where the keywords can be anything (basically IDs in this case) but I do know what I want the values to be#2019-01-2413:01aisamuSomething involving #{}, perhaps?#2019-01-2413:04borkdude@victorbjelkholm429 what about (s/map-of any? #{:value1 :value2})#2019-01-2413:06victorbThanks @aisamu and @borkdude. map-of any? seems to be what I was looking for.#2019-01-2413:08victorbEnded up with (s/map-of keyword? :my/other-spec) šŸ™‚#2019-01-2413:15fmjreyBetter to change your spec before coding than changing it after coding!#2019-01-2414:38victorb@fmjrey funny you say that as I currently sit and trying to retrofit clojure.spec after getting the basic mvp features in place šŸ™‚#2019-01-2510:54borkdude@alexmiller left a few comments about the new SHA here https://github.com/borkdude/speculative/issues/124#issuecomment-457532789 and here https://github.com/borkdude/speculative/issues/124#issuecomment-457533593#2019-01-2513:55Ben HammondI've just started playing with . I can see that the :args are getting verified, but I cannot see how to get the :ret verified#2019-01-2513:56Ben Hammondthis is the code that I pinched from https://blog.taylorwood.io/2017/10/15/fspec.html
(defn digits
  "Takes just an int and returns the set of its digit characters."
  [just-an-int]
  (into #{} (str just-an-int)))
=> #'dev/digits
(s/fdef digits
        :args (s/cat :just-an-int int?)
        :ret (s/coll-of char? :kind set? :min-count 1))
=> dev/digits
(spec.test/instrument `digits)
=> [dev/digits]
(digits 1)
=> #{\1}
(digits "1")
Execution error - invalid arguments to dev/digits at (form-init1603011188380229902.clj:1).
"1" - failed: int? at: [:just-an-int]
#2019-01-2513:57Ben Hammondso far so good; now I want to modify the :ret to something impossible and retest#2019-01-2513:57Ben Hammond
(s/fdef digits
        :args (s/cat :just-an-int int?)
        :ret (s/coll-of keyword? :kind set? :min-count 1))
=> dev/digits
(digits 1)
=> #{\1}
#2019-01-2513:58Ben HammondHmm I was expecting the return value to fail. I can see the change in the doc string
(doc digits)
-------------------------
dev/digits
([just-an-int])
  Takes just an int and returns the set of its digit characters.
Spec
  args: (cat :just-an-int int?)
  ret: (coll-of keyword? :min-count 1 :kind set?)
=> nil
#2019-01-2513:59Ben Hammondwhat do I need to do to get the :ret spec verified?#2019-01-2513:59borkdude@ben.hammond ret is not checked during instrumentation#2019-01-2513:59borkdudeit is during generative testing#2019-01-2513:59borkdudethis is FAQ number 1 I guess#2019-01-2514:00Ben Hammondis that a feature? or a bug?#2019-01-2514:00borkdudedesign decision#2019-01-2514:01Ben Hammondokay, so I have to manually call (spec/conform that's not a biggie Thanks#2019-01-2514:01borkdude@ben.hammond this library can help you with checking examples: https://github.com/borkdude/respeced/blob/master/README.md#check-call#2019-01-2514:01borkdudeThereā€™s also orchestra which turns on ret checking in instrumentation#2019-01-2514:02Ben HammondWell I was just trying to pace out my basic understanding don't need to get too fancy at this point#2019-01-2514:02Ben Hammondthanks#2019-01-2514:05vemvis there a good predicate for checking if x can be used as a spec? should be true for int?, (spec/spec ...), #{1 2 3}#2019-01-2514:09Ben Hammond
(doc spec/def)
-------------------------
clojure.spec.alpha/def
([k spec-form])
Macro
  Given a namespace-qualified keyword or resolvable symbol k, and a
  spec, spec-name, predicate or regex-op makes an entry in the
  registry mapping k to the spec. Use nil to remove an entry in
  the registry for k.
=> nil
#2019-01-2514:09Ben Hammondany predicate should be fine#2019-01-2514:09Ben Hammond(which will include int? and hashsets)#2019-01-2514:10Ben Hammondor the keyword identifier of an existing spec#2019-01-2514:10Ben Hammondor an inline spec declaration#2019-01-2514:12Ben Hammondby 'regex-op` I think they mean combinations of (spec/+, (spec/cat, ,(spec/alt` etc..?#2019-01-2514:13Ben Hammondhow do you tell if an arbritary function is a predicate? I'm not sure if you can distinguish function arity from the outside#2019-01-2514:13Ben Hammondother than by provoking ArityExceptions#2019-01-2514:15Ben HammondI guess you could get java.lang.reflect to tell you and there is probably a clojurey wrapping somewhere#2019-01-2514:15vemvnot sure if we're talking about the same thing#2019-01-2514:16vemv(defn spec? "whether x can can be used as a spec" [x] ...)#2019-01-2514:16vemv^ I'm seeking this#2019-01-2514:16borkdude@vemv you want a predicate that tells you if a thing fits inside (s/fdef x --> ? <--) ?#2019-01-2514:17borkdudespec already has a spec? predicate, but that doesnā€™t do the same#2019-01-2514:17vemvyeah, not s/spec?#2019-01-2514:18danielnealifn?#2019-01-2514:18vemvseeking a filler for (s/def ::foo ->>> x <<<-)#2019-01-2514:18borkdudeI guess fn? and keyword? and spec? maybe?#2019-01-2514:18vemv(s/or :ifn ifn? :spec s/spec?)?#2019-01-2514:19borkdudemay work.#2019-01-2514:20vemvyeah, guess so, the fun part of my question was seeking a built-in that did that#2019-01-2514:35Alex Miller (Clojure team)thereā€™s some stuff in CLJ-2112 (specs for specs) patch#2019-01-2514:41vemvthanks!#2019-01-2514:35Alex Miller (Clojure team)keep in mind that all of these answers will be different in spec2 :)#2019-01-2615:59borkdudehash-map canā€™t be instrumented and then called in CLJ but it can be in CLJS.
$ clj -A:test -m cljs.main -re node

ClojureScript 1.10.439
cljs.user=> (require '[speculative.core])
nil
cljs.user=> (require '[clojure.spec.test.alpha :as stest])
nil
cljs.user=> (stest/instrument `hash-map)
[cljs.core/hash-map]
cljs.user=> (apply hash-map [1])
repl:13
throw e__6573__auto__;
^

Error: Call to #'cljs.core/hash-map did not conform to spec.

$ clj -A:test

Clojure 1.10.0
user=> (require '[speculative.core])
nil
user=> (require '[clojure.spec.test.alpha :as stest])
nil
user=> (stest/instrument `hash-map)
[clojure.core/hash-map]
user=> (apply hash-map [1])
Execution error (StackOverflowError) at (REPL:1).
null
#2019-01-2617:29borkdudeI donā€™t understand why, because this has never been the case before with CLJ. With CLJS these are pretty common, because it has no direct linking.#2019-01-2618:46borkdudeThe cause is probably that spec-checking-fn calls with-instrument-disabled, which calls binding which expands in a call to hash-map, so thereā€™s a loop.#2019-01-2619:04borkdudeA possible solution would be to replace in the binding macro:
(push-thread-bindings (hash-map 
with
(push-thread-bindings (clojure.lang.PersistentHashMap/create ~(var-ize bindings)))
i.e. inline the hash-map call. Probably very low priority, but if thereā€™s interest, Iā€™ll make the JIRA ticket.
#2019-01-2621:07Alex Miller (Clojure team)Donā€™t want to depend on the class like that#2019-01-2621:08Alex Miller (Clojure team)If anything would be better to use RT/map or whatever #2019-01-2622:11borkdudeRT/map is tricky, because then I need into-array which is not defined yet#2019-01-2622:12borkdudeWhen I depend on a local version of spec.alpha (not spec-alpha2), I get:
Execution error (ClassNotFoundException) at java.net.URLClassLoader/findClass (URLClassLoader.java:382).
clojure.spec.alpha$explain_out
when I call an instrumented function with invalid args. No clue what this is.
#2019-01-2622:15borkdudeWhen I install it with mvn it works#2019-01-2622:21borkdudeit seems like spec.alpha is AOT-compiled?#2019-01-2701:49Alex Miller (Clojure team)Yes#2019-01-2701:49Alex Miller (Clojure team)But core.specs.alpha is not#2019-01-2809:55jeroenvandijk@borkdude maybe something ugly as this would work:
(let [hash-map0 (var-get #'hash-map)]
  (alter-var-root #'clojure.core/hash-map (constantly (fn [& x] 
    (with-redefs [clojure.core/hash-map hash-map0]
      ;; validation here?
      (apply clojure.core/hash-map x)
      )))))
#2019-01-2809:58borkdude@jeroenvandijk if youā€™re interested, I have a potential fix for hash-map not being able to be instrumented here: https://github.com/borkdude/speculative/issues/264#2019-01-2809:58borkdudeIā€™ll probably give up on it for now. CLJS has plenty of similar issues like this (instrumenting apply, seq, etc.)#2019-01-2809:59borkdudeIā€™m not sure what your solution accomplishes, but you could add it to the issue with some context#2019-01-2810:01jeroenvandijkFrom what I understand, hash-map calls hash-map itself when instrumented. So I thought of aliasing hash-map to the old version of hash-map during the validation itself. Just an idea, maybe it's shortsighted#2019-01-2810:03borkduderight. the spec-checking-fn replacing hash-map calls itself (because binding emits a call to hash-map), so it ends up in a loop.#2019-01-2810:19jeroenvandijkMaybe a complete clojure.core fix would be to have (stest/instrument x) to have x to be aliased to it's previous value so it cannot interfere with the instrumentation itself when x is used for the instrumentation.#2019-01-2810:20borkdudethat would be the more fundamental solution yes#2019-01-2810:21borkdudeit would also speed up core instrumentation probably#2019-01-2817:35buttergunsHi. I'm experimenting with Spec. I think I'm approaching it (incorrectly) from an OO background. Suppose I have a spec ::vehicle #{:car :truck :semi-truck :bicycle :motorbike}. It is part of a map (s/keys :req [::vehicle ::color]). At some time later in processing, I have a function that requires 2-wheeled vehicles, so I might have ::two-wheeled-vehicle (s/and ::car has-two-wheels?). My problem is I've already used the ::vehicle key for the more "generic" specification of a vehicle. I would need to create a new map with new keys (`(s/keys :req [::two-wheeled-vehicle ::color])`), and copy all the values over, in order to validate the stricter specification. Am I doing it wrong?#2019-01-2915:44jkrasnayIā€™m trying to use s/keys with a vector of keys, such that I can use the same vector of keys elsewhere:
(def my-keys [:emp/id :emp/name])
(s/def ::my-spec (s/keys :req my-keys))
ā€¦but s/keys complains. Is there any way to do this?
#2019-01-2915:51taylorthere are changes coming to spec to make this easier, but for now you gotta eval#2019-01-2915:45mpenet> (eval `(s/def ::my-spec (s/keys :req ~my-keys)))#2019-01-2915:49jkrasnayAh, cool. Thanks.#2019-01-3008:41Josip GracinHi! clojure.spec.test.alpha/check silently ignores symbols which are not "checkable". This seems counter-intuitive to me because I'd expect it to fail when I call check on a symbol and the symbol does not have an associated spec (e.g. it is misspelled).#2019-01-3008:41Josip Gracinthe definition of check looks like this:#2019-01-3008:41Josip Gracin(defn check ([] (check (checkable-syms))) ([sym-or-syms] (check sym-or-syms nil)) ([sym-or-syms opts] (->> (collectionize sym-or-syms) (filter (checkable-syms opts)) (pmap #(check-1 (sym->check-map %) opts)))))#2019-01-3008:42Josip Gracinit seems to me that the (filter (checkable-syms opts)) might better go to the no-arg variant#2019-01-3008:42Josip Gracindoes this make sense?#2019-01-3013:10Alex Miller (Clojure team)File a jira#2019-01-3013:11Josip Gracinok, tnx!#2019-01-3015:15mpingHi, is there any good way to xform specā€™s :problems into something that could make sense for a human over a json api?#2019-01-3016:07bbrinckI suppose it depends on how technical the user is, but you could potentially use Expound here. #2019-01-3016:10bbrinckThe default options might be too noisy (they include the specs), but this can be changed easily. You can also customize the messages in some cases. Let me know if you want help with configuring expound or just need some sample code for your use case. #2019-01-3016:15bbrinckPhrase is another option#2019-01-3016:36mpingtks, I basically want to convert the specā€™s :problems into something more tractable for a human#2019-01-3016:37mpingit seems that iā€™d have to change *explain-out* right?#2019-01-3016:38bbrinckYep, thatā€™s what Expound does :)#2019-01-3016:39bbrinckYou can just replace explain-out with the printer provided by Expound #2019-01-3016:39bbrinckhttps://github.com/bhb/expound#2019-01-3023:31mishais there a rationalization why aliased specs can't be overwritten in spec1?
(s/def :user/foo string?)
(s/def :user/bar :user/foo) ;;alias
(s/exercise :user/bar 1 {:user/bar #(s/gen #{"quux"})}) ;;=> (["" ""])
(s/exercise :user/bar 1 {:user/foo #(s/gen #{"quux"})}) ;;=> (["quux" "quux"])

(s/def :user/bar string?) ;;not alias
(s/exercise :user/bar 1 {:user/bar #(s/gen #{"quux"})}) ;;=> (["quux" "quux"])
Can I somehow decouple gen-override call site from the knowledge of spec (not) being an alias?
#2019-01-3023:37mishaSticking generator on the spec at s/def time works, but I'd like to avoid nailing generator to spec.#2019-01-3023:42mishaOverride for alias with custom generator works though:
(s/def :user/foo string?)
(s/def :user/bar (s/with-gen :user/foo #(s/gen #{"bar"})))
(s/exercise :user/bar 1)
=> (["bar" "bar"])
(s/exercise :user/bar 1 {:user/bar #(s/gen #{"quux"})})
=> (["quux" "quux"])
#2019-01-3023:43Alex Miller (Clojure team)itā€™s a bug#2019-01-3023:43Alex Miller (Clojure team)thereā€™s a ticket for it#2019-01-3023:43mishadoh#2019-01-3023:43mishashould I wait for spec2? or spec 1 gets things fixed too?#2019-01-3023:44Alex Miller (Clojure team)hard to say right now#2019-01-3023:44Alex Miller (Clojure team)I think I actually fixed it already in spec 2, but I havenā€™t tested it explicitly#2019-01-3023:45Alex Miller (Clojure team)but no release of spec 2 yet#2019-01-3023:45mishado you have any guestimate for closest spec2 release?#2019-01-3023:46Alex Miller (Clojure team)2019#2019-01-3023:46Alex Miller (Clojure team);)#2019-01-3023:47mishaopieop#2019-01-3100:04seancorfieldFWIW, from spec-alpha2, latest SHA:
user=> (require '[clojure.spec-alpha2 :as s])
nil
user=> (s/def :user/foo string?)
:user/foo
user=> (s/def :user/bar :user/foo) ;;alias
:user/bar
user=> (s/exercise :user/bar 1 {:user/bar #(s/gen #{"quux"})}) ;;=> (["" ""])
(["" ""])
user=> (s/exercise :user/bar 1 {:user/foo #(s/gen #{"quux"})}) ;;=> (["quux" "quux"])
Execution error (IllegalArgumentException) at clojure.spec-alpha2.protocols/eval173$fn$G (protocols.clj:11).
No implementation of method: :gen* of protocol: #'clojure.spec-alpha2.protocols/Spec found for class: clojure.lang.PersistentHashSet
user=> (s/def :user/bar string?) ;;not alias
:user/bar
user=> (s/exercise :user/bar 1 {:user/foo #(s/gen #{"quux"})}) ;;=> (["quux" "quux"])
(["" ""])
user=> 
#2019-01-3100:05seancorfieldand your custom generator example:
user=> (s/def :user/foo string?)
:user/foo
user=> (s/def :user/bar (s/with-gen :user/foo #(s/gen #{"bar"})))
:user/bar
user=> (s/exercise :user/bar 1)
Execution error (IllegalArgumentException) at clojure.spec-alpha2.protocols/eval173$fn$G (protocols.clj:11).
No implementation of method: :gen* of protocol: #'clojure.spec-alpha2.protocols/Spec found for class: clojure.lang.PersistentHashSet
user=> (s/exercise :user/bar 1 {:user/bar #(s/gen #{"quux"})})
Execution error (IllegalArgumentException) at clojure.spec-alpha2.protocols/eval173$fn$G (protocols.clj:11).
No implementation of method: :gen* of protocol: #'clojure.spec-alpha2.protocols/Spec found for class: clojure.lang.PersistentHashSet
user=> 
#2019-01-3101:27Alex Miller (Clojure team)I havenā€™t changed anything since we last talked#2019-01-3103:49seancorfieldThat was for @misha not you @alexmiller, since he asked how his code would behave on spec2 šŸ™‚#2019-01-3104:00Alex Miller (Clojure team)ah, well the thing still to do on aliases for you affects it :)#2019-01-3118:57buttergunsI'd like to substitute a generator for an "inner" spec while testing. i.e.
;; src file
(s/def ::zip pos-int?)
(s/def ::address (s/keys :req [::zip]))
(s/def ::person (s/keys :req [::address]))

;; test file
(gen/sample (s/gen ::person) 1)
=> {::address {::zip 1}}

;; pseudo code
(gen/sample (redefine-a-spec (s/gen ::person) ::zip #{10001 10002}))
=> {::address {::zip 10001}}
How can I do this?
#2019-01-3118:58taylors/gen takes an overrides map#2019-01-3118:58borkdude@mattmorten stest/check also takes an overrides map in case you need that#2019-01-3118:58taylorsomething like (s/gen ::person {::zip new-spec}) I think?#2019-01-3119:00buttergunsThat was even simpler than I hoped for. Thanks!#2019-01-3120:45buttergunsI have a related question. I have a coll-of ::animal, where ::animal is a multi-spec. How do I provide overrides to force the ::animal that is generated?#2019-01-3120:45butterguns
(s/def ::animal-type #{:dog :cat})
(s/def ::cat-attribute #{"very catty"})
(s/def ::dog-attribute #{"dog dog"})

(defmulti animal-type ::animal-type)
(defmethod animal-type :cat [_] (s/keys :req [::cat-attribute ::animal-type]))
(defmethod animal-type :dog [_] (s/keys :req [::dog-attribute ::animal-type]))
(s/def ::animal (s/multi-spec animal-type ::animal-type))

(s/def ::animals (s/coll-of ::animal))
(s/def ::petshop (s/keys :req [::animals]))

(gen/sample (s/gen ::petshop) 1)
=>(#:{:animals [#:{:cat-attribute "very catty", :animal-type :cat}
                #:{:dog-attribute "dog dog", :animal-type :dog}]})

;; How do I override s/gen to get a list of :cat's only? 
(gen/sample (s/gen ::petshop {??}) 1)
#2019-01-3120:47borkdudeNarrator: and this was harder than he hoped for.#2019-01-3120:49borkdudeso the spec for cat is (s/keys :req [::cat-attribute ::animal-type])?#2019-01-3120:50buttergunsyep#2019-01-3120:50borkdudeI donā€™t have experience with spec and multimethods, so Iā€™ll shut up now#2019-01-3120:51Alex Miller (Clojure team)name the spec returned by the defmethod and override it?#2019-01-3120:52Alex Miller (Clojure team)or youā€™re looking for a particular branch of the multimethod to be poked?#2019-01-3120:53buttergunsI just need cats!#2019-01-3120:53Alex Miller (Clojure team)override ::animal with the generator for the spec returned by (animal-type {::animal-type :cat}) ?#2019-01-3120:53borkdudeis multi-spec relevant here?#2019-01-3120:54buttergunsGenius#2019-01-3120:55butterguns(gen/sample (s/gen ::petshop {::animal #(s/gen (animal-type {::animal-type :cat}))}) 1)#2019-02-0109:50borkdudeI would like to have a better solution for this problem: https://github.com/borkdude/speculative/issues/276#2019-02-0111:12borkdudeIs there something like s/undef in spec? I guess not?#2019-02-0111:15eval2020IIRC thereā€™s a ticket for that#2019-02-0111:17borkdudeWhat I actually want is to define a an fdef that doesnā€™t get instrumented when calling (stest/instrument), but this may be too niche.#2019-02-0111:19eval2020I had this ticket in mind: https://dev.clojure.org/jira/browse/CLJ-2060#2019-02-0111:20borkdudeoooh, nice šŸ™‚#2019-02-0111:20borkdudethis is already in spec#2019-02-0111:21borkdudebut this doesnā€™t work for fdef?#2019-02-0111:22borkdude(s/def clojure.core/hash-map nil) works#2019-02-0111:23borkdudeseems to work also in CLJS#2019-02-0111:23eval2020would expect fdefā€™s to be put in the registry#2019-02-0111:25borkdudecool, I might be able to use this, thanks @eval2020!#2019-02-0201:12dronerunning into issues with circular references between specs (::a references ::b in itā€™s definition and vice versa). taking a step back to consider alternative ways to model the specs, but will spec2 support referring to undefined specs? As in:
(s/def ::a ::b)
(s/def ::b <spec>)
#2019-02-0203:34Alex Miller (Clojure team)Yes, and I have done some more work there to defer lookups more pervasively#2019-02-0202:38kennyI'd be nice to use s/assert in production but if you s/assert a spec that contains a fspec, it will require test.check to check the value. Is is possible to disable this functionality in production but still use the s/assert check?#2019-02-0203:29seancorfieldHow would it check the spec then @kenny?#2019-02-0203:30seancorfield(and you've got the same problem if you use s/valid? or s/conform, right?)#2019-02-0203:34kennyfn? is all I need in prod. #2019-02-0203:34kennyItā€™d be nice to get Spec error messages without the overhead of generation.#2019-02-0203:38seancorfield(s/valid? fn? thing)#2019-02-0203:39seancorfieldIf you want checks in production, put them in the code explicitly with s/valid? and/or s/conform.#2019-02-0203:40seancorfieldAsserts in production are a code smell, IMO, since they throw an Error, not an Exception so you're basically going to "kill" your request/process.#2019-02-0203:45seancorfieldIf you want "Spec error messages" then you don't really want s/assert -- you want explicit code to check validity and call explain or something.#2019-02-0415:12daniel-tcgplayerHey everyone, I'm experiencing something odd trying to implement spec into our project for the first time. I'm using fdef to define my function's spec, however it's not validating my arguments. I'm using clojure 1.10. Here's the code:
(defn inc-num [x]
  (inc x))

(s/fdef inc-num
  :args (s/cat :x number?)
  :ret number?)
Calling this function (inc-num "a") throws a ClassCastException instead of the spec error. Any ideas?
#2019-02-0415:14taylordid you s/instrument the function too?#2019-02-0415:15taylorthat's what enables the :args checking at runtime#2019-02-0415:16daniel-tcgplayerI did not! Let me go ahead and do this, maybe I got ahead of myself in the documentation šŸ™‚#2019-02-0415:16borkdudeyes, writing an fdef does not turn on instrumentation automatically. also note when you re-define the fdef, you have to call instrument again to make it effective#2019-02-0415:18daniel-tcgplayerI see. So for all functions that I'm relaying on fdef to spec, I must also instrument those functions#2019-02-0415:19borkdudeyes#2019-02-0415:19borkdudeyou can instrument all function at once by calling (stest/instrument)#2019-02-0415:19daniel-tcgplayerPer namespace?#2019-02-0415:19borkdudeif youā€™re using component, you might want to hook it up to the start/stop cycle. no, globally#2019-02-0415:20tayloryep that'll instrument everything that's been loaded, across all namespaces. there's also unstrument to do the opposite#2019-02-0415:21daniel-tcgplayerAwesome! Thanks everyone for the quick feedback, I'm movin' again. I'll go through the documentation more thoroughly#2019-02-0415:22taylor@U6TUZTAAF I wrote some more function spec examples here too: https://blog.taylorwood.io/2017/10/15/fspec.html#2019-02-0415:25borkdudeIā€™ve never used fspec. does spec really check the arity of such a function when you call a higher order function whose function argument is specā€™ed with it?#2019-02-0415:27taylor@U04V15CAJ fdef uses fspec internally, so there's not much difference AFAIK. when you spec a function that takes another function as an argument, spec invokes the passed function ~20 times ā€” a kind of mini-check ā€” to see if it conforms#2019-02-0415:27borkdudespec invokes then function when?#2019-02-0415:27taylorwhenever you call the higher-order function, if it's instrumented#2019-02-0415:28borkdudeit doesnā€™t just call it when itā€™s called normally?#2019-02-0415:29taylornot sure I understand the question#2019-02-0415:29taylor
(defn f [g] (g 1))
(s/fdef f :args (s/cat :g (s/fspec :args (s/tuple int?))))
(st/instrument `f)
(f #(doto % prn inc))
-1
0
-2
...
59
-69770
-1165
1
=> 1
#2019-02-0415:31taylorso whenever you call f (instrumented), spec is going to do a lil mini-check of the function you pass to f to see if it conforms to the fspec, and if it does then of course f will call g again#2019-02-0415:31borkdudewhy is that sane behavior? I can see this being useful when doing generative testing#2019-02-0415:33borkdudewutā€¦
$ clj
Clojure 1.10.0
user=> (require '[clojure.spec.alpha :as s])
nil
user=> (require '[clojure.spec.test.alpha :as stest])
nil
user=> (s/fdef clojure.core/keep :args (s/cat :f ::keep-fn :coll seqable?))
clojure.core/keep
user=> (s/def ::keep-fn (s/fspec :args (s/cat :x any?) :ret any?))
:user/keep-fn
user=> (stest/instrument `keep)
[clojure.core/keep]
user=> (keep inc [1 2 3])
Execution error (FileNotFoundException) at user/eval144 (REPL:1).
Could not locate clojure/test/check/generators__init.class, clojure/test/check/generators.clj or clojure/test/check/generators.cljc on classpath.
user=>
#2019-02-0415:34taylorI think you need test.check on your classpath#2019-02-0415:34borkdudeyeah, I know why this happens, Iā€™m just surprised that it suddenly wants to do generative testing while it can just conform while running, like normal functions#2019-02-0415:35taylorthere's no way to check an fspec without invoking the function though#2019-02-0415:36borkdudeoh, now I see, because they are not replaced by an instrumented versionā€¦ right.#2019-02-0415:36borkdudebut still, itā€™s a little weird this.#2019-02-0415:37borkdudeyou would have to be very wary of using side effects in fspecā€™ed functions#2019-02-0415:38borkdudeI havenā€™t used fspecs in speculative yet and this may be a reason Iā€™m not going to..#2019-02-0415:42taylorI think the problem is more about speccing higher-order functions in general, than it is purely about fspec, b/c fdef is just a shorthand for s/def + s/fspec. another option is to just use fn? or ifn? when you're speccing HOFs, that way spec doesn't mini-check them#2019-02-0415:42borkdudeyes, thatā€™s what weā€™re doing now#2019-02-0415:43borkdudein most cases itā€™s sufficient, although it would maybe nice to be able to speak about function arity#2019-02-0415:47taylorone case where I can see value in fspecing args to HOFs is when you check the HOF, spec will pass it a dummy function that expects the right :args and returns values generated according to the :ret spec#2019-02-0415:48borkdudethat would be useful, but I want this to be pluggable. so my fdef would not use fspecs, but I do want to plug them instead of ifn? during generative testing.#2019-02-0415:49borkdudeyou can do that by overriding spec generators#2019-02-0415:50borkdudebut overriding ifn? would be a bit too global, since the args and ret could use ifn? with different properties#2019-02-0415:50borkdudeso then you would have to give the argument list a named spec#2019-02-0415:50borkdudewhich I what I do sometimes#2019-02-0514:55dronechiming in that the generator requirement for HOFs also does not seem sane to me. should be able to instrument without assumption that youā€™re using generative testing#2019-02-0514:55borkdudeIā€™d say so too. @alexmiller maybe should be an option to turn this off on spec2?#2019-02-0515:02Alex Miller (Clojure team)itā€™s a topic weā€™ll revisit#2019-02-0420:16hmaurerIs there a way quick way, without using an external lib, to generate a random value from a spec while limiting the max depth when the value is a map?#2019-02-0420:17hmaurere.g. I have nested s/keys specs and I would like to generate a random value for the spec but limit the depth of the data structure#2019-02-0420:17hmaurerI know there are some libs on github that do this, but I am wondering if there is a short way to do it out of the box with specs
#2019-02-0420:18tayloris it a recursive spec? if so, there's a *recursion-limit* binding you can use during generation. the other thing that comes to mind is passing in a small/fixed size arg to the sample function#2019-02-0420:21hmaurer@taylor yes itā€™s recursive, although not directly. The spec is modelling an AST#2019-02-0420:23taylornot sure if this would have the effect you want, but maybe https://gist.github.com/taylorwood/232129ccd3cb809281fea591d46f1b8a#file-infix-spec-clj-L58#2019-02-0420:23hmaurer@taylor Iā€™ll try that, ty! Also I am trying to generate random values for this spec:
(s/def :elogic/term (s/and (s/keys :req [:elogic.term/type])
                           (s/multi-spec elogic-term-type :elogic.term/type)))
but I am getting ā€œCouldnā€™t satisfy such-that predicate after 100 tries.ā€
#2019-02-0420:23hmaurerany idea how I can make it work?#2019-02-0420:25tayloryeah that's b/c the generator for your s/and is only based on the first inner s/keys spec, and the generated values from that don't conform to the second multi-spec#2019-02-0420:25hmaurer@taylor ah. Any way around this?#2019-02-0420:25tayloryou can wrap that s/and with s/with-gen to specify a custom generator, or override the generator elsewhere#2019-02-0420:26taylorsome of the spec functions take an overrides map for this#2019-02-0421:01hmaureris there a way to define a spec on a string without seqā€™ing it first?#2019-02-0421:01hmaurera regex spec#2019-02-0421:04Alex Miller (Clojure team)no#2019-02-0421:08hmaurer@alexmiller is it a generally bad idea to use specs to parse strings?#2019-02-0421:08Alex Miller (Clojure team)itā€™s not recommended, but I literally canā€™t stop you :)#2019-02-0421:08Alex Miller (Clojure team)itā€™s not recommended, but I literally canā€™t stop you :)#2019-02-0421:09hmaureršŸ˜„ what is the recommendation against it?#2019-02-0421:31souenzzoDo your own spec-inspired (and maybe backended by spec) library to specify string?!?!#2019-02-0421:33borkdudeI recently found this lib: https://github.com/miner/strgen Itā€™s cool, but maybe comes with limitations#2019-02-0421:33hmaurer@U2J4FRT2T not a bad idea šŸ˜„ I would need to first learn more about how spec works though#2019-02-0421:33borkdudeThe nice thing about using spec for strings is that you get nice error messages and can generate examples. For better error messages one could also use a parser combinator or parser generator.#2019-02-0421:35hmaurer@U04V15CAJ being able to generate examples is definitly useful#2019-02-0421:38Alex Miller (Clojure team)sorry, was on a call#2019-02-0421:38Alex Miller (Clojure team)the point is that spec regex were designed to spec sequential collections#2019-02-0421:39Alex Miller (Clojure team)strings are not sequential collections and the performance of the regex specs on them might be bad#2019-02-0421:39Alex Miller (Clojure team)and there are quite good options for parsing strings (namely, the regex built into the JDK and exposed by Clojure)#2019-02-0421:40Alex Miller (Clojure team)test.chuck has a function that generates strings from regexes https://github.com/gfredericks/test.chuck if you need that#2019-02-0422:32seancorfieldFWIW, we use test.chucks string regex generator quite a bit -- we're very happy with that, combined with regular string regex. I would not recommend trying to use spec's sequence regexes on a string.#2019-02-0422:53borkdude@U04V70XH6 good to hear. how does it compare to minerā€™s lib? I see his name in the README of test.chuck. Was his lib merged into test.chuck?#2019-02-0422:53borkdude@U052852ES I think this thread is worth reading since you asked me about a related problem youā€™re solving with spec#2019-02-0423:12souenzzoWe can compile specs into regexp at macro time šŸ‘€#2019-02-0500:35seancorfield@U04V15CAJ No idea. First I've heard of Steve Miner's regex lib.#2019-02-0507:55thomas@U04V15CAJ I have seen it and I will have a look again, thank you.#2019-02-0508:27borkdudeI meant the test.chuck regex thing#2019-02-0509:25Vincent CantinWhere do clojurists usually place their fdef? - In the same namespace of their functions? - In the same namespace with ā€œ.specā€ prepended? - In a separate, global /src/spec/ namespace tree? - other?#2019-02-0511:02n2oI usually put it directly below the function definition, but i am not really happy with it to be honest.#2019-02-0511:16Vincent CantinWhy arenā€™t you fully satisfied with it?#2019-02-0512:04n2othe source code feels a bit messy. Simple clean functions, then a new definition below it just for specs... Just a personal impression. Some kind of choppy in my opinion#2019-02-0512:05n2oI tried to put them into a separate namespace but as you have to instrument them to really use the specs, the specs are like to be forgotten. So this does not seem to be a good solution#2019-02-0512:05n2oI then tried to put them below my source code, but in the same namespace (at the bottom of the file). I almost forgot all of them and was wondering, that I wrote specs for those functions when I scrolled down the file after some months. So this is also not a good solution#2019-02-0512:07n2oCurrently I put them directly together, so I don't forget them
(defn foo [x y] ,,,)
(s/fdef foo
  :args (s/cat :x number? :y pos-int?)) 
#2019-02-0517:47seancorfieldIf I want the code to be usable without spec (i.e., on Clojure 1.8), I put fdefs in a separate namespace (see clojure.java.jdbc for example). Otherwise I put the fdef immediately above the function it refers to.#2019-02-0609:10n2oOk, in my case I do not need to be downwards-compatible. But yes, then you don't have a different choice.#2019-02-0509:34borkdude@vincent.cantin The answer is: up to you. There is no ā€œbest wayā€ regarding this. Personally, I co-locate it with the defn.#2019-02-0513:53abdullahibraHello guys#2019-02-0513:53abdullahibrahow could i check the values of map only using clojure.spec, i know how can i do this for map keys names + values, but i'm in situation which i don't know what key names are but i know what types of values should be ?#2019-02-0513:53borkdudee.g. for checking string values: (s/map-of any? string?)#2019-02-0513:53abdullahibrashould handle the values using s/coll-of#2019-02-0513:54abdullahibraah great#2019-02-0513:54abdullahibra@borkdude thanks#2019-02-0513:56abdullahibrais clojure.spec good alternative to type system in language like Haskell, ocaml ?#2019-02-0520:25hmaurermost definitely not#2019-02-0520:25hmaureršŸ˜„#2019-02-0520:25hmaureritā€™s very different and by no mean a substitute for static typing (as much as I like spec)#2019-02-0513:57borkdudeit has different applications, everythingā€™s runtime, so you get no compile time guarantees, macros being the exception#2019-02-0513:58borkdudebut itā€™s also more flexible than a type system. different trade offs.#2019-02-0514:06manutter51Also spec is still evolving, there are going to be changes, especially concerning how maps are specā€™ed.#2019-02-0514:15victorbfor more details about the changes: https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha#2019-02-0514:22borkdudewould it make sense to co-develop the CLJS version of spec2 in the same repo and pull it outside CLJS itself, so CLJS never is behind on spec?#2019-02-0514:22borkdudelike test.check#2019-02-0514:26borkdudeam I right that you canā€™t use anonymous functions in specs anymore in spec2 so they would always have to be defined top-level? what about with-gen?#2019-02-0514:26borkdudethanks for the link @victorbjelkholm429, useful information#2019-02-0514:29Alex Miller (Clojure team)I do not think you should be tracking spec 2 changes in cljs yet - still in flux#2019-02-0514:30Alex Miller (Clojure team)you can use anonymous functions#2019-02-0514:30Alex Miller (Clojure team)what about with-gen?#2019-02-0514:31borkdudecan?#2019-02-0514:31Alex Miller (Clojure team)?#2019-02-0514:32borkdudeIā€™m reading about ā€œsymbolic specsā€, it seems that you cannot use anonymous functions in there?#2019-02-0514:32Alex Miller (Clojure team)you can#2019-02-0514:32borkdude
Symbolic specs consist only of:

Spec forms (lists/seqs), with spec op in function position, composed of other symbolic specs
Qualified keywords (names that can be looked up in the registry)
Qualified symbols (predicate function references)
Sets of constant values (an enumeration)
#2019-02-0514:33Alex Miller (Clojure team)spec op can be fn#2019-02-0514:33Alex Miller (Clojure team)so those are valid spec forms#2019-02-0514:33borkdudeah ok. I was confused, because of feedback you had earlier, I thought it wasnā€™t allowed anymore, but then I misunderstood#2019-02-0514:34Alex Miller (Clojure team)thereā€™s a picture at http://insideclojure.org/2019/02/02/journal/ if that helps#2019-02-0519:54taylorI really like the music notes at the end of each post! There's some good lesser-known stuff on Zeppelin's In Through the Out Door too šŸ™‚ are you a fan of The Meters?#2019-02-0519:54Alex Miller (Clojure team)yep and yep#2019-02-0519:56Alex Miller (Clojure team)been listening to the first Meters album a lot lately actually :)#2019-02-0514:34Alex Miller (Clojure team)it has a note about fn in the middle there#2019-02-0514:35Alex Miller (Clojure team)if you want to use anonymous functions at the top level with the api functions, you need to wrap them in s/spec#2019-02-0514:37borkdudealright. I donā€™t know how much in flux things are, but if you want me to give it a try again with speculative, let me know.#2019-02-0514:38Alex Miller (Clojure team)(s/valid? (s/spec #(> % 5)) 10)#2019-02-0514:39Alex Miller (Clojure team)Iā€™ve got at least one thing to fix still from Seanā€™s feedback#2019-02-0514:39Alex Miller (Clojure team)and weā€™re working through best direction for some of the things you saw last time still#2019-02-0514:40Alex Miller (Clojure team)so probably not worth doing anything yet#2019-02-0523:05dronemaybe using spec in an unintended way, but I ran into a situation where there were two spec forms in an s/and, the result of the first conformed the input and the second spec form in the s/and was a predicate expecting the original input, not the modified/conformed form. it feels like validation should be independent of conforming#2019-02-0523:06droneand I donā€™t understand the purpose of conforming. is it intended to be like schemaā€™s coercion? or is it to produce some intermediate form useful for spec-checking?#2019-02-0523:46Alex Miller (Clojure team)Re and, we have long considered having both ā€œflowingā€ and nonflowing variants of and#2019-02-0523:46Alex Miller (Clojure team)And that might still happen in spec 2#2019-02-0523:47Alex Miller (Clojure team)The purpose of confirming is not coercion - itā€™s to tell you which choices were made during validation for optional specs or components#2019-02-0523:48Alex Miller (Clojure team)That is, why was it valid?#2019-02-0523:58droneSounds good. The flowing was definitely surprising.#2019-02-0523:58droneThanks, got it#2019-02-0523:58droneA lot of design decisions #2019-02-0603:22flyboarderSo this is a spec error I ran into today and Im here thinking there must be a better way to get to the root of the problem#2019-02-0603:22flyboarder#2019-02-0603:34taylorwhatā€™s on line 28 in bootstrap/feature.clj? is that your project?#2019-02-0603:36flyboarder(defn- feature-dispatch [[k v]] k)#2019-02-0603:37tayloritā€™s probably the destructuring of the first arg, maybe a keyword is being passed as the first arg#2019-02-0603:37flyboarder^ I was able to solve this the problem was (spec/explain-data ::feature-gate feature) when I needed (spec/explain-data ::feature-gate [feature])#2019-02-0603:37flyboarder@taylor ^#2019-02-0613:02borkdude@alexmiller I updated this comment just now: https://github.com/borkdude/speculative/issues/124#issuecomment-460944994#2019-02-0620:27borkdude@alexmiller do you plan to support this for spec-alpha2 as well? https://dev.clojure.org/jira/browse/CLJ-2060 I noticed it does not yet work in CLJS.#2019-02-0620:29Alex Miller (Clojure team)should work, havenā€™t looked at why yet, but wasnā€™t intentionally broken#2019-02-0620:29borkdudehttps://dev.clojure.org/jira/browse/CLJS-3049#2019-02-0621:19seancorfieldFYI, I just ran out entire test suite against the latest spec2 and there's just one failure (for s/valid? on a large data structure for a very complex, nested spec) -- which was the unexplained failure I had before. The generator problems I mentioned before are fixed by the latest work.#2019-02-0621:28seancorfield@alexmiller What in spec (or spec2) causes this predicate to be used? (or (nil? %) (sequential? %))#2019-02-0621:29seancorfield(it's not in our code so I assume it's what underlies s/cat or something?)#2019-02-0621:48Alex Miller (Clojure team)I think thatā€™s checked for all regex specs#2019-02-0621:48Alex Miller (Clojure team)yeah#2019-02-0621:48Alex Miller (Clojure team)I assume you see that fail?#2019-02-0621:51seancorfieldI've tracked it down a bit further...
(def s [ ... sequence of hash maps ...])
(count s) ;;=> 6
(s/explain ::ba-decline (take 3 s)) ;;=> Success!
(s/explain ::ba-decline (drop 3 s)) ;;=> Success!
(s/explain (s/cat :one ::ba-decline :two ::ba-decline) s)
;; fails, claiming that the first element of s does not satify: (or (nil? %) (sequential? %)) in: [0] at: [:one]
#2019-02-0621:52seancorfield(this works with spec1)#2019-02-0621:53seancorfieldAm I missing something obvious about combining regex specs?#2019-02-0621:53seancorfieldThe ::ba-decline spec is an s/cat of three items.#2019-02-0621:54seancorfieldI'll see if I can create a minimal example that fails, based on this (unfortunately there are a lot of complex specs behind this).#2019-02-0622:03seancorfield@alexmiller
(s/def ::x string?)
(s/def ::y int?)
(s/def ::z keyword?)
(s/def ::a (s/keys :req-un [::x]))
(s/def ::b (s/keys :req-un [::y]))
(s/def ::c (s/keys :req-un [::z]))
(s/def ::abc (s/cat :a ::a :b ::b :c ::c))
(def a {:x "x"})
(def b {:y 42})
(def c {:z :bar})
(s/explain ::abc [a b c])
(s/explain (s/cat :one ::abc :two ::abc) [a b c a b c])
works on spec1, fails on spec2
#2019-02-0622:03seancorfieldon spec2 {:x "x"} - failed: (or (nil? %) (sequential? %)) in: [0] at: [:one]#2019-02-0623:07Alex Miller (Clojure team)
user=> (s/explain (s/cat :one ::abc :two ::abc) [[a b c] [a b c]])
Success!
certainly is suspicious :)
#2019-02-0623:08seancorfieldIt's almost like regex specs don't unroll anymore! :rolling_on_the_floor_laughing:#2019-02-0623:14Alex Miller (Clojure team)I may have actually introduced that by introducing the delayed resolution of aliased specs#2019-02-0623:17Alex Miller (Clojure team)basically regex specs in the registry are shielded by the keyword (which is not a regex spec), so the code doesnā€™t believe they can be combined#2019-02-0623:18Alex Miller (Clojure team)so this may fight with the last changes - I canā€™t remember now if that was from forward references or from something about gens?#2019-02-0623:18seancorfieldThis was broken before you fixed either of those I believe. Certainly broken before the gensub fix.#2019-02-0623:19Alex Miller (Clojure team)well good to know then :)#2019-02-0623:19seancorfieldAlthough, with a quick clj -Sdeps ... it's enough to test that repro case against any version of spec2 šŸ™‚#2019-02-0622:16Alex Miller (Clojure team)Iā€™ll take a look, seems buggy to me#2019-02-0622:17Alex Miller (Clojure team)There is a lot gnarly code in that form/object transition. I could easily have broken something subtle#2019-02-0622:17Alex Miller (Clojure team)Thanks for the repro#2019-02-0622:18seancorfieldI was lucky that the first simple repro I tried still broke šŸ™‚#2019-02-0622:19Alex Miller (Clojure team)Hey, thatā€™s a good sign#2019-02-0622:21seancorfieldAt this point, it really is a pretty minimal set of changes in our code to go from spec1 to spec2. It's frustrating that we can no longer use comp and partial to construct predicates and have to resort to #(..) or (fn [x] ..) but, fortunately, that didn't affect much code.#2019-02-0622:23seancorfieldWe have a few places we're having to use s/spec* now so the "helper macros" mentioned in your latest journal will help clean that up when they drop in GitHub.#2019-02-0622:54Alex Miller (Clojure team)yeah, that has further solidified today and I think will be very useful#2019-02-0622:31borkdudeI still have at least 3 bugs with spec alpha 2 (see issue link above)#2019-02-0622:36seancorfieldInteresting. I never knew you could define a spec to nil to make it go away...#2019-02-0622:37borkdudeItā€™s a fairly recent addition.#2019-02-0622:52borkdudeMake that 2. The (s/def spec nil) was a mistake on my part, I had to call unstrument first (which failed because ::s/invalid is not a valid any? which I ran into)#2019-02-0622:52Alex Miller (Clojure team)oh good :)#2019-02-0622:52borkdudeundefining still works:
$ clj -A:test
Clojure 1.10.0
user=>  (require '[clojure.spec-alpha2 :as s])
nil
user=> (require '[clojure.spec-alpha2.test :as stest])
nil
user=> (defn foo [x] x)
#'user/foo
user=> (s/fdef foo :args (s/cat :x number?) :ret number?)
user/foo
user=> (stest/instrument `foo)
[user/foo]
user=> (foo "a")
Execution error - invalid arguments to user/foo at (test.clj:129).
"a" - failed: number? at: [:x]
user=> (stest/unstrument)
[user/foo]
user=> (s/def foo nil)
user/foo
user=> (stest/instrument `foo)
[]
#2019-02-0712:30borkdudeI thought I had to make upgrading existing spec libraries less painful is to have something like a reader conditional that can dispatch on the version of spec. E.g.:
#?(:spec2 (:require [clojure.spec-alpha2] :as s) :default (:require [clojure.spec.alpha :as s]))
#?(:spec2 (s/def ā€¦) :default (s/def ā€¦))
This way a library can maintain compatibility with both versions of spec
#2019-02-0716:23dominicm
(try
  (require '[clojure.spec-alpha2 :as s])
  (catch Exception e
    (require '[clojure.spec.alpha :as s])))
No? šŸ˜„
#2019-02-0716:23dominicmdoesn't work for cljs though#2019-02-0716:23borkdudethatā€™s the big deal, CLJS#2019-02-0716:24borkdudewe could use a goog-define for it in CLJS, but then still you cannot do anything on the namespace declaration level#2019-02-0716:25dominicmit's a shame that it doesn't work in cljs given the work that was done to make require work there.#2019-02-0716:25borkdudethatā€™s only for self-hosted AFAIK#2019-02-0716:25borkdudeand REPL usage#2019-02-0716:26borkdudeanyway, it would be nice to have a way have libraries support both versions. I bet it will be 2 branches and versions with ā€œ-spec2ā€ suffixes for a while#2019-02-0716:27dominicmyeah, I think you're right#2019-02-0716:28dominicmyou could use a macro, and perform the require voodoo at the clojure level perhaps šŸ¤”#2019-02-0716:28dominicmbut still not great#2019-02-0716:29borkdudeoh right, I think I saw someone ā€œabuseā€ a data reader for that recently. Maybe it was @U050PJ2EU?#2019-02-0716:29dominicmit would work in shadow-cljs, which has the ability to specify custom :keywords which run#2019-02-0713:49borkdudewill spec-alpha2 be released at the same time as clojure 1.11 as one bundle?#2019-02-0713:54Alex Miller (Clojure team)unknown#2019-02-0713:59borkdudeI have yet to test the latest SHA. Will update you, probably in the weekend.#2019-02-0714:05Alex Miller (Clojure team)no worries#2019-02-0714:05Alex Miller (Clojure team)good catches#2019-02-0718:55joefromctcan anyone point out something i may have missedā€¦ if i try to generate from a s/keys spec un-req i get all blank mapsā€¦ but if i use the same with req i can generate data. Do you need a custom generator for anything un-req ?#2019-02-0718:56Alex Miller (Clojure team)do you mean req-un ?#2019-02-0719:03seancorfield@joefromct
user=> (require '[clojure.spec.alpha :as s])
nil
user=> (s/def ::a int?)
:user/a
user=> (s/def ::b string?)
:user/b
user=> (s/exercise (s/keys :req-un [::a ::b]))
([{:a -1, :b ""} {:a -1, :b ""}] [{:a 0, :b ""} {:a 0, :b ""}] [{:a -1, :b "75"} {:a -1, :b "75"}] [{:a 0, :b "zW7"} {:a 0, :b "zW7"}] [{:a -3, :b ""} {:a -3, :b ""}] [{:a -2, :b "Hlf"} {:a -2, :b "Hlf"}] [{:a 11, :b "M8r"} {:a 11, :b "M8r"}] [{:a -4, :b "K9Dn"} {:a -4, :b "K9Dn"}] [{:a 44, :b "o2rAl"} {:a 44, :b "o2rAl"}] [{:a -4, :b "SzHY1pTSN"} {:a -4, :b "SzHY1pTSN"}])
user=> 
#2019-02-0719:03joefromctha, yup. that was it. very funny.#2019-02-0719:06joefromctattention to detail is important. thanks.
clojure
(s/def :my-test/name (s/and string? #(not= "" %)))
(s/def :my-test/developer (s/keys :req [:my-test/name ]))
(s/exercise :my-test/developer 2)
([#:my-test{:name "W"} #:my-test{:name "W"}] [#:my-test{:name "7"} #:my-test{:name "7"}])

;; d'oh `un-req` not a thing.  
(s/def :my-test/developer (s/keys :un-req [:my-test/name ]))
(s/exercise :my-test/developer 2)
([{} {}] [{} {}])
#2019-02-0719:07seancorfieldI'm a bit surprised s/keys doesn't complain that all its known options are missing
user=> (s/exercise (s/keys))
([{} {}] [{} {}] [{} {}] [{} {}] [{} {}] [{} {}] [{} {}] [{} {}] [{} {}] [{} {}])
user=> 
#2019-02-0719:09Alex Miller (Clojure team)s/keys is a valid spec that will validate all registered keys in the map#2019-02-0719:09joefromctyeah i guess i did come across in the guide the mention of (s/keys) was valid and i guess the rule to fnā€™s are ā€œextra keys no-big-dealā€#2019-02-0719:09Alex Miller (Clojure team)but of course it doesnā€™t know how to gen#2019-02-0719:10Alex Miller (Clojure team)just use this mnemonic ā€¦ ā€œI req-un the map should have a key like thisā€#2019-02-0719:59jimbobā€¦.nice#2019-02-0720:00jimbobwhats the idiomatic way to mandate presence of keys? related: whats also the idiomatic way to mandate the lack of extraneous / undefined keys?#2019-02-0720:01jimbobi suppose for spec you can do something like (s/and (contains % #{mandated-keys})#2019-02-0720:02borkdudemandate the presence of keys: req-un not allow extra keys: this is not according to the philosophy of spec/Clojure I think#2019-02-0720:02jimbobinteresting. thanks#2019-02-0720:03taylorthere are some code snippets around for disallowing "extra" keys though @ben.borders even though it's counter to the design philosophy#2019-02-0720:03borkdudeyou can always hack around the design philosophy of Clojure šŸ˜›#2019-02-0720:04jimbobright, i figured.. seems like i should probably try to allign more with the philosophy#2019-02-0720:04jimbobiā€™m sure there are valid reasons for that.#2019-02-0720:04jimbobwould just like to read about it i suppose#2019-02-0720:04Alex Miller (Clojure team)Rich did a whole talk about it#2019-02-0720:05Alex Miller (Clojure team)well, this covers a lot of other stuff, but there is a section in here: https://github.com/matthiasn/talk-transcripts/blob/master/Hickey_Rich/Spec_ulation.md#2019-02-0720:06Alex Miller (Clojure team)grep for ā€œcode for growthā€#2019-02-0720:06jimbobthank you#2019-02-0720:06taylorI think one of the metaphors is "if you're expecting a truck to deliver your TV, you shouldn't care what else is on the truck" or something like that#2019-02-0720:06borkdudeThere have been some conversations about what spec is and isnā€™t suited for. E.g. it isnā€™t designed for coercion or ā€œclosed worldā€ assumptions (not allowing extra keys), e.g. using it as API boundary protection#2019-02-0720:07Alex Miller (Clojure team)I think the greater point is actually that you shouldnā€™t design your systems that way, and if you donā€™t, spec will mesh well :)#2019-02-0720:08jimbobright.. we donā€™t now.. we allow our maps to have whatever values they have, and we grab the ones we care about#2019-02-0720:08jimbobwhich tends to be most or all of them.#2019-02-0720:08borkdudewell, sometimes I donā€™t want to have crap in my jsonb field, but thereā€™s always select-keys, etc.#2019-02-0720:08jimbobright.#2019-02-0720:08Alex Miller (Clojure team)and you might be interested in the new stuff coming in spec 2 around that https://github.com/matthiasn/talk-transcripts/blob/master/Hickey_Rich/MaybeNot.md#2019-02-0720:55seancorfield> I think the greater point is actually that you shouldnā€™t design your systems that way, and if you donā€™t, spec will mesh well :) It's fine for a (REST) API to be passed a bunch of parameters it doesn't care about. It can just ignore them. Then spec works well for parameter validation.#2019-02-1013:14Jp SoaresI started to use REST API with compojure-api 2.0.0 that can coerce data with spec and generate swagger docs from it. I thought it would be the trend for specs, but reading this conversation maybe not. It's better to use compojure-api with schema to follow the clojure philosophy in REST?#2019-02-1018:24seancorfieldI don't understand your question : what do you mean "clojure philosophy in REST"?#2019-02-0720:57seancorfield> well, sometimes I donā€™t want to have crap in my jsonb field, but thereā€™s always select-keys, etc. And you can pull the "known" keys out of a spec fairly easily (usually!) so that you can narrow your set of fields down to just what's in the spec (we do this around database insertion where we have a spec for the row representation of data).#2019-02-0721:00joefromcthow would a spec look for the output (https://github.com/clojure/clojure/blob/master/src/clj/clojure/xml.clj#L78) from clojure.xml\parse look? Below is what iā€™m guessing at so far, but the ::content references ::element and vice versaā€¦ not sure how to handle this. (Iā€™m not worried about namespaces at the moment): I think its tricky for me because itā€™s a hierarchy.
(s/def ::tag keyword?)
(s/def ::attrs (s/map-of keyword? string?
                         :conform-keys true))

(s/def ::content-strings (s/coll-of string?))
(s/def ::content-maps    (s/coll-of ::element?))

(s/def ::content
  (s/or :content-map ::content-maps 
        :content-string ::content-strings )) 

(s/def ::element (s/keys :req-un [::tag]
                         :opt-un [::content ::attrs]))
#2019-02-0723:45butterguns(gen/sample (s/gen string?) 10000)#2019-02-0723:45butterguns...only gives me alphanumeric, for 10,000 iterations. Why is that?#2019-02-0723:46buttergunsNo symbols / unicode#2019-02-0723:47seancorfieldThat's probably all the string? generator is coded to produce.#2019-02-0723:48seancorfieldtest.chuck has a regex string generator that can produce a much wider range of character values, if you need to test that.#2019-02-0723:48buttergunsHmmm. I was trying to follow this blog post: https://lispcast.com/testing-stateful-and-concurrent-systems-using-test-check/#2019-02-0723:48seancorfieldFor example, here are some generated "email" strings, using that generator:
(["脀@Z.Gp" "脀@Z.Gp"] ["ņ«‘Ÿ@OG.lr" "ņ«‘Ÿ@OG.lr"] ["\"ņ’…Ø\"@z.DtS.K.Qpu" "\"ņ’…Ø\"@z.DtS.K.Qpu"] ["ńœŖń•ž.ņ²ØņŒ²žņ‘ ”󒓶@2N8.SuYn.FY.zBf" "ńœŖń•ž.ņ²ØņŒ²žņ‘ ”󒓶@2N8.SuYn.FY.zBf"] ["ńˆ±¢ó”®¢ń‘¬§ņ¹’ ōŽ²±@3.QBB" "ńˆ±¢ó”®¢ń‘¬§ņ¹’ ōŽ²±@3.QBB"] ["šø£„ńØ®¤ņ£ä±ó„¼­.ņ‹¾.ņž˜®.󦑁š£©·ó½‡±š–øƒ.ģ‘†óŽ˜…ń®“¶ōŒ¹›š–„ƒńŒ‹¶.š•ŗ»óš£ń‰ˆ€ó˜…™ņ”“Žņ¢ž™@[3.61.526.313]" "šø£„ńØ®¤ņ£ä±ó„¼­.ņ‹¾.ņž˜®.󦑁š£©·ó½‡±š–øƒ.ģ‘†óŽ˜…ń®“¶ōŒ¹›š–„ƒńŒ‹¶.š•ŗ»óš£ń‰ˆ€ó˜…™ņ”“Žņ¢ž™@[3.61.526.313]"] ["\"ń¹­ƒš°„–šæŖµņ™‡‘\"@[78.35.80.6]" "\"ń¹­ƒš°„–šæŖµņ™‡‘\"@[78.35.80.6]"] ["\"ń‘™¬ņ؍øńž»\"@[00.5.0.8]" "\"ń‘™¬ņ؍øńž»\"@[00.5.0.8]"] ["\"ń±©\"@-Npu4W.unVx6R.DdNnk.4.A6tM.y.EJFHfw92N.ecYbisymo" "\"ń±©\"@-Npu4W.unVx6R.DdNnk.4.A6tM.y.EJFHfw92N.ecYbisymo"] ["\"ņµŖ¶ņ¤†š¶Ÿņƒš­š«Šń¤¬ņ‹‚¼ńŗš·ń† §ņŖ³\"@[0.881.6.929]" "\"ņµŖ¶ņ¤†š¶Ÿņƒš­š«Šń¤¬ņ‹‚¼ńŗš·ń† §ņŖ³\"@[0.881.6.929]"])
#2019-02-0723:49seancorfield(my editor does not render them all correctly šŸ˜ž)#2019-02-0723:49butterguns... scrolling down to "Lets set up some Generators", he uses gen/string instead of string? and it produces the full range#2019-02-0723:50seancorfieldAh, there you go!#2019-02-0723:50buttergunsWould be nice not to have to override all my string? specs with gen/string however šŸ™‚#2019-02-0723:52seancorfieldI guess it depends on how important it is that you exercise your code with more exotic strings?#2019-02-0723:53buttergunsYeah, I guess so. It just surprised me. Testing against exotic strings strikes me as exactly within generative testing's wheelhouse#2019-02-0723:54seancorfieldWell... it is but this is a case where you need to opt into it.#2019-02-0723:57seancorfieldIt really depends on what parts of your data processing you need to stress test. If you only pass a string through your system and into the DB, you don't really care whether it's alphanumeric or all poop emojis.#2019-02-0723:58seancorfieldIf you have specific string processing that does some sort of parsing, it might well be useful to stress test it with really exotic strings.#2019-02-0723:58seancorfieldNot everything is important to test to the same level.#2019-02-0723:59seancorfieldIt might be more useful, for example, to use a string generator that only ever produces an empty string, a short readable string, or a giant long string.#2019-02-0800:01buttergunsThat's a good point. I think this perhaps exposes a weakness in my understanding of generative testing. My brain says "what will this really test that can't be better accomplished with targeted unit testing"#2019-02-0800:01buttergunswacky strings is often my go-to answer#2019-02-0800:02buttergunsBut this is a larger question, somewhat off topic#2019-02-0800:04seancorfieldMy experience so far leads me to think that the power of generative testing is more in the combinations of data it uses to exercise code rather than the specific values of an individual item.#2019-02-0800:08buttergunsCan you elaborate? To that point: I feel like that is what makes gen testing so difficult. E.g. I may have a "Command" that has two parameters. But only a very narrow combination of those two parameters actually make sense. Otherwise the "Command" is just gibberish.#2019-02-0800:09buttergunsAnd when I start using (gen/elements ["very" "small" "number" "of" "valid" "inputs"]) it sorta doesn't feel like generative testing anymore#2019-02-0800:15seancorfieldIf you have interdependent parameters, that's certainly a bit trickier to test. You'd need to write a spec for the possible combinations that were valid, and make sure it's generatable.#2019-02-0800:17seancorfieldAn example from one of our apps. We have a spec for the set/sequence of valid interactions a user (an admin in this case) can have with a sequence of data. Given that (complex!) spec, we can generate arbitrary but valid combinations of operations the user could run. We use those generated combinations of operations to drive an HtmlUnit test to verify that a) they are actually possible and don't produce errors and b) that the end state for the transformed sequence of data is still valid (using its own set of specs).#2019-02-0800:18seancorfieldAutomated UAT of a web app, via generative testing šŸ™‚#2019-02-0800:23buttergunsAh, ok this makes a lot of sense! I think I'm getting hung up on the idea that I have to check the result of applying these operations, in literal sense. Like, "after a generated operation, I should inspect the operation and foo should == bar". It sounds like you're saying "after a generated operation, make sure everything conforms to spec, plus no errors thrown, and then just move on"#2019-02-0800:36seancorfieldIt's really going to depend on your specific system under test. In another part of our system, we have certain pairs of functions that are inverses of each other, so for those we'll do full-on property-based testing to ensure that, for arbitrary "valid input", calling (= (inverse-fn (some-fn some-input)) some-input).#2019-02-0800:38seancorfieldAnd then in other situations, you might want to specifically test handling of bad data so you might write a spec for some of the things that your input cannot be, and then run tests to ensure that for arbitrary "bad input" you get some sort of appropriate failure response (i.e., that given bad input you don't get a success response).#2019-02-0800:38seancorfield(that might help you detect combinations of bad input that you unexpectedly allow through your app -- a gap in your validation)#2019-02-0801:27ericnormandjohn hughes (creator of quickcheck and now property-based testing consultant) always talks about how prop-based testing is exploratory#2019-02-0801:27ericnormandyou model the system you are checking, and often the tests donā€™t pass#2019-02-0801:28ericnormandyou investigate and find that your test was missing something#2019-02-0801:28ericnormandso you modify it and run it again#2019-02-0801:28ericnormandit fails again, it was missing something else#2019-02-0801:29ericnormandmy point is that property based testing always requires work to model the system#2019-02-0801:29ericnormandthere is no ā€œone wayā€ to test#2019-02-0801:30ericnormandi think property based testing is a good name. youā€™re looking for algebraic properties to test for#2019-02-0801:31ericnormand@seancorfieldā€™s example of inverses is a good property#2019-02-0801:31ericnormandanother is idempotence#2019-02-0801:31seancorfieldYup, you often start with a property you believe should hold and sometimes you discover it doesn't -- so either the property is wrong or your system doesn't preserve when it should šŸ™‚#2019-02-0801:32ericnormandcommutativity, associativity, identity, zero#2019-02-0801:32ericnormandall good properties to start with#2019-02-0801:32ericnormandthen you can get creative#2019-02-0801:32ericnormandas in ā€œitā€™s commutative under these conditionsā€#2019-02-0801:33ericnormandor itā€™s commutative under this comparison operator#2019-02-0801:33ericnormand(instead of =)#2019-02-0801:35ericnormandas for generators, i have made use of three#2019-02-0801:35seancorfieldThis output value should always exceed this input value. The output should always be an ordered sequence (regardless of its actual contents). The output value(s) should always be between these bounds (possibly based on input values). Etc.#2019-02-0801:35ericnormandalways valid data, junk data, and almost valid data#2019-02-0801:35ericnormandthe almost valid is the hardest to model#2019-02-0801:36ericnormandfor me at least#2019-02-0801:37ericnormanditā€™s almost as if you need to start with valid data and break it#2019-02-0801:37ericnormandin a random way#2019-02-0801:37ericnormandthose are to test error conditions as sean was saying#2019-02-0801:38seancorfieldMutation testing.#2019-02-0801:38ericnormandah!#2019-02-0801:38ericnormanddoes such a thing exist in clojure?#2019-02-0801:39seancorfieldIn theory, you could build such a system (to mutate code). Easier to mutate input data and see if your system breaks in unexpected ways šŸ™‚#2019-02-0801:39ericnormandyes#2019-02-0801:40ericnormandif d doesnā€™t pass spec, f should throw an exception#2019-02-0801:40ericnormanditā€™s hard to test that thoroughly#2019-02-0801:41ericnormandto feel confident youā€™ve found all of the ways you could do the data wrong #2019-02-0801:41ericnormandit would be interesting to make a generator that could mutate any other value randomly#2019-02-0801:42ericnormandadd or subtract 1 from numbers#2019-02-0801:42ericnormandadd a char to a string#2019-02-0801:42ericnormanddrop an element from a list#2019-02-0801:42ericnormandetc#2019-02-0815:10borkdude@alexmiller one down. https://github.com/borkdude/speculative/issues/124#issuecomment-461833318#2019-02-0815:11borkdudeItā€™s probably an issue with not fully qualifying something in a macro#2019-02-0815:14borkdudemight be this one: https://github.com/clojure/spec-alpha2/blob/master/src/main/clojure/clojure/spec_alpha2.clj#L931#2019-02-0815:28Alex Miller (Clojure team)I fixed it, then broke it again :) Iā€™ve reverted my last changes#2019-02-0815:38borkdudeit works now, except for some error messages that look strange (see issue). I think weā€™ve now seen the first time that all speculative specs can be instrumented. šŸ™‚#2019-02-0815:41borkdudeI have to port respeced to spec-alpha2 before I can run the test suite, Iā€™ll do that sometime soon#2019-02-0815:46Alex Miller (Clojure team)Iā€™m not getting whatā€™s weird about the errors#2019-02-0815:47borkdudeok.
(atom 1 {:validator 1})
Execution error - invalid arguments to clojure.core/atom at (test.clj:129).
{:validator 1} - failed: keyword? at: [:options :clojure.spec-alpha2.impl/k]
The failed keyword should be something like :atom/validator
#2019-02-0815:47borkdudeI think?#2019-02-0815:48Alex Miller (Clojure team)spec 1 version of the first one is the same#2019-02-0815:48Alex Miller (Clojure team)
user=> (atom 1 {:validator 1})
Execution error - invalid arguments to clojure.core/atom at (REPL:1).
{:validator 1} - failed: keyword? at: [:options :clojure.spec.alpha/k]
#2019-02-0815:48Alex Miller (Clojure team)this may be a pre-existing weakness of keys*#2019-02-0815:48borkdudeinteresting, Iā€™ll try#2019-02-0815:48Alex Miller (Clojure team)whatā€™s weird about the dissoc one?#2019-02-0815:49borkdudeOn spec 1 I get:
user=> (dissoc 1)
Execution error - invalid arguments to clojure.core/dissoc at (REPL:1).
1 - failed: map? at: [:map :clojure.spec.alpha/pred] spec: :speculative.specs/map
1 - failed: nil? at: [:map :clojure.spec.alpha/nil]
On spec 2 I get:
user=> (dissoc 1)
Execution error - invalid arguments to clojure.core/dissoc at (test.clj:129).
1 - failed: map? at: [:map :clojure.spec-alpha2/pred]
1 - failed: nil? at: [:map :clojure.spec-alpha2/nil]
#2019-02-0815:50borkdudeso itā€™s clear which of my own specs the argument is violating#2019-02-0815:50Alex Miller (Clojure team)so this bit: spec: :speculative.specs/map#2019-02-0815:50borkdudeI guess so.#2019-02-0815:51borkdudeMaybe it would also be good for expound to test this behavior, since it relies on things like this (@bbrinck)#2019-02-0815:52Alex Miller (Clojure team)yeah, Iā€™m sure there are some subtle things like this that have broken, particularly in regex world#2019-02-0815:52borkdudeI donā€™t have specific demands for this, but expound probably has. Nowā€™s the chance to get it fixed šŸ™‚#2019-02-0815:54borkdudeAre spec objects that are defined in the registry backward compatible with spec1? I donā€™t mean how you are defining them, but the actual result of defining a spec?#2019-02-0815:54Alex Miller (Clojure team)no#2019-02-0815:54bbrinck@borkdude Good idea! I will start a spec2 branch soon for expound to see what works and what needs changes.#2019-02-0815:54Alex Miller (Clojure team)for the moment they are the same protocol, but that protocol is in a different ns so not compatible#2019-02-0815:55Alex Miller (Clojure team)migration/release is another whole thing - weā€™ve talked about it at length but not going to worry about it for now till weā€™re much closer#2019-02-0815:55borkdude@alexmiller if that were true, I think there would be a more smooth upgrade path. spec2 would just register it in both the old and new registry and old consumers would still work#2019-02-0815:55borkdudeah ok, makes sense.#2019-02-0815:56bbrinckIā€™m wondering if expound may need to depend on spec1 and spec2 at the same time. Even if expound has different namespaces that use spec1 and spec2, presumably a lib could upgrade to spec2 but still use a lib that uses spec1. #2019-02-0815:57Alex Miller (Clojure team)that should be fine (presuming you have a version of clojure aware of both registries, etc)#2019-02-0815:57Alex Miller (Clojure team)the clojure integration aspects will require a clojure release - lots of open questions about exactly what that will cover still tbd#2019-02-0815:58bbrinckYeah#2019-02-0815:59borkdudeIā€™ve thought about supporting two versions of the lib, one with a -spec2 suffix in the version. Could work.#2019-02-0816:00bbrinckThat makes sense, as you think about migration/release it would be useful for lib authors to come up with some practices as well, informed by your decisions. I suspect many libs will be in the situation where they want to be spec2 compat but must assume that some deps use spec1. #2019-02-0816:00borkdudebut then again, thereā€™s only one or two places right now where I needed to change something? the finicky difference is the namespace declarations#2019-02-0816:01borkdudeanyway, now it probably not the time > but not going to worry about it for now till weā€™re much closer šŸ™‚#2019-02-0816:01bbrinckIf all spec-libs require both spec1 and spec2, then at least the incremental cost of using any spec-lib wonā€™t be that much :)
#2019-02-0816:04borkdudeAnd then we have CLJS in the mix as well, so if you write CLJC there is that to consider too#2019-02-0818:59gklijsIs there a cljs way to statisfy inst? I used it in a cljc spec and ran into trouble reading the java Instant, then just switched to number, but there might be a nicer way?#2019-02-0819:01gklijsIs this it? https://cljs.github.io/api/syntax/inst-literal#2019-02-0819:14lilactownare you asking how to make a value that is an inst? AFAIK it's just a (js/Date.)#2019-02-0819:32gklijsI should explain in more detail. I have a list of images in de back-end (Clojure) with just an id of the image and the time it was uploaded for now. I want to spec and use this list in the front-end (Clojurescript). I already have some (de)serialize code based on the spec to not repeat the keys everywhere. So I think I could get away with transforming the Clojure Inst to a string in such a way it's easy for Clojurescript to make it into a js/Date. Which gives me my answer by using a DateTimeFormatter.ISO_INSTANT to format the Instant to string, it should be possible to use Instant.parse(x) to get back the Instant while at the same time in Clojurescript I can use #inst x to get the js/Date.#2019-02-0819:36lilactownif I understand correctly, what you are trying to figure out is how to transfer the instant over the wire?#2019-02-0819:36lilactownfrom the back-end to the front-end?#2019-02-0819:37lilactownFWIW, you can't call #inst x where x is a var or some other binding; #inst is a literal, so you'd have to write the date literally #inst "1980-01-01"#2019-02-0819:38lilactownwhat I would suggest is using something like EDN or Transit to communicate between your back end and front-end, so that they can transfer the inst literally without having to format to a string and then re-parse the string#2019-02-0819:57gklijsI already use EDN, but that goes wrong, because there is no Instant object in cljs. You where right about the literal, I need (js/Date. x) to create it.#2019-02-0819:57lilactownare you reading the EDN in?#2019-02-0819:57gklijsyes#2019-02-0819:58lilactownyou might just need to provide the correct arguments to the reader#2019-02-0819:59lilactownhttps://github.com/Lokeh/punk/blob/e6bb3b719571e79a6540597db50163fc4b9d8a4f/ui/src/punk/ui/core.cljs#L464#2019-02-0820:03gklijsThanks, might work better. I actually use reader/read-string now..#2019-02-0819:40Frank HenardHey everyone... question about using s/merge with s/multi-spec: https://stackoverflow.com/q/54599149/59439 I put it on SO for easier findability#2019-02-0822:12borkdude@alexmiller I now ported my testing library to spec-alpha2, this works. Now Iā€™m running the tests of speculative and I ran into a bug which I can reproduce as follows:
user=> (require '[clojure.spec-alpha2.gen :as gen])
nil
user=> (gen/sample (s/gen any?))
Execution error (IllegalArgumentException) at clojure.spec-alpha2.protocols/eval189$fn$G (protocols.clj:11).
No implementation of method: :gen* of protocol: #'clojure.spec-alpha2.protocols/Spec found for class: clojure.core$any_QMARK_
#2019-02-0822:13borkdudeI can probably workaround this by using the test.check generator directly, but just fyi.#2019-02-0822:36borkdudeThis does work: (gen/sample (gen/gen-for-pred string?))#2019-02-0822:40borkdudeThis too:
user=> (gen/sample (s/gen (s/spec string?)))
("" "I" "" "4Cv" "" "Cs" "" "864w" "4" "w8TIh7")
#2019-02-0822:59seancorfield
user=> (gen/sample (s/gen (s/spec any?)))
([] nil [] () {[] ()} () nil nil {} ([{\M :nK21+I.s*-J_ce9P+.-B!ll?._0?!07zMs*.k4N-y_A.?T?b.V*!mC/c-m-?k7, \q .z?} [:?57*-_?*x0:wg!!*v66+:+!Nwv:?*:Vti3*78I7:2_EVDZ8Dij:1:xC0 -1.375]]))
user=> 
#2019-02-0823:00seancorfieldBecause any? isn't a symbol, so you need s/spec now.#2019-02-0823:00seancorfieldThis seems expected to me, given the spec1 -> spec2 changes.#2019-02-0823:00seancorfield^ @borkdude#2019-02-0823:18borkdudeIā€™m not 100% clear on this. E.g. it says in https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha > Symbolic specs consist only of: ā€¦ > Qualified symbols (predicate function references) If Iā€™m reading that, I would say that
(gen/sample (s/gen `any?))
should work since itā€™s a qualified symbol referencing a predicate function?
#2019-02-0823:20borkdudeIt might as well be an accidental breaking change. Iā€™ll wonder what Alex thinks. Iā€™m off to bed now.#2019-02-0823:41seancorfieldGiven that (s/valid? any? :foo) etc all reject any? (and string? etc), I think it's pretty clear it's deliberate @borkdude#2019-02-0823:42seancorfieldany? isn't a spec, it's a predicate. You can define a spec in terms of a predicate. So (s/def ::foo any?) is valid and then (s/exercise ::foo) will be valid but (s/exercise any?) is not.#2019-02-0823:43seancorfieldAnd then (s/spec any?) produces a spec from a predicate. So (s/exercise (s/spec any?)) is valid.#2019-02-0823:44seancorfieldThe line between specs and predicates is very blurred in spec1.#2019-02-0900:43Alex Miller (Clojure team)Sean is correct. s/gen takes a spec object. A predicate symbol or function is not a spec object. Invoking s/spec with a predicate will expand into a qualified symbolic spec and pass it through spec*, yielding a spec object#2019-02-0900:43Alex Miller (Clojure team)So (s/spec any?) or you can do the work of the spec expander and do (s/spec* `any?)#2019-02-0902:07buttergunsQuick question: any way to supply a seed to gen/sample?#2019-02-0902:09buttergunsI want to do some performance testing / benchmarking. Hence getting a predictable sample is critical#2019-02-0902:28seancorfield@mattmorten It sounds like you really need to provide your own overridden generators that produce the same sequence over and over again?#2019-02-0902:36buttergunsHmmm. That would involve a lot of duplication. In my test ns, I have 20+ lines of generator code to produce a complex data-structure in my app. I'm very happy with it. I'd rather say "give me seed 1 of this generator" than go back and write duplicate generators for each of my (many, very nested) attributes that produce a constant value#2019-02-0905:11flyboarderWhy doesnā€™t spec/assert let me set my own message, seems like an optional 3rd arg would be really convenient#2019-02-0907:33borkdude@seancorfield @alexmiller alright, thanks. Now that this is clear Iā€™ll change those #2019-02-0910:00borkdudeFound a new bug:
speculative.specs=> (s/def ::or-spec (s/with-gen (s/or :vector vector) #(s/gen (s/spec vector?))))
:speculative.specs/or-spec
speculative.specs=> (s/valid? ::or-spec [1 2 3])
false
#2019-02-0910:00borkdudewithout s/with-gen it works, with it doesnā€™t#2019-02-1105:07Alex Miller (Clojure team)fixed in latest#2019-02-0910:03borkdudeIt also works when not using s/or + s/with-gen so it seems s/or specific#2019-02-0910:24borkdudeI bumped into another one, but this might be related, so Iā€™ll wait for this one to be resolved#2019-02-0913:42Alex Miller (Clojure team)You have vector in the spec, not vector?#2019-02-0913:42Alex Miller (Clojure team)Not sure thatā€™s the cause but seems like a thing to fix first#2019-02-0915:37borkdudeStill doesnā€™t work with vector?.#2019-02-1011:24borkdudeThis works in spec1 and 2: (gen/sample (s/gen (s/every number? :kind vector?))) This works in spec1 but doesnā€™t work in spec2. Should it? (gen/sample (s/gen (s/every number? :kind coll?)))#2019-02-1014:14Alex Miller (Clojure team)I think so? Not sure why it wouldnā€™t #2019-02-1016:22borkdudeWell it doesnā€™t yet. I posted it in the issue#2019-02-1105:05Alex Miller (Clojure team)I did find and fix a bug in the gen from a :kind, but this particular example should and does fail in both versions. coll? can generate any kind of collection, including maps, which will not pass with something like:#2019-02-1105:06Alex Miller (Clojure team)
user=> (gen/sample (s/gen (s/every number? :kind coll?)))
Error printing return value (IllegalArgumentException) at clojure.lang.RT/seqFrom (RT.java:553).
Don't know how to create ISeq from: java.lang.Double
#2019-02-1105:06Alex Miller (Clojure team)thatā€™s with spec 1, but I see same on spec 2 now. You might sometimes get lucky and get enough non-maps for this to work occasionally.#2019-02-1011:39borkdudeApart from this issue and the or-spec issue, the speculative tests pass with spec-alpha2!#2019-02-1012:23borkdudeThe specs are now backward compatible with spec1. So if spec2 and spec1 would have the same namespace names, it could just be a drop-in replacement.#2019-02-1012:26borkdudeFirst succesful build of the spec-alpha2 branch šŸ˜„ https://circleci.com/gh/borkdude/speculative/tree/spec-alpha2#2019-02-1012:54borkdudeIt just occurred to me that libs can maintain compatibility with spec1 and spec2 by doing this: (:require [clojure.spec.alpha :as s1] [clojure.spec-alpha2 :as s2]) and then define specs for each version of spec: (s1/def :foo number?) (s2/def :foo number?)#2019-02-1013:20borkdude@jpsoares106 FWIW yada is not moving to spec for this reason. Schema will remain to be used to API boundary validation/coercion.#2019-02-1018:27seancorfield@borkdude what is "this reason" in your comment above? We use spec extensively for "API boundary validation / coercion" and it works great. #2019-02-1018:37borkdude@seancorfield Malcolm talked about it here: https://clojurians-log.clojureverse.org/yada/2019-01-18/1547833475.076000#2019-02-1018:38seancorfieldThat doesn't answer the question -- WHY?#2019-02-1018:40borkdudeMalcolm referred to this post on SO where it says that clojure.spec is not indented for coercion: https://stackoverflow.com/a/49056441/6264 Iā€™m not sure what I said anymore, lost track.#2019-02-1018:40seancorfield(And, yes, I read a lot of the follow up discussion there -- none of it seems substantive. It's all opinion)#2019-02-1018:41borkdudeThis opinion comes from the clojure.core team. But if it works for you, nobody will stop you I guess šŸ™‚#2019-02-1018:42seancorfieldI think it's misrepresenting what Alex said. #2019-02-1018:43seancorfieldAnd it specifically omits Alex's justification, which was that it forces all clients of the spec to use the coercion. Which in this case is EXACTLY what you want here. #2019-02-1018:44borkdudeWhat do you mean by ā€œitā€, the SO article?#2019-02-1018:44seancorfieldDave's answer there yes #2019-02-1018:49borkdudeI havenā€™t tried using spec in this way. What I want at the API level is not having junk to come into my system which makes it all the way into the database in some jsonb field. Also I want coercion at the API level (e.g. query params are always strings and some have to be numbers, array, etc.). All I know is that yada does this for me using Schema and I havenā€™t tried to make this work with spec. The main author of yada doesnā€™t want to use spec for this anymore. Thatā€™s all I know šŸ™‚.#2019-02-1018:50borkdudeI wish there was a clear and more elaborate article on this subject so I had more clarity on this myself.#2019-02-1018:53seancorfieldWhen I'm not just on my phone I'll write more about this. We were an early adopter of spec and use it very heavily in production for a lot of different things. I've probably spent more time discussing this aspect with Alex than anyone else outside of Cognitect šŸ™ƒ #2019-02-1018:54borkdudePlease do!#2019-02-1018:57borkdude> We do this for parameters in our REST API, for long, double, Boolean, date, etc ā€“ we have two specs for each: one that is a spec for the target type in the domain model (which in these cases is just defined as the appropriate built-in predicate), and one that is a spec for the API level (which accepts either the target type or a string that can be coerced to the target type). Then we use the appropriate spec at the appropriate ā€œlevelā€ in our application. How do you deal with ā€œextraā€ data coming into your API? How do you deal with preventing junk going into your persistence layer?#2019-02-1018:59lilactownselect-keys? seems like thatā€™s something spec wonā€™t save you from#2019-02-1018:59borkdudeyeah, a nested select keys actually which is what Schema and other tools are good at#2019-02-1019:01borkdudee.g. EQL#2019-02-1019:06lilactowncurious to know how sean handles that too. Weā€™ve been using spec for our services, but just relying on whatever reititā€™s integration with spec-tools does#2019-02-1019:16ikitommispec-tools allows one to drop all extra keys while doing coercion.#2019-02-1019:16ikitommiitā€™s automatically on in reitit & compojure-api#2019-02-1019:20seancorfieldRegarding that quote from me: we have different specs for different layers. We have a spec for the persistence layer and we have code around the persistence layer that derives the set of keys (i.e., the set of columns) from the specs and uses that to narrow the keys to just those the database will accept. I.e., we are open for extensions, as far as the maps are concerned, until we have to close the set for storing it in a system that isn't open šŸ™‚#2019-02-1019:24seancorfieldAt the API boundary, we have specs that are specific to the REST parameters -- strings as inputs (of course) and validation that does the minimal coercion in order to validate those arguments.#2019-02-1019:25borkdudeIt would be useful to have a function that worked generically on a spec that selected just the data that the spec describes, without changing the spec. Is that public?#2019-02-1019:27seancorfieldGiven that a REST API requires coercion, you have a limited number of choices here. You must do some coercion somewhere. You do it before, during, or after the validation step. You can't validate without some coercion. There isn't much point in doing the coercion twice -- and you don't want different code inside validation and outside it. You have the choice of doing all that coercion upfront and then validating the result, or you can do it in place via spec as part of the validation. Anyone criticizing combining that in spec should also be criticizing Schema for the same thing, IMO.#2019-02-1019:31seancorfieldThe objections to doing coercion in a spec are based on a number of things. Alex has repeatedly pointed out that if you do coercion in a spec, you are forcing that coercion on all clients of that spec -- which is a concern if you're building reusable specs and I agree that you shouldn't do coercion in a general, reusable spec.#2019-02-1019:33borkdudeTo be clear, whatā€™s the problem of forcing coercion on a user of the spec?#2019-02-1019:33ikitommiAlex talks about using conform to do coercion.#2019-02-1019:33ikitommiitā€™s always on.#2019-02-1019:35seancorfieldRight. So you only want to use it in a context where you need that coercion in order to do the validation.#2019-02-1019:35ikitommidid a gist how spec-tools (and reitit + c-api) solves this: https://gist.github.com/ikitommi/68f662a399a90e8a70308ffcd4b3e752#2019-02-1019:35ikitommiwould like to see the solution baked into spec itself, hereā€™s the most relevant issue: https://dev.clojure.org/jira/browse/CLJ-2251#2019-02-1019:40seancorfieldI'm with Alex that spec shouldn't be used for JSON transformation/parsing šŸ™‚ Our REST API mostly has simple string input that gets coerced as part of the validation that those strings are valid numbers, dates, booleans -- but where we have JSON input, we use a JSON library to do that aspect of parsing/coercion.#2019-02-1019:42ikitommisure, the json strings get parsed with Jsonista/Cheshire and then the values are coerced. I would argue that ā€œdropping keys that are not part of the specā€ happens in the coercion part. Or you do it manually, which is IMO not a good idea.#2019-02-1019:43seancorfieldWe disagree on that, but that's fine.#2019-02-1019:43ikitommihopefully spec will add support for coercion. the current ways of doing it (conform or form walking) are kinda hacks and need to go.#2019-02-1019:45seancorfieldI doubt it, given how much and how often Alex et al have argued against coercion in spec šŸ™‚#2019-02-1019:46borkdudeClearly thereā€™s a tension between some users of spec and the authors of spec on this subject (coercion/stripping). I believe a good article/guide on http://clojure.org about this topic and the recommended way to go about it would be cool.#2019-02-1019:53ikitommiI guess the upcoming select could be used to strip out data not defined in it?#2019-02-1019:54seancorfieldNo, it's for selecting which keys to check in a given context.#2019-02-1019:56seancorfieldSo that it can decomplect the overall shape of the data spec from the conformance needed in specific situations.#2019-02-1020:56ikitommi@borkdude you asked about a function just to strip out just keys, just remembered that there is a existing transformer doing only that. So, this works (but requires spec-tools):
(require '[spec-tools.core :as st])

(s/def ::zip int?)
(s/def ::country keyword?)
(s/def ::address (s/keys :req-un [::zip ::country]))
(s/def ::name string?)
(s/def ::user (s/keys :req-un [::name ::address]))

(st/coerce
  ::user
  {:name "liisa"
   :TOO "MUCH"
   :address {:zip 33800
             :INFOR "MATION"
             :country "INVALID"}}
  st/strip-extra-keys-transformer)
; {:address {:zip 33800, :country "INVALID"}, :name "liisa"}
#2019-02-1020:56borkdudecool šŸ™‚#2019-02-1020:57ikitommiand you can compose json-transformer with strip-extra-keys-transformer so itā€™s applied in single pass.#2019-02-1101:19joshkhhow might i go about creating a generator for BigDecimal values? is this a job for fmap?
(defn bigdec? [n] (instance? BigDecimal n))
(s/def :bank/balance bigdec?)
(s/gen :bank/balance)
ExceptionInfo Unable to construct gen at: [] for: :bank/balance  clojure.core/ex-info (core.clj:4739)
#2019-02-1109:18misha@joshkh
;; Clojure 1.10.0
(s/exercise decimal?)
=>
([0.5M 0.5M]
 [1.0M 1.0M]
 [2.0M 2.0M]
...
#2019-02-1109:18misha
(defn decimal?
  "Returns true if n is a BigDecimal"
  {:added "1.0"
   :static true}
  [n] (instance? BigDecimal n))
#2019-02-1114:48borkdudemaybe stupid question, but whatā€™s the benefit of s/defop over a macro? https://github.com/borkdude/speculative/issues/124#issuecomment-462352972#2019-02-1114:52mpenetit's basically a parameterized spec, not 2 different specs#2019-02-1114:54Alex Miller (Clojure team)the benefit is that it forms back to the op#2019-02-1114:54mpenetI didn't check the source but reading the comments about it on the dev notes that what it felt like#2019-02-1114:55Alex Miller (Clojure team)that is, you have added a new symbol to the spec op language#2019-02-1114:55Alex Miller (Clojure team)and yes, itā€™s a parameterized fixed spec op#2019-02-1114:55Alex Miller (Clojure team)not a combinator of arbitrary other spec ops#2019-02-1114:56Alex Miller (Clojure team)itā€™s not intended to cover every possible case, just address one common need#2019-02-1114:56borkdudewhatā€™s the benefit of ā€œforms back to the opā€?#2019-02-1114:57Alex Miller (Clojure team)as in the example at https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha, you have created something with your own semantics#2019-02-1114:57mpenets/form return value is your op#2019-02-1114:57Alex Miller (Clojure team)
user=> (s/form ::zip)
(user/bounded-string 5 9)
#2019-02-1114:57Alex Miller (Clojure team)so you retain your semantics#2019-02-1114:57borkdudeah. I was just trying this out. I imagined you would get a specific error message for bounded-string:
user=> (s/explain (bounded-string 0 2) "foooo")
"foooo" - failed: (<= 0 (count %) 2)
#2019-02-1114:58Alex Miller (Clojure team)youā€™re getting the more specific error messages from the definition of bounded-string#2019-02-1114:58mpenetyou can also map the op to something useful in your context for other means I guess#2019-02-1114:59mpenetuser/json-object#2019-02-1114:59mpenetetc#2019-02-1114:59Alex Miller (Clojure team)many of the spec ops have internal constraints that will generate explain problems#2019-02-1114:59Alex Miller (Clojure team)so this is the same as other spec ops#2019-02-1114:59Alex Miller (Clojure team)like regexes check whether the input is nil? or sequential?#2019-02-1114:59Alex Miller (Clojure team)keys checks for map?#2019-02-1114:59Alex Miller (Clojure team)etc#2019-02-1115:00Alex Miller (Clojure team)it may be that some further adjustments should be made in the explain generator (just deferring to the definition spec right now)#2019-02-1115:02Alex Miller (Clojure team)I didnā€™t highlight it, but inst-in, int-in, and double-in are all derived specs and I reimplemented all of those in terms of defop#2019-02-1115:02Alex Miller (Clojure team)they are effectively all parameterized compound specs#2019-02-1115:02borkdudeThis gives more or less the same result when it comes to error messages:
user=> (defmacro bounded-string2 [min max] `(s/and string? #(<= ~min (count %) ~max)))
#'user/bounded-string2
user=> (s/explain (bounded-string2 0 2) "foooo")
"foooo" - failed: (<= 0 (count %) 2)
but s/form will give the expanded spec form, so thatā€™s indeed different
#2019-02-1115:04Alex Miller (Clojure team)the other big thing is that with defop, we actually def a macro#2019-02-1115:04Alex Miller (Clojure team)well I guess youā€™ll get the same effect if youā€™re using defmacro here#2019-02-1115:05Alex Miller (Clojure team)(vs just implementing the lower-level create-spec)#2019-02-1115:08borkdudehttps://github.com/clojure/spec-alpha2/blob/bfba0e37f1b72b2723eda9a7887a96a08e92698d/src/main/clojure/clojure/spec_alpha2.clj#L757 šŸ™‚#2019-02-1115:14borkdude@mpenet > itā€™s basically a parameterized spec, not 2 different specs Not sure what you mean by this. If you call (bounded-string 1 10) and (bounded-string 1 20) you will get two different spec objects#2019-02-1115:15mpenetyeah but the form share quite a bit, I guess the facts it's 2 distinct spec objects is an impl detail#2019-02-1115:16mpenetas I said I don't know more than what I read on the blog post + intuition of what's the final intent#2019-02-1115:19mpenetfor lib authors it's nicer to read (s/sorted-coll-of x comparator) than (s/and (s/coll-of x) #(..))#2019-02-1115:20borkdudewhen would a lib author read this? not trying to argue for the sake of arguing, just want to get it clear for myself šŸ™‚#2019-02-1115:21borkdudeI havenā€™t used s/form and friends myself yet, so I havenā€™t needed this feature much (probably out of ignorance)#2019-02-1115:21mpenetex: spec-tools when trying to understand specs to build json schemas#2019-02-1115:22mpenethere we have specs that are used to validate json params also, one of which is some kind of derivative of coll-of, it makes dealing with it much nicer too#2019-02-1115:22borkdudebut compared to a macro you donā€™t see a benefit there yet? I mean when reading and writing the literal code?#2019-02-1115:23mpenetnot sure what you mean#2019-02-1115:23mpenetmore readable s/form is a plus for me, ability to build your own s/coll-of like ops is good too#2019-02-1115:24borkdudeI mean, when you define a macro you get to write same code:
user=> (defmacro bounded-string2 [min max] `(s/and string? #(<= ~min (count %) ~max)))
#'user/bounded-string2
user=> (bounded-string2 1 100)
#object[clojure.spec_alpha2.impl$and_spec_impl$reify__1407 0x64b3b1ce "
#2019-02-1115:24borkdudewhen are you using s/form?#2019-02-1115:24borkdudemaybe this is used by explain logic?#2019-02-1115:26mpenetanytime you want to understand what's behind a spec#2019-02-1115:26mpenetprogramatically#2019-02-1115:26borkdude> ability to build your own s/coll-of like ops is good too this is limited with s/defop since you cannot pass in any spec argument. e.g. (my-coll-of (s/nilable ::my-spec)) wonā€™t work?#2019-02-1115:26mpenetno idea#2019-02-1115:26mpenetas I said I read only the blog post#2019-02-1115:26mpenetand what's here#2019-02-1115:27borkdudeI was referring to this link that I posted earlier: https://github.com/borkdude/speculative/issues/124#issuecomment-462261842#2019-02-1115:27borkdudethatā€™s what left me wondering about defmacro vs s/defop#2019-02-1115:29borkdudemaybe not a huge limitation since you can always def the ā€œanonymousā€ spec first#2019-02-1115:44Alex Miller (Clojure team)int-in, inst-in, double-in are all good examples where this is useful too#2019-02-1115:44Alex Miller (Clojure team)they are all compound parameterized specs#2019-02-1115:45borkdude> where this is useful What exactly are you referring to?#2019-02-1115:46borkdudethe result of s/form?#2019-02-1115:47Alex Miller (Clojure team)I just mean general cases where s/defop is better#2019-02-1115:47Alex Miller (Clojure team)symbolic specs form a language, defined by the ops#2019-02-1115:48Alex Miller (Clojure team)using a macro that expands to a compound spec is fine - youā€™re using an initial step to produce something in the language#2019-02-1115:49Alex Miller (Clojure team)using defop lets you create new ops in the language for the special case where the op can be defined in terms of other spec ops (and is not parameterized by another spec)#2019-02-1115:50Alex Miller (Clojure team)and if you need that, you can drop down another level and do the same thing the provided ops are doing - implement the create-spec multimethod to return a Spec protocol instance#2019-02-1115:50Alex Miller (Clojure team)(but a lot more code is required)#2019-02-1115:51borkdudeis nilable an op? then why can (s/nilable ::foo) not be passed as an argument to defop, but ::foo can? sorry, bit confused about ā€œsymbolic specsā€#2019-02-1115:51Alex Miller (Clojure team)nilable is an op#2019-02-1115:51Alex Miller (Clojure team)as I said above, defop is not designed to be parameterized by other symbolic specs#2019-02-1115:52borkdudeso passing ::foo just accidentally works?#2019-02-1115:52Alex Miller (Clojure team)whatā€™s the example?#2019-02-1115:52borkdude(seqable-of ::foo) vs. (seqable-of (s/nilable ::foo))#2019-02-1115:54Alex Miller (Clojure team)it really goes to the evaluation model. spec names (fq keywords) are a little special in that they eval to themselves and also they are the only thing other than a spec object explicitly handled in the spec api functions.#2019-02-1115:56Alex Miller (Clojure team)in this case, ::foo evaluates to a valid symbolic spec (where another spec form evaluates to a spec object, which is not a symbolic spec)#2019-02-1115:57Alex Miller (Clojure team)so it will work, but youā€™ve created a narrow constraint on how it can be used#2019-02-1115:57borkduderight, so itā€™s something that happens to work, but not really the common use case for defop#2019-02-1115:58Alex Miller (Clojure team)yeah, I hadnā€™t really thought about that#2019-02-1115:58Alex Miller (Clojure team)youā€™re also not going to get proper explain path-ing with it as a spec created by defop is considered to be a single op#2019-02-1115:59Alex Miller (Clojure team)so if the sub-spec fails, you wonā€™t get the parent spec in the path#2019-02-1116:01borkdudeok. to conclude: defop is not designed to support spec arguments. If you want that, either write a macro and accept less helpful error messages and s/form output, or ā€œdrop down another level and do the same thing the provided ops are doing - implement the create-spec multimethod to return a Spec protocol instanceā€ which requires more code#2019-02-1116:03Alex Miller (Clojure team)yes#2019-02-1116:03Alex Miller (Clojure team)although I think in the macro case, the errors are almost exactly the same#2019-02-1116:03Alex Miller (Clojure team)so I would maybe quibble with that part#2019-02-1116:08borkdudenot going to publish this anywhere, so I think for now itā€™s clear šŸ˜‰#2019-02-1116:15borkdudeI notice that regular pre-defined predicates are also not supported in defop:
user=> (s/defop first-pred [pred] (s/and (pred (first %))))
#'user/first-pred
user=> (s/valid? (first-pred number?) [1 "f"])
Maybe a bad example
#2019-02-1116:30Alex Miller (Clojure team)again, evaluation#2019-02-1116:31Alex Miller (Clojure team)the definition in defop is not going to evaluated - it should be a valid symbolic spec but where the parameterized values are substituted (defop is literally going through and replacing the args with their values)#2019-02-1116:47borkdudeMade a typo. This works:
user=>  (s/defop first-pred [pred] (s/and #(pred (first %))))
user=> (s/valid? (first-pred number?) [1 "f"])
true
#2019-02-1116:53seancorfieldFor specs parameterized by other specs, you can do something like
(defn domain-keywords
  "Given a spec, return a new spec that can conform a keyword or string to
  a keyword that is constrained by the given spec."
  [spec]
  (s/spec* `(s/with-gen (s/and ::keyword ~spec)
              (fn [] (g/fmap name (s/gen ~spec))))))
(that and bounded-string above come from the World Singles Networks' codebase)
#2019-02-1116:54borkdude@seancorfield what benefit does that have over writing domain-keywords as a macro?#2019-02-1116:56Alex Miller (Clojure team)well, itā€™s a function so you can apply it, so usual benefits of function over macro#2019-02-1116:57Alex Miller (Clojure team)(with the caveat that the spec arg needs to be a form, not an object, here)#2019-02-1116:57seancorfieldIt was a function in the spec1 world (without (s/spec* ..) and the quote/unquote, so we made it a function in the spec2 world. Minimal change (and it still composes and applies etc).#2019-02-1116:58borkdude@seancorfield I had a similar thing with seqable-of. Function in spec1, but when I turned it into a function in spec2 using s/spec*, I could not provide specs like (s/nilable ::foo) because I got an error, so then I made it a macro.#2019-02-1116:58seancorfieldI plan on writing up a (probably long) blog post on our conversion from spec1 to spec2 when Alex tells me spec2 is stable enough for that to be widely valuable šŸ™‚#2019-02-1116:58Alex Miller (Clojure team)you canā€™t use this with in s/def though (like (s/def ::x (domain-keywords ...)))#2019-02-1116:59seancorfieldRight. And we use s/defop for those sorts of things.#2019-02-1116:59Alex Miller (Clojure team)but you could with the functional entry point s/register which takes an object (which is what domain-keywords returns)#2019-02-1117:00Alex Miller (Clojure team)@seancorfield btw, I spent some time working on the regex thing and I need to undo the changes I made to support forward references in regexes#2019-02-1117:00Alex Miller (Clojure team)which will fix the nesting issue, but re-break forward references#2019-02-1117:00seancorfield@borkdude Mostly, we've found changing our defn spec builders over to s/defop has been all we've needed in the most common cases. A few have needed s/spec* instead.#2019-02-1117:00borkdude> Right. And we use s/defop for those sorts of things. Sorry, referring to what?#2019-02-1117:01seancorfield@alexmiller That's fine -- the forward reference issue only affected one spec in our entire code base so I can just move it below the sub-specs šŸ™‚#2019-02-1117:01Alex Miller (Clojure team)Forward refs in regex is solvable via different means but I need to talk to Rich before I commit to a path on that and heā€™s out today#2019-02-1117:01seancorfield@borkdude "those sorts of things" = "use this with in s/def"#2019-02-1117:02borkdudeaā€™ight#2019-02-1117:04borkdudeI canā€™t write a very long blogpost about transitioning to spec2. All I had to do is report bugs, which were all fixed, wrap a bunch of predicates in s/spec, refactor one predicate to #(not (sequential %)) instead of (complement sequential?) and turn a private function into a macro.#2019-02-1117:05borkdudeI think I might write a tweet about it instead.#2019-02-1117:06Alex Miller (Clojure team):)#2019-02-1117:06Alex Miller (Clojure team)@seancorfield reverted the fwd reference fix, which should fix the nesting issue (but break that fwd reference)#2019-02-1117:08borkdudedid forward referencing ever work? didnā€™t know#2019-02-1117:10Alex Miller (Clojure team)yes, in many cases#2019-02-1117:11Alex Miller (Clojure team)in general, specs delay lookup of named specs until use#2019-02-1117:11Alex Miller (Clojure team)I fixed several places where that wasnā€™t being done#2019-02-1117:13Alex Miller (Clojure team)but changes in how regexes are implemented mean that we effectively broke it for them#2019-02-1117:14Alex Miller (Clojure team)regex impls used to not be specs but would get spec-ized when needed. in spec 2, regexes actually are spec instances (thanks to metadata protocol implementation!) which simplifies the code in many places, but removed the point where this delay naturally happened before#2019-02-1117:15Alex Miller (Clojure team)fixing it is ā€¦ tedious#2019-02-1117:15borkdude(s/declare ::foo) šŸ˜‰#2019-02-1117:15Alex Miller (Clojure team)yeah, no thanks :)#2019-02-1118:08seancorfield@alexmiller Good to know. I'll run a full test suite with the latest spec2 shortly.#2019-02-1119:47seancorfield@alexmiller Confirming that our full test suite runs on the latest spec2, with that one forward reference regex spec moved after the specs that it refers to.#2019-02-1119:49Alex Miller (Clojure team)šŸ‘#2019-02-1119:49borkdudespeculative still works as well#2019-02-1213:56guyIf youā€™re going to be using spec in a project is it better to use spec-2 now instead?#2019-02-1214:07Alex Miller (Clojure team)no#2019-02-1214:07Alex Miller (Clojure team)we have not yet done any releases of spec 2#2019-02-1221:30guyThanks!#2019-02-1217:05seancorfield(if you're using CLI / deps.edn, you can at least start testing against spec2 to see what code changes you might need to make at some future point @guy)#2019-02-1221:31guythanks!#2019-02-1221:02mishahttps://clojurians.slack.com/archives/C03RZMDSH/p1550005322074900#2019-02-1221:25aisamuWe usually create aliases (`alias`) and then use ::#2019-02-1221:30misha1) assuming alias :datomic.client.protocol = pro: ::pro/response is fine, but ::pro.response/body just does not work, (as keyword ns is just a string with no "structure" inside, afaik). So you'd need to make an alias for :datomic.client.protocol.response as well. And if you don't have datomic.client.protocol.response and datomic.client.protocol namespaces as files, thats 2 lines for create-ns, and 2 for alias, all just to save 5 words :( 2) afaik, clojurescript does not have alias function (or I did not find it yesterday)#2019-02-1221:35mishayeah, https://github.com/clojure/clojurescript/blob/master/src/main/clojure/cljs/core.cljc#L42
(ns cljs.core
  (:refer-clojure :exclude [ ...
                            #
#2019-02-1306:07caleb.macdonaldblackIs there a way to create an anonymous spec? So instead of defining it, I can just create one and pass it into spec functions?#2019-02-1306:13seancorfield@caleb.macdonaldblack I suspect the answer is "yes" but it's different between spec1 and spec2...#2019-02-1306:14caleb.macdonaldblackspec2 as in the next version of spec?#2019-02-1306:14seancorfieldIn spec1, you can mostly use a predicate interchangeably with a spec. In spec2, you can construct spec objects on the fly.#2019-02-1306:14seancorfieldYes, clojure.spec-alpha2#2019-02-1306:14caleb.macdonaldblackWell I'm now excited for spec2#2019-02-1306:15seancorfieldHahaha... well, there are no releases yet, but if you're using deps.edn, you can test against it.#2019-02-1306:17caleb.macdonaldblackI'm using leiningen. I may as well wait until an official release. I'm glad to see spec moving in that direction#2019-02-1306:19seancorfieldHave you seen Rich's talk from Conj? (Maybe Not)#2019-02-1306:19seancorfieldHe talks about future direction for spec...#2019-02-1306:20caleb.macdonaldblack"Maybe Not"?#2019-02-1306:20caleb.macdonaldblackif so I haven't seen it. I'll have a look though#2019-02-1306:22seancorfieldhttps://youtu.be/YR5WdGrpoug#2019-02-1306:23seancorfieldSpecifically he talks about how s/keys complects the shape of data and the actual checks that are needed in any given context.#2019-02-1306:27caleb.macdonaldblackinteresting#2019-02-1306:51Alex Miller (Clojure team)in both spec 1 and 2 you can create a spec object and pass it to any of the spec api functions#2019-02-1306:51Alex Miller (Clojure team)with the caveat that s/keys relies on having registered key specs to rely upon#2019-02-1306:52Alex Miller (Clojure team)there may be more support in spec 2 for map selection specs with anonymous key specs#2019-02-1306:53Alex Miller (Clojure team)I am working in that area right now#2019-02-1307:17caleb.macdonaldblackAh okay i didnt know that#2019-02-1314:25rascioHi here! I'm playing with clojure specs, I'm struggling in trying to use generators for a spec with regex: (s/def ::entity (s/and string? #(re-matches #"DEST:\d+"))) Am I wrong in defining something? Or the regex can't be generated by the default lib?#2019-02-1314:27borkdude@manuelrascioni youā€™re missing a %:
user=> (s/def ::entity (s/and string? #(re-matches #"DEST:\d+" %)))
:user/entity
user=> (s/valid? ::entity "DEST:1")
true
#2019-02-1314:30borkdudeYour spec will generate, but the likelihood that it generates strings that will satisfy the predicate is extremely small. You probably want to provide a generator with the spec, using s/with-gen#2019-02-1314:39rascio@borkdude ah...thank you! It seems I still have to train my eye to check for this kind of mistakes...I will check the docs for the with-gen thank you for the hint!#2019-02-1314:44borkdude@manuelrascioni E.g.:
(s/def ::entity (s/with-gen (s/and string? #(re-matches #"DEST:\d+" %)) #(gen/fmap (fn [i] (str "DEST:" i)) (s/gen nat-int?))))
(gen/sample (s/gen ::entity))
("DEST:1" "DEST:0" "DEST:1" "DEST:2" "DEST:2" "DEST:7" "DEST:16" "DEST:41" "DEST:1" "DEST:4")
#2019-02-1314:47rasciogreat! thank you, it is very helpful!#2019-02-1314:50rasciojust to check if I understood well, the fmap is used to "customize" the value generated by a generator, and return a generator, right?#2019-02-1314:54borkdudefmap returns a new generator which transforms the values generated by the mapped-over generator using a function#2019-02-1315:02buttergunshttps://github.com/miner/strgen#2019-02-1315:22Alex Miller (Clojure team)a newer better version of that is in test.chuck#2019-02-1315:23Alex Miller (Clojure team)https://github.com/gfredericks/test.chuck#string-from-regex#2019-02-1315:26borkdudeI recently ran orchestra which tests ret-specs with speculative on a body of code. The only ret spec which wasnā€™t correct was for an fdef for which I had not used generative testing.#2019-02-1315:28borkdudeit was e.g. re-find, re-matches, etc. for which I had not considered that it could also return nils, as in
(re-find #"(a)?(b)" "b")
["b" nil "b"]
I wonder how I could have written a generator for this.
#2019-02-1315:29borkdudeI would have to generate regexes and strings that would sometimes match, sometimes not#2019-02-1316:13borkdudeThis fun experiment is able to generate regexes that seem to not terminate when executedā€¦
(defn test-re-find []
  (let [regex-gen (gen/fmap (fn [parts]
                              (let [s (str/join parts)]
                                (re-pattern (str/join parts))))
                            (s/gen (s/* (s/cat :part
                                               (s/or :string string?
                                                     :group (s/with-gen string?
                                                              #(gen/fmap (fn [s]
                                                                           (str "(" s ")"))
                                                                         (s/gen string?))))
                                               :maybe (s/? (s/with-gen string?
                                                             #(gen/fmap (fn [s]
                                                                          (str s "?"))
                                                                        (s/gen string?))))))))
        matcher-gen (gen/fmap (fn [[r s]]
                                (re-matcher r s))
                              (gen/tuple regex-gen (s/gen string?)))]
    (map re-find (gen/sample matcher-gen 100))))
#2019-02-1316:13borkdude
user=> (gen/sample regex-gen)
(#"" #"" #"" #"()15?" #"(2)" #"(3)" #"()9?F?JWwff1" #"xE7(re9)(79W)26E?()jKqXKk5?(DY1Xa)()(4m)2qNS3?" #"gW7GAJ9p22Z4o4eWJ?" #"(Gi8r)RQ22uBC?(jj0PolFmd)h7?Taz()(7)GwE0")
#2019-02-1319:38borkdudeIā€™m trying this now:
(s/def ::regex.char #{"a" "b"})
  (s/def ::regex.group (s/with-gen string?
                         #(gen/fmap (fn [s]
                                      (str "(" s ")"))
                                    (s/gen ::regex.pattern))))
  (s/def ::regex.maybe (s/? (s/with-gen string?
                              #(gen/fmap (fn [s]
                                           (str s "?"))
                                         (s/gen ::regex.pattern)))))
  (s/def ::regex.pattern (s/* (s/or :char ::regex.char
                                    :group ::regex.group
                                    :maybe ::regex.maybe)))
This gives me a stackoverflowā€¦
(binding [s/*recursion-limit* 1]
    (gen/sample (s/gen ::regex.pattern)))
#2019-02-1320:13borkdudeHow do I get generators to play nice with conformers?
(s/def ::my-spec (s/and int? (s/conformer str)))
(gen/sample (s/gen ::my-spec)) ;; => (0 -1 0 0 0 0 6 42 -1 -3) <- want strings here
#2019-02-1320:25seancorfieldAs written, your spec accepts (only) numbers -- and it is generating numbers that your spec accepts. That's the correct behavior.#2019-02-1320:27seancorfieldConforming numbers to strings as part of a spec feels very wrong to me (and you know what an advocate I am for certain types of coercion in specs! šŸ™‚)#2019-02-1320:28borkdudethis was just an example, not something Iā€™m doing for real#2019-02-1320:29borkdudethe thing I was doing for real was the regex.pattern above, where I want to generate strings, but describe those strings in terms of spec#2019-02-1320:30seancorfieldGenerators must produce values that are acceptable to your spec.#2019-02-1320:31seancorfield(and we've had repeated cautions from @alexmiller not to use spec regex for string parsing/generation stuff šŸ™‚ )#2019-02-1320:33borkdudefor parsing yes, because of performance, there are better tools, but for generation, I currently donā€™t know a better tool šŸ˜›#2019-02-1320:35seancorfieldWhy not use test.chucks regex string generator?#2019-02-1320:35seancorfield(or did I miss your rationale for not using that?)#2019-02-1320:35borkdudeI want to generate regexes, not strings that are matched by a given regex#2019-02-1320:38borkdudeonce I have that, I can use test.chuck to generate strings from the generated regexes. and then I can use stest/check to test re-find, etc.#2019-02-1320:52Alex Miller (Clojure team)why not make a regex for regexes?#2019-02-1322:38aisamuCan't tell if serious#2019-02-1323:08Alex Miller (Clojure team)I'm not sure either#2019-02-1320:52Alex Miller (Clojure team)then use test.chuck on it#2019-02-1320:58borkduderegex language cannot be expressed with a regex, I think you need a CFG tool like spec#2019-02-1321:45borkdudethis kinda works:
(s/def ::regex.pattern
    (s/* (s/cat :pattern
                (s/alt :char #{\a \b}
                       :group (s/cat :open-paren #{\(}
                                     :inner-pattern ::regex.pattern
                                     :closing-paren #{\)}))
                :maybe (s/? #{\?}))))
  (s/valid? ::regex.pattern (seq "(ab)"))
  (s/valid? ::regex.pattern (seq "ab(ab)?"))
  (map str/join (binding [s/*recursion-limit* 2]
                  (gen/sample (s/gen ::regex.pattern))))
  
  (defn test-re-find []
    (let [regex-gen (gen/fmap (fn [r]
                                (re-pattern (str/join r)))
                              (s/gen ::regex.pattern))
          matcher-gen (gen/fmap (fn [[r strs]]
                                  (re-matcher r (str/join strs)))
                                (gen/tuple regex-gen (s/gen (s/* #{"a" "b"}))))]
      (let [matchers (gen/sample matcher-gen)]
        (map re-find matchers))))

  (test-re-find)
At least Iā€™m now finding return values that I didnā€™t account for in an early version of the spec, e.g.:
["ba" "a" "a" "" "" nil nil "" "" ""]
#2019-02-1321:46borkdudeafk now#2019-02-1410:01borkdudeSeems to work now. Iā€™m generating regexes using a simplified regex spec (that exists solely for generating) and Iā€™m generating strings that match it using test.check. Iā€™m using these combinations to test re-find, etc. Thanks for the suggestions all šŸ™‚#2019-02-1410:01borkdudehttps://twitter.com/borkdude/status/1095985994590027777#2019-02-1410:02borkdudeThe only problem left is making it work on CLJS and self-hosted CLJS šŸ˜• https://github.com/gfredericks/test.chuck/blob/master/src/com/gfredericks/test/chuck/generators.cljc#L244#2019-02-1410:04borkdudeI think Iā€™ll just replace the generator on CLJS with a simpler one#2019-02-1411:33gfredericks@borkdude there's a PR for that šŸ™‚#2019-02-1411:33gfredericksI'm not sure if it works though#2019-02-1411:34borkdudePR seems out of date. Even if itā€™s incomplete I think itā€™s better than nothing? @wilkerlucio#2019-02-1411:36gfredericksyou're suggesting I just merge it? I'm not sure what you mean by "out of date"#2019-02-1411:37borkdudethat the PR has merge conflicts
#2019-02-1411:41wilkerlucioI wouldn't merge it, the range of things it works is quite narrow at this point, I'm using something simpler to generate strings, if we want to get that working in cljs (which would be awesome) that code needs better testing and impl, not good to merge as is IMO.#2019-02-1411:42wilkerluciowhat I'm using for cljs these days is a much simpler string generator that knows just some basic patterns (numbers, letters, alphanum):#2019-02-1411:42wilkerlucio
(ns string-gen
  (:require [com.wsscode.test.chuck.charsets :as charsets]
            [clojure.test.check.generators :as gen]))

(defn charset->gen [charset]
  (let [size (charsets/size charset)]
    (if (zero? size)
      (throw (ex-info "Cannot generate characters from empty class!"
               {:type ::ungeneratable}))
      (gen/fmap (partial charsets/nth charset)
        (gen/choose 0 (dec size))))))

(def type->charset
  {"D" (charsets/range "0" "9")
   "W" (charsets/union (charsets/range "0" "9") (charsets/range "a" "z") (charsets/range "A" "Z"))
   "A" (charsets/range "A" "Z")})

(defn parse-int [x]
  (js/parseInt x))

(defn token->gen [token]
  (cond
    (string? token)
    (gen/return token)

    (keyword? token)
    (if-let [[_ t n] (re-find #"([DAW])(\d+)" (name token))]
      (gen/fmap #(apply str %)
        (gen/vector (charset->gen (type->charset t)) (parse-int n)))
      (throw (ex-info "Invalid keyword token" {:token token})))

    :else
    (throw (ex-info "Invalid token" {:token token}))))

(defn string-gen [tokens]
  (->> tokens
       (mapv token->gen)
       (apply gen/tuple)
       (gen/fmap #(apply str %))))
#2019-02-1411:47borkdudeIā€™ll consider tweaking this. Thanks#2019-02-1411:35borkdudeI just realized that core.match could play nice with clojure spec conform results (in general, not related to this regex discussion)? https://stackoverflow.com/a/54687183/6264#2019-02-1411:36borkdude(probably realized a little late)#2019-02-1413:02joshkhi'm probably missing something obvious -- how can i define a spec so that its value conforms to one of a collection of specs? for example, a person can have a pet that's either a mammal or a fish, neither of which share a common attribute.
(s/def :animal.class/mammal (s/keys :req [:utters/count]))
(s/def :animal.class/fish (s/keys :req [:fins/count]))
(s/def :person/pet (s/or :animal.class/mammal :animal.class/fish))

(s/explain :person/pet {:utters/count 4})
val: #:utters{:count 4} fails spec: :animal.class/fish at: [:animal.class/mammal] predicate: (contains? % :fins/count)
=> nil
#2019-02-1413:03joshkhwhoops - that was meant for #beginner#2019-02-1413:04borkdude(s/or :animal.class/mammal :animal.class/fish) => (s/or :mammal :animal.class/mammal :fish :animal.class/fish)#2019-02-1413:04borkdudeyou need to tag the alternatives with a key#2019-02-1413:04joshkhah!#2019-02-1413:04joshkhi saw that in cat/alt, didn't realise it applied to /or. thanks as usual, borkdude.#2019-02-1413:40andy.fingerhutborkdude, are you basically looking for bugs in the JVM's and/or JavaScript's regex library using spec and generative testing?#2019-02-1413:42andy.fingerhutI wouldn't be surprised if you find bugs in those, and/or test.chuck's generate-a-string-matching-a-regex-from-a-regex code, or all of the above. Just so you realize that the time scale for fixing the JVM and/or JavaScript's regex library might be a bit long šŸ™‚ Those libraries must be challenging to maintain.#2019-02-1413:45borkdudeThe situation is like this: someone used Orchestra with speculative specs. Orchestra checks ret specs at runtime (in contrast to spec). It turned out the ret specs of regex functions were incomplete and I could have found this if I had used generative testing (which I have for almost all specā€™ed functions except these). Now I have almost fixed this generative testing, but I ran into the CLJS limitation of test.chuck as a last issue.#2019-02-1413:46borkdudeI can bypass this by generating simpler strings for CLJS. It will still find the case I forgot to spec.#2019-02-1413:48borkdudeIā€™m not using it for anything more serious than this, just wanted to see how for I could go with it.#2019-02-1414:16mpenetany planned changes (or options) about fn as argument triggering gen, something like wrapping their arguments with asserts instead so that they fail at invoke time ?#2019-02-1414:17mpenetI think there was a jira issue about this#2019-02-1414:21Alex Miller (Clojure team)will revisit, haven't yet#2019-02-1417:02djtangoIs there a way to provide a custom generator for a spec distant to the spec's definition/local to the test context? The docs for instrument say that the :replace only relates to fn-specs and :gen option is only for stubbed vars#2019-02-1417:03borkdude@djtango do you want to test with stest/check or instrument?#2019-02-1417:07djtangoI think right now want to manually generate some inputs using gen/sample#2019-02-1417:07djtangothough could also do stest/check - was it stest/check that lets you also patch in your own generators?#2019-02-1417:07djtangothat seems familiar#2019-02-1417:07borkdudeyes,
(stest/check `my-function {:gen {::my-spec (fn [] (gen/ā€¦))}})
#2019-02-1417:10djtangoawesome thanks#2019-02-1417:10djtangobut with instrument no luck?#2019-02-1417:10borkdude> :gen overrides are used only for :stub generation.#2019-02-1417:11borkdudeIā€™m not sure what :replace does, I have never used it šŸ™‚#2019-02-1417:12borkdude
:replace replaces a fn with a fn that checks args conformance, then
invokes the fn you provide, enabling arbitrary stubbing and mocking.
#2019-02-1417:12borkdudešŸ¤”#2019-02-1509:11borkdude@wilkerlucio FWIW:
$ clj -R:test -m cljs.main -re node
Downloading: com/wsscode/test-chuck-string-from-regex-cljs/com.wsscode.test-chuck-string-from-regex-cljs/0.3.0/com.wsscode.test-chuck-string-from-regex-cljs-0.3.0.pom from 
Downloading: com/wsscode/test-chuck-string-from-regex-cljs/com.wsscode.test-chuck-string-from-regex-cljs/0.3.0/com.wsscode.test-chuck-string-from-regex-cljs-0.3.0.jar from 
ClojureScript 1.10.516
cljs.user=> (require '[com.wsscode.test.chuck.core :as sfr])
Unexpected error macroexpanding instaparse.core/defparser at (regexes.cljs:18:1).
Error parsing grammar specification:
Parse error at line 1, column 1:
./resources/com/gfredericks/test/chuck/regex-cljs.bnf
^
Expected one of:
<
Īµ
eps
EPSILON
epsilon
Epsilon
(*
#"[^, \r\t\n<>(){}\[\]+*?:=|'"#&!;./]+(?x) #Non-terminal"
#2019-02-1609:47borkdude
(require '[clojure.spec.alpha :as s])
(require '[clojure.spec.gen.alpha :as gen])
(s/def ::foo (s/keys* :req-un [::a ::b]))
(s/def ::a number?)
(s/def ::b number?)
(s/valid? ::foo [:a 1 :b 2]) ;; true
(gen/generate (s/gen ::foo {::a (fn [] (gen/return 1))})) ;;=> (:a -15164 :b 24.0)
any reason I canā€™t override a key generator in a keys* spec?
#2019-02-1609:59Alex Miller (Clojure team)no, probably a bug#2019-02-1611:58borkdude@alexmiller JIRA here: https://dev.clojure.org/jira/browse/CLJ-2483 I could take a look myself if youā€™re still accepting patches for spec1.#2019-02-1612:51borkdudeI think itā€™s a more general problem:
(gen/generate
   (s/gen (s/with-gen ::foo
            (fn []
              (s/gen (s/cat :key-a #{:a} :val-a ::a :key-b #{:b} :val-b ::b))))
          {::a (fn [] (gen/return 1))})) ;;=> (:a -1 :b "OXi2")
Overriding the generator on a spec thatā€™s made with with-gen has no effect apparently?
#2019-02-1614:49Alex Miller (Clojure team)Oh, thatā€™s true based on the impl for sure#2019-02-1620:10don.dwoskeI finally watched the Maybe Not talk and have some comments. Agree totally based on our history of modeling a biomedical research domain that schema/select is a great idea. A context is needed in order to know the required/optionality. For us information gets added to entities over time, what is an optional field in one stage in the pipeline, may be required in a future stage. However, I think something additional is missing. Our predicates also depend on the context. The allowable values for a key depend on the context, so we want to attach narrowing predicates at that 'select' time, not just the required/optionality part. Hopefully a simple example: if we define a Car entity, with a make, the make is validated against a CV list that has lots of makes in it. However, if we are working within a particular dealership context - a dealership which only sells Jeeps and Land Rovers, we want to add a new predicate at that select time which not only requires the make be present, but that it also be one of those two values, and not just any kind of make. Not only is the shape gaining requirements in some contexts, the allowable values are as well. The predicates added at select time are always and- they reduce the set of allowable values and cannot make it more general - thus, there is no breaking change to the original s/def contract for the key. Rich was close to this idea during his talk and applied it to shapes, but not to predicates. Hopefully, I've described all this well enough, and I'm not missing something obvious, but my Google/Slack-fu didn't turn up anything relevant on this. Thanks.#2019-02-1712:30wilkerlucio@U4NQYBCU8 I trully believe in this idea of information getting adding over selections over time, maybe you will be interesting to check a library that I maintain that pushes this idea futher, and provides a system where by defining inputs and outputs as maps (with schema/select like definitions), the system can transition from some input to some output, given there is some available path in the system. this is the lib: https://github.com/wilkerlucio/pathom and if you like you can see I talk I gave about it (also in this conj :)) https://www.youtube.com/watch?v=yyVKf2U8YVg#2019-02-1922:37johanatanThereā€™s no documentation on this project yet but this is the intended usage of: https://github.com/johanatan/validaze#2019-02-1922:38johanatanYou can get an idea how to use it by reading the tests (including spec generators) and source code.#2019-02-1922:39johanatanThe strong point of this lib is very user-friendly validation failure error messages.#2019-02-1621:06Alex Miller (Clojure team)you can s/and additional predicates any time you like#2019-02-1822:44don.dwoskeBut that changes things in a global context... an earlier stage context doesn't want to 'see' the additional s/and for that def.#2019-02-1823:00Alex Miller (Clojure team)Iā€™m saying you can s/and additions predicates into a new spec, either at the point of use or registered#2019-02-1823:16don.dwoskeBut isn't the point of schema/select being able to reuse the same schema in different select contexts, so that registering new specs is not necessary. e.g. I want to use the same schema in two contexts 1) makes allow only land rovers and jeeps and 2) makes allow fords and chryslers ... what I'm doing there is wanting to add predicates at the select stage. Am I missing something here?#2019-02-1823:20don.dwoskeIt's not always a linear accretion and narrowing of information - there are forks where one part of the system has different rules for validating cars than another part (e.g. one dealership has different rules than another) but the schema for a car is always the same.#2019-02-1918:51buttergunsWould something like this work in spec2?
(s/def ::model string?)
(s/def ::car (s/keys [::model]))

;; Would "select" the ::model key AND apply additional rules
(s/select ::car [(s/and ::model #(is-made-by-jeep? %))])

(s/select ::car [(s/and ::model #(is-made-by-ford? %))])
#2019-02-1918:52butterguns(pseudo code of course)#2019-02-1919:01Alex Miller (Clojure team)not currently#2019-02-1919:04buttergunsI'm not sure I understand when you say "you can s/and additional predicates any time you like". I want to add additional predicates at "selection-time", like above#2019-02-1919:05Alex Miller (Clojure team)I get what you're asking#2019-02-1919:06Alex Miller (Clojure team)I don't have an answer for you right now#2019-02-1919:08buttergunsNo problem! Was just spit-balling answers for @U4NQYBCU8#2019-02-2018:44don.dwoskeThanks. Just want to be understood and acknowledged, no answers needed... I think you get it. @UFQAPAUU8 example is approx. the thing we want. However, another thing to make clear is that selecting for required/optionality is orthogonal to adding predicates to keys on the schema within a context. A key may still be optional in a spec context - but if the key/value happens to be there, I want to add predicates to that key's spec. Some psuedo-stuff :
(s/def ::id int?)
(s/def ::make string?)
(s/def ::model string?)
(s/def ::color string?)

(s/def ::car (s/schema [[::id ::make ::model ::color]]))

(favorite-car car) needs ::make, ::model
(favorite-car) checks (::make is 'jeep') and (::model is 'wrangler')
(favorite-car) checks ::color is 'red' or 'black' 
For favorite-car, make and model are required, color is still optional. Predicates in the context are added to both required and optional fields to narrow the specification.
#2019-02-1721:23hmaurerSpeaking of schema/select, whatā€™s the current state of this? Is alpha code already available?#2019-02-1721:52Alex Miller (Clojure team)Spec 2 is at https://github.com/clojure/spec-alpha2#2019-02-1721:52Alex Miller (Clojure team)But nothing about select there yet#2019-02-1807:46jaihindhreddyI want to spec a fn target, like this:
(s/fdef target
        :args (s/+ ::a)
        :ret ::a
        :fn #(= (f (:ret %) i)
                (apply g (map (fn [x] (f x i))
                              (:args %)))))
#2019-02-1807:47jaihindhreddyWhere f and g are functions I've written, and i is any integer i.e, for any integer i, this property must hold. Can I express this with spec, or should I drop down to test.check?#2019-02-1813:36misha> Iā€™ve been working on the new select functionality for spec 2 this week ā™„#2019-02-1814:18gfredericks@jaihindh.reddy I think if i is not an arg to target then you can't easily do that with spec; you'd have to maybe write another function target* that also takes i, and you'd only use it for testing, which is a bit weird#2019-02-1912:23sonnytocan anyone answer this question about spec? https://groups.google.com/forum/#!topic/clojure/IPY9YukiLI0#2019-02-1912:56Alex Miller (Clojure team)You need to force eval somehow so macro around it, use eval, etc. this is an area where there are more and different answers in spec 2#2019-02-1912:57sonnytothanks. i used def-impl directly . is this a problem?#2019-02-1913:53Alex Miller (Clojure team)it's fine (although won't work in spec 2)#2019-02-1913:54Alex Miller (Clojure team)there is a new function in spec 2 called register that is similar#2019-02-1916:07borkdudeIs metadata and docstrings on specs considered for spec2? E.g. Iā€™d like to conditionally instrument them based on metadata#2019-02-1916:16Alex Miller (Clojure team)yes#2019-02-1918:29borkdudebananadance#2019-02-1918:33Alex Miller (Clojure team)it was in scope for spec 1 we just never got to it :)#2019-02-2001:02jsa-aerialI have question that more experienced people here will likely think obvious. Say I have a map with some keys, two of which need to have the same value. For example m, {::id v1 ::nm v2 ...}, and I want to enforce (= (m ::id) (m ::nm)). I haven't seen this sort of example, but presumably it is covered by spec?? Thanks in advance for any insight!#2019-02-2001:30jsa-aerialHmmmm, this doesn't give errors, but it doesn't work either:#2019-02-2001:31jsa-aerial#2019-02-2001:40jsa-aerialHmmmm, looks like id-eq-nm? needs to use (m :sid) and (m :snm)#2019-02-2015:42kvltHey all. I'm writing specs for a few controller functions. They take in request maps and hand off those maps to have work done. The issue I'm experiencing is that a collection inside of that (`body-params`) can vary by quite a bit depending on which endpoint is called and I don't like the idea of writing a really loose spec that can work with all of the say, user endpoints. As such: (s/def ::body-params (s/keys :opt [::username ::email ::....])) and would prefer to have specs setup for each call: (s/def ::body-params (s/keys :req [::email] :opt [:...])). Without putting each endpoint into a new file. How do I go about handling this?#2019-02-2017:24johanatanIsnā€™t it possible to have more than one ns form per file?#2019-02-2015:47borkdudeI think this is a problem which spec2 solves#2019-02-2015:48borkdudeyou can watch the latest Rich Hickey talk on youtube if you want to know more#2019-02-2015:48borkdudenot that this is solving your problem right nowā€¦ sorry šŸ™‚#2019-02-2015:52kvltYeah, I saw the talk. I was just kinda hoping that there was something I could do naow#2019-02-2016:07buttergunsCould a multi-spec be used for this?#2019-02-2016:14butterguns
(s/def ::email string?)
(s/def ::username number?)
(s/def ::request-type #{:change-email :show-username})

(s/def ::change-email-request (s/keys :req [::email] :opt [::username]))
(s/def ::show-username-request (s/keys :req [::username] :opt [::email]))


(defmulti request-type ::request-type)
(defmethod request-type :change-email [_] ::change-email-request)
(defmethod request-type :show-username [_] ::show-username-request)

(s/def ::body-params (s/multi-spec request-type ::request-type))

(s/valid? ::body-params {::request-type :change-email ::email "boop"})
=> true
(s/valid? ::body-params {::request-type :change-email ::username 1234})
=> false
(s/valid? ::body-params {::request-type :show-username ::email "boop"})
=> false
(s/valid? ::body-params {::request-type :show-username ::username 1234})
=> true
#2019-02-2016:15buttergunsYou'll have to do a step first that assigns the request-type key to the map, depending on the controller that was callled#2019-02-2020:45guyMight be the wrong place to ask, But with https://github.com/jeaye/orchestra are you still supposed to use, https://clojure.github.io/spec.alpha/clojure.spec.test.alpha-api.html#clojure.spec.test.alpha/check To check the instrumented fdefā€™s?#2019-02-2021:01borkdude@guy as far as I know orchestra only changes instrumentation (it includes ret + fn checks). you still define specs with normal spec, so everything should still work. orchestra uses different namespaces, so it doesnā€™t patch/replace spec itself (I think)#2019-02-2021:02borkdude@jeaye might be able to confirm or correct this#2019-02-2021:07guyThanks!#2019-02-2021:07guyAlso iā€™m a little confused about how to use :fn#2019-02-2021:07guyas part of an fdef#2019-02-2021:08borkdudehereā€™s an example: https://github.com/borkdude/speculative/blob/master/src/speculative/core.cljc#L465#2019-02-2021:08guyAm i supposed to try and link the :args and :ret using :fn ? As in when a certain input is something, then the return should be something, using fn to validate that?#2019-02-2021:09jeayeYep, you still use spec for everything. Just use orchestra to enable instrumentation and consider using defn-spec to clean up your fns.#2019-02-2021:12guyok thanks šŸ™‚#2019-02-2021:09borkdude@guy with fn you can do additional checking on arg + ret and dependencies between them#2019-02-2021:11jeaye:fn is optional, so you'll likely know when you need it.#2019-02-2021:11borkdude@guy e.g. in the group-by spec I linked I did an additional check that group-by should not produce more elements than the input collection had#2019-02-2021:13guyšŸ¤”#2019-02-2021:13guyok#2019-02-2021:15guyI guess it might be an issue with my understanding. Lets say you have a function that takes some args and returns true or false. Would you ever want to use :fn to sort of specify that when the args are nil, you return false#2019-02-2021:15guyIā€™m not sure if thats a bad example, but when i check an instrumented function, i want the args and return to make sense#2019-02-2021:16guyor maybe iā€™m thinking about it the wrong way :shrug:#2019-02-2021:16borkdudeI think in the case of simple predicates fn specs might not be the most helpful. The clojure spec guide might provide you with more helpful ones.#2019-02-2021:17guyIā€™ll read the guide again and check that speculative lib too. Thanks šŸ™‚#2019-02-2022:45droneā¤ļø defn-spec#2019-02-2022:47droneweā€™ve rolled our own for defrecord, but would like to see standard spec-enhanced forms be a thing. but I have a feeling theyā€™re most useful for those of us using spec as a poor manā€™s type system, which is probably something Hickey wants to discourage#2019-02-2023:02Alex Miller (Clojure team)can you explain what you're talking about?#2019-02-2023:03Alex Miller (Clojure team)what is "spec-enhanced forms"?#2019-02-2100:00dronejust like defn-spec adds inlined specs for function return values and parameters, defrecord-spec adds inlined specs for fields, along with adding specs to the built-in record construction functions#2019-02-2100:00droneeh, s/inlined/inplace#2019-02-2100:01Alex Miller (Clojure team)Ah. Rich is considering a spec-enhanced defn.#2019-02-2100:12dronedefn and defrecord are the obvious ones to gain spec-enhanced forms. specs for protocol implementations could also be nice (in defrecord/deftype or extend-type). default specs for protocols (in defprotocol)? protocol related stuff I havenā€™t thought enough about. but have noticed it feels like a ā€œspec gapā€#2019-02-2100:52Alex Miller (Clojure team)Protocols canā€™t be specā€™ed#2019-02-2101:02flyboarder@mrevelle if you keep your protocol an implementation detail and dont expose it as your public api itā€™s not a problem#2019-02-2101:03flyboarderfor example (my-func) could be a specā€™d function and (-my-func) could be a protocol function#2019-02-2101:04flyboarderthen (my-func) calls (-my-func) internally#2019-02-2101:04flyboarderThis is what we did with hoplon#2019-02-2101:42Alex Miller (Clojure team)These days we consider that a best practice in the core team#2019-02-2103:35droneprotocols could be specā€™d#2019-02-2103:40drone@flyboarder yeah, I suppose. not a fan of the redundancy and lack of documented expectations that specs may provide. it can also be useful to allow others to implement your protocols (see loom and ubergraph)#2019-02-2103:58dorabWhat am I doing wrong here?#2019-02-2103:58dorab
user> (require '[clojure.spec.alpha :as s])
nil
user> (require '[clojure.spec.test.alpha :as stest])
nil
user> (require '[clojure.spec.gen.alpha :as sgen])
nil
user> (s/def ::one-arg-fn (s/fspec :args (s/cat :x any?)))
:user/one-arg-fn
user> (defn map-vals
        "Map the function f over all the values of the associative collection coll."
        [f coll]
        (reduce-kv (fn [m k v]
                     (assoc m k (f v)))
                   (empty coll)
                   coll))
#'user/map-vals
user> (s/fdef map-vals :args (s/cat :f ::one-arg-fn :coll associative?))
user/map-vals
user> (def xmap {:a " foo" :b "bar "})
#'user/xmap
user> (stest/instrument)
[user/map-vals]
user> (map-vals clojure.string/trim xmap)
Execution error - invalid arguments to user/map-vals at (REPL:65).                            
(nil) - failed: (apply fn) at: [:f] spec: :user/one-arg-fn                                    
user> 
#2019-02-2104:10Alex Miller (Clojure team)when you use an instrumented fspec, it will actually generate values according to the fspec args spec and invoke the function you pass with them#2019-02-2104:10Alex Miller (Clojure team)here you declared that the fspec function takes an any? arg#2019-02-2104:10Alex Miller (Clojure team)the generator generated nil and invoked clojure.string/trim with it#2019-02-2104:10Alex Miller (Clojure team)which throws#2019-02-2104:12dorabAh. Thanks. So, I should use a s/with-gen in the spec definition of ::one-arg-fn?#2019-02-2104:12Alex Miller (Clojure team)so it's effectively telling you that when passing clojure.string/trim to map-vals, you have passed a function that will not take an any?#2019-02-2104:12Alex Miller (Clojure team)you can#2019-02-2104:12Alex Miller (Clojure team)or instead of fspec, many people find it easier to just use ifn?#2019-02-2104:13Alex Miller (Clojure team)this generative test on instrument of fspec is a feature that many people find surprising (or wrong)#2019-02-2104:13Alex Miller (Clojure team)and something we're going to revisit in spec 2#2019-02-2104:14dorabOK. Thanks for the explanation. What would be the recommended way in spec1 to write a spec for a single arity function?#2019-02-2104:17dorabOr, is the recommendation to just use ifn? as you mention above?#2019-02-2104:28Alex Miller (Clojure team)I would just do that#2019-02-2104:28Alex Miller (Clojure team)with current state, I don't find that I gain anything but headaches with fspecs#2019-02-2104:32dorabThanks.#2019-02-2105:57gklijsWhat would be the best way to prepare for spec 2. I now sometimes use the :opt is it useful to rewrite those to :req and make the values also allow nil?#2019-02-2106:27seancorfield@gklijs s/keys will stay as-is, as far as I know, so you will be able to move to s/schema and s/select piecemeal over time as you need it.#2019-02-2106:28seancorfieldI have our codebase running against spec2 with minimal changes (by depending on the latest git SHA of clojure/spec-alpha2 -- since we use CLI/`deps.edn`) but it's still very much a moving target with no formal release (and still some bugs being worked out).#2019-02-2106:30seancorfieldThe main thing you may trip over is if you have used a predicate in a context that is really expecting a spec -- spec1 allows that but spec2 does not. Also, some constructs are now stricter about what constitutes a "spec".#2019-02-2106:31seancorfieldYour best preparation at this point is to read https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha#2019-02-2106:31Alex Miller (Clojure team)the fate of s/keys is not yet determined#2019-02-2106:31seancorfieldOooh... so it may go away before spec2 is "released"?#2019-02-2106:32Alex Miller (Clojure team)maybe, but really don't know yet#2019-02-2106:32seancorfieldThat would make the migration from spec1 to spec2 a lot more work!#2019-02-2106:32Alex Miller (Clojure team)at the moment I'd say it's more likely to still be there#2019-02-2106:33Alex Miller (Clojure team)the idea behind schema has shifted a lot since Rich's talk, not sure if it will survive in the form described then#2019-02-2106:33Alex Miller (Clojure team)he's deep in the hammock :)#2019-02-2106:33seancorfieldInteresting... I look forward to seeing how this evolves then...#2019-02-2106:34seancorfieldAnd, yeah, I feel that hammock... I'm still deep in thought about next.jdbc šŸ™‚#2019-02-2106:34Alex Miller (Clojure team)@gklijs I would not rewrite opts to nilable reqs - I think that's the wrong direction#2019-02-2106:34Alex Miller (Clojure team)really, unless you're a "riding the bleeding edge" guy like Sean, I would just wait#2019-02-2106:35Alex Miller (Clojure team)select will have both required and optional selections#2019-02-2106:36Alex Miller (Clojure team)although the semantics around optional stuff is a little different#2019-02-2106:39gklijsOk, I'll leave it for now then. I now use it as part of (de)serialisation, with optional added defaults on a subset of what can be specced, also using the spec's to generate the cljs functions to edit the data. I probably need to write some sort of migration at a point, but that should be trivial. Might use another namespace to spec the existing data in spec2 at some point.#2019-02-2115:26guyIf you have two specs but you want to have a shared id between the two, is that possible in a generator? But also the idā€™s are generated as well. šŸ‘€#2019-02-2115:29borkdudeexample please?#2019-02-2115:31mishayou'll need custom generator, which probably will have to generate examples for both specs at the same time. Another funky option is to override default id gen to generate predefined small set of ids, and then filter out examples generated by "default" gens which happen to share an id#2019-02-2115:31mishayou'll need custom generator, which probably will have to generate examples for both specs at the same time. Another funky option is to override default id gen to generate predefined small set of ids, and then filter out examples generated by "default" gens which happen to share an id#2019-02-2115:32guythanks! that makes sense#2019-02-2115:34mishafirst option would look like: 1) gen set of ids, 2) gen set of obj1 with default gen 3) gen set of obj2 with default gen 4) override ids in pairs of obj1 obj2 with ids from 1)#2019-02-2115:35mishathis will allow you to leverage default spec gens w/o any modification, and just build higher level one#2019-02-2115:37mishawriting a gen for entire family of interdependent objs is hard -_-#2019-02-2201:35jsa-aerialSo, what is the status of CLJ-2320? In my case I can limp along with s/merge, but this is (according to the issue itself) a 'major' priority but does not appear to have had any activity since last June. OTOH, it doesn't have any votes either. I don't have an account so can't log in to vote anyway...#2019-02-2201:37jsa-aerialAlso, why would what is effectively a 'logic system' use short circuiting semantics for an 'and' operator???#2019-02-2203:22seancorfield@jsa-aerial Anyone can create a JIRA account, I believe...#2019-02-2203:23seancorfieldI think s/and really has to be short-circuiting since some predicates may blow up if they don't have a guard predicate ahead of them...?#2019-02-2216:07jsa-aerial@U04V70XH6 Sounds like a likely explanation, but in this context that 'should not happen'. Put another way, that should be up the user's predicates. If a 'guard' operator is wanted - call it 'guard' or some such.#2019-02-2208:09Christian JohansenIs there any way to associate a docstring with a spec?#2019-02-2208:10Josip Gracinnot in spec1, AFAIK#2019-02-2208:10Christian Johansendo you know if it's planned for 2?#2019-02-2208:12Josip GracinI should probably let Alex answer that, but I believe it will probably be in spec2#2019-02-2208:15Josip Gracinmy belief is based on previous discussions of this on slack and mailing lists#2019-02-2208:26Christian Johansenok, cool. I haven't followed either of those closely, so crossing my fingers šŸ™‚#2019-02-2208:27mishahttps://clojurians.slack.com/archives/C1B1BB2Q3/p1550592458214700 https://clojurians.slack.com/archives/C1B1BB2Q3/p1550601206215400#2019-02-2209:06Christian Johansenah, cool, thanks!#2019-02-2213:22slipset@christian767 I think @bbrinck has done some trickery in expound wrt docstrings on specs....#2019-02-2213:23slipsetNo, that was custom error messages. #2019-02-2213:25Christian Johansenok šŸ™‚#2019-02-2221:38kennyIs there a good way to debug slow generators when running a Spec check? #2019-02-2310:42misha@kenny https://www.youtube.com/watch?v=FihU5JxmnBg#2019-02-2310:42mishaĀÆ\(惄)/ĀÆ#2019-02-2316:23adc17Is there a way of following progress on the s/schema/`s/select` functionality that Rich mentioned in his talk? I can't see an issue tracking this on Jira (I also can't see spec in the clojure-contrib libraries)?#2019-02-2316:33misha@alexchalk17 https://github.com/clojure/spec-alpha2 and http://insideclojure.org/#2019-02-2316:48Alex Miller (Clojure team)clojure-contrib is very old and deprecated#2019-02-2421:04eoliphantHi, trying to muddle through a modeling scenario in the context of spec. In one of my domains I work with grants. so a grant has a unique :grant/number thatā€™s a string of some format ā€œGRT-999ā€, etc., string? + regex and Iā€™m good. But i also have to deal with the fact that various ā€˜issuersā€™ of grantā€™s have their own formats. Issuer A, uses ā€œGRANT-999", B uses ā€œG_999ā€,etc. So, Iā€™m grappling with somehow providing ā€˜contextā€™ to the spec. Or are these better modeled as different things entirely :issuer-a/grant-number, :issuer-b/grant-number?#2019-02-2421:09borkdudeI can imagine that a grant has a :grant/issuer and based on that value you can verify if the :grant/number is valid, possibly using s/and or maybe using a multi-spec?#2019-02-2507:31misha#2019-02-2507:34misha@eoliphant ^^^ this is a bit simplified, as you probably will not have issuers as keyword, but it can be expanded with mapping of issuer to number format.#2019-02-2507:36mishayou'd also need to customize generators a bit, if you need them.#2019-02-2510:49Audriuswhat if to use Spec with regular (non namespaced) keywords? should it work OK?#2019-02-2510:49Audriusis there a too t generate spec from data structure?#2019-02-2510:51borkdude@masta I believe spec-tools has such a tool#2019-02-2511:49misha@masta read through this chapter https://clojure.org/guides/spec#_entity_maps#2019-02-2511:49mishathere is https://github.com/stathissideris/spec-provider#2019-02-2514:08eoliphantThanks @misha, @borkdude. One other bit of fun is that at the end of the day, the number of issuers/formats could be large and might end up in a db#2019-02-2514:35rickmoynihanIs it possible to turn clojure/core.specs off during macroexpansion? So you can see the failing expansion?#2019-02-2514:37Alex Miller (Clojure team)https://clojure.org/guides/faq#skip_macros#2019-02-2514:37rickmoynihanšŸ‘#2019-02-2514:37rickmoynihanthanks#2019-02-2514:39rickmoynihanIā€™m assuming thereā€™s a reason this couldnā€™t also be a set!able var? Not that restarting my REPL is that difficult! šŸ™‚#2019-02-2514:40Alex Miller (Clojure team)well it's in a system property as stuff is potentially checked while you're loading before you have a chance to set it#2019-02-2514:41Alex Miller (Clojure team)it could probably also be a dynvar (although that would need to be checked on every macroexpansion, so probably some perf impacts)#2019-02-2514:46Audrius
(spec/fdef data->graph
  :args ::data
  :ret ::graph)
why this does not work? I mean it will take for ever to start...
#2019-02-2514:47Alex Miller (Clojure team)"not work" == ?#2019-02-2514:51Audriusprocess starts for ever#2019-02-2514:54Alex Miller (Clojure team)is data->graph a macro or a function?#2019-02-2514:55Alex Miller (Clojure team)if a function, it shouldn't do anything until you instrument#2019-02-2514:55Audriusit is a function...#2019-02-2515:09Audriusstill running šŸ¤” maybe it is too complex for Spec to handle...#2019-02-2515:10Audrius
(spec/def ::graph (clojure.spec.alpha/def
                    :importer.datamodel/graph
                    (clojure.spec.alpha/coll-of
                     (clojure.spec.alpha/or
                      :collection
                      (clojure.spec.alpha/coll-of
                       (clojure.spec.alpha/keys
                        :req-un
                        [:importer.datamodel/label]
                        :opt-un
                        [:importer.datamodel/center
                         :importer.datamodel/edge_id
                         :importer.datamodel/global_id
                         :importer.datamodel/name
                         :importer.datamodel/source
                         :importer.datamodel/target
                         :importer.datamodel/typ]))
                      :simple
                      clojure.core/keyword?))))

(spec/def ::data (clojure.spec.alpha/def
                   :importer.datamodel/data
                   (clojure.spec.alpha/coll-of
                    (clojure.spec.alpha/or
                     :collection
                     (clojure.spec.alpha/coll-of
                      (clojure.spec.alpha/keys
                       :req
                       [:importer.datamodel/global-id]
                       :opt
                       [:importer.datamodel/center
                        :importer.datamodel/part-of
                        :importer.datamodel/type]
                       :opt-un
                       [:importer.datamodel/attributes
                        :importer.datamodel/datasource
                        :importer.datamodel/features
                        :importer.datamodel/ioi-slice
                        :importer.datamodel/name
                        :importer.datamodel/url]))
                     :simple
                     clojure.core/keyword?))))

(spec/fdef data->graph
  :args ::data
  :ret ::graph)
#2019-02-2515:10AudriusI have this...#2019-02-2515:10Audriusand runs for ever to start...#2019-02-2515:23guyHave you tried (exercise ::data 3) to see if outputs anything?#2019-02-2515:29Audriusalso runs for ever...#2019-02-2515:31AudriusFound it...#2019-02-2515:31Audrius(spec/def ::data (clojure.spec.alpha/def is recursive or something šŸ˜„#2019-02-2519:52jrwdunhamHi here! I'm hoping someone can help me out with what is probably a basic question about clojure spec#2019-02-2519:52jrwdunhamI have this function:#2019-02-2519:52jrwdunham
(defn starts-with-any?
  "Return truthy if s starts with any character in chrs; otherwise nil."
  [s chrs]
  (seq (filter (fn [c] (string/starts-with? s (str c))) chrs)))
#2019-02-2519:52jrwdunhamand this spec:#2019-02-2519:52jrwdunham
(s/fdef starts-with-any?
  :args (s/cat :s string? :chrs string?)
  :ret (s/nilable (s/coll-of char?))
  :fn (s/or :nil-case #(-> % nil?)
            :non-nil-case
            (s/and #(= (-> % :ret set first) (-> % :args :s first))
                   #(= 1 (-> % :ret set count))
                   (fn [x] (subset? (-> x :ret set) (-> x :args :chrs set))))))
#2019-02-2519:53jrwdunhamif I run (stest/check starts-with-any?)` on this I get a failure#2019-02-2519:53Alex Miller (Clojure team)#(-> % nil?) == nil? btw#2019-02-2519:53jrwdunhamyeah, thanks, I'll change that#2019-02-2519:54borkdude> I get a failure. needs more detail.#2019-02-2519:54jrwdunham
{:clojure.spec.alpha/problems ({:path [:fn :nil-case], :pred (clojure.core/fn [%] (clojure.core/-> % clojure.core/nil?)), :val {:args {:s "", :chrs ""}, :ret nil}, :via [], :in []} {:path [:fn :non-nil-case], :pred (clojure.core/fn [%] (clojure.core/= 1 (clojure.core/-> % :ret clojure.core/set clojure.core/count))), :val {:args {:s "", :chrs ""}, :ret nil}, :via [], :in []}), :clojure.spec.alpha/spec #object[clojure.spec.alpha$or_spec_impl$reify__2046 0x3d686af1 "
#2019-02-2519:54jrwdunhamit's failing on a nil ret, but shouldn't it pass on the first branch of s/or?#2019-02-2519:55borkdudeitā€™s failing on the path [:fn :nil-case], so that would be the first branch#2019-02-2519:55jrwdunhamoh right, and it needs :ret#2019-02-2519:55jrwdunhamd'oh#2019-02-2519:56Alex Miller (Clojure team)yeah#2019-02-2519:56jrwdunhamhaha, thanks. sorry for the typo question#2019-02-2519:57Alex Miller (Clojure team)duckie#2019-02-2522:53robertfwIs there a recommended/succinct way to reuse a string regex spec in keyword form? That is, I have a spec that uses a regex to define a string format (in this case, it's an ID format). In some places, that value is used as a keyword in a map. I could break out the specifics, but was wondering if there is an easy way to do something in the spirit of (s/def ::id-as-keyword (as-keyword ::id-spec))#2019-02-2612:12Audriushow to make spec for list of maps? '({}{}{}) ?#2019-02-2612:12mpenet(s/coll-of map?)#2019-02-2612:13mpenetbut most of the time you want something more precise than map?#2019-02-2612:13Audriusbut how to make that the maps also conforms to a spec?#2019-02-2612:15mpenetreplace map? with a namespaced keyword that points to a spec#2019-02-2612:15mpenetor a spec directly#2019-02-2615:53jsa-aerialAny reason why (s/explain-data a-spec a-value) does exactly what is expected (and is correct) and given the exact same input (s/valid? a-spec a-value) fails with a class cast exception? This is on 1.9#2019-02-2615:54borkdudedo you have the spec and example data? did you test in clj 1.10?#2019-02-2616:08favila@jsa-aerial https://dev.clojure.org/jira/browse/CLJ-2372 maybe?#2019-02-2616:45jsa-aerial@favila doesn't look to be that (might be peripherally related). I think the issue is related to a misuse of s/merge in my case. I have a set of predicates def'd to specs. None of these are map/key related (like with s/keys) so none involve a map. Then I use s/merge (because s/and short circuits...) on these. Now, s/merge is happy to do this. s/explain-data is happy to do 'the right thing' and check all predicates of the merged spec. But s/valid? has a bug as well, but instead of 'doing the right thing' it just blows up. I put 'right thing' in quotes because I suppose s/merge should not allow this in the first place. All of this is a shame - s/and should not be short circuiting in the first place. This is a logic system after all. If you are concerned about guarding flow through to other predicates then your predicates are what's broken. I suppose you could have a guard operator for that sort of case, but IMO, that would still be bogus. ĀÆ\(惄)/ĀÆ#2019-02-2616:46favilawhy would you want s/and not to short-circuit?#2019-02-2616:48jsa-aerialBecause you are saying that the spec should satisfy all predicates. Like in propositional logic. Not programming languages#2019-02-2616:49jsa-aerialThe reason this is really useful is because you would like to catch all the errors in one go - not get one report to user. fix that one, get the next, report to user, etc. That is really annoying and just wrong#2019-02-2618:51favilabut s/and accepts specs and conformers#2019-02-2618:51favilanot just predicates#2019-02-2618:51favilawell, nm just conforming is a problem#2019-02-2618:52favilaso you would have to drop conformers, or introduce some trickery about evaluation#2019-02-2618:52favilaeven if all were evaluated order would still have to matter#2019-02-2618:55seancorfield@jsa-aerial s/and flows data through all the specs so it must be short-circuiting -- it has an inherent order and you can't rely on being able to apply subsequent specs if an earlier spec fails. (and I feel we've had this conversation before)#2019-02-2619:13jsa-aerialIt is what it is - I think if you wanted that 'guarded flow` capability that another name would have been far better. I don't really care TBH, I just think it was a poor choice.#2019-02-2619:04mishaYeah, each next spec in s/and receives conformed value from prior one. So if one step is invalid ā€“ next would just blow up with random noise instead of useful explain data.#2019-02-2619:09mishaotherwise how would you s/conform to s/and spec? conform only last step? none? Most of the s/and specs I saw ā€“ check fields first, and then some relation between them next (e.g. this id and that id are the same.), which is basically: 1st step is conforming, last is not#2019-02-2619:09favilaif s/and were restricted to predicates, and predicates were written defensively (which they should be IMO), then I think what @jsa-aerial proposes would be fine; order could be used for making generators, but all could be checked in parallel#2019-02-2619:10mishahow do you restrict to predicates only?#2019-02-2619:10favilaI mean if conformers were not legal#2019-02-2619:11favilabascially, you would have to drop conformers as a feature of s/and#2019-02-2619:11mishawell, this excludes s/keys opieop#2019-02-2619:11mishaor anything containing s/or#2019-02-2619:12favilaI don't follow?#2019-02-2619:13favilaI'm just talking about s/and. Maybe for clarity I should say this would not be s/and anymore. s/nonconforming-and or s/parallel-and#2019-02-2619:13favilas/all maybe#2019-02-2619:14mishaI'd call it s/every or something. But "flowing" behavior was surprising to me back then, I agree#2019-02-2619:15mishabut, how would such s/every be more useful than clojure.core/and?#2019-02-2619:15favilaevery predicate would be tested unconditionally#2019-02-2619:15favilaI should say, could be#2019-02-2619:16borkdudeis this for form validation or something?#2019-02-2619:16favilaI'm guessing#2019-02-2619:16mishaok, clojure.core/every-pred#2019-02-2619:16favilasomething user-facing#2019-02-2619:18jsa-aerialNot necessarily, could be anything. You just want to be able to report all findable problems at once - not piecemeal#2019-02-2619:16favilabut I haven't found spec very useful for things like that#2019-02-2619:18jsa-aerialActually it works amazingly good when coupled with phrase#2019-02-2619:16mishawhat extra would it provide?#2019-02-2619:16favilareport all errors instead of just the first one#2019-02-2619:17borkdudemaybe build your tool on top of spec, but not use s/and to collect all the errors. just call s/explain for each field for example#2019-02-2619:17mishayou can't exercise it, can you? because it involves conforming#2019-02-2619:17favilaconforming and excercising seem orthogonal?#2019-02-2619:18mishaexercise gives you pairs of [generated conformed], I think?#2019-02-2619:19favilathe conformed of s/every would always be identity#2019-02-2619:19favilaor whatever, s/every-pred#2019-02-2619:19mishaI think you'll be able to implement such s/every on top of spec2#2019-02-2619:19favilano conforming#2019-02-2619:20favilaI'm more curious what the explain-data would look like#2019-02-2619:20mishaok, what if one of the items in s/every would be spec with s/and inside? it'll propagate conformed value, I think#2019-02-2619:21seancorfieldIt's worth pointing out that Spec2 draws a harder line between specs and predicates -- and s/and accepts specs, not arbitrary predicates. See https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha#2019-02-2619:21favilathe point is, this spec would simply not do that#2019-02-2619:21mishawell yeah, explain - "conforms"#2019-02-2619:23seancorfield
(! 591)-> clj
Clojure 1.10.0
user=> (require '[clojure.spec.alpha :as s])
nil
user=> (s/def ::foo (s/and int? (partial > 10)))
:user/foo
user=> (s/valid? ::foo 10)
false
user=> (s/valid? ::foo 1)
true
user=> ^D

Tue Feb 26 11:22:41
(sean)-(jobs:0)-(~/clojure)
(! 592)-> clj -A:spec2
Clojure 1.10.0
user=> (require '[clojure.spec-alpha2 :as s])
nil
user=> (s/def ::foo (s/and int? (partial > 10)))
:user/foo
user=> (s/valid? ::foo 10)
Execution error (IllegalArgumentException) at clojure.spec-alpha2/spec* (spec_alpha2.clj:197).
No method in multimethod 'create-spec' for dispatch value: clojure.core/partial
user=> 
#2019-02-2619:23misha@seancorfield I don't see "not arbitrary predicates" there. It says "Qualified symbols (predicate function references)" are ok#2019-02-2619:23seancorfield@misha See the code I just posted above.#2019-02-2619:24mishawhat if you defn it first, instead of partial?#2019-02-2619:25favilaprobably it needs an fspec#2019-02-2619:25favila(just guessing)
#2019-02-2619:25favilawhich sounds great to me#2019-02-2619:25borkdude
(s/def ::foo (s/and #(int? %) #(> 10 %)))
works in spec-alpha2
#2019-02-2619:26mishaI think it should be orthogonal: fdef and predicate fn#2019-02-2619:26seancorfieldYes, you can def a predicate from (partial > 10) and then it falls into the set of things that can be treated as a symbol spec. But the point I'm making is that s/and isn't intended to operate on predicates, it's for specs.#2019-02-2619:26mishain spec2? or in both 1 and 2?#2019-02-2619:26seancorfieldWe got our entire codebase up and running on Spec2 and we had several places where we had "mixed" specs and predicates and those were rejected and we had to rewrite things somewhat.#2019-02-2619:27favilaso a def-ed predicate is a spec#2019-02-2619:27mishamost of my use cases for s/and were s/keys + fn#2019-02-2619:31jsa-aerialIf you have s/keys you can use s/merge which does satisfy all semantics#2019-02-2619:27favilawell symbolic spec, not spec object#2019-02-2619:27borkdude@favila see above. anonymous functions are allowed, but you have to write them with fn or #(ā€¦ %)#2019-02-2619:28favilathat doesn't jive with the doc @seancorfield linked#2019-02-2619:28borkdudeTo be honest, I found it confusing too, but thatā€™s how it works now#2019-02-2619:28mishaalso, my impression was, that any predicate is sort of first class in spec by design#2019-02-2619:29seancorfieldIt's an important (but subtle) distinction. Spec1 doesn't enforce it but Spec2 does.#2019-02-2619:29favilawhat about michiel's counter-example?#2019-02-2619:30seancorfieldWhat counter-example?#2019-02-2619:30borkdude(fn [x] int?) is a ā€œsymbolic specā€, itā€™s just what they call it now#2019-02-2619:30favila"fn" is a spec form?#2019-02-2619:30borkdudebut just simple symbol isnā€™t, like int?#2019-02-2619:30mishaI see how spec2 could disallow fns returning fns as predicates (partial), but forbidding anonymous fns would be strange (and doing it only for few spec forms ā€“ just awkward)#2019-02-2619:31borkdudeI think this will be a source of much confusion.#2019-02-2619:31seancorfieldint? is acceptable as a symbolic spec.#2019-02-2619:31favila> Spec forms (lists/seqs), with spec op in function position, composed of other symbolic specs > Qualified keywords (names that can be looked up in the registry) > Qualified symbols (predicate function references) > Sets of constant values (an enumeration)#2019-02-2619:32favilaunless "fn" is a spec form, I don't see how anonymous inline fns would be allowed#2019-02-2619:32favilaeven then that seems iffy#2019-02-2619:32misha@jsa-aerial s/merge is for "map-validating specs", it's not for things like "also check that value in [:foo :bar] is the same as in [:baz]" but s/and - is exactly for that#2019-02-2619:33jsa-aerialYes, I know that - just saying it does satisfy all semantics#2019-02-2619:33mishawhich "all semantics"?#2019-02-2619:34seancorfieldHe means "not short-circuiting"#2019-02-2619:34jsa-aerialIf you merge a bunch of specs it check them all#2019-02-2619:34mishaI meant, it is (s/and keys-spec custom-pred) or (s/and regex-spec custom-pred)#2019-02-2619:36mishakeys-spec could be (s/merge keys-spec1 keys-spec2 ...), yes. but important part is a custom-pred, which is usually (for me) goes last, and does not conform anything, but often requires conformed input, in case there are branches with different "postvalidation" semantics required#2019-02-2620:14ikitommia new guide on coercion with spec1 and spec-tools: https://cljdoc.org/d/metosin/spec-tools/0.9.0/doc/spec-coercion#2019-02-2714:58AudriusCan I have spec for a map entry that it has keyword, but corresponds to a different spec? I have such map {::value "" ::other-map {::value {::bla 1}}} so my ::value sometimes is string sometimes is map...#2019-02-2714:58AudriusCan I have spec for a map entry that it has keyword, but corresponds to a different spec? I have such map {::value "" ::other-map {::value {::bla 1}}} so my ::value sometimes is string sometimes is map...#2019-02-2715:32favilaNo. Spec is philosophically opposed to this distinction: https://clojure.org/about/spec#_map_specs_should_be_of_keysets_only#2019-02-2715:32favilaYou could write your own spec form (hard) but it will never be built in.#2019-02-2715:33favilaI wrote one out of necessity, but I don't necessarily recommend using it: https://gist.github.com/favila/ab03ba63e6854a449d64d509aae74618#2019-02-2715:36favilaMy spec isn't going to help in your case#2019-02-2715:36favilaIf you can, you should change your map to use a different keyword#2019-02-2715:36favilaor, it's possible you should be specing a higher-level language#2019-02-2715:32valeraukowhy would you want to do that?#2019-02-2715:35AudriusI amended my question...#2019-02-2715:33favila#2019-02-2715:34favila@vale Real World Data (tm) has fields whose type may change depending on what map it is in#2019-02-2715:34favilaI appreciate and understand the philosophical stance of spec here, but it does cause friction#2019-02-2715:38favilaThe case I always seem to run in to is a field which has a baseline type (e.g. it should be number?) wherever it appears, but in certain kinds of maps (defined with s/keys) it must be some subset of its allowed values (e.g. even?)#2019-02-2716:01valeraukoyou could namespace those specific use cases#2019-02-2716:02valeraukolike :map-with-strings/name and :map-with-whatevers/name#2019-02-2716:02valeraukoand then :key-un to ignore the namespace#2019-02-2716:05valerauko::outer-map/value string? ::inner-map/value (s/keys ...)#2019-02-2716:06valeraukothat is imo the most spec-friendly way to say "a value as in outer-map and value as in inner-map"#2019-02-2716:10favilayeah, so basically put a transformation in front of all these functions#2019-02-2716:10favilalike I said, friction#2019-02-2716:11favila@audrius if you control the keys and the shape of your data, you should use different keys (even just namespaces) as @vale suggests#2019-02-2716:12favilathe spec philosophy of "one key one type" is a worthy goal#2019-02-2716:12valeraukono, you don't have to have different keys in your data#2019-02-2716:12valeraukojust spec them in different namespaces#2019-02-2716:12favilathis forces the concrete maps to use un-namespaced keys#2019-02-2716:14valeraukowhy would you have two names in the same namespace refer to multiple different things#2019-02-2716:14valeraukoyou're right i don't know your use case#2019-02-2716:14favilait's pretty easy to encounter data shaped like this#2019-02-2716:15favilae.g. datomic transaction maps#2019-02-2716:15valeraukoi'll let smarter people argue the point#2019-02-2716:18lilactownAn example would be like we are trying to spec a userā€™s medical records. They have a coverage type: :coverage/type that can be #{:medical :dental :vision} and then in a certain context, you want to specify it must be :medical along with some other facts about the userā€™s data#2019-02-2716:18lilactownright?#2019-02-2716:18favilathat is exactly the case I encounter constantly#2019-02-2716:18favilabut it is different from the OP's case which is more extreme#2019-02-2716:19favilahere's an extreme example for a datomic transaction map: {:db/id (string, or tempid-record, or entity-id-long, or keyword-ident, or tuple[entity-id-long or keyword-ident, indexable-datomic-valuetype)}#2019-02-2716:19favilaand suppose for a particular function you want to specify that no tempids are legal for :db/id#2019-02-2716:19favilayou can't respec :db/id contextually#2019-02-2716:21favilayou can add a predicate, but you have the burden of maintaining the predicate, a less useful s/merge, and custom generators#2019-02-2716:21favilaor, you can make a new field and remap key names on the way in#2019-02-2716:22favila(i.e. transform your data for typing reasons)#2019-02-2716:25lilactownyeah. atm we have a specific medical-coverage? predicate that we combine with others. itā€™s not the greatest#2019-02-2716:36favilaThat gist I linked earlier may help you#2019-02-2716:36favilait was written exactly for the use case you described#2019-02-2716:38favila(keys+ :req [:coverage/type] :conf {:coverage/type #{:medical}}) would produce an s/keys spec where :coverage/type was checked both against its natural spec and any override you have in :conf, and :conf is the one used for generators#2019-02-2716:40lilactownthat's awesome! thanks, I'll check it out#2019-02-2716:54dronefor things related (but not the same) to this, we use different records with specs. the records may have overlapping keys, and records of different kinds can be determined by their record (really, class) type#2019-02-2721:07hiredmanthat sort of sounds like a multi spec#2019-02-2721:07hiredmankeyed on :coverage/type#2019-02-2721:53lilactownthe coverage type might be 1 part of many keys#2019-02-2721:56lilactownex: {:coverage/type :medical :coverage/code 1234 :coverage/status :active} answers "does user have XYZ program", for example#2019-02-2723:23dronemulti-spec uses a multi-method to retrieve the correct spec based on whatever dispatch function you choose. so you could dispatch on the value of :coverage/type and return different specs for each of your coverage record ā€œtypesā€#2019-02-2723:24favilathat's not the goal here#2019-02-2723:24dronethere seem to be at least two different things being discussed#2019-02-2723:24dronewhat is the goal#2019-02-2723:25favilaWill and I were discussing the situation where we want to spec a map where an entry is only allowed some subset of what is normally allowed#2019-02-2723:26favila:coverage/type doesn't alter the type of the map (what multispec would allow)#2019-02-2723:26favilathis is "my function takes coverage maps but only :medical ones"#2019-02-2723:27favilathere's also an open vs closed tradeoff here#2019-02-2723:27favilamultispec can only be open#2019-02-2723:30drone> spec a map where an entry is only allowed some subset of what is normally allowed you mean a coverage map tagged as :medical can only contain some subset of all coverage map keys?#2019-02-2723:31favilano#2019-02-2723:31favilawe mean there's a map type "coverage", one of its keys is coverage/type which can be #{:medical :dental} this function only deals with :coverage/type = :medical#2019-02-2723:33droneand specā€™ing the function with medical-coverage? (mentioned above) is too much work?#2019-02-2723:34dronegiven this data model, Iā€™m not seeing how else youā€™re proposing to check this aside from looking at the tag?#2019-02-2723:35favila(s/and ::coverage #(= (:coverage/type %) :medical)) you mean?#2019-02-2723:35favilathis is awkward is all#2019-02-2723:35favilayou need to override the generator too#2019-02-2723:35favilayou need to chain a bunch of one-off predicates if you have more subset constraints#2019-02-2723:36favilait would be nice to override in s/keys directly#2019-02-2723:36favilaI wrote something hacky that does this#2019-02-2723:36favila(keys+ :req [:coverage/type] :conf {:coverage/type #{:medical}}) is what it looks like#2019-02-2723:37favilathe spec in the :conf map is used for conforming and generation#2019-02-2723:37favila(oh, also you can't s/merge s/and)#2019-02-2723:38favilaanother alternative is key rewriting#2019-02-2723:39favila:coverage/type -> :coverage/type=medical, then you can (s/keys :conf [:coverage/type=medical), but that just shifts the pain around#2019-02-2723:41droneso, why not just have specs to represent each of the kinds of coverage: medical-coverage, dental-coverage, ... and then a type that represents the common, generic coverage?#2019-02-2723:41drones/type/spec#2019-02-2723:42favilathe constraint is, :coverage/type is the concrete key#2019-02-2723:42favilawhen you have a coverage map, it must have :coverage/type as the key#2019-02-2723:42favilabut different functions want different values for that key#2019-02-2723:43favilaso you can either chain predicates to your s/key with s/and (and lose niceties like s/merge, automatically correct generators, fewer predicates), or you can rewrite your key so that type and value match (and end up with key rewriting layers everywhere)#2019-02-2723:44favilawhat you cant do is say "for this keys spec, only allow some of what would normally be in :coverage/type"#2019-02-2723:52lilactown@mrevelle the point is that I don't actually care about the coverage type. what I do care about is answering some higher-level question about the data#2019-02-2723:52lilactownlike I might want a spec that is called :user/super-cool-program#2019-02-2723:53drone@favila ah, thank you. I am tracking now#2019-02-2723:53lilactownand their coverage map should look like:
{:coverage/code 1234
 :coverage/type :medical
 :coverage/status :active}
#2019-02-2723:53favilaS/keys blurs the lines between a spec and a predicate somewhat #2019-02-2723:54lilactownthe code, type and status tell me that the user has "Super Cool Program"#2019-02-2723:54favilaItā€™s a fancy composable predicate system for map-shaped data#2019-02-2723:55favilaBut you either obey its ā€œspec key = typeā€ constraint or you lose all the niceties it provides#2019-02-2800:03dronemaybe spec is trying to do too much. itā€™s acting as a specification for: contracts, test value generators, and data coercions#2019-02-2800:06dronewhen you make use of chaining predicates to solve your problem, you can express the contract youā€™d like. but at the cost of support for the other areas#2019-02-2800:08droneI use spec only for contracts, so hadnā€™t experienced the pain#2019-02-2800:31droneitā€™s ironic that the argument for spec over any kind of static type system leans significantly on expressiveness. but with that expressiveness (i.e., arbitrary predicates) comes tradeoffs that make difficult to accomplish some of the other goals of spec#2019-02-2802:05misha@lilactown I think your :user/super-cool-program (or something as specific as such fields combination) should be just predicate with a custom generator.#2019-02-2802:09misha@favila what is the downside (and is there any?) of always having conformed datomic's :db/id tagged, like [:lookup-ref [:user/name "foo"]] or [:perm-int 12345]. Where do you actually need to constantly check against e.g. not-a-temp-id, so you'd be forced to come up with funky qualifying s/and, etc.?#2019-02-2802:11favilaA function may only work with lookupable db ids#2019-02-2802:11mishaofc, I'd love to always get conformed {:db/id 1234} out of queries/pulls. But is it a high price?#2019-02-2802:12mishathat means you'd need to conform first, and have function working with conformed map, right?#2019-02-2802:12misha... or conform db/id within fn#2019-02-2802:13mishado you actually have an example of such fn? can't think of one from the top of my head#2019-02-2802:13favilaTyping the result of pull is easy#2019-02-2802:13favilaItā€™s consistent#2019-02-2802:14favilaTyping a map intended for a tx is harder because youā€™re typing a particular instance of a dsl #2019-02-2802:15favilaBut thatā€™s the issue right there#2019-02-2802:15mishahow is it different, given you specced db/id as s/or? oh wait. db/refs -_-#2019-02-2802:16favila:db/id or :my/ref-attr has different types #2019-02-2802:16mishayeah, db/refs are pain. even in pull btw.#2019-02-2802:16favilaDepending on whether it is produced by a pull or embedded in the map of a TX Map dsl #2019-02-2802:17mishayeah, does this mean entire app's specs will know about datomic (app uses datomic and the :user/friends is a :db.type/ref)? or is there another way?#2019-02-2802:18favilaWell thatā€™s exactly the point#2019-02-2802:19favilaSome functions deal with :db/id (just to pick one attr) in a pull result context #2019-02-2802:19favilaI those causes they want to assert such values passing through them are always Kongā€™s#2019-02-2802:19favilaLongs #2019-02-2802:20Alex Miller (Clojure team)you don't have to spec every attribute#2019-02-2802:21favilaIn other map contexts or parts of the app the same attr means something different (eg itā€™s meant for a tx later, or it can be anything d/entid can resolve)#2019-02-2802:25mishaAlex, true. I need to think through what I will not get if I omit :db.type/refs from spec registry, e.g. generative testing or validation -wise#2019-02-2808:32Josip GracinIs there any reason the following two definitions would not be equivalent? I'm having some problems with the one using partial.#2019-02-2808:54borkdude@josip.gracin Works for me:
user => (defprotocol Foo
  (foo [this]))
Foo
user=> (extend-type java.lang.Long Foo (foo [this] this))
nil
user=> (s/valid? :n/k 1)
false
user=> (satisfies? Foo 1)
true
user=> (s/valid? :n/k 1)
false
user=> (s/def :n/k (partial satisfies? Foo))
:n/k
user=> (s/valid? :n/k 1)
true
#2019-02-2808:55borkdudeit could be that after you change the protocol, you have to re-define the spec. to be sure, just reload your program entirely#2019-02-2808:57Josip Gracinthanks! I just needed a sanity check before I go further into debugging. I have a situation where those two behave differently.#2019-02-2810:12misha@josip.gracin there are some nuances between the 2 in spec2: read ~10-20msgs starting from https://clojurians.slack.com/archives/C1B1BB2Q3/p1551208861176100#2019-02-2810:13mishamight be related, if you use spec2#2019-02-2810:55Josip Gracin@misha thanks! I'm using spec1 for this.#2019-02-2810:56borkdudespec2 isnā€™t really finished, so not recommended to use yet I think#2019-03-0112:08djtangois there a way to provide a generator to a spec you want to use at runtime for validation, but have test.check only as a dev/test dependency?#2019-03-0112:09djtangocan you augment a spec in the registry in a separate namespace?#2019-03-0112:13gfredericks@djtango the whole point of spec's proxy namespace for generators is to allow that#2019-03-0112:13gfredericksyou reference/build the generator using that namespace, and as long as you don't use it, t.c isn't needed#2019-03-0112:15djtangohmm I must be doing something wrong#2019-03-0112:15gfredericksI think any function calls in that ns have to be delayed inside a (fn [] ...) so that they don't actually happen#2019-03-0112:16gfrederickswhich is part of why the spec api requires you to supply a 0-arg-function-that-returns-a-generator#2019-03-0112:19djtangoah got it#2019-03-0112:27borkdudeIā€™m using this feature a lot, but it took a trick to get the string-from-regex generator to be decoupled from production codeā€¦ https://github.com/borkdude/speculative/blob/master/src/speculative/specs.cljc#L180#2019-03-0112:28borkdudeI could have moved this code to the test code and use a generator overrride#2019-03-0200:17aghas anyone had this problem? It seems instrument doesnā€™t see my fdefs and they simply are ignored. fdef and the function and instrument - all in Clojurescript#2019-03-0200:18agIā€™m trying to use orchestraā€™s instrument - doesnā€™t work#2019-03-0200:25seancorfield@ag You've required the namespace containing the functions and the fdefs before running instrument?#2019-03-0200:26aghmmā€¦ when I eval instrument it returns vector with speced symbols, but it doesnā€™t fail the spec#2019-03-0200:27agIā€™m trying to check if wrong data would cause it to throw, but it doesnā€™t#2019-03-0200:27seancorfield(I'm not sure how instrument works in cljs so you may want to ask for help in #clojurescript if this seems to be a cljs-specific problem?)#2019-03-0200:27seancorfieldHard to offer more advice without seeing your code...#2019-03-0200:28seancorfieldAlso, have you tried spec's instrument directly, to eliminate an issue with Orchestra?#2019-03-0200:28agyeah, neither working ;(#2019-03-0200:29seancorfieldI would try it in a Clojure REPL to see if you can confirm the basics work as you expect. If they do, try that exact same example in cljs if you can.#2019-03-0200:33agthe basicsā€¦ workā€¦ if I simply create a function and call it in the repl, but when itā€™s within the app - it doesnā€™t work#2019-03-0200:33agprobably has something to do with re-frame#2019-03-0200:38agyeah, seems Iā€™m rightā€¦ if I call fdefed function directly - it works. But doesnā€™t automatically validate when used in the app#2019-03-0200:38agand the fn is definitely being called#2019-03-0204:27seancorfield@ag Ah, I suspect the code holds onto the original function value and doesn't see the instrumented version...?#2019-03-0211:55metametadataHi. Is there any library which allows enriching the error message of the function-based spec? Default:
(defn alright? [x] false)
(s/def ::alright alright?)
(s/explain ::alright 123)
; Output: 123 - failed: alright? spec: :user/alright
With lib:
(s/def ::alright (lib/with-explanation alright? (fn [x] (str x " is not alright!")))
(s/explain ::alright 123)
; Output: 123 - failed: alright? (123 is not alright!) spec: :user/alright
#2019-03-0212:00borkdudemaybe using the explain-printer stuff? expound would be a library to look at for an example of this#2019-03-0212:50metametadatasetting custom :reason does the trick for me:
(defn with-reason-fn
  "Will use (reason-fn [x]) to set :reason in explain data for the specified spec."
  [spec reason-fn]
  {:pre [(s/spec? spec) (ifn? reason-fn)]}
  (reify s/Spec
    (explain*
      [_ path via in x]
      (let [data (s/explain* spec path via in x)]
        (map #(assoc % :reason (reason-fn x)) data)))

    ; Do not modify the rest of methods
    (conform* [_ x] (s/conform* spec x))
    (unform* [_ y] (s/unform* spec y))
    (gen* [_ overrides path rmap] (s/gen* spec overrides path rmap))
    (with-gen* [_ gfn] (s/with-gen* spec gfn))
    (describe* [_] (s/describe* spec))))

(s/def ::alright (with-reason-fn (s/spec alright?) (fn [x] (str x " is not alright"))))

(s/explain ::alright 123)
; Output: 123 - failed: 123 is not alright spec: :user/alright
#2019-03-0214:34Josip GracinI'm trying to figure out why the following code is not working (using Clojure 1.10.0). The test fails. However, if I move the s/def definition of :n/repository after the extend-type line, it passes.#2019-03-0214:37borkdudewhat happens if you donā€™t use partial but #(satisfies? % Repository)? Iā€™m not sure if that solved it, just curious#2019-03-0214:38Josip Gracinthen it works. šŸ™‚ you remembered my original question.#2019-03-0214:39Josip Gracin#(satisfies? Repository %)#2019-03-0214:39borkdudesorry yes#2019-03-0214:40Josip Gracinstrange, isn't it?#2019-03-0214:40borkdudethere was a discussion about this in the #beginners channel I believe#2019-03-0214:41borkdudepartial just binds the value of the var too early to see re-definitions, I think itā€™s related to that, although Iā€™m not sure how it works with protocols#2019-03-0214:41Josip Gracinyeah, I'm suspecting some interplay between protocol var, auto-generated interface and closures.#2019-03-0214:42Josip Gracinis extend-type redefining something?#2019-03-0214:42Josip GracinI'll check#2019-03-0214:42borkdudeI think a protocol is a var that gets redefined everytime you extend a type:
(-reset-methods (alter-var-root (:var proto) assoc-in [:impls atype] mmap))
so that explains it
#2019-03-0214:43Josip GracinI see. Thanks!#2019-03-0416:43jsa-aerialIs there a preferred idiom for the use of 'world information' in predicates and specs? This can occur in data validation. To clarify some, you have a data value, which you can check the 'shape/type' of via predicate and spec involving that value. Once that checks OK, a further validation requirement would be to see if what the value represents exists (in a DB or directory or ...) and is 'properly configured'. Those checks require looking outside the value so a predicate doing these checks needs more than the value. Maybe that is access to an in memory cache or a DB key or whatever. I can think of several ways to do this, but they all feel a bit off. So, is there an idom / preferred way to have that information available in predicates and specs?#2019-03-0416:44borkdude@jsa-aerial My gut feeling tells me that itā€™s not a great idea to do side effects in specs#2019-03-0416:52Alex Miller (Clojure team)specs are not a great match for these kinds of validations#2019-03-0416:54Alex Miller (Clojure team)in general, I've seen better success with dynamically generated static specs (like running some code to register a spec with a set of allowed values pulled from a db) than with dynamic specs (which somehow close over state required to dynamically check validity in some way)#2019-03-0416:55jsa-aerialI see - I can understand that. Basically this is 'out of scope'.#2019-03-0416:55Alex Miller (Clojure team)pushing the limits :)#2019-03-0416:56Alex Miller (Clojure team)might be easier to implement as an explicit validation implemented outside of spec#2019-03-0416:57jsa-aerialYeah, that would be one of the ways I've considered#2019-03-0416:56jsa-aerialI will say there isn't any side effecting here, but it does use / need 'outside/ world' access#2019-03-0416:57borkdudemaybe something like this would work though:
(defn foo [conn]
  (let [vals (query conn)]
       spec (my-custom-spec vals)]
  (s/assert spec ...))
This keeps the spec entirely pure
#2019-03-0416:58borkdudedepending on how you use spec -- this wonā€™t work for e.g. fdefs#2019-03-0417:02jsa-aerialSome other ways: augment the value first with the 'key' for outside data so that the 'value' is wrapped and contains the necessary information. That's not too bad. If you have a map, augment with the 'outside key' and explicitly check the map contents (as opposed to using spec'd keys). Again, not too bad but kind of not using spec idiomatically...#2019-03-0417:04jsa-aerialRather worse: use a dynamic var to hold the outside 'key'... This actually feels a bit dirty.#2019-03-0417:11Alex Miller (Clojure team)it is#2019-03-0420:02seancorfieldIt's also worth mentioning that, strictly speaking, reading values from a DB is side-effecting because if you repeat the call, you will not necessarily get the same result. It's not mutating anything directly, but it is not a pure operation either. I think a lot of people tend to think that SELECT * FROM whatever is "readonly" and therefore not subject to side-effects...#2019-03-0422:42jsa-aerialIf nothing changes (anywhere), I don't see how it is 'side effecting' to read. The only way that would make sense is if the database were to change out from underneath you. In general that could happen, but in this case nothing of the sort is happening. Even more to the point, if it is already read and cached, nothing associated with the operations at hand is effecting anything anywhere.#2019-03-0423:01seancorfieldThat's why I said "strictly speaking" -- in the general case, where a database is a mutable thing that other processes or other threads could be updating, when you do a live read from the DB, you may get different results from different calls over time. Thus, the function doing the reading behaves as if it has side-effects because it is not pure. I was specifically excluding the case where the data is read once and cached.#2019-03-0420:03seancorfield(unless you're using Datomic where you can get the entire DB as an immutable value for whatever scope of request processing you want)#2019-03-0421:03aghow do I write fdef args for a function that takes: [[a b] _]?#2019-03-0421:05Alex Miller (Clojure team)there are several ways to potentially spec "[a b]" - what have you tried?#2019-03-0421:05Alex Miller (Clojure team)could be an s/tuple, s/coll-of with :count 2, a nested s/cat#2019-03-0421:06ag(s/cat (s/cat :a int? :b int? ) :_ any?)#2019-03-0421:06Alex Miller (Clojure team)nested regexes describe the same collection#2019-03-0421:06Alex Miller (Clojure team)wrap s/spec around the inner s/cat#2019-03-0421:06Alex Miller (Clojure team)to describe a nested regex collection inside the outer regex collection#2019-03-0421:07Alex Miller (Clojure team)(s/cat (s/spec (s/cat ...)))#2019-03-0421:11agnot working šŸ˜ž#2019-03-0421:13borkdude@ag
user=> (s/valid? (s/cat :v (s/spec (s/cat :a int? :b int?))) [[1 2]])
true
#2019-03-0421:14agah okayā€¦ now I get it!#2019-03-0421:14agawesome! thank you @alexmiller and @borkdude!#2019-03-0421:18borkdudein spec2 this will be called (s/nest ..)#2019-03-0422:44robertfwapologies if this isn't the right place to ask - I've recently started using the replacement instrument function from jeaye/orchestra. I'm getting an error while generating one of my specs, the error is
Execution error - invalid arguments to orchestra.spec.test/spec-checking-fn$conform! at (test.cljc:115).
(\8 \9 \7 \9 \2 \5 \4 \2 \0 \1 \0 \8 \1 \6 \9 \3 \1 \7) - failed: string? at: [:args :digits]
#2019-03-0422:44robertfwer, one moment while i clean that formatting#2019-03-0422:45robertfwI'm wondering if I'm missing something as I would expect to get a more detailed/specific error about what spec failed#2019-03-0422:45robertfwwhereas the error seems to be something internal to orchestra#2019-03-0422:58seancorfield@robertfrederickwarner That looks to me like it expected a string but was passed (seq s) -- i.e., a sequence of characters instead.#2019-03-0423:00robertfwYup - that's the problem within the spec. My confusion is more that the error from orchestra isn't what I'd expect to see - looking at https://github.com/jeaye/orchestra/blob/2019.02.06-1/src/clj/orchestra/spec/test.cljc @ line 115 (the line from the error above), I'd expect to see an error prefixed with "Call to..." followed by details about the spec that failed#2019-03-0423:01robertfwThe problem itself - passing a sequence of chars instead of a string - is occurring within a generator for a spec so I am wondering if that is causing problems for the usual reporting#2019-03-0423:02robertfwI saw some discussion of orchestra here earlier so thought I would throw it out incase anyone here recognized something obvious, I'll dig around some more but may open a bug to see what @jeaye thinks#2019-03-0423:51robertfwdid some further digging and tried out the standard instrument. Orchestra was hiding the error location a little bit but otherwise returns similar to the standard error#2019-03-0423:52jeaye@robertfrederickwarner So the issue exists in the upstream spec instrumentation as well?#2019-03-0423:52robertfwThe upstream error I got was
Execution error - invalid arguments to ws.data/luhn-check-digit at (data.clj:39).
(\8 \9 \2 \2 \5 \0 \7 \9 \2 \3 \9 \6 \4 \5 \3 \2 \6 \8) - failed: string? at: [:digits]
#2019-03-0423:53robertfwSo at least that let me see the line in question (data.clj:39)#2019-03-0423:53jeayeOk, it's likely an issue with the way your spec is written then.#2019-03-0423:53jeayeWill you show your code, or a minimal repro case?#2019-03-0423:54robertfwsure, let me just strip out some specifics here#2019-03-0500:03robertfwI think the relevant pieces are all here: https://gist.github.com/robertfw/4abd13c91feba4c96d3e1d14c87c0c13#2019-03-0500:04robertfwWe have an ID value that uses a luhn check digit. The error was being caused by line 27 - where it is using subs, it was using drop-last which ended up returning the sequence of characters#2019-03-0500:05robertfwIs the expected behaviour to see what spec failed? e.g., hone in on luhn-check-digit being the failing fdef?#2019-03-0500:21jeayeWhat is regex-spec?#2019-03-0500:21jeayeSeems like that would be the problem to me.#2019-03-0500:30robertfwah sorry I missed including that. It is...
(defmacro regex-spec
  [regex]
  `(s/with-gen (s/and string? #(re-matches ~regex %))
     #(gen'/string-from-regex ~regex)))
#2019-03-0500:35robertfwI do find that there are quite a few exception messages that could be a little more helpful. For example, right now I'm hunting down a problem with a misbehaving generator, and all I have to go off of is
Execution error (ExceptionInfo) at clojure.test.check.generators/such-that-helper (generators.cljc:320).
Couldn't satisfy such-that predicate after 100 tries.
#2019-03-0500:36robertfwIt's made a little more manageable through workflow (e.g., I know what spec I'm working on), but it would still be very helpful to get more details on what went wrong to help debugging#2019-03-0500:47robertfw(That's a general spec/test.check comment, not so much orchestra)#2019-03-0500:48jeayeI feel like you have a lot going on there, so it's not quite a minimal repro case at all.#2019-03-0500:49jeayeTo figure out the source of that string sequence issue, keep removing code until the issue goes away. šŸ™‚ If it goes away, add that code back and remove some other code until it goes away again. Keep repeating until the only code left is necessary for the reproduction.#2019-03-0500:49jeayeNamely, that spec error likely has nothing to do with generators or your custom macros or even all of that business logic around luhn digits.#2019-03-0500:55robertfwAs mentioned earlier - I already fixed the actual issue with the code, my question is more about debugging flow and finding the culprit. In this case the error I was getting back was not very helpful in narrowing down where to look, and I was wondering if there was something I was doing wrong to impact that.#2019-03-0501:04jeayeOh, that's my mistake. I came in last and must've missed that. I think the one thing I'd say we should keep in mind is that specs themselves should be simple enough to not really need debugging. To me, specs are for debugging, so it's awful to need to debug the debugging code. If you're stuck on weird issues with your specs, perhaps your approach can be simplified.#2019-03-0501:05jeayeNote, I also never use generators, since Orchestra + good unit and functional test coverage is everything I care about. However, especially given that this functionality (ret + fn instrumentation) has been removed from Clojure core and a lot of the spec press has been around generative testing, I may not be among the majority.#2019-03-0501:09robertfwWe're still fairly fresh with it - we have one other clojure project in our repos but it is pre-spec. The generative testing has been useful so far, it's already caught a few issues that we probably wouldn't have spotted with our usual approach. But yeah.. it's definitely tricky going at times with more complex specs, especially for relatively new clojure devs like myself#2019-03-0501:09jeayeWe have thousands of lines of specs and hundreds of spec'd functions in our code (at my company). I don't recall the last time I spent a noticeable amount of time debugging an issue with one of my specs. This is likely because they're all very simple. "This thing requires these keys. This is a number, this is a non-blank string, this matches this regex." Not too much more than that, for our entire front-end and back-end.#2019-03-0501:11robertfwThe bulk of ours are like that and haven't given us any issue. Todays fun was caused by the more complicated spec for the luhned ID, and some specs defining incoming requests on our API that have some internal consistency properties#2019-03-0501:11jeayeGotcha.#2019-03-0501:12robertfwThanks for taking a look! I appreciate it. Orchestra was a great find for us, having the :ret and :fn checking on instrument is really simplifying our workflow#2019-03-0501:12jeayeI'm happy you're enjoying it. šŸ™‚#2019-03-0501:12robertfwWe were wondering why that wasn't part of the standard instrument#2019-03-0501:12robertfwIt seems like a natural thing to do#2019-03-0501:14jeayeIt was and then it was removed. My understanding is that the Clojure team didn't like to emphasize that usage of spec and noted the performance implications. To me, that is the usage of spec and the performance implications are negligible compared to the safety benefits.#2019-03-0501:15jeayeI wouldn't want to misrepresent them and I haven't actually spoken with them about it. Would love to sit down with Rich and talk it out, but who wouldn't love to sit down with Rich and talk anyway?#2019-03-0501:17robertfwInteresting. Yeah, no kidding! Definitely a brain to pick#2019-03-0501:51seancorfieldIt's because instrument is for checking calls pass the right data -- the :args key -- and check is for checking the functions themselves behave correctly (given conforming :args, does the output of the function satisfy :ret and :fn).#2019-03-0501:52seancorfieldOrchestra complects those two, very different, types of checks.#2019-03-0501:52seancorfield(which is fine as long as folks understand that is what's going on and they are comfortable making that choice to go against how spec is designed)#2019-03-0501:57jeayeThere's the official opinion. šŸ™‚ Thanks, Sean. Fortunately, people now have the choice.#2019-03-0503:16seancorfieldI wouldn't say "official", coming from me -- I'm just repeating what I understand the Cognitect folks to have said about function specs šŸ™‚#2019-03-0504:48Alex Miller (Clojure team)yeah, that#2019-03-0504:49Alex Miller (Clojure team)ret specs are probably going to change a lot in spec 2 as well#2019-03-0505:21jeayeHopefully not in a way which is incompatible with this form of instrumentation, I hope, Alex.#2019-03-0505:21jeayeEither upstream or via a soft fork such as Orchestra.#2019-03-0505:24seancorfield@jeaye At work we have a branch of our code running on Spec2 -- there were several substantive changes (beside the "obvious" renaming of namespaces in :require clauses). They weren't big changes, but they were breaking changes.#2019-03-0505:24jeayeGood to know, Sean. Thanks for the info.#2019-03-0505:28jeayeI'll wait and see how things are, when the dust settles. The libs, spec and spec2, aren't so large as to be very difficult to either work into this case or replace with something which does. There's superb thought work going into spec2, no doubt, and I have no interest in competing with that. I just want to make sure my team and other Orchestra users will be able to take advantage of the new spec features along with the instrumentation.#2019-03-0510:47prnc
Clojure 1.10.0
user=> (require '[clojure.spec.alpha :as s])
nil
user=> (s/def ::foo :bar/this)
Syntax error compiling at (REPL:1:1).
Unable to resolve spec: :bar/this
user=> (s/def ::foo (s/keys :req-un [:bar/this]))
:user/foo
user=> (s/def ::foo (s/keys :req [:bar/this]))
:user/foo
Could someone offer a good way of thinking about the above ^^^? I.e. it is possible to refer to 'non-existent' spec inside map spec (through keys) but it's not possible at the "top level" s/def. I guess my confusion is coming from the fact that semantically those uses seem equivalent to me--both use ns-qualified keyword as a way to refer to spec--one fails the other doesn't.
#2019-03-0513:05Alex Miller (Clojure team)Aliased specs are not as delayed as they should be in this case, which is a known issue. I have been working on fixing it in spec 2.#2019-03-0514:19prncAmazing, thanks @alexmiller mario-star#2019-03-2617:18rickmoynihanahh might be an old spec versionā€¦#2019-03-2617:24rickmoynihanYeah I had an old spec; can confirm the issue is gone when using latest spec1 šŸ˜Œ#2019-03-2621:11agwhen defining a fdef spec for a function that is in different namespace, I can use fully qualified name. How can I alias fully-qualified part? e.g: - fn implementation of foo is in core.common - spec is in core.specs - I can do (s/fdef core.common/foo bla-bla-bla) I cannot require core.common in core.specs, to avoid circular reference since specs are used in core.common. But because Iā€™m using qualified name of foo - instrumentation still works. But how can I use alias for core.common, so I donā€™t have to repeatedly write it fully? clojure.core/alias doesnā€™t work, because core.common cannot be imported in this namespace#2019-03-2621:21didibusI don't think there's a way.#2019-03-2621:23Alex Miller (Clojure team)currently, you can't do this (other than by creating it as a namespace, which will then allow you to alias it).#2019-03-2621:25ag(ā•ÆĀ°ā–”Ā°ļ¼‰ā•Æļøµ ā”»ā”ā”» Iā€™m sad and dissatisfied. It is totally entire Cognitectā€™s fault.#2019-03-2621:26lilactownitā€™s beginning to feel like we need something like alias but that doesnā€™t require the namespace to be loaded#2019-03-2621:27agyeahā€¦ please have itā€¦ or Iā€™m gonnaā€¦ about to.. leave Clojure and doā€¦. I dunno something elseā€¦ woodworking or papier mache figurines to sell them on farmerā€™s market#2019-03-2621:27lilactownesp. in ClojureScript where you canā€™t even ad-hoc create the namespace#2019-03-2621:44Alex Miller (Clojure team)yeah, this is something we've talked about for a while and there is a ticket placeholder for it#2019-03-2621:45Alex Miller (Clojure team)we would like to have a lighter-weight aliasing mechanism (lighter than "creating namespaces") and I expect that could be used in both clj and cljs#2019-03-2621:45Alex Miller (Clojure team)placeholder is https://dev.clojure.org/jira/browse/CLJ-2123
#2019-03-2621:45Alex Miller (Clojure team)hoping to do some work on this in 1.11#2019-03-2621:46Alex Miller (Clojure team)I feel your pain :)#2019-03-2711:09zcljDo anyone know of a library that can produce specs given a json schema?#2019-03-2712:05ikitommi@zclj havenā€™t heard, but would have use for that too. there is an issue in spec-tools, but no work going on: https://github.com/metosin/spec-tools/issues/154. Happy to help if someone starts doing that.#2019-03-2712:31zclj@ikitommi thanks for the info. I will do some experiments on my specific case and if it is generic enough I would be happy to contribute to the issue in spec-tools#2019-03-2713:05ikitommiIs there a way to force a fdef to validate always the inputs? Schema had the :always-validate metadata for this.#2019-03-2713:08ikitommie.g.
(require '[schema.core :as s])

(s/defn ^:always-validate interceptor-x
  [opts :- {:string? s/Bool}]
  ...)

(interceptor-x {})
; Syntax error (ExceptionInfo) compiling at (test.cljc:150:1).
; Input to interceptor-x does not match schema:
;
;       [(named {:string? missing-required-key} opts)]
#2019-03-2713:07ikitommiSomething like stest/instrument but for non-test use.#2019-03-2713:11rickmoynihanNot that I know of. You can use a :pre condition on a function that calls s/valid?, though youā€™ll not get an s/explain-* style error if you do.#2019-03-2713:12rickmoynihanIf you actually want that though, why canā€™t you just call
(st/instrument #{`the.fdef.i.want.to.always/validate})
#2019-03-2713:14rickmoynihanIā€™d probably just call s/valid? and raise an s/explain-data if the check failed explicitly inside the function though if I wanted that ā€” or however you want to report the error.#2019-03-2713:14ikitommiThat would work, but that would not contribute to the function documentation.#2019-03-2713:15rickmoynihanIt would if it was both an fdef and that šŸ™‚#2019-03-2713:16ikitommicalling clojure.spec.test.alpha from non-test code also doesnā€™t sound like a good idea.#2019-03-2713:20ikitommiIā€™ll write an issue out of that.#2019-03-2713:32ikitommihttps://dev.clojure.org/jira/browse/CLJ-2498#2019-03-2714:26rickmoynihanregarding configuring components on startup you might want to look at what integrant does: https://github.com/weavejester/integrant/#specs#2019-03-2714:46ikitommiThatā€™s nice, thanks! Doing about the same with reitit: https://github.com/metosin/reitit/blob/master/modules/reitit-http/src/reitit/http/coercion.cljc#L12#2019-03-2714:48ikitommiI would still like a support functional way of doing that: have a function that takes options and returns the component (that has a spec). And the function args should be validated too.#2019-03-2714:21drone@ikitommi Iā€™m not sure it addresses your exact issue, but orchestraā€™s instrument enables checking of :ret and :fn#2019-03-2714:24dronethis has been better for how we use spec; where we enable instrumentation during development to try and get type-checking-like feedback while writing code#2019-03-2714:29Alex Miller (Clojure team)@ikitommi it is intentional that this functionality doesn't exist for functions as the goal is to have no overhead for function specs at runtime#2019-03-2714:29Alex Miller (Clojure team)but I'll leave it open for now as maybe it would be useful to have an opt-in feature for that#2019-03-2714:31ikitommiopt-in would be great.#2019-03-2714:34ikitommi@mrevelle orchestra is great, but it also has the dynamic *instrument-enabled* which can disable the instrumentation.#2019-03-2723:42jeayeWe just use something like the following in our code:
(when-not-prod
    (let [instrumented (sort (stest/instrument))] ; [orchestra.spec.test :as stest]
      (timbre/trace :instrumenting instrumented)))
#2019-03-2819:31borkdude@ikitommi If you want to always validate, also in prod, why not use s/valid? for this?#2019-03-2819:32borkdude
(s/valid? (:args (s/get-spec `my-fn)) args)
something like that
#2019-03-2819:34borkdudeor if you donā€™t like using s/get-spec you can write the args spec separately and use that in fdef, youā€™ll still get the docstring#2019-03-2819:39dronewe use s/valid? in asserts for things that should never happen and both s/valid? and s/conform in control flow (`if`, condp, match, etc). not advocating to do that, but sharing a related need we felt and what weā€™re trying out to address it#2019-03-2819:45borkdude
user=> (s/fdef foo :args (s/cat :i int?))
user/foo
user=> (def args-spec (:args (s/get-spec `foo)))
#'user/args-spec
user=> (defn foo [i] (if (s/valid? args-spec [i]) i (throw (ex-info "n00" (s/explain-data args-spec [i])))))
#'user/foo
user=> (foo 1)
1
user=> (foo "1")
Execution error - invalid arguments to user/foo at (REPL:1).
"1" - failed: int? at: [:i]
#2019-03-2819:48borkdude^ @ikitommi#2019-03-2819:48borkdudes/assert also works, but we have assertions turned off in prod#2019-03-2819:48borkdudeso s/valid? always works#2019-03-2819:49borkdudeand I bet you can wrap this in a macro like (def-checking-fn name & body)#2019-03-2820:26ikitommi@borkdude thanks! looks just right. In my case, it's a framework calling the specced functions at runtime, so I can use apply & args to make a generic functional proxy.#2019-03-2820:32ikitommi... or just extract the function spec if that exist and validate by the framework. indeed.#2019-03-2820:29ikitommithat said, tempted to add an s/defn with identical syntax as Schema somewhere. There are several ones (all bit different) out there already. The Lisp Curse?#2019-03-2820:39droneI think the only public one thatā€™s maintained is orchestraā€™s defn-spec#2019-03-2821:05Alex Miller (Clojure team)fwiw, I think weā€™re probably going to add something with spec 2 (or maybe Clojure 1.11)#2019-03-2821:18metametadataI like https://github.com/Provisdom/defn-spec syntax as it's friendly out-of-the-box to Cursive IDE (specs are defined inside defn metadata). but maybe orchestra's macro is also supported by Cursive#2019-03-2821:49buttergunsThe other day, I asked a question about speccing Java object fields https://clojurians.slack.com/archives/C1B1BB2Q3/p1553612761304000 I thought I'd check back in, and say that I got intimate with s/conformer! My solution is to use a conformer to convert the Java object into a clojure map of {<fieldname-keyword> <field-value>}, and then just use s/keys
(s/def ::device (s/and (s/conformer (conform-java-fields ::summary #(.getSummary %)
                                                         ::report #(.getReport %)))
                       (s/keys :req [::summary ::report])))
I learnt something! šŸ™‚
#2019-03-2821:58Alex Miller (Clojure team)note that this spec won't work backwards with unform though#2019-03-2822:00Alex Miller (Clojure team)in general, this kind of thing is frowned upon as it bakes transformation into the spec and doesn't give registry consumers the option of whether to do the transformation. I would probably recommend to instead actually convert the objects to Clojure data, then use simple data specs on it.#2019-03-2822:01buttergunsSo, doing the transformation in code, instead of within a spec?#2019-03-2822:02Alex Miller (Clojure team)yeah#2019-03-2822:02Alex Miller (Clojure team)conformer was really imagined primarily as a tool for spec op creators, not for api users#2019-03-2822:03buttergunsOK, that makes sense. Thanks for the advice. I'm making progress#2019-03-2821:58borkdudere: https://clojurians.slack.com/archives/C1B1BB2Q3/p1553807158018600 it would be nice if spec offered a bit more integration between fdef args and defn args somehow#2019-03-2822:02Alex Miller (Clojure team)well the big thing likely here is to integrate the requires/provides semantics via select, and also to introduce a way to talk about returns in terms of args more descriptively. the :ret/:fn as it stands now is likely to change#2019-03-2901:20kenny@borkdude @mrevelle I've run into problems using s/valid? in production. If my specs use fspecs, it will throw in prod because no test.check. We work around this by never passing a function to a function that uses s/valid? but seems a bit dangerous. Would be nice to be able to disable fspec generative checking in certain production.#2019-03-2908:18ikitommiIs there a way to extract the fdef specs, given one has a function at hand? e.g. need to get the symbol out of the function to be able to look up for itā€™s spec. Functions are quite opaque in clojure (or Iā€™m just doing it wrong)#2019-03-2908:21ikitommisomething like this, but for real (and without eval):
(defn kikka [])

(-> kikka
    str
    (str/replace #"@.*" "")
    (str/replace #"_" "-")
    (str/split #"\$")
    (->> (apply symbol)))
; user/kikka
#2019-03-2912:05Alex Miller (Clojure team)You can demunge the class name#2019-03-2912:05Alex Miller (Clojure team)But wonā€™t always work#2019-03-2912:20Alex Miller (Clojure team)In general, itā€™s better if you can start from a symbol than start from a function object#2019-03-2913:14ikitommiThanks. In my case, I just donā€™t have the symbols available, just functions. Knowing the arity of the functions at runtime would be useful too to be able to fail fast in case of invalid arity of the (framework) functions (middleware, interceptors etc). Any change of getting the function symbol & arity queryable for runtime?#2019-03-2913:47Alex Miller (Clojure team)function objects are just function objects - the fact that a particular var is referring to them is in some ways incidental.#2019-03-2913:48Alex Miller (Clojure team)same for arity, although that one in particular is one that affects code evolution over time (it's common for arities to expand but still be backwards compatible)#2019-03-2913:48Alex Miller (Clojure team)I know these seem like obvious things to have on a function, but I think the implications are probably a lot more far reaching and subtle than you expect#2019-03-2913:50Alex Miller (Clojure team)there are a lot of good reasons to fight this kind of introspection and instead try to drive from symbols or vars#2019-03-2913:50Alex Miller (Clojure team)so, in short, no, not eagerly looking to make these changes#2019-03-2914:04drone@kenny we donā€™t use fdef because of this. And I agree, disabling the test generator req would help. I think it was mentioned as possibly coming in spec2#2019-03-2914:35stathissiderisa bit of a philosophical question: I know spec uses qualified keywords, but do people normally go for the :com.my-company.invoice/total style of keywords or the :invoice/total style?#2019-03-2914:48Alex Miller (Clojure team)if you are writing a public library, then I think you endeavor to start your qualifier with something you control (by trademark, domain name, etc).#2019-03-2914:48Alex Miller (Clojure team)if you are writing a private app, then you should make qualifiers "sufficiently unique"#2019-03-2914:49Alex Miller (Clojure team)whether that means including company, department, product, whatever is relevant#2019-03-2915:22stathissideris@alexmiller thanks, thatā€™s a balanced view which I think makes sense#2019-03-2918:08dominicmI'm starting to regret using particularly unique keywords that include the company name and sub project as a prefix for an application. I'm considering adding aliases to human written edn because of it.#2019-03-2919:10carocadhey guys, does somebody knows what is the url of the api docs for spec-alpha2 ? The link seems to be broken in https://github.com/clojure/spec-alpha2 šŸ˜•#2019-03-2919:12Alex Miller (Clojure team)I'm not sure I've ever built it#2019-03-2919:12Alex Miller (Clojure team)the doc builder works off of a release and we haven't released a version of it yet#2019-03-2919:15Alex Miller (Clojure team)in other words, the link is right, the target is absent :)#2019-03-2919:15Alex Miller (Clojure team)I can try to build it but I'm probably not going to have time today#2019-03-2919:21Alex Miller (Clojure team)Posted a writeup on the (still in work) schema and select in spec 2: https://github.com/clojure/spec-alpha2/wiki/Schema-and-select#2019-03-2919:29carocadthanks a lot šŸ™‚#2019-03-2920:23seancorfieldI know what I'll be doing this weekend (while my wife's in China)... šŸ™‚#2019-03-3114:09eggsyntaxLooking good, thanks Alex! Couple of thoughts/questions on that document: - on first encounter, it seems confusing (and complecting šŸ˜‰ ) to treat schemas sometimes as though they're specs (`s/def`, same registry) and sometimes not (can't be used for valid? etc). Why not either a) let them act as specs (maybe a schema acts like (s/select my-schema), ie all optional, when treated as a spec?) or b) make them fully separate (separate registry, can't use s/def to register them)? - What do you mean by 'SPI'? 'Serial-parallel interface'? šŸ¤” - In the "get-movie-times" example, is there a way to say that address is optional, but if present it must include zip? I know I've encountered use cases like that in practice.#2019-03-3120:27seancorfieldYes, omit address from the vector but still provide the map for the sub-select
user=> (require '[clojure.spec-alpha2 :as s])
nil
user=> (s/def ::name string?)
:user/name
user=> (s/def ::street string?)
:user/street
user=> (s/def ::city string?)
:user/city
user=> (s/def ::zip string?)
:user/zip
user=> (s/def ::addr (s/schema [::street ::city ::zip]))
:user/addr
user=> (s/def ::user (s/schema [::name ::addr]))
:user/user
user=> (s/def ::data (s/select ::user [::name {::addr [::zip]}]))
:user/data
user=> (s/valid? ::data {::name "Sean"}) ; ::addr is optional
true
user=> (s/valid? ::data {::name "Sean" ::addr {}}) : but when provided it must contain ::zip
false
user=> (s/valid? ::data {::name "Sean" ::addr {::zip "94546"}})
true
user=> 
#2019-04-0103:35Alex Miller (Clojure team)@U077BEWNQ - schemas as specs would be pretty useless since map specs are open (any map would validate) so we decided to make them different. There are some real subtle questions in the idea of wanting to have both for different purposes though and I haven't had a chance to talk through it with Rich yet. might change. - SPI = service provider interface, for those building new kinds of specs/schemas, vs API which is what consumers use. probably should have explained that more clearly.#2019-03-3017:28didibusHow would you spec the following: [:keyword1 '(:one :or) '(:more :lists) :keyword2 '(:one :or) '(:more :lists)] where each pair of :keywordN '(:one :or) '(:more :lists) is optional?#2019-03-3017:29didibusI tried:
(s/cat :keyword1-pair (s/?
                  (s/cat :keyword1 #{:keyword1}
                         :keyword1-lists (s/+ list?)))
         :keyword2-pair (s/?
                 (s/cat :keyword2 #{:keyword2}
                        :keyword2-lists (s/+ list?))))
But it doesn't work.
#2019-03-3017:33didibusNevermind, it does work. I had a typo.#2019-03-3017:34didibusI have another question though. It seems that conform evaluates the forms?#2019-03-3017:35didibusBasically, my above spec is for a macro. And I expect the lists to be code forms. Then I call conform to get it conformed, but it seems to evaluate the forms.#2019-03-3017:36didibus
(defmacro foo [& bar]
  (s/conform ::my-spec bar))

(foo [:keyword1 (+ 1 1)])
Hum... Unless its evaluated on the print.
#2019-03-3017:38didibusAh yes, nevermind again#2019-03-3021:34seancorfield@alexmiller I'm trying the latest Spec2 and running into a new failure
(s/defop length-limited-string
  "Given n return a spec for a string that is less then n characters
  long."
  [n]
  (s/and string? #(>= n (count %)))) 
;=> #'ws.billing.specs/length-limited-string
(s/def :wsbilling/braintree-id (s/nilable (length-limited-string 64))) 
;=> :wsbilling/braintree-id
(s/def :wsbilling/risk-id :wsbilling/braintree-id)
;=> :wsbilling/risk-id
(s/get-spec :wsbilling/braintree-id)
;=> #object[clojure.spec_alpha2.impl$nilable_impl$reify__12249 0x689396e2 "
This worked fine on the last version I tried, and that last s/get-spec call returned a spec, as expected, not nil.
#2019-03-3021:34seancorfieldIf I change :wsbilling/risk-id to
(s/def :wsbilling/risk-id (s/spec :wsbilling/braintree-id))
then it works.
#2019-03-3021:39seancorfieldUnfortunately we have a lot of specs that are defined as aliases of other specs. I happened to spot this one because we have a test that verifies every key in a related spec has a spec defined for it...#2019-03-3021:40seancorfieldAh, it looks like that spec exercises and conforms just fine, it's only get-spec that "fails" by returning nil.#2019-03-3021:52seancorfieldOK, it looks like we used to call s/spec in these cases with Spec1, but that stopped working in Spec2 with the API split into symbolic specs and spec values, so I changed it to s/get-spec and checked it returned non-`nil`. But what I probably should have done was used s/form instead which seems to do the right thing in Spec2. It still seems surprising that s/get-spec returns nil for the case above tho'...#2019-03-3022:54Alex Miller (Clojure team)I know why youā€™re seeing that#2019-03-3023:02seancorfieldIs it a bug or should I just not be doing it that way?#2019-03-3023:02seancorfield(I have almost everything passing again on Spec2 -- still trying to track down one failure in one of our newer apps)#2019-03-3023:07seancorfieldOK, everything passes. The only change I had to make was that get-spec issue (in three places).#2019-03-3023:07seancorfieldI'll take a break for a bit and then start looking at schema/`select`...#2019-03-3101:50Alex Miller (Clojure team) Bug#2019-03-3103:55seancorfieldNeed a JIRA for it @alexmiller?#2019-03-3103:55Alex Miller (Clojure team)Nah#2019-03-3107:36danielcomptonhttps://danielcompton.net/2019/03/31/announcing-defn-spec#2019-03-3109:06ikitommi@danielcompton awesome! It seems to use schema directly, have you though about copying just the needed parts? Takes 1sec to load just the schema.core to the repl and most of it is not needed here, I guess?#2019-03-3113:01y.khmelevskii@danielcompton as I can see defn-spec is slightly similar to https://github.com/gnl/ghostwheel, right?#2019-03-3116:45danielcompton@ikitommi yeah I've pretty much copied most of the impl from schema into defn-spec, there was just a little bit left in the macros namespace and I wanted to get a release out over the weekend#2019-03-3116:45danielcomptonBut that's on the list to do, shouldn't be much more work there#2019-04-0107:34tangrammerhi spec people!! Iā€™m trying to temporarily redefine the spec global registry in the same way as with-redefsdoes ā€¦ My intention is to reduce possible valid values, eg: instead of only checking that value is a string, validate that is exactly foo value so far, this is the code that iā€™ve reachedā€¦
(s/def ::simple string?)

(s/def ::composed
  (s/keys :req [::simple]))

(with-redefs [s/registry (constantly (assoc (s/registry) ::simple (#'s/res #{"foo"})))]
    [(s/valid? ::composed {::simple "other"})
     (s/valid? ::composed {::simple "foo"})] 
    ) ;; [false true]

(s/valid? ::composed {::simple "other"}) ;; true

šŸ¤” what do you think about using with-redefs to redefine specs?
#2019-04-0112:09Alex Miller (Clojure team)Why?#2019-04-0112:29tangrammerthe final idea is to apply authorisation#2019-04-0112:32tangrammerbasically if certain path values of the speced data are ā€œallowedā€, the full data could be viewed by a user or not#2019-04-0112:33tangrammeriā€™s it more clear now?#2019-04-0112:38Alex Miller (Clojure team)Sounds like a nightmare for such a critical role#2019-04-0112:48tangrammercould you expand a bit why you think it could be a nightmare?#2019-04-0113:04Alex Miller (Clojure team)why would you base something as critical as auth on something as fragile as with-redefs?#2019-04-0113:07tangrammersorry i didnā€™t realise about ā€œThese temporary changes will be visible in all threadsā€ šŸ˜¬ ā€¦ I was trying for something thread isolated ā€¦ but with-redefs was my quick first attempt#2019-04-0115:20djtangoif you want thread-local rebinding - use binding#2019-04-0115:21djtangothough so far the only things I've found with-redef useful for is orthogonal instrumentation of functions (e.g. timing functions) or stubbing in testing#2019-04-0115:22tangrammer
(def ^:dynamic my-fun string?)

(s/def ::simple #'my-fun)

(s/def ::composed (s/keys :req [::simple]))

(s/def ::extra-composed (s/keys :req [::composed]))

(assert (s/valid? ::extra-composed {::composed {::simple "one"}}))

(assert (= (binding [my-fun #{"foo"} ]
             [(s/valid? ::extra-composed {::composed {::simple "other"}})
              (s/valid? ::extra-composed {::composed {::simple "foo"}})])
           [false true]))

(assert (s/valid? ::extra-composed {::composed {::simple "other"}}))


#2019-04-0115:22djtango(though since spec I've not really redef'd fns for testing)#2019-04-0115:22tangrammerexample using binding ^^ šŸ™‚#2019-04-0115:23djtangoya - still hard to see how / when you might apply that but at least it's thread-safe now#2019-04-0115:23tangrammer:+1:#2019-04-0115:26tangrammer@U064X3EF3 maybe less nightmare now with binding ?#2019-04-0115:34Alex Miller (Clojure team)Less but the idea of changing specs still seems like an idea likely to cause problems later #2019-04-0115:46tangrammerI'll take into account, thanks both for your help!#2019-04-0116:06djtangoIf it's a top level spec you can do:
(let [the-spec (if p? ::a ::b)]
  (s/valid? the-spec data))
If it's a nested spec, you could try using :req-un to have (s/keys :req-un [:very-specific/a]) and (s/keys :req-un [:general/a]) alternatively, you could try experimenting (s/or ...) If you're using namespaced keys on your data, it's hard to understand why the spec for a namespaced entity would change
#2019-04-0121:33tangrammermore than change, the spec should be more specific. so instead of having a predicate string? would be #{"value1" "value2" "value3"}#2019-04-0121:34tangrammerbeing these values the returned of a sql execution#2019-04-0112:42tangrammerthe thing (at this level) is getting spec validation messages#2019-04-0113:05djtangogiven that specs are typically named with just data can't you dispatch at real-time the spec you might want to validate against#2019-04-0113:05djtango(with no other knowledge of what you're trying to achieve)#2019-04-0116:04mathpunkI've got a question about spec design. The situation: I have an entity in mind, which I call a run (for test run). It has two variants: it might be a test run from CI, or it might be a test run from my local machine.#2019-04-0116:06mathpunkSome data is present in runs of any variant, like results and a revision (the SHA of the codebase during the run). Only CI runs have a pipeline. Both variants have logs but only the local variant has api logs under that key.#2019-04-0116:08mathpunkMy design problem is, I'm trying to use mostly namespaced keywords, so that I can nest specs. For instance,
(s/def ::revision (fn [s] (boolean (re-find #"[a-f0-9]+" s))))
(s/def ::meta-v1
  (s/keys :req [::date ::job ::pipeline ::revision]
          :opt [::suites]))
#2019-04-0116:08mathpunkBut then I realized, ooooh when I start modeling runs of the local variant, project.ci.runs/revision is overly specific#2019-04-0116:09mathpunkDo you have any thoughts on what I should keep in mind while spec'ing these variants out? Thanks!#2019-04-0116:11mathpunkI guess I should add, I'm trying to be careful about not over-spec'ing things --- this could be an example of, Don't spec this part#2019-04-0116:18mathpunk(The value-add that I see of spec'ing this data is, I've been doing these runs for months, and iterating on how they're shaped, so I'm interested in validating that I understand what 'type' of run I'm looking at)#2019-04-0116:19Alex Miller (Clojure team)a good guiding principle is to just try to spec the truth - what are all the values an attribute can actually take on?#2019-04-0116:20Alex Miller (Clojure team)if it's not constrained to the regex, then that's not the truth, it's just one variant#2019-04-0116:20Alex Miller (Clojure team)it's always possible to spec generally, then s/and an additional constraint for a particular context#2019-04-0116:23mathpunkšŸ¤” So maybe,
:project.run/revision
    :
#2019-04-0116:23mathpunkoh wait, you said 's/and', not 'and'... I'll have to think about that a minute#2019-04-0117:14kennyHow do you guys use clojure.test.check.generators/let with Spec? It doesn't appear to be exposed in clojure.spec.gen.alpha.#2019-04-0201:14taylorI just add the dependency explicitly to the test check namespace #2019-04-0201:15kennyWe don't allow test.check as a runtime dependency.#2019-04-0201:15taylorInstead of using the aliased stuff from spec#2019-04-0201:15taylorYou can add it as a dev only dependency #2019-04-0201:16kennyRight but then you can't define your generators next to your specs.#2019-04-0201:16taylorAlso, youā€™re depending on it already if youā€™re actually using any of its functionality#2019-04-0201:17kennyWe don't use the functionality in production but want to define the generators next to our specs.#2019-04-0201:17taylorThe stuff in spec are merely aliases to the test check definitions#2019-04-0201:17kennyTrue but it does it lazily -- it only requires test.check if you actually invoke a function in the gen namespace.#2019-04-0201:18taylorI supposed you could use the same strategy#2019-04-0201:19kennyYes but that begs the question of why it wasn't included to begin with. It may be because it is a macro.#2019-04-0210:24gfredericksyes, a macro has to run at compile time#2019-04-0210:25gfredericksso the normal trick that allows you to not have a runtime dependency on test.check doesn't work#2019-04-0217:14kennySo the solution is to copy and paste the let macro elsewhere?#2019-04-0218:58gfredericksYou can do that, you could relax your requirement to not have test.check in prod, you could wrap in a simple proxying macro that expands to an exception throwing generator when tc isn't on the classpath#2019-04-0219:00kennyThe problem with including test.check on the cp in prod is certain code paths check input structures via s/valid?. If, by accident, a function is passed in a map of things, it would get generatively checked. That would have a significant impact on performance.#2019-04-0219:03gfredericksThey skip that based on the namespaces being requirable or not?#2019-04-0219:03kennyNot sure what you mean.#2019-04-0219:04kennyIf test.check isn't available on the cp and a fn is passed to the s/valid? call, an exception will get thrown.#2019-04-0219:05kennyIf it is, no exception is thrown and the performance will slow down silently.#2019-04-0219:06gfredericksAh, I see, I didn't notice the "by accident" part In any case, the other two approaches would work though#2019-04-0219:07kennyThe latter seems like a good approach. Seems like a worthy addition to the Spec gen namespace as well.#2019-04-0301:11gfredericksI can see that#2019-04-0217:07zalkyHi all, is there an predicate for whether something is a valid spec? Something like specize (not public api), but returns true or false. I guess I could do:
(defn spec?
  [spec]
  (try
    (do (s/conform spec nil) true)
    (catch Exception e
      false)))
But maybe I missed something easy?
#2019-04-0217:11buttergunshttps://clojuredocs.org/clojure.spec.alpha/spec_q#2019-04-0217:13zalky@mattmorten, thanks for the response. The issue with that is it returns false things that are valid specs, like functions and keywords that resolve to spec objects.#2019-04-0217:14buttergunsAh, sorry, misunderstood. No I don't think I know of anything like you ask#2019-04-0218:13robertfwAre there any suggestions for tracking down "Couldn't satisfy such-that predicate" errors? The exception it throws doesn't appear to give any clues as to which predicate is the problem. I'm wondering if there is a faster option that my current approach of narrowing down my recent changes#2019-04-0218:16Alex Miller (Clojure team)unfortunately, not really atm. I will tell you that most of the time it's from an s/and#2019-04-0218:16Alex Miller (Clojure team)where the initial gen is too wide for the subsequent filter(s)#2019-04-0218:17robertfwyeah, I narrowed it down pretty quickly and it was indeed an and. just one of those minor annoyances.#2019-04-0218:22Alex Miller (Clojure team)there is some new stuff in the latest test.check but we haven't yet looked at how to make it better to expose the info out through spec#2019-04-0218:22Alex Miller (Clojure team)certainly, it's annoying, and I have spent a lot of time doing the same#2019-04-0223:20didibusQuestion, How do you s/keys a Java Map? Is that even possible? It fails because it says a Java Map is not a map?#2019-04-0223:26robertfwMy first guess would be to write your own predicate function to validate it yourself, I wouldn't expect s/keys to support it#2019-04-0301:49Alex Miller (Clojure team)spec is based on Clojure maps and is not intended to support Java colls#2019-04-0303:19didibusYa, I think I was confused, I thought all Java collections were Clojure's, but its the other way around only. And since Java Lists are seqable, anyways. I got around it.#2019-04-0303:19didibusthanks#2019-04-0310:22djtango@alexmiller I seem to have some vague memory that you guys were maybe looking at alternatives to generatively testing functions that are argument inputs or map entries, but can't seem to find anything to follow on this subject?#2019-04-0310:22djtangoam I remembering correct? This was prompted by some of the discussion @kenny and @gfredericks were talking about#2019-04-0315:57Alex Miller (Clojure team)not sure what you're talking a bout#2019-04-0315:58Alex Miller (Clojure team)can you give me some more hints? :)#2019-04-0316:17kennyI believe he may be referencing the ability to disable generative fspec testing in production.#2019-04-0317:27djtango^ this#2019-04-0317:28djtangobut am I imagining that there would have been an alternative way to check if the specs were still correct?#2019-04-0317:38Alex Miller (Clojure team)oh, yeah, my current thought on this is to instead of generatively testing the function to wrap the function with something that checks arg specs of the inputs#2019-04-0409:09djtangoAh nice - is any of the ideas/progress on this happening anywhere public?#2019-04-0411:05Alex Miller (Clojure team)https://github.com/clojure/spec-alpha2#2019-04-0411:06Alex Miller (Clojure team)And I write a weekly journal at http://insideclojure.org#2019-04-0413:45djtangoha - ok is there a jira on this specific bit?#2019-04-0413:47Alex Miller (Clojure team)there is an old ticket for it#2019-04-0413:51Alex Miller (Clojure team)https://dev.clojure.org/jira/browse/CLJ-2217#2019-04-0413:58djtango:+1: awesome thanks!#2019-04-0413:59Alex Miller (Clojure team)there may be others#2019-04-0319:27mpenetThat would be a huge improvement#2019-04-0321:04hadilsQ: I want to define a spec some thing like:
(s/def ::firstName string?)
(s/def ::lastName string?)
(s/def ::email utils/email-address?)
(s/def ::type #{:receive-only :personal :business})
(s/def ::businessName string?)
(s/def ::receive-only-user (s/and (s/keys :req-un [::firstName ::lastName ::email ::type]) (::type equals :receive-only))
#2019-04-0321:05hadilsIs this possible?#2019-04-0321:28hadilsNvm, I answered my own questionā€¦#2019-04-0411:32orestisIs there a way for spec to express ā€œif this key is present, this other key must also be presentā€?#2019-04-0411:33orestisActually - if this key has this value, this other key must also be present#2019-04-0411:34orestisI suspect itā€™s something you can do with multi-spec?#2019-04-0411:58orestisGot something working multi-spec, but it seems that I have to name the intermediates ā€” like, if I need a timestamp to be present if something is of type A, I have to name a spec ā€œsomething-with-timeā€, whereas the normal case is ā€œsomething-plainā€. Or perhaps naming isnā€™t necessary, and my multi-method can just return an (s/keys [])?#2019-04-0413:31Alex Miller (Clojure team)(s/keys) is valid#2019-04-0413:31Alex Miller (Clojure team)and will still validate all registered keys#2019-04-0413:32Alex Miller (Clojure team)sounds like an excellent use case for schema/select in spec2#2019-04-0413:43orestisOh thatā€™s right ā€” I guess what the actual requirement is, is that for some specific subtype of foo, you have to provide ā€œbarā€. If you provide ā€œbarā€ for a different subtype, well, itā€™s still valid but going to be ignored anyway.#2019-04-0413:46orestisLooking forward to seeing how schema/select looks like.#2019-04-0413:51Alex Miller (Clojure team)it looks like https://github.com/clojure/spec-alpha2/wiki/Schema-and-select#2019-04-0413:52Alex Miller (Clojure team)you can try it now if you like#2019-04-0413:58djtangoThis feels like a bit of an abuse of s/keys but is this what you mean @orestis (s/def ::m (s/keys :req [::a (or ::a (and ::foo ::bar))] :opt [::foo ::bar]))#2019-04-0413:59Alex Miller (Clojure team)note that or here is not xor#2019-04-0414:00djtangoindeed - got caught out by that once#2019-04-0414:00djtangobut really was just loose reasoning more than anything#2019-04-0414:02orestisI thought that there were incompatibilities between current spec and spec2 ā€” thatā€™s why I shied away so far.#2019-04-0414:03orestis(As in, Iā€™d rather migrate my specs and introduce the new stuff to the team when itā€™s released)#2019-04-0414:11orestisBTW today was the first time I shared a spec between frontend and backend. So nice! Thanks a lot to everyone involved šŸ˜„#2019-04-0416:35Alex Miller (Clojure team)correct - spec and spec2 are not compatible, but you could try it if you like (I wouldn't use spec2 for anything yet)#2019-04-0417:10seancorfield@orestis We have a branch of our 80K+ line codebase at work running on Spec2 but it's still changing a lot and we've tripped over numerous bugs that have led us to huddle with @alexmiller several times in order to figure out workarounds (and sometimes just waited for the bug to get fixed).#2019-04-0417:10seancorfieldI agree with Alex that it isn't production ready yet but it's definitely interesting to try out.#2019-04-0417:11seancorfieldOur biggest issue was that we had sort of mixed predicates and specs in some places -- that Spec1 allows but Spec2 draws a harder line in the sand around.#2019-04-0417:12orestisIrrelevant to spec2: Someone mentioned that in 1.10 destructuring is done via a spec, but it seems like destructure is same as before. Is there a spec for what could go into a destructuring binding?#2019-04-0417:12orestisThanks for chipping in @seancorfield ā€” Iā€™ll put it down on my list to try out, but the backlog is very big šŸ˜„#2019-04-0417:13seancorfieldI know that feeling šŸ™‚#2019-04-0417:13Alex Miller (Clojure team)there is a spec for destructuring in core.specs.alpha, used by many other specs#2019-04-0417:14Alex Miller (Clojure team)it's not used for destructuring#2019-04-0417:14seancorfieldWe've been on the bleeding edge for a long time at World Singles Networks. We regularly take alpha/beta builds to production, and we were early adopters of spec (and very heavy users of it in production). But even for us, Spec2 isn't quite ready yet šŸ™‚#2019-04-0417:14Alex Miller (Clojure team)https://github.com/clojure/core.specs.alpha/blob/master/src/main/clojure/clojure/core/specs/alpha.clj#L4-L44#2019-04-0417:14orestisCool!#2019-04-0417:15Alex Miller (Clojure team)btw, Sean, current spec2 should fix that issue you reported (as a side effect of other changes)#2019-04-0417:15Alex Miller (Clojure team)schema/select have been changing a bit this week, but probably not in ways that break anything you've done#2019-04-0417:15seancorfield(oh @alexmiller meant to mention that we've had 1.10.1-beta1 in production for all processes since 3/26 -- no problems!)#2019-04-0417:15Alex Miller (Clojure team)cool#2019-04-0417:16Alex Miller (Clojure team)I've been keeping https://github.com/clojure/spec-alpha2/wiki/Schema-and-select basically up to date though#2019-04-0417:16Alex Miller (Clojure team)will write up some more in weekly journal#2019-04-0417:16seancorfieldThanks, Alex. I may go back to our branch tomorrow and update it and see where we are now. I was working on an enhancement over the last few days that was just screaming out for schema/`select` BTW.#2019-04-0417:18seancorfieldBusiness wanted an extra field added to a database table (for which we have a spec) and that cascaded up through several layers -- and several specs -- to two of our REST APIs (which also have specs). Dealing with optionality across those tiers is something that schema/`select` will really help us with, I think.#2019-04-0417:18orestisOn the destructuring front then ā€” Iā€™d like to write a macro that given a destructuring form as in ::binding-form, and some kind of other specification (a GraphQL input object, specifically), ensures that the symbols used in the destructuring are present and and in the correct place in the other specification. This is a partly an aid to catch typos, partly a cool thing to show off. Would that ::binding-form spec be of any use? Perhaps conform could give something back?#2019-04-0417:19orestisOr should I ā€œjustā€ hand-roll a validator?#2019-04-0417:27seancorfield@orestis My understanding is that binding form spec already exists and you can use it directly (we do this in one place in our code -- which led to an interesting workaround when we tried Spec2 since the core specs continue to use Spec1 and we needed to bridge between them!).#2019-04-0417:28orestisYeah, TIL: (s/valid? ::cs/binding-form '{:keys [a b c :d]}) ;;=> true#2019-04-0417:28seancorfield
;; ugly workaround to pull spec1 internals into spec2 world:
(eval '(s/def :clojure.core.specs.alpha/bindings
         (s1/get-spec :clojure.core.specs.alpha/bindings)))
(s/fdef with-connection
        :args (s/cat :bindings :clojure.core.specs.alpha/bindings
                     :body (s/* any?))
        :ret any?)
#2019-04-0417:28orestisNever expected that a keyword would be a valid destructuring key.#2019-04-0417:29seancorfield(I'm shocked that the above workaround even works but...)#2019-04-0417:34Alex Miller (Clojure team)we added keyword support a few years ago, specifically so that you could use ::a, ::x/a, etc and get autoresolve functionality#2019-04-0417:34Alex Miller (Clojure team)that's somewhat been superseded now by :x/keys support but still there for backwards compatibility#2019-04-0417:37orestisGotcha.#2019-04-0417:39orestisSeems like spec isnā€™t the tool Iā€™m looking for the job I want to do. I would like something that given something like {foo :bar} and {:keys [a :b]} would give me the keys that will be destructured on the map. Perhaps I could do some hackery by actually calling destructure and looking at the generated bindings ā€” but that seems a recipe for trouble.#2019-04-0417:42orestisSorry for the stream of thought about this. I am wondering if this is actually viable or possible, and if I could get some value out of the core specs (apart from seeing that there are a ton of possible use cases that I never knew existed šŸ˜„ )#2019-04-0417:56Alex Miller (Clojure team)oh, yeah, I don't know of anything that will do this job#2019-04-0417:57Alex Miller (Clojure team)unless you're will to eval, in which case you could probably abuse the destructure function#2019-04-0418:00Alex Miller (Clojure team)
user=> (->> '[{:keys [a :b]} {foo :bar}] destructure (apply hash-map) keys)
(a map__2953 b)
#2019-04-0418:01Alex Miller (Clojure team)you will see some gensym cruft (although the cruft is real! those are intermediates that are getting bound)#2019-04-0418:01orestisThis will probably lose the structure for nested restructuring though. Since it will linearize all the bindings. #2019-04-0418:02Alex Miller (Clojure team)yes, it resolves to a single let binding#2019-04-0418:02Alex Miller (Clojure team)but that is the reality of what's happening. going in reverse seems challenging.#2019-04-0418:03orestisI thing walking the tree and having a cond based on the core specs to see which kind of restructuring is there and recurring accordingly would make more sense. I will have to maintain the ā€œcurrentā€ position against an external tree structure anyway, to answer questions such as ā€œis this key valid for this sub treeā€#2019-04-0418:05Alex Miller (Clojure team)there's a lot of destructuring options so lots to do there#2019-04-0418:06orestisHopefully thereā€™s a 80/20 reward (as is usually the case with this kind of linting).#2019-04-0418:08orestisOr there are dozens of academic paper arguing about whether this is possible šŸ˜„#2019-04-0418:08Alex Miller (Clojure team)doesn't seem impossible :)#2019-04-0418:08Alex Miller (Clojure team)just tedious#2019-04-0418:20orestis
(defn get-destructured-keys [binding]
  (->> (destructure [binding {}])
       (filter list?)
       (filter #(= (first %) 'clojure.core/get))
       (map #(nth % 2))))


(get-destructured-keys '{:keys [foo bar]})
(get-destructured-keys '{a :foo
                         b :bar})

#2019-04-0418:20orestis70% of the value šŸ˜›#2019-04-0418:25orestis
(let [(gql-args {foo :foo} :something) {:foo "bar"}]
  foo)
=> this complains that let doesnā€™t conform to spec. Bummer. I hoped that my macro would run first.
#2019-04-0418:26Alex Miller (Clojure team)nope :)#2019-04-0418:26Alex Miller (Clojure team)due to delayed eval, outer is in control first#2019-04-0418:28orestisPerhaps I could do a tagged literal?#2019-04-0418:36Alex Miller (Clojure team)tagged literals are read as objects at read time, can't see that's going to help you in expansion#2019-04-0418:36Alex Miller (Clojure team)you could make your own let variant#2019-04-0418:38orestisWas hoping that I could replace the tagged literal with a valid let binding form at read time, then the let macro wouldnā€™t know the difference (if I understand the compilation phases correctly)#2019-04-0418:43orestisToo much for a Thursday evening ā€” gotta run! Thanks for your time.#2019-04-0418:57Alex Miller (Clojure team)might actually work to some limited degree (I imagine some editors and context would run into issues with this)#2019-04-0418:59Alex Miller (Clojure team)
user=> (set! *data-readers* {'foo (fn [x] ['a x])})
{foo #object[user$eval2967$fn__2968 0x5b0902b4 "
#2019-04-0419:00orestisOh, thatā€™s how you do a dynamic data reader? I was looking at https://clojure.org/reference/reader#tagged_literals and it suggested binding which didnā€™t work in my case.#2019-04-0419:02Alex Miller (Clojure team)it's a dynvar and it's set by the RT so you can set! it for thread-local scope#2019-04-0419:03Alex Miller (Clojure team)or you can set it up in the data_readers.clj file#2019-04-0419:10orestiswhat about the binding suggestion in the docs? Would that work?#2019-04-0419:12Alex Miller (Clojure team)it should, but seems inconvenient for this#2019-04-0419:13Alex Miller (Clojure team)
user=> (binding [*data-readers* {'foo (fn [x] ['a x])}]
         (let #foo 5 a))
5
#2019-04-0419:15orestisHm, I was doing something wrong. Oh well, thanks for clarifying. Doing it as a tagged literal seems weird but I donā€™t want to write a separate let macro (perhaps in practice it wonā€™t be an issue). Thanks again :)#2019-04-0419:18bronsawhat, that should not work#2019-04-0419:18bronsa@alexmiller was that from the same repl you set! earlier? I think it was only working because of that#2019-04-0420:37Alex Miller (Clojure team)ah, it was#2019-04-0602:08gfredericksset! in that context, used as a past tense verb, is pronounced "set-bung"#2019-04-0612:54ikitommiIs there a way to construct the selection in s/select programmatically? e.g. have the selection somewhere as data (in db, edn, etc) and make a select out of it at runtime.#2019-04-0612:55ikitommisame for s/schemaā€¦#2019-04-0612:59ikitommialso, could the selection syntax allow something special that effects the whole keyset, not present an individual key. Could be just a function or a special Protocol.
(s/def ::passwords-must-equal #(= (:password1 %) (:password2 %)))

(s/select ::user [::user ::password1 ::password2 ::passwords-must-equal]) 
;; or
(s/select ::user [::user ::password1 ::password2 #(= (:password1 %) (:password2 %))]) 
#2019-04-0613:07ikitommiā€¦ if that special keyset-validator would be implemented with a protocol, it would be great to have a function which takes both the spec and the value as arguments, so one could easily write something like ::closed-keyset: the function could extract the list of defined keys from the spec argument and use that to verify that the value doesnā€™t have any extra keys. And if the validation could return the ::s/problems instead of just ::s/invalid, it would be easy to write those, e.g. no need to write separate explain too.#2019-04-0613:40metametadata+1, would be nice to have something like that. because currently relying on forms, macros or low-level *-impl helpers is quite cumbersome (e.g. see my attempt at implementing merge-able closed keys spec with user-friendly explain here: https://groups.google.com/d/msg/clojure/duY3ojPwPYo/Wgvk9PsaCAAJ)#2019-04-0614:36ikitommiI too have almost mergeable closed specs with Spec1, but next hurdle is to get deep-mergeable specs...#2019-04-0617:40Alex Miller (Clojure team)You have two routes to programmable specs in spec 2#2019-04-0617:40Alex Miller (Clojure team)You can make forms, and call spec*#2019-04-0617:41Alex Miller (Clojure team)Or you can make your own syntax that wraps something with defop#2019-04-0617:41Alex Miller (Clojure team)Or actually third option is making your own spec instance from scratch but thatā€™s a much heavier lift#2019-04-0617:42Alex Miller (Clojure team)We are evaluating support for preds in specs - tbd#2019-04-0617:43Alex Miller (Clojure team)But we are working on another kind of checking for closed specs#2019-04-0617:43Alex Miller (Clojure team)Sorry, gotta run
#2019-04-0703:57didibus> But we are working on another kind of checking for closed specs That's really good news. I've had to implement closed keys spec in spec 1 to deal with the following three common case of defects: 1. Keys are mistyped, wrong qualified ns or name, like thinking it is id instead of identifier. Or just plain typo. 2. Keys are misplaced. Like putting the address on the user when it actually goes in the nested profile map. Address on profile being optional, open specs don't catch this bug. 3. Combinations are mistaken. Like say a map which has a query and insert key, this is supposed to be interpreted by some engine, but it is now ambiguous if the operation is supposed to be to query or to insert.#2019-04-0704:01didibusI don't favour the use of closed specs, because they can cause unecessary breakage. And I'm not saying closed specs is the best solution to these above issues. But my team has faced these often, leading to real production bugs. And I haven't found a better solution to them yet.#2019-04-0704:04didibusThe only better idea we had was to seperate the spec of the producer from the one of the consumer. Specifically, code that looks to an input map to read from doesn't need to validate the input as being closed. As all the above bugs are on the producing code. That way, consumers are not broken by producer accretion that are backward compatible.#2019-04-0704:04didibusBut producers would validate their output maps using Closed spec. To ensure they didn't make any of the above mistakes.#2019-04-0704:05didibusWe never implemented it, because with spec1, it would be too tedious to keep seperate producer/consumer specs.#2019-04-0705:37orestisYeah, catching typos is a great use of closed specs. I was attempting going at it from a usage point of view, warning if you are trying to a lookup a key that isnā€™t guaranteed to be there. #2019-04-0705:37orestisA spec-aware linter would be great.#2019-04-0706:56ikitommi@orestis maybe you could work with @borkdude on clj-kondo for that? Would be awesome.#2019-04-0706:57orestisYeah, I was actually looking into this before. A tricky thing is that you probably need runtime info to get to the spec registry #2019-04-0706:58orestisYou could perhaps just collect s/keys calls statically for a good first approximation though. #2019-04-0707:00ikitommi@didibus there is https://github.com/bhauman/spell-spec for spec1 just for that. Sadly, it also macros so can't program/compose with those.#2019-04-0707:17ikitommi@alexmiller thanks for the info. My use case is this: reitit components can declare partial specs for the route data they are mounted to, e.g. a parameter-coercion-interceptor says that the route data can have :parameters key, with a map with keys :query, :path etc. for a given route. All the partial specs of the components mounted to a route will be deep-merged for the final effective spec. e.g. interceptor requiring data of form {:parameters {:path ...}} and another with {:parameters {:multipart ...}} should be merged to require effectively {:parameters {:path ā€¦, :multipart ā€¦}}. And there should be an way to validate that merged spec as ā€œclosedā€ to fail-fast on typos. I have just that working using data-specs + spell-spec + spec-tools coercion and will demo that soon, but I would want the users of the library not to require anything else than the core.spec to define the component specs. With spec2, I would like to support the selection / schema syntax as data so that one could say that ā€œmy component requires [{:parameters [{:query ::parameter-map}]}] and the other [{:parameters [{:multipart ::parameter-map}]}] and I could just merge those + spec to close them (with a small helper) into [{:parameters [{:query ::parameter-map} {:multipart ::parameter-map} ::closed-spec]} ::closed-spec] kinda thing, that could be used to check the strict configuration.#2019-04-0707:20ikitommialso, noticed that the nested non-qualified keys donā€™t work yet, is that a bug or a feature? [{:a int?}] seems to work, but [{:a [{:b int?}]}] doesnā€™t.#2019-04-0711:11Alex Miller (Clojure team)It works if you wrap the inner in s/schema (or register the inner with a name)#2019-04-0707:27borkdude@orestis > You could perhaps just collect s/keys calls statically for a good first approximation though. Thatā€™s a nice idea. Iā€™ll consider that for clj-kondo.#2019-04-0707:29ikitommiwith top-level merging & spec1#2019-04-0707:32borkdudesomething like this? https://github.com/borkdude/clj-kondo/issues/56#2019-04-0707:58didibusThat would be neat yes#2019-04-0707:59didibusOne thing I was thinking also is to make a custom validate function which can be called with an option to be closed or open.#2019-04-0708:00didibusThat way I could use the same spec, but have the producer code validate it assuming no added keys can be present#2019-04-0711:12Alex Miller (Clojure team)Thatā€™s exactly the direction weā€™re going #2019-04-0711:12Alex Miller (Clojure team)Itā€™s in how you check, not in the spec#2019-04-0711:38orestisThat makes a ton of sense!#2019-04-0715:26gfredericksliterally friday at work somebody was debugging some code that walks & transforms a plumatic/schema to add openness everywhere, with exactly the same motivation#2019-04-0719:00quollIā€™m pushing up against leaving for my daughterā€™s swimming training, so I figure I should ask in hereā€¦ Iā€™m using an ns form with metadata, in the same way that clojure.core does. i.e.
(ns ^{:doc "some docs" :author "me"} my-ns)
#2019-04-0719:00quollHowever, this fails on Clojure 1.10, due to ā€œExtra inputā€#2019-04-0719:01gfredericksworks fine for me if I paste that in a repl#2019-04-0719:01quollsure, I can just go back to a standard docstring and drop the :author metadata, but given that clojure.core uses this I thought Iā€™d question it#2019-04-0719:01quollyup, same here#2019-04-0719:01quollbut leiningen hates to build it šŸ™‚#2019-04-0719:02quolllet me put together something minimal#2019-04-0719:02gfredericksyeah, I just tried it in leiningen and it worked fine#2019-04-0719:03gfrederickshalf of the time when I'm debugging something, trying to get a minimal reproduction also exposes the real problem#2019-04-0719:09quollOK, itā€™s looking like I had a typo elsewhere that got through the compiler before spec was introduced šŸ˜³#2019-04-1114:48vemvIs there something like (describe-spec :my/spec) that will print what :my/spec is about? Maybe recursively. Loosely analog to macroexpand/`macroexpand-all`#2019-04-1115:02vemv(for now I use Emacs niceties but I'm interested in editor-agnostic tooling)#2019-04-1115:19Alex Miller (Clojure team)doc works on registered specs#2019-04-1115:20Alex Miller (Clojure team)it is, however, not deep/recursive#2019-04-1115:20Alex Miller (Clojure team)(doc :my/spec)#2019-04-1116:38fedregHi all, can someone help me with specā€™ing a higher-order fn? One of the arguments to a fdef is a higher order fn that Iā€™ve defined as:
(s/def ::foo-fn (s/fspec :args any? :ret ::foo)) 
This works except running something like test.check on the fn that takes foo-fn as an arg is super slow. ā€¦If I just pass ::foo in instead of ::foo-fn everything generates as quickly as usual so Iā€™m guessing Iā€™m doing something wrongā€¦ šŸ˜ž . Any thoughts? thx!! ā€¦ohā€¦ and ::foo has no args but not sure how to express that.. :args nil doesnā€™t work
#2019-04-1117:06Alex Miller (Clojure team):args (s/cat) is prob best#2019-04-1117:07Alex Miller (Clojure team)fspecs are instrumented by generating args and invoking the passed function a bunch of times#2019-04-1117:07Alex Miller (Clojure team)this is sometimes surprising and/or bad for people#2019-04-1117:07Alex Miller (Clojure team)an alternate option to using an s/fspec here is to just spec it as ifn?#2019-04-1117:10fedreg@alexmiller Will try those out! Thanks!!#2019-04-1117:18fedregā€¦Using :args (s/cat) instead of any? definitely sped things up.. Canā€™t use ifn? for my specific use case but still generating / running through my system about 5 results per second which is pretty fast. ā€¦just spoiled by how fast test.check usually runs data!#2019-04-1117:34Alex Miller (Clojure team)any? can generate very large nested colls#2019-04-1117:35Alex Miller (Clojure team)(s/cat) can only generate one thing, ()#2019-04-1117:43fedregyes.. I knew that any was slowing things down but couldnā€™t figure out how to do no argsā€¦ Didnā€™t think to try (s/cat) so thanks for that!#2019-04-1217:06fedregCan anyone point me to an example of passing custom generators into the options of spec.test.alpha/check?
:gen        map from spec names to generator overrides
ā€¦I need to limit the range of some generated ints and canā€™t seem to get it. thx!
#2019-04-1506:23Jakub HolĆ½ (HolyJak)Given a spec, how can I generate random value(s)? Does spec.gen/generate take a spec or is there another fn? Thank you!#2019-04-1507:46jaihindhreddyclojure.spec.alpha/gen takes a spec and returns a generator for it. clojure.spec.gen.alpha/sample takes a generator and gives you a few examples. Takes an optional number of examples to generate clojure.spec.alpha/exercise takes a spec and gives you a seq of pairs, each containing generated example value, and the same value conformed. Check out their docstrings. Also, it's highly recommended to read the Spec guide in its entirety (https://clojure.org/guides/spec) It clears a lot of these things up (including potential gotchas like you need to include test.check for some of the functionality to work)#2019-04-1515:43jaihindhreddy@holyjak ^#2019-04-1614:22borkdudecan macro spec checking (for core macros) be disabled from the REPL instead of with a Java property?#2019-04-1614:32Alex Miller (Clojure team)not currently. such a thing could be done by just setting the clojure.lang.RT/CHECK_SPECS flag, but that's not public currently#2019-04-1616:40jumarIs there any reason why the clojure.spec.skip-macros isn't re-loaded dynamically in Compiler every time it's needed? Performance?#2019-04-1616:43Alex Miller (Clojure team)Didnā€™t seem necessary#2019-04-1614:33Alex Miller (Clojure team)you could remove those specs from the registry#2019-04-1614:33Alex Miller (Clojure team)s/def with nil will remove#2019-04-1614:34borkdudeI tried (clojure.spec.alpha/def clojure.core/for nil) already, but that didnā€™t do it#2019-04-1614:34borkdudeoh it does work, sorry#2019-04-1614:35borkdudeI was testing if for supported beginning with a let but it doesnā€™t šŸ™‚
(for [:let [x 1] y (range x)] y)
#2019-04-1614:36borkdudeand I wanted to make sure I wasnā€™t getting an error from the spec. thanks#2019-04-1621:18hiredmanI dunno if nit picking on spec-alpha2 is useful feedback yet, but https://github.com/clojure/spec-alpha2/blob/master/src/main/clojure/clojure/spec_alpha2.clj#L51-L54 is checking if an object implements the interface behind the Schema protocol instead of checking for satisfying the Schema protocol (the namespace imports the interface behind the Spec and the Schema protocols, but is careful to use the Spec protocol everywhere and not the interface)#2019-04-1621:47Alex Miller (Clojure team)Yeah, thatā€™s actually intentional for perf, same in spec 1#2019-04-1621:47Alex Miller (Clojure team)Well same for spec?#2019-04-1621:48Alex Miller (Clojure team)May change#2019-04-1621:52hiredmanah, I see, I missed the use of the interface there. and it looks like spec? will return true if you implement the interface, or if you have metadata with the conform* function(satisfy the protocol via metadata), but will return false if you extend the protocol to some type, which seems idiosyncratic#2019-04-1621:55hiredmanI guess I'll stay tuned#2019-04-1622:05Alex Miller (Clojure team)Yeah#2019-04-1622:05Alex Miller (Clojure team)In any case, Iā€™m aware :)#2019-04-1623:05bronsahttps://dev.clojure.org/jira/browse/CLJ-1814 homerdisappear#2019-04-1916:58lilactownI have a multi-spec for different query types to a service. I also want to dispatch on the query type to actually query the service#2019-04-1916:58lilactownwhat's the best way to consolidate these two things?#2019-04-1917:00lilactowncause atm I have:
(defmulti query-type :type)

(defmethod query-type :view/id [_]
  (s/keys :req-un [:content.query/id]))

(defmethod query-type :fragment/id [_]
  (s/keys :req-un [:content.query/id]))
and then for implementation:
(defmulti query (fn [conn q] (:type q)))

(defmethod query :view/id
  [{:keys [url]} {:keys [id]}]
  {:url (str url "/view/" id)
   :method :get
   :as :json})

(defmethod query :fragment/id
  [{:keys [url]} {:keys [id]}]
  {:url (str url "/view/" id)
   :method :get
   :as :json})
and then I need to create a wrapper function so I can fdef the implementation, right?
#2019-04-1918:12mishaafaik spec does not work for multimethods, but you can declare implementation function(s), fdef it/them, and make your multimethod call it/them#2019-04-1921:39lilactownit looks like multi-specs s/keys check all available keys, even when not specified?#2019-04-1921:39lilactownactually not sure if this is multi-specs or s/keys doing this#2019-04-1921:40Alex Miller (Clojure team)s/keys does this#2019-04-1921:41lilactownhrm#2019-04-1921:41seancorfieldIf you have a qualified key that matches a spec, it will be checked, whether it's specified in s/keys or not.#2019-04-1921:41Alex Miller (Clojure team)this is documented in the s/keys docstring#2019-04-1921:41lilactowngotcha - it's just not what I want right now šŸ˜‚#2019-04-1921:56dominicm@alexmiller just read your inside clojure. The api seems very global for closed specs. Have you given much thought to how it might be scoped?#2019-04-1921:58Alex Miller (Clojure team)Scope in what dimension?#2019-04-1921:58Alex Miller (Clojure team)It is very global, like instrumentation #2019-04-1921:59dominicmMaybe with a dynamic binding, so thread local.#2019-04-1922:00dominicmThinking of a web server with many concurrent requests where you want the closed spec on the boundary, but internally within processing the open spec may be used (as other sources are merged).#2019-04-1922:01borkdudeInteresting choice to turn off and on ā€œclosednessā€ for specs. Would it make sense to combine this with select somehow? Like (s/def ::closed-spec (s/select ::open-spec ...))?#2019-04-1922:02Alex Miller (Clojure team)It should work with select too, I just havenā€™t done that yet. Thereā€™s just a lot of copied code there at the moment#2019-04-1922:03borkdudeno, I mean, turn an open spec into a closed one using select, not by a global flag#2019-04-1922:03borkdude(then select would have to support that)#2019-04-1922:03Alex Miller (Clojure team)then, no :)#2019-04-1922:04Alex Miller (Clojure team)Closeness is intentionally not part of the spec language and wonā€™t be#2019-04-1922:04Alex Miller (Clojure team)Itā€™s part of the api#2019-04-1922:04Alex Miller (Clojure team)@dominicm still thinking about this aspect#2019-04-1922:05borkdudebut when you would like to use a spec in different ways, you would have to toggle this flag all the time?#2019-04-1922:05Alex Miller (Clojure team)Thatā€™s the aspect weā€™re think about still#2019-04-1922:06borkdudeI guess you can just make a copy of the spec, like (s/def ::foo ::bar), but if you close bar, will foo also be closed?#2019-04-1922:07Alex Miller (Clojure team)Well thatā€™s not a copy, itā€™s an alias#2019-04-1922:07borkdudeyeah#2019-04-1922:07dominicm@borkdude feels like working around the intentional limitation to make closed&open copies.#2019-04-1922:07Alex Miller (Clojure team)The answer is tricky, and may change#2019-04-1922:09dominicm@alexmiller btw, why the global on/off switch? My reaction was that I'm going to see someone turning it on globally unconditionally in some project and removing the upside of open specs.#2019-04-1922:10Alex Miller (Clojure team)Do you just mean the no-arity version of close/open?#2019-04-1922:23dominicmI do, yeah#2019-04-1922:11borkdudeis s/valid? part of the spec language? why not make it an option to those functions?#2019-04-1922:11Alex Miller (Clojure team)s/valid? is api, not spec language#2019-04-1922:12borkdudeyeah ok. I think having it as an option is more flexible than setting it globally#2019-04-1922:12Alex Miller (Clojure team)Still might do that. Breaks all existing APIs and specs though#2019-04-1922:12borkdudeextra arity doesnā€™t break?#2019-04-1922:13Alex Miller (Clojure team)It will break the protocol#2019-04-1922:14Alex Miller (Clojure team)Weā€™ve also been thinking about a fast s/valid? path. Right now thatā€™s tunneled through the conform* protocol, which does a lot of useless work if just validating. So that could also be a flag, or a separate protocol method#2019-04-1922:14borkdude(s/valid2? ā€¦) šŸ˜‰#2019-04-1922:15Alex Miller (Clojure team)Those could all be flags internally on conform*#2019-04-1922:15Alex Miller (Clojure team)We are already in a new namespace, no need to number#2019-04-1922:16Alex Miller (Clojure team)But ideally could minimize breakage for some of those spec libs#2019-04-1922:17Alex Miller (Clojure team)Anyhow, what Iā€™m interested in hearing now is how you would use it#2019-04-1922:17Alex Miller (Clojure team)We can work out the best answers from that#2019-04-1922:17borkdudea fast s/valid? path would certainly help things like core specs which quickly become noticable in performance#2019-04-1922:18Alex Miller (Clojure team)Itā€™s not going to be like 10x or anything#2019-04-1922:18Alex Miller (Clojure team)It really matters when you are constructing conformed outputs#2019-04-1922:18borkdudeI think the closedness aspect is something where people choose different tools or use workarounds for spec right now at API level things, like maybe Reitit and yada (which now uses Schema), but Iā€™ll let those maintainers chime in#2019-04-1922:22Alex Miller (Clojure team)Like how often are you likely to use the same spec in both open and closed check modes on the same spec? And where different, is it per usage or per thread or per what?#2019-04-1922:27dominicm@alexmiller I can imagine closed for the incoming boundary, open internally when merging additional internal data, and potentially closed again when sending it on to another consumer (e.g. A service which accepts too much or a database)#2019-04-1922:29borkdudethatā€™s exactly how I would use it too and probably in combination with select#2019-04-1922:30dominicmI'm not the maintainer of yada, but I'm influential there. For incoming web requests, I'm more interested in: - not checking extra keys - not seeing extra keys#2019-04-1922:31dominicmI don't want to throw if there's extra keys, probably, just silently drop them.#2019-04-1922:33dominicmSorta, but recursive :)#2019-04-1922:57seancorfieldMy first reaction is: ugh, no! Mutable global state on spec behavior just feels horribly wrong. I could have a spec defined in one place and just looking at that and the uses of it no longer tells me whether extra keys are permitted or not, because any random code, anywhere in the system can toggle that setting :face_vomiting: :exploding_head: šŸ˜ #2019-04-1922:58seancorfieldIt seems like the absolute worst of all worlds, to be honest.#2019-04-2000:27droneso, we use records and have a macro that generates a spec which includes checking fields as well as asserting the object is an instance of the backing record class (the latter part can also be used in isolation as a fast check using instance?). because of this, I didnā€™t realize keys generates open specs#2019-04-2000:30droneah, partial nm. forgot about :req and :req-un, which we use behind the scenes#2019-04-2000:30dronewhatā€™s the use case for a spec that describes keys but permits maps that have none of those keys?#2019-04-2000:32droneand agreed with above that mutating closedness seems like a bad idea#2019-04-2000:55seancorfield@mrevelle One observation about records is that they are open too -- you can assoc in any additional keys you want and it remains an instance that record type.#2019-04-2000:56droneyeah, and that makes sense. but my expectation is that a record (or keys spec) provides a base set of keys that should be present#2019-04-2000:56seancorfieldMy view of :opt in s/keys is that it mostly exists for generative testing (and partly for documentation).#2019-04-2000:57drone
(s/def ::f string?)
(s/def ::l string?)
(s/def ::s (s/schema [::f ::l]))
(s/valid? ::s {::x 10})  ;; "extra" keys are ok
;;=> true
#2019-04-2000:57dronethat seems wrong. but maybe Iā€™m missing something#2019-04-2000:58seancorfieldselect is how you specify required-ness in spec2.#2019-04-2000:58droneso the default, when no select is provided, is to not include anything?#2019-04-2001:04seancorfieldhttps://github.com/clojure/spec-alpha2/wiki/Schema-and-select#2019-04-2001:05seancorfield"Schemas do not define "required" or "optional" - it's up to selections to declare this in a context."#2019-04-2001:06seancorfield
user=> (s/valid? (s/select ::s [::f ::l]) {::x 10})
false
user=>
#2019-04-2001:07dronethanks, sorry for the easily researched question. and ughā€¦#2019-04-2001:07seancorfield
user=> (s/valid? (s/select ::s [::f ::l]) {::x 10 ::f "F" ::l "L"})
true
user=> 
#2019-04-2001:11seancorfieldThe schema/`select` stuff is going to be very helpful for us -- it will simplify our API specs quite a bit. Instead of needing to have different specs for otherwise very similar API calls, we can have a schema for the "family" of parameters that can be passed in and then use select to specify what a particular API requires.#2019-04-2001:12seancorfieldI'm still thinking about how it can help with the specs around our database, but I think it can help with the required-for-insert vs required-for-update scenarios.#2019-04-2001:17droneyeah, sub-spec selection seems like a good idea. I think itā€™s just that the undefined default behavior is to not include any of the keys/fields#2019-04-2001:18drone
spec2-scratch.core> (s/valid? ::s {::x 10 ::f 10})
false
#2019-04-2003:58seancorfield@mrevelle This is not valid because ::f's value is not string? per your spec for ::f. That makes perfect sense to me.#2019-04-2003:59seancorfields/keys works that way today.#2019-04-2004:01droneOh, I donā€™t disagree. Was pointing out that a unselected spec field becomes active when it matches a key even though itā€™s not selected.#2019-04-2004:01seancorfieldYes, which makes sense in the context of the way Spec has always worked.#2019-04-2004:02seancorfieldIf you added a spec for ::x as string? then (s/valid? ::s {::x 10}) would fail -- because 10 is not string?.#2019-04-2004:03seancorfieldThat's how s/keys works today. That's how s/schema and s/select work in Spec2: if you provide additional keys, they're still checked if there's a spec for them.#2019-04-2004:05seancorfield
user=> (s/def ::y string?)
:user/y
user=> (s/valid? (s/select ::s []) {::x 10})
true
user=> (s/valid? (s/select ::s []) {::x 10 ::y 11})
false
user=> (s/valid? (s/select ::s [::f]) {::x 10 ::y 11 ::f "s"})
false
user=> (s/valid? (s/select ::s [::f]) {::x 10 ::y "a" ::f "s"})
true
#2019-04-2004:06droneagain, I understand and am not disagreeing with you. But you pointed out that the docs for spec2 say use of a schema without select is undefined and Iā€™m providing examples of how saying something is undefined doesnā€™t mean there isnā€™t a default#2019-04-2004:07droneand for my use case, itā€™s a poor default#2019-04-2004:07seancorfieldI didn't say it was undefined (as in "undefined behavior"). I quoted the part that says schema says nothing about required-ness -- that's what select is for -- and that decoupling is exactly what Rich was talking about in Maybe Not.#2019-04-2004:09seancorfieldWhat I do think is odd, right now, is that you can select keys that are not in the .#2019-04-2004:10dronefrom the docs: > Schemas do not define ā€œrequiredā€ or ā€œoptionalā€#2019-04-2004:11droneand that is weird#2019-04-2004:11seancorfieldI disagree. It's exactly what Rich talked about.#2019-04-2004:11seancorfieldThis, on the other hand, seems wrong:
user=> (s/def ::s (s/schema [::f ::l]))
:user/s
user=> (s/valid? (s/select ::s [::f ::x]) {::f "s"})
false
user=> (s/valid? (s/select ::s [::f ::x]) {::f "s" ::x 10})
true
#2019-04-2004:12seancorfield(there's no spec for ::x but the s/select call makes it required, even tho' it isn't part of ::s)#2019-04-2004:12droneI guess my point is, based on Richā€™s talk and the current spec2 docs, (s/valid? ::s x) shouldnā€™t be permitted since the schema is being used without a select#2019-04-2004:13seancorfieldIn the first cut of schema/`select` that landed in the repo, schemas were not specs, so that was not allowed.#2019-04-2004:13seancorfieldI'm not sure why the change was made to turn schemas into specs.#2019-04-2004:15droneah, I see#2019-04-2004:16seancorfieldThis is the commit that changed that https://github.com/clojure/spec-alpha2/commit/1aa141dcdadca88c25afa73e486b3707eaed9d99#2019-04-2004:17droneIā€™m mostly just bummed that spec seems to be going away from being a more-flexible structural typing substitute and into something that Iā€™m not sure fits at all with how I use Clojure#2019-04-2004:18dronethanks for the link#2019-04-2004:22seancorfieldI'm not sure I follow you... select is much more powerful than s/keys -- and now you can define the overall shape of your data once, and then validate against the relevant parts of it as needed, without having to write multiple s/keys specs that all overlap.#2019-04-2004:24seancorfieldReading over the wiki page again in more detail, I wonder if the drive for that change (the commit above) came from wanting to treat schemas like specs because of data generation?#2019-04-2004:26seancorfieldThere's an example of generating against a schema, which produces random selections of keys from the set in the schema. Which would all be "valid" in any context that accepted that schema without placing any required-ness constraints on the contents. Seems like a slim driver to me tho'...#2019-04-2004:32dronethatā€™s true, itā€™s not that something canā€™t be done. but itā€™s unclear how compatible it is with other use cases. donā€™t want to end up working against the language#2019-04-2004:33droneand I think youā€™re probably right about generation as motivation#2019-04-2017:40seancorfieldAlex responded to my question on that commit: "The big thing that tipped it is the unqualified key support which does enforce the specs of unqualified keys (in addition to checking the values of any qualified keys against the registry)." -- because the unqualified key schemas include the spec and don't exist outside the schema (so the schema is a useful spec mostly for unqualified keys which would not otherwise be checkable).#2019-04-2001:18dronebut I didnā€™t select ::f from ::sā€¦#2019-04-2001:24ikitommi@alexmiller About the closed specs. I agree on @dominicm on the open/closed on the borders. We want to drop out extra keys at the borders (recursively) and also coerce the values. Inside they can be open. Spec-tools (and by so, reitit) does both stripping extra keys & coercion already, but using the s/form parsing, which feels wrong. There is CLJ-2251 I wrote about the different ways to walk the Specs from inside, not outside. Also the fast path of just validating.#2019-04-2001:25dronei.e., based on what Rich presented and my lurking on here, Iā€™d expect select to be used to choose a subset of keys when the full record isnā€™t expected or needed. having it default to none seems silly when itā€™s much easier to write (s/select ::s []) when none are required than enumerate all the fields in a spec when you want all to be required. I also think itā€™s less surprising, but thatā€™s subjective#2019-04-2007:25seancorfield> when itā€™s much easier to write (s/select ::s []) when none are required than enumerate all the fields in a spec when you want all to be required FYI (s/select ::s [*]) is the "all fields required" case#2019-04-2015:47dronecool, thatā€™s a bit better#2019-04-2001:34ikitommiI would put the closed-checking into select: Schemas would be open by default, but one can define select default to open or closed maps, which can be explicitly defined to different (sub)selects too.#2019-04-2001:35ikitommi
(s/select ::user [::id ::name ::address {::address [::street s/closed]} s/closed])
#2019-04-2001:37ikitommi
(s/select ::user [::id ::name ::address {::address [::street]}] [s/closed]) ;;3rd arg will be merged into all submaps
#2019-04-2001:46wilkerluciomaybe moving to the select get back on the closed problem again, it will always be closed, I agree with @borkdude in this one, maybe the point to make this decision is s/valid?, this way you keep the possibility to stay open, but can do closed checks when it makes sense#2019-04-2001:53ikitommiwith s/valid? is would be all or nothing. In s/select, one could close it just partially. Not sure if there is a real life case for that thou.#2019-04-2405:48didibusIt seems you register which spec are to be validated as closed and open. So the s/valid could be partially closed as well, if you broke it up into named specs.#2019-04-2003:24eggsyntaxIndependently wrote a response to the open/closed approach which pretty much echoes the same things other folks are saying above ā€” global-stateful-ness bothers me, what about threads, etc. The need for closedness seems to me like itā€™s usually context-specific and scope-limited, so I was expecting a limited-scope approach. My first inclination would be to be able to do something like
(s/valid? (s/closed-variant ::bar) my-bar)
but I understand youā€™re trying to make sure it doesnā€™t become part of the spec language. If itā€™s about limiting it to the API, what about a separate s/closedly-valid? (or strictly-valid? or strictly-correct? or whatever)? And maybe an equivalent for s/explain. Or I like what I think @wilkerlucio is suggesting, making it available as an extra-arity arg to s/valid? (off by default so it doesnā€™t break existing code).
#2019-04-2003:56seancorfield@eggsyntax My first thought was s/valid-closed? but then you need a variant of each of the explain* functions and conform and probably several others... which gets ugly fast...#2019-04-2020:42eggsyntaxWhat about maybe a separate s/closed? function that just verifies whether the data structure contains only the keys in the spec? It doesnā€™t seem like a big burden to do
(and (s/closed? ::spec x) (s/valid? ::spec x))
#2019-04-2007:29metametadataIf closed/open status becomes an option to some s/valid* fn then I don't see how the closed spec can be specified to validate args in defn-spec or something like that. So "closedness" should be attached to the spec instance. And I guess I'd also intuitively expect (s/close-specs ::s) to return a new instance of the spec. Even though I don't yet have cases where I need to open or close the already created spec, I'd expect it to return new values instead of "mutating" existing ones. Otherwise, there's a risk to have all the old problems with mutable data like threading, reasoning about code, etc.#2019-04-2017:30adamShould I be using spec for server side form validation or https://funcool.github.io/struct/latest/ or something else?#2019-04-2017:41seancorfield@somedude314 We use Spec for that. We use it for API parameter validation, server-side form validation, and several other places in our production code.#2019-04-2017:42adam@seancorfield do you use any helper libraries built on top the spec or just vanilla spec?#2019-04-2017:44seancorfieldJust vanilla spec. We wrote some code on top of explain-data to turn Spec failures into domain-specific error messages but that's it.#2019-04-2017:45seancorfieldAnd we mostly do it like this:
(let [params (s/conform ::my-spec (:params req))]
  (if (s/invalid? params)
    (report-error (s/explain-data ::myspec (:params req)))
    (process-form-data params)))
(pseudo-code)
#2019-04-2017:47adamCool thanks. I will give it a try. This is the first time I am trying to validate stuff in Clojure so still unsure what is practical and what is not.#2019-04-2022:47ikitommiDid a demo in Clojure/North about closed specs, with spec.alpha, spec-tools, spell-spec and expound, using the new reitit error formatter: https://vimeo.com/331602826. The code calls spec-tools.spell/closed-spec function on a spec and gets a closed (spell-)spec back.#2019-04-2320:45robertfwI have a question about how people handle writing specs for an API endpoint. I'm speccing out ring request maps, and have several endpoints in the same namespace. a ring spec typically involves providing specs for :status, :params, sometimes :body. However my spec names end up being quite wordy; for example, the expected body for a successful response of a given endpoint ends up being :my.ns.endpoint-name.response.success/body. Over the course of defining several endpoints, with potentially multiple possible responses, I end up with quite a lot of typing. I've been considering either moving each endpoint spec into its own namespace, so that I can use :: shorthand, but I'm not wild about the proliferation of files that will create. My other thought is to create some macro or function to help generate these specs. Any thoughts?#2019-04-2321:35seancorfield@robertfrederickwarner You can create a namespace alias without needing an actual namespace file.#2019-04-2321:53robertfwthis looks to work nicely šŸ™‚ thanks#2019-04-2405:23didibusWow didn't know that#2019-04-2321:36seancorfield
(alias 'm  (create-ns 'ws.domain.member))
and then use ::m/whatever
#2019-04-2321:37robertfwah, I did not know about alias. I had considered trying using (ns ...) but rejected it as it seemed to be pretty unidiomatic#2019-04-2321:39seancorfield@alexmiller has mentioned the possibility of improving alias usage but I don't think any concrete proposal has been made yet.#2019-04-2321:39robertfwIt would be nice if it were possible to define a map spec assigning specs to given keys; I know that seems to be going against the grain of spec but it would cut down some lines of code needed to spec out json endpoints#2019-04-2321:40robertfwe.g... we get data coming in as :some_value whereas in our system it's :some-value, so there's another spec to handle the conversion#2019-04-2321:40robertfwI've been digging around to see if anyone has written helpers for that but no luck#2019-04-2321:43Alex Miller (Clojure team)spec2 has support for this for unqualified keys in s/schema#2019-04-2321:43robertfwprayers: answered#2019-04-2321:44Alex Miller (Clojure team)https://github.com/clojure/spec-alpha2/wiki/Schema-and-select#2019-04-2321:45robertfwIs there a rough idea of when spec2 might be "ready"?#2019-04-2321:46seancorfieldDefine "ready" šŸ™‚ We have a branch of our codebase at work that runs on Spec2.#2019-04-2321:48robertfwgood question - I guess, active development finished and moving into a phase of broader community use, similar to where spec.alpha is now#2019-04-2321:52Alex Miller (Clojure team)I would say it's likely on order of months given the set of work we have talked about#2019-04-2321:53seancorfieldCould you give us an overview of what additional work on Spec2 is still in the pipeline?#2019-04-2321:53Alex Miller (Clojure team)hard to say whether that's 2 months or 6 but certainly I would love have it before the conj#2019-04-2321:54robertfwthat's what i suspected. thanks šŸ™‚#2019-04-2321:58Alex Miller (Clojure team)we have a list of 10 areas of work for spec2#2019-04-2322:00Alex Miller (Clojure team)some of those we'd like to work through before "release" and re-integration, some could be add-ons later#2019-04-2322:01seancorfieldI'd like to cut our project at work over to Spec2 but I don't know how much rework to expect (I mean, I'm used to some level of rework given how often we rely on Clojure alphas šŸ™‚ )#2019-04-2322:03Alex Miller (Clojure team)you mean rework between now and "the end" ?#2019-04-2322:05seancorfieldBetween now and non-alpha status.#2019-04-2322:05Alex Miller (Clojure team)well, I wouldn't recommend that yet but I'm not expecting that the existing apis are going to change much#2019-04-2322:05seancorfieldAlthough we're heavily reliant on Spec1 Alpha in production so I don't know why I'm all that worried...#2019-04-2322:11eggsyntax@alexmiller don't know if you already saw it, but there are a few more ideas above on s/closed.#2019-04-2518:33pyrHi#2019-04-2518:33pyrnow that usage of spec is pervasive and given the fact that defrecord produces handy functions#2019-04-2518:35pyrwould it make sense to add another one which would create a predicate#2019-04-2518:35pyrto test for protocol satisfaction?#2019-04-2518:36pyri.e (defrecord MyRecord) ;; => implies ->MyRecord, map->MyRecord, and MyRecord?#2019-04-2518:38borkdudeyou mean like (instance? Foo (->Foo))?#2019-04-2518:38dronewhere MyRecord? is a predicate returning true if its argument is an instance of the backing MyRecord class and/or it has the required record fields with values that match some field specs?#2019-04-2518:39pyr@mrevelle: yes#2019-04-2518:40pyrthe former#2019-04-2518:40pyrmeh, now that I say it out loud, it makes more sense for protocols#2019-04-2518:40pyrnevermind#2019-04-2518:43droneI think thereā€™s value in specā€™ing fields of records (just as there is in specā€™ing plain-old maps). as for protocols there are some technical reasons they arenā€™t supported. the workaround that was thrown around in the past is having protocol method implementations call a specā€™d function#2019-04-2518:45pyrok#2019-04-2518:48dronere: records. it sounds like most people donā€™t use them (?) and stick with maps and occasionally add some :tag or :type entry to achieve the ability to determine the ā€œkindā€ of map#2019-04-2518:51dronenot sure if records were too JVM-centric or if they feel too restrictive to the free-spirited majority opposed to types#2019-04-2518:52borkdudewerenā€™t they a little bit better for performance?#2019-04-2518:53borkdudealso in Clojure Applied it is recommended for ā€œdomainā€ objects, although I rarely see them used#2019-04-2518:53pyrI like the fact that you can directly attach protocols#2019-04-2518:54droneyeah, you get fast single dispatch (like a Java class, since theyā€™re implemented as Java classes)#2019-04-2518:54Alex Miller (Clojure team)fwiw, I would probably de-emphasize records in a 2nd ed of Clojure Applied#2019-04-2518:56pyrextend-via-metadata will help with that#2019-04-2518:57droneI use them all over in machine learning, data analysis, and program analysis work#2019-04-2518:57pyr@alexmiller so fewer records but protocols still emphasized, correct?#2019-04-2518:58pyrspec and qualified keywords make maps more attractive indeed#2019-04-2518:58pyrbreaking out of the protocol + record-implementation will be hard#2019-04-2518:58pyrespecially in our component heavy codebase#2019-04-2518:58droneseems like thereā€™s plenty of boiler plate that packaging things as records could manage for you#2019-04-2519:00dronedefrecord-spec where each field has an associated spec. specā€™d record creation functions, reference the field specs elsewhere when performing a subset select with select#2019-04-2519:01dronethe only place I see them being awkward is when youā€™re building up the fields for a record. which I believe is an example Rich referenced in Maybe Not or another recent-ish talk#2019-04-2519:02dronebut, itā€™s not that bad to just use maps with a sub-select spec until youā€™ve gathered all the required fields and then throw it into a record#2019-04-2519:04dronefeels like serialization is motivating much of this. e.g., you may have some fields that are handy but can be computed from other fields in the map. so when you serialize maps for storage in a database, you drop those derived fields. but then results from the database are ā€œincompleteā€ and processing is required to generate the derived fields before you could construct the record#2019-04-2519:08dronedatomic killed records, is what Iā€™m saying (with tongue in cheek)#2019-04-2710:55kommenplaying around with spec-alpha2, I was wondering if there is a way to make s/conform of an s/select only return keys of a map which are actually specified in the select.#2019-04-2710:56kommen
(s/def :foo/id string?)
(s/def :foo/text1 string?)
(s/def :foo/text2 string?)
(s/def ::foo (s/schema [:foo/id :foo/text1 :foo/text2]))
(s/def ::foo-only-text1 (s/select ::foo [:foo/id :foo/text1]))

(s/conform ::foo-only-text1 {:foo/id "bar" :foo/text1 "t1" :foo/text2 "t2"})
;; => #:foo{:id "bar", :text1 "t1", :text2 "t2"}
#2019-04-2716:15Alex Miller (Clojure team)No, by design#2019-04-2716:16Alex Miller (Clojure team)Itā€™s not working this way at the moment but select over a closed schema should throw invalid on that case #2019-04-2716:16Alex Miller (Clojure team)The close stuff is going to get completely overhauled though so current api will change #2019-04-2914:02dominicmI was curious about this, could you share what the current thinking is? Even with the disclaimer of it being heavy WIP?#2019-04-2914:05Alex Miller (Clojure team)rather than the stateful api (which was intended to be similar to instrument), will be more op-specific, and actually more generic than just "closing", will open up some api space for other kinds of checking variants#2019-04-2914:05Alex Miller (Clojure team)so, new options on valid?, conform, assert, etc - anything with checking#2019-04-2914:11dominicmSounds really interesting. Looking forward to seeing how much leverage this provides.#2019-04-3008:20ikitommichecking variants like runtime transformations / coercion? or just validating? anyway, generic sounds great. :+1:#2019-04-3011:41Alex Miller (Clojure team)Not transformations#2019-05-0115:03metametadata> so, new options on valid?, conform, assert, etc - anything with checking it looks like it will create friction for adopting closed maps in defn-spec and other libs which don't know about new options in the methods and rely on the fact that every spec itself describes everything needed for validation#2019-05-0115:05Alex Miller (Clojure team)that's something defn-spec will have to reckon with. function specs themselves are going to go through significant changes and making an integrated defn+function spec syntax is in scope.#2019-05-0115:14metametadatathanks. fwiw, this is the typical way I add specs to protocol methods using defn-spec at the moment (`sp` here contains custom spec helpers):
(:require [common.logic.specs :as specs]
            [spec-plus.core :as sp]
            [clojure.spec.alpha :as s]
            [defn-spec.core :as ds]))

(defprotocol Protocol
  (-fetch-places [_ postal]))

(ds/defn-spec fetch-places
  {::s/args (sp/pos any? ::specs/postal)
   ::s/ret  (sp/seq-of ::specs/place)}
  [this postal]
  (-fetch-places this postal))
#2019-05-0115:16dominicmI guess there's a open question of whether the function level is the right place for deciding closedness. Maybe the other flags will clarify this.#2019-05-0115:19metametadatayes, it's interesting to see how function specs will play with closed maps. and speaking of that, this is how I use closed maps in functions currently:
(ds/defn-spec build-and-push
  {::s/args (sp/pos ::component
                    (sp/speced-keys :req-un [::path ::image-id ::tag]))} ; sp/speced-keys implements a closed s/keys spec
  [component {:keys [path image-id tag]}]
...
#2019-05-0115:25dominicmI'm suggesting that maybe it's up to the caller sometimes whether to apply certain constraints.#2019-05-0115:32Alex Miller (Clojure team)these flags will probably be available on calls like instrument or check so you could supply those constraints that way#2019-05-0115:37metametadatawould be cool if it allowed to spec a function in such a way that some args are closed and some are not, e.g. (foo <closed-map> <integer> <open-map>)#2019-05-0115:41dominicmThat sounds like an important filter.#2019-05-0116:58Alex Miller (Clojure team)the planned api will allow that#2019-05-0116:59Alex Miller (Clojure team)well, on a per-spec basis that is. if you want to use the same spec in both modes in a single context, not sure about that (or whether that's even a real use case)#2019-05-0117:28dominicmHmm, merging an old-user with new-user where the new user is coming over the wire and needs to be locked down, but old-user is from the db?#2019-04-2913:21johnhttps://nakedsecurity.sophos.com/2019/04/29/nist-tool-boosts-chances-of-finding-dangerous-software-flaws/ Apparently this just reduces the number of combinations one might test by choosing more unique combos per iteration. Seems like maybe a dimensionality reduction scheme over random parameters. Thought it might be related to spec gen scenarios.#2019-04-3008:25ikitommiBending spec1 to support things that we need for validating configuration, and that hopefully are available with spec2 itself: closing (data-)specs recursively. No macros, just data.#2019-04-3008:26ikitommiuses spell-spec and expound for most of the work.#2019-04-3010:26vemv
(spec/def a int?)
(spec/valid? `a 2) ;; -> true
Sometimes I make this typo (`a` vs. ::a) but it happens to work Is it intentional?
#2019-04-3011:54Alex Miller (Clojure team)Itā€™s a side effect of the impl#2019-04-3011:55Alex Miller (Clojure team)Symbols are used as valid names for function specs (s/fdef)#2019-04-3012:29vemvgot it, thanks!#2019-05-0116:28vemv
(spec/def :foo/bar int?)

(spec/def :baz/quux string?)

(spec/keys :req [...])
Is it possible to define a keys spec such that the :foo/bar key is required, but the :baz/quux spec is validated for that specific key? i.e. I want to override a spec without renaming the key (rationale: :foo/bar is a Datomic attribute. I cannot or don't want to rename it. And :foo/bar has a existing spec, useful/correct 99% of the times)
#2019-05-0116:40valeraukonot that i know of, no#2019-05-0116:42vemvWorking around that with a fn for now šŸ™‚#2019-05-0120:29eskemojoe007I have a hash-map where the keys and values are used almost as a linked list {"a" "b", "b" "c", "c" "a"} where a,b,c could be other strings. This is used in a map where I have a spec, but I don't know how to write a spec that says, let the keys and values be any ol' string.#2019-05-0120:33drone(s/map-of string? string?)#2019-05-0120:38eskemojoe007Well that was easy...#2019-05-0120:39eskemojoe007One more question for now. I have a unique ID for many items. They are used throughout the speced map. When using generators, it doesn't understand the relationships between the IDs. Any way to enforce that?#2019-05-0121:50favilahttps://github.com/reifyhealth/specmonstah is in this space
#2019-05-0120:41seancorfield@david.folkner You can either (s/and (s/map-of ...) uniqueness-predicate) or write your own generator. The former is "easy" if it's able to actually satisfy the criteria. The latter is harder but more likely to work.#2019-05-0120:42eskemojoe007Perfect. I'll take a look at making a generator, that will likely fit my application a bit better. I already have the functions to build up the data for my actual application, so a custom generator can leverage those.#2019-05-0121:13Alex Miller (Clojure team)for the latter, see gen/fmap and/or gen/bind#2019-05-0121:14Alex Miller (Clojure team)http://blog.cognitect.com/blog/2016/8/10/clojure-spec-screencast-customizing-generators might be a good example#2019-05-0301:36Nolanjust trying out alpha2 for the first time and finding (s/or ...) to be StackOverflowErroring on everything i try. seems like im missing something pretty big but havent been able to piece it togetherā€”do the regex macros need to be used with s/spec*?#2019-05-0301:43Nolanah, seems like its nrepl relatedā€¦#2019-05-0301:47Nolanseems to have resolved itself, what a baffling experience indeed.#2019-05-0302:20Nolanis there a concise way of expressing something similar to s/or where data that conforms to multiple predicates returns all of the matches instead of just the first? something similar to:
(s/def ::example (s/... :e even? :s #(< % 42))
(s/conform ::example 2) ;; => ([:e 2] [:s 2])
#2019-05-0302:29Alex Miller (Clojure team)no, not really#2019-05-0302:31Nolangood to know. thank you alex!#2019-05-0312:54jumarI'm not sure how to approach this problem... I have multiple implementations of an "auth provider" (e.g. db, ldap, etc.) and they are represented as a namespaced map like this:
#:auth-provider{:active? true
                                     :type "ldap"
                                     :config {:port 636
                                              :host ""
                                              :bind-dn-format "uid={username},cn=users,cn=accounts,dc=demo1,dc=freeipa,dc=org"
                                              :search-base "dc=demo1,dc=freeipa,dc=org"
                                              :connection-timeout 10000
                                              :response-timeout 10000
                                              :ssl? true}}
The problem here is the :config key. It's widely different across different implementations and I'd like to have different specs for that. My problem is that AFAIK you can have only one spec for such a key - in my case that's fairly generic:
(s/def :auth-provider/config (s/map-of keyword? any?))
(s/def ::auth-provider-spec (s/keys :req [:auth-provider/type
                                          :auth-provider/active?]
                                    :opt [:auth-provider/id
                                          :auth-provider/default-role
                                          :auth-provider/config
                                          :auth-provider/priority-order
                                          :auth-provider/role-mapping]))
Now I want to "override" spec for the :auth-provider/config key for each implemantation - currently I use this hacky approach:
(s/def ::ldap-config (s/keys :req-un [::host ::port ::connection-timeout ::response-timeout]
                             :opt-un [::bind-dn-format ::search-base]))

(s/def ::ldap-provider-spec (s/and
                             ::auth-specs/auth-provider-spec
                             ;; this is less descriptive than using `s/keys` directly
                             ;; but we cannot use `s/keys` because `:auth-provider/config` default
                             ;; spec is already registered in auth-specs namespace
                             ;; and you cannot have two different specs for the same namespaced key (i.e. `:auth-provider/config`)
                             #(s/valid? ::ldap-config (:auth-provider/config %))))
But that's not only awkward it's also problematic from the "error reporting" point of view - in case something inside the :config key is invalid I get pretty useless error message that the spec failed but I don't know why:
... Reason: Invalid LDAP configuration: -- Spec failed ---------
... Relevant specs ------- :auth.providers.ldap-provider/ldap-provider-spec: (clojure.spec.alpha/merge :auth.specs/auth-provider-spec (clojure.core/fn [%] (clojure.spec.alpha/valid? :auth.providers.ldap-provider/ldap-config (:auth-provider/config %)))) ------------------------- Detected 1 error
#2019-05-0312:56jumarI tried merge and multi-spec (might not know how it should be used properly) but failed basically for the same reason - cannot define different specs for the key :auth-provider/config#2019-05-0313:03manutter51Wonder if you could do something with the namespaces on the key? Like, instead of :auth-provider/config for all the variants, have :auth-provider-ldap/config, :auth-provider-db/config, etc.#2019-05-0313:06jumarThat's not an option I'm afraid.#2019-05-0313:04Alex Miller (Clojure team)Youā€™re in the ballpark of s/multi-spec - have you tried that?#2019-05-0313:07jumarAs commented in the reply, I got stuck basically for the same reason (not able to define different specs for the same namespaced-key) - I might be doing it wrong, though.#2019-05-0313:10jumarroughly something like this:
(defmulti auth-specs/provider-type :auth-provider/type)

(defmethod auth-specs/provider-type "ldap" [_]
  (s/keys :req [:auth-provider/config]))

#2019-05-0313:11jumarNow I don't know how to define different versions of :auth-provider/config...#2019-05-0313:20Alex Miller (Clojure team)one way would be to do this one level up - the map containing :auth-provider/config#2019-05-0313:21Alex Miller (Clojure team)or I find it's always helpful to come back to the truth of the matter - what values can a key take on? here :auth-provider/config has multiple sets of things#2019-05-0313:21Alex Miller (Clojure team)so spec it as s/or of different s/keys specs#2019-05-0313:21Alex Miller (Clojure team)(some of which might reuse the same attributes)#2019-05-0313:26jumaridea with s/or is interestig - I'll try that. Thanks!#2019-05-0318:33denikit looks like unform is not aware of nonconforming or am I doing sth wrong?
(s/def ::bar 
  (s/or :baz integer?))

(s/def ::foo
  (s/nonconforming   ; <========
   (s/keys :req-un [::bar])))

(s/def ::tez 
  (s/tuple any? ::foo))

(s/conform ::tez [123 {:bar 4}])
; [123 {:bar 4}]
(s/unform ::tez [123 {:bar 4}])
; Error: nth not supported on this type function Number() { [native code] }
;    at Function.
#2019-05-0318:38deniksmaller example
(s/def ::bar 
  (s/or :baz integer?))

(s/def ::foo
  (s/nonconforming
   (s/keys :req-un [::bar])))

(s/conform ::foo {:bar 4})
(s/unform ::foo {:bar 4})
#2019-05-0322:21Alex Miller (Clojure team)those both roundtrip fine for me w/o error#2019-05-0322:22Alex Miller (Clojure team)are you on cljs? or clj?#2019-05-0617:29denikcljs#2019-05-0617:42denikjust tested and can't reproduce in CLJ.#2019-05-0619:53yuhanIs it good practice to place all specs together in a dedicated my-project.specs namespace?#2019-05-0620:05Jakub HolĆ½ (HolyJak)I do not know but on my project I created a separate .domain ns with specs so that I could share them with multiple namespaces and it is nice not to have it all through the core ns. So it works for me šŸ™‚#2019-05-0620:33yuhanThanks! I was reading a few posts and it seems there isn't really a one "correct" way of doing things#2019-05-0620:34yuhansharing specs between namespaces is the reason I'm thinking of doing it as well, and it probably makes sense for s/fdef specs to remain next to their respective functions#2019-05-0718:26buttergunsI personally put specs above the code, in the same namespace. It makes sense that logically, if I want to see the "Event" spec, it would be in the event.clj namespace for example#2019-05-0718:28buttergunsI also have a common.clj namespace, which contains shared functions. It also contains common specs, like ::timestamp, which is used in many places#2019-05-1015:52rickmoynihanIā€™d say it can make sense for libraries at a certain scaleā€¦ though even there you may just want to put them where you need them. For an application youā€™ll inevitably end up wanting to spread things out.#2019-05-0619:58yuhanI also avoid using namespace alias ::keyword syntax and most of the specs have short 3-8 character prefixes that don't actually correspond to any of my project namespaces, wondering if that's good practice too#2019-05-0620:06Jakub HolĆ½ (HolyJak)They don't need to correspond to actual namespaces, the main thing is I guess readability.#2019-05-0819:55favilawhat's the best way to spec that some relationship holds between the keys and values of a map?#2019-05-0819:56favilain my case I have a map whose keys describe some range, and the vals are vecs of items which should be in the range described by the key#2019-05-0819:57kennys/and?#2019-05-0819:58favila(s/and (s/map-of ::x ::y) kv-rel-holds?)#2019-05-0819:58favilanothing better?#2019-05-0820:00Alex Miller (Clojure team)You can spec it as a coll of entry tuples#2019-05-0820:00Alex Miller (Clojure team)Then you get both k and v#2019-05-0820:02favila(s/coll-of (s/and (s/tuple ::x ::y) kv-rel-holds?) :into {} :kind map?)#2019-05-0820:02favilalike that?#2019-05-0820:02favilagenerator for that seems easier to make#2019-05-0911:07benthis is probably a really simple question, but what are the practical differences between using :pre and :post vs spec/fdef for function validation? As far as I can tell, fdef gives you a bit more control, but otherwise :pre/`:post` are just there because theyā€™re more ā€œlightweightā€/easier to read?#2019-05-0913:31eskemojoe007I'm a noob with this stuff, but I think :pre and :post run at runtime. fdef doesn't run at runtime, only during testing.#2019-05-0913:53benthis rings true with the docs now I read them back, actually#2019-05-0913:53benthanks!#2019-05-0916:46jumarfn specs defined via fdef don't run by default - they run in whatever environment you want if you toggle those via stest/instrument (https://clojure.github.io/spec.alpha/clojure.spec.test.alpha-api.html#clojure.spec.test.alpha/instrument)#2019-05-0916:47jumarAnd yes, it's not usually recommended for production use, only for testing/development#2019-05-1007:08thumbnail:pre, and :post can be toggled by rebinding *assert* as well though#2019-05-1017:41eskemojoe007I'm trying to make a custom generator for the deck of cards example in the spec guide. I have a spec that looks like (s/def ::player (s/keys :req [::name ::score ::hand ::scored])). Each key already has the proper tested generator, but for ::player It shouldn't allow duplicate cards in the hand and scored category for example. I need to make a custom generator for ::player that checks that forces them to be unique.#2019-05-1017:42eskemojoe007I guess I didn't ask a question. Question1: Is it possible to make a custom generator for just part of the ::player?#2019-05-1017:45seancorfieldI'm not sure I'm reading your problem statement correctly: are you just trying to ensure ::hand generates unique cards, and ::scored generates unique cards, or are you also trying to ensure there's no duplication between ::hand and ::scored?#2019-05-1017:45seancorfieldIf it's the former, you can just write custom generators for those specs. If it's the latter, you pretty much have to write a generator for the whole ::player spec.#2019-05-1017:47eskemojoe007Its the later. Given a typical deck of 52 cards. Scored is cards that have scored, then are out of play (Making go-fish).#2019-05-1017:48seancorfieldThen, yeah, you have no choice really -- you need a generator for ::player#2019-05-1017:48eskemojoe007So I have to make a custom generator for the whole ::player and I can't rely on the generator even for the other keys such as ::score?#2019-05-1017:49seancorfieldYour ::player generator will run the generators for the individual elements if it wants default behavior.#2019-05-1017:50eskemojoe007What does that syntax look like?#2019-05-1017:51seancorfield(s/gen ::my-spec) gets you a generator for a spec#2019-05-1017:52eskemojoe007Ahh...perfect!#2019-05-1017:52eskemojoe007That makes sense, just didn't realize thats what it was doing.#2019-05-1017:52eskemojoe007Thank you.#2019-05-1017:54seancorfieldAnd (gen/hash-map ...) to generate the ::player hash map from those generators.#2019-05-1017:54seancorfield(assuming gen is clojure.spec.gen.alpha)#2019-05-1017:58eskemojoe007So a very redundant generator would look something like this:
(s/def ::player (s/with-gen
                  (s/keys :req [::name ::score ::hand ::scored])
                  #(gen/hash-map ::name (s/gen ::name)
                                 ::score (s/gen ::score)
                                 ::hand (s/gen ::hand)
                                 ::scored (s/gen ::scored))))
#2019-05-1017:59seancorfieldYup, that would get you started.#2019-05-1017:59eskemojoe007Perfect. I'll post my solution just for reference, but I appreciate the help.#2019-05-1018:00seancorfieldI'd probably have a spec for a deck of cards with a custom generator to ensure they're all distinct (it could just shuffle an ordered generation of all the cards), then ::scored would be the first random N of those and ::hand would be the next random M of them.#2019-05-1018:04eskemojoe007The ::scored spec generator is weird, it takes exactly 4 of the same rank card to score. So I was going to run that first, then ommit any results from the ::hand based on that.#2019-05-1018:05eskemojoe007But taht seems like a terrible idea.#2019-05-1018:05eskemojoe007I'm gonna make a full deck spec.#2019-05-1317:55eskemojoe007Finally got back to this. I came up with something that looks like
(gen/sample (gen/bind (s/gen ::scored)
                      #(gen/hash-map ::name (s/gen ::name)
                                     ::score (gen/return (/ (count %) 4))
                                     ::hand (gen-hand %)
                                     ::scored (gen/return %))))

(defn remove-by-rank
  [cards ranks]
  (vec (reduce (fn [new-cards rank]
                (remove #(rank-match? % rank) new-cards))
           cards
           ranks)))

(defn gen-hand
  [scored-cards]
  (gen/bind (s/gen ::hand)
            #(gen/return (remove-by-rank % (should-score? scored-cards)))))
#2019-05-1317:56eskemojoe007So it generates the scored cards first, uses gen/bind to get those, and bases the the other generation based on that (I didn't include all the functions).#2019-05-1116:48denikI'm using s/conform to give a richer shape (e.g. places -> names / sequential -> associative) to data flowing through my program. However, after I conformed a value, it trickles through my program and is updated. The crux is that I don't have a spec for the conformed value, which would be useful. Since conform is doing the reshaping work, it must be possible to generate that spec. Currently, I'm hacking this behavior with s/unform and then s/conform#2019-05-1119:11Alex Miller (Clojure team)that's why you shouldn't use conform to transform your data#2019-05-1119:12Alex Miller (Clojure team)conform is designed to tell you how it parsed, not to serve as a general purpose transformation engine#2019-05-1202:35denikIs anyone aware of a library that is designed to transform data based on a spec or similar? cc @alexmiller#2019-05-1202:40Alex Miller (Clojure team)https://github.com/wilkerlucio/spec-coerce#2019-05-1203:17seancorfieldResist the temptation tho' @denik -- spec is not intended to be used to drive transformations šŸ™‚#2019-05-1204:00ikitommithere is also https://cljdoc.org/d/metosin/spec-tools/0.9.2/doc/spec-coercion#2019-05-1215:46denik@alexmiller @seancorfield @ikitommi to be clear, I'm not looking to coerce data, rather I want to to give it a richer shape (e.g. places -> names / sequential -> associative). I have a concise place-based syntax like [:user/add [:post 5 :authors] {:id 10 :user/name "Eve"}] that I want to turn into
{:op :user/add
 :path [:post 5 :authors]
 : entity {:id 10 :user/name "Eve"}}
This would be easy in vanilla clojure but there are some gotchas, for example path is optional. With other edge cases writing that in vanilla clojure would get ugly fast. Spec's s/? + conform (or similar) seem like a perfect fit. So if it's not conform, I'm wondering if there is a straightforward approach to parse and unparse while leveraging specs?
#2019-05-1215:49stathissideris@denik s/conform would work for this, but Iā€™ve seen people advising against using it#2019-05-1217:38Alex Miller (Clojure team)cond-> is great for optional stuff#2019-05-1217:38Alex Miller (Clojure team)Iā€™d just use core stuff#2019-05-1218:53denik@alexmiller not for parsing#2019-05-1219:31Alex Miller (Clojure team)there are libs for data structure parsing like https://github.com/cgrand/seqexp that do the "parsing" of data#2019-05-1315:33denikthanks! unfortunately clj only#2019-05-1220:18ikitommiPuzzled. I would have said that s/conform would have been a perfect tool (without any add-onns) for parsing just that example. What are the downsides of using spec for parsing?#2019-05-1221:59Alex Miller (Clojure team)conform is fine for parsing#2019-05-1222:00Alex Miller (Clojure team)Request above is for additional transformation and enhancement of the result#2019-05-1315:31denikconforming/parsing + generation of a spec for the parsed result (comformed shape) so that it can be checked in other function invocations as well as fspecs. I would find this very useful: 1. sparse data comes in from a request, e.g. [:user/add [:post 5 :authors] {:id 10 :user/name "Eve"}] 2. conform data to use names instead of indexes/places=>
{:op :user/add
 :path [:post 5 :authors]
 :entity {:id 10 :user/name "Eve"}}
3. use a derived spec of the conformed value from (2) to check validity of the value as it is updated throughout the program
#2019-05-1312:02benI have a bunch of specs that are quite repetitive to write out manually. Is there an easy way to define spec (e.g. w/in the current ns) with a function?#2019-05-1312:10benEssentially I have a map that looks something like:
{:event    :some-kw
 :metadata {:a 1 :b 2}}
where the spec of :metadata depends on the value of :event, which I want to check with spec
#2019-05-1312:13benSo I think I could do something like:
(s/def :event1/event ...)
(s/def :event2/event ...)
;; and so on

(s/def :event1/metadata ...)
(s/def :event2/metadata ...)
;; and so on

(s/def :event1/message (s/keys :req-un [:event1/event :event1/metadata]))
(s/def :event2/message (s/keys :req-un [:event2/event :event2/metadata]))
;; etc

(s/def ::message (s/or :event1/message :event2/message ...))
But this seems extremely inelegant. Feels like Iā€™m missing something obvious but Iā€™m not sure what
#2019-05-1312:26codonnellSounds like a good use case for a multispec.#2019-05-1312:36Alex Miller (Clojure team)Yes, also repetitive code can be made less repetitive with a macro#2019-05-1312:59ben> One common occurrence in Clojure is to use maps as tagged entities and a special field that indicates the ā€œtypeā€ of the map where type indicates a potentially open set of types, often with shared attributes across the types. yes it does šŸ™‚ thank you, @codonnell#2019-05-1322:59yogthosdoes anybody know if there's a workaround for this issue https://dev.clojure.org/jira/browse/CLJ-2482#2019-05-1323:06yogthosI guess dropping down to clj 1.9.0 works#2019-05-1323:08Alex Miller (Clojure team)The linked issue in the comments has some patches people have been using.#2019-05-1323:08Alex Miller (Clojure team)Patches to Clojure that is#2019-05-1323:09Alex Miller (Clojure team)Iā€™m not sure which of the many approaches on there is really the best#2019-05-1323:09Alex Miller (Clojure team)But we will definitely take a look at it in 1.11#2019-05-1323:17yogthosthanks, and 1.9 seems to work so I'll just stick with that for the time being#2019-05-1412:57rickmoynihanAny ideas on how to spec this? Essentially I have a heterogenous map; if a key in that map is a vector of length N then Iā€™d like to spec that the value of that key in the map is a sequence of tuples also of length N. If a key in the map is not a vector, than its key can be any?#2019-05-1413:02Alex Miller (Clojure team)spec it as a collection of kv tuples#2019-05-1420:03Alex WhittWhen defining your fspec's :fn, what's the intended way to interact with args that are s/or specs? For example: {:fn #(-> % :args :arg-name) for such an arg yields a vector that starts with the keyword for the s/or variant. Here's how I could approach it:
(s/def ::example (s/or :map map?
                       :num int?))

(defn myfn [int-arg example-arg])

(s/fdef myfn
  :args (s/cat :int-arg int?
               :example-arg ::example)
  :fn #(-> % :args :example-arg
           (as-> [path value]
               (or (not= path :map)
                   (contains? value (-> % :args :int-arg))))))

(myfn 1 {2 :a 3 :b}) ;; => fails
Here, :fn ensures that the first arg is a key in the second arg if the second arg is a map. Is this how I should be writing these :fns, or is there a better way?
#2019-05-1421:21Alex Miller (Clojure team)if it's just an inter-arg dependency, you can s/& that predicate onto the args spec#2019-05-1421:21Alex Miller (Clojure team)rather than using :fn, which has access to both args and ret#2019-05-1421:22Alex Miller (Clojure team)alternately, you could encode the args with an s/alt for your two alternatives#2019-05-1421:26Alex Miller (Clojure team)
(s/alt :not-map (s/cat :int-arg int? :example-arg #(not (map? %)))
       :map (s/& (s/cat :int-arg int? :example-arg map?) #(contains? %2 %1)))
#2019-05-1421:26Alex Miller (Clojure team)something like that for the args if I understood all that right#2019-05-1421:27Alex Miller (Clojure team)and as an aside, when something is hard to spec like this with options, it's often a good sign that your function is doing two things and having 2 functions might be better#2019-05-1514:02Alex WhittAhh that looks better! And thank you for the advice, I'll think about whether I can split this up or not.#2019-05-1521:13lellisHi all! There is some lib that i can create an ER model based in my spec definition?#2019-05-1607:47jaihindhreddyTake a look at Hodur. Tries to DRY the essence of the domain model. You define one central model from which you generate the models for each thing you use (search engine, database, message queues etc.) https://github.com/luchiniatwork/hodur-engine#2019-05-1607:47jaihindhreddyI'm sure you can extend hodur to make it generate an ER model too. Hodur's possibly too heavy conceptually for your specific use case.#2019-05-1613:38lellisTks!#2019-05-1620:47robertfwI'm trying to create an fdef :args spec for a multi-arity function in the general form of (defn my-func ([x] (my-func x nil)) ([x y] (do-thing x y))) I have a handful more args than just x & y, and don't want to repeat myself, so my first thought was to do something like (def base-args [:x ::my-x-spec]) (s/fdef my-func :args (s/alt :without-y (apply s/cat base-args) :with-y (apply s/cat (conj base-args :y ::my-y-spec))) - but alas, s/cat is a macro so I can't do that. I've had another peruse of the spec docs but nothing jumped out at me. any suggestions?#2019-05-1620:49robertfwCan I make an (s/cat base-args) and then append onto it somehow?#2019-05-1620:53robertfwI understand that I can just provide a full list of [:x ::my-x :y ::my-y], but I'd like to be able to generate examples of each arity#2019-05-1620:58Alex Miller (Clojure team)Just build a single s/cat with nested optional additional args with s/?#2019-05-1620:59Alex Miller (Clojure team)Or you could build each arity as an s/cat that combined the prior with an additional arg#2019-05-1621:27robertfwThanks, I'll look at those options#2019-05-1710:07vemvIs there a performance difference between running essentially the same spec checking via instrumentation, vs. via :pre?#2019-05-1719:28andy.fingerhutFor a general spec, I have no idea. For a very specific very simple spec of checking whether some clojure.set functions are given arguments return true for the predicate set? or not, there is a pretty big performance difference I measured, with measurements given starting in about the second screenful of the README for this project: https://github.com/jafingerhut/funjible#2019-05-1818:17vemvnice one! But my question was focused on instrumentation vs :pre. Guess I can try it myself though#2019-05-1711:10kszaboAm I right to assume that until spec2 gets finished the way to go to emulate s/schema behavior is to only define (s/keys :opt [:schema/values]) kind of specs and use fdefs with use-case specific :req specs?#2019-05-1711:11kszabothat way those can be later transformed in s/selectā€™s#2019-05-1712:23Alex Miller (Clojure team)Yeah#2019-05-1805:03dottedmagI'm trying to write a dissector of a complex protocol for Wireshark. I have started by writing it manually in C, but it seems to be a very tedious task. I'm going to consume this protocol from Clojure, so I was thinking about specifying it using spec, and then generating C code from it. Has anyone seen anything similar?#2019-05-1805:04dottedmag(C can be replaced with Python or Lua ā€” does not matter much in this context)#2019-05-1805:04seancorfieldI bet if you ask that during Pacific daylight hours, my colleague @hiredman would have a fair bit to say. He's a big fan of generating stuff from spec and similar data-based systems.#2019-05-1805:49dottedmagIs there a way to have "switch-by-value" without describing all variants in one gigantic (s/alt)? I'm dealing with a protocol that has a command ID byte that describes format of what is following, and then another subcommand ID in some commands etc. I don't want to have (s/alt :0x00 ::0x00-command :0x01 ::0x01-command ...) ā€” maybe I can register them separately?#2019-05-1805:49dottedmagSomething like a multi-spec for sequences.#2019-05-1818:15y.khmelevskiiHi! Does anybody know when spec2 for clojurescript will be available? Also, just curious, was spec2 release date announced?#2019-05-1818:45Alex Miller (Clojure team)Still months to go probably#2019-05-1818:45Alex Miller (Clojure team)No date#2019-05-1818:54y.khmelevskiigot it, thank you#2019-05-1819:26Alex Miller (Clojure team)Although at some point maybe weā€™ll decide to lock it and just make additive changes after that so who knows #2019-05-1820:02y.khmelevskiiIt would be great to deploy cljs.spec-alpha2 in the same state as clojure.spec-alpha2 now. During development not stable clojure.spec-alpha2 is ok but I need cljs.spec-alpha2 as well for sharing new specs between frontend and backend#2019-05-1820:19Alex Miller (Clojure team)afaik no one has actually started working on a port of the changes in spec-alpha2 to cljs yet. The implementation changes are fairly large (and still under way) so I'm not sure it makes sense to start that work until we feel the internals have started to stabilize, and I wouldn't say that yet.#2019-05-1820:21y.khmelevskiiunderstand, thanks for info! I will look forward to it#2019-05-1920:31Jakub HolĆ½ (HolyJak)how do you do generative testing with spec? I now use
(defspec xyz
  100
  (prop/for-all [v (s/gen ::my-spec), res (myfn  v)]
    (do (s/assert ::my-result res)
          (my-check-some-property-on v res))))
but shouldn't s/exercise-fn be usable for this? Though, contrary to s/gen it doesn't allow me to specify overrides and thus test more special / infrequent cases?
#2019-05-2005:05Jakub HolĆ½ (HolyJak)I see I should have used https://clojure.github.io/spec.alpha/clojure.spec.test.alpha-api.html#clojure.spec.test.alpha/check instead. Still, how to best integrate with clojure.test? Something like
(deftest xyz 
  (if-let [f (:failure (st/check ::my-spec)]
    (throw f) 
    (is true)))
?
#2019-05-2010:37conanI use this namespace:
clojure
(ns ic.test-util
  (:require [clojure.spec.test.alpha :as stest]
            [clojure.test :refer :all]
            [expound.alpha :as expound]))

(defn check
  "Passes sym to stest/check with a :max-size option of 3 (i.e. generated sequences will have no more than 3 elements,
   returning true if the test passes or the explained error if not"
  [sym]
  (let [check-result (stest/check sym {:clojure.spec.test.check/opts {:max-size 3}})
        result (-> check-result
                   first ;; stest/check accepts a variable number of syms, this does not
                   :clojure.spec.test.check/ret
                   :result)]
    (when-not (true? result)
      (expound/explain-results check-result))
    result))
#2019-05-2010:37conanthen my tests look like this:
(ns ic.date-test
  (:require
   [clojure.test :refer :all]
   [ic.date :as date]
   [ic.test-util :as tu]))

(deftest inst->local-date-time-test
  (is (true? (tu/check `date/inst->local-date-time))))
#2019-05-2010:38conanthis will either pass, or give an expound-formatted error showing where the :args, :ret and :fn specs went wrong for the generative tests run for my function (in this case, inst->local-date-time)#2019-05-2016:33Jakub HolĆ½ (HolyJak)thanks a lot!#2019-05-2011:53borkdude@holyjak Iā€™m using this lib for it: https://github.com/borkdude/respeced#2019-05-2011:53borkdudeso in the context of this question: https://github.com/borkdude/respeced#successful#2019-05-2011:54borkdudethat function also checks if the sequence of results is not empty#2019-05-2016:23boyanbDoes anybody have experience they would be willing to share in regard of human readable messages at API boundaries via spec? We've looked at expound(which doesn't fit) and phrase(which could do the job, but I am personally not convinced by the predicate focused approach). It feels that a simpler solution focused around explain-data and a message registry(similar to the one found in expound) would produce a better result. Has anybody implemented/is currently using spec for this purpose?#2019-05-2016:27jeroenvandijk@boyanb Can you share how you feel expound is not a fit?#2019-05-2016:30boyanbIt's not really designed with the idea of message formatting for "users". Expound could very easily be enhanced or parts/ideas of it lifted into a library that could fit it. AFAIK, while looking at the code, there were several places where we needed paramtrization/additional control that is currently not available by the public facade of the library,#2019-05-2016:30boyanb(had to do with custom printers and expected outputs)#2019-05-2016:32boyanbI could probably dig in a little further and find the concrete examples. Are you using it with user facing messages in any way?#2019-05-2016:32jeroenvandijkI've used expound like this
(defn validate-data [spec data]
  (if (s/valid? spec data)
    :ok
    (if-let [explain-data (s/explain-data spec data)]
      (let [expound-state
            (try
              (expound/expound spec data)
              :ok
              (catch Exception e
                (println "Expound had difficulties using " explain-data)
                :error))]
        (throw (ex-info (if (= expound-state :ok)
                          "Spec error (see stdout)"
                          "Spec error (see explain data)") {:explain explain-data})))
      (throw (ex-info "Specs are in a weird state, as we can't explain why data is invalid" {})))))

#2019-05-2016:32jeroenvandijkBut I agree it is not perfect#2019-05-2016:34jeroenvandijkI've actually used this in a clojurescript environment (using with-out-str). Worked pretty will when you do a validation on every key change#2019-05-2016:34boyanbYes#2019-05-2016:34kenny@boyanb We had a similar problem. We wanted to spec our API using Clojure Spec but we didn't want to expose Spec's error messages to our users (most people are not used to seeing error messages in that format). This immediately meant Expound was out of the question (too similar to Spec). I looked at Phrase but it seemed like you'd need to duplicate some code to get error messages. I didn't really like that. The overall approach Phrase took made sense -- take the output of explain-data, match it, and convert it to error messages. The matching part seemed like a perfect fit for core.match. So I took that direction and wrote https://github.com/Provisdom/user-explain. I didn't have time to write docs for it šŸ˜ž IMO the library is much more general than Phrase due to all the features of core.match. It's also pretty simple -- only 75 LOC šŸ™‚#2019-05-2021:06ericstewartthank you for sharing this! Going to take a look as I have been on the same path as you and you are further ahead it seems.#2019-05-2021:07kennyOf course! LMK if it works out for you.#2019-05-2016:34boyanbour implementation was with-out-str exactly.#2019-05-2016:35boyanb(when with expound). In the end, we still didn't have the formatting we wanted. Main point is, underlying users are really not familiar with anything clojure and shouldn't care about implemenmtation and in the end we needed fine grained control to explain-data to be able to format path within spec + spec message as we needed.#2019-05-2016:36boyanb@kenny - what you describe matches exactly our needs. I'll take a look.#2019-05-2016:39kennyThe tests may be helpful for documentation. I'd recommend just reading the 75 line implementation though. There's a lot of areas I want to improve. As you start diving into the data produced by explain-data, you realize that there's a ton of "core" predicates that need to be handled. A common case is #(contains? % kw) which is used with s/keys to validate keywords exist on the map. I'd like to provide a way get automatic nice error messages for all the "core" predicates.#2019-05-2016:40boyanbOooh, that's great. @kenny. Looking at the tests it's almost exactly what we are looking for. I'll play with it and let you know if we decide to ship.#2019-05-2016:40boyanbYes, the tests are where I started ;o)#2019-05-2016:49kennyThere's a couple weird things about the implementation that I really need to write down before I forget: 1. Since defexplainer does not have a name associated with it, the only way to uniquely identify "explainers" is via the matching map. If you change the matching map (i.e. add or remove keys), the old matcher will still be def'ed. You'll need to run clear! to reset everything. This is really annoying and I don't have a great solution atm. The most obvious solution would be to name every defexplainer. 2. Order of defexplainers does matter. If you def the most general explainer first, it will always get matched first. Ensure more specific matches are def'ed before general ones. Technically this could be fixed by sorting based on some sort of heuristic but I didn't have the time to work out what that should be.#2019-05-2016:50boyanbThank you#2019-05-2016:51kennySure. LMK if it ends up working out or if you guys take another approach.#2019-05-2016:51boyanbWill do#2019-05-2114:25Jakub HolĆ½ (HolyJak)does (spec.test/instrument) (w/o args) instrument 1. all functions (ie vars) in all namespaces, mine, libs, and clojure? or 2. just the current one? thanks!#2019-05-2114:25Jakub HolĆ½ (HolyJak)does (spec.test/instrument) (w/o args) instrument 1. all functions (ie vars) in all namespaces, mine, libs, and clojure? or 2. just the current one? thanks!#2019-05-2114:33kszabohttps://clojure.github.io/spec.alpha/clojure.spec.test.alpha-api.html#clojure.spec.test.alpha/instrument#2019-05-2114:34kszaboall vars#2019-05-2116:13Jakub HolĆ½ (HolyJak)yeah I read that. So option 1 right?#2019-05-2116:17kszabohttps://github.com/clojure/spec.alpha/blob/0269b2cfefe4df7b68710decc94c623db4a6f630/src/main/clojure/clojure/spec/test/alpha.clj#L248 everything returned by instrumentable-syms#2019-05-2116:18Jakub HolĆ½ (HolyJak)OK I guess I can run it to get the answer ,thx#2019-05-2116:28seancorfieldIt will instrument all the s/fdefs that have been loaded.#2019-05-2116:29seancorfieldAs an example, clojure.java.jdbc has all its s/fdefs in a separate namespace so the library's functions would only be instrumented by (s/instrument) if you have required the clojure.java.jdbc.specs namespace.#2019-05-2211:38Jakub HolĆ½ (HolyJak)thanks a lot, Sean!#2019-05-2118:19ejelomeThis might be answered, but I watched Matthias Felleisen's talk about types: "Types are like the Weather, Type Systems are like Weathermen". And I remembered clojure.spec and noticed that it somehow reflects his ideas (putting contracts instead of making a type system for an untyped language, e.g. typed.clojure [which afaik is now discouraged/discontinued] and that type inference is near to impossible to implement in a untyped language like Clojure). So the question is ... is clojure.spec the answer to Felleisen's talk?#2019-05-2118:28Alex Miller (Clojure team)no, although spec was influenced by Racket's contracts, along with other stuff#2019-05-2118:29Alex Miller (Clojure team)spec had been in development for about 4 months at the time of that talk#2019-05-2118:33seancorfield"typed.clojure [which afaik is now discouraged/discontinued]" -- I thought Ambrose was still actively working on this...?#2019-05-2118:35Alex Miller (Clojure team)he just defended his dissertation on it#2019-05-2118:35Alex Miller (Clojure team)so it has been, but not sure where it's going from here. would be a good question for him#2019-05-2118:38Alex Miller (Clojure team)iirc, while I was away at Clojure/west when that talk was given, Rich wrote all the regex stuff in spec#2019-05-2118:54ejelomethanks @alexmiller, although it didn't came directly from racket's contracts, I'm still glad that they're going almost on the same direction (no type inference, or implementing a type system [just to say we also can do type checking]).#2019-05-2118:59Alex Miller (Clojure team)well, it's not type checking, and the difference is important#2019-05-2118:59Alex Miller (Clojure team)type checking is about proving things early#2019-05-2119:00Alex Miller (Clojure team)spec is about verifying predicative constraints dynamically#2019-05-2211:51Jakub HolĆ½ (HolyJak)@alexmiller To improve docs of the lazy-loaded gen/* functions, since including the whole docstring is not feasible according to https://clojure.atlassian.net/browse/CLJ-2018, what about at least changing the docstring to contain the URL of the function's online documentation, such as https://clojure.github.io/test.check/clojure.test.check.generators.html#var-elements for elements?#2019-05-2212:13Alex Miller (Clojure team)Sure#2019-05-2212:14Jakub HolĆ½ (HolyJak)Should I make a jira issue and send a patch?#2019-05-2212:20Alex Miller (Clojure team)Go for it#2019-05-2212:23Alex Miller (Clojure team)I assume you are aware we have migrated jira to new system...#2019-05-2217:10Jakub HolĆ½ (HolyJak)@alexmiller I wanted to try my patch by including local clone of spec.alpha in my project but starting clj in my project then fails with
...
Caused by: java.lang.Exception: #object[clojure.spec.alpha$and_spec_impl$reify__1047 0xf9b5552 "
Any idea what I do wrong? I have cloned to /Users/me/tmp/spec.alpha, git hash 5228bb7. In my project's deps.edn:
{:deps {org.clojure/clojure {:mvn/version "1.10.1-beta2"}
        org.clojure/spec.alpha {:local/root "/Users/me/tmp/spec.alpha"}
...}
(This is before I did any changes to the code. java -version 1.8.0_192, OSX) But mvn package in the spec.alpha project runs just fine.
#2019-05-2217:13Jakub HolĆ½ (HolyJak)Same problem when running clj in the spec.alpha project:
šŸŸ  clj
Clojure 1.10.0
user=> (load-file "src/main/clojure/clojure/spec/alpha.clj")
Syntax error macroexpanding clojure.core/defn at (alpha.clj:78:1).
#object[clojure.spec.alpha$and_spec_impl$reify__2183 0x1698fc68 "
#2019-05-2217:22Jakub HolĆ½ (HolyJak)(Added https://clojure.atlassian.net/browse/CLJ-2512 for the docstring change)#2019-05-2212:14Jakub HolĆ½ (HolyJak)Question: I often do something like the following, to restrict the domain of the generated values to ensure interesting conflicts:
(s/fdef filter-adult-users
        :args (s/cat :youngsters (s/coll-of ::uid :kind set?), :users ::users) #_... )
(deftest filter-adult-users-spec
  (let [[user-ids] (sg/sample (sg/set (s/gen ::uid)
                                      {:min-elements 1
                                       :max-elements 20})
                              1)]
    (is (true? (check `filter-adult-users
                      {:gen {::uid (constantly (sg/elements user-ids))}})))))
I.e. I have a let where I use sample to generate a random, small set of data then used in generator overrides in the test itself. Do you do that too? Is there a better way? update Sometimes I need to use the customized random data from multiple generators, which prevents I. believe the usage of simple generator derivation such as gen/let and gen/rmap
#2019-05-2214:08mishahave a look at test.check/fmap, test.check/bind and test.check/let: https://github.com/clojure/test.check/blob/master/doc/cheatsheet.md#combinators https://github.com/clojure/test.check/blob/master/doc/intro.md#bind @holyjak https://github.com/clojure/test.check/blob/master/src/main/clojure/clojure/test/check/generators.cljc#L1570#2019-05-2214:41Jakub HolĆ½ (HolyJak)thank you! but what if eg 2 different generators need the value? any tips?#2019-05-2214:56tangrammeryep, thatā€™s a very interesting need ā€¦#2019-05-2215:09Jakub HolĆ½ (HolyJak)example: having a function taking a map with known values and a list of "things", the fn throws if any " thing" has an unknown value. If I want to test its other functionality I must ensure that things only use the values in the map get I prefer not to hardcode the map.#2019-05-2215:46mishayou can test.check/let the generator, which generates [map things] tuples, and then apply function you want to test to the generated tuples#2019-05-2216:27Jakub HolĆ½ (HolyJak)good idea! But I guess I would need to invoke it manually instead of using spec.test/check#2019-05-2310:05mishadefining wrapper which applies your function to the tuple, and specing/checking wrapper instead might be an option for you#2019-05-2310:08tangrammercould anyone write/share a gist detailed example šŸ™ ? šŸ™‚#2019-05-2311:11Jakub HolĆ½ (HolyJak)BTW I tried to replace my (let [uids ..] (check .. {:gen {::uid (const. (sg/elements uids)))}})) with
(check .. 
       {:gen {::uid (constantly
                      (gen/bind
                        (sg/set (s/gen ::kd/sid) :num-elements 1)
                        sg/elements))}})
but it does not seem to really work. If I replace it with (constantly (sg/return (s/gen ::uid))) then I get many (desired) "collisions" in the tests but with ā˜ļø I get none. So my hypothesis is that the set of values is re-created every time that a new value fpr ::uid is generated, instead of creating it once and reusing it every time ::uid is requested.
#2019-05-2406:51Jakub HolĆ½ (HolyJak)@U051KJGTX a gist of what exactly? As mentioned above, bind did not work for me, and showing a let wrapping check with some :gen overrides is perhaps not all that interesting?#2019-05-2412:52Jakub HolĆ½ (HolyJak)@U051KJGTX here is an example of what misha proposed, a wrapper fn taking a tuple of related inputs:
(defn apply-profile-compute-totals-wrapper
  "Smart wrapper around apply-profile-compute-totals that 'unpacks' the subscr-summaries+userid->profile tuple
   we generate so that both have the same subscribers before invoking the wrapped fn
   "
  [subscr-summaries+userid->profile]
  (let [[subscr-summaries userid->profile] subscr-summaries+userid->profile]
    (apply-profile-compute-totals
      {:db/userid->profile userid->profile}
      subscr-summaries)))

(s/def ::subscr-summaries+userid->profile
  (s/with-gen
    (s/cat :subscr-summaries ::kd/subscr-summaries, :userid->profile :db/userid->profile)
    ;; GENERATOR: Ensure that the generated userid->profile have a profile for every subscriber in
    ;;            subscr-summaries (b/c those without profile would have been filtered out before)
    (constantly
      (gen/let
        [subscr-summaries (s/gen ::kd/subscr-summaries)
         profiles         (sg/vector (s/gen :db/profile) (count subscr-summaries))]
        [subscr-summaries
         (zipmap
           (keys subscr-summaries)
           profiles)]))))

(s/fdef apply-profile-compute-totals-wrapper
        :args (s/cat :1 ::subscr-summaries+userid->profile)
        ;:args (s/cat
        ;        :org  (s/keys :req [:db/userid->profile])
        ;        :subscr-summaries ::kd/subscr-summaries)
        :ret ::kd/subscrs+profile-usages)

(st/check `apply-profile-compute-totals-wrapper)
#2019-05-2413:12tangrammerthanks a lot for sharing ā€¦ Iā€™ll give a try next week!#2019-05-2216:03mishahere is more explanation about fmap and bind: https://youtu.be/F4VZPxLZUdA?t=655#2019-05-2306:41yuhanHow do I instrument/unstrument functions on a per-namespace basis?#2019-05-2306:50Jakub HolĆ½ (HolyJak)Not sure but perhaps https://clojure.github.io/spec.alpha/clojure.spec.test.alpha-api.html#clojure.spec.test.alpha/enumerate-namespace could help?#2019-05-2306:54Jakub HolĆ½ (HolyJak)(st/instrument (st/enumerate-namespace 'my.core)) Enum. enumerates all vars but instrument seems to be happy with that and presumabely skips those that are not spec-instrumentable.#2019-05-2307:03yuhanthat's just what I was looking for, thanks!#2019-05-2307:07yuhan(stest/instrument (map symbol (vals (ns-interns *ns*))))#2019-05-2312:36Nolanam i missing something regarding s/union and s/select? iā€™m trying to do something like this:
(require '[clojure.spec-alpha2 :as s])
(s/def ::n number?)
(s/def ::o odd?)
(s/def ::schema1 (s/schema [::n]))
(s/def ::union1 (s/union ::schema1 [::o]))
(s/select ::union1 [*])                  ;; => IllegalArgumentException
(s/select (s/union ::schema1 [::o]) [*]) ;; => IllegalArgumentException
#2019-05-2312:55Alex Miller (Clojure team)probably broken atm. top-level union is probably going to go away anyways#2019-05-2312:56Alex Miller (Clojure team)I'll take a look when I next get a chance#2019-05-2313:05Nolanok, perfect. appreciate the info!#2019-05-2319:36ikitommicould the s/def fail-fast on totally bogus spec-forms? what would be a spec for the spec-form arg in s/def?#2019-05-2319:36ikitommi
(s/def ::a 1)
;; :user/a

(s/valid? ::a 1)
;; Syntax error (ClassCastException)
;; java.lang.Long cannot be cast to clojure.lang.IFn
#2019-05-2319:39Alex Miller (Clojure team)it could check some things better, for sure#2019-05-2319:55Alex Miller (Clojure team)what's valid is changing between spec 1 and 2#2019-05-2319:58Alex Miller (Clojure team)in spec 2 the valid symbolic forms are keywords, sets, symbols, and lists/sequences which are spec forms where a known spec op is in op position (ops are an extension point via multimethod)#2019-05-2319:59Alex Miller (Clojure team)we may add another datafied/map form, still tbd#2019-05-2320:00Alex Miller (Clojure team)notably, function objects are not allowed (different than spec 1)#2019-05-2320:00Diegohey all, does anyone know why canā€™t I do something like this with spec?
(s/def ::profile (s/keys :req-un {::id int?
                                                      ::name string?
                                                       ::photos (s/* (s/keys :req-un {::type int?}))}))
#2019-05-2320:02Alex Miller (Clojure team)you mean, inline a spec in s/keys?#2019-05-2320:02Alex Miller (Clojure team)by design, spec is trying to enable a shared global registry of attribute specs#2019-05-2320:03Alex Miller (Clojure team)the attribute spec is considered to be more important than containers of those attributes#2019-05-2320:03Alex Miller (Clojure team)as such, we require you to strongly associate a spec with an attribute name#2019-05-2320:04Alex Miller (Clojure team)spec 2 will have more support for inlining attribute spec defs for unqualified keys (common with json interop)#2019-05-2320:09DiegoI see. Being new to spec I find that not being able to inline specs makes it harder for me to reason about them but I suppose that might change once I get more familiar with it, and one I learn about the best practices on creating specs (any suggested links?).#2019-05-2320:13Diegobtw, thanks for the response @alexmiller#2019-05-2320:18Alex Miller (Clojure team)have you read the guide? https://clojure.org/guides/spec ?#2019-05-2320:18Alex Miller (Clojure team)or the rationale? https://clojure.org/about/spec#2019-05-2321:06Nolancurious if there is a significant expected runtime difference between computing s/selects dynamically and defing them prior. e.g. if the following s/valid calls were going to be called in a loop:
(require '[clojure.spec-alpha2 :as s])

(s/def ::schema1 (s/schema [...]))
(s/def ::select1 (s/select ::schema1 [...]))
(def select2 (s/select ::schema1 [...]))

(s/valid? ::select1 {...})
(s/valid? select2 {...})
(s/valid? (s/select ::schema1 [...]) {...})
#2019-05-2321:16Alex Miller (Clojure team)you should expect that we have spent no time yet on perf aspects of select :)#2019-05-2321:19Alex Miller (Clojure team)in general though, the first two are reusing the same spec object, which means work can be done once at spec object creation time (optimization work is basically shifting as much work as possible into construction time here) whereas the last one would pay that cost every time#2019-05-2321:19Nolanexactlyā€”was about to say that i suppose iā€™d always def it, if it werenā€™t truly dynamic. was just tangentially curious about the perceived cost of the dynamism there#2019-05-2321:20Alex Miller (Clojure team)so in general, I would tend toward either using an object saved in var or from the registry (those are probably approx the same)#2019-05-2321:20Alex Miller (Clojure team)but also note that the tradeoff, as usual, is in not picking up changes to specs in the ... there#2019-05-2321:21Alex Miller (Clojure team)so you might make the opposite choice at the repl, if you're in development#2019-05-2321:22Alex Miller (Clojure team)generally, I am just working in a file, that has all the specs I'm working on in it, and I just reload the file, which registers and recompiles all of the specs, and I don't care#2019-05-2321:22Alex Miller (Clojure team)it is useful to have that mental model though#2019-05-2321:25Nolanright, right. got it. thats a premium tip. spec2 is awesome, have been having a blast getting some experience. it alleviates essentially all of my old s/keys woes šŸ„³#2019-05-2321:34Alex Miller (Clojure team)great!#2019-05-2406:54Jakub HolĆ½ (HolyJak)Does anybody have any idea why might I be getting the error > Syntax error macroexpanding clojure.core/defn at (alpha.clj:78:1). > #object[clojure.spec.alpha$and_spec_impl$reify__2183 0x1698fc68 "clojure.spec.al/cdn-cgi/l/email-protection"] is not a fn, expected predicate fn when trying to start a REPL in the spec.alpha project and loading the code? I did this:
$ git clone  # sha 5228bb75fa10
$ cd spec.alpha
$ clj
Clojure 1.10.0
user=> user=> (load-file "src/main/clojure/clojure/spec/alpha.clj")
Syntax error macroexpanding clojure.core/defn at (alpha.clj:78:1).
#object[clojure.spec.alpha$and_spec_impl$reify__2183 0x1698fc68 "
#2019-05-2407:06Alex Miller (Clojure team)spec is aot compiled, might be a conflict since you are forcing a load of the clj, which recompiles the protocol (yet all the old instances are from the prior protocol). should probably use load instead?#2019-05-2407:46Jakub HolĆ½ (HolyJak)Ah, OK, thank you, that helped! I am used to load-file and do not really know how it works, I guess I should learn šŸ™‚#2019-05-2417:47seancorfield@U0522TWDA I'm curious how you got into the habit of using load-file in the first place? It seems like require would be the "obvious" thing to learn first...#2019-05-2417:49Alex Miller (Clojure team)yeah, that seems weird#2019-05-2417:51Jakub HolĆ½ (HolyJak)Perhaps a bad habit from Node.js where require is idempotent and does not actually reload the code from the disk. Also, I see that Cursive's "load file in repl" used to reload the code actually calls load-file, at least when running REPL via main (instead of via lein)#2019-05-2417:52Jakub HolĆ½ (HolyJak)Ah, no, Clojure require does not reload the code either > Loads libs, skipping any that are already loaded. So what are you saying? That the 1st time I should use require but after I change the code, to get the changes in, then I should use what?#2019-05-2417:53Jakub HolĆ½ (HolyJak)(I guess I will get the answer after I finish Eric's REPL course)#2019-05-2417:53seancorfield(require ... :reload) or (require ... :reload-all)#2019-05-2417:54Jakub HolĆ½ (HolyJak)Also, require requires that the code is on the classpath while load-file does not care about that. But I guess that is not a problem if I only use lein repl in lein project and clj in deps.edn projects as those have the correct paths autom.#2019-05-2417:54seancorfieldI think editors are likely to use load-file since they have a filesystem path, rather than a namespace -- but I view load-file as a tooling/system-level hook. Always interesting to hear how other folks developed their Clojure habits...#2019-05-2407:06Alex Miller (Clojure team)or require?#2019-05-2614:41y.khmelevskiiHi everyone! Can you please explain me why this spec doesnā€™t work correctly:
(s/def ::name string?)
(s/def ::src string?)
(s/def ::width pos-int?)
(s/def ::height pos-int?)

(s/def ::size
  (s/schema {:src    ::src
             :width  ::width
             :height ::height}))

(s/def ::file
  (s/select
   (s/schema {:name ::name
              :size  ::size})
   [:name :size {:size [*]}]))

(s/valid? ::file {:name "test"
                  :size {}})
;; => true
;; but should be false
#2019-05-2614:47y.khmelevskiibut when I use fully-qualified keywords, s/valid? works correctly#2019-05-2620:30seancorfield@y.khmelevskii I tried the following and it also produces true
user=> (s/def ::file (s/select (s/schema {:name ::name :size ::size}) [:name :size {:size [:src :width :height]}]))
:user/file
user=> (s/valid? ::file {:name "x" :size {}})
true
I'm not sure what restrictions are in place for unqualified key usage. It is all alpha, after all.
#2019-05-2623:54benzapHas clojure.spec 2.0 landed already, or is it still in the development stage?#2019-05-2623:59seancorfieldVery much alpha at the moment @benzap#2019-05-2623:59seancorfieldThere hasn't even been an "alpha" release yet -- it's just on GitHub.#2019-05-2700:05benzapOkay, thanks for the update!#2019-05-2700:09seancorfieldAt work we have a branch of our code running on spec-alpha2 just to keep an eye on changes. We haven't started using any of the schema/`select` stuff yet. Alex has repeatedly told me it's not ready to run in production yet šŸ™‚ Even for us, who run alpha stuff in production all the time šŸ™‚#2019-05-2700:12benzaphaha, well it sounds like it's on it's way then, that's good to hear šŸ™‚#2019-05-2700:13seancorfieldYeah, still some weird quirks to be ironed out but it wasn't too painful to get our code running on the new version straight from GitHub (since we're an all-`deps.edn` setup at work)#2019-05-2808:25zcljany advice in how to create a spec for a "string-date" where the string should be a valid RFC3339 format?#2019-05-2810:24Mikko HarjuHi! Is there a mechanism to get exhaustive errors from a spec or how would one approach validating maps with required keys and keys that depend on the content of another one? For instance, given a spec like
(s/and (s/keys :req-un [::start-date ::end-date ::foo]) end-date-after-start-date?)
Iā€™d like to be able to have the end-date-after-start-date? error to be shown also when the key :foo is also missing from the map.
#2019-05-2810:25Mikko HarjuBy default s/and does short circuiting on the first failing spec#2019-05-2810:43Mikko HarjuOne option would be to split the spec and use s/merge, are there any other valid options?#2019-05-2811:49Jakub HolĆ½ (HolyJak)Any idea why, given this spec
(s/def ::account (s/with-gen delay? (constantly (sg/return (delay nil)))))
does (sg/sample (s/gen ::account)) fail with > Unable to construct gen at: [] for: :myapp/account ? Doesn't the with-gen add the generator?
#2019-05-2901:49Chris ReyesIā€™m interested in using spec (for the first time) for a side project Iā€™m working on. Where should I put the spec definitions? (Or is that a controversial question?)#2019-05-2901:50Chris ReyesI found this https://stackoverflow.com/questions/37942495/where-to-put-specs-for-clojure-spec But Iā€™m not sure they came to a consensus in any of the answers/comments. (Or maybe Iā€™m not sure how to interpret it because Iā€™m still pretty new to spec)#2019-05-2901:53seancorfield@chrisreyes Most people tend to put data specs in their own namespace, possibly with a few utilities for processing that data, but put function specs next to (above) the functions to which they apply.#2019-05-2901:54seancorfieldIf you want specs to be "optional" for some functions, it makes sense to put them in a separate namespace, so that users can decide whether to load them or not.#2019-05-2901:55Chris ReyesOkay, thanks!#2019-05-2901:56seancorfieldThere is no "right way" or "wrong way" -- whatever is most convenient/makes the most sense.#2019-05-2901:57seancorfieldWhen I added function specs to next.jdbc recently, I put them all in a separate namespace, so users could choose whether to use them or not https://github.com/seancorfield/next-jdbc/blob/master/src/next/jdbc/specs.clj#2019-05-2901:58seancorfieldThat ns actually contains fdefs for two other namespaces within the next.jdbc project, just because that was the most natural/convenient way to set things up.#2019-05-2901:58seancorfieldBut having fdefs separate isn't as common as having s/defs for data separate.#2019-05-2901:59seancorfieldPart of the issue is that specs can serve a lot of different purposes. They can be used for testing in several ways. They can be used to support development (`st/instrument` for example). They can be used in production for data validation (and other things).#2019-05-2909:23hlolliWhat's & rest in spec. I want to spec [:a :B :C :D] and make sure that first is :a and the rest can by of any amount of any type? (s/cat :need-a ::need-a (&rest?))#2019-05-2909:31djtango(s/cat :need-a ::need-a :rest (s/* any?))#2019-05-2909:32djtangothe nested s/* (and other regex ops) is flattened out by default so that doesn't mean [:a [:b ...]]
#2019-05-2909:34hlolliahh I see šŸ™‚ thanks dj! Still after 2 years of spec, these basics are still troubling me#2019-05-2909:34djtangothat one definitely requires "grokking"#2019-05-2909:35hlolliyes I would have expected a nested sequence for that#2019-05-2909:36djtangoI guess with a lisp hat on you would expect it to nest, but if you were trying to imagine expressing regex via s-expressions it would look like that too so :shrug:#2019-05-2909:37djtangos/tuple behaves how you might expect iirc#2019-05-2909:47djtangoDo people do generative testing over side-effectful code and/or integration tests? I feel it is nice to be able to put an fdef spec over an API endpoint on a webserver (e.g. required params and possible responses) but am unconvinced setting up the check as it starts becoming a bit like a super verbose example test. I suppose instrument with :stub option could work but I'm trying to find a nice way of tying fdefs into integration tests...#2019-05-2909:57Jakub HolĆ½ (HolyJak)Speaking about integration / system tests, these talks https://lispcast.com/testing-stateful-and-concurrent-systems-using-test-check/, https://youtu.be/zjbcayvTcKQ and Datomic Simulat could be of interest. Regarding side-effects, I guess it depends on their kind. Some you certainly want to avoid when testing. (x with datmoic, you can use an "in-memory" local copy of the DB and never "commit" / transact into the original DB yet have everything there for verification.#2019-05-2910:01djtangothanks for sharing#2019-05-2910:01djtangowill take a look!#2019-05-2910:03djtangoalas we're not using Datomic at work though#2019-05-3010:44hlolliSo I have this fdef
(s/fdef define-fx
  [env]
  :args (s/cat :fx-name ::fx-name
               (s/keys* :req [::fx-name]
                        :req-un [(or ::orc-string ::orc-filepath ::orc-internal-filepath)]
                        :opt [::fx-form ::ctl-instr ::num-outs
                              ::init-hook ::release-hook
                              ::release-time ::config]))
  :ret :instrument-controller fn?)
for this macro
(defmacro define-fx   [fx-name & {:keys [orc-string fx-form ctl-instr num-outs release-time
                     init-hook release-hook instance-config] :as env}] )
but there seems to be no instrumentation if I forget to provide required args, or type them incorrectly? Any obvious error here?
#2019-05-3016:59dronethereā€™s no key for the second key-pred pair in s/cat#2019-05-3017:13droneand [env] seems wrong should be removed, unless thereā€™s some form of fdef Iā€™m unfamiliar with#2019-06-0416:25hlolliyup that's right, works now, thanks!#2019-05-3119:40mpdairyso i have an idea for instrumentation with spec. It would be nice if the arguments to a function with a failing fdef were saved somewhere so they could be inspected in the repl, since a lot of times the spec error message is not very useful, and sometimes is so large it nearly crashes emacs.#2019-05-3119:41mpdairyspeaking of that, is there a way to truncate the length of spec error messages?#2019-05-3119:42ghadispec error messages should be really short as of the last changes#2019-05-3119:42ghadiwhat version clojure + spec are you using?#2019-05-3119:46mpdairyorg.clojure/spec.alpha "0.2.176" and clojure 1.10.0#2019-05-3119:48ghadiare you sure you're not mistaking the ex-data for the exception message?#2019-05-3119:48ghadi(that's the latest release)#2019-05-3119:49mpdairyyeah, it's the explain data. it's when it's showing the "actual" datatype, which can be huge#2019-05-3119:50ghadithat's a tooling thing... CIDER should have a toggle for it#2019-05-3119:50mpdairyoh ok#2019-05-3119:50ghadi(It used to be that the ex-data was also serialized into the message string, which is admittedly awful, but that is no longer the case)#2019-05-3119:51mpdairyoh yeah i remember those times#2019-06-0410:38Jakub HolĆ½ (HolyJak)Hello, how do I spec protocol methods? Just (s/fdef protocol-method :args (s/cat :this any? ...)) ?#2019-06-0410:56metametadataYou'll have to wrap the protocol method into a spec-able function. E.g. using defn-spec:
(defprotocol Protocol
  (-foo [_ x]))

(ds/defn-spec foo
  {::s/args (sp/pos any? ::specs/x)
   ::s/ret  ::specs/y}
  [this x]
  (-foo this x))
#2019-06-0412:32djtangowhat is (sp/pos ...) ?#2019-06-0414:57metametadata@U0HJD63RN ah, it's a local helper to reduce verbosity:
#?(:clj
   (defmacro -cat
     [& body]
     (if (-cljs-env? &env)
       `(cljs.spec.alpha/cat 
#2019-06-0415:06djtangoah nice - I had a feeling that was how it worked, just got my hopes up it might be something in spec-2 šŸ˜ž#2019-06-0417:56colinkahnIs there a way to control the recursion depth for spec generators?#2019-06-0417:57colinkahnMy use case is a recursive tree-like spec that Iā€™m generating using (gen/generate (s/gen ::my-spec))#2019-06-0417:58Alex Miller (Clojure team)there are some dynamic var knobs in the spec.alpha namespace#2019-06-0417:59Alex Miller (Clojure team)https://clojure.github.io/spec.alpha/clojure.spec.alpha-api.html#clojure.spec.alpha/*recursion-limit*#2019-06-0417:59Alex Miller (Clojure team)also, if you have any collection specs, I usually use :gen-max 3 to limit nested collections from going out of control#2019-06-0418:00Alex Miller (Clojure team)even given those, I think there are some known issues where the recursion is not well controlled#2019-06-0418:03colinkahnAwesome, :gen-max 3 is working for me, thanks!#2019-06-0421:35hlolliso for a string? bool in a macro, I get
{:path [:define-fx-params :orc-string],
                          :pred clojure.core/string?,
                          :val
                          (str
                           (slurp
                            (io/resource
                             "panaeolus/csound/fx/udo/shred.udo"))
                           "\ngkTransPose init 1\ngkTransRand init 0.1"),
                          :via [:panaeolus.csound.macros/orc-string],
                          :in [:orc-string]}
So it's obviously a string, but it's a macro, so I understand it sees a list at this point, any good tip to make this string check?
#2019-06-0421:38hlolli(s/def ::orc-string #(string? (eval %))) works actually#2019-06-0423:05Alex Miller (Clojure team)Thatā€™s not a good pattern#2019-06-0423:06Alex Miller (Clojure team)By evalā€˜ing here youā€™re basically ruining the lazy evaluation of macros and could even cause issues#2019-06-0423:07Alex Miller (Clojure team)In general macro specs are often tricky to write unless youā€™re trying to enforce positional constraints#2019-06-0423:07Alex Miller (Clojure team)For something like this I would probably not spec it at all#2019-06-0423:08Alex Miller (Clojure team)But would spec that arg as any? if I was#2019-06-0500:00seancorfield@hlolli Remember that macros take code as input and produce code as output -- they don't see the runtime type of that code.#2019-06-0500:05hlollihmm, my macros are the few things in my app that I actually want to spec, because of their interface nature to my app. Yeh, I can ofc spec a presence/absence of an argument. Or spec a function that the macro calls... also a solution.#2019-06-0501:00Alex Miller (Clojure team)the latter is a good idea#2019-06-0501:01Alex Miller (Clojure team)keep in mind that macro specs are checked at compile time. so they can only check things you know at compile time.#2019-06-0501:02Alex Miller (Clojure team)they can't check things about the values, because the values don't exist yet#2019-06-0501:02Alex Miller (Clojure team)so macro specs are great for checking syntax (in core we use them for ns, defn, destructuring, etc) - things that are macros with their own syntax defined by the macro#2019-06-0501:03Alex Miller (Clojure team)function specs are checked (when instrumented) at runtime when you have values in hand#2019-06-0501:03hlolliyup, so this makes total sense to me. I should keep it a habit to do as little work as possible in a macro, I only need the def on a symbol, and I can forward the rest to a function*#2019-06-0501:04Alex Miller (Clojure team)in general, having pairs like that is one common thing people do#2019-06-0501:04Alex Miller (Clojure team)with the caveat that the macro expands to a function call, so if you want that to be part of your public api, available outside the ns, then the function needs to be public too#2019-06-0501:05Alex Miller (Clojure team)sometimes that feels dirty#2019-06-0501:05Alex Miller (Clojure team)spec itself has this all over (s/and is a macro that expands to s/and-spec-impl etc)#2019-06-0501:06Alex Miller (Clojure team)we have significantly changed that in spec 2... but that's a longer story#2019-06-0501:07hlolliyeh I see, it's dirty in a forgiveable way. Other plus is that when I'm working in the repl, I only need to change the function once (given that I have a `(def ~symbol function) pattern), instead of re-evaling all def instances#2019-06-0501:09Alex Miller (Clojure team)yep#2019-06-0516:58dangercoderHi! Never looked at how Spec works when it comes to data that depend on other data. Is it possible to use clojure.spec to validate and generate relational data? Let's say I have a map like this:
{:min-amount 100.00M
 :max-amount 10000000.00M
 :amount 133.00M}
:amount must be within the bounds of min-amount and max-amount.
#2019-06-0516:59Alex Miller (Clojure team)you can do this with custom generators#2019-06-0517:00Alex Miller (Clojure team)gen the min and the max and then use gen/bind to create a generator that produces values in the range and packages them all together#2019-06-0517:00kszaboa tool in this area: https://github.com/reifyhealth/specmonstah#2019-06-0517:01dangercoderYeah I am listening to a podcast about Specmonstah right now#2019-06-0517:01kszabonot for the usecase you mentioned, but for relational data generation#2019-06-0517:01dangercoderThanks @alexmiller I will look more into how custom generators work! šŸ™‚#2019-06-0522:08plinshey everyone, I want to spec a map with 2 values (`:key1` :key2), they are both ints so ill use integer?.. but the second key must be bigger than the first, whats the best way to achieve that?#2019-06-0523:34seancorfield@plins wrap the s/keys in s/and and add a predicate#2019-06-0523:38seancorfield@plins if you need more concrete guidance than that, LMK and I'll paste an example#2019-06-0600:07plinsthank you very much but I've managed to do it šŸ™‚#2019-06-0615:45sveri#2019-06-0615:50Alex Miller (Clojure team)you're just getting the spammed ex-info data there, not a crafted message#2019-06-0615:50Alex Miller (Clojure team)which clojure version and where are you seeing this?#2019-06-0615:51Alex Miller (Clojure team)seeing it in repl (if so, which repl) or from lein command or other command?#2019-06-0709:31sveriGetting back to this problem. I did some digging and it turns out, if I run this function as part of a web request it gets wrapped into a exception clojure.lang.ExceptionInfo and printed out like seen above vs calling the function from the repl and seeing the more readable output. What would be the idiomatic way to use spec in this environment? I could catch the exception, and display some better error? Is it possible to extract a more readable error from the exception? Two things that bug me here. 1. ExceptionInfo seems like a very generic exception and I would have to look into it to check if its a exception regarding spec. 2. I would have to do this for every function that accepts / returns a web request while I would like to have it printed readable everytime, without having to do manual exception handling, if possible.#2019-06-0615:57sveriI see this in the cursive REPL when running a function of mine that I specced. There is also a lengthy stacktrace available and what fails is clojure.spec.test.alpha$spec_checking_fn$conform_BANG___3024.invoke Clojure version is 1.10.1 and spec is spec.alpha 0.2.176#2019-06-0616:06Alex Miller (Clojure team)what kind of repl in Cursive?#2019-06-0616:07Alex Miller (Clojure team)I'm most likely going to suggest you ask in #cursive as its really dependent on who is controlling the printing, which here is either cursive or nrepl#2019-06-0616:08Alex Miller (Clojure team)that is, if you used lein or clj in this same scenario, I think you would get a much better output#2019-06-0616:58vemvI post here a variation of something that I posted in #clojurescript this morning, but is essentially a different problem (it affects JVM clojure) I have a ns l from library L which requires clojure.core.specs.alpha, and uses one of its specs via spec/def. And I have a ns a from app A that requires l. When running A, I get java.lang.Exception: Unable to resolve spec: :clojure.core.specs.alpha/params+body. This doesn't happen when running L. I use the Reloaded workflow with vanilla requires and project organisation. All deps are the latest#2019-06-0617:19vemv@alexmiller : The problem appears to boil down to the fact that just that specific spec is absent from the registry#2019-06-0617:19vemv#2019-06-0618:10Alex Miller (Clojure team)there were some core spec renames in the 1.10-era core.specs.alpha - are you seeing mismatches maybe?#2019-06-0618:11Alex Miller (Clojure team)yeah params+body was a rename in 1.10-era from 1.9-era#2019-06-0618:12Alex Miller (Clojure team)changes happened in core.specs.alpha 0.2.44#2019-06-0618:14Alex Miller (Clojure team)so would be good to know the version of that being used by L and A, and also whether aot is happening on L#2019-06-0618:15vemvhi! I've been doing archeology for the last hour and I still can't make sense of things even when using 1.10.1 and the latest clojure.core.specs.alpha dep (stable, or snapshot, or none at all, since it's bundled), (require '[clojure.core.specs.alpha] :reload-all) won't get me the params+body spec in this particular project maybe Leiningen is doing sth evil#2019-06-0618:19vemvno aot in either side. tried lein clean, no luck either#2019-06-0618:23Alex Miller (Clojure team)
Clojure 1.10.0
user=> (require '[clojure.core.specs.alpha :as sa])
nil
user=> ::sa/params+body
:clojure.core.specs.alpha/params+body
user=> (require '[clojure.spec.alpha :as s])
nil
user=> (s/get-spec ::sa/params+body)
{:clojure.spec.alpha/op :clojure.spec.alpha/pcat, :ps [:clojure.core.specs.alpha/param-list {:clojure.spec.alpha/op :clojure.spec.alpha/alt, :ps ({:clojure.s...
#2019-06-0618:23Alex Miller (Clojure team)just using clj here#2019-06-0618:29vemvyes, in the original library ("L") it just works in a larger app ("A") using L it doesn't Did whatever I could to bisect, no luck I'll try creating another app, since A is quite large tbh, increasing the chances for weird interactions#2019-06-0618:41Alex Miller (Clojure team)does the registry contain any of the core.specs.alpha stuff?#2019-06-0618:41Alex Miller (Clojure team)(keys (s/registry))#2019-06-0618:48vemvmost of them yes. see #clojure , found the thing#2019-06-0619:17sveri@alexmiller its the same in the clj repl. But I have a lot of libraries loaded, not sure of something messes around with the output.#2019-06-0622:48plinsdid some googling but not found the anwser, is it possible to share specs between clj and cljs files? do I need to write .cljx files?#2019-06-0622:57Alex Miller (Clojure team)you can write specs in .cljc files and share them#2019-06-0623:40plinsany recommended documentation on how to setup clc files and share them? something has to be done in project.clj?#2019-06-0700:10seancorfield@U3QUAHZJ6 It's mostly automatic. The compilers look for both .cljc files and their own suffix (`.clj` or .cljs).#2019-06-0622:58Alex Miller (Clojure team).cljx is old and shouldn't be used anymore#2019-06-0707:08sveri@alexmiller I just checked my error in a minimal example and indeed the output is very different:
Execution error - invalid arguments to off/add-nutriments at (core.clj:9).
"" - failed: #{"kJ" "kcal" "kCal" "g" "mg"} at: [:product :nutriments :clojure.spec.alpha/pred :sugars_unit] spec: :off/unit
"" - failed: #{"kJ" "kcal" "kCal" "g" "mg"} at: [:product :nutriments :clojure.spec.alpha/pred :fat_unit] spec: :off/unit
"" - failed: #{"kJ" "kcal" "kCal" "g" "mg"} at: [:product :nutriments :clojure.spec.alpha/pred :salt_unit] spec: :off/unit
{:salt "3.5", :sugars_unit "", :energy-kcal "201.81514210652017", :energy_unit "kcal", :energy_value 202, :proteins_value "17.6", :proteins_unit "", :carbohydrates_unit "", :saturated-fat_value "8.2", :fat 12, :energy 845, :salt_value "3.5", :saturated-fat_unit "", :fat_100g 12, :sugars_value 6, :sodium_100g "1.37795275590551", :carbohydrates_value 6, :proteins_100g "17.6", :fat_unit "", :saturated-fat "8.2", :sugars_100g 6, :sodium "1.37795275590551", :sugars 6, :carbohydrates 6, :energy_100g 845, :salt_100g "3.5", :saturated-fat_100g "8.2", :salt_unit "", :carbohydrates_100g 6, :proteins "17.6", :fat_value 12} - failed: nil? at: [:product :nutriments :clojure.spec.alpha/nil] spec: :off/nutriments
I do use the same cursive repl so it must be some library that tinkers with the spec output settings. Thanks for the hint.
#2019-06-0707:09sveriAnd yea, this output definitely is much more helpful.#2019-06-0712:11Alex Miller (Clojure team)In 1.10.1 you should also get a full path location in the first line too#2019-06-0811:11mishais there a better way to make s/conform label values conformed to s/coll-of other than wrap s/coll-of in s/or?
(s/conform (s/coll-of any?) [])
=> []
(s/conform (s/or :vector (s/coll-of any?)) [])
=> [:vector []]
#2019-06-0811:13mishaneed it for a recursive spec, to uniformly label all tree nodes, to dispatch on conform-labels in a case#2019-06-0812:48Alex Miller (Clojure team)Canā€™t say I have any better idea other than to do this walk after the conform instead#2019-06-0813:33misha(re-)labeling conformed tree is doing (almost) double work. and implies having double knowledge of "how nodes look like" in the code: in spec and inline during the walk. Or, worse, dispatching with call to s/valid? or s/conform again, which will conform entire subtree from current node down. the (s/or) wrappers are hacky, but not that bad for irregular trees, all things considered:
(s/def :user/leaf int?)
(s/def :user/foo (s/coll-of (s/or :user/leaf :user/leaf :user/bar :user/bar :user/foo :user/foo)))
(s/def :user/bar (s/coll-of (s/or :user/leaf :user/leaf)))
(s/def :user/tree
  (s/or
    :user/leaf :user/leaf
    :user/foo :user/foo))

(s/conform :user/tree [[1] 2 [[3]]])
;;=> [:user/foo [[:user/bar [[:user/leaf 1]]]
                 [:user/leaf 2]
                 [:user/foo [[:user/bar [[:user/leaf 3]]]]]]]
#2019-06-0814:48Alex Miller (Clojure team)my complaint is basically that you're turning conform into a meat grinder to get a specific data structure you want, and that's not what it's designed for - it's designed to tell you a) is this valid? and b) why?#2019-06-0815:15potetmI thought it was also deigned to be ā€œdestructuring on steroidsā€#2019-06-0815:15potetm(I forget where I heard that.)#2019-06-0815:23Alex Miller (Clojure team)it has some capability in that regard when conforming regex for syntax structures in macros, but even then it's not destructuring, more making structures amenable to destructuring#2019-06-0906:38mishaI completely agree with your complaint. But it works! kappa#2019-06-0906:39misha(went with loop+case and weaker jit errors instead of spec for such tree this time, though)#2019-06-0907:08mishafurther destructuring of a form conformed to regex-spec and to or-spec is different: map vs "twople". In case of a tree, you'll walk 1 node at a time, which makes
(let [[tag form] conformed]
  (case tag ...))
preferable to
(let [{:keys [tag1 tag2 tag3 ...]} conformed]
  (cond ;; I guess?
    tag1 ...
    tag2 ...))
#2019-06-0907:09mishabut latter brings the same edge cases as (contains? #{...} x) vs. just (#{...} x): nil, false#2019-06-1002:38andy.fingerhutThere is an example spec using s/double-in on the spec guide page: (s/def ::dubs (s/double-in :min -100.0 :max 100.0 :NaN? false :infinite? false)) I have tried changing the false occurrences to true, and leaving out the :NaN? and :infinite? keys, but I always see (s/valid? ::dubs ##Inf) evaluate to false, as well as (s/valid? ::dubs ##-Inf) and (s/valid? ::dubs ##NaN). Is that expected?#2019-06-1002:44andy.fingerhutI guess it makes sense for ##-Inf to be invalid if a :min value is specified that is a double value other than ##-Inf, and similarly for :max values and ##Inf.#2019-06-1002:44andy.fingerhutBut it seems like specifying either :min or :max changes the validity of ##NaN from true to false.#2019-06-1002:44andy.fingerhutno matter what the value associated with the :NaN? key happens to be#2019-06-1002:49Alex Miller (Clojure team)might be the ordering of the checks#2019-06-1002:50Alex Miller (Clojure team)if you (s/form :;dubs) you can see what it expands to#2019-06-1003:02Alex Miller (Clojure team)maybe buggy at a glance as it's an s/and and it starts flowing the boolean rather than the double#2019-06-1018:37andy.fingerhutI doubt that is a hot ticket item for anyone, but created a JIRA ticket as a reminder: https://clojure.atlassian.net/browse/CLJ-2516#2019-06-1213:23carkhmulti-spec dispatching on the first value in a sequence and returning the syntax for the rest of that sequence, possible at all ?#2019-06-1213:24carkhall examples of multi-specs i could find were about maps#2019-06-1213:26carkhwhat i'm trying to do : (s/def ::node (s/cat :tag keyword? (multispec dispatching on the tag here)))#2019-06-1213:30Alex Miller (Clojure team)it is not only maps, can definitely be made to work with other things (I've got some stuff doing that)#2019-06-1213:32Alex Miller (Clojure team)you can make your multimethod dispatch on first#2019-06-1213:32Alex Miller (Clojure team)and then you have to be a little careful with the retag function#2019-06-1213:32Alex Miller (Clojure team)I actually have an example of this in CLJ-2112 (specs on specs) I think#2019-06-1213:33Alex Miller (Clojure team)https://clojure.atlassian.net/secure/attachment/10625/spec-forms.patch#2019-06-1213:33Alex Miller (Clojure team)you'll see spec-form is a defmulti on first#2019-06-1213:33Alex Miller (Clojure team)and the ::spec spec uses (s/multi-spec spec-form (fn [val tag] val))#2019-06-1213:34Alex Miller (Clojure team)and then the defmethods cover the different cases with different starting symbols#2019-06-1213:38carkhthanks looking at your example#2019-06-1213:42carkhok got it working on the whole "sequence", i'm guessing this can't be done the way i want, dispatching on the first element in order to spec the rest of the sequence#2019-06-1213:42carkhnot a huge deal, just a little usability issue i guess#2019-06-1213:44Alex Miller (Clojure team)isn't the example above what you're describing?#2019-06-1213:44Alex Miller (Clojure team)I'm using it to spec a form sequence based on the operator type at the beginning, so seems like the same thing?#2019-06-1213:52carkh(s/cat :tag keyword? :rest (s/multi-spec node-type :rest)) it looks like it's looking for a sequence in the rest#2019-06-1213:53carkhthe mmultispec is returning something like (s/cat ....) a flat thing#2019-06-1213:53carkhis multispec like s/spec in this regard ?#2019-06-1213:54carkh(i have a hard time following your example)#2019-06-1213:55Alex Miller (Clojure team)you don't want an s/cat here at all - the multi-spec is the sequence spec#2019-06-1213:55carkhbut i need something to put around my stuff returned from the multimethod =)#2019-06-1213:55carkhbut yes i understand#2019-06-1213:56Alex Miller (Clojure team)I don't understand what that meant#2019-06-1213:56Alex Miller (Clojure team)the multimethod returns the spec for the sequence#2019-06-1213:56carkhright#2019-06-1213:56carkhin the typical example for a typed map, you spec for the type keyword outside the multimethod#2019-06-1213:57carkhthen only spec for what's relevant to the multispec user in the multimethod#2019-06-1213:57carkhbut i think what i'm asking is impossible#2019-06-1213:58Alex Miller (Clojure team)afaict what you're asking for is the example I gave. I don't get why it's not what you want.#2019-06-1213:59Alex Miller (Clojure team)are you looking for something different in the conformed value shape?#2019-06-1214:00carkhin your example, i want the :f #{'clojure.core/fn} out of the multimethod#2019-06-1214:00carkhjust like for the typical typed map examples#2019-06-1214:01carkhactually i'm wrong on that#2019-06-1214:01carkhevent/type is indeed present in the multi-method in the guide#2019-06-1214:02carkhok then sorry for the noise ><#2019-06-1214:03Alex Miller (Clojure team)you can post-process the conformed value to do whatever you want of course#2019-06-1214:04carkhi was just weary that all the users of the multimethod will have to spec for the "tag"#2019-06-1214:04carkhnot a huge deal since all those users are me =)#2019-06-1214:04carkhthanks for your time !#2019-06-1214:16Alex Miller (Clojure team)yeah, you can just spec it as any? if you like#2019-06-1214:16Alex Miller (Clojure team)it's already been chosen by the time you get into the multimethod and the value will flow through#2019-06-1214:17Alex Miller (Clojure team)I spec'ed it in that code just for clarity#2019-06-1217:04colinkahnWhen multi-specing where the specs are s/keys and the retag is a keyword, is it best practice to include the retag key as :req in the defmethods keys spec?#2019-06-1217:23Alex Miller (Clojure team)not sure what you mean by "include the retag key as :req"#2019-06-1217:23Alex Miller (Clojure team)oh, you mean as a :req in the s/keys#2019-06-1217:24Alex Miller (Clojure team)well it is probably required to get to that point in the first place#2019-06-1217:25Alex Miller (Clojure team)and either way it's going to end up in the conformed value#2019-06-1217:25Alex Miller (Clojure team)I'm not sure that it matters either way, depends on whether you see the retag key as part of the data or as frame around the data, but that's probably dependent on your code#2019-06-1217:51colinkahnOk cool, yeah I was doing it quite consistently in :req and ran into something where I was adding a :gen to the keys spec in the defmethod and found it a bit unergonomic to have to include the retag in the generator as well, which got me feeling like maybe it was better not to include it at all. I have been also specing the retag key mostly as something rather generic like keyword? too.#2019-06-1311:12vemvHey there, I wrote down this design https://github.com/reducecombine/fridge/issues/9 50% for fun, 50% b/c it might actually be necessary at work. It doesn't seem too extravagant (in fact I view it as very aligned with RH-isms: immutable/never-broken specs, growth by accretion, no renames etc), but still, there is a gap between theory and practice šŸ™‚ i.e. my design might actually have flaws in a real system. It would be super appreciated if someone with relevant experience could comment in the issue.#2019-06-1311:12vemvHey there, I wrote down this design https://github.com/reducecombine/fridge/issues/9 50% for fun, 50% b/c it might actually be necessary at work. It doesn't seem too extravagant (in fact I view it as very aligned with RH-isms: immutable/never-broken specs, growth by accretion, no renames etc), but still, there is a gap between theory and practice šŸ™‚ i.e. my design might actually have flaws in a real system. It would be super appreciated if someone with relevant experience could comment in the issue.#2019-06-1311:45Alex Miller (Clojure team)Seems fine. I guess my only real question is whether apps and models really need that level of absolute separation that comes from putting the app name in the attribute qualifier#2019-06-1311:47Alex Miller (Clojure team)Seems like you will end up basically copying the same attribute definition everywhere (and require unnecessary data transformation at each step). The whole idea of the global registry is to get reuse#2019-06-1311:48Alex Miller (Clojure team)There is more coming on datafied specs in spec 2 btw#2019-06-1312:15vemv> Seems like you will end up basically copying the same attribute definition everywhere Not sure I can follow. Let's say I have N repos, 1 per microservice, plus 1 special repo, the 'immutable spec registry' Over there, I (spec/def :messaging.blue-app.v1/age integer?) And microservices can consume that spec (and quite concisely, via require ... :as ...). There's no copying in that > and require unnecessary data transformation at each step What did you mean?#2019-06-1313:22Alex Miller (Clojure team)I assume these services pass data to each other. It seems improbable that you won't have multiple services handling the same semantic attribute (like "age"). in your scheme, that attribute will show up in the messages for all the services - :messaging.blue-app.v1/age, :messaging.red-app.v1/age etc#2019-06-1313:23Alex Miller (Clojure team)if a service ever takes a map with one variant and returns a map with the other variant, you have rebuild the map and change the attribute name#2019-06-1313:24Alex Miller (Clojure team)whereas if you had a :messaging.common.v1/age, it could flow between services and be shared#2019-06-1313:24Alex Miller (Clojure team)which was the whole point of the global registry#2019-06-1317:47seancorfieldThis is why we tend to use more generic domain-level prefixes for our specs, rather than specific/namespace-qualified names.#2019-06-1318:24vemvUnderstood! I'll give it a think, however I find some value in having this design: Each microservice can retain a specific meaning and type for age. That way, services can evolve independently, without coordination. The opposite might resemble a distributed monolith, where making a change/decision needs asking everyone else (other codebases / other team members) first (btw, messaging is via Kafka so the map-in-map-out philosophy doesn't apply that much)#2019-06-1321:23rapskalian@U04V70XH6 could you give an example of what you mean by ā€œdomain-levelā€? Would you tend to favor :user/email over :myapp.auth.user/email?#2019-06-1321:49seancorfieldFor stuff that stays "inside the company", the qualifier doesn't need to be any more specific than the broadest level of uniqueness needed. So, yeah, if you want to standardize on an email representation for all your users across the whole company, :user/email is fine. It shouldn't be part of any public APIs/libraries you create -- but using commonly known domain names inside your company helps reuse across multiple systems.#2019-06-1321:52seancorfieldA specific example is :wsbilling for us -- for a lot of things that are related to our World Singles Billing services rather than being tied to a single app, these specs are reused by all apps that support and interact with those services. We've used :wsbilling there rather than just :billing because there are other generic billing services we interact with, so this just hints that it's about our billing services rather than anything else we interact with.#2019-06-1323:01rapskalianVery helpful, thanks#2019-06-1406:36jaihindhreddy@U45T93RA6 "making a change/decision needs asking everyone else (other codebases / other team members) first", as RH says in Spec-ulation, the way we use the word change tends to complect growth and breakage, which are completely different. As long as you're only doing growth, these is no need to ask (or even tell) anyone.#2019-06-1406:45seancorfieldThat's why it's so important to me that next.jdbc will never make breaking changes -- only accretive/fixative changes, since it moved into beta. I learned a lot of lessons from maintaining clojure.java.jdbc for the last eight years! šŸ™‚#2019-06-1407:36vemv@ jaihindh, how do you tell if a change is breaking or not? as hinted in the document, One man's non-breaking is another's breaking. My approach is pessimistic in that it assumes all changes are potentially breaking. At the same time, it encourages services to keep sending/accepting messages at old versions, so you don't actually break 3rd parties. This philosophy is partly justified by the fact that atm we don't have a tool that automatedly analyses spec compatibility. Might be unfeasible given that specs can be backed by arbitrary code#2019-06-1411:22jaihindhreddyI don't have the answer (don't even have an answer) for how to partition "change" into "growth" and "breakage". And no matter how hard we try, at some point people will rely on things that are broken, making one man's fixation another man's breakage, etc. But if we keep changing our names by incorporating versions into them and ask people to assume they are potentially breaking, we still lack a stable set of names that can act as a stable basis to reason about our data across time. That, is the real systems problem right? Sorry for the blabbering & noise šŸ˜…#2019-06-1416:06seancorfieldYou don't change the names. Just introduce new names. The old names still exist and have the old behavior. Folks can continue to rely on those old names "forever". #2019-06-1413:30dmarjenburghIs there a convenient way to separate specs definitions and their generators? I want to define my specs with my main code, but only use generators in test code. I include test.check in my classpath only with the test alias. I know spec loads generators lazily for this purpose, but not sure how to use s/with-gen when the generator argument uses stuff depending on test.check#2019-06-1413:42dmarjenburghOkay, I think I got it. I was calling gen/fmap when my namespace of custom generators was loaded.#2019-06-1516:47misha
(->> [0 1 2 3]
  (partition-all 2)
  (into {}))
ClassCastException   [trace missing]

(->> [[0 1] [2 3]]
  (into {}))
=> {0 1, 2 3}
šŸ˜ž
#2019-06-1516:48misha
(->> [0 1 2 3]
  (partition-all 2)
  (map vec)
  (into {}))
=> {0 1, 2 3}
#2019-06-1519:20andy.fingerhutone of the places in Clojure where vectors vs. non-vectors makes a difference.#2019-06-1519:34y.khmelevskiihey, can you please help me to understand why lein throw this error
If there are a lot of uncached dependencies this might take a while ...
clojure.lang.ExceptionInfo: Could not find artifact org.clojure:spec-alpha2:jar:0.2.177-SNAPSHOT in central ()
{:lib org.clojure/spec-alpha2, :coord {:mvn/version "0.2.177-SNAPSHOT", :deps/manifest :mvn}}
#2019-06-1519:37y.khmelevskiiI use leiningen together with deps.edn#2019-06-1519:40bronsasnapshot releases are not in maven central, they're in sonatype#2019-06-1519:40bronsatry adding in your repos#2019-06-1519:41y.khmelevskiiwhich way? via :mvn/repos?#2019-06-1519:41bronsayeah#2019-06-1519:43y.khmelevskii
:mvn/repos {"clojars" {:url ""}
             "central" {:url ""}}
it doesnā€™t work
#2019-06-1519:58y.khmelevskiiand my dependency org.clojure/spec-alpha2 {:mvn/version "0.2.177-SNAPSHOT"}#2019-06-1520:00bronsanot sure if lein picks up repos from deps.edn#2019-06-1520:00bronsaif you're sure that snapshot exists, maybe trying adding that repo in .lein/profiles.clj and retry#2019-06-1520:00bronsadon't have any other suggestions, sorry#2019-06-1609:52hlollihow is it possible to prevent a generator from doing a retry without re-calling a generator function? Here's an ugly phone number generator
(defn phone-number-generator []
  (gen/fmap (fn [[cc phone-num]]
              (str "+" cc " (0) " phone-num))
            (gen/tuple (gen/return (rand-nth ["1" "37" "39" "44" "49"]))
                       (gen/return (-> (* Integer/MAX_VALUE (Math/random))
                                       int
                                       str)))))
printing the validation from the spec
"VALID?" "+1 (0) 265957916" false
it gets print 100 times exactly the same, the generator is only called 1x or 2x that all ends with
Execution error (ExceptionInfo) at clojure.test.check.generators/such-that-helper (generators.cljc:320).
Couldn't satisfy such-that predicate after 100 tries.
#2019-06-1611:20gfredericks@hlolli generally you design the generator so that it has no need to retry#2019-06-1611:20gfredericksI can't see the spec so I don't know what that would be#2019-06-1611:21hlolliI see, it's this here
(defn-spec phone-number-valid? boolean?
  [phone-number string?]
  (let [instance (PhoneNumberUtil/getInstance)
        phone-number-parsed (.parse instance phone-number "")]
    (.isValidNumber instance phone-number-parsed)))

(s/def :user/phone-number ;  format
  (s/with-gen phone-number-valid?
    generators/phone-number-generator))
#2019-06-1611:23gfredericksokay well there's some stuff buried in the jvm method but for example, is the last portion supposed to be a particular number of digits?#2019-06-1611:25hlolliyes, it depends I think which country code is used, so I just multiply some random number to sizeof(int), should give me a more likely way of number that purely random number generator, I think?#2019-06-1611:25hlolliOr I could choose 1 counry code and substring to a specific length#2019-06-1611:25gfredericks(* Integer/MAX_VALUE (Math/random)) is going to be pretty skewed towards certain lengths#2019-06-1611:26gfredericksif it's easy to get a length associated with each country code, then you could do it much better#2019-06-1611:27hlolliyes, these rules are difficuly, despite all lengths being right, sometimes the first digit must fit some standard#2019-06-1611:27hlolliI can read more about this ofc#2019-06-1611:27hlollibut I thought the generator could try again and again until it's valid#2019-06-1611:27hlollilike with regex specs#2019-06-1611:27gfredericks
(gen/for [[country-code length] (gen/elements country-codes-and-lengths)]
          number (apply gen/tuple (repeat length (gen/choose 0 9)))]
  (str "+" country-code " (0) " number))
is the sort of thing I would aim for if I Had that list
#2019-06-1611:28hlollicool, instersting šŸ™‚#2019-06-1611:28gfredericksrelying on the generator trying again is not a good plan unless you're sure it's highly likely to succeed#2019-06-1611:28gfredericksbecause you can easily be in a situation where it only succeeds e.g. 0.001% of the time#2019-06-1611:28gfredericksand then you'll be burning a lot of CPU#2019-06-1611:29hlolliyes true, I thought the idea would be to give the generator a bit of help. The odds of alpha-numberic random gen hitting a valid phone number are probably astronomical, I'd assume#2019-06-1611:29gfredericksyeah, you can definitely do worse than what you have šŸ™‚#2019-06-1612:11gfredericks@hlolli incidentally, here's a generator that's evenly distributed between digit lengths
(gen/for [digit-count (gen/choose 5/10)
          digits (apply gen/tuple (repeat digit-count (gen/choose 0 9)))]
  digits)
#2019-06-1612:12gfredericksthat might work better for you#2019-06-1612:12hlolliok, but I didn't get gen/for, in which namespace is it?#2019-06-1612:13hlolliI'll try soon and let you know, I'm generating fake firebase users for testing#2019-06-1612:14gfredericksclojure.test.check.generators#2019-06-1612:14gfredericksit's just syntax sugar for gen/fmap and gen/bind mostly#2019-06-1612:14gfredericksyou can do the above just using gen/bind#2019-06-1612:17hlolliah I see, I was searching for it in clojure.spec.gen.alpha#2019-06-1612:18hlollithanks šŸ™‚#2019-06-1701:21Alexander StavoninIs it possible to check that a spec argument is a function with expected signature?#2019-06-1701:24Alexander StavoninI can easily implement such check with protocols, but in one particular case function is preferable for me.#2019-06-1702:40taylorspec alpha1 ā€œchecksā€ function values by invoking them; so yes, if that meets your needs you could spec function args#2019-06-1702:40taylormaybe you could do something with meta and :arglists?#2019-06-1703:11Alexander StavoninI have a function which accept other function as argument. Iā€™d like to be sure, the argument is: a) function, b) function with expected signature (lets say with 2 arguments). I suppose, fn? could be helpful here, but what should I do with function arguments test?#2019-06-1704:33seancorfield@UKHQCLTQV clojure.spec isn't a type system.#2019-06-1704:36Alexander StavoninI know, but sometimes type-system like guarantees are very convenient. This is especially important in cross components calls. Can you suggest any other way to which will provide such guarantees?#2019-06-1704:39seancorfieldMaybe you want to look at Typed Clojure instead, in order to analyze code and ensure that you're passing the right sort of function as an argument?#2019-06-1704:44Alexander Stavonin@U0JLGECPK, are you talking about https://github.com/clojure/core.typed? I just looked thru readme and got feeling Typed Clojure means ā€“ whole project should be statically typed. This is not a goal for me, as Iā€™m more or less happy with dynamic typing, and the only needs is ā€“ cross components interface data and types validity guarantees.#2019-06-1704:48Alexander StavoninOk, I should look deeper here:
This work adds static type checking (and some of its benefits) to Clojure, a dynamically typed language, while still preserving idioms that characterise the language. It allows static and dynamically typed code to be mixed so the programmer can use whichever is more appropriate.
#2019-06-1704:48Alexander Stavoninthanks!#2019-06-1704:51Alexander Stavoninjust interesting reading about this project: https://circleci.com/blog/why-were-no-longer-using-core-typed/#2019-06-1704:57seancorfieldYeah, there are trade offs. And that's an old article. core.typed has had a lot of work since then.#2019-06-1704:58seancorfieldWe also tried it back then and gave up for all the same reasons CircleCI gave up.#2019-06-1704:58seancorfieldBut the question you're asking is best answered by core.typed rather than spec.alpha#2019-06-1710:46djtangothat said - Racket was able figure out a solution to higher-order functions with their run-time contracts system, that also doesn't involve doing gen-testing on the input function#2019-06-1710:47djtangobut function equivalence is undecidable so tradeoffs are always going to have to be made in this space...#2019-06-1712:52droneTyped Clojure seems to be pretty inactive. While spec isnā€™t a type system, I could see function arity being part of a contract. E.g., for a function passed to reduce. But I also think the spec designers would argue that function arity errors already exist and solve most of the problem.#2019-06-1716:02seancorfieldAmbrose just successfully defended his thesis work on Typed Clojure. It's still an active project. #2019-06-1716:10droneThere was a flurry of activity last winter, and then other than a few commits to the core.typed.analyzer.jvm project in April (likely in preparation for defense) there has been no activity for seven months#2019-06-1716:17seancorfieldIf you've watched any of his talks you know that he has a lot of design work ongoing about dealing with the typed / untyped code boundary. Just because there have been no recent commits doesn't mean there's no recent work -- this is an extremely hard topic area that requires a lot of "hammock" time. #2019-06-1716:23dronesure, but itā€™s also just as likely he is moving on from the project after completing his PhD program#2019-06-1719:59seancorfieldNot based on what he's talked about at conferences.#2019-06-1722:11dronethatā€™s great if heā€™s still working on it. but based on observable output, including: repo commits, the projectā€™s twitter account, his patreon, and activity in #core-typed; not much is going on in the project and thatā€™s what I meant by ā€œinactiveā€. this seems like a weird thing to argue aboutā€¦#2019-06-1712:47Jakub HolĆ½ (HolyJak)Is there a shorter, nicer way to do this (without nesting if in let)?
(defn conform! [x spec]
  (let [conformed [(s/conform spec x)]]
    (if (#{::s/invalid} conformed)
      (throw (ex-info
               (s/explain-str spec x)
               (s/explain-data spec x)))
      conformed)))
#2019-06-1712:52Charles FourdrignierThis looks like a lot to Stuart Halloway conform!, so I guess you can't do better. https://github.com/Datomic/mbrainz-importer/blob/master/src/cognitect/xform/spec.clj#2019-06-1816:13OlicalI came up with something similar but as a macro
(defmacro on-invalid [conformed fallback]
  `(let [conformed# ~conformed]
     (if (s/invalid? conformed#)
       ~fallback
       conformed#)))

(defn validate [x spec message]
  (when-not (s/valid? spec x)
    (throw (ex-info (str message "\n\n" (expound/expound-str spec x)) {})))
  x)
#2019-06-1713:32ghadi@holyjak don't do that because you'll be putting a ton of data in the exception message (via explain-str)#2019-06-1713:33Jakub HolĆ½ (HolyJak)hm, good point, thanks!#2019-06-1713:37Jakub HolĆ½ (HolyJak)maybe I should just use (-> explain-data ::s/problems first (dissoc :val :value)) instead?#2019-06-1823:01mishaimplementing this as macro has an advantage of being able to include calling form into error data/message, which saves a lot of time for exceptions purposed for developer and repl. or just have an error-msg arg in a second arity, with default message like "does not conform to spec <spec form>" because you'll expand exception data in repl anyway, because "first problem" is not always "the only problem"#2019-06-1916:16Jakub HolĆ½ (HolyJak)Thanks!#2019-06-1914:42hmaurero/. Quick question: is the new way of handling optionality in clojure spec (with selections etc) already usable?#2019-06-1915:14dronelike nilable? or do you mean :opt and :opt-un in keys?#2019-06-1915:18seancorfield@mrevelle He means in spec-alpha2. Right @hmaurer?#2019-06-1915:18droneah..#2019-06-1915:18hmaurer@seancorfield I think yes; what Rich Hickey talked about at this autumnā€™s conj#2019-06-1915:19seancorfieldWe have a branch at work that runs with spec-alpha2, but itā€™s still very much a moving target right now, and has a number of bugs.#2019-06-1915:19seancorfieldItā€™s probably fine to ā€œplayā€ with but Alex has advised against using it for any serious work yet.#2019-06-1915:24hmaureršŸ‘Œ thanks#2019-06-1915:34ghadihttps://github.com/clojure/spec-alpha2/wiki/Schema-and-select some docs here, everything subject to change#2019-06-1915:37hmaurerty!#2019-06-1915:34ghadi@hmaurer#2019-06-2007:56Ben HammondI am calling generate on a complex data structure. Sometimes I see this error. I would normally troubleshoot this by commenting out parts of the spec to zone in on the problem; but I'm wondering if there is a smarter way to debug generators#2019-06-2007:57Ben Hammondis there a direct way to dicover the specific sub-spec that has the problem?#2019-06-2009:51Ben Hammondokay another question; I want a generator that has two modes of operation ā€¢ when there is data available in the dynamc variable *existing-ids* then I want it to behave like (element *existing-ids)` ā€¢ when *existing-ids* is empty then I want it to behave like (spec/gen string?) this must be a fairly common use case how has it been solved before?#2019-06-2009:55Ben HammondI'm thinking something like
(test.gen/bind
  (test.gen/return nil)
  (fn [_]
      (if (not-empty *existing-ids*)
        (test.gen/elements *existing-ids*)
        test.gen/string-alpha-numeric)))
#2019-06-2010:00Ben Hammondthat doesn't seem a very smart solution though#2019-06-2010:11Ben HammondI think that's the best I can do, unless I hack directly into
(defn- bind-helper
or
(defn- make-gen
but that doesn't seem like a very good idea either
#2019-06-2010:15Ben Hammond
(def ^:dynamic *existing-ids* nil)
=> #'dev/*existing-ids*
(def gg (test.gen/bind
          (test.gen/return nil)
          (fn [_] (if (not-empty *existing-ids*)
                    (test.gen/elements *existing-ids*)
                    test.gen/string-alphanumeric))))
=> #'dev/gg
(spec.gen/generate gg)
=> "9Dlv2SuFqT4Jb"
(binding [*existing-ids* [:this :that :tother]]
  (spec.gen/generate gg))
=> :that
#2019-06-2010:18Ben HammondI suppose I could store the test.gen/elements in the *existing-ids* var#2019-06-2010:23Ben Hammond(spec/with-gen takes a gen-fn, and I'd hope that I could use this to choose between generators, but of course it doesn't get called very often#2019-06-2010:31Ben Hammondthis might be better actually
(defn build-indirect-gen 
  [vargen] 
  (test.gen/bind
    (test.gen/return nil)
    (fn [_] (deref vargen))))

(def ^:dynamic *existingids-gen* test.gen/string-alphanumeric)
(def gg (build-indirect-gen #'*existingids-gen*))

(binding [*existingids-gen* (test.gen/elements [:this :that])]
  (test.gen/generate gg))
=> :that 

(test.gen/generate gg)
=> "Sjeu5IwBWbMUZl"
#2019-06-2015:02misha@ben.hammond afair, Gary touched on it debugging "Couldn't satisfy such-that predicate after 100 tries" here: https://www.youtube.com/watch?v=F4VZPxLZUdA#2019-06-2015:04mishathis is the timestamp#2019-06-2015:04mishahttps://youtu.be/F4VZPxLZUdA?t=976#2019-06-2015:17Ben HammondI've not seen that video. that's very interesting#2019-06-2016:12Ben Hammondi'd not seen the size/scale explained properly before#2019-06-2016:12Ben HammondI'd assumed they represented mean/standard deviation#2019-06-2018:04Alex WhittI'm sure this has been asked and answered, but I can't find the answer... Why can I not do the following?
(s/def ::a any?)
(s/def ::thing (let [ks [::a]] (s/keys :req ks)))
That yields Don't know how to create Iseq from: clojure.lang.Symbol.
#2019-06-2018:27Alex WhittI'm guessing that macros are the only way to dynamically define specs?#2019-06-2018:39favilapretty much#2019-06-2018:39favilaI think spec2 has some improvements to this, but still isn't going to let you plop a let in there#2019-06-2018:40favilaI think spec2 will expose a public ast-like interface for programmatic spec creation/manipulation of specs as data#2019-06-2018:40favilabut the surface syntax will still be macros#2019-06-2018:41Alex WhittHmm, alright. I'm interested in the spec2 feature you mentioned so I'll take a look. Thanks!#2019-06-2019:23seancorfieldSpec 2 has a macro layer and a function layer that implements the macros. You can programmatically construct pretty much any spec in Spec 2.#2019-06-2019:24seancorfield(we have a branch of our code base at work that uses Spec 2 -- so we can track the sort of differences we'll need to make when we transition from Spec 1)#2019-06-2113:21Alex WhittAh, that's exciting!#2019-06-2107:48Ben Hammondfollowing on from yesterday; I think the tidiest way to achieve > generator uses id from a database, > when there is a database is to use the overrides feature of (spec/gen)
(doc spec/gen)
-------------------------
clojure.spec.alpha/gen
([spec] [spec overrides])
  Given a spec, returns the generator for it, or throws if none can
  be constructed. Optionally an overrides map can be provided which
  should map spec names or paths (vectors of keywords) to no-arg
  generator-creating fns. These will be used instead of the generators at those
  names/paths. Note that parent generator (in the spec or overrides
  map) will supersede those of any subtrees. A generator for a regex
  op must always return a sequential collection (i.e. a generator for
  s/? should return either an empty sequence/vector or a
  sequence/vector with one item in it)
=> nil

#2019-06-2111:40djtangothanks for this btw - am enjoying your writeup! Shame it'll be gone whenever the history runs out#2019-06-2115:03vemvI was thinking that a spec/not could make sense as exemplified here
(spec/def ::version (fn [s]
                      ...))

(spec/def ::alpha-version (spec/and ::version
                                    (fn []
                                      ...)))

(spec/def ::stable-version (spec/and ::version
                                     (spec/not ::alpha-version))) ;; <- imaginary API
has this been considered already?
#2019-06-2115:04vemv(I guess the generative part would suffer)#2019-06-2115:14Alex Miller (Clojure team)I've written a ton of specs and have never needed an s/not#2019-06-2115:15vemvI've also written a fair number of specs for a few years. Just now I think of not, so that kind of proof is limited#2019-06-2115:16Alex Miller (Clojure team)you could do something like (spec/def ::stable-version #(spec/invalid? (spec/valid? ::alpha-version %)))#2019-06-2115:16Alex Miller (Clojure team)it is probably inherently difficult to auto-gen#2019-06-2115:17Alex Miller (Clojure team)in any case, we have no plans to add it#2019-06-2115:17vemvšŸ‘ this was more curiosity than anything else. One could always write his own not anyway#2019-06-2115:39Ben HammondI've got quite an interesting situation; I have a spec that generates a fairly complex data structure I sometimes want to overide some of the generators to make it use foreign keys from the database but it only works intermittently: I have hooked up a snitching ILookup to tell me what is happening around
(if-let [g (c/or (when-let [gfn (c/or (get overrides (c/or (spec-name spec) spec))
                                          (get overrides path))]
and what I see is that When a recompile that spec directly to REPL, then the overridy will wok When I recompile the entire namespace to REPL, it does not work Which is intriguing behavour
#2019-06-2115:40Ben Hammondso this my ILookup
(reify ILookup
        (valAt [_ k]
          (println (str "valAt*1: " k))
          (println (str "==>" (get m k)))
          (get m k))

        (valAt [_ k nf]
          (println (str "valAt*2: " k ":" nf))
          (println (str "==>" (get m k nf)))
          (get m k nf))
        )
#2019-06-2115:41Ben Hammondwhen it works, I see
valAt*1: :db.generators.offers/offer
==>
valAt*1: []
==>
valAt*1: :db.generators.offers/offer_headline
==>
valAt*1: [:offer_headline]
...
#2019-06-2115:41Ben Hammondwhen it does not work, I see
valAt*1: :db.generators.offers/offer
==>
valAt*1: []
==>
#2019-06-2115:42Ben Hammondis this ringing any bells?#2019-06-2115:45Alex Miller (Clojure team)specs compile in their dependent specs so if you modify a spec, you need to reload any specs that depend on it. that's a likely reason you'd see different results for the two cases#2019-06-2115:45Alex Miller (Clojure team)I'd expect "recompile the entire namespace" to give you the more accurate answer.#2019-06-2115:46Alex Miller (Clojure team)hard for me to tell from this what the actual problem is though#2019-06-2115:47Ben Hammondperhaps I have misunderstood usage. I looked at
(doc spec/gen)
-------------------------
clojure.spec.alpha/gen
([spec] [spec overrides])
and hoped that I could plug in some overrides that would get me proper foreign keys, and it would just work
#2019-06-2115:48Ben Hammondwould you expect to re generate all of the specs to handle a overrides map?#2019-06-2115:50Alex Miller (Clojure team)you should be able to plug in overrides that way. I was responding to
When a recompile that spec directly to REPL, then the overridy will wok
When I recompile the entire namespace to REPL, it does not work
which seemed like a pretty textbook outcome from spec compilation
#2019-06-2115:50Alex Miller (Clojure team)you haven't actually shown what you're doing, so I can't really comment#2019-06-2115:50Alex Miller (Clojure team)can you give a full example?#2019-06-2115:52Ben Hammond
(spec/def ::offer
  (spec/keys
    :req-un [::offer_headline
             ::offer_classifiers
             ::offer_title
             ::offer_title_short
             ::offer_merchant
             ::offer_voucher_codes
             ::offer_description
             ::offer_terms_and_conditions
             ::offer_terms_and_conditions_url
             ::offer_claim_restrictions
             ::offer_presentation]
    :opt-un [::offer_pre_claim_advice
             ::offer_key_terms
             ::offer_redemption_guidelines
             ::offer_taxable_value
             ::offer_identifiers
             ::offer_images
             ::offer_approval_required
             ::offer_discount_mechanic
             ::claim_condition_msisdn_list]))


(spec/def ::offer_merchant ::ingestion/merchant_id)
and then I have an ingestion namespace that says
(s/def ::merchant_id (s/and string? #(<= 1 (count %) 64)))
is the main offer spec
#2019-06-2115:53Ben Hammondso I'm a bit suspicious about that (spec/def ::offer_merchant#2019-06-2115:54Ben Hammondand now I want to generate a bunch off offers where the merchant ids have come out of the database#2019-06-2115:54Ben Hammondso I write
(:offer_merchant
 (test.gen/generate
   (spec/gen :db.generators.offers/offer
     (let [m {:customer.ingestion/merchant_id (constantly (test.gen/return "OVERRIDEa"))
              :db.generators.offers/offer_merchant (constantly (test.gen/return "OVERRIDEb"))
              :offer_merchant (constantly (test.gen/return "OVERRIDEc"))}]
       (reify ILookup
         (valAt [_ k]
           (println (str "valAt*1: " k))
           (println (str "==>" (get m k)))
           (get m k))

         (valAt [_ k nf]
           (println (str "valAt*2: " k ":" nf))
           (println (str "==>" (get m k nf)))
           (get m k nf))
         )))))
#2019-06-2115:55Alex Miller (Clojure team)there is a known issue with specifying generator overrides on spec aliases#2019-06-2115:55Alex Miller (Clojure team)in that, it doesn't work#2019-06-2115:55Ben Hammondokay. that's a simple explanation#2019-06-2115:55Ben Hammondis there a workaround?#2019-06-2115:56Ben Hammondshould I copy-and-paste it?#2019-06-2115:56Alex Miller (Clojure team)I think it should work if you specify it on the aliased spec :customer.ingestion/merchant_id, but seems like you are?#2019-06-2115:58Alex Miller (Clojure team)I would expect OVERRIDEb and OVERRIDEc to never work here#2019-06-2115:58Ben Hammondso if I ttry to override :db.generators.offers/offer_title instead#2019-06-2115:59Ben Hammondit still doesn'tt work though#2019-06-2115:59Ben Hammond
(:offer_title (test.gen/generate
                (spec/gen :db.generators.offers/offer
                  {:db.generators.offers/offer_title (constantly (test.gen/return "OVERRIDEb"))})))
=> "SC3T9wmvO54Q2TNx4"
`
#2019-06-2116:00Alex Miller (Clojure team)what is test.gen?#2019-06-2116:00Ben Hammond
[clojure.test.check.generators :as test.gen]
#2019-06-2116:01Alex Miller (Clojure team)so test.check.generators expects something different than clojure.spec.gen.alpha - namely, the spec version takes generator thunks, whereas I think test.check just takes generators#2019-06-2116:01Alex Miller (Clojure team)can you try it with spec's version?#2019-06-2116:02Alex Miller (Clojure team)I'm not sure what test.check supports as far as override marking - I'm not sure it even knows about the attributes as that's all spec registry based#2019-06-2116:03Ben Hammondwell it looks like all logic is buried inside clojure.spec.alpha/gensub#2019-06-2116:04Ben Hammondso perhaps I don't understand what you are asking. If I run
(:offer_title (spec.gen/generate
                (spec/gen :db.generators.offers/offer
                  {:db.generators.offers/offer_title (constantly (test.gen/return "OVERRIDEb"))})))
I get the same outcome as when ran test.gen/generate
#2019-06-2116:15Ben Hammondah#2019-06-2116:15Ben Hammondthe good news is that I'm an idiot#2019-06-2116:18Ben Hammondso what I'm doing to sabotage the overridee#2019-06-2116:20Ben HammondThe declaring file goes like this
(ns db.generators.offers
...
(defmacro add-gen
  "update spec to add generator"
  [k g]
  `(spec/def ~k
    (spec/with-gen ~k
      (constantly ~g)) ))

(defmacro update-gen
  [k f]
  `(add-gen ~k (~f (clojure.spec.alpha/gen ~k))))
...
(spec/def ::offer
  (spec/keys
    :req-un [::offer_headline
...
(update-gen ::offer (fn [g] (test.gen/fmap add-msisdn-csv-bytes g)))
...
#2019-06-2116:22Ben Hammondso after the offer spec is declared I go and set its generator using with-gen to mix in an extra bit of functionality#2019-06-2116:22Ben Hammondand the existence of this gfn means that#2019-06-2116:23Ben Hammondclojure.spec.alpha/map-spec-impl sees
(gen* [_ overrides path rmap]
        (if gfn
          (gfn)
          (let [rmap (inck rmap id)
...
#2019-06-2116:23Ben Hammondand says >Ooh a gfn. My work here is done#2019-06-2116:24Ben HammondThankyou for your help#2019-06-2116:24Ben HammondI'm not sure how I'm going to fix it but at least I understand it#2019-06-2116:27Alex Miller (Clojure team)Some of this stuff really needs a rethink, itā€™s pretty tricky#2019-06-2116:27Ben Hammondwell I thought the the next spec counts as a rethink?#2019-06-2116:28Ben Hammonddiscovering where the tripwires are is tricky#2019-06-2116:29Ben Hammondthat's true of my entire time with Clojure#2019-06-2116:29Ben Hammondyou have to trip 'em to find 'em#2019-06-2116:29Alex Miller (Clojure team)Yeah, we may get to it in spec 2#2019-06-2116:32Ben Hammondright its the weekend in my timezone#2019-06-2116:32Ben Hammondhave a goodd weekend Alex#2019-06-2116:50Alex Miller (Clojure team)later#2019-06-2118:54mishawhat is the difference between: (s/coll-of ... :kind vector?) and (s/coll-of ... :into []) ?#2019-06-2119:32Alex Miller (Clojure team)kind adds a validation on the input#2019-06-2119:32Alex Miller (Clojure team)into is about gen and conform (output)#2019-06-2119:33Alex Miller (Clojure team)(but kind also impacts the starting coll for gen/conform)#2019-06-2119:42mishaso :into [] is "list/seq is fine, but generate me a vector"?#2019-06-2120:03Alex Miller (Clojure team)Yes#2019-06-2120:03Alex Miller (Clojure team)And conform to vector#2019-06-2119:44mishathanks. had an impression, that :into is a younger replacement for :kind.#2019-06-2120:02Alex Miller (Clojure team)No, they are different purposes#2019-06-2120:35mishawhat is the best :ret spec for "predicaty" function? boolean? is insufficient because nil is falsey. and because everything but false/nil - is truthy. any?#2019-06-2120:50Alex Miller (Clojure team)Depends on what the function returns#2019-06-2120:50Alex Miller (Clojure team)Spec the truth#2019-06-2120:50Alex Miller (Clojure team)(s/nilable boolean?) is useful sometimes#2019-06-2120:51Alex Miller (Clojure team)Donā€™t use sets for logically false values#2019-06-2120:55mishafunction is supplied by user, and is used as predicate, but return value is not compared to true or false.#2019-06-2120:56mishalimiting it to :ret boolean? or :ret (s/nilable boolean?) would be... limiting#2019-06-2120:58mishawhat is the best way to spec atom fn arg? #(instance clojure.lang.Atom %)? any??#2019-06-2121:00mishait actually will be an atom, not just something dereferable. and fn might call swap! or reset! on this arg#2019-06-2121:49Alex Miller (Clojure team)on the boolean question, I'd just not spec it then#2019-06-2121:49Alex Miller (Clojure team)not spec the ret at all#2019-06-2121:50Alex Miller (Clojure team)nothing to check#2019-06-2121:50Alex Miller (Clojure team)on atom, I'd do the first one#2019-06-2121:50Alex Miller (Clojure team)or actually, use IAtom, the interface, not the concrete class#2019-06-2221:54carkhi have this spec : (s/cat :tag ...... :children (s/* ::child) I have a special case where i want a single child, but i still want it to be conformed as a list of one item.... how would i express this ?#2019-06-2221:56carkhi have the dispatching ok,, been using conformer and it works, but it swallows the explaining of errors down the tree#2019-06-2221:57carkhso what i would want is a replacement for s/* i guess#2019-06-2222:57seancorfields/+ ā€” one or more.#2019-06-2222:59seancorfieldMaybe wrapped in s/and to check the count is 1?#2019-06-2222:59seancorfieldNot sure how s/and combines with the sequence regex stuff thoā€™ā€¦ Iā€™d have to experiment.#2019-06-2223:03carkhoh i can try that thanks#2019-06-2223:11carkhnot working for me but that was a good try =)#2019-06-2223:12carkhi'm really trying to use spec as a parser for sexp, but it's not actually one, i guess the limitations are due to the gen part#2019-06-2223:17carkhThis works, but the real thing is not doing integers, and the tree may be deep. so finding a syntax error deep down might be hard without proper explaining from spec#2019-06-2223:28carkhahhaaa ! this might work#2019-06-2223:32carkhand i get nice errors deep down the tree, nice#2019-06-2323:40misha@carkh
(s/conform
  (s/cat :tag keyword? :children (s/& (s/+ integer?) #(-> % count (= 1))))
  [:a 3])
=> {:tag :a, :children [3]}
#2019-06-2711:29Jakub HolĆ½ (HolyJak)Thanks! I took the liberty of adding the example to clojuredocs http://clojuredocs.org/clojure.spec.alpha/&amp;#example-5d14a98fe4b0ca44402ef771#2019-06-2323:41mishahttps://clojure.github.io/spec.alpha/clojure.spec.alpha-api.html#clojure.spec.alpha/&amp;#2019-06-2323:41carkhgonna have to test this before turning out, thanks#2019-06-2323:43carkhyes that's exactly it, that combinator flew under my radar, thanks @misha#2019-06-2501:07miguelbHi everyone, Iā€™m trying to to write a spect that has ā€œrelationshipsā€ to other specs. (apologies for the language, Iā€™m still new to spec). For example, a group has a total number of people, number of active people and number of inactive people. active-people + inactive-people = total num of people. I can write a spec for the total number of people but how do I use that spec for in the spec for both inactive and active. Also how does that work with generators? Ultimately I would like to generate a group where total, active and inactive are set.#2019-06-2501:18seancorfield@miguelb Are these specs all used together in a map?#2019-06-2501:19miguelbyea eventually, right now iā€™m defining each with their own s/def#2019-06-2501:20miguelbnot sure if this is the right way to go about it, define each part and then compose together#2019-06-2501:20seancorfield(s/and (s/keys :req [::active-people ::inactive-people ::total-people]) #(= (::total-people %) (+ (::active-people %) (::inactive-people %))))#2019-06-2501:20seancorfieldYou can only apply relationship predicates to something that contains all the various related keys.#2019-06-2501:21seancorfieldHowever, I would question the model design: since that's an invariant that should always hold, you don't need all three values (and probably should not try to have all three). Any two gives you all the information you need.#2019-06-2501:22miguelbgood point#2019-06-2501:22miguelbmy end goal here is to make a generator that will generate groups#2019-06-2501:22miguelbI was aiming to have the person count part of a ā€œvalidā€ group#2019-06-2501:22ghadiyou'll end up needing to manually control the generator#2019-06-2501:23ghadithe default spec generators will not find something that satisfies the invariant within 100 tries#2019-06-2501:23ghadiunless you're feeling really really lucky#2019-06-2501:23miguelbheh#2019-06-2501:23seancorfieldYes, that's another reason that I think it would be easier without the constrained total-people number#2019-06-2501:24seancorfieldIf you just have active and inactive, then regular generators will work#2019-06-2501:24seancorfield(since there's no need for an additional constraint)#2019-06-2501:24miguelbIā€™ll try that way, makes way more sense that what I was about to try#2019-06-2501:24miguelbty very much!#2019-06-2608:14kwrooijenHi, when instrumentation is enabled and I call a function with from arguments within a def, instrumentation doesnā€™t trigger an error. Whatā€™s the reason for this? Hereā€™s an example:#2019-06-2608:14kwrooijen#2019-06-2608:15kwrooijenWhen running inc-and-format, when itā€™s defined as a function, instrumentation crashes When running inc-and-format, when itā€™s defined as a def, instrumentation doesnā€™t crash#2019-06-2711:39Jakub HolĆ½ (HolyJak)Is it possible to "and the rest of the arguments should match this others spec"? Example:
(s/def ::kid (s/cat :name string? :age int?))
(s/def ::input-explicit-username (s/cat
                                   :username uuid?
                                   :kid ::kid))
(s/def ::input (s/or :implicit-user ::kid, :explicit-user ::input-explicit-username))
The thing is that I either have a short input or something+the short input. How to spec this without copy&paste? Thank you! Answer: s/cat do not nest, they become one so the code above actually works as desired:
(s/conform ::input ["name" 12])
=> [:implicit-user {:name "name", :age 12}]
(s/conform ::input [(java.util.UUID/randomUUID) "name" 12])
=> [:explicit-user {:username #uuid"c19bc1f5-033f-4ca1-b2e0-5877657cfa8d", :kid {:name "name", :age 12}}]
#2019-06-2711:48jaihindhreddyyup#2019-06-2712:08mishaIf you want some of regex specs to match nested structure, wrap them in s/spec, @holyjak #2019-06-2713:26Jakub HolĆ½ (HolyJak)thx! I wouldn't figure that one out šŸ™‚#2019-06-2712:16misha(s/+ int?) matches 1 2 3 (s/spec (s/+ int?)) matches (1 2 3) s/def wraps form in s/spec, so from (s/def ::x (s/+ int?)) it might appear that (s/+ int?) matches (1 2 3) (with parens). It does not.#2019-06-2712:28Alex Miller (Clojure team)I wouldn't say it that way#2019-06-2712:28Alex Miller (Clojure team)
user=> (s/valid? (s/+ int?) [1 2 3])
true
user=> (s/valid? (s/spec (s/+ int?)) [1 2 3])
true
#2019-06-2712:29Alex Miller (Clojure team)the more accurate way to look at it is that (s/+ int?) is a regex op (which is not actually a spec)#2019-06-2712:29Alex Miller (Clojure team)regex ops combine#2019-06-2712:29Alex Miller (Clojure team)when used, regex ops are automatically "spec-ized"#2019-06-2712:30Alex Miller (Clojure team)s/spec is an explicit "spec-ize" step#2019-06-2712:30Alex Miller (Clojure team)in all cases, regex ops are matching the elements in a sequential collection though#2019-06-2714:26mishaThatā€™s what I meant kappa #2019-06-2714:30Alex Miller (Clojure team)I realize that, it's just the wrong mental model#2019-06-2716:43jeroenvandijkIs it a good practise to use keyword inheritance for multi-spec dispatch? e.g.
(defmulti example-config :entity/type)

(s/def :entity/type #(contains? (set (methods example-config)) %))

(defmethod example-config :default [_]
  (s/keys :req [:entity/type]))

(s/def :entity/config (s/multi-spec example-config :entity/type))

(defmethod example-config :entity.type/generic [x]
  (getx x :entity/type))


;; Use keyword inheritance to get to the right spec
(derive :entity.type/specific :entity.type/generic)

(s/def :entity.type/specific integer?)

(s/explain-data :entity/config {:entity/type :entity.type/spfecific})
#2019-06-2717:56Alex Miller (Clojure team)sounds fine#2019-06-2717:57Alex Miller (Clojure team)not sure if there are any gotchas in gen, but might be#2019-06-2718:33jeroenvandijkThanks, I'll give it a go. Removes some boiler plate code#2019-06-2810:27Daniel Hines(reposted from #clojure) I've got a problem I think spec would be perfect for, but I'm a total spec noob and have no idea how to spec it. I'm using pseudo set notation to describe the problem (read "m E S" as "m is a member of set S"). --- Given: There's a set of types, T, There's a set of attributes, A, There's a set of specs S There's a map such that every attribute has a corresponding spec from S. What is the spec for 2-tuples where: - The first element R1 is a vector of where every element e E T. - The second element R2 is a set of tuples in the form [e a v], where e E {1..length of R1}, a E A, and the value of v conforms to the spec corresponding to a. Any help is appreciated!#2019-06-2813:02djtangoso the first spec seems easy enough:
(s/def ::T #{:t1 :t2 ,,,})
You could map A and S manually by doing:
(s/def ::attribute-that-is-a-number number?
,,,
If you need to make the set of A to be concrete you'd probably want to look up the registry Then the final spec for your 2-tuple probably involves quite a hairy predicate (remember that all specs are 1-arg preds) This could then check that every first-element of the R2 is is a member of the set of ints from 1 up to (count R1) Then a spec that looks up value of a in R2 and does (s/valid? a v)
#2019-06-2813:02djtangohopefully that's enough to get the brain flowing#2019-06-2813:08Daniel HinesLol, why did I have to pick this as my first venture into specs šŸ˜›. Letā€™s try making it more concrete.
(s/def ::T #{:t1 :t2})
(s/def ::height number?)
(s/def ::name string?)
(s/def ::A #{::height ::name})
(s/def ::Goal (s/cat :R1 int? :R2 ???))
#2019-06-2813:10Daniel Hinesā€¦ Thatā€™s all I got so far! Can I get a hint on the rest of the syntax?#2019-06-2813:16Daniel HinesMaybe
(s/def ::Goal (s/cat :R1 int? :R2 (s/* (s/cat :e (set (range (count R1))) :a ??? v: ???)))
#2019-06-2814:35djtangoR1 is a vector right? not an int#2019-06-2814:37djtangoif you don't think about ::Goal as a spec but more a function that takes the whole 2-tuple what would that function look like#2019-06-2815:16Daniel HinesYeah, youā€™re right, :R1 is more like (s/col-of ?int).#2019-06-2815:17droneand ::R2 is (s/coll-of ::R2-elem :kind set?)#2019-06-2815:24Daniel HinesYeah, technically, t#2019-06-2815:24Daniel HinesThereā€™s more to it though, right @UCF779XFS? I need some predicate function that @U0HJD63RN was alluding to.#2019-06-2815:26droneyouā€™d need to a predicate that will take the R2/a and verify the R2/v passes s/valid?#2019-06-2815:30droneso youā€™d have something like:
(defn valid-val?
  [[idx a v]]
  (s/valid? a v)) 

(s/and (s/cat _ _ _ _ _ _)
             valid-val?)
#2019-06-2815:30dronesomething like that/ where the empty cat is whatever youā€™re using for the three values in the R2 element#2019-06-2815:30Daniel HinesOk.. this is starting to come together!#2019-06-2815:31dronetake with a grain of salt, just woke up and may be stupid#2019-06-2816:44djtangowell from the description of the specification - R2's permissible values for e depends on R1#2019-06-2816:44djtangoso you need the whole 2-tuple#2019-06-2816:45djtangoso you would want a predicate that can compare R1 and R2#2019-06-2816:45Daniel HinesYeah, that's right.#2019-06-2816:46djtango
(let [[r1 r2] tup
     allowed-e? (->> r1 count inc (range 1) (into #{}))]
    (->> r2 (map first) (every? allowed-e?))
something like this maybe
#2019-06-2817:01Daniel HinesIā€™ll give that a shot later, thanks!#2019-06-2817:23djtangoso I guess you could split the spec for your tuple into three checks: That R1 is valid all e are valid wrt R1 all v conform to their corresponding a#2019-06-2817:25Daniel HinesYes! That sounds very doable#2019-06-2817:26Daniel HinesI was hoping to get generators for free, but even besides this, there are other constraints that between entities of different types that are much too complicated without some constraint solver.#2019-06-2812:08salokristianI'm working on creating a spec-based validator for creating human-readable error messages. I have a working implementation which locates errors based on the in key in the returned spec errors for explain-data. This works fine for errors concerning leaf-level specs, i.e. entries in a collection. However, I have not yet found a way to create useful error messages for custom predicates that concern a whole collection. Is there a (built-in or custom) way to return some sort of metadata from a custom spec predicate, which could be accessed in the spec error message? For example, the error message for this spec is not useful for locating the error, i.e. the duplicate values.
(s/def ::num int?)
(s/def ::list (s/and (s/coll-of ::num) #(apply distinct? %)))
(s/explain-data ::list [1 2 3 1])
I would like to be able to return some custom metadata from (apply distinct? %), which would tell me which elements caused the error, and allow to locate the errors precisely. Of course this is not just an error with this particular custom predicate, but all custom predicates. There doesn't seem to be a way to add metadata to the error messages generated by spec. Is this doable? If it isn't, what kind of a workaround would you suggest for this? I would like to use spec for checking correctness and generate errors based on spec errors.
#2019-06-2812:25jumarhow does it compare to https://github.com/alexanderkiel/phrase ?#2019-06-2815:33yuhanor https://github.com/bhb/expound#2019-06-2906:07salokristianI'm actually using phrase to generate end-user friendly errors. The problem is passing additional error-related metadata from custom spec predicates to spec errors (and therefore to phrase).#2019-07-0109:16djtangoI'm not convinced you can get helpful error messages from the general form of (s/def ::the-name #(apply distinct? %)) - if you consider that the spec is a one-arg predicate function, it would be hard to see how to determine which entry caused the failure. For the specific case of duplication, you'd probably need to replay the data or reconstruct it - as spec would only recall the input unless you walk it element by element. Even the inbuilt :distinct field in (s/every ...) doesn't seem to know which specific member breaks uniqueness#2019-06-2815:42Daniel HinesCan test.check reverse regex specs out of the box?#2019-06-2815:47nwjsmithNo, youā€™ll need test.chuck for that#2019-06-2815:47nwjsmithoh.#2019-06-2815:47nwjsmithregex specs#2019-06-2815:47nwjsmithI donā€™t think test.check is spec-aware#2019-06-2815:49Daniel HinesIs there a built in spec for UUIDā€™s that works in CLJS?#2019-06-2815:49Daniel HinesJust need the strings.#2019-06-2815:56nwjsmithDo you need a spec for string-encoded UUIDs, or would uuid? work?#2019-06-2816:47Daniel HinesThe former.#2019-06-2817:43dronethere was a whole thread about that somewhere. a Google search would probably find it#2019-06-2817:47dronefor reference: https://clojure.atlassian.net/browse/CLJ-1925#2019-06-2817:44droneuuid across JVM and JS#2019-06-2817:45nwjsmithI think this will work:
(s/def ::uuid-string
  (s/with-gen
    #(re-find #"[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}" %)
    #(gen/fmap str (gen/uuid))))
#2019-06-2817:46nwjsmithOn both platforms#2019-06-2817:50Joe LaneDoes that handle all UUID versions?#2019-06-2817:53nwjsmithIt should, but IIRC test.check.generators/uuid will only generate version 4 UUIDs#2019-06-2817:54nwjsmithThe textual representation of UUIDs is the same across all versions#2019-06-2817:56Daniel HinesThanks a bunch!#2019-06-2819:38Daniel HinesAre there any spec power tools for defining large entity maps? Not really sure what Iā€™m looking for - just something more concise if possible.#2019-06-2900:59kennyI'm encountering a strange error when using st/check. I'm getting an IllegalArgumentException with the message Additional data must be non-nil. (full trace here: https://gist.github.com/kennyjwilli/e7fcfb809a771db130a1ff37b3d3cb79). The stacktrace points to this line: https://github.com/clojure/spec.alpha/blob/5228bb75fa10b7b13ea088d84f4d031cce74a969/src/main/clojure/clojure/spec/test/alpha.clj#L280 which clearly has the possibility of passing nil to an ex-info call which would result in this error. Not sure why this is happening in the first place though. It's likely an issue in my code but this exception is not very helpful.#2019-06-2900:59kenny#2019-06-2901:45Alex Miller (Clojure team)A value that is invalid according to its spec should always produce explain data. Would be interested to know the spec/value causing this#2019-06-2918:20Daniel HinesCan anyone offer any advice on how to transform a json object into a proper Clojure object based on a given spec? The final result has a mix of keywords and strings, and Iā€™m not sure how to say ā€œHereā€™s a spec of the end result, and hereā€™s the JSON, now give me the end resultā€#2019-06-2920:26jaihindhreddynot sure how to do it with vanilla spec but Metosin's spec-tools has coercion for this purpose, check out https://github.com/metosin/spec-tools/blob/master/docs/01_coercion.md#2019-06-2920:45Daniel HinesThanks. Speaking of, how do you get data specs to work with test.check and generators? Is that possible?#2019-06-2920:53Daniel HinesFor example, this confuses me:
(s/def ::foo #{:bar})
(s/def ::foo-map (s/keys :req [::foo]))
(gen/generate (s/gen ::foo-map)) ;; => {::foo :bar}
(def data-foo (ds/spec {::foo (s/spec #{:bar})}))
(s/valid? foo-map {::foo :bar}) ;; => true
(gen/generate (s/gen data-foo)) ;; => nil
#2019-06-2920:58jaihindhreddyI've only been watching spec-tools and schema from afar. Sorry#2019-06-2920:58Daniel HinesThanks anyway!#2019-06-2920:58Daniel HinesLooks like itā€™s something to do with this. options include one :gen. https://github.com/metosin/spec-tools/blob/master/docs/07_spec_records.md#2019-06-3001:46Daniel HinesHow is this possible?
(s/conform workspace ws-clj) ;; => :cljs.spec.alpha/invalid
(s/explain-str workspace ws-clj) ;; => "Success!\n"
#2019-06-3001:46Daniel Hiness/valid? also returns false.#2019-06-3001:50Daniel HinesOh, itā€™s because the invocation was erring out šŸ˜•#2019-06-3001:51Daniel HinesFor s/explain-str to return ā€œSuccess!\nā€ when an exception is thrown seems like a bug.#2019-06-3001:52Alex Miller (Clojure team)It is typically. Whatā€™s the spec and the value?#2019-07-0117:18Daniel HinesI tried reproducing it and couldnā€™t figure out exactly what I did (I was working at the REPL). If I run into it again Iā€™ll try to find a solid repro.#2019-07-0115:32gfredericksnew test.check release: https://clojurians.slack.com/archives/C0JKW8K62/p1561995125014300#2019-07-0201:27seancorfieldUpdated our code at work to use 0.10.0-RC1 and to switch away from the deprecated generators -- and we were using pos-int on the assumption that it generated strictly positive numbers (although zero doesn't break our tests).#2019-07-0210:57gfredericksthat makes me glad I deprecated it šŸ™‚ did you switch to (gen/fmap inc gen/nat), or something else?#2019-07-0212:58seancorfieldFor now we just switched to gen/nat to preserve the behavior but we will revisit that test and see what might happen if it ever generates zero. At least it is clear now!#2019-07-0212:59gfredericksI can assure you it generates zero quite often šŸ™‚#2019-07-0213:08seancorfieldThat test should probably fail with zero (but it doesn't) šŸ˜•#2019-07-0213:28gfrederickscomputers are crazypants#2019-07-0116:46seancorfield@gfredericks What's the thinking behind deprecating those generators?#2019-07-0119:29gfrederickspos-int and neg-int are highly misleading w.r.t. 0, and all five of them are misleading w.r.t. size; I think generators by default should generate a healthy range of things, and if you want to restrict yourself to small numbers you should have to be explicit#2019-07-0119:30gfredericksso it's basically entirely a naming issue#2019-07-0119:31gfredericksspose I could've also deprecate-renamed gen/ratio to gen/small-ratio#2019-07-0120:00seancorfieldAre you actually planning to remove them at some point (a breaking change) or just "strongly discourage" their usage?#2019-07-0120:04gfredericksthe latter#2019-07-0120:04gfredericksthere's already another deprecated generator of that sort#2019-07-0120:05seancorfieldI found 15 occurrences of "deprecated" in that file šŸ™‚#2019-07-0120:05seancorfield(seven functions)#2019-07-0120:05gfredericksentirely for naming, apparently:
(def ^{:deprecated "0.6.0"}
  char-alpha-numeric
  "Deprecated - use char-alphanumeric instead.

  Generates alphanumeric characters."
  char-alphanumeric)
#2019-07-0120:06seancorfieldNaming is hard. I struggled with deprecation/breakage in clojure.java.jdbc and ended up breaking the API twice to get it consistent and idiomatic. With hindsight I would have done it differently.#2019-07-0120:19gfredericksyeah I've tried to figure out how to overhaul things so many times, and largely haven't#2019-07-0118:11rapskalianDoes spec have a simple way to express xor with map keys? Meaning map m can have key :a or key :b, but not both.#2019-07-0118:15Alex Miller (Clojure team)Not built in, but you can s/and any predicate you like#2019-07-0118:16akielHas someone a solution for using st/instrument with :stub were the function is called more than once and I like to have return values depending on the inputs? With Java mock frameworks such things are possible. With spec I donā€™t have an idea how to do it. Thanks!#2019-07-0118:18Alex Miller (Clojure team)You canā€™t with stub, but you could make a stateful :replace fn#2019-07-0118:27akiel@alexmiller Work perfect. Thanks! Not as elegant like mock libs but one could build a helper function which maps inputs to outputs.#2019-07-0208:54roklenarcicIs there a way to get around the limitation of s/merge that it only conforms one branch of merge?#2019-07-0208:55roklenarcicThis is really really really detrimental.#2019-07-0407:09Jakub HolĆ½ (HolyJak)Hello! I get some kind of infinite loop in conform after I have renamed and moved around a couple of spec names, any tips how to troubleshoot it? It seems deep-resolve and reg-resolve are involved in the loop as well as a particular spec (`:my.ns/account-invoices`) and if the debugger is correct then we regularly arrive to a point in reg-resolve where both spec and k are null. Update: I can replicate - calling (reg-resolve :my.ns/account-invoices) never returns. (Clojure 1.10.1, spec 0.2.44) This is really weird - all parts of the spec resolve but not the spec itself. I have this:
(s/def ::account-invoices
  (s/nilable
    (s/every-kv ::accid ::acc+invoice)))
and both (reg-resolve! :my.ns/acc+invoice) and (reg-resolve! :my.ns/accid) return a spec successfully yet (reg-resolve! :my.ns/account-invoices) never returns.... If I change the name of the spec to :tmp/account-invoices then (reg-resolve! :tmp/account-invoices) succeeds. SOLVED: This line little more down was the issue (s/def ::account-invoices ::account-invoices) I wish Spec had a loop detection...
#2019-07-0613:23misha@roklenarcic can you give an example, where/how it limits you?#2019-07-0615:02roklenarcicLet's say you define (s/def ::date-time (s/conformer to-date-time-string from-date-time-string)). This conforming spec will print date time object into a string (or vice versa on unform call). If you define several s/keys specs each containing some fields of this type and then you defined a merged spec with s/merge you will end up with having a situation where after conforming an object only a third of date time properties will be conformed to strings the rest will remain datetime objects and it will fail.#2019-07-0618:22mishaThis sounds reasonable, although alexmiller stated many times that conformers are not for (this kind of) data transformation.#2019-07-0618:23mishae.g. https://stackoverflow.com/a/49056441/1128985#2019-07-0618:41misha
(s/def :foo/s-int (s/conformer #(Integer/valueOf ^String %) str))
(s/def :foo/a :foo/s-int)
(s/def :foo/b :foo/s-int)
(s/def :foo/merge
  (s/merge 
    (s/keys :req [:foo/a])
    (s/keys :req [:foo/b])))

(s/conform :foo/s-int "1") ;; => 1
(s/unform :foo/s-int 1) ;;=> "1"

(s/valid? :foo/merge {:foo/a "1"})  ;;=> false
(s/valid? :foo/merge {:foo/a "1" :foo/b "2"}) ;; => true
(s/conform :foo/merge {:foo/a "1" :foo/b "2"})  ;;=> #:foo{:a 1, :b 2}
(s/unform :foo/merge {:foo/a 1 :foo/b 2}) ;;=> #:foo{:a "1", :b "2"}
#2019-07-0715:25roklenarcicIf data transformation is not the intent, then what IS the intent of conforming?#2019-07-0715:29mishaTo tell you how exactly data is valid or invalid. The line is somewhat blurry though.#2019-07-0715:31mishaRead up threads linked in SO answer#2019-07-0613:26mishayou might be able to use s/and, which "flows" conformed values as part of validation:
(s/conform   (s/and (s/or :k keyword? :i int?) vector?)  1)
=> [:i 1]
(s/valid?  (s/and (s/or :k keyword? :i int?) vector?)  1)
=> true
#2019-07-0613:27misha(but intent readability suffers, imo)#2019-07-0617:09Diego Melendezhow do I write a s/fdef for a function that does not receive args?#2019-07-0617:22Jakub HolĆ½ (HolyJak)I guess leave out args or eg (s/coll any? :count 0)?#2019-07-0617:22Jakub HolĆ½ (HolyJak)Or just empty?#2019-07-0618:00Diego MelendezšŸ‘#2019-07-0619:07Alex Miller (Clojure team)I use (s/cat)#2019-07-0808:19StefanHi, Iā€™d like to write a spec for an entry in a map that must always be a specific string, and also can be generated. So I have {:my-key "my constant string"} (as part of a bigger structure). A spec like this works:
(s/def ::my-key (partial = "my constant string"))
Now I try to generate data inside my test, and this fails. Iā€™m trying to use s/with-gen, but I think Iā€™m missing something.
(s/def ::my-key
  (s/with-gen (partial = "complexity")
              (constantly "complexity")))
yields:
Assert failed: Second arg to such-that must be a generator
(generator? gen)
How should I approach this?
#2019-07-0808:34valeraukotry #(constantly "complexity")#2019-07-0808:36yuhanYou could also use a 1-element set:
(s/def ::my-key #{"complexity"})
#2019-07-0808:37aisamuhttps://clojure.github.io/test.check/clojure.test.check.generators.html#var-return#2019-07-0808:37aisamu(although the set approach above is the preferred way)#2019-07-0808:38Stefan@UAEH11THP I tried that too, also gives an error. I got it to work though after some trial and error:
(s/def ::my-key
  (s/with-gen (partial = "complexity")
              #(gen/return "complexity")))
#2019-07-0808:38Stefan@U1UQEM078 Thanks! Why is that preferred?#2019-07-0808:39aisamuBecause then you get the generator for free!#2019-07-0808:39StefanOh wait thatā€™s ALL the code Iā€™d need šŸ™‚#2019-07-0808:39StefanMuch simpler indeed šŸ™‚#2019-07-0808:56valeraukowow every day i learn something new#2019-07-0810:15mishayeah, it is idiomatic to spec constants with a single element set. however, if that constant is either false or nil ā€“ use false? and nil?, because
(s/valid? #{false} false)
=> false
(s/valid? #{nil} nil)
=> false
#2019-07-0810:18Stefan@U051HUZLD Iā€™m probably misunderstanding:
(s/valid? #{false} false)
=> false
(s/valid? #{false?} false)
=> false
(s/valid? #{nil} nil)
=> false
(s/valid? #{nil?} nil)
=> false
#2019-07-0810:26misha
(s/valid? nil? nil)
=> true
#2019-07-0810:26mishanil? false? instead of set#2019-07-0810:34yuhanthat's an edge case to look out for in general when using sets as predicate functions, which is really all that spec is doing here#2019-07-0810:37yuhan
(if (#{1 2 3} x) ;; returns truthy values (1, 2, or 3) if x is in the set
   ...)

 (if (#{false} x) ;; returns a falsey value no matter what x is!
   ...)
#2019-07-0810:43mishait is#2019-07-0810:48mishacan be worked around with contains? though#2019-07-0913:45arohnerI have an s/keys with a key of type uuid?. I have a generative test where I generate a few hundred records, and insert them into postgres, where the uuid uniqueness constraint is checked. Occasionally the test fails due to duplicate uuids. Is there a way to tell spec/test.check to only generate unique uuids?#2019-07-0914:41ghadithere are ways of generating unique data within spec/t.c. @arohner but for this usecase I would dedupe between generation and inserting records#2019-07-0915:04arohnerThanks#2019-07-0915:39kennyIt seems useful to have s/with-gen be passed the generator for the spec it is overriding. Often I find myself generating values from the spec's gen and then fmap'ing to ensure certain constraints are met.#2019-07-0915:56kennyIt'd also be useful to have a variant of s/and that doesn't pass the conformed value to s/and predicates. We have a lot of this in our specs:
(s/and
  (s/keys :req [:metric/tags :metric/lower-bound :metric/upper-bound])
  #(metric-tags-valid? (s/unform ::base-metric %))
  #(metric-bounds-valid? (s/unform ::base-metric %)))
#2019-07-0916:48mishasometimes changing order inside s/and works#2019-07-0916:48kennyThese such-that errors are quite frustrating because they provide no info:
Error printing return value (ExceptionInfo) at clojure.test.check.generators/fn (generators.cljc:417).
Couldn't satisfy such-that predicate after 100 tries.
clojure.lang.ExceptionInfo: null #:clojure.error{:phase :print-eval-result}
	at clojure.main$repl$read_eval_print__9068.invoke(main.clj:419)
clojure.lang.ExceptionInfo: Couldn't satisfy such-that predicate after 100 tries. {:pred #object[clojure.spec.alpha$gensub$fn__1876 0x71807ca2 "
At the very least, including the predicate form would be extremely helpful.
#2019-07-0916:50misha@kenny you can sometimes enhance that by naming your anon fns#2019-07-0916:51mishae.g. (fn f1 [x] (metric-tags-valid? (s/unform ::base-metric x))) instead of #(metric-tags-valid? (s/unform ::base-metric %))#2019-07-0916:52mishait might show up in stacktrace as something like /cdn-cgi/l/email-protection instead of /cdn-cgi/l/email-protection#2019-07-0917:04kennyYeah I suppose I could do that. It'd be great if there was a tighter integration between spec and test.check so I don't need to do that.#2019-07-0921:33kennyclojure.spec.gen.alpha/gen should have up to 3 args (https://clojure.github.io/test.check/clojure.test.check.generators.html#var-generate) when it only has one.#2019-07-1014:23gfredericksThe second two args are new#2019-07-1014:24gfredericks@arohner the uuid generator is fully random with a uniform distribution, so you shouldn't be any more likely to get collisions than you are in Real Life, i.e. it shouldn't ever happen#2019-07-1014:25arohnerWhat about during shrinking?#2019-07-1014:25gfredericksMy guess is you're seeing the effects of shrinking, where your test gets run repeatedly with similar data#2019-07-1014:26gfredericksSo the UUIDs won't shrink, but your test is run again, so if you have state in a database that you're not resetting between trials, that'd be a problem#2019-07-1014:31gfredericksI agree it's annoying, because UUIDs otherwise seem like such an effective way of implicitly partitioning a stateful external system by trial#2019-07-1014:32gfredericksI'm curious whether, in practice, blasting lots of data into a test DB w/o deleting it is faster than truncating between trials#2019-07-1612:52eoliphantHi, Iā€™m trying to figure out how to model this with spec. Say I have (s/def ::rule-type #{:rule1 :rule2}) I then want to create (s/def ::rule1 (s/keys ...), etc where aside from the keys that are selected ::rule-type is ā€˜narrowedā€™ to specific value, such that I can then have a ::rule spec thatā€™s an or of those guys#2019-07-1612:54gfredericksmight be what multi-spec is for?
#2019-07-1612:55Alex Miller (Clojure team)yes, seems like exactly what s/multi-spec is for#2019-07-1613:00eoliphantah ok, actually just got it working with and s/and ā€˜passthruā€™ but will try the mult-spec as well#2019-07-1613:01eoliphantdid this
(s/and (s/keys :req [::rule-type ::rule-id ::rule-action ::object-locator]
                            :opt [::load-order])
                     #(= (::rule-type %) :selection)))
but yeah multi-spec seems a bit more elegant spec2 looks exciting @alexmiller
#2019-07-1613:20jeroenvandijkIt also gives much better feedback (more specific) when using with expound for instance#2019-07-1613:02Alex Miller (Clojure team)s/multi-spec is also open so you can add more rules later#2019-07-1613:04eoliphantsweet. Hey is spec2 going to have fspec support for mutimethods?#2019-07-1613:05Alex Miller (Clojure team)not something we're working on right now, but maybe eventually#2019-07-1613:18eoliphantcool#2019-07-1714:30talgiatIs there a way to reuse fspec in clojurscript? the only way to spec functions in clojurescript seems to use fdef but that canā€™t take fspec defined somewhere else as an argument. My goal is to define a function spec in one place and re-use it for all functions that have the same function spec. This can be done in clojure the following way (I believe):
(def n->n-fn-spec (s/fspec :args (s/cat :n number?) :ret number?))

(s/def add5 n->n-fn-spec)
(defn add5 [n] (+ n 5))

(s/def add6 n->n-fn-spec)
(defn add6 [n] (+ n 6))

#2019-07-1714:35Alex Miller (Clojure team)instead of def, use s/def#2019-07-1714:36Alex Miller (Clojure team)oh, you're trying to reuse the whole fspec as a function's spec#2019-07-1714:38Alex Miller (Clojure team)not sure I know enough about the cljs impl to say if this is possible#2019-07-1714:53talgiatyes thatā€™s what Iā€™m trying to do. But this example should work in clojure (I think without s/def in the first expression)#2019-07-1715:18Alex Miller (Clojure team)it won't work exactly like this in spec 2 in Clojure#2019-07-1715:19Alex Miller (Clojure team)but there is a similar path#2019-07-1715:20talgiatI tested and it works both with def and s/def in this example (but with s/def it should be ::n->n-fn-spec ā€¦)#2019-07-1715:21talgiatLooking at clojurescript fdef code it adds the fn-sym to specedvars atom that is used with instrument.#2019-07-1715:22talgiatso you canā€™t use s/def in clojurescript to spec function symbols (at least I canā€™t get it to work)#2019-07-1715:23Alex Miller (Clojure team)the fact that it even works in Clojure is somewhat accidental#2019-07-1715:49talgiatyou mean with def or even with s/def, or you mean reusing fspec in general?#2019-07-1715:50Alex Miller (Clojure team)being able to use s/def with function spec objects#2019-07-1715:51Alex Miller (Clojure team)that won't work in spec 2, but there is a new function s/register that will be available to do the equivalent#2019-07-1715:51talgiatwill it work with def (not s/def)?#2019-07-1715:52Alex Miller (Clojure team)that's really incidental - def doesn't do anything with the registry so no change there#2019-07-1715:52Alex Miller (Clojure team)the changes are around what s/def does#2019-07-1715:52talgiatyeah def just binds a symbol to fspec#2019-07-1715:53Alex Miller (Clojure team)importantly, an fspec object (not a form)#2019-07-1715:53Alex Miller (Clojure team)s/def doesn't work with objects anymore in spec 2, only forms#2019-07-1715:53talgiatright#2019-07-1715:54Alex Miller (Clojure team)but s/register is a new function in spec 2 that s/def (macro) uses#2019-07-1715:55talgiatI guess my question is, is reusing fspec is a valid use case for spec? or what is the correct way to have reusable function spec?#2019-07-1715:55talgiatthat works in clojure and clojurescript#2019-07-1715:58Alex Miller (Clojure team)I can't really answer the latter, but in spec 2 it's possible by registering the same object via s/register. cljs spec 2 work has not been started so tbd#2019-07-1716:02talgiatThanks Alex, Iā€™ll chase David for clojurescript#2019-07-1716:02Alex Miller (Clojure team)I think they're going to wait until we're much further along with spec 2 before they start working on it#2019-07-1716:03talgiatIt will be great if with spec 2 things will work the same way in Clojure and Clojurescript#2019-07-1716:04Alex Miller (Clojure team)well that's certainly the intent (other than things that can't work the same way)#2019-07-2312:43henryw374I want to have generators associated with predicates, like you have already with the clojure.core preds. Are people re-binding gen.alpha/gen-builtins or is there some other way?#2019-07-2313:12Alex Miller (Clojure team)make a spec with a custom gen#2019-07-2313:31henryw374but with string?, I can use that both as a normal pred, and as a spec with a generator. If I make a spec with the pred, that's two separate things isn't it?#2019-07-2313:34Alex Miller (Clojure team)yes, but you can just use the spec#2019-07-2313:37henryw374ok, so I'm hearing the core pred fns are a bit special wrt specs. I'm creating a library and really it'd be nice for the preds to be used in the same way as the core fns. https://github.com/henryw374/time-specs/blob/master/src/time_specs/core.cljc#2019-07-2313:38Alex Miller (Clojure team)the recommended way to do this with spec 1 is to make a spec with a custom generator#2019-07-2313:39Alex Miller (Clojure team)future answer may be different#2019-07-2313:39henryw374ok thanks for the info.#2019-07-2313:40Alex Miller (Clojure team)rebinding gen-builtins from a lib is problematic - users have to do that for it to work, and whatever you do has to be compatible with whatever some other lib might do#2019-07-2313:40henryw374yeah... I'll avoid that I think. bound to be problems#2019-07-2313:40henryw374no pun intended#2019-07-2313:41Alex Miller (Clojure team)heh#2019-07-2313:41Alex Miller (Clojure team)spec 2 has some capabilities to create your own derived spec types with custom gens which is maybe another option#2019-07-2313:42Alex Miller (Clojure team)https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha#simple-spec-ops#2019-07-2313:43Alex Miller (Clojure team)but that's really trying to handle the case of parameterized things more so than pred generators#2019-07-2313:45Alex Miller (Clojure team)the example at that link doesn't show how to define a custom gen in a defop, but it has that capability#2019-07-2313:54henryw374ok cool. I'll have a look at that with an eye to the future. on the topic of spec2, with select, say I have a schema with a key a the spec of which is a coll-of x or map-of string? x and x is a schema with keys foo, bar. could you do a select so say you want a and in the value of a you want to select key foo from x? I guess not from what I've read so far, but I think that'd be useful.#2019-07-2314:01Alex Miller (Clojure team)not there now, but we have been thinking about how to add it#2019-07-2314:01Alex Miller (Clojure team)definitely a common case#2019-07-2314:01henryw374cool thanks, looking forward to spec2 definitely#2019-07-2322:03robertfwWhat is the recommended way to spec a map that uses strings for keys?#2019-07-2322:05Alex Miller (Clojure team)it's a little tricky right now#2019-07-2322:05Alex Miller (Clojure team)you can spec it as an s/map-of s/tuple for map entries#2019-07-2322:05Alex Miller (Clojure team)possible to s/multi-spec the tuple on the key to choose the val spec#2019-07-2322:32robertfwok, thanks#2019-07-2322:32robertfwWill that become easier with spec2?#2019-07-2401:23Alex Miller (Clojure team)not sure#2019-07-2406:40y.khmelevskiiHi gents! Is there any way to use spec/keys definition for creating new spec as set of keys? For example I have spec:
(s/def ::schema
  (s/keys :req-un [::id
                   ::name
                   ::active?]
               :opt-un [::age]))
I want to create other spec based on ::schema
(s/def ::fields
  #{:id :name :active? :age})
#2019-07-2412:53danierouxHow would I generate a random walk with clojure.spec.gen? Or, how would I generate a successive value based on the current value?#2019-07-2414:08respatializedI think the easiest way would be to use spec.gen for starting values and then use iterate to do the actual random walks. you could use specs inside the fn you pass to iterate to constrain the return values.#2019-07-2414:10respatializeddoes anyone have any thoughts on making spec predicates usable through a java API? should I just use spec/conform to define the class methods I want to expose to the Java API?#2019-07-2414:18Alex Miller (Clojure team)can you explain more what you're trying to do?#2019-07-2414:24respatializedI'm using core.spec to define a series of data integrity/quality predicates that some data needs to conform to. I would like for those specs to be available as widely as possible, including for other consumers of the data who may be using Java instead of Clojure.#2019-07-2504:01jjttjjis there an example out there of a spec on arguments that look like
([a])
([a b])
([a b c])
([a b c d])
([a b c d e])
where later arguments are optional and can be omitted and replaced with defaults? I feel like there has to be a better way than just a big s/alt with a bunch of s/cats but I can't think of it right now
#2019-07-2505:22Alex Miller (Clojure team)(s/cat :a ::a (s/? :b ::b (s/? :c ::c (s/? :d ::d (s/? :e ::e)))))#2019-07-2505:23Alex Miller (Clojure team)but also, sometimes s/alt is clearer, just depends#2019-07-2505:28seancorfieldI didn't realize you could cascade s/? like that... I've only seen (s/cat :a ::a :b (s/? ::b)) before I think?#2019-07-2505:30seancorfieldHmm, according to the docs/source, s/? can only take a single pred-form @alexmiller#2019-07-2505:43maxphow to write spec that conforms input value to positive int or to zero when value missing?#2019-07-2505:44maxplike that
(s/def ::offset
  (s/or
    :nil (s/and nil? (s/conformer (constantly 0)))
    :int (s/and int? #(<= 0 %))))
#2019-07-2505:45maxpbut it returns vector (s/conform ::offset nil) [:nil 0]#2019-07-2506:03Alex Miller (Clojure team)@seancorfield sorry, tired#2019-07-2506:03Alex Miller (Clojure team)should be s/cats in there#2019-07-2506:03Alex Miller (Clojure team)(s/cat :a ::a :an (s/? (s/cat :b ::b :bn (s/? (s/cat ...#2019-07-2506:04Alex Miller (Clojure team)@maxp imo, you should not do that with specs#2019-07-2506:06Alex Miller (Clojure team)s/or always conforms to a tagged result where the tag matches the branch that was taken#2019-07-2506:07Alex Miller (Clojure team)but general data transformation is best applied outside spec, not as part of conforming anyways#2019-07-2506:08maxpthank you#2019-07-2507:21maxpis it possible to "attach" some human readable message to the spec to get it in (explain-data) ?#2019-07-2507:23carkh@maxp check the expound library#2019-07-2507:23carkhhttps://github.com/bhb/expound#2019-07-2507:24carkhnot exactly what you're asking but it helps#2019-07-2507:25maxpgreat!#2019-07-2515:55jjttjjIs there a way to get the resolved value for a given spec? I have a set I generate as a spec and I'm wondering if there's a way to access that set from the spec api:
(s/def ::sec-type (set (map str (Types$SecType/values))))
(s/describe ::sec-type) ;;=> (set (map str (values))) ;can't eval this
#2019-07-2515:58seancorfield@jjttjj Try s/form instead.#2019-07-2515:58seancorfields/describe returns an "abbreviated" version.#2019-07-2515:58jjttjjperfect, thanks again sean#2019-07-2519:17y.khmelevskiihey, can anybody help me with it? https://clojurians.slack.com/archives/C1B1BB2Q3/p1563950441028500#2019-07-2616:00kennyAny good tools for debugging slow generators? We have lots of specs with tens of s/and preds & custom gens associated. I'd like to know which predicate is causing a such-that to fail often.#2019-07-2616:05gfredericksI bet you could alter-var-root such-that to default to a lower max-tries#2019-07-2616:05gfrederickslooks like spec just uses the default (10) as far as I can tell#2019-07-2616:05gfredericksso if you change that to, e.g., 2, you might get more failures#2019-07-2616:06gfredericksbut maybe that's not useful if the "telling you which s/and failed" feature isn't wired up yet#2019-07-2616:07kenny> isn't wired up yet Is a feature like this on the roadmap?#2019-07-2616:08kennyGood idea on the such-that max-tries thing. That should help a bit. Though, the such-that errors are usually pretty obtuse, hiding the actual name of the predicate behind an anonymously named test.check pred.#2019-07-2616:09gfredericksspec's roadmap, yes, as far as I know; that feature couldn't exist prior to the 0.10.0 versions of test.check#2019-07-2616:09gfredericks> the such-that errors are usually pretty obtuse that's exactly the problem that the aforementioned up-wiring would address#2019-07-2616:11kennyAh, very cool. Debugging these issues has been a huge time sink for us. It seems like it should be a fairly easy change to spec. What feature in 0.10.0 allows this better up-wiring?#2019-07-2616:16gfredericksfrom the such-that docstring:
You can customize `such-that` by passing an optional third argument, which can
  either be an integer representing the maximum number of times test.check
  will try to generate a value matching the predicate, or a map:

      :max-tries  positive integer, the maximum number of tries (default 10)
      :ex-fn      a function of one arg that will be called if test.check cannot
                  generate a matching value; it will be passed a map with `:gen`,
                  `:pred`, and `:max-tries` and should return an exception
#2019-07-2616:23kennyI see. Would it be easy to hack this in to spec as a temporary fix?#2019-07-2616:24gfredericksI have no idea, @U064X3EF3 probably knows at least the easy vs hard part#2019-07-2616:27Alex Miller (Clojure team)probably not easy?#2019-07-2616:28Alex Miller (Clojure team)well, not sure#2019-07-2616:28gfredericksI'd think you'd want have some kind of path on hand at the point where such-that is called#2019-07-2616:29gfredericksso might take some wiring to make that path available#2019-07-2616:29gfredericksor maybe it's already there, since spec likes to put paths in its error messages#2019-07-2616:29kennyThis would be a huge value add for us. If adding/hacking this in is a day's worth of work, it's definitely worth it for us to dive in#2019-07-2616:29Alex Miller (Clojure team)it should be - gens have the path#2019-07-2616:30Alex Miller (Clojure team)but they aren't using such-that right now#2019-07-2616:30Alex Miller (Clojure team)so I'm not sure how this stuff connects#2019-07-2616:30gfredericksthey? this is about s/and in particular I think#2019-07-2616:31gfredericksI can't imagine the gen for s/and doesn't use such-that#2019-07-2616:31Alex Miller (Clojure team)I guess it does, just not directly#2019-07-2616:32Alex Miller (Clojure team)that's generically in gensub, which handles the recursion check stuff#2019-07-2616:33kennyIt appears there's only one place that such-that is used in spec, gensub as you say. It also is passed other info that seems valuable to report back.#2019-07-2616:36Alex Miller (Clojure team)are you using deps.edn ?#2019-07-2616:36kennyYes#2019-07-2616:36Alex Miller (Clojure team)if so I could commit something on a branch and you could test it as a git dep#2019-07-2616:36kennySure#2019-07-2616:37kennyI currently don't have a repro of one of these such-that issues though, just the slow gen thing. The such-that errors are very frequent though#2019-07-2616:49Alex Miller (Clojure team)easy to repro with any restrictive s/and pred#2019-07-2616:49Alex Miller (Clojure team)my first hack did not yield anything useful though#2019-07-2616:49Alex Miller (Clojure team)I need to step away for a bit, but may play with it later#2019-07-2616:50kennyOk. Let me know if I can help in any way.#2019-07-2616:13Alex Miller (Clojure team)@kenny 95% of the time the issue with slow generators is generating large (and particularly nested large) collections#2019-07-2616:14Alex Miller (Clojure team)setting :gen-max 3 in s/coll-of, s/map-of etc is often a big difference#2019-07-2616:14kennyThis structure is highly nested & has lots of colls. We have these opts set:
:coll-check-limit 10
:coll-error-limit 10
:fspec-iterations 10
:recursion-limit  1
#2019-07-2616:15Alex Miller (Clojure team)none of those affect the gen'ed count#2019-07-2616:16Alex Miller (Clojure team)with a 3 level coll, you could have 10x10x10 nested elements#2019-07-2616:16kennyOh, right - that param is on the actual generator. Lemme see if we have those set.#2019-07-2616:16Alex Miller (Clojure team)well, recursion limit might affect the depth (although I think there are some scenarios where it is not getting applied right now)#2019-07-2616:18Alex Miller (Clojure team)if you have fspecs, that's another possible issue where you might want to consider simpler preds#2019-07-2616:18Alex Miller (Clojure team)like ifn?#2019-07-2616:19kennyWe have considered that a number of times simply due to the massive amount of time debugging generators take. Every time, however, we always decide not to because those fspec generators catch valuable bugs haha#2019-07-2616:19gfredericksthe eternal test.check conundrum#2019-07-2616:19Alex Miller (Clojure team)another testing tip is that gens will get run 1000 times in check and size/complexity increases#2019-07-2616:20Alex Miller (Clojure team)so gen/sample by default (20) will not expose generator issues#2019-07-2616:20Alex Miller (Clojure team)but you can gen/sample with 1000 and will see those#2019-07-2616:20Alex Miller (Clojure team)if you separate the args spec for an fdef into its own spec, it's easy to test that#2019-07-2616:21Alex Miller (Clojure team)(s/fdef foo :args ::args-spec), then (gen/sample (s/gen ::args-spec) 1000)#2019-07-2616:22kennyI searched for :gen-max in this service and didn't find any so it may not be used. Though, there are tons of custom gens which use the gen/* functions which don't take in :gen-max, I think. So that's the usual way to debug the slow gens -- something like this (do (doall (gen/sample (s/gen ::db/db-control-args) 1000)) nil). Then that will hang forever and that's where the pain starts.#2019-07-2616:23Alex Miller (Clojure team)I would try just blindly adding :gen-max 3 to all the collection specs#2019-07-2616:23gfredericks@kenny what version of test.check are you using?#2019-07-2616:23Alex Miller (Clojure team)there is a ticket specifically for changing this default (or exposing a global default) btw#2019-07-2616:26Alex Miller (Clojure team)which I am completely failing to find#2019-07-2616:26Alex Miller (Clojure team)but I wrote the patch for it, so I know it exists#2019-07-2616:43kennyThe global default would still respect coll-of :min-count, right?#2019-07-2617:12Alex Miller (Clojure team)there's actually a bug report about that#2019-07-2617:12Alex Miller (Clojure team)https://clojure.atlassian.net/browse/CLJ-2202#2019-07-2617:13Alex Miller (Clojure team)so be careful on that#2019-07-2616:24kennyWill try that. We're using 0.10.0-alpha3.#2019-07-2616:24kennyYeah, that would help to figure out if the problem is collection gen or something else.#2019-07-2616:25gfredericksokay, that should avoid some exacerbatory issues with the 0.9.0 release#2019-07-2616:25kennyHaha, yeah. I remember a huge perf bump when switching from 0.9.0#2019-07-2616:25gfredericksoh good#2019-07-2616:47kennyAnother thing that slows down debugging slow gens is that the check tests still run in the background even though I interrupted the current op in the REPL.#2019-07-2616:47kennyI often will need to restart the REPL to get it to stop.#2019-07-2616:47gfredericksis that a pmap thing?#2019-07-2616:47kennyWe're not using that. Does test.check?#2019-07-2616:50gfredericksI think spec does#2019-07-2616:50gfredericksbut could be wrong#2019-07-2616:56kennypmap is in the check impl.#2019-07-2616:57kennyMaybe a call to shutdown-agents would help?#2019-07-2617:00kennyGetting a frequencies map returned after generating a spec with s/and would also be super helpful. The keys would be the forms of each s/and pred.#2019-07-2617:10Alex Miller (Clojure team)well shutdown-agents will shutdown the pool never to restart again, so you'd still need to bounce your repl#2019-07-2617:11Alex Miller (Clojure team)pmap is lazily parallel#2019-07-2617:11Alex Miller (Clojure team)so when you stop using the result, up to n-1 (where n is number of processors) may still be working to compute the next few results in the lazy seq#2019-07-2617:16kennyHmm. It seems to go on forever. Iā€™ll add prints to some functions to debug the slow gens and it never stops printing after interrupting. #2019-07-2617:24kennyPerhaps because of the doall in this line? (do (doall (gen/sample (s/gen ::db/db-control-args) 100)) nil)#2019-07-2617:52kennyThis is helpful :
(let [such-that gen/such-that]
  (with-redefs [gen/such-that (fn
                                ([pred gen]
                                 (such-that pred gen 1))
                                ([pred gen opts]
                                 (such-that pred gen 1)))]
    (gen/generate (s/gen ::db-control-args))))
#2019-07-2617:54kennyActually, @gfredericks the problem with taking the above approach is simple generators sometimes need the such-that. e.g.
(s/def ::int+-interval
  (s/and (s/tuple ::m/int+ ::m/int+)
         (fn [[x1 x2]] (>= x2 x1))))
#2019-07-2617:55kennyI suppose you could write a custom gen for an interval.#2019-07-2619:26Alex Miller (Clojure team)there already is an interval spec with a generator - s/int-in#2019-07-2619:26kennyThat generates an integer. This one is an actual interval. e.g. [0 10]#2019-07-2619:26Alex Miller (Clojure team)ah, I see#2019-07-2617:58gfredericks@kenny yeah the general rule with such-that is that the predicate needs to be highly-likely to succeed; so for something like an interval you'd want to write your own generator, which is pretty easy#2019-07-2617:58kennyI mean, it is highly likely to succeed in the above case.#2019-07-2617:59gfredericksisn't it just 50%?#2019-07-2618:00kennyI think so#2019-07-2618:00gfredericksmore precisely what you want is "astronomically unlikely to fail 10 times in a row"#2019-07-2618:00gfredericksand something that only passes 50% of the time doesn't have that quality#2019-07-2618:00kennyFair enough#2019-07-2618:01gfredericksas such-that retries, it's also incrementing size, so often it's good enough that the probability of failure goes down with larger size#2019-07-2618:01gfrederickse.g., gen/not-empty relies on that#2019-07-2618:03kennyIt almost seems like you should be able to enable a warning that tells you when your specs use s/and and don't have a custom gen.#2019-07-2618:31kennyWriting the generator for an interval, like the above is tricky without such-that because you don't know how to generate a value that will match the spec. I was originally thinking it'd just be a gen/bind but that doesn't work because I don't know how to generate a ::m/int+ such that the second one is greater than the first without using such-that.#2019-07-2619:28Alex Miller (Clojure team)the trick is to generate a starting value and the range size, then gen/fmap the pair from that#2019-07-2618:33bfabryone option: before doing a such-that add something like this https://github.com/bfabry/specify-it/blob/master/src/specify_it/bst_spec.clj#L268 that just reports on how often the such-that would succeed. then tune your inputs until itā€™s a reasonable frequency#2019-07-2618:34kennyWell this is a reasonable frequency but it's certainly not "astronomically unlikely to fail 10 times in a row"#2019-07-2618:35kennyIt's fairly unlikely though.#2019-07-2618:36bfabryin the case that youā€™re going to throw away generations that donā€™t match your predicate, I would put ā€œreasonable frequencyā€ at like 50%#2019-07-2618:37kennyThat seems right. This would go back to the original problem then where, in practice, overriding such-that doesn't really work as a debugging technique.#2019-07-2618:42bfabrya test.check generator mode where all of the various predicates print out their form line number and % success rate at the end would be cool (and quite hard to do)#2019-07-2618:42kennyThat's exactly what I want too šŸ™‚#2019-07-2618:52gfredericks@kenny w.r.t. avoiding such-that, how about fmap with sort?#2019-07-2618:52kennyOoo, I like it!#2019-07-2618:53gfredericksfmap can be used to avoid such-that in a lot of cases another example is here: https://clojure.org/guides/test_check_beginner#_generator_combinators#2019-07-2619:27kenny@gfredericks That cleaned up this code nicely šŸ™‚ Thanks for the idea! https://github.com/Provisdom/math/blob/488573a16947eab78ecd096c7ef71a33f94cd0d7/src/provisdom/math/intervals.clj#L14-L55#2019-07-2619:29Alex Miller (Clojure team)I would generate the starting value and the range size, then fmap to get [start (+ start size)]#2019-07-2619:30kennyI don't think that'd work because you'd need to know how to generate the range.#2019-07-2619:30kennyFor example, we have a "prob-interval" where the interval's lower and upper bounds must be between 0 and 1.#2019-07-2619:32kennyOne issue with @gfredericks approach, however, is if you have a strict-interval. You have a chance of hitting the such-that then.#2019-07-2619:33gfredericksSort+inc?#2019-07-2619:34kennyCan't be that simple because of the above interval case.#2019-07-2619:34kenny"prob-interval", that is#2019-07-2619:36kennyYou could add Double/MIN_VALUE in that case. But if the interval is integers only, that won't work.#2019-07-2619:36kennyWell, even adding Double/MIN_VALUE may not work because that may go outside the range.#2019-07-2620:01gfredericksMath/nextUp šŸ™‚#2019-07-2620:25kennyI think that's still problematic for the same reason though.#2019-07-2620:26kennyWhy isn't generator? public in spec's gen ns?#2019-07-2620:34Alex Miller (Clojure team)Might be an oversight, might be a reason it wonā€™t work with dynaload#2019-07-2620:38kennySame with vector-distinct-by#2019-07-2620:40kennyMight be able to add them to our own libs by using lazy-combinators#2019-07-2620:42kennyWon't work because lazy-combinator isn't qualified šŸ˜ž https://github.com/clojure/spec.alpha/blob/5228bb75fa10b7b13ea088d84f4d031cce74a969/src/main/clojure/clojure/spec/gen/alpha.clj#L92#2019-07-2620:47kennyCould s/coll-of take a new option for :distinct-by?#2019-07-2621:01souenzzoI need something like (s/tuple (s/tuple ::a) (s/tuple ::b)), but should allow values like [["a" "d"] ["b"]] (ignore "extra" tuple values)#2019-07-2621:04souenzzo
(s/cat :a (s/spec (s/cat :a ::a
                         :extra (s/* any?)))
       :b (s/spec (s/cat :b ::b
                         :extra (s/* any?))))
^ this one works, but it's really ugly
#2019-07-2621:13kennyCan you use a s/or?#2019-07-2621:21souenzzoor where?#2019-07-2621:23kenny
(s/def ::tup1 (s/tuple string?))
(s/def ::tup2 (s/tuple string? string?))
(s/def ::tup (s/or :tup1 ::tup1 :tup2 ::tup2))
(s/tuple (s/tuple ::tup ::tup))
#2019-07-2621:26souenzzonot sure how many ::tupN is needed. The fact is: just the 3 first values are required/used. (each one has it own spec) But sometimes a larger collection if passed as arg.#2019-07-2621:27kennyTuple is usually for a list of a known length. Sounds like you really want cat, like your example#2019-07-2919:45respatializedat risk of asking an obvious question: how do I refer to a spec from another namespace? here's an example: in namespace-0, I have some specs defined using (s/def ::consistent ...). In namespace-1, I am using (:require [namespace-0)], but making calls to the specs in the other namespace by doing something like (s/valid :namespace-0/consistent data) fails with a Unable to resolve spec :namespace-0/consistent. How do I refer to these specs from another namespace using fully qualified keys?#2019-07-2919:57respatializedactually, it works as expected using the fully qualified keys. my issue was just a typo in the spec name. šŸ˜‘#2019-07-2921:54kennyIs this how :gen-max is supposed to work? It appears to be a bug.
(distinct (map count
               (gen/sample
                 (s/gen (s/map-of string? string?
                                  :min-count 1
                                  :gen-max 1)) 100)))
=> (1 2)
#2019-07-2921:56Alex Miller (Clojure team)https://clojure.atlassian.net/browse/CLJ-2202#2019-07-2921:56Alex Miller (Clojure team)tis bug#2019-07-2921:57kennyAh, ok. Switched everything to use :gen-max and generators got 2-2.5x slower šŸ˜¬#2019-07-2921:57Alex Miller (Clojure team)doh#2019-07-2921:58kennyI think a lot of places are only generating one item now generate up to 2.#2019-07-3015:23misha@afoltzm "required" aliases work too:
(require '[clojure.spec.alpha :as s])
::s/invalid
;=> :clojure.spec.alpha/invalid
#2019-07-3015:27Alex Miller (Clojure team)they work, assuming that the qualifier actually corresponds to a real clj namespace#2019-07-3021:14dharriganIs there a spec that tests for an empty map (as a value) i.e., {:foo {}}?#2019-07-3021:30minimal(s/def ::empty-map #{{}})#2019-07-3107:24dharrigansweet#2019-07-3107:24dharriganta šŸ™‚#2019-08-0114:06ben.mumfordare there any best practices anywhere on where in the code to define specs? at the moment i'm defining function specs alongside the function they go with and don't have a consistent strategy for where to define the specs (spec/def ...) - usually in a few large files#2019-08-0114:08jjttjj@ben.mumford620 there's a discussion on this here: https://stackoverflow.com/questions/37942495/where-to-put-specs-for-clojure-spec#2019-08-0114:10ben.mumfordcheers. i'm guessing then that are no suggestions from rich and the crew?#2019-08-0114:16jjttjjThis is extremely shaky ground on which to make a decision, but here's a cognitect example of a spec only namespace: https://github.com/cognitect-labs/day-of-datomic-cloud/blob/b4103e4a8f14518ed3f6d7f66f56cbf863117974/src/datomic/dodc/repl_ui/specs.clj#2019-08-0114:16Alex Miller (Clojure team)https://clojure.org/guides/faq#spec_location#2019-08-0114:20ben.mumfordthanks alex#2019-08-0216:40kennyI am getting this exception "Additional data must be non-nil." when using st/check on my function. I have gotten this before and then it mysteriously went away. I am now getting it constantly and am curious what is going on. It is coming from these Spec lines: https://github.com/clojure/spec.alpha/blob/5228bb75fa10b7b13ea088d84f4d031cce74a969/src/main/clojure/clojure/spec/test/alpha.clj#L278-L284.#2019-08-0216:43ghadiif you have a reproducible example case, please paste it @kenny#2019-08-0216:44kennyWorking on that. It's pretty coupled to this code but I should be able to pull something out.#2019-08-0216:44ghadicool -- thanks!#2019-08-0216:44ghadithis has been reported before, but AFAIK there's not a deterministically reproducible case#2019-08-0216:45kennyHopefully I can get one! It's definitely reproducible for me right now.#2019-08-0216:49Alex Miller (Clojure team)the error occurs when you try to create an ex-info with nil data. the case where this happens here is when spec validation fails, but explain-data succeeds (which ideally should never happen). usually, this is a bug in spec. occasionally, it's a bad predicate in user code.#2019-08-0216:50kennyThis error is quite confusing. Seems like a bug either way šŸ˜‰#2019-08-0216:50Alex Miller (Clojure team)well the error should never happen#2019-08-0216:50ghadiright, it's an invariant violation#2019-08-0216:52kennyhaha, the best kind of bugs#2019-08-0216:53kennyI pulled some code out to get a repro case and was running st/check on it. It failed with a regular specification error. I thought that was weird because it hasn't failed yet. Turned on instrumentation and ran again and got the "Additional data must be non-nil." message. May be related to instrument.#2019-08-0216:57Alex Miller (Clojure team)this won't help you, but I've made a change in spec-alpha2 to better report this case#2019-08-0217:22kennyAlright, here's a repro: https://gist.github.com/kennyjwilli/f324b94eaadc404cb72fdfe41067b469. Not minimal but I've gotta go make some food. Within 3 or so runs of (st/check formula->fn)`, it will get the exception.#2019-08-0218:23Alex Miller (Clojure team)so I misdiagnosed above - even more subtle... s/conform and s/valid? are returning different answers when run on the same input#2019-08-0218:24Alex Miller (Clojure team)which is because the spec is an fspec, which generates its own source of random inputs#2019-08-0218:25Alex Miller (Clojure team)this is a known issue and generally we've been suggesting people don't use fspec because it has all these unexpected consequences#2019-08-0218:50kennyfspec has proven to be quite valuable though. What is the alternative? Just fn?#2019-08-0218:56Alex Miller (Clojure team)ifn?#2019-08-0218:57Alex Miller (Clojure team)unfortunately it's not even easy to override in a spec alias or anything#2019-08-0218:58kennyWon't that just generate any old function during gen tests and pass it to my function?#2019-08-0219:07Alex Miller (Clojure team)yes, it is significantly harder to use it in check. I'm not saying any of this is good.#2019-08-0219:09Alex Miller (Clojure team)this is a known area I'm hoping to spend some rethink time on in spec 2, but we're going to overhaul the function return value stuff first#2019-08-0219:16kennyOh ok. Any workaround for now to avoid this error?#2019-08-0219:21Alex Miller (Clojure team)I don't think I have any simple advice for you, other than to remove your :ret spec :(#2019-08-0219:25kenny:ret on the fspec of the top level fn?#2019-08-0219:27Alex Miller (Clojure team)no, the :ret on the fdef#2019-08-0219:27Alex Miller (Clojure team)that is an fspec#2019-08-0220:10kennyI was trying to add some gen improvements to clojure.spec.alpha but I cannot load the ns in a REPL. I get
CompilerException java.lang.Exception: #object[clojure.spec.alpha$and_spec_impl$reify__2171 0x4a332076 "
Is there some magic that needs to happen to have a REPL with Spec itself?
#2019-08-0220:49y.khmelevskiihi gents! How I can define spec of vector where I know only first element and donā€™t know length of vector. For example [:in 1 2 3 4]#2019-08-0220:56y.khmelevskiiit would be great to use something like
(s/tuple #{:in :or} & nat-int?)
#2019-08-0220:58Alex Miller (Clojure team)(s/cat :in #{:in :or} :nums (s/* nat-int?))#2019-08-0220:59Alex Miller (Clojure team)this is exactly the job the regex ops are made for#2019-08-0221:01y.khmelevskiiI see, thank you!#2019-08-0317:54hjrnunesHi. Do predicates need to accept MapEntry arguments to be used with s/or? This predicate
(defn str-date-time? [v]
  (try
    (some? (f/parse (f/formatters :date-time) v))
    (catch IllegalArgumentException _
      false)))
Causes ClassCastException: clojure.lang.MapEntry cannot be cast to java.lang.String when used in an or spec. If I change the catch to Exception I get a failed validation with
Part of the value

  {:post/updated "2017-05-05T11:55:00.000Z", ...}

when conformed as

  [:str-date-time "2017-05-05T11:55:00.000Z"]
with spec
(s/def :post/updated (s/or :date-time ::tspec/date-time
                                             :str-date-time str-date-time?))
used as :req in a s/keys form. Any thoughts? Thanks
#2019-08-0318:21seancorfield@hjrnunes That sounds like you're passing a conformed value into str-date-time?#2019-08-0318:23hjrnunesYes it does. But it's utterly puzzling. I'm merely calling s/valid? on a map#2019-08-0318:23hjrnunesDoes s/or conform values?#2019-08-0318:23seancorfieldThe result of s/or is a tagged pair, yes.#2019-08-0318:24hjrnunesso, I should cater for this in the predicate?#2019-08-0318:24seancorfieldNo.#2019-08-0318:24seancorfieldI need to see more of your specs to see where you're flowing a conformed value into that spec.#2019-08-0318:24hjrnunesactually, I have the same result with string? or int?#2019-08-0318:25seancorfieldShow the context.#2019-08-0318:26hjrnunesyes, I'm trimming stuff down. Thanks for the help#2019-08-0318:27seancorfieldDo you have s/and somewhere? That's often where a conformed value ends up flowing into another predicate.#2019-08-0318:30hjrnunesYes I do#2019-08-0318:30hjrnunes
(s/def :post/published (s/or :date-time ::tspec/date-time
                             :str-date-time str-date-time?)) ;; should be DateTime or :date-time format

(s/def :post/updated (s/or :date-time ::tspec/date-time
                           :str-date-time str-date-time?))  ;; should be DateTime or :date-time format


(s/def ::post-found (s/keys :req [:post/published
                                  :post/updated]))

(s/def :post/missing coll?)

(s/def ::post-missing (s/and ::post-found
                             (s/keys :req [:post/missing])))
#2019-08-0318:31seancorfieldYou probably want s/merge not s/and there.#2019-08-0318:32hjrnunesHmm, perhaps. So the idea is to re-use ::post-found and add another key - which is what actually happens to the data#2019-08-0318:32hjrnunesso s/merge is the right way then?
#2019-08-0318:33seancorfieldThat's what s/merge is for -- to blend two s/keys specs.#2019-08-0318:33hjrnunesI see, I read somewhere, the mailing list I think, that s/and would work. Perhaps it was in a previous version#2019-08-0318:34hjrnunesYep. That does it. Thank you very much!#2019-08-0318:35hjrnunesBTW, I listened to the defn podcast you were on recently - very interesting! Best of luck and thanks for all the work!#2019-08-0318:39seancorfields/and is specifically for flowing a conformed value into another spec. For example, you might s/and one s/keys-based spec and then a predicate that checks two keys have distinct values perhaps.#2019-08-0318:39seancorfields/merge is for extending an s/keys-based spec with additional key specs -- like merge for maps.#2019-08-0505:27akondexplain-data produces a list of problems, but i've only seen a single item in it. can somebody show a case when the :problems vector contains more than one item?#2019-08-0512:39Alex Miller (Clojure team)(s/explain-data (s/or :i int? :k keyword?) "abc")#2019-08-0512:46akond@alexmillerthank you#2019-08-0512:49akondbut is it still possible to produce more then 1 problem without s/or's?#2019-08-0512:50Alex Miller (Clojure team)yes, there are several places where it can happen#2019-08-0604:21dorabCan someone please explain this behavior?#2019-08-0604:22dorab
$ clj -Srepro
Clojure 1.10.1
user=> (require '[clojure.spec.alpha :as s])
nil
user=> (require '[clojure.spec.gen.alpha :as sg])
nil
user=> (require '[clojure.spec.test.alpha :as st])
nil
user=> (s/def ::vec vector?)
:user/vec
user=> (s/def ::str string?)
:user/str
user=> (s/def ::str-vec (s/coll-of ::str :kind ::vec))
:user/str-vec
user=> (sg/generate (s/gen (s/coll-of string? :kind vector?)))
["2m4A8TKjUU5TJ61KkKRzbnk2z21" "xAvkdc78i5qo6O8Gf6" "cHbZwfz88d6xZj5qp54L5g" "K6MdrEaiFWm2P27BCD1913h9r1" "2jpUYE74Vvf" "PG9SrB729W5pWo6G5xt7K28tR5M33" "DTAJijHs2t9mEC" "6zkimTFtp51q698N" "eAgGEU08gJF7R5C2tb6ZdcTRO" "Ewx60Cb94NWxMMXjQJn"]
user=> (sg/generate (s/gen ::str-vec))
Execution error (ExceptionInfo) at clojure.test.check.generators/such-that-helper (generators.cljc:320).
Couldn't satisfy such-that predicate after 100 tries.
user=> 
#2019-08-0604:23dorabWhy does the sg/generate work in one case, but not the other?#2019-08-0604:54seancorfield@dorab :kind should not be a spec, it should be a predicate#2019-08-0604:54seancorfieldIf you define ::str-vec with :kind vector? it works just as expected.#2019-08-0604:55seancorfieldIf you read the docstring for s/every (which applies to s/coll-of), it says
:kind - a pred that the collection type must satisfy, e.g. vector?
        (default nil) Note that if :kind is specified and :into is
        not, this pred must generate in order for every to generate.
#2019-08-0604:56seancorfieldPredicates can be treated as specs, but specs are not predicates (in general).#2019-08-0615:29dorabThanks#2019-08-0903:42valeraukois there any roadmap for spec 2? we plan to add spec validations to our codebase and if spec 2 is reasonably close we'd go straight with that#2019-08-0903:49Alex Miller (Clojure team)roadmap in terms of things to do or timing#2019-08-0903:49Alex Miller (Clojure team)I guess you mean timing#2019-08-0903:50Alex Miller (Clojure team)I would not expect it to be "done" for at least another month or two#2019-08-0903:51valeraukowe could work on adding it from this fall (october <) so that sounds promising to me#2019-08-0903:51Alex Miller (Clojure team)well, you might be fine then#2019-08-0903:51Alex Miller (Clojure team)you can use it right now as a git dep, but it is definitely a work in progress#2019-08-0903:52Alex Miller (Clojure team)https://github.com/clojure/spec-alpha2#releases-and-dependency-information#2019-08-0903:54seancorfield@vale We have a branch of our code base running on Spec 2. There are definitely some interesting differences from Spec 1, and if you do anything where you need to share specs between 1 & 2 (perhaps because you use specs from a library that depends on Spec 1), you'll need to jump through some hoops.#2019-08-0903:55Alex Miller (Clojure team)I've landed two big sets of changes in the last week there, will have a writeup on some new stuff tomorrow#2019-08-0903:56seancorfield(just in time for me to go to England for a week so I won't get to play with it until I get back!)#2019-08-0903:57valeraukoi heard you mention that on the defn podcast, but i'm not adventurous enough to use it in production code#2019-08-0903:58valeraukowe're adding spec from zero, and that 1/2 difference thing is exactly the reason i'd prefer to go straight to 2 if it's feasible#2019-08-0904:01valeraukoon a similar note, any plan on making it not-alpha?#2019-08-0904:11seancorfieldJust reviewing those commits @alexmiller -- love the new closed spec approach!#2019-08-0904:11seancorfield@vale Oh that branch isn't in production -- just dev -- until Alex green lights production use (even in alpha).#2019-08-0904:12seancorfieldWe started using Spec 1 in early alpha in production. We take most Clojure alpha builds to production. But Spec 2 isn't yet at that point.#2019-08-0904:13seancorfieldBased on past experience, I'd expect Spec 2 to stay alpha for quite a while before it moves to a beta. tools.deps is still alpha and it's core to the CLI / deps.edn tooling.#2019-08-0904:33Alex Miller (Clojure team)yeah, we're still actively changing the api in spec 2 and I don't think that's done yet. we're not going to have a "release" until that settles down#2019-08-0904:34Alex Miller (Clojure team)my hope is that it can be stable enough to be non-alpha and so it will never release as spec-alpha2 at all, but will depend on timing#2019-08-0904:34Alex Miller (Clojure team)tools.deps edges ever closer to me being comfortable with taking it off alpha#2019-08-0904:35Alex Miller (Clojure team)the api is pretty stable there#2019-08-0904:43seancorfield@alexmiller Good to hear that about Spec 2. Targeting "release" by Conj?#2019-08-0904:43seancorfieldRe: tools.deps -- any decision on add-lib yet?#2019-08-0905:41valeraukooh nice! it could be tricky to explain to stakeholders how something labelled alpha is production ready#2019-08-0905:46seancorfield@vale Stakeholders outside the Clojure world, yes. But inside the Clojure world there's much more stability than almost any other language.#2019-08-0905:47seancorfieldWe went to production originally, back in 2011, on Clojure 1.3 Alpha 7 or 8, and we've run almost every single pre-release build of Clojure in production ever since.#2019-08-0905:49seancorfieldQuite a few Contrib libs have been in pre-release mode for ages on various revisions and it's "expected" that folks will use them, reporting any problems, but on the assumption that they're "solid". There is, after all, a mindset -- driven from the top down -- of accretive/fixative change rather than breakage.#2019-08-0905:50seancorfield(that said, some libs have had breaking changes several times -- including clojure.java.jdbc that I have maintained for about eight years!)#2019-08-0908:41dharriganupvote for add-lib in tools.deps please šŸ™‚#2019-08-0911:37ikitommi@alexmiller Closed spec looks so much better now (also read the commits)! As the settings open up more options to do things, it would be great if you would time to re-check the proposed patch in https://clojure.atlassian.net/browse/CLJ-2116, as it had similar approach, but I believe, would be even more generic way to do this, for even to apply coercion. Or if thatā€™s an bad idea, that should be declined as itā€™s now partially overlapping with the new design.#2019-08-0911:39Alex Miller (Clojure team)Well, all spec patches in jira are probably long broken given that Iā€™ve rewritten pretty big chunks of it multiple times#2019-08-0911:40Alex Miller (Clojure team)Once we get through the majority of new feature type stuff, I will go through them all but no point now #2019-08-0911:40Alex Miller (Clojure team)As Iā€™ve said in the past, I think itā€™s highly unlikely that weā€™re going to take clj-2116 regardless#2019-08-0911:43ikitommiI donā€™t think itā€™s a good way anymore either (2116), wrote a the spec walker issue later that could be the one generic way to walk over specs + values, and could be used for both.#2019-08-0911:44Alex Miller (Clojure team)Well, weā€™re working on the walking stuff too#2019-08-0911:44ikitommicurrently spec-tools & spec-coerce walk the spec forms which is IMO not a right way to do that.#2019-08-0911:45Alex Miller (Clojure team)The other half of what Iā€™m working on now is dataficatiom of specs#2019-08-0911:45ikitommicool. just for browsing or also transforming?#2019-08-0911:46Alex Miller (Clojure team)Either - the input side is there now#2019-08-0911:46Alex Miller (Clojure team)Once you have the output side, then transformation is just normal clojure stuff#2019-08-0911:48ikitommione common case for closed specs has been the ā€œstrip out extra keysā€ functionality, is that doable with the new closed feature? e.g. before saving to the database / when reading input from untrusted clients / intenet.#2019-08-0911:49Alex Miller (Clojure team)Havenā€™t really considered it. Why not just use select-keys?#2019-08-0911:50ikitommiselect-keys for a deeply nested structure mean double-maintaining the keysets. some helper would be nice#2019-08-0911:51Alex Miller (Clojure team)I think weā€™re giving you a lot of new tools to build your own spec that does something like this (flag passed through settings) but we have not talked about any plans to add this#2019-08-0911:53Alex Miller (Clojure team)This again feels like a coercion/transformation use case which we donā€™t believe is a job for conform#2019-08-0911:53ikitommiok. the data is there (keysets) already to feed a select-keys. would love to throw away stuff from spec-tools, when they are doable with spec itself. currently:
(s/def ::name string?)
(s/def ::street string?)
(s/def ::address (s/keys :req-un [::street]))
(s/def ::user (s/keys :req-un [::name ::address]))

(def inkeri
  {:name "Inkeri"
   :age 102
   :address {:street "Satamakatu"
             :city "Tampere"}})

(st/select-spec ::user inkeri)
; {:name "Inkeri"
;  :address {:street "Satamakatu"}}
#2019-08-0911:53ikitommispec-tools does that nowadays with form walking btw.#2019-08-0911:55ikitommiI think that would use the upcoming ā€œwalkerā€, so a good test case when you develop it šŸ˜‰#2019-08-1013:26norton@alexmiller Hello. I have a question regarding spec-alpha2. Specs and tests written with specs stopped working from this commit. https://github.com/clojure/spec-alpha2/commit/2bd68ffee3a775e6fa0f925260b94a9c421787c0 I am reviewing the diff and I spotted one item that Iā€™m curious about. The third argument to conform* on this line 220 https://github.com/clojure/spec-alpha2/blob/2bd68ffee3a775e6fa0f925260b94a9c421787c0/src/main/clojure/clojure/spec_alpha2.clj#L220 and on line 326 https://github.com/clojure/spec-alpha2/blob/2bd68ffee3a775e6fa0f925260b94a9c421787c0/src/main/clojure/clojure/spec_alpha2.clj#L326 looks inconsistent. Is this difference intentional ā€¦ x on line 220 and spec on line 326?#2019-08-1013:29Alex Miller (Clojure team)Yeah, the first one should be spec#2019-08-1013:29Alex Miller (Clojure team)Do you have an example of whatā€™s not working for you?#2019-08-1013:30nortonSorry, I do but not a small sample that I can share.#2019-08-1013:31Alex Miller (Clojure team)Ok, would be happy to take a look if you can describe#2019-08-1013:31nortonOr ā€¦ can you share a recipe how I can test out locally by making this change?#2019-08-1013:39nortonThe essence of the test is as follows: (doseq [spec (filter #(contains? STRING-NAMESPACE (namespace %)) (keys s/registry)))] (let [p (prop/for-all [g (s/gen spec)] (s/valid? spec g))] (is (= {:result true} (abbrev-result (tc/quick-check 100 p))))))#2019-08-1013:39nortonwhere STRING-NAMESPACE is a string used to filter keys from the s/registry#2019-08-1013:39nortonI might not have all of the parens correct ā€¦ just pseudo-code šŸ˜‰#2019-08-1014:45Alex Miller (Clojure team)that conform thing above was fixed in the next commit btw#2019-08-1014:46Alex Miller (Clojure team)so what fails? which spec and what's the failure?#2019-08-1015:05nortonHere is a subset of the stack trace. I bumped up to the latest version 'ca131c1bec353a6ebe4fe8e0a8b6f8825734ef42'.#2019-08-1015:17nortonHere is a subset of the stack trace. I pinned it at the version '000d7d83a98ca3af58a44c20d7bd9fea0f4b03ab'#2019-08-1015:18nortonInterestingly, there seems to be a different failure(s) between these two commits.#2019-08-1015:23nortonThe commit '922b0f5886735641e0cdded58fc85011f79cc292' has the same stack trace as 'ca131c1bec353a6ebe4fe8e0a8b6f8825734ef42'. This commit introduced the second failure.#2019-08-1015:24nortonAs I posted originally, the commit '2bd68ffee3a775e6fa0f925260b94a9c421787c0' introduced the first failure.#2019-08-1015:39nortonI will try to narrow down the issue to a small repository.#2019-08-1015:41Alex Miller (Clojure team)sorry, hard for me to do much without seeing the spec#2019-08-1015:43nortonUnderstand#2019-08-1015:54Alex Miller (Clojure team)Some spec updates here: http://insideclojure.org/2019/08/10/journal/#2019-08-1016:14eval2020I would expect the first (s/valid?...) to be without the {:closed ...}#2019-08-1017:41Alex Miller (Clojure team)yes, I whiffed some copy/pasta on the example, fixed#2019-08-1114:52nortonWorking example with 'dc960158ccb9cf8b5dc555842eea5e1af12f28c2'#2019-08-1114:53nortonFailing example with '2bd68ffee3a775e6fa0f925260b94a9c421787c0'#2019-08-1114:57norton@alexmiller Please see these two examples. The latest version 'ca131c1bec353a6ebe4fe8e0a8b6f8825734ef42' behaves the same as the failing example.#2019-08-1120:48seancorfieldDefinitely related to (s/def ::bar ::baz) since you can do this
user=> (s/def ::quux (s/keys :req-un [::baz]))
:user/quux
user=> (gen/sample (s/gen ::quux))
({:baz 0} {:baz -1} {:baz -1} {:baz -3} {:baz -3} {:baz -2} {:baz 0} {:baz 3} {:baz 0} {:baz -1})
user=>    
#2019-08-1121:26Alex Miller (Clojure team)fixed#2019-08-1121:36seancorfieldYup. Just confirmed that works now!#2019-08-1200:32norton@alexmiller Thank you. That issue is resolved. I have a new one. Here is a failing example taken from the spec Guide. This is with the latest version 'e58788107ee2ab765a7d46854b2cffd85429e339'.#2019-08-1200:55seancorfield@norton I'll defer to Alex for the final word but I think that's a case where Spec 2 and Spec 1 differ. I think I ran across something like that when I tried converting our code over to Spec 2...#2019-08-1200:56norton@seancorfield thank you. I will take another look.#2019-08-1201:03seancorfieldI'm trying a few things locally but it isn't jogging my memory of what I had to do here. So it may be an unintended bug.#2019-08-1201:11seancorfieldHmm, the more I look at this, the more I think it is a new bug. I pulled up the Spec 2 branch of our code and we have some specs that look very similar to the above and used to work. Right now I can't test the Spec 2 branch on my laptop so I can't dig in any further until tomorrow @norton#2019-08-1201:34norton@seancorfield Yes, I think so. This type of spec was working with the 'dc960158ccb9cf8b5dc555842eea5e1af12f28c2' commit.#2019-08-1202:54Alex Miller (Clojure team)fixed, just missed the expand-spec for with-gen#2019-08-1205:20seancorfield@alexmiller That fixes the problem reported above, but you can't do (s/exercise ::kws) -- that produces
user=> (s/exercise ::kws)
Execution error (IllegalArgumentException) at clojure.spec-alpha2.protocols/eval173$fn$G (protocols.clj:11).
No implementation of method: :gen* of protocol: #'clojure.spec-alpha2.protocols/Spec found for class: clojure.lang.PersistentHashSet
user=> 
#2019-08-1209:24Alex Miller (Clojure team)That is a spec 2 difference - the gen needs to wrap s/spec around the set#2019-08-1215:55seancorfieldAh yes. That was the difference I was thinking of when I first thought this was a Spec 1/2 difference but still couldn't get that example working! /cc @norton #2019-08-1209:51norton@alexmiller @seancorfield Thank you. The issue above is fixed. There is one more.#2019-08-1209:52nortonThis is an abridged version of a larger spec that illustrates the failure.#2019-08-1214:43Alex Miller (Clojure team)fixed, added some tests to catch this and prior#2019-08-1214:49schmeeCan you alias a spec to another spec? I have a situation like this where I would like to not repeat the definition of task-state for both old and new:
(s/def ::task-state #{... big set ...})
(s/def ::old ::task-state)
(s/def ::new ::task-state)
(s/def ::event (s/keys :req [::old ::new]))
#2019-08-1214:50schmeethe code above doesnā€™t work, but it hopefully shows the intent#2019-08-1214:52jjttjjThat code should work though?#2019-08-1214:53Alex Miller (Clojure team)should work (in spec 1)#2019-08-1214:54schmeeyouā€™re right, this was PEBKACā€¦ facepalm sorry for the noise#2019-08-1217:13ataggartAnyone know of guidance around organizing specs and the code being specā€™d? E.g., putting specs in the same file as the things being specā€™d, or in separate parallel nses, or in a separate common spec ns?#2019-08-1217:25Alex Miller (Clojure team)https://clojure.org/guides/faq#spec_location#2019-08-1217:31ataggartI swear I read that doc! šŸ™‚#2019-08-1217:31ataggartThanks#2019-08-1218:32seancorfield@alexmiller I'm updating our branch to the latest Spec 2 at work and ran into this exception "No implementation of method: :conform* of protocol: #'clojure.spec-alpha2.protocols/Spec found for class: java.lang.String", -- would that be from this spec: (s/def ::empty-id (s/or :empty #{""} :id ::pos-int)) ?#2019-08-1218:33Alex Miller (Clojure team)Dunno, would have to take a closer look#2019-08-1218:34seancorfieldI'll try wrapping #{""} in (s/spec ,,,) and see if that fixes it... but we have a lot of set-as-predicate specs and this has all worked with earlier Spec 2 builds...#2019-08-1218:37seancorfieldNope. Problem lies deeper. Digging...#2019-08-1218:38seancorfieldSeems to be related to this
(defmacro param-spec
  [coerce str-or-spec & [spec]]
  (let [[to-str spec] (if spec [str-or-spec spec] [str str-or-spec])]
    `(s/with-gen (s/and (s/conformer ~coerce) ~spec)
       (fn [] (g/fmap ~to-str (s/gen ~spec))))))
So it's around dynamically built specs.
#2019-08-1218:55gfrederickshttps://clojurians.slack.com/archives/C0JKW8K62/p1565636109003400#2019-08-1219:02Alex Miller (Clojure team)@seancorfield you shouldn't have to wrap that so I'd say it's a bug if that fixes it :)#2019-08-1219:04Alex Miller (Clojure team)it's probably not that it's dynamically built but rather something about what's being built (my first suspicion would be with conformer)#2019-08-1219:17seancorfield
user=> (require '[clojure.spec-alpha2 :as s])
nil
user=> (s/def ::foo (s/keys))
:user/foo
user=> (s/valid? ::foo {"bar" 42})
Execution error (IllegalArgumentException) at clojure.spec-alpha2.protocols/eval13332$fn$G (protocols.clj:11).
No implementation of method: :conform* of protocol: #'clojure.spec-alpha2.protocols/Spec found for class: java.lang.String
user=> 
Only fails if the map being checked has string keys, rather than keyword keys.
#2019-08-1219:19seancorfield@alexmiller Smallest repro ^#2019-08-1219:30Alex Miller (Clojure team)ah, thx#2019-08-1219:30Alex Miller (Clojure team)I changed how some of that keys stuff worked, obviously did not test it very well (but not really emphasizing keys as that's going to be replaced by schema/select)#2019-08-1219:49seancorfieldI assume it's trying to check whether unspecified keys conform and not guarding that with a check for whether the key name could actually be a spec?#2019-08-1219:50seancorfieldSwitching a large code base from s/keys to s/schema/`s/select` will be a massive piece of work so I hope that s/keys will continue to work while we make those changes šŸ™‚#2019-08-1219:56Alex Miller (Clojure team)yeah, I'm trying to keep it working, I'll fix it up#2019-08-1222:26nortonAll tests are running fine now with 6af1f3.#2019-08-1305:09murtaza52I have a sorted collection, I want to define the relationship between 2 elements of the collection ie the first element is greater than the second element. Is this possible ?#2019-08-1305:32seancorfield@murtaza52 (sort comparator coll) lets you pass in a comparator that defines the relationship on which you are sorting.#2019-08-1305:36seancorfield
user=> (sort (reify java.util.Comparator (compare [_ a b] (cond (even? a) -1 (even? b) 1 :else 0))) (range 20))
(18 16 14 12 10 8 6 4 2 0 1 3 5 7 9 11 13 15 17 19)
user=> 
#2019-08-1305:36seancorfield(a silly example, but it sorts all even numbers ahead of all odd numbers šŸ™‚ )#2019-08-1305:37seancorfieldIs that an answer to the question you were asking? (I wasn't entirely sure what you were asking)#2019-08-1305:39murtaza52@seancorfield thanks for the example šŸ™‚. However , I am already using a custom comparator for my sorting, however how do I use that in my spec definition ?#2019-08-1305:39seancorfieldI guess I don't understand what you're trying to do...#2019-08-1305:39murtaza52I have a function that does the sorting. Now I want to spec the fn, to specify that the output collection is sorted#2019-08-1312:21Alex Miller (Clojure team)We are kicking around some ideas for stating this constraint in spec 2#2019-08-1312:22murtaza52cool thanks#2019-08-1305:39seancorfieldAh...#2019-08-1305:40seancorfieldThis is one of those cases where the spec would almost need to be the implementation. I think that's over-specification to be honest.#2019-08-1305:40seancorfieldI very rarely specify the result to that level of detail.#2019-08-1305:41seancorfieldAfter all, it's not like instrumentation checks :ret or :fn so this is really only about generative testing and I think that can be better served by property-based testing, not direct :ret/`:fn` spec testing.#2019-08-1305:42seancorfieldFor example, for any collection with count > 1, good properties to test are that the comparator produces the expected result for the first and second items, and for the first and last items.#2019-08-1305:43seancorfieldIt's tempting to "spec everything" when you get started but it's really not very productive in my experience.#2019-08-1305:44murtaza52ack makes sense from a testing perspective. however what I like about specs is also the documentation it produces. So when I spec a relationship, it also helps other devs to know what the fn is going to return.#2019-08-1305:44seancorfieldSpec should be "just enough" to sanity check what you're writing -- unless you are specifically using it to validate input data (or output data), rather than specific functions.#2019-08-1305:48seancorfield@murtaza52 I'm not sure that using Spec to that level of detail, just for documentation, is productive. I would say "most functions do not need to be Spec'd" but you could describe the behavior in the docstring without adding the overhead of Spec...#2019-08-1305:49seancorfieldSpec makes the most sense on boundaries and for the "critical" functions. Spec'ing everything makes code brittle.#2019-08-1305:49seancorfield(and verbose)#2019-08-1305:49murtaza52@seancorfield thanks for the insight#2019-08-1306:09seancorfieldIn my mind, this is why type systems can make code brittle -- you end up with types everywhere rather than just where they "make sense". It's one of the things I really like about Clojure: you can write a lot of generic code and a lot of abstractions without introducing that brittleness -- but you also have Spec for defining data structures and for certain functions on the edges of modules.#2019-08-1306:09seancorfieldIn my mind, this is why type systems can make code brittle -- you end up with types everywhere rather than just where they "make sense". It's one of the things I really like about Clojure: you can write a lot of generic code and a lot of abstractions without introducing that brittleness -- but you also have Spec for defining data structures and for certain functions on the edges of modules.#2019-08-1306:40murtaza52yup makes sense#2019-08-1310:19murtaza52@seancorfield so trying to think through, in what situations does specā€™ing a fn makes sense, so that it could instrumented later ? Bcoz as u mentioned specā€™ing all fns dont make sense. The ones on the boundary I am already validating. So where does the optional specā€™ing come in ?#2019-08-1316:29seancorfieldSpec'ing functions can be useful for development / testing (instrumentation / generative checking) but I would generally only spec functions that are part of module APIs or "important" functions. There's no hard and fast rules here -- just spec what is "useful" to spec. For example, in next.jdbc, there are optional instrumentation specs for the public API functions -- for users of the library -- and those are instrumented during test runs too, for me as the developer of the library (but it's also important run the tests from time to time without instrumentation to see what behavior/errors crop up in known bad input data).#2019-08-1316:29seancorfieldI think you just have to find what works for you in terms of the amount of checking you need during development/testing and in terms of what you want to provide for clients of your code.#2019-08-1316:30seancorfieldIt's a bit like the question "Which functions should I unit test?" -- the answer is "Not all of them" but there are no hard and fast rules there either.#2019-08-1306:39murtaza52In the below spec -
(s/def ::id (s/and string? #(not (clojure.string/blank? %))))

  (s/def ::my-event (s/keys :req [::id]))
this is valid - (s/explain ::my-event {::id "a"}) this is invalid - (s/explain ::my-event {:id "a"}) So do all my keys in the input have to be namespaced ? Because I am not able to validate data when the keys are not namespaced.
#2019-08-1306:46seancorfieldIf you use :req then, yes, the keys must be namespaced. If you use :req-un then you have have simple keywords.#2019-08-1306:47seancorfieldWith :req-un, you can have ::foo/bar as the spec for the key -- but the key will be :bar. That allows you to have different specs for the same key name in different contexts.#2019-08-1306:56murtaza52thanks, I was confused bcoz when I used gen, it generated data without namespaced keywords. So my assumption was that it will also me to validate ones without ns.#2019-08-1306:56murtaza52So moving ahead is it more idiomatic to have namespaced keys in data too ?#2019-08-1307:42schmeeI have run into what I find to be a tricky modeling problem with spec. I have data that looks like this (excluding namespaces on the kws for brevity)
:event-type :state-transition
:old-state :running
:new-state :exception
Those three keys are always included, but if the event type is :state-transition and the new state is :exception, then there will be an :exception key as well (there are a couple of more special cases like this). What I would like is some form of ā€œwildcardā€, so that I can dispatch like so:
defmethod foo [:state-transition :exception]
defmethod foo [:state-transition _] <-- wildcard for the default case
Without the wildcard, I need a defmethod for all possible combinations state-transition/new-state which will lead to a lot of duplication. Is there a better way to accomplish this, in either spec1 or spec2?
#2019-08-1308:51jaihindhreddyName the special cases and use s/multi-spec with a dispatch fn that returns these names, then impl them by extending the multimethod.#2019-08-1312:42murtaza52Is there a pred for specifying only 1 param in a collection. I find this pattern when I want to spec the input args of a fn. Lets say if the fn expects a hash-map, I usually define the spec for args as :args (s/? ::map-spec). This pattern works, however is not correct bcoz it allows no values too.#2019-08-1312:44Alex Miller (Clojure team)s/cat is usually the best top level spec for args#2019-08-1312:44Alex Miller (Clojure team)(s/cat :m ::map-spec) #2019-08-1312:45Alex Miller (Clojure team)You can match up the tags and the args too#2019-08-1312:52murtaza52thanks, yup that is better#2019-08-1313:09Alex Miller (Clojure team)if you haven't seen the guide, it has several examples https://clojure.org/guides/spec#2019-08-1313:26murtaza52yup have gone through it, and its open on machine as a reference, however sometimes tend to miss a detail.#2019-08-1313:37murtaza52what is a good way to generate date time, I have a spec - (s/def ::start inst?), however the dates that it generates are very similar, and most are the same. (gen/sample (s/gen ::ks/start) 5) =>
(#inst "1969-12-31T23:59:59.999-00:00"
 #inst "1969-12-31T23:59:59.999-00:00"
 #inst "1969-12-31T23:59:59.999-00:00"
 #inst "1970-01-01T00:00:00.001-00:00"
 #inst "1970-01-01T00:00:00.000-00:00")
#2019-08-1313:41Joe Lane@murtaza52 https://clojure.github.io/spec.alpha/clojure.spec.alpha-api.html#clojure.spec.alpha/inst-in or write a custom generator.#2019-08-1313:43Alex Miller (Clojure team)inst-in will generate only dates near the start range in the first few samples but will then range farther#2019-08-1313:45Joe LaneIs that why sometimes I see a pattern to (drop 1000 ...) with some of the generators?#2019-08-1313:50ghadimy general pattern for this stuff is to generate a bunch of deltas, and add them to a constant "anchor" value, in this case a date#2019-08-1313:51ghadi(But since inst-in exists, use it)#2019-08-1313:51Alex Miller (Clojure team)which is exactly how inst-in generator works but it "grows" from the start date#2019-08-1313:51Alex Miller (Clojure team)so test.check "shrinking" will shrink towards the start date#2019-08-1313:54Joe LaneIs that a property of the Rose tree or is it designed explicitly for that?#2019-08-1313:56murtaza52changed it to inst-in and this worked (drop 990 (gen/sample (s/gen ::ks/start) 1000))#2019-08-1313:57Alex Miller (Clojure team)the rose tree is designed to shrink from greater to lesser "size" and inst-in generator builds from start date + int generator (which has its own "size")#2019-08-1313:58Alex Miller (Clojure team)whether or not this is good is a separate question :)#2019-08-1314:00Joe Lanehahah cool, thanks for the insight alex. It's great to read your insideclojure blog, always a joy to see what you're working on.#2019-08-1315:56murtaza52is there a way to merge distinct s/every-kv like s/keys can be done ?#2019-08-1318:14murtaza52I have speced a few fns with :args and :ret, however it throws an error only when the :args does not conform, no error is thrown when the :ret spec does not conform. Any ideas what I am missing ?#2019-08-1318:18seancorfield@murtaza52 Per our thread discussion earlier: instrument checks :args but you need generative checking for :ret (and :fn).#2019-08-1318:19seancorfieldGenerative checking is about testing whether the function has the expected behavior (given generated conforming arguments).#2019-08-1318:19seancorfieldInstrumentation is about checking that other code calls this function correctly during development/testing.#2019-08-1318:25murtaza52thanks @seancorfield#2019-08-1404:12murtaza52just trying to understand why does a empty seq satisfy this spec (s/valid? (s/every integer?) []), shouldnā€™t it fail ?#2019-08-1404:32Alex Miller (Clojure team)because every element in the collection is an integer#2019-08-1404:32Alex Miller (Clojure team)for all none elements#2019-08-1405:19murtaza52could you please expand on none elements, did not grasp it#2019-08-1406:14Alex Miller (Clojure team)(s/every? integer?) specifies a collection where each element is an integer. In this case [] meets that spec, there just happen to be no elements.#2019-08-1406:17murtaza52thanks @U064X3EF3 that explains it#2019-08-1405:22murtaza52When I run (stest/summarize-results (stest/check)), I see the following error printed in my repl - No implementation of method: :specize* of protocol: #'clojure.spec.alpha/Specize found for class: nil all the tests are passing and the :sym names are being printed correctly, however at the end it prints the above too.#2019-08-1406:15Alex Miller (Clojure team)don't know, can't tell from just that#2019-08-1415:17kennyI get this often. Typically it's from forgetting to call s/gen on a spec.#2019-08-1415:20murtaza52There was a typo in my s/fdef association with the sym, once that was corrected, the error disappeared#2019-08-1406:14Alex Miller (Clojure team)if you require a non-empty collection, you can use (s/every? integer? :min-count 1)#2019-08-1421:33ataggartconformer doc states: > takes a predicate function with the semantics of conform i.e. it should return either a (possibly converted) value or :clojure.spec.alpha/invalid And yet defining such a function gives me an error (minimal reproducible example):
user=> (defn f [] :clojure.spec.alpha/invalid)
Syntax error macroexpanding clojure.core/defn at (/private/var/folders/m9/5hxwnkyj0bxf6plprrbf5vzw0000gn/T/form-init9446902011342467351.clj:1:1).
:clojure.spec.alpha/invalid - failed: map? at: [:fn-tail :arity-1 :body :prepost+body :prepost] spec: :clojure.core.specs.alpha/defn-args
:clojure.spec.alpha/invalid - failed: any? at: [:fn-tail :arity-1 :body :body] spec: :clojure.core.specs.alpha/defn-args
Am I missing something silly?
#2019-08-1421:35Alex Miller (Clojure team)defining that function fails the spec for defn itself#2019-08-1421:35Alex Miller (Clojure team)as it has the magic "invalid" value#2019-08-1421:37Alex Miller (Clojure team)a workaround is#2019-08-1421:37Alex Miller (Clojure team)
user=> (def i :clojure.spec.alpha/invalid)
#'user/i
user=> (defn f [] i)
#'user/f
#2019-08-1422:01ataggartThanks!#2019-08-1422:04ataggartI think I have managed to avoid running into this by having predicates before the conformer (surrounded by s/and), thus obviating the need for the forming fn to explicitly yield ::s/invalid.#2019-08-1505:16valerauko>magic šŸ˜¢#2019-08-1507:06Jakub HolĆ½ (HolyJak)Hello, when I run
(require '[clojure.spec.test.alpha :as st])
(alias 'stc 'clojure.spec.test.check)
(st/check `my-awesome-fn
          {::stc/opts {:num-tests 50, :max-size 3}})
I expect the random generated data, whenever there is a collection, to have max three elements (including collections nested in collections, speaking of lists/vectors/maps) but that is not the case. What do I wrong? Thank you!!!
#2019-08-1512:18Alex Miller (Clojure team)What are the specs?#2019-08-1513:02Jakub HolĆ½ (HolyJak)The problematic argument to the function is ::account-invoices, the core of its specs is:
(s/def ::invoice-row-one (s/keys :req [::serviceType ::totalAmount ::period]))
(s/def ::invoiceRow (s/coll-of ::invoice-row-one :min-count 1 :kind sequential?))
(s/def ::invoice-group-single (s/keys :req [::name ::invoiceRow]))
(s/def ::invoiceGroup (s/coll-of ::invoice-group-single :min-count 1 :kind sequential?))
(s/def ::invoice-subscription-one (s/keys :req [::invoiceGroup ::invoiceSubscriptionUser]))
(s/def ::invoiceSubscription (s/coll-of ::invoice-subscription-one :min-count 1 :kind sequential?))
(s/def ::invoice (s/keys :req [::invoiceNumber ::billingDate ::invoiceSubscription ::period]))
(s/def ::acc+invoice (s/keys :req-un [::account ::invoice]))
(s/def ::account-invoices (s/nilable (s/every-kv string? ::acc+invoice))
#2019-08-1513:27Alex Miller (Clojure team)off the top of my head, not sure. those :min-count's may be causing something with size to be set in the constructed generator, not sure#2019-08-1513:29Jakub HolĆ½ (HolyJak)Yes, that seems likely based on the docs. The question is why the generated data is larger than that....#2019-08-1514:00Alex Miller (Clojure team)not something I have time to look at atm, feel free to file a ticket if needed#2019-08-1518:12zlrthis it possible to make a spec for map that has two keys, and if the value of one key matches, then the value of the other has to match something else? but if the value of the one key doesnā€™t match, then the spec is valid, regardless of the second value#2019-08-1519:04Alex Miller (Clojure team)can you write a function that says yes or no to that?#2019-08-1519:04Alex Miller (Clojure team)if so, that predicate is now a spec#2019-08-1519:04Alex Miller (Clojure team)specs are limited only to functions ... which can do anything#2019-08-1519:05zlrthah of course--thanks very much#2019-08-1519:05Alex Miller (Clojure team)perhaps a better questions is whether this is a good idea :)#2019-08-1519:05Alex Miller (Clojure team)ime, you're probably doing too much#2019-08-1519:05Alex Miller (Clojure team)don't strangle the code, just check it#2019-08-1609:16metametadataLooking at the latest blog post about spec2: (s/valid? ::s {::f "Bugs" ::l "Bunny" ::x 10} {:closed #{::s}}). Does this mean that if ::f is also a map then in order to close it the user will have to explicitly put it into set too? I.e.: (s/valid? ::s {::f {:a 1 :b 2} ::l "Bunny" ::x 10} {:closed #{::s ::f}}). If yes, then it seems to be inconvenient for cases with nested maps because when validating a top map I don't want to remember and manually specify the "closed-ness" of all the maps used in validation of every particular key.#2019-08-1610:51Alex Miller (Clojure team)Yes, thatā€™s what it means, but it doesnā€™t preclude some kind of :all setting#2019-08-1611:54metametadataI see, thanks. I was wondering how easy it will be to migrate my current codebase to this approach. As currently the knowledge of closedness is tied to each spec (I have a custom macro: (sp/speced-keys :closed? true :req ...)). And "parent" specs are not aware of this. :all is fine if all the specs are closed but will not work in case some specs are closed and others are open.#2019-08-1612:01metametadatae.g.
::foo = {::a <vector of closed ::x maps> ::b <open ::y map>}
(s/valid? ::foo myfoo {:closed ???})
#2019-08-1612:10Alex Miller (Clojure team)Keep in mind also that s/keys itself is being replaced with s/schema and s/select#2019-08-1612:10Alex Miller (Clojure team)The closed checking is currently implemented only in s/schema, not s/keys#2019-08-1614:26sundbpIā€™ve got this use-case: Iā€™ve got a couple of macros/fns that I use to dynamically create a few ā€œconceptsā€ and associated helper fns (stuff like serialization/deserialization etc). As part of that Iā€™m also creating specs. Iā€™m looking at spec-alpha2 and getting stuck wanting to do something like this: (s/register k (s/spec (fn [x] ..something using a captured local value..) This doesnā€™t fly, the local value (from e.g. an encapsulating let form) ends up not scope captured as Iā€™d expect and instead breaks everything. Any tips how I can accomplish what I need?#2019-08-1614:29Alex Miller (Clojure team)the (fn [x] ...) part here is a symbolic form (not evaluated)#2019-08-1614:30Alex Miller (Clojure team)so if you want to use a local value, you need to do something to construct the right symbolic form#2019-08-1614:30Alex Miller (Clojure team)you could for example use
`( ... ~foo ... )
#2019-08-1614:33sundbpah. ok. iā€™ll give that a go. the symbol stuff still doesnā€™t really compute very well with me - it feels like even though spec2 allows being driven by code (as opposed to manual s/defā€™s), I still donā€™t find that Iā€™m able to do so doing things the way I expect. The macros still ā€œget in the wayā€ for me a lot of the time.#2019-08-1614:35Alex Miller (Clojure team)there are no macros involved in your code above. how can they be in the way?#2019-08-1614:35Alex Miller (Clojure team)well I guess s/spec#2019-08-1614:35sundbpso I tried (s/register k (s/spec* (fn [x] .. ~foo ..))` - no cigar.#2019-08-1614:36sundbpMeant the spec2 macros#2019-08-1614:36Alex Miller (Clojure team)can you share more of your code?#2019-08-1614:37sundbpthe 2nd register is what Iā€™m struggling with.#2019-08-1614:38Alex Miller (Clojure team)what is entity?#2019-08-1614:39Alex Miller (Clojure team)I guess a symbol? kw?#2019-08-1614:40sundbpentity is a kw#2019-08-1614:42Alex Miller (Clojure team)and in how does this not work? error or spec doesn't do what you expect?#2019-08-1614:43Alex Miller (Clojure team)is there some reason you're using s/spec and not s/spec* in last line?#2019-08-1614:43Alex Miller (Clojure team)I think you'd want the latter there#2019-08-1614:44sundbp#2019-08-1614:45Alex Miller (Clojure team)try spec* in that last line#2019-08-1614:45sundbp#2019-08-1614:46sundbp(sorry, for screenshots - I donā€™t have slack access on a remote machine where the code is and canā€™t copy across)#2019-08-1614:50Alex Miller (Clojure team)that's actually a bug in spec 2 function explication#2019-08-1614:50Alex Miller (Clojure team)the [x] in the fn is getting ns'ed#2019-08-1614:50sundbpthank god for that - I was starting to run out of ideas šŸ™‚#2019-08-1614:51Alex Miller (Clojure team)well, maybe#2019-08-1614:51sundbpcan I work around that?#2019-08-1614:51Alex Miller (Clojure team)in that fn, instead of x can you do ~'x everywhere?#2019-08-1614:52Alex Miller (Clojure team)I'm not sure if the error here is in your expansion or inside spec 2#2019-08-1614:53sundbpcool. that compiles at least, so hoping that means Iā€™m on my way. wont be able to confirm overall functionality without some more work.#2019-08-1614:53Alex Miller (Clojure team)basically you're fixing this issue:#2019-08-1614:53Alex Miller (Clojure team)
user=> `(fn [x] (+ x 1))
(clojure.core/fn [user/x] (clojure.core/+ user/x 1))
#2019-08-1614:54Alex Miller (Clojure team)I have seen places where spec 2 has trouble with this, but here I think it's in your expansion#2019-08-1614:54sundbpmore context#2019-08-1614:55Alex Miller (Clojure team)the code you now have is not using any macros and there is no "magic" here - it's just passing forms to functions#2019-08-1614:57sundbpI think that macro+adapt-spec wrapper was a trial and error solution to similar issue I had with passing in a fn (e.g. string?). if I already had the fn and not the symbol pointing to the fn at the time I want to s/register I couldnā€™t work out any way to proceed.#2019-08-1615:10sundbpbtw, that seems to get me back on track and define the sort of spec I have in mind. thanks a lot for the help - and let me know if you want any more info in terms of it being me vs spec2.#2019-08-1615:46Alex Miller (Clojure team)if you already have a function object, then you can't go through the symbolic API (spec*)#2019-08-1615:46Alex Miller (Clojure team)as it's not a symbol#2019-08-1615:46Alex Miller (Clojure team)but you can directly create a spec object that implements the protocol#2019-08-1615:47Alex Miller (Clojure team)w/the caveat that you're not going to be able to s/form or nest that kind of thing in other symbolic forms#2019-08-1615:47Alex Miller (Clojure team)which is true to some degree in spec 1 too#2019-08-1616:29sundbpyeah. compared to a more data driven API both spec1 and spec2 throws up some speed bumps if one is generating specs by code. but spec2 seems to have enough flex to make it possible to get what I need even if itā€™s not yet intuitive to me to work in the symbolic space (as compared to more data composition).#2019-08-1616:29sundbpso Iā€™m a bit split on the API so far. it doesnā€™t feel like ā€œnormalā€ clojure when one comes at it from generating specs in code. however, it does compose fine as long as youā€™re manually specā€™ing things. net-net Iā€™d love a more data-driven approach. Iā€™ve seen https://github.com/metosin/spec-tools/blob/master/docs/02_data_specs.md and perhaps what Iā€™m looking for is doable on top of the symbolic api.#2019-08-1617:30Alex Miller (Clojure team)well there is now a map-focused format as well (as of last week)#2019-08-1617:30Alex Miller (Clojure team)that is almost certainly going to change formats so I wouldn't touch it yet#2019-08-1617:33Alex Miller (Clojure team)
{:clojure.spec/op `s/keys
 :req (conj (map first required) ::entity-type)
 :opt (map first optional)}
#2019-08-1617:33Alex Miller (Clojure team)is something you could pass to s/spec* for example and get the identical spec#2019-08-1617:34Alex Miller (Clojure team)and there is an s/expand-spec that can do form->map#2019-08-1617:34Alex Miller (Clojure team)but this should be much more amenable to doing spec->map->transform->spec type things#2019-08-1617:37Alex Miller (Clojure team)there are now 3 or 4 different ways to tap into spec creation/transformation and we're looking at some meta stuff on top of that which might make some of them more accessible, but really there's a lot of options now#2019-08-1820:33sundbp@alexmiller cool. yeah, Iā€™m not saying there isnā€™t ways to hook things up. For manual use, which obviously is the major use case, things have felt natural and I havenā€™t had to read source etc. For manipulating specs programatically Iā€™m convinced all I want is doable, but it hasnā€™t felt ā€œnaturalā€ and Iā€™ve had to read the implementation to try to grok the explicate/spec/spec* machinery. The map syntax you hint at looks likely to be more intuitive to me, so I like that. Thanks for both helping me across my stumbling block and for keeping on improving spec!#2019-08-1823:28cflemingWhat are the current workarounds for closed spec-like behaviour? Iā€™m trying to retrofit specs to some existing code, and Iā€™d like to ensure that I havenā€™t missed any cases during development.#2019-08-1823:28cflemingIn particular, how can I specify a map that should only have the keys Iā€™ve specified?#2019-08-1823:46taylorhereā€™s an example https://github.com/gfredericks/schpec/blob/b2d80cff29861925e7e3407ef3e5de25a90fa7cc/src/com/gfredericks/schpec.clj#L13#2019-08-1823:49cflemingAwesome, thanks!#2019-08-1823:42cflemingAlso, Iā€™d like to spec a map which has some standard keyword keys, but also might have string keys with values specified by a spec. Can I do that? Neither s/keys nor s/map-of seem to fit that.#2019-08-1823:47Alex Miller (Clojure team)You can, but not easily. You can do an s/coll-of :kind map? and then spec the possible mapentry types#2019-08-1823:48Alex Miller (Clojure team)Itā€™s tedious#2019-08-1823:50Alex Miller (Clojure team)Another path (if you care less about gen) is to use a conformer to keywordize the map then s/keys#2019-08-1823:50cflemingYeah, I donā€™t care about gen in this case.#2019-08-1823:50cflemingJust trying to spec an existing system.#2019-08-1900:07cflemingThis seems to work:
(s/def ::entry (s/or :project (s/cat :file string? :data ::project)
                     :profiles (s/cat :key (s/and keyword? #(= (name %) "profiles"))
                                      :data (s/coll-of string? :kind vector?))))
(s/def ::projects (s/coll-of ::entry :kind map?))
#2019-08-1900:07cflemingNot pretty, but it does the trick.#2019-08-1900:24metametadata@cfleming there's a support for closed maps in spec-tools (https://github.com/metosin/spec-tools/blob/master/CHANGELOG.md#092-alpha1). Also I posted my take a while back: https://gist.github.com/metametadata/53a847cd3b02056e8e4c124e63d9ae5a.#2019-08-1900:26cfleming@metametadata Thanks! Iā€™ll try yours out since it looks like the error messages might be more helpful.
#2019-08-1900:32metametadatacool! we use an evolved version in company projects (which e.g. is more convoluted because of https://github.com/metosin/compojure-api/issues/417), but didn't have time to publish it yet#2019-08-1914:32NickIā€™m trying to write a spec for a payload wrapper map. For simplicity, the spec Iā€™m trying to write is for a map with two keys, :query and :payload. For the purposes of this example, Iā€™m using extremely simple examples of the payload with just maps of one key. ` (s/def ::query (and keyword? #{:avalue :anothervalue :thirdvalue})) (s/def ::email string?) (s/def ::user-id integer?) (s/def ::avalue (s/keys :req-un [::email])) (s/def ::anothervalue (s/keys :req-un [::user-id])) {:query :avalue :payload {:email ā€œ<mailto:/cdn-cgi/l/email-protection|/cdn-cgi/l/email-protection>ā€ }} {:query :anothervalue :payload {:user-id 123}} ` I can use multi-spec and a defmulti to key off the :query object, but that doesnā€™t let me use the shared :payload keyword. Is the only way around this to define ::avalue and ::anothervalue in different namespaces ::avalue/payload ::anothervalue/payload? I havenā€™t created a lot of namespaced keywords in clojure yet.#2019-08-1914:39Alex Miller (Clojure team)yes, you would need to use different namespaces in this case#2019-08-1914:39Alex Miller (Clojure team)the ::query spec is redundant btw, just (s/def ::query #{:avalue :anothervalue :thirdvalue}) would be sufficient#2019-08-1914:43NickDo I need to create a separate valid namespace, or can I define the specs with a a namespace that doesnā€™t correspond to an existing clojure file? ie, does there need to be a (ns anothervalue) somewhere before I can do a (s/def ::anothervalue/payload string?)#2019-08-1914:51Alex Miller (Clojure team)technically here, it's better to use the term "qualifier"#2019-08-1914:52Alex Miller (Clojure team)which may correspond to an actual namespace, but doesn't have to#2019-08-1914:52Alex Miller (Clojure team)you can use a keyword with any qualifier#2019-08-1914:54NickThanks, that's quite helpful.#2019-08-1914:55Alex Miller (Clojure team)a lot of docs in Clojure are not particularly clear on this point#2019-08-2019:29johanatanis this guide still the latest info on spec? https://clojure.org/guides/spec#2019-08-2019:29johanatanhave things changed sufficiently that one should follow a different guide?#2019-08-2019:30johanatanit still seems almost identical to ~1.5-2 years ago when I first used spec and I thought things were changing dramatically for v2#2019-08-2019:30Alex Miller (Clojure team)v2 has not been released yet, so that is still up to date for v1#2019-08-2019:31johanatanhm, are there certain things we could do to "future proof" our specs in prep for v2 ?#2019-08-2019:31Alex Miller (Clojure team)given how things are in flux, I would not worry about it much yet#2019-08-2019:31Alex Miller (Clojure team)but you can read more about v2 at https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha and https://github.com/clojure/spec-alpha2/wiki/Schema-and-select#2019-08-2019:31johanatanok, thanks!#2019-08-2121:02johanatani have the following deps in my deps.edn:
{:deps {org.clojure/clojure {:mvn/version "1.10.0"}
        org.clojure/clojurescript {:mvn/version "1.10.520"}
        ...
...
and yet i'm getting the following errors when trying to use clojure.spec.test.alpha:
Use of undeclared Var clojure.spec.test.alpha/instrument

   5              [clojure.spec.alpha :as s]
   6              ;[clojure.spec.test.alpha :as stest]
   7              [clojure.test.check.generators :as gen]))
   8  
   9  (defn instrument-all-syms []
  10    (clojure.spec.test.alpha/instrument (clojure.spec.test.alpha/instrumentable-syms)))
         ^---
from figwheel-main
#2019-08-2121:02johanatanUncommenting line 6 causes a cannot resolve dependency error#2019-08-2121:02johanatanis there something else that has to be done to get clojure.spec.test.alpha to be available?#2019-08-2121:03johanatanlooks like this: https://cljs.github.io/api/cljs.spec.test.alpha/instrumentable-syms indicates that it should be there#2019-08-2121:04johanatan[i have tried subbing cljs for clojure as well]#2019-08-2200:08seancorfield@johanatan Is there a reason you're calling instrumentable-syms there, rather than just (stest/instrument)? The latter seems to work with figwheel, based on my limited testing, but the former seems to behave strangely (based on my very limited testing).#2019-08-2201:57johanatan@seancorfield it was just copypasta from a previous project. it was the first thing that worked. no particular reason.#2019-08-2201:57johanatandoes (stest/instrument) have the same behavior as I'm attempting to get there?#2019-08-2202:24seancorfieldIt instruments all functions... which should be the same as all instrumentable-syms but maybe @alexmiller can shed light on that?#2019-08-2202:54johanatan[btw, that change did get my code to compile, strangely]#2019-08-2320:11johanatanis there a way to ask spec: "give me all embedded values in the nested map satisfying spec ::x?" (either in the std alpha or in 3rd party extensions). I'm thinking it wouldn't be too hard to do this with specter + spec but just wondered if something like it already exists...#2019-08-2320:20Alex Miller (Clojure team)not in spec#2019-08-2320:20Alex Miller (Clojure team)but it's just something like (defn deep-match [s x] (->> x (tree-seq coll? seq) (filter #(s/valid? s %))))#2019-08-2320:20johanatanšŸ™‚ yea. cool, thx#2019-08-2420:07tylerAre unqualified keywords not supported for closed specs? Closed specs seem to work as expected with namespaced keyword specs but the unqualified keyword form doesnā€™t seem to work with closed.#2019-08-2420:14johanatanare you referring to s/keys req-un or opt-un specifically? are there other aspects of specs that support "unqualified" ?#2019-08-2420:26tylerRefering to the examples here https://github.com/clojure/spec-alpha2/wiki/Schema-and-select#literal-schemas the unqualified form doesnā€™t seem to be recognized by the closed option.#2019-08-2420:26tylerThis is using schema, not s/keys#2019-08-2420:33johanatanAh, specs2 gotcha . Specs1 didnā€™t support unqualified except via keys#2019-08-2420:58Alex Miller (Clojure team)No, not supported right now#2019-08-2615:33unbalancedokay so I'm doing this pattern a lot and I'm wondering if there's a more concise way to do it, perhaps to register a s/def with another s/def at creation time?
(s/def :platform.confirm/display-attribute keyword?)
(s/def :platform.confirm/temp-attribute string?)
(s/def :platform.confirm/id string?)
(s/def :platform.confirm/value string?)
(s/def :platform.confirm/edit-attribute keyword?)

(s/def ::confirm (s/keys :req [:platform.confirm/display-attribute
                               :platform.confirm/temp-attribute
                               :platform.confirm/id
                               :platform.confirm/value
                               :platform.confirm/edit-attribute]))
#2019-08-2615:34unbalancedand, while I'm at it ... I'm wondering if anyone has any best practices in combining spec with datomic? Since we already go to all the trouble of making the schema for datomic, has anyone looked into using that for spec?#2019-08-2617:26seancorfield@goomba This popped up as the second result on a Bing search for using clojure.spec to create datomic schema for me: https://github.com/Provisdom/spectomic -- seems to be what you're looking for?#2019-08-2617:27unbalancedoh man, and great name too#2019-08-2617:28unbalancedthat's interesting ... they went the opposite way I would've expected, i.e., s/def -> schema#2019-08-2617:28unbalancedI would've expected schema-ish -> s/def#2019-08-2617:29seancorfieldThat makes sense to me. We treat spec as the "system of record" and generate stuff from it.#2019-08-2617:29unbalancedmakes sense. I'm usually just paranoid about "losing" my schema.#2019-08-2617:30unbalancedI like to see them all written out explicitly. But that would be easy enough with eval-and-replace#2019-08-2617:30seancorfieldI recently answered a Quora question about using clojure.spec https://twitter.com/seancorfield/status/1164714132647530496 (don't know if you can read the answer without a Quora account?)#2019-08-2617:30kenny@goomba clojure.spec can express more info than Datomic schema can so it makes more sense to go from spec -> schema.#2019-08-2617:30unbalancedwheeee yess! That was another question I had!#2019-08-2617:31unbalanced@kenny solid point. I was just expecting something more like a hybrid spec/schema, uh... spec#2019-08-2617:31unbalancedhave it all in one place#2019-08-2617:32unbalancedthis is totally fine though#2019-08-2617:33unbalancedalso got a lot of good answers out of it and experienced @seancorfield do a "let me Bing that for you" which is fantastic šŸ˜†#2019-08-2617:35seancorfieldI hope you didn't think I was being mean with my comment about Bing? You've said you're sometimes not finding search results for stuff so I figured sharing the search string I used might be helpful. Quite a few of my friends ping me about finding stuff because apparently I'm better at searches than them... šŸ™‚#2019-08-2618:20unbalancedno hahaha I thought it was hilarious šŸ˜„ I loved it#2019-08-2617:33Alex Miller (Clojure team)I've seen people do both directions, and also start from a 3rd "source of truth" and generate the Datomic schema and spec too#2019-08-2617:33unbalancedmmm chimeratomic#2019-08-2617:39kennyAttaching schema data directly to the specs is something that interested us too @goomba. From what I remember when we wrote that lib, it wasn't immediately obvious what the best path to doing that would be. Ideally it would've been something simple like attaching metadata to the spec's name (e.g. (def ^{:db/isComponent true} my-spec map?)). That wouldn't work for specs because they are typically defined as keywords which do not accept metadata.#2019-08-2617:42kennyThere were 2 other options IIRC. 1) Create a new s/def which accepts a new 3rd parameter. 2) add a function similar to s/with-gen which would attach metadata to the spec itself.#2019-08-2617:42unbalancednod... in my lazy imagination it would be something like
{:db/ident       :boot.user/string
 :db/valueType   :db.type/string
 :db/cardinality :db.cardinality/one
 :spec/id        ::string
 :spec/spec      string?}
and then specs and schema would be gen'd from that. But listen I'm a lazy application layer dev asking for handouts, what you did is great it's just not what I would've thought of immediately
#2019-08-2617:43kennyI mean, that's basically a complete schema map.#2019-08-2617:43unbalancedI'm not even sold on the :spec/spec with a function there because now that's not even a value anymore#2019-08-2617:44unbalancedyeah I'm excited to check out spectomic!#2019-08-2617:44unbalancedI do a ton of datascript/spec work so this is great#2019-08-2617:49kennyI haven't looked at this at all but I'm assuming that this sort of thing would be easier to do with Spec2, given Spec2 allows specs to have docstrings.#2019-08-2619:18ikitommihello all. is there a reason that pos? and neg? donā€™t have built-in generators?#2019-08-2619:28Alex Miller (Clojure team)yes, they are not specific to one numeric type#2019-08-2619:28Alex Miller (Clojure team)use (s/and int? pos?) or whatever numeric type you need#2019-08-2619:39ikitommiok, thanks!#2019-08-2619:51Alex Miller (Clojure team)or something like pos-int?#2019-08-2715:44joefromctHi, Iā€™ve read the spec guide and couldnā€™t really come up with the correct way to do the following; Apologies if i had missed something. I am looking to define a spec for a ::paragraph that consists of ā€œ::wordsā€ , ā€œ::space(s)ā€ , and ā€œless oftenā€ ::punctuation. It (of course) does not need to be grammatically correct. It seems like i need something like s/cat however i want the results to be not a vec but a larger string (that consists of smaller ā€œspecā€ strings...ā€œ). Iā€™m interested in having a spec for ::paragraph as well as a generator to play with a natural language processing app iā€™m working on. Any tips appreciated,#2019-08-2820:03ataggartI managed to do something like that by using conformer to tokenize the string first. E.g.:
(s/and string?
       (s/conformer #(str/split % #",") #(str/join "," %))
       (s/cat ...))
But it wonā€™t be as fast as a real parser.
#2019-08-2820:04ataggartTyped that on my phone from memory, so apologies if I got things incorrect. #2019-08-2715:57Alex Miller (Clojure team)I think youā€™re probably asking for more out of spec than it is designed to provide #2019-08-2716:01joefromctok, that makes sense. thanks, iā€™ll come up with something.#2019-08-2720:37vemvIs there a version (in some library, etc) of s/keys that allows me do manipulate the spec definition before proceeding? My current pattern for that is:
(eval `(spec/keys :req ~(conj dependencies ::foo-options)))
...Generally it works fine, but now my needs also include clojurescript, so I can't use eval
#2019-08-2721:57Alex Miller (Clojure team)I don't think there is an easy pattern for it in spec 1. CLJ-2112 has a sketch of specs for specs which lets you conform, modify then unform, but it's pretty cumbersome due to the structure of s/keys.#2019-08-2721:58Alex Miller (Clojure team)spec 2 has several ways to do this and a new symbolic map form in work in particular#2019-08-2723:16vemvAppreciate the confirmation, thanks!#2019-08-2721:38johanatanwhat would the regex to match "one or more forward slashes" be? #"\/+" doesn't seem to compile or work#2019-08-2721:38johanatanalso tried several other permutations#2019-08-2721:40johanatanthis seems to have worked: (js/RegExp. "\\/+")#2019-08-2721:40johanatanbut pretty sure #"\\/+" did not#2019-08-2721:54jahson
user=> (re-seq #"\/+" "///////asd////")
("///////" "////")
It definetely compiles.
#2019-08-2804:44johanatanhuh? brain fart apparently. this works:
cljs.user=> (clojure.string/replace "/////blah/blah/blaz///" #"\/+" "/")
"/blah/blah/blaz/"
cljs.user=> 
#2019-08-2818:12johanatanhmm, ^^ that was in lumo.#2019-08-2818:12johanatanwhen it's in my source file and compiled by figwheel-main I get the following js error:
Uncaught SyntaxError: missing ) after argument list
#2019-08-2818:12johanatanso, yea, a compilation error apparently#2019-08-2818:14johanatanand i just realized this is the very wrong channel for this topic lol#2019-08-2818:14johanatan[sorry about that]#2019-08-2819:37chancerussellI noticed that alpha2 supports s/def ing symbols instead of keywords and got really curious about itā€¦then realized that the first version supports the same thing#2019-08-2819:38chancerussellReally curious about the use case for that though#2019-08-2819:40Alex Miller (Clojure team)that's how function specs are registered (s/fdef)#2019-08-2819:40chancerusselllol duh#2019-08-2819:40chancerussellthanks#2019-08-2822:46agcan someone throw at me with an example of a spec for a string that has to be of specic length (within range) and should only allow letters. The catch though: it should not fail generation with Couldn't satisfy such-that predicate message. I swear, I used to have those, somehow Iā€™m feeling stupid and canā€™t make them anymore.#2019-08-2822:51Alex Miller (Clojure team)you could use test.chuck's support for making generators from a regex#2019-08-2822:54agah test.chuck. Yeah it has a few nice gems there. But I feel Iā€™m missing something fairly small#2019-08-2822:59agmy best effort for now is to use with-gen with gen/such-that with max-tries jacked-up to a huge number. But it seems to be slow and still not guaranteed to not to fail to satisfy#2019-08-2823:05seancorfield@ag Another "vote" for test.chuck here since you can use a (string) regex such as #"[a-zA-Z]{1,32}" for example, for pure alpha strings up to 32 chars in length and test.chuck provides a generator that works out of the box. We use it to generate email addresses and a bunch of other stuff.#2019-08-2823:11seancorfieldFYI https://www.dropbox.com/s/tvvby5apbzyra7j/Screenshot%202019-08-28%2016.11.34.png?dl=0#2019-08-2823:14agLOL#2019-08-2823:12agOn related note: Gary is awesome! He was at defn podcast (that I started listening but couldnā€™t finish this morning)#2019-08-2920:06joefromctis there a way to make a recursive spec to generate the output from clojure.data.xml ?#2019-08-2920:07joefromctnot sure what iā€™m doing wrong there#2019-08-2920:09Alex Miller (Clojure team)you might try limiting the collection sizes#2019-08-2920:10Alex Miller (Clojure team)(s/def ::attrs (s/map-of keyword? string? :gen-max 3))#2019-08-2920:10Alex Miller (Clojure team)(s/def ::content (s/coll-of ::a-content :kind vector? :gen-max 3))#2019-08-2920:11Alex Miller (Clojure team)oh, it's probably that the unqualified keys are foiling the recursion limiter#2019-08-2920:11Alex Miller (Clojure team)try it with :req and see if that works#2019-08-2920:14joefromcthmm.. no luck with req and gen-max 3
(s/def ::tag       keyword?)
(s/def ::attrs (s/map-of keyword? string? :gen-max 3))
(s/def ::a-content (s/keys :req [::tag ::attrs ::content] ))
(s/def ::content (s/coll-of ::a-content :kind vector? :gen-max 3))

(->> ::content
     s/gen
     gen/generate
     (binding [s/*recursion-limit* 1]))

;;=> Execution error (StackOverflowError) at clojure.test.check.generators/tuple
;;(generators.cljc:534).
#2019-08-2920:27Alex Miller (Clojure team)We have some open tickets about SO on recursive gen. I have not dug into any of them recently#2019-08-2920:33joefromctok sounds good, iā€™ll keep an eye out.#2019-08-2920:34joefromcti appreciate how you always try to respond when iā€™m sure your busy with strange loop very soon.#2019-08-2920:34joefromcti hope i donā€™t miss it this year because iā€™m stuck in a tube in the museum. j/k#2019-08-2920:35Alex Miller (Clojure team)me too :)#2019-08-3005:10johanatandoes spec-alpha2 support clojurescript?#2019-08-3012:23Alex Miller (Clojure team)No, we will probably hold off on that till churn slows down #2019-08-3019:25johanatanthx#2019-08-3008:31hkjelsGenerating from the spec: (s/inst-in #inst "1984" #inst "2019") will just return dates from 1984 Whatā€™s the idiomatic way of generating random dates within a range? #inst only works with string literals and not computed strings#2019-08-3012:24Alex Miller (Clojure team)Samples are biased to beginning of the range, but ā€œgrowā€ as you make more#2019-08-3012:24Alex Miller (Clojure team)So if you sample a larger number youā€™ll see more variety #2019-08-3010:06mpenet(java.util.Date. x) with some random number within the range#2019-08-3010:47hkjelsproblem was actually the string I created. Your absolutely correct @mpenet#2019-08-3120:30seancorfield@j.m.frith Can you share the spec and the function that you're having problems with? (perhaps in a cut down version if it's a lot of code)#2019-08-3120:32JazzerThanks @U04V70XH6 I'm just working something up. In the actual code, the problem bit is fairly deep in a map, so I'm going to try to simplify#2019-08-3120:32JazzerBack in a few minutes...#2019-08-3120:31JazzerHi everyone, I'm getting issues when running stest/check while a function is instrumented. It seems to be conforming the input before passing it to the function. I'll try and condense my code into a minimal example...#2019-08-3120:31JazzerHi everyone, I'm getting issues when running stest/check while a function is instrumented. It seems to be conforming the input before passing it to the function. I'll try and condense my code into a minimal example...#2019-08-3120:37JazzerNuts! My minimal example has no problems#2019-08-3120:37JazzerI'll try expanding the code until I see an issue#2019-08-3120:39JazzerThe symptoms I'm seeing are: if I run (stest/unstrument) and then (stest/check 'my-fn) (not sure how to write a backtick in code but my-fun is quoted with a backtick not a single quote) I get tests all passing#2019-08-3120:41JazzerIf instead I run (stest/instrument) followed by (stest/check 'my-fn) then I get errors that the input has not conformed to spec.#2019-08-3120:42Alex Miller (Clojure team)Could you just share the error?#2019-08-3120:45Jazzer#2019-08-3120:46JazzerI'm not sure that's quite as helpful as it might be, which is why I'm trying to scale down to get to the root of the problem#2019-08-3120:47JazzerThere's a whole bunch of data in there which isn't very helpful#2019-08-3120:49JazzerAlthough right at the front of line 4 it's the part [:decision #:log{:decision {}}] this is definitely a conformed value - the spec has an (s/or ...) with :decision being one of the options#2019-08-3120:49seancorfield@j.m.frith Are you sure that function works with instrumentation on when called manually from the REPL?#2019-08-3120:50JazzerOne moment, will re-try now#2019-08-3120:51JazzerIt's working fine for me now#2019-08-3120:51JazzerAnd it gives and returns a map as expected without the conform path#2019-08-3120:53JazzerHave also just checked that (gen/generate ...) gives me a map that conforms fine to the spec.#2019-08-3120:54JazzerI think it'll be easiest if I take a bit of time to whittle away all the nonsense so that things are a bit easier to follow.#2019-08-3120:57seancorfieldOne approach that I strongly recommend when working with Spec is to write and check each spec as you write each function -- that way you don't get a bunch of code for which you are trying to write a bunch of specs all at once and then testing all at once. The REPL allows you to take small steps and get feedback immediately on each piece of code as you write it.#2019-08-3120:59seancorfieldFor example, I have a (comment ,,,) block in each file where I'm working on code + spec and I have (s/exercise ,,,) calls in that comment that I eval to check each spec will generate (not all specs need to generate but I think it's important to check the ones that should as you write them).#2019-08-3120:59seancorfieldThat "Rich Comment Form" also includes instrument and check calls as needed.#2019-08-3121:01JazzerThat is definitely good advice, which I should have taken before getting to this point. I was too eager to learn the language first and built a working prototype and am now going back to learn how to use spec.#2019-08-3121:02seancorfieldAlso bear in mind that "nothing everything needs a spec" šŸ™‚ Spec is great for boundaries and for "key functions" that you really need to be sure of. Spec isn't a type system, so you don't need a spec on every function.#2019-08-3121:03seancorfieldAt work, we've used Spec since the first alpha builds in the 1.9 cycle and we mostly spec data, rather than functions. We use s/conform and s/invalid? in production code (and s/explain-data to help drive error message generation).#2019-08-3121:04JazzerI guess I have a "base layer" of functions which directly interact with my state-map (in a functional way) and I'm currently only speccing those functions. The higher layers I may get to at some point#2019-08-3121:04JazzerAnd the spec is mostly so that I can run generative tests on that portion of the code#2019-08-3121:05seancorfieldWe use instrument to support development work, mostly. We use check sparingly.#2019-08-3121:05JazzerThe problem is that one part of the system now has an (s/or in the spec and that is throwing off the earlier function which worked alright when I didn't have that in the spec...#2019-08-3121:07JazzerAgain, that sounds about right. I was really using check just to try and find edge cases. Maybe I should stick to either using instrument or check at a time. I'm getting no problems at all when using one but not the other#2019-08-3121:08seancorfieldI'll be interested to see the cut down example that breaks because of s/or -- I've never run into that (in several years of working with Spec).#2019-08-3121:12JazzerI'll work it out. It seems even stranger at the moment because the spec is actually saying it's failing on a function that shouldn't even be called. It's failing on game/player-count and I'm checking game/set-lead-player. set-lead-player has two lines, neither of which call player-count. It looks like I've got a bit of digging around ahead of me... I'll be back in the next few days once I've simplified things down#2019-08-3121:14seancorfieldIf you instrument a function that takes a function as an argument, and you have a fspec for that argument, it will be generatively tested when it is called.#2019-08-3121:15seancorfield(that sometimes surprises people)#2019-08-3121:16JazzerI don't think that is the case here, but that is good to know. I have to head out now, but I'll be back šŸ™‚#2019-08-3121:17JazzerThank you for your time - I have to say that I'm impressed both by clojure and the community here#2019-08-3123:46JazzerI have got almost to the bottom of this! The issue was that I was calling a function within the spec definition, and not realising that spec passes a conformed value into whatever you give Code looks like this:
(s/fdef game/set-lead-player :args (s/and
                                     (s/cat :game :game/game
                                            :lead-player :player/player-no)
                                     #(<= (:lead-player %) (game/player-count (:game %))))
                             :ret :game/game)
I hadn't realised that (:game %) returns a conformed value of the argument passed in, rather than just passing on the value itself
#2019-08-3123:49JazzerTwo questions would follow from this: 1. Is there any way to use a function in this way and pass in the original argument, rather than a conformed value 2. Should I in fact be avoiding this conflation of spec and function? i.e. any specs should depend purely on the data they are presented (and core clojure functions)#2019-08-3123:53JazzerAs to my question 1 above, the answer is s/unform. Worked perfectly with that in there. Yay! Answered my own question šŸ˜›#2019-09-0100:24seancorfieldWhen you're writing compound predicates like that, it's expected that you take into account the shape of the conformed value -- calling s/unform is kind of a sledgehammer...#2019-09-0100:25seancorfieldI'm a bit surprised you need a function to get the player count from a game tho'... isn't that just an element in the game's map? Or are you calculating it on every call from something inside the game map?#2019-09-0100:28seancorfieldBut your :game/game spec has s/or at the top-level? What are the alternatives there? (sorry if you posted it elsewhere, I'm only looking at this thread right now)#2019-09-0111:39JazzerThanks @U04V70XH6. Agreed that s/unform seems overkill, I just thought Iā€™d share that it does indeed achieve my (probably misguided) aim of being able to use the original argument rather than a conformed version.#2019-09-0111:41JazzerThe reason for the function is that, as I developed things, I kept changing up the structure of the map. Rather than having to change everywhere in the code which refers to this element, I made a function to pull it out and then only need to update that one method if/when I changed my mind again.#2019-09-0111:46JazzerAnd no, the :game/game spec at the top level is just s/keys. One such key is :game/log which is a s/coll-of :log/log-entry :kind vector? and finally :log/log-entry is where the s/or happens to choose between various different types of log entry#2019-09-0111:47JazzerSome log entries are for player actions, others are for rules being applied and each has a different structure.#2019-09-0312:04Vesa NoriloHi guys! Is there a way to destructure and walk specs? Suppose I have (update-in document [some path] transform). Is there a way to derive a spec for transform output by reaching into the spec for document?#2019-09-0312:28Alex Miller (Clojure team)Not right now#2019-09-0312:53Vesa Noriloright, thanks!#2019-09-0418:01johanatani have a function from ::a -> ::b and i am testing it with stest/check. i have set :num-tests to 1 via: (stest/check 'the-sym {:clojure.spec.test.check/opts {:num-tests 1}}). does it make sense that i would see two ::as generated?#2019-09-0418:03johanatan[i've verified that both generated ::as are non-trivial/non-empty so it shouldn't be stest peeking at the first and considering it "not good enough" then proceeding to the second].#2019-09-0418:07johanatanhmm... i guess there could be a hidden such-that or some other "retry" mechanism behind the scenes.#2019-09-0418:07johanatanhmm... i guess there could be a hidden such-that or some other "retry" mechanism behind the scenes.#2019-09-0418:07taylorgot a small repro example?#2019-09-0418:13Alex Miller (Clojure team)you may see it gen more than once#2019-09-0418:14Alex Miller (Clojure team)even in a single test#2019-09-0511:28pvillegas12I have a spec which reuses another, something like
(s/def ::a
  (s/and #(s/valid? ::sub-a %)
  (s/keys :req [:a/only-attribute]))
#2019-09-0511:29pvillegas12However, my problem is that s/explain does not show the ::sub-a spec violations, it just says that ::sub-a is invalid#2019-09-0511:29pvillegas12is there a way for spec to ā€œwalkā€ down ::sub-a in this case?#2019-09-0511:33pvillegas12I could, of course, copy over the (s/keys) in ::sub-a to the spec of ::a but I was looking for a solution that reuses the ::a declaration#2019-09-0512:01Alex Miller (Clojure team)Right now, this is what I would recommend. Much better options coming for this in spec 2#2019-09-0512:10pvillegas12I actually got good error messages by doing
(s/def ::a
  ::sub-a
  (s/keys :req [:a/only-attribute]))
#2019-09-0512:10pvillegas12It this bad @U064X3EF3?#2019-09-0512:22Alex Miller (Clojure team)That looks invalid, what are you seeing?#2019-09-0512:26pvillegas12Iā€™m actually getting the error messages for ::sub-a should that not work?#2019-09-0512:41pvillegas12@U064X3EF3 got inspiration from https://gist.github.com/robert-stuttaford/e1bc7bfbcdf62020277dda1a277394ca#2019-09-0512:41pvillegas12https://gist.github.com/robert-stuttaford/e1bc7bfbcdf62020277dda1a277394ca#file-keys_with_and_question-clj-L13#2019-09-0511:48Leonhow can i describe a map where a key could either contain a normal value ::some-object or, in an error-case, an error-value? I don't want to add the error-case to ::some-object as that would greatly decrease the expressiveness and reliability,...#2019-09-0511:59Alex Miller (Clojure team)Spec is about expressing the true range of values. If it can contain special error values, then that is part of that truth #2019-09-0512:00Alex Miller (Clojure team)If thatā€™s weird, you might consider using an alternate attribute for the error or using exceptions#2019-09-0512:03Leonexceptions are an extremely ugly solution (at least for me, coming from ADT Either-type land)... and my data cannot contain errors, but the place they're transmitted can. i guess i'll create a wrapper object that can be either of type success or of type failure, and then contain the values respectively#2019-09-0512:04Alex Miller (Clojure team)Exceptions are the idiomatic solution on the jvm and in Clojure#2019-09-0512:06Leoni guess thats an idiom i dislike ^^ i really hate the complexity that braking out of my control flow adds, especially when otherwise working with pure functions that are easy to reason about ;D and in my case exceptions aren't really a simple option, as im working in clojurescript with re-frame, where im dispatching events and then dispatching other events on success#2019-09-0512:49potetmProblem is: if you go down the path of ā€œexceptions are bad,ā€ youā€™re fighting not just the clojure ecosystem, but the Java(Script) one. Leveraging host libs is one of the primary reasons clojure(script) exists at all!#2019-09-0512:51potetmNot to say you canā€™t write clj(s) w/o host libs. Many try. But know that youā€™re giving up a primary value-add for the lang.#2019-09-0512:53potetmMany have also tried to improve upon exceptions (e.g. slingshot). Iā€™ve found them to be marginal gains at best. Not worth their overhead.#2019-09-0514:01Leonyea i know, i don't have any direct issue with exceptions, i just really hate using them where not necessary. and as i said, in my case, they are pretty much not a solution, as the one observing the result isn't the same thing that calls the thing that could throw. for such cases, where error-data makes more sense than exceptions, some solution would really be great. but with the current a spec only talks about the keys it gets, the type of the values HAVE to be defined as part of that key really makes something like this nearly impossible, and strongly increases the complexity in knowing what shape your data actually has, because i CANNOT actually spec it without introducing additional complexity (and removing a hell of a lot of expressivity) to my domain model#2019-09-0514:04Alex Miller (Clojure team)if your data can contain error values, then spec'ing it means spec'ing those values. spec is just exposing that complexity that you have introduced#2019-09-0514:10Leonmy data cannot contain errors. a book doesn't contain any error. a find-book-response can contain that error. the big problem is that spec forces me to define find-book-response specs for everything, and by that introduces an additional layer of complexity where something like generics could really help. the whole notion of needing to have a spec for a key in a map makes many things really complicated for no good reason. say you have a utility function, that takes a (s/keys*) map as arguments (maybe it does some complex math where you want the arguments to be named for readability. now you need to create spec defs for each int? that the function can take, thus adding extreme amounts of complexity where a (s/kv :req {:num1 int?, :num2 float?}) would solve your problem perfectly.#2019-09-0514:12Leonenforcing best practices is a good thing, don't get me wrong. but if these best practices only apply in some areas, but you enforce them in every little situation, that really screws up what could have been an incredibly powerful and nice solution to safety and type-validation in a fully dynamic environment#2019-09-0514:12Alex Miller (Clojure team)I don't understand what's forcing you to define anything#2019-09-0514:13Alex Miller (Clojure team)keys specs are open - you don't have to spec every attribute#2019-09-0514:15Leonsay you have some complex, user-input based form, where the data doesn't really mean anything specific from a domain-model point of view. if you want to validate that data in a map context, you'll need to define seperate, single use specs for each entry in that map that you want to validate.#2019-09-0514:15LeonI'd say that "just don't spec things that would take to much effort" isn't a desirable solution.#2019-09-0514:16Leonnot only does your safety suffer from that, but it also defeats the purpose of spec in general. because then you could just say "just manually write (if (not (int? my-number)) (throw Exception. "foo")),... thats not the point#2019-09-0514:16Leonthe point is to make validations easier and expressive#2019-09-0514:17Alex Miller (Clojure team)well yes, but not for every purpose#2019-09-0514:17Alex Miller (Clojure team)given that these are not for your particular domain, but a truly dynamic problem, you're a bit off the goals of spec#2019-09-0514:18Alex Miller (Clojure team)you certainly could use a pairing of fields and specs to generically validate (but don't validate the aggregate map)#2019-09-0514:18Alex Miller (Clojure team)then no registration is needed#2019-09-0514:19Leonstill, what is the actual reason there is no spec-function that takes keys ( be they namespace qualified or not) and their types (predicates, etc) and validates a map against them? is it just to encourage good practises?#2019-09-0514:19Alex Miller (Clojure team)I don't know that I explain it better than Rich, who has written/talked about this extensively#2019-09-0514:19Leoncould you give me a summary?#2019-09-0514:20Alex Miller (Clojure team)the whole idea is to imbue semantics to attributes, and register those for use globally#2019-09-0514:20Alex Miller (Clojure team)and to de-emphasize the role of the aggregate#2019-09-0514:20Alex Miller (Clojure team)that is, the attributes are the driver, not the aggregation (the map)#2019-09-0514:20Alex Miller (Clojure team)this is taken much further in the schema/select work in spec 2 (and s/keys is going to go away)#2019-09-0514:21Alex Miller (Clojure team)spec 2 schemas do have inline support for ad hoc un-namespaced key specs#2019-09-0514:22Alex Miller (Clojure team)https://github.com/clojure/spec-alpha2/wiki/Schema-and-select#unqualified-keys#2019-09-0514:23Alex Miller (Clojure team)the ability to programatically create and use specs is greatly enhanced in spec 2 as well#2019-09-0514:35Leonso spec 2 will support that kind of thing i'm requesting? as i said, i fully understand the "attributes are the driver, not the aggregation" argument, and i think as a base philosophy it's actually rather genius. but there still are a lot of cases where these attribute-semantics are single-use function arguments. and in these cases, having to give names to things that only make sense in a very specific context is increasing complexity and hurting readability. the "attributes are the driver" argument stops working as soon as you leave your direct domain-model and go into functions that achieve very specific purposes with specific parts of that data. having to describe the shape of every little thing there really makes this a lot more complicated. especially in my current project, a clojurescript re-frame based frontend application, i really want to spec out every view function. not strictly for validation or domain-expressivity, but actually primarily just to get any kind of good errors. knowing that a view function got its arguments in the wrong shape/type is a lot more helpful than knowing that some reagent-internal had problems.... for these purposes spec would nearly be perfect. dynamic validation, that is actually close to dependent type-systems, would be the perfect solution to greatly specify what each function expects as arguments, no matter how detached that function is from the domain model. but the way that spec currently prohibits the generation of map-specs inline makes a lot of things hugely more complex. having a view function that takes 4 arguments look like this:
(s/fdef foo
  :args (s/cat :title string?, :subtitle string?, :rating ::domain/rating, :content string?))

(defn foo [title subtitle rating content]
  ...)
is great for the spec. but calling the function gets anoyying and unreadable, so you want to add keyworded args:
(s/def ::title string?)
(s/def ::subtitle string?)
(s/def ::content string?)

(s/fdef foo
  :args (s/cat :props (s/keys* :req-un [::title ::subtitle ::domain/rating ::content])))

(defn foo [& {:keys [title subtitle rating content]}]
  ...)
this is easier to call and read from caller side.... but extremely unnecessarily complex from implementation side. (yes, title subtitle and content could be domain specific stuff, but maybe im writing a component library that doesn't HAVE any domain specific information.) do you see where i come from?
#2019-09-0514:40Alex Miller (Clojure team)those seem almost the same. so not sure I'm getting "extremely unnecessarily complex"#2019-09-0514:41Alex Miller (Clojure team)just the extra s/defs?#2019-09-0514:43Alex Miller (Clojure team)I don't think spec 2, at this exact moment, helps in this particular situation. however, it's not really easy to judge as we are currently completely reworking how function specs are defined and used and it's probably not going to bear much similarity to spec 1#2019-09-0514:45Alex Miller (Clojure team)something like (s/schema {:title string? :subtitle string? :rating int? :content string?}) is sufficient at current moment to spec a map with those (unqualified) keys. can't currently apply that ala s/keys* - that's still a tbd right now (I don't think it's a big deal to address, just not current focus)#2019-09-0517:42LeonIn this case it's not that Bad, but in more complex cases, and especially all over the Code, it gets very anoyying to type and Deal with. It would be great if that could be Adressed in spec2, as it then would be useful in pretty much any Situation without being overly verbose#2019-09-0517:53andy.fingerhutSorry, I do not have much constructive to say, but I do recall an explicit statement made by one of the Clojure core team (probably Stu Halloway) that something can be simple (i.e. not complex) but still be verbose.#2019-09-0519:16Leonyes, thats the java philosophy. "make simple things seem 5 times as complex by introducing unnecessary abstractions, syntactical overhead and overengineered design patterns". that its POSSIBLE to make something simple overly verbose isn't a good thing, and absolutely nothing you should WANT#2019-09-0602:37potetmit seems like you want something like: https://cljdoc.org/d/metosin/spec-tools/0.10.0/doc/data-specs#2019-09-0602:38potetmthe upside of a well-broken-down design is you can always layer on syntax to taste#2019-09-0602:40potetmthe fundamental diff between ā€œcomplex = verboseā€ and ā€œcomplex = a few things glommed togetherā€ is the former can turn to the latter, but the inverse is not true šŸ™‚#2019-09-0602:42taylor>A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over, beginning with a working simple system.#2019-09-0602:48potetmthis is aligned with my experience#2019-09-0602:48taylorit took me a long time to learn this lesson šŸ™‚#2019-09-0602:50taylorI think this quote resonates with my enjoyment of writing Clojure and how easy it is to compose little solutions into big ones#2019-09-0602:52potetmhonestly, the biggest win there is probably immutable data#2019-09-0602:52potetmbut Iā€™m continuously surprised at how much simpler thing get if you design them to be run from the REPL#2019-09-0602:53potetmevery TDDer: Itā€™s like Iā€™ve been saying for yeeeears! troll#2019-09-0602:54tayloryeah, I felt the same way with other ML-like functional/immutable languages before Clojure. OCaml/F# is nice, but Clojure is more enjoyable for me#2019-09-0609:28LeonThats exactly what im looking for! Thanks, @potetm i was about to spin my own solution and dig into how to create custom spec validators for that (might still do, but only for the learning experience) As i said, i fully agree that well broken down Systems are easier to reason about and work with. But originally coming from Java, and then going to expressive languages like kotlin, haskell, Rust (and now clojure), ive really started to value expressivity. I like to talk in my Code like i would talk to a Person. I like to describe what im trying to achieve, how different Things relate to each other, and what Things represent, and i like to Use expressive Tools to achieve that. The Same way that a for Loop isnt really the best way to describe the relation of a list of numbers to the Same list with every number squared, i feel about using spec. Defining Things that Look and feel like Domain Model/ more complex stuff for simple validations works, but it doesnt Show what i want to express. In my analogy, i want to actually express (map #(* % %) my-list) . Simmiarly, in my spec i dont want to say "the arg title is of type ::title, where ::title is a String." but just "this function takes a String that will be displayed as the tite." It's not so much about complexity in a Code sense, but just about expressivity, readability and also just typing and reading time. And yes. Lines of Code do matter. Otherwise lisps would put their closing parens on New lines#2019-09-0611:51potetmYeah that makes sense to me.#2019-09-0611:53potetmOne thing that I really disliked about spec is that it encouraged creating names where none would suffice.#2019-09-0612:17Leonyes, that is my exact point ;D glad we agree here, and i hope spec2 will change that#2019-09-0611:55mishaas a datapoint, in ~2 years I specced like 5 functions, and am really amused how much attention s/fdef and instrument get#2019-09-0612:07potetmwhyso? do you just not get a lot of value out of fdef?#2019-09-0612:13Leoni spec pretty much all my functions, but mainly because its the only way to get immediately helpful error messages. especially in clojurescript, sometimes the errors really dont help. i just get some errors in some G_1235 generated function name and have no idea what happened. also, I'd say a big help for speccing all functions are the defn-spec feature of orchestra or things like ds/defn from defn-spec that lets you add specs directly into the function signature#2019-09-0612:28mishathere are few functions which benefit from gen-testing. which somewhat rhymes with "you really need just few refs per project". but other things too: - read/write overhead, - maintenance, - "spec pushes you to have global names where none suffice", - project size/complexity is not that out of control, so "specing 100% of the code is the only way out" - wide contact surface with mutable DOM in cljs - etc.#2019-09-0612:34Leonmaintainance isn't really a big problem, and the error reporting benefits are just huge#2019-09-0612:39mishaI prefer to understand what system is doing, when I can afford it. Admittedly, you are not always in position to have this as an option (project/team size, domain complexity, etc.)#2019-09-0612:45mishasome analogies would be: - turning off code highlighter to pay more attention to code. - writing code w/o autocomplete/autosuggest to know better whats there in the project. these things keep your mind in a tonus#2019-09-0612:47mishaand more stuff you will know you will have to keep track of, more reluctant you will be to let bloat slide in.#2019-09-0613:15potetm@misha yes, to all that#2019-09-0613:16potetmIā€™ve honestly had very little urge to use spec at all. Though Iā€™ve spent the last 9mo on a novel thing, so nothing is solidified to the point where I want to invest in pouring some concrete.#2019-09-0613:17potetmActually, there are a fair number of unit tests, and IMO even those arenā€™t worth enough to bother.#2019-09-0613:18potetm(They have to be touched fairly often and have caught very few/no bugs afaict)#2019-09-0613:25mishaand still, pouring concrete on every private function ā€“ is questionable#2019-09-0613:30Alex Miller (Clojure team)from spec hq: agree#2019-09-0619:15seancorfieldFrom our code base (World Singles Networks):
Clojure build/config 52 files 936 total loc
Clojure source 299 files 70282 total loc,
    3207 fns, 662 of which are private,
    441 vars, 29 macros, 62 atoms,
    602 specs, 21 function specs.
Clojure tests 330 files 19116 total loc,
    4 specs, 1 function specs.
We were an early adopter of Spec in production code (`conform`/`valid?`/`explain-data`) but we mostly spec data rather than functions, as you can see above.
#2019-09-0622:48gerred@seancorfield are you using valid? and conform in production paths or in tests or their own specs.clj file? that is to say, are you paying the performance costs at runtime for the specs you do have?#2019-09-0623:21seancorfield@gerred We use conform and invalid? mostly, then explain-data to drive heuristics for turning spec errors into codes/messages we can return to client apps. But, yes, that's all production path code.#2019-09-0623:23seancorfieldA lot of our input form data and most of our API input parameters are run through Spec. We have conforming Specs that accept strings and will conform them to simple numbers, dates, Booleans etc. Something that Alex frowns on šŸ™‚ but it has worked exceptionally well for us.#2019-09-0623:24seancorfieldWe usually write the specs to accept either the native type we want or a string that can be parsed/coerced/conformed to that native type, and our specs have generators that produce strings (we mostly generate the underlying spec and then fmap str over that).#2019-09-0700:29johanatanis stest/instrument known to work for spec'd reagent components (returning hiccup data structures)? once i enabled it, my browser started complaining that it didn't recognize the elements emitted (which were non-standard html elements which had apparently not been interpreted by react/reagent beforehand).#2019-09-0721:36borkdude@johanatan fwiw, I do have an fdef for one reagent component in our app, but that's a form-3 component: (defn foo [opts] (r/create-class ...))#2019-09-0722:24johanatanYea perhaps it would be less of a problem for form-2 or form-3 components. @borkdude #2019-09-0919:17Nolanmay have been discussed before, but is there a way to install a custom randomness source into spec/gen?#2019-09-0919:21Alex Miller (Clojure team)gen rides on test.check#2019-09-0919:21Alex Miller (Clojure team)test.check does have pluggable randomness, although there are some caveats about it as it's designed to be repeatable#2019-09-0919:22Alex Miller (Clojure team)prob best to hit up @gfredericks for the details#2019-09-0919:22Alex Miller (Clojure team)from spec side, the big thing is that stest/check reports the seed it used and accepts a seed as an option to pass along#2019-09-0919:27gfrederickst.c isn't wired up for that, but the protocol is in place, so currently you could pull it off by monkeypatching random/make-random#2019-09-0919:28Nolanright. i traced it down to the relevant files, and quickly realized that there are a number of features about the existing random interface that id want to keep in almost every case#2019-09-0919:29gfredericksWhat's the motivation?#2019-09-0919:30Nolanusing the current spec-alpha2.gen whats the best way to create a generator that basically calls a function and returns the value? looking for something like gen/return that evaluates x in each generation#2019-09-0919:30Nolanit may sound weird, but im aiming for a generator that produces a fixed number of CSR bytes#2019-09-0919:32gfredericksSounds like something better addressed at the generator or spec level, if I'm understanding you#2019-09-0919:32gfredericksOr is this some kind of security thing where you want High Quality Randomness?#2019-09-0919:32Nolanyeah, it definitely is. i was mistaken to think that it would be interesting to aim at the test.check level#2019-09-0919:34gfredericksgen/fmap is the normal trick for making a generator from an impure function, with all the usual caveats#2019-09-0919:34Nolani do want a certain quality of randomness, but more generally im looking for something that would let me (gen/let [x (some-fn-producing-x)] x), but gen/let isnt exposed in spec-alpha2.gen. im using fmap now, but didnā€™t know if that was the right way:tm:. appreciate the input!#2019-09-0919:38gfredericksThere is no right way, because it circumvents determinism, growth, and shrinking; but spec has generally made theses things a lot murkier#2019-09-0919:40Nolanmakes total sense. thats what i quickly realized when i was digging into the test.check randomness. those are some really neat files, way above what im trying to do#2019-09-0919:40Nolanawesome work!#2019-09-0919:41Nolanam i correct in labeling something like this (partial gen/fmap f) a ā€œcombinatorā€?#2019-09-0919:41Nolanor would that have a different name#2019-09-0919:49gfredericksfmap is a generator combinator, and partial is a function combinator; the result is also a generator combinator I suppose#2019-09-0919:51Nolanideal#2019-09-1014:21NickIā€™d like to be able to add a docstring to my specs, to document particular points of interest about one spec versus a similar one. Looks like thereā€™s an issue on the JIRA but it hasnā€™t been updated in some time. Is there a recommendation as to how to handle this yet? https://clojure.atlassian.net/browse/CLJ-1965#2019-09-1021:09thumbnailAt work we use/develop speced.def, which includes a helper exactly for this with tooling support; https://github.com/nedap/speced.def/blob/a6e4e40fa6b92cf66ca6b0f1542eeb8cbd09b039/src/nedap/speced/def.cljc#L24#2019-09-1014:22Alex Miller (Clojure team)We're planning to work on it for spec 2, but currently, no#2019-09-1014:44NickCool, thanks. Iā€™ll just drop the comment in my code next to the spec def then.#2019-09-1118:33Nolanonce it leaves alpha of course, is it a priority for spec to expose the same api between CLJ and CLJS?#2019-09-1118:47Alex Miller (Clojure team)yes#2019-09-1118:48Alex Miller (Clojure team)I'm not sure whether it's possible to guarantee 100% fidelity between the two, but certainly should be close. afaik it is mostly the same now (there are some necessary exceptions around the gen namespace, but those should be able to move closer)
#2019-09-1119:08Nolanimpeccable!!! thanks for the info. and totallyā€”they are (at least for us) entirely cross-compatible for spec1. ive gotten so fond of s/schema and s/select that now im almost disenchanted when working with our current specs šŸ˜‚#2019-09-1218:21leongrapenthinim wondering whether spec2 is missing feedback/adoption because of cljs?#2019-09-1218:23hkjelsI would definitely have used spec2 if it was available in cljs#2019-09-1418:10johanatansame#2019-09-1218:21leongrapenthinid love to try it out and give feedback already, but all my projects are fullstack or cljc#2019-09-1218:29ghadi@leongrapenthin spec2 isn't even done#2019-09-1218:30ghadionce the design settles, will be ported to cljs#2019-09-1218:30leongrapenthincool#2019-09-1219:17seancorfieldHeh, we're about as leading edge as you'll find, for running pre-release versions of Clojure and Contrib libs but even we're not using Spec2 on the backend yet. Still far too "not done". We do have a branch based on it, passing all our tests, but was a bunch of work to get the Spec1 code running on Spec2.#2019-09-1223:11andy.fingerhutThis is perhaps a bit tangential to spec, but I was reminded recently that a handy technique for making generative testing more time-efficient in finding problems in your code, is if you can tweak parameters in your code to make sub-parts of your data structures smaller. For example, a 32-way branching tree structure like PersistentVector takes 32 conj's or pop's before it actually "does something" to affect the structure of the tree, but if you tweak the code to make it 4-way branching instead, you can make more interesting tree structures and test them with generative testing much, much faster, and the bugs are likely to be identical, except for the size of the test cases.#2019-09-1223:12andy.fingerhutA technique I saw used in hardware simulation tests long ago (where the simulations are much much slower than the real hardware devices), but hadn't been reminded of in a while.#2019-09-1322:26gerredi must be tired and missing something#2019-09-1322:26gerredi'm converting a spec from s/? to requiring exactly one argument.#2019-09-1322:26gerredi'm staring at the source and can't figure out the arg spec I want.#2019-09-1322:28gerredoh my gosh#2019-09-1322:28gerrednm#2019-09-1509:20ikitommiIs it intentional that s/valid? can throw? should it return false in case the validation fails on exception?#2019-09-1509:20ikitommie.g.
(s/valid? empty? 1)
; Syntax error (IllegalArgumentException)
; Don't know how to create ISeq from: java.lang.Long
#2019-09-1704:07ataggartCan't speak to intention, but that predicate has an invariant assumption: that the arg is a collection. Neither true nor false would be valid answers. The error can be avoided by first checking that the invariant assumption holds:
user=> (s/valid? (s/and coll? empty?) 1)
false
#2019-09-1714:31dmarjenburghIs it possible to dynamically generate specs at runtime? For example, based on a runtime value, I call a fn that returns a vector of namespaced keys. I then generate a spec like (s/keys :req vector-of-keys). Since keys is a macro, I havenā€™t been able to find a way#2019-09-1714:46Alex Miller (Clojure team)you'll need to eval somehow, either by explicitly using eval or by wrapping in another macro that uses backtick#2019-09-1714:47Alex Miller (Clojure team)(in spec 2, we've made this much easier)#2019-09-1714:51dmarjenburghGreat, looking forward to spec2#2019-09-1916:11dominicmHas anyone experimented with making specs for stateful things, with the intention of using mocks for those values?#2019-09-1916:11dominicmThinking in the context of dependency injection somewhat.#2019-09-1916:13Joe Lane@dominicm Do you mean something like
(stest/instrument `invoke-service {:stub #{`invoke-service}})
found at https://clojure.org/guides/spec#_combining_check_and_instrument
#2019-09-1916:15dominicmoh wow, that's fantastic!#2019-09-1916:16dominicmoh, wait. That's not quite what I was looking for. But still very interesting in what I'm thinking of šŸ¤”#2019-09-1916:17dominicmthat's really neat. I didn't know you could do that. That's actually really handy for what I'm working on.#2019-09-1916:19dominicmI'm actually thinking of the case where you have a function like invoke-service, and you want a dumb wrapper which can figure out what to call it with (from it's registry of "stuff") based on the spec. I'm a bit frustrated with the pattern of passing around a grab-bag of state which inevitably grows into an unrepl-able mess.#2019-09-2009:21djtangoinstrument actually lets you go even further and completely replace the function#2019-09-2009:23dominicmYeah, that's great at the repl. I don't want to do that in production though šŸ™‚#2019-09-2009:24djtangoah - sorry, I am still catching up with the discussion though have found the intercepting properties useful at test-time#2019-09-2009:25dominicmI'm a little unsure on that. Parallelizing tests is valuable for speed (until Rich invents his tool for running only the required subset of the tests). I don't know how much I love that idea really.#2019-09-2009:25dominicmBut yeah, if you're willing to trade those two properties, it's all good.#2019-09-2009:34djtangomm yeah fair enough. So is your use-case this: use fdef to document what deps your function needs have some kind of wrapper that then knows how to extract and inject only those deps into a calling function?#2019-09-2009:37dominicmYeah. I'm coming round to the idea that I might be barking up the wrong tree though šŸ™‚ I'm thinking a little more time in the hammock should help me puzzle out the use cases.#2019-09-2009:37dominicmI'm trying to exploit maximum leverage without creating unnecessary verbosity. It's a hard balance šŸ™‚#2019-09-2009:45djtangoI guess I can in theory see how you could fetch the fspec then use a generator, to satisfy the s/keys :req portion but feels brittle, if only as I'm not convinced it's an appropriate usage of the tools#2019-09-2009:47dominicmMe neither šŸ™‚#2019-09-2009:49djtangosounds tricky - would be interested to know how you get on#2019-09-2009:51dominicmI'm hoping to publish my results somewhere, I think I'm onto something, I've reduced the initial problem space into something more palatable. But I still have to handle the stateful dependencies part.#2019-09-1916:22Joe LaneI'm not sure I understand the usecase for the dumb wrapper. What is registry of "stuff"?#2019-09-1916:24dominicmI suppose something to the effect of, I have a "system" like: {:db db-conn :some-stateful-service statey} and I want to direct things directly to a function defined like:
(defn add-user
  [db add-user-data] ā€¦)
So the "registry" is that first map.
#2019-09-1916:25Joe LaneSo, in component terms, a "system"#2019-09-1916:25Joe Lanek#2019-09-1916:25Joe Lane(Or a pedestal context map)#2019-09-1916:26dominicmYeah. Exactly. But passing around systems is an anti-pattern. And I also can't figure out what's in my pedestal context map half the time, because there's lots of interceptors messing with it, and it's unclear why routeA gets :mongodb and routeB doesn't. (So I'm trying to come at this with a new angle of positional parameters)#2019-09-1916:31Joe Lanethinking#2019-09-1916:33Joe LaneSo, is the goal here to assert a property about the dumb wrapper but you're trying to determine the best way to inject actually stateful mocks?#2019-09-1916:34dominicmI guess I could write a spec parser for s/cat? And that would let me figure out the arguments in order. Well, actually I was thinking that it would be handy to just start by being able to call add-user by using it's fdef to figure out that it needs a database, and getting one from the system. (and somehow connecting ::db/db as a spec to that)#2019-09-1916:35Joe Lane(= dumb-wrapper add-user)#2019-09-1916:35Joe Laneright?#2019-09-1916:35Joe Laneoh.#2019-09-1916:36dominicmyeah, right šŸ™‚#2019-09-1916:37Joe LaneI might be wrong, but it sounds like you want an integration between your DI tool (are you using component?) and the s/select facilities in spec2. If you're using component though, shouldn't that fdef to figure out.... step be happening at system/lifecycle start time?#2019-09-1916:37dominicmBut I'd be using this in production, not just for testing. So I would know that this function only takes 1 non-stateful argument (the "event", or maybe "req" for http). So I just need to get the rest of the arguments.#2019-09-1916:39dominicmI'm not using component. The reason being that a system probably ends up with ~50 or so handlers, and the boilerplate involved is quickly tedious. Although maybe I should just use a macro for that šŸ˜›#2019-09-1916:39dominicmThe way I see it, I have actual stateful things (e.g. db conn) and things that want to use those things.#2019-09-1916:40Joe LaneThis is starting to feel more and more like spring style annotation DI vs data oriented component DI. And further away from a problem related to spec.#2019-09-1916:42dominicmA little, yeah. And that's somewhat intentional. I see one of two patterns in this space: 1. create a map with everything in, call it "deps" and hope the function you're passing it to understands it (and there's a bunch of bad patterns which fall out of this). 2. roll entire namespaces into being partial applied with their state available, and passed in as arguments.#2019-09-1916:44dominicmTbh, I expect there's not a good spec function for figuring this out šŸ™‚ As I have a lot of contextual awareness that spec doesn't have. (e.g. I know it's always a list, and I will always know everything except 1 argument).#2019-09-1916:45Joe LaneI think the 3rd pattern is component/system/mount oriented, where functions are passed positional arguments (not a deps/req/context map) which were determined by the DI tool. I don't think you can reduce that kind of complexity, only choose where to solve it, 1 large place (component) or in every dumb-wrapper by declaring the deps at the callsite. I definitely don't think spec is going to be helpful here.#2019-09-1916:47dominicmI haven't seen 3? what does that look like#2019-09-1916:49dominicmoh, maybe I do know what you mean.#2019-09-1916:50dominicmI guess pattern 3 here is implemented in terms of my listed patterns 1 & 2.#2019-09-1919:28jaihindhreddyI just started solving some basic programming puzzles with Clojure, with the constraint that I want to spec the solutions. I hit a snag right on the first one. Here's the fn I want to spec:
(defn two-sum
  "Returns indices of two elems in ints that add up to sum"
  {::url ""}
  [ints sum]
  (loop [l 0
         r (dec (count ints))]
    (let [s (+ (nth ints l) (nth ints r))]
      (cond (= l r) nil
            (= s sum) [l r]
            (< s sum) (recur (inc l) r)
            :else (recur l (dec r))))))
And here's what I came up with:
(s/fdef two-sum
  :args (s/cat :ints (s/coll-of int? :min-count 2 :kind vector?) :sum int?)
  :ret (s/and (s/tuple (s/and int? #(>= % 0)) pos-int?) (fn [[l r]] (< l r)))
  :fn (s/and
        #(< (-> % :ret second) (-> % :args :ints count))
        #(= (-> % :args :sum)
            (+ (nth (-> % :args :ints) (-> % :ret first))
               (nth (-> % :args :ints) (-> % :ret second))))))
#2019-09-1919:30jaihindhreddyThe puzzle is to find two (different) indices s.t. the elements in ints in those indices add up to sum. The catch is: there is always exactly one solution. How do I spec my args to get gen right?#2019-09-1919:31jaihindhreddyUnless I use the fn itself in it's spec (or trusted equivalent), I can't seem to get gen. to work.#2019-09-1919:52seancorfield@jaihindhreddy That sounds like an external constraint on the data? An arbitrary vector of 2+ ints and an arbitrary sum aren't going to satisfy that condition so the behavior of two-sum on such data will be...? Undefined? nil?#2019-09-1919:52seancorfieldYour :ret spec doesn't account for the function returning nil -- which it clearly can -- so your spec isn't right as it stands.#2019-09-1919:55seancorfield
user=> (two-sum (into [] (range 10 20)) 9)
nil
So your :ret spec needs to be s/nilable or use s/or and then your :fn spec needs to accept that (:ret %) can be nil (or should satisfy that complex predicate).
#2019-09-1919:58jaihindhreddyIt is an external constraint on the data. I thought of using s/nilable there but actually the fn is not supposed to be called with such args and its UB, and I'm trying to get the generation to work in such a way that there exists exactly one answer.#2019-09-1919:59jaihindhreddyI'll probably look into using fmap to generate the sum from the ints. Thanks for your help!#2019-09-1920:03seancorfieldAnd it's also an ascending sequence of ints, yes? (based on the < logic in there)#2019-09-1920:05seancorfieldI think what you're attempting is a bit self-defeating: the generator and the :fn spec together are pretty much going to be a re-implementation of the function logic at this point...#2019-09-1920:06seancorfield(i.e., you're over-spec'ing things, IMO)#2019-09-1920:07jaihindhreddyKinda reckoned that myself.#2019-09-1920:09jaihindhreddyThe fn returns a pair of indexes into the ints arg, so I'm checking that the second one in the return value is less than the count of ints.#2019-09-1920:45vlaaadnoob question: why spec forms sometimes have qualified, and sometimes unqualified symbols?#2019-09-1921:00vlaaadminimal example:
(s/def ::vec vector?)

(:pred (first (::s/problems (s/explain-data ::vec {:a 1}))))
; => clojure.core/vector?

(:pred (first (::s/problems (s/explain-data (s/coll-of vector?) [{:a 1}]))))
; => vector?
#2019-09-1921:03Alex Miller (Clojure team)Because bugs#2019-09-1921:03Alex Miller (Clojure team)Should always be qualified#2019-09-1921:03vlaaadagree!#2019-09-1921:04Alex Miller (Clojure team)There are some pending patches for this stuff#2019-09-1921:08vlaaadI'm fascinated by spec, sprinkling it a bit here and there to document/enforce contracts where it makes sense... do you think it's the future of strong gradually static type systems?#2019-09-1921:16vlaaadlike being able to say something along the lines of
let n: int? = (get-some-int) 
let x: (and int? even?) = n ;; checks for `even?` during casting
#2019-09-1921:16Joe LaneI think it will push significantly more boundaries than just "type systems"#2019-09-1921:18Joe LaneI also think the traditional notion of static type systems look very different if you can work in a dynamic system.#2019-09-1921:19Joe LaneBut what do I know :man-shrugging:#2019-09-1922:05seancorfieldI don't consider it "like" a type system (and I think it's a bit misleading to think of it in those terms).#2019-09-1922:07seancorfieldWe've been using spec in production code very heavily since it first appeared. We don't use it much on functions (which is the only "like types" part of it -- and even that isn't much like a "type system").#2019-09-1922:09seancorfieldWe mostly use data specs and validation/conformance. We use it to generate (random, conforming) data for example-based tests. We use it for generative testing of some things. We use it via instrument a little bit during dev/test.#2019-09-2005:41vlaaadWell, it's not like a type systems of today :)#2019-09-2017:51andy.fingerhutWell, the full generality of spec allows arbitrary code in predicates, so it is in general computationally undecidable to statically check that a program meets such a general spec.#2019-09-2017:53andy.fingerhutspec has been designed with run-time checking (and generation, etc.) in mind, not trying to limit itself to what can be statically checked, although subsets of it can be: https://github.com/arohner/spectrum#2019-09-2020:50seancorfieldBecause folks keep asking about our "coercing specs" at work, we just open-sourced them https://github.com/worldsingles/web-specs worldsingles/web-specs {:mvn/version "0.1.0"}#2019-09-2105:20dominicmAny regrets using conformers?#2019-09-2105:26seancorfieldNope, none at all. I originally submitted a talk to Conj about it (and it was accepted) but I pulled it because Alex was so "anti" the whole conformer thing. That was years ago. But I still think we made the right decision and I'd stand by it every day.#2019-09-2107:04jumarAwesome, thanks for that.#2019-09-2209:56Sebastiano BarreraHi y'all... I'm a beginner tinkering around with spec (on CLJS) and I was wondering: is there a way to encode a spec so that a certain keyset is required only if another key has a certain value? For example, I might have {:type :assignment, :left ..., :right ...} or {:type :identifier, :name 'some-name}; could I express the requirement that :left and :right must both appear iff :type is :assignment?#2019-09-2210:03Sebastiano BarreraWait, I think I figured it out... I'm guessing
(defn node-is? [type] #(= (:type %) type))
(s/or
  (s/keys :req-un [::type (node-is? :assignment) ::left ::right])
  (s/keys :req-un [::type (node-is? :identifier) ::name]))
#2019-09-2210:59vlaaad@sebastiano.barrera_cl I think what you need is multi-spec#2019-09-2211:00vlaaadhttps://clojure.org/guides/spec#_multi_spec#2019-09-2211:02Sebastiano BarreraOh, let me take a look at that#2019-09-2517:07sundbpIā€™ve got a question about spec2, how do I do something like this: (s/def ::foo (partial bar 1)) - in spec2 that fails. Wrapping the partial in e.g. s/spec is no solution.#2019-09-2517:57seancorfieldI believe function literals are accepted @sundbp -- try (s/def ::foo #(bar 1 %))#2019-09-2614:02sundbpThanks. Will give that a go#2019-09-2518:06seancorfieldYeah, that works:
user=> (require '[clojure.spec-alpha2 :as s])
nil
user=> (defn bar [n y] (< n y))
#'user/bar
user=> (s/def ::foo #(bar 1 %))
:user/foo
user=> (s/conform ::foo 2)
2
user=> (s/conform ::foo 1)
:clojure.spec-alpha2/invalid
user=> 
#2019-09-2519:01andy.fingerhutI can dig into the implementation and find out, but does anyone happen to know whether test.check/quick-check calls clojure.test/is or clojure.test/are internally, and thus is intended to be called inside of a clojure.test/deftest form directly?#2019-09-2519:01andy.fingerhutThe doc string says it runs multiple tests, but isn't clear about whether there is a return value#2019-09-2519:03andy.fingerhutPerhaps one reasonable way to use test.check/quick-check inside of a clojure.test/deftest would be (is (:pass? (test.check/quick-check ...))) ?#2019-09-2519:06andy.fingerhutAh, from looking at test.check's own deftest's, it appears that it often uses (is (:result (test.check/quick-check ...))). I will use that, too.#2019-09-2519:10andy.fingerhutIs there some existing test.check docs that describe the return value of quick-check that I could learn this from? Or perhaps a small addition to the quick-check doc string might be welcome?#2019-09-2519:16gfredericks@andy.fingerhut hi#2019-09-2519:17gfredericksThe docstring of the latest version is pretty explicit about the return value#2019-09-2519:17gfredericksYou're correct that the quickcheck function is unrelated to clojure.test; you can integrate manually as you noted, or by using the t.c.clojure-test namespace#2019-09-2519:24andy.fingerhutok, I may have been checking the quick-check doc string of an older version that core.rrb-vector is using. Let me look at the latest.#2019-09-2519:24gfredericksThe past was a terrible time#2019-09-2519:25andy.fingerhutOK, much better now. Thanks!#2019-09-2519:27gfredericksNP#2019-09-2710:09lmergenis it possible to somehow tokenize a string using specs, in a s/cat-like way? e.g.
(s/def :directory (fn [x] (re-matches ...))
(s/def :filename (fn [x] (re-matches ...))
(s/def :absolute-path (s/cat :d :directory :f :filename))
(which doesn't work for obvious reasons, but that's the idea of what i'm looking for)
#2019-09-2710:11lmergenactually i'm probably looking for a way to compose regexes more than i'm looking for this spec-like functionality... šŸ¤”#2019-09-2710:21Alex Miller (Clojure team)You can do this (by seq-ing a string) but I generally think itā€™s a bad idea. There is actually a regex composition library for Clojure, but Iā€™m blanking on the name#2019-09-2710:23Alex Miller (Clojure team)Maybe https://github.com/fhur/regie is what Iā€™m thinking of?#2019-10-1616:49rickmoynihanCould it also have been: https://github.com/cgrand/seqexp ? cc @U0M8Y3G6N#2019-09-2710:27lmergeni see, that actually looks interesting#2019-09-2710:27lmergenthanks#2019-09-2712:37jonpitherHi. is there a generalised pattern / approach for handling config, in that a) defining the specs for the config attributes, b) documenting the individual options, c) providing default values for options? I realise this Q is a bit wide ranging, but I'm wanting to take a step back and to see what others usually do for config input#2019-09-2713:47taylorI donā€™t have an answer for your exact question, but Iā€™ve specā€™d CLI args before which is kinda like config https://github.com/taylorwood/clojurl/blob/c69cef7c1f2b46ac9034cf75ea1c5f7d3abf6d2b/src/clojurl.clj#L13-L42 Re: documentation, spec doesnā€™t really support it (yet?), and I probably wouldnā€™t use spec to influence config default values. That said, thereā€™s probably an opportunity to pull pieces of this together into an ā€œopinionatedā€ config library :man-shrugging:#2019-09-2714:27jonpitherThanks @U3DAE8HMG will check this out#2019-09-2721:11jumar@U050DD55V I've done something similar for one of our applications. It's a rather hairy code and I'm sure there's a better approach for validation and replacing the invalid values with defaults but here's essentially what we've had in production for at least 1 year: https://github.com/jumarko/clojure-experiments/blob/master/src/clojure_experiments/config/config.clj#L1 See this commit which shows all relevant parts: https://github.com/jumarko/clojure-experiments/commit/35d429e1641806bba78b470e4f4e249207739e96#2019-09-3008:56jaihindhreddyspec might support docstrings but probably won't support metadata, because that might cause specs to become bloated and less suitable for a variety of usecases. Because ns-qualified keywords come with no strings attached, spec is able to associate global semantics in particular contexts (when calling s/validate, for example). In a same way, maybe something like aero can maintain its own global registry of documentation, defaults etc. and be useful in contexts that make sense for it, perhaps pawning off the job of validation to spec. @U050DD55V sorry for being vague šŸ˜… And pointing to your own library (`aero`) šŸ˜›#2019-10-1617:02rickmoynihan@U050DD55V we have an inhouse tool that we use to specify service requirements and dependencies in EDNā€¦ itā€™s meant to tie together CI services with what they expect at runtime etc, and help in pulling clojure services and service dependencies together across various environmentsā€¦ e.g. in dev you want to run the services with repls and various things setā€¦ in prod you want packer images built with config files written in the right places that define parameters to be injected into env files and config files etc. And all environments want to pull in dependent services. Anyway part of it is aero; and each service declares things like:
:omni/keys {:stardog/stardog-home {:doc "Location of the stardog home directory"
                                    :default [:install "stardog-home"]
                                    :env-var STARDOG_HOME}
             :stardog/java-args {:doc "Arguments to the JVM when running stardog"
                                 :env-var STARDOG_SERVER_JAVA_ARGS}
             :stardog/temp-dir {:doc "Java temp directory for the stardog process"
                                :env-var STARDOG_JAVA_TEMP_DIR}
             :stardog/logs-dir {:doc "Directory to write the stardog.log file to"
                                :default "."}}
#2019-10-1617:03rickmoynihanwhich includes the doc string, and where the variable comes from etc.#2019-10-1617:04rickmoynihanso services can inform things further up the chain what they need to set; and display a useful error when something is missing#2019-09-3010:08shanBasic question about spec, but I would expect clojure to complain if a spec doesnā€™t exist but seem it just seems to ignore the fact:
(s/def ::test-spec
  (s/keys :req-un [::req-does-not-exist]
          :opt-un [::opt-does-not-exist]))

(s/valid? ::test-spec {:req-does-not-exist :foo})
;; => true


(s/valid? ::test-spec {:req-does-not-exist :foo
                       :opt-does-not-exist :bar})
;; => true
Any idea if thereā€™s a reason for this or how you usually handle being aware of missing specs?
#2019-09-3010:16jaihindhreddyI'm guessing that's because spec is an open system. When you say I need ::a to be present in the map, s/keys will verify that, and if you have specified what ::a should look like, then it will verify (and generate) that too correctly. This is what allows us to spec as little or as much as we want about our domain.#2019-09-3010:17jaihindhreddyAlso comes in handy when doing interactive development, where spec doesn't become this thing that is constantly bothering you about missing spec definitions. You get to decide how much you want to spec, and accordingly you get back as much value and precision in validation and generation. Hope that answers your question.#2019-09-3010:29vlaaadCode above looks like a bug to me.#2019-09-3010:30vlaaadby specifying :req-un, you asked spec to check that :req-does-not-exist key should be checked against ::req-does-not-exist spec during a call to valid?#2019-09-3010:31vlaaadit's one thing to have forward declarations in spec where you don't check if referenced spec is declared during "definition phase", and another to ignore required keys during "validation phase"#2019-09-3010:40shan^^ yep, thatā€™s what I was thinking#2019-09-3010:58jaihindhreddyYeah. That does seem like a bug. Didn't read the whole thing. My bad.#2019-09-3014:05jaihindhreddyActually, s/valid is behaving correctly here. (s/valid? ::test-spec {}) correctly returns false.#2019-09-3010:42shanit seems to mean you could add a missing spec to :req/opt-un and add it to the data but this wouldnā€™t actually be checked or give any indication the spec doesnā€™t exist#2019-09-3014:28minimalStuart H wrote a gist to check missing keys a couple of years ago https://gist.github.com/stuarthalloway/f4c4297d344651c99827769e1c3d34e9#2019-09-3023:04shanCheers dude. Youā€™re always a good source for clojure knowledge šŸ™Œ#2019-09-3013:02conanThis seems fine to me. You're ensuring that a map contains a specific key, but making no claims about the value of that key. I do this frequently when receiving required but complex data from external systems - i want to check that the data is there, but i don't know enough to write a comprehensive spec for it.#2019-09-3016:55sgerguriHere is my situation: I have a top-level spec that allows for two situations - cluster updated and cluster removed. This is handled by two multispecs as follows:
(defmulti cluster-updated :cluster-type)
(defmulti cluster-removed :cluster-type)
(s/def ::cluster-updated (s/multi-spec cluster-updated :cluster-type))
(s/def ::cluster-removed (s/multi-spec cluster-removed :cluster-type))

(s/def ::cluster (s/or :cluster-removed ::cluster-removed
                       :cluster-updated ::cluster-updated))
#2019-09-3016:56sgerguriEach of those multispecs has two implementations. The implementations generate nested maps, where some of the nested maps have generators that generate matching input, but conforming strips away extra data as specified.#2019-09-3016:58sgerguriNow, I would like to call (s/conform ::cluster (g/generate (s/gen ::cluster))) and get either a :cluster-removed or :cluster-updated data, but conformed recursively all the way through. However, conformance does not seem to propagate to the level of the multispecs, it simply stops at the level of ::cluster.#2019-09-3016:59sgerguriI can work around it by examining the tag I get from ::cluster, then conforming again against either ::cluster-updated or ::cluster-removed, which does the correct thing against the correct implementation. But ideally, I would like to conform only once and have the correct thing happen. Is there a way to do that?#2019-09-3017:00sgerguriFYI - I have two multispecs there as unfortunately multi-spec seems to expect a single field as a dispatch tag, even though the multimethods themselves can dispatch on more than one field (through the use of juxt).#2019-09-3017:01sgerguri@alexmiller ^ Will definitely appreciate some thoughts on this situation! šŸ™‡#2019-09-3017:05Alex Miller (Clojure team)well on the last point first, you can use an arbitrary function for the multi-spec, doesn't have to be a keyword#2019-09-3017:05Alex Miller (Clojure team)so you could for example use juxt to produce a vector and dispatch from a single multimethod#2019-09-3017:05Alex Miller (Clojure team)maybe that solves your whole problem, not sure#2019-09-3017:06sgerguriWhat would I use for a retagging function though?#2019-09-3017:06Alex Miller (Clojure team)it's used to drive gen#2019-09-3017:06Alex Miller (Clojure team)it will pick a random multimethod, gen from that, then "retag" to something passed to the multimethod#2019-09-3017:07Alex Miller (Clojure team)here it would need to go from the dispatch value (the vector output of juxt) to modify the generated defmethod map to get back to your original expected input#2019-09-3017:08Alex Miller (Clojure team)depending on your method generators, there may be nothing to do and you could just use identity#2019-09-3017:08Alex Miller (Clojure team)or you could assoc back in the keys and the values from the juxt vector#2019-09-3017:08sgerguriWith juxt I'd be ideally looking at something as follows:
(defmulti cluster (juxt :cluster-type :action))
(s/def ::cluster (s/multi-spec cluster ?))
I'm just not sure what to put in instead of the ?.
#2019-09-3017:09Alex Miller (Clojure team)as above, identity might work, or you could use (fn [[type action]] (assoc % :cluster-type type :action action))#2019-09-3017:09sgerguriAh, identity might just do the trick. Let me give that a try!#2019-09-3017:10Alex Miller (Clojure team)or maybe I've got the signature wrong there - does retag fn take 2 args?#2019-09-3017:10Alex Miller (Clojure team)yeah, it's value and dispatch value#2019-09-3017:10sgerguriI think it only takes one? It can also be a keyword, which would support that argument.#2019-09-3017:10Alex Miller (Clojure team)so should have been (fn [gen-val [type action]] (assoc gen-val :cluster-type type :action action))#2019-09-3017:10sgerguriAh no, it actually takes two. But apparently one can supply a keyword as well.#2019-09-3017:12Alex Miller (Clojure team)I think that's right. I haven't done one of these in a while.#2019-09-3017:12sgerguriWhy is the retagging even necessary, when the multimethods return a spec that is narrow enough to always generate the right data? I guess that's one thing that's confusing me.#2019-09-3017:20sgerguriHm, this did not really solve the problem - but thanks anyway. I'll post here if I manage to solve this in a satisfactory fashion.#2019-09-3017:25sgerguriActually, I take that back - I had a dangling redef on the multispec elsewhere. Doing it as per Alex's suggestion:
(defmulti cluster (juxt :cluster-type :action))
(s/def ::cluster (s/multi-spec cluster (fn [gen-val [type action]] (assoc gen-val :cluster-type type :action action))))
does the trick. Thanks for the help, @alexmiller!
#2019-09-3017:26Alex Miller (Clojure team)Np#2019-10-0123:01Oliver GeorgeIt'd be nice if defn did s/instrument when you're working at the repl.#2019-10-0123:02Oliver GeorgeAny chance something like that is on the radar?#2019-10-0123:14Alex Miller (Clojure team)I think thatā€™d be wrong for everyone sometimes#2019-10-0123:15Alex Miller (Clojure team)Iā€™d prefer to have a lot more control over things like that#2019-10-0123:17seancorfieldAgreed. I definitely would not want auto-instrument at the REPL (and how would the compiler tell the difference between REPL and regular running since the semantics are the same?).#2019-10-0123:18seancorfield(remember that not everyone AOT's code!)#2019-10-0211:29Jakub HolĆ½ (HolyJak)@alexmiller I have a tiny improvement to gensub to provide a meaningful error when such-that fails, is it meaningful for send a patch to https://clojure.atlassian.net/browse/CLJ-2097? Essentially (gen/such-that #(valid? spec %) g 100) -> (gen/such-that #(valid? spec %) g {:max-tries 100, :ex-fn (fn [{:keys [max-tries]}] (ex-info (str "Couldn't satisfy such-that predicate after " max-tries " tries.") {:path path}))}) (Since path is the only thing that looks meaningful, contrary to test.check's gen, pred and spec's spec, form) I have updated the issue, let me know there whether to send a patch. Thank you!#2019-10-0212:39Alex Miller (Clojure team)test.check 0.10.0 has some new hooks for this case too#2019-10-0212:41Alex Miller (Clojure team)if you have a patch, go for it, but I'm not sure whether we're going to go back and patch anything on spec.alpha. this is tbd, but my current thought is that once we get close to a spec 2 release I would reassess all the spec tickets (I know some have been fixed) and try to do a clean-up wave. this will require re-working any of the patches as I'd expect none of them to apply to spec 2.#2019-10-0215:05Jakub HolĆ½ (HolyJak)Thanks, @alexmiller! Question 2: I discovered that it is even more useful to include an example of why the spec failed on sample data - knowing which spec failed to match is good but knowing why is even better. In my case the spec failed was something like ::person because of invalid :person :address :zip. So, to troubleshoot my issue, I include this in the thrown error:
{:max  max-tries
 :path path
 :sample-explain (->> (first (gen/sample g 1))
                      (explain-data spec)
                      :clojure.spec.alpha/problems)}
- which was extremely useful for me but not sure whether it would be an appropriate general solution.
#2019-10-0215:06kenny@holyjak YES. The failing args and ret should be easily shown in the error message. I don't think that particular approach will work because there's no guarantee you'll hit the failing case with one sample but I like the idea.#2019-10-0215:08Alex Miller (Clojure team)showing an example which isn't the one that actually failed seems more confusing to me#2019-10-0215:09kennyYes - it needs to be the failed one.#2019-10-0215:09Jakub HolĆ½ (HolyJak)ALL the examples fail#2019-10-0215:09kennyWe have internal code that does this already but it's a hack on gensub.#2019-10-0215:10kennyNot always @holyjak.#2019-10-0215:10Jakub HolĆ½ (HolyJak)Well, in theory you are right. What we call is `(gen/such-that #(valid? spec %) g {:max-tries 1000 ...}) - none of the 1000 generated samples failed. What is the chance that the one we generate in the error does not fail?#2019-10-0215:11kennyDepends on how you're controlling the size.#2019-10-0215:12Jakub HolĆ½ (HolyJak)What do you mean?#2019-10-0215:12Jakub HolĆ½ (HolyJak)Of course I could add a check and include :sample-explain only if it really fails. Would that be satisfactory? If not - what is a good way of capturing some generated, failing data?#2019-10-0215:12kennyIIRC, gen/sample does not use all 200 size values.#2019-10-0215:13kennyUsing quick-check or st/check will.#2019-10-0215:13Jakub HolĆ½ (HolyJak)I am not sure how such-that invokes the generator but I would assume that it does so in a way very similar to sample?#2019-10-0215:15Jakub HolĆ½ (HolyJak)Anyway, what do you think about the "best effort" approach - generate a sample, if it fails the spec, include :sample-explain in the error. I think that would work in 99.9% cases?#2019-10-0215:15kennyAnyway, the piece I find valuable is being able to easily see why a check failed, whether it be by generator failure or check failure. When a generator fails, I would like to see the failed samples. This usually lets me figure out which predicate is causing the issue pretty quick. In the case the check fails, I'd like to see the args and the ret.#2019-10-0215:17kennyYou should know which samples failed while generating @holyjak, no need to generate new ones.#2019-10-0215:18Jakub HolĆ½ (HolyJak)Well, I do not generate anything, it is such-that that does it - and in its failure-handling function I have no idea what values were actually tried. Or? BTW here is the code https://gist.github.com/holyjak/8cadc0d939c8e637ef6bf75b070d28b4#file-clojure-spec-alpha-clj-L17#2019-10-0215:20kennyOur hack (not acceptable for a patch but let's us debug generators without going crazy) is to store invalid values in an atom and display that in the error message.#2019-10-0215:20kennySo we just change the such-that predicate to do that.#2019-10-0215:22Jakub HolĆ½ (HolyJak)Smart! The question is: What approach would be acceptible for a patch (and useful for the users!)?#2019-10-0215:23kennyI don't know. It definitely feels too hacky for a patch. It may be the only way to do it though. When I was looking at this, I don't think I saw any test.check hook to grab the failed values.#2019-10-0215:24kennyJust speculating but I imagine it'd be somewhat easy to add that hook to test.check if it isn't already there and is the proper solution.#2019-10-0215:25kennyI would definitely be in favor of adding some sort of patch to spec1 alpha given we don't know when spec2 is coming. Debugging generators is super painful as it is.#2019-10-0215:25gfredericksCheck the such-that docstring in the latest version#2019-10-0215:26kennyDoesn't appear to have anything that stores failed samples.#2019-10-0215:27gfredericksWould you want all of them? Just the latest?#2019-10-0215:27gfredericksHaving the generator means it should be easy to generate more#2019-10-0215:27kennyProbably all.#2019-10-0215:28kennyIf I have all, I can easily only have the latest if I want.#2019-10-0215:28Jakub HolĆ½ (HolyJak)I believe a single value is enough (preferably the 1st as it is the simplest one)?#2019-10-0215:29kennyTrue.#2019-10-0215:29gfredericksKeeping all could be a GC issue#2019-10-0215:29Jakub HolĆ½ (HolyJak)> Having the generator means it should be easy to generate more That is what I do in my solution (generate a new value using sample) but it has been argued that using one of the actually failed would be better.#2019-10-0215:29kennyI guess I only ever work with one of the failed samples anyway.#2019-10-0215:31Jakub HolĆ½ (HolyJak)@gfredericks As I explained above, this popped up when working with specs - a custom generator generated value that did not conform to the spec. I can get the name of the spec but want to see why the custom gen failed to provide anything valid. There, having a sample value and running s/explain-data on it is sufficient and very useful.#2019-10-0215:31kennyLooks easy to add in such-that-helper.#2019-10-0215:31Jakub HolĆ½ (HolyJak)I am creating an issue on test.check to pass a sample failed value to ex-fn. Stop me if I should not šŸ™‚#2019-10-0215:32gfredericksYou shouldn't not#2019-10-0215:36Jakub HolĆ½ (HolyJak)Conclusion: I create a patch for Spec to include the spec name (or rather path) in the such-that failures and we wait for test.check to expose a sample failed value to the :ex-fn before adding explain-data of it.#2019-10-0215:39Jakub HolĆ½ (HolyJak)@alexmiller is there any point in providing also a similar patch for Spec 2 or is it too much in flux?#2019-10-0215:40Alex Miller (Clojure team)I'm not going to do the sample thing, so not interested in a patch for that#2019-10-0215:41Jakub HolĆ½ (HolyJak)no, not that, just including path#2019-10-0215:41Alex Miller (Clojure team)that's fine#2019-10-0215:41Alex Miller (Clojure team)I don't think there's any point for spec 2#2019-10-0215:41Alex Miller (Clojure team)one possible outcome is that we repackage everything in spec 2 to remove alpha designation, which will (again) break any patches#2019-10-0215:45Alex Miller (Clojure team)I think in test.check, while generating in such-that, it would be possible to just retain the first or last gen'ed every time (before the pred check) and report that if you hit the retry limit. that way you're reporting an example that was actually tried.#2019-10-0215:50gfredericksAn example generated after the fact is just as meaningful, philosophically#2019-10-0215:53Alex Miller (Clojure team)if only a percentage of them fail, and you happen to generate a sample that doesn't fail, that seems strongly less useful than one of the actual examples that didn't pass the predicate#2019-10-0215:53Alex Miller (Clojure team)so I'ma disagree with you on that#2019-10-0215:53gfredericksI was assuming you'd filter on failure šŸ˜›#2019-10-0215:53Alex Miller (Clojure team)well you failed to mention that :)#2019-10-0215:53Alex Miller (Clojure team)but that would be fine#2019-10-0215:53gfredericksWhich, ironically, could fail šŸ™„ though presumably is unlikely to#2019-10-0215:54gfredericksBut that might be a good enough reason not to do it#2019-10-0215:55gfredericksAdding the size to the failure data would assist with that though#2019-10-0215:57Jakub HolĆ½ (HolyJak)@gfredericks Will you consider adding a sample failed value to the arguments of such-that's :ex-fn or some alternative improvement?#2019-10-0216:04gfredericks@holyjak I'm not going to be doing active work on test.check for the foreseeable future, so that'll be up to somebody else; but a jira ticket is definitely the right way to make sure it gets looked at#2019-10-0216:25Jakub HolĆ½ (HolyJak)Hm, I was about to create it but you told me I should not https://clojurians.slack.com/archives/C1B1BB2Q3/p1570030324189400 I guess it was just misunderstanding.#2019-10-0216:25Jakub HolĆ½ (HolyJak)I'd be happy to provide a patch to test.check - and even happier to get any pointers regarding the best solution.#2019-10-0216:29gfredericksNo I meant you should create the ticket#2019-10-0216:29gfredericksSorry, I was being cute with English double negatives, it was probably a bad idea#2019-10-0216:34Jakub HolĆ½ (HolyJak)I see šŸ™‚ My bad, I should have read more carefully. Also, I believed that English normally doesn't permit double negatives. Always learning šŸ™‚#2019-10-0216:35Jakub HolĆ½ (HolyJak)Any pointers regarding the best way to implement this? Is failed-value a good key name to pass to :ex-fn?#2019-10-0216:35Jakub HolĆ½ (HolyJak)FYI I have created https://clojure.atlassian.net/browse/TCHECK-156 Provide sample failed value to such-that's ex-fn for better error messages#2019-10-0216:54Jakub HolĆ½ (HolyJak)FYI I have sumbitted a patch for ā˜ļø#2019-10-0223:10nopromptWhat is the proper way to lawfully copy/paste source code from core.specs.alpha?#2019-10-0223:10nopromptThis patch has caused me some headache: https://github.com/clojure/core.specs.alpha/commit/938c0a9725a284095baa2387dff1a29c3f1e26ac#2019-10-0223:11nopromptWhat Iā€™d like to do is copy ::defn-args and its dependencies into my own project.#2019-10-0223:14nopromptDo I just put the copy right information above the code I want to copy?#2019-10-0223:14nopromptAlso, my project is MIT licensed, so Iā€™m not sure how all that works.#2019-10-0223:18noprompt> What are my obligations if I copy source code obtained from http://Eclipse.org and licensed under the Eclipse Public License and include it in my product that I then distribute? > Source code licensed under the EPL may only be redistributed under the EPL.#2019-10-0223:19nopromptIā€™m guessing I need to handwrite my own version of spec.#2019-10-0223:33seancorfield@noprompt I'm curious as to how you're depending on specs from that namespace since they are intended to support clojure.core macros?#2019-10-0223:34seancorfield(and, yes, you're going to be on some pretty shaky ground if you copy'n'paste that code into your system and then distribute it)#2019-10-0223:34noprompt@seancorfield Imagine you want to write a macro that has the form as defn.#2019-10-0223:36nopromptAnd imagine that I want to use conform to parse ::core.specs/defn-args.#2019-10-0223:36nopromptThats my situation.#2019-10-0223:37seancorfieldAh, so your code was relying on the internal names used in the those specs?#2019-10-0223:37nopromptIā€™m noticing youā€™re using the word ā€œinternalā€.#2019-10-0223:37seancorfieldOnly two top-level specs changed their names.#2019-10-0223:38seancorfieldIf you conform, you have to rely on the implementation of the spec.#2019-10-0223:38nopromptYeah but specs are globally publicā€¦#2019-10-0223:38seancorfieldWe've used alpha versions of Clojure libs in production for over eight years and sometimes Cognitect change stuff while it's alpha and break your code. That's the risk of using alpha libs.#2019-10-0223:39nopromptI hope you can appreciate the irony in this.#2019-10-0223:39nopromptšŸ™‚#2019-10-0223:39nopromptTo me its a bit surprising given the critique of SemVer, etc.#2019-10-0223:39nopromptSo suddenly calling it ā€œalphaā€ makes it okay to make breaking changes?#2019-10-0223:40noprompt::defn-args2?#2019-10-0223:40seancorfieldThey've been very clear that alpha means "can change/break". Things are only guaranteed additive/fixative once the alpha label goes away.#2019-10-0223:40nopromptIts not on the README.#2019-10-0223:40nopromptWhere were they clear?#2019-10-0223:41seancorfieldThey've said it repeatedly publicly. Pretty sure even Rich has mentioned that in talks...?#2019-10-0223:41nopromptPublicly?#2019-10-0306:57jaihindhreddyRich says it in the same talk where he contends SemVer is broken. Search for alpha in https://github.com/matthiasn/talk-transcripts/blob/master/Hickey_Rich/Spec_ulation.md Funny thing though, Rich also says something like:
But, that is not to say just leave your thing 0.0.967. At a certain point, you are going to have users, and whether you change it to 1.0 or not, they are going to be depending on your stuff.
And that is where spec is going right now IMHO. But his Maybe Not talk and Alex Miller's blog eased my fears of spec remaining in this state forever.
#2019-10-0223:41nopromptLike on Twitter?#2019-10-0223:41nopromptThe ML?#2019-10-0223:42nopromptI rarely, if ever, watch talks.#2019-10-0223:42nopromptThe README and the source code are far more public.#2019-10-0223:42nopromptThe disclaimer should be there.#2019-10-0223:43taylorwhat would it say#2019-10-0223:43nopromptWhat Sean said.#2019-10-0223:44nopromptHow about > Alpha means this library ā€œcan change/breakā€ do not rely on it.#2019-10-0223:44nopromptBecause I obviously misunderstood the situation.#2019-10-0223:45seancorfieldI'm sorry you've missed it being discussed on the mailing list and in Rich's talks and here on Slack (where it has come up several times in several situations).#2019-10-0223:46nopromptI just want answers to my questions.#2019-10-0223:47nopromptCan/How do I copy the source code?#2019-10-0223:48seancorfieldNot legally into anything you are distributing under an incompatible license.#2019-10-0223:49seancorfieldIf you want to use EPL code in your OSS project, consider changing your license to EPL so it's compatible.#2019-10-0223:49nopromptOkay, thatā€™s what I gathered by reading the EPL website.#2019-10-0223:50nopromptIts fine, I can write the specs myself in this case. I was just hoping I could take a short cut.#2019-10-0223:50nopromptšŸ™‚#2019-10-0223:50seancorfieldGiven the day I've had working on our Spec 2 branch, I do sympathize (even if it may not sound like it).#2019-10-0223:51nopromptHonestly, it didnā€™t feel that way but you saying that makes me feel like you do.
#2019-10-0223:51nopromptI just want to get my users and teammates out of a broken state quickly.#2019-10-0223:52nopromptIā€™ve given up on trying to supply feedback or contribute to anything in the clojure repo due to conversations that go the way they do regarding these kinds of frustrations.#2019-10-0223:53seancorfieldYeah, I can detect a fair amount of frustration/anger...#2019-10-0223:53nopromptTransparency would be nice on these topics; one canā€™t be expected to keep up with every feed of data.#2019-10-0223:53nopromptThe repo and source code is about as public and as inclusive as any place to explain semantics.#2019-10-0223:54nopromptIā€™ve made the effort in Meander to call out the versioning semantics I employ so there is no misunderstanding.#2019-10-0223:54seancorfieldWhen Rich introduced spec in 2016, he was pretty clear that it was alpha and subject to change (I just checked the transcript of one of his talks from back then). He also called out alphas as "potentially breaking" in his Spec-ulation talk (which is where his main criticism of SemVer was made.#2019-10-0223:55seancorfieldSemVer itself is broken tho'. While you're in alpha, you can make (breaking) changes. Once you have a non-alpha, you should only make accretive/fixative changes. SemVer doesn't help with that.#2019-10-0223:55nopromptThat makes this assumption that people will be exposed to those things or, if you have my memory, remember them.#2019-10-0223:56nopromptJust slap a phrase on the README that says, upfront, what the version semantics are instead of relying on them being implicitly understood.#2019-10-0223:57nopromptWhat I took from Rich was if Iā€™m going to make a breaking change, I make a new namespace, artifact, etc.#2019-10-0223:57nopromptAnd that works fantastic.#2019-10-0223:57seancorfieldIf you're not in alpha, yeah.#2019-10-0223:57nopromptIts a greek letter.#2019-10-0223:57seancorfieldIt was an important caveat in his talk šŸ™‚#2019-10-0223:57nopromptRich doesnā€™t get to decide what that means.#2019-10-0223:58nopromptHe canā€™t have it both ways.#2019-10-0223:58seancorfield(and it's been the practice in Clojure overall for years)#2019-10-0223:58nopromptSaying that ā€œalphaā€ can break is just weird thing.#2019-10-0223:58nopromptNone of these versions can break but this one can.#2019-10-0223:58nopromptWhat?#2019-10-0223:59nopromptBut, hey, if the semantics were with the source, fine, whatever, those are Richā€™s rules. Cool.#2019-10-0223:59nopromptMy point is theyā€™re not transparent.#2019-10-0300:00nopromptPut the semantics with the source.#2019-10-0300:00seancorfieldWell, we're just going to disagree on how clear we think the Clojure/core team have been on that... ĀÆ\(惄)/ĀÆ#2019-10-0300:00Alex Miller (Clojure team)I've added a warning to the readme of both core.specs.alpha and spec.alpha#2019-10-0300:00seancorfieldProblem solved! šŸ™‚#2019-10-0300:00noprompt@alexmiller THANK YOU!#2019-10-0300:00Alex Miller (Clojure team)If you have another suggestion of how to signal this, please let me know#2019-10-0300:00nopromptI literally have tears in my eyes. šŸ˜‚#2019-10-0300:01Alex Miller (Clojure team)sorry it wasn't clear#2019-10-0300:01nopromptBut they are now! Which is great!#2019-10-0300:01nopromptšŸ˜„#2019-10-0300:01nopromptYou have literally changed my attitude.#2019-10-0300:02Alex Miller (Clojure team)I think making alpha ok with breaking stuff is fine, with the caveat that having something in alpha for years and deps of official releases is admittedly weak sauce#2019-10-0300:02Alex Miller (Clojure team)yet, that's where we are#2019-10-0300:02Alex Miller (Clojure team)I have a burning desire to move these libs out of alpha and stabilize#2019-10-0300:03nopromptSpeaking from my experience with the greek letter style namespaces, its been incredibly liberating to not have to worry about breaking users.#2019-10-0300:04Alex Miller (Clojure team)and as fair warning, core.specs.alpha is eventually going to get updated to depend on spec-alpha2 (or some future non-alpha variant) and will necessarily break again. But on the upside, these will be new names, not breaking versions of old names.#2019-10-0315:49nopromptIs there an appropriate place to discuss this or get an explanation? It doesnā€™t seem like a necessary requirement for spec alpha to depend on any other version of spec.#2019-10-0315:53nopromptPut another way: is there a path forward that doesnā€™t cause breakage? Breakage should be taken seriously and avoided at all costs. It takes real time and effort on behalf of users to deal with it.#2019-10-0316:07Alex Miller (Clojure team)We have had some discussion about it but are deferring until later the full analysis of what we will do for migration. Most likely, there will be a new lib (probably core.specs) with new namespaces and new (spec 2) specs. There are a number of possible paths to continuing to support spec 1 specs (via some adapters possibly) and many choices about to what extent they are supported. Exploring this in full depth is more than I'm going to do in this slack thread.#2019-10-0316:08Alex Miller (Clojure team)You're preaching to the minister here on breakage, but as an alpha library the plan was always to ultimately be replaced and we've been pretty vocal about that from the start.#2019-10-0316:16nopromptWhere does the full depth discussion take place?#2019-10-0316:52Alex Miller (Clojure team)N months from now, Rich and I will go through a design process to try to flush out all the questions and answers around this#2019-10-0316:52Alex Miller (Clojure team)how and what kind of feedback we'll need at that point I'll defer to that point#2019-10-0316:53Alex Miller (Clojure team)what the world will look like then, I have no idea so I'm not going to worry about it now#2019-10-0300:04nopromptThats the win Iā€™ve experienced.#2019-10-0300:05seancorfieldclojure.spec.alpha -> clojure.spec-alpha2 has some major breakage for users who choose to migrate. Most mainstream stuff is unchanged but there's a lot of "dragons" if you've ventured off that path šŸ™‚#2019-10-0300:05nopromptIts been incredibly liberating for me to be able to make new versions of my library without worrying about breaking users or, worse, stagnating.#2019-10-0300:09nopromptMaybe I took what Rich said to the extreme but, in practice, what Iā€™ve come to discover is that strive for a ā€œstable versionā€ is just a withdrawal symptom of breaking away from a bad versioning habit and way of thinking about a project.#2019-10-0300:10nopromptLike, it no longer bothers me that the next version of Meander, when I get to it, will live at meander/zeta. I donā€™t care that I canā€™t (:require [meander.core :as m]). (:require [meander.zeta :as m]) is fine.#2019-10-0300:11nopromptThe whole idea of a ā€œstable versionā€ is odd to me when we all know the only thing we can rely on as software developers is that things are rarely, if ever, stable.#2019-10-0300:14nopromptSo, I would hope, that spec doesnā€™t devolve to SemVer at some point.#2019-10-0300:15nopromptI think it would be a step back to one day to have [clojure/spec X.X.X].#2019-10-0300:16nopromptThat just puts you back in the soup.#2019-10-0300:16Alex Miller (Clojure team)well, you will have that#2019-10-0300:16Alex Miller (Clojure team)but at the point we go there, we will try to maintain only additive changes after that point#2019-10-0300:17nopromptRight but the point is by not doing that you donā€™t have to do what you just said.#2019-10-0300:17nopromptYou donā€™t lock yourself in.#2019-10-0300:17nopromptOnce you go back to SemVer, youā€™re locked in.#2019-10-0300:17Alex Miller (Clojure team)it's not sem ver#2019-10-0300:17nopromptYou have to have the discipline to do what you just said.#2019-10-0300:17Alex Miller (Clojure team)it's an increasing set of numbers that make only additions#2019-10-0300:18Alex Miller (Clojure team)I believe we have that discipline :)#2019-10-0300:18Alex Miller (Clojure team)and will submit the rest of clojure as evidence :)#2019-10-0300:18nopromptBut why put yourself in a situation where you necessarily must have discipline.#2019-10-0300:18nopromptšŸ™‚#2019-10-0300:18Alex Miller (Clojure team)because supporting an ever changing set of namespaces across versions is irritating?#2019-10-0300:19Alex Miller (Clojure team)there's a balance of tensions here#2019-10-0300:19nopromptHow would that be any different from just depending on a fork?#2019-10-0300:19nopromptI canā€™t speak for others, but I donā€™t see that as an irritant.#2019-10-0300:20nopromptThe question is ā€œhow do you know youā€™re at a stable point where its possible to only sustain the software on discipline you can be sure you have?ā€#2019-10-0300:20Alex Miller (Clojure team)in general, I would rather just bump versions than bump versions AND update the namespaces in all my code#2019-10-0300:22nopromptIn principal you can do that already with any artifact that uses SemVer in honestly and never has a major version bump.#2019-10-0300:22Alex Miller (Clojure team)I'm going to just say experience. but changing to new namespaces or even new lib names is not an option that's lost - we can always do that if needed#2019-10-0300:23Alex Miller (Clojure team)I'm suggesting a small thing for users while still retaining the option to do the big thing for users#2019-10-0300:23Alex Miller (Clojure team)that seems better regardless#2019-10-0300:24nopromptWell, Alex, Iā€™m not going anywhere anytime soon as far as I can tell so Iā€™m looking forward to seeing where this goes. šŸ™‚#2019-10-0300:24Alex Miller (Clojure team)cool#2019-10-0300:25nopromptThough, I donā€™t mind a little typing. šŸ˜‰#2019-10-0300:25nopromptI gotta go be a dad but, again, thanks @alexmiller for making that change to the README.#2019-10-0300:31Alex Miller (Clojure team)ditto on dad time#2019-10-0308:53roklenarcicCurrent spec alpha has a function s/merge which works like s/and in that it will only conform the last spec. I am wondering if spec-tools coercion suffers from same problem?#2019-10-0309:26ikitommi@roklenarcic there is spec-tools.core/merge#2019-10-0309:29ikitommi> .. that selects only the specced keys from each conformed result, then merges those results onto the original input. This avoids overwriting conformed values with unconformed values while preserving all unspecced keys of the input. Fixes #90. By Arttu Kaipiainen.#2019-10-0309:39roklenarcicCool thanks#2019-10-0309:39roklenarcicdoes it work this way with s/conform or only with coerce?#2019-10-0309:40ikitomminot sure, please test šŸ™‚#2019-10-0520:44roklenarcicHm, ended up not using coerce, because it doesn't use any conforms defined in specs.#2019-10-0604:53ikitommithere is also decode, which does both. It's quite messy to try to implement transforming from a 3rd party library. https://clojure.atlassian.net/plugins/servlet/mobile?originPath=%2Fbrowse%2FCLJ-2251#issue/CLJ-2251 would make things better.#2019-10-0604:53ikitommihttps://github.com/metosin/spec-tools/blob/master/src/spec_tools/core.cljc#L232-L248#2019-10-0311:56practicalli-john@alexmiller just like to say thanks for all the work you and the team have done and continue to do with Spec. It is much appreciated. Thank you.#2019-10-0316:08jonathanHi all. I'm trying out spec alpha2. I noticed I accidentally forgot to include a key in my schema, but s/select does not seem to validate that the key exists in the schema.
(s/def ::first string?)
(s/def ::last string?)
(s/def ::user (s/schema [::first ::last]))
; Should the select blow up? email is not in spec
(s/def ::create-user (s/select ::user [::last ::email]))
#2019-10-0316:09ghadiit doesn't care in spec1 either#2019-10-0316:09Alex Miller (Clojure team)s/schema and s/select both support open maps (extra keys are ok), just like s/keys#2019-10-0316:10jonathanI thought (s/select ::user [::last ::email])) is specifically saying I'm choosing to require ::email from the ::user schema#2019-10-0316:10Alex Miller (Clojure team)whether you should get some feedback if s/select requires things not in the schema is a reasonable question, and not something we've talked about#2019-10-0316:10jonathan:+1:#2019-10-0316:12jonathanWithout any validation it seems like I don't need the schema at all except for documentation purposes#2019-10-0316:48Alex Miller (Clojure team)several things - 1) supports gen in both schema and select, 2) basis for closed spec checking if you want to do that, 3) yes, docs#2019-10-0318:40jonathanThanks. I appreciate your work on spec!#2019-10-0316:14jonathan
(s/explain ::create-user {::last "Smith"})
#:user{:last "Smith"} - failed: (fn [m] (contains? m :user/email)) spec: :user/create-user
^^ Validates email perfectly fine
#2019-10-0316:23seancorfieldOne question that came to my mind yesterday as I worked through getting our entire code base switched to the latest Spec 2 and our full test suite passing: how will folks deal with the world when they want to use Spec 2 but they rely on a chain of dependencies where one of the downstream libs still depends on Spec 1. https://github.com/cognitect-labs/anomalies/blob/master/src/cognitect/anomalies.cljc is a good example: very small, focused lib that defines Spec 1 specs and could be depended on by any number of libs in your tool chain.#2019-10-0316:25seancorfieldTo get our code base passing all its tests, I had to essentially recreate Anomalies' specs in my code, I had to create Spec 2 compatible versions of expectations.clojure.test (one I maintain), and worldsingles/web-specs (work maintains), since those are downstream dependencies using Spec 1.#2019-10-0316:26seancorfield@alexmiller If Spec 2 becomes clojure.spec (no alpha), how will folks deal with a mixed Spec 1 (alpha) / Spec 2 (contrib) code base?#2019-10-0316:27seancorfieldIn particular, in our case, we create specs in our own code (Spec 1 on master, Spec 2 on a branch) that incorporate the specs from Anomalies -- which I suspect is going to be a fairly common situation that folks get themselves into...#2019-10-0316:49Alex Miller (Clojure team)I think it's possible to build an adapter layer from spec 1 to spec 2, but we're going to push off answering these questions till we have something good enough to migrate to#2019-10-0316:54seancorfieldFair enough šŸ™‚ Is there a specific outline of work left to do on Spec 2 at this point? Or is even that aspect not "baked" yet?#2019-10-0317:46Alex Miller (Clojure team)there's a rough list at the end of my clojutre talk#2019-10-0317:56taylorv nice talk#2019-10-0317:47Alex Miller (Clojure team)
* More on map forms
* Function specs, ret/fn specs, defn
* Metadata / doc strings
* Open enumerated sets
* Spec traversal
#2019-10-0317:48Alex Miller (Clojure team)that doesn't all necessarily have to be done to be at a release point if we feel it's additive from that point#2019-10-0317:49Alex Miller (Clojure team)so the actual list may be shorter (or longer) but certainly nailing down 1,2, and probably 4#2019-10-0317:49seancorfieldAh, and I just watched that the other day and forgot about that list. Sorry.#2019-10-0317:50seancorfieldWhat do you mean by "Open enumerated sets" in that context?#2019-10-0317:50seancorfieldAnd "More on map forms" means the unqualified, inline s/schema definitions?#2019-10-0317:55Alex Miller (Clojure team)re open enumerated sets - using a set as a predicate #{:red :green :blue} is a different form of "closed" predicate. Often you want to say "these constants", but also other things later#2019-10-0317:56Alex Miller (Clojure team)"more on map forms" = more work on the "map" form of specs#2019-10-0317:56Alex Miller (Clojure team)as the maps currently in there are a first pass and we will likely change the form of some of them and probably automate the form<->map definition somehow#2019-10-0317:58seancorfieldAh, the "AST" stuff... OK.#2019-10-0317:59seancorfieldRe: enumerated sets. That sounds interesting. I'm not sure I can come up with a use case right away tho', but I can see how set literals are currently "closed".#2019-10-0318:04Alex Miller (Clojure team)options are a common case#2019-10-0321:02kennyAnyone know of a library that lets you strip extra keys of a multi-spec/parameterized map? For example, if a map with :type set to :a requires keys :a1 and :a2, any additional keys would be removed.#2019-10-0511:57dmarjenburghYou can use select-keys, but the problem is that you define the key set in two places. I had a similar problem and defined the schema as data {:req [::a1 ::a2] :opt [::a3]} . From there, I use the data to generate the spec and the keyset.#2019-10-0519:42nenadalmspec-tools can do that (in case I understand correctly): https://github.com/metosin/spec-tools/blob/master/docs/01_coercion.md#spec-coercion#2019-10-0521:54kennyIt needs to be a bit smarter than that because multi-spec is used. Iā€™d only like to select the keys that the multi-spec for a particular dispatch defines. #2019-10-0521:54kennyIdeally itā€™d also be recursive. #2019-10-0321:33hmaurerHello šŸ™‚ Possibly a silly question but what is the recommended way of working with specs whose data they model I want to serialise and deserialise to JSON with unnamespaced keys?#2019-10-0321:34hmaurerso, in clojure the keys should be namespaced (since, as far as I understand, thatā€™s the only way to work with spec), but in JSON they should not, and when deserialising JSON into a clojure datastructure I want to automatically namespace the keys based on some spec#2019-10-0321:34hmaurerI thought s/conform would do this if I specified :req-un, but it doesnā€™t appear to#2019-10-0321:38hmaurerseems like https://github.com/metosin/spec-tools/blob/master/docs/01_coercion.md is what I am looking for?#2019-10-0322:09seancorfield:req-un and :opt-un are for unqualified key names -- but let you associate qualifiers for the matching spec names.#2019-10-0322:11seancorfield
(s/def :foo/bar string?)
(s/def :foo/map (s/keys :req-un [:foo/bar]))
(s/def :quux/bar int?)
(s/def :quux/map (s/keys :req-un [:quux/bar]))
;; so you have have {:bar "s"} in one context and {:bar 42} in a different context
Does that help @hmaurer?
#2019-10-0322:18hmaurer@seancorfield thanks! I think it does; initially I was thinking I would want some sort of way to deserialise {"bar": 42} (json) into something like {:foo/bar 42} in a spec-driven way, but I could just use req-un and have unqualified keywords everywhere#2019-10-0322:29seancorfieldYeah, I'm not sure what to recommend for switching between "JSON" style maps and "internal" (qualified) maps. I think a lot of that is context-dependent / application-dependent so I'm not sure there's a generic approach that suits everyone.#2019-10-0322:31seancorfieldGiven the specs, you can always get the form of the spec and get the list(s) of keywords used in the spec and you could create a generic way to call clojure.set/rename-keys or simply process the (JSON) map to qualify the keys yourself. Much will depend on exactly what your application needs and why you need to perform that transformation.#2019-10-0322:40hmaurerAlright; I'll look into that. Thank you#2019-10-0502:23gfrederickshttps://twitter.com/gfredericks_/status/1180307358297346053#2019-10-0514:15mishaare there any pre-requisites? so you wouldn't have to baby-sit too much#2019-10-0520:59roklenarcicSeems there's an inconsistency about how conforming works if you have qualified or unqualified s/keys. Given specs:
(s/def ::a (s/conformer str))
(s/def ::b (s/keys :req []))
(s/def ::a1 (s/merge (s/keys :req-un [::a]) ::b))
(s/def ::a2 (s/merge (s/keys :req [::a]) ::b))
You get:
(s/conform ::a1 {:a 1})
=> {:a 1}
(s/conform ::a2 {::a 1})
=> #:clj-rest-client.core{:a "1"}
Weird.
#2019-10-0521:28seancorfield@roklenarcic Interesting. I tried it with the latest Spec 2 and it's the same. Also the same behavior using s/schema (with [::a] for qualified keys and {:a ::a} for unqualified keys).#2019-10-0521:38ikitommiI think the reason is that clojure.spec.alpha/merge separately conforms each merged spec, then merges the results, so the unconformed value from the lateer overrides the result#2019-10-0521:38Alex Miller (Clojure team)Merge only confirms the last spec#2019-10-0521:39ikitommiSecond works because all namespaced are always confirmed, even if they are part of the spec.#2019-10-0521:39Alex Miller (Clojure team)^^#2019-10-0521:50seancorfieldAh yeah. That makes sense!#2019-10-0522:51roklenarcicHow? Makes no sense at all to me that it matters whether I use namespaced keys or not.#2019-10-0522:53roklenarcicAlex says that merge only conforms the last spec, but if I s/merge multiple s/keys specs that use conformers and that use namespaced keys, all the keys are conformed not just the ones from the last merge argument#2019-10-0600:46seancorfield@roklenarcic If you have a spec :foo/bar and you conform any map that contains :foo/bar, it will get checked
spec.demo=> (s/def :foo/bar int?)
:foo/bar
spec.demo=> (s/conform (s/keys) {:foo/bar "x"})
:clojure.spec.alpha/invalid
spec.demo=> (s/conform (s/keys) {:foo/bar 42})
#:foo{:bar 42}
spec.demo=>   
#2019-10-0600:49seancorfieldSo the s/merge with the unnamespaced key doesn't get conformed because s/merge only conforms the last spec -- ::b -- but the qualified keys all get conformed because regardless of whether they're even in the spec.#2019-10-0608:16roklenarcicwait, so namespaced keys in maps get checks even if they are not in the spec I'm using to check them with?#2019-10-0608:46mpenetYes#2019-10-0608:46mpenetYou can confirm this with (s/keys)#2019-10-0711:03sgerguriHow can I force spec to not validate a namespaced key against an optional spec in a map? I have a multispec with two clauses where the only difference is in the inclusion of a namespaced key, but because the key matches a spec the library will simply go ahead and validate that key against the spec even though it should not for that specific clause.#2019-10-0712:44Alex Miller (Clojure team)keys will always validate namespaced keys#2019-10-0712:45Alex Miller (Clojure team)Thatā€™s kind of central to the whole design of spec#2019-10-0713:33sgerguriIt just feels inconsistent to how un-namespaced maps are treated - though I guess one could technically argue that if I'm saying a namespaced key should have a particular shape then it should have it whether or not it is required in a map. I have since tweaked it so that this is indeed the case, though given how namespacing is a particularly burdensome thing in our service (I have stripped it from everywhere except for one namespace, and even there it feels a bit redundant) this was quite the surprise in behaviour.#2019-10-0713:37mpenetif you think about them (kinda sorta) like Types it makes sense. a :animal/lion is always the same thing.#2019-10-0713:38favilaRich has a philosophical position that maps should be a bag of keys, and type information comes primarily from keys not from the bag#2019-10-0713:39favilahttps://clojure.org/about/spec#_map_specs_should_be_of_keysets_only#2019-10-0714:13dominicmI think is inaccurate to characterise it as philosophical. Having consistent meaning associated with names in a program makes your program easier to reason about.#2019-10-0716:37sgerguriI found that in our specific use case, where we have lots of things that look the same up to a certain level and can diverge in some specifics, namespacing became a burden both during testing and during main program flow. Our use case might just be a special scenario, though I suspect I would probably do it the same way again if I was facing largely similar scenario - i.e., internal entities with roughly similar shapes, as I really don't like the idea of mixing different namespaces in a single map.#2019-10-0716:48dominicmI hate working on programs where the thing I'm looking at shares a name with something else but is different. E.g. if "products" in one place referred to the ids of products you're allowed to see, but elsewhere referred to a list of product entities from the database.#2019-10-0716:50dominicmMixed namespaces don't create any bother for me, I believe that exists so you can have separate pieces of code which operates on the same entity as long as it fulfils the data contract. E.g. It's easier to understand code which operates on "Twitter account" which is a limited number of details, rather than "user" which is much larger and requires more juggling in your head.#2019-10-0717:19sgerguriYeah - and I'd imagine that would work well if the maps truly mixed contexts. In our case, they mostly mix different datatypes that tend to look similar to one another, with a common "envelope" map around them. Given the benefits we gain out of being able to shove everything into a common processing pathway stripping the namespacing ended up making things a lot less unwieldy than it was.#2019-10-0717:30dominicmSounds like stamp coupling#2019-10-0717:31dominicmWhen taken to an extreme, these ideas are still coupling. Functional connections are the only kind that's free from this, but that's a lot of typing and not much abstraction.#2019-10-0718:10favilaI donā€™t disagree itā€™s nice when keys = types and they are well namespaced and shared across entire application. I think Dominic took ā€œphilosophicalā€ as a slur which I absolutely do not mean#2019-10-0721:34dominicmI didn't think you were necessarily disparaging it, but I hear the term "philosophical" a lot when people feel a stance is without pragmatics and more rooted in perfectionism at cost to real software. I just wanted to attempt to explain the importance of this stance.#2019-10-0721:34favilafair enough#2019-10-0721:35favilayeah, I think it is slightly impractical, but aspirational#2019-10-0721:36favilathe impracticality for me is entirely centered around those two use cases: typing someone elseā€™s data which doesnā€™t share that philosophy (which, honestly, is the vast majority of systems! most think property meanings should come from the record type they are in), or cases where the key is used ā€œas a valueā€ in e.g. a DSL instead of its ā€œnaturalā€ meaning.#2019-10-0721:38seancorfieldFor me, that's why there is the split between unqualified and qualified keys: the former are "out there" and you have to assign meaning on a per-case basis; the latter should be unique keys within your domain and should have a single meaning for each name (in terms of the data type).#2019-10-0721:38dominicmIf you're a piece in a pipeline and you need to keep the data fairly untainted, I can appreciate that difficulty.#2019-10-0721:41seancorfield(and as an adjunct to my comment: if you can't assign a single meaning for a name "globally" then don't use a qualified keyword for it -- stick with unqualified keys that you have more flexibility with)#2019-10-0718:10favilaHowever, keys in fact do not always have the same type, e.g. a map key in a DSL vs a map key in data#2019-10-0718:11favilae.g. datomic pull expression map vs the value of the pull#2019-10-0718:11favilathe ā€œbagā€ sometimes imparts meaning to the keys, IOW#2019-10-0718:12favilanot to mention having to deal with other systems which donā€™t share your ā€œmaps are only keysetsā€ philosophy, but I think the spec2 select machinery is an acknowledgement of that problem#2019-10-0718:13Alex Miller (Clojure team)spec 2 select just says that the notion of required/optional keys is contextual#2019-10-0718:13Alex Miller (Clojure team)it still maintains strong semantics for attributes across aggregates#2019-10-0718:13favilabut it also has contextual key renaming, no?#2019-10-0718:13Alex Miller (Clojure team)no, it doesn't#2019-10-0718:14Alex Miller (Clojure team)it has the ability to do binding of un-qualified keys to specs, but un-qualified keys only#2019-10-0718:18favilathatā€™s all I meant#2019-10-0718:19favila{:first :my.app.name-part/first-name} for example#2019-10-0718:20favilaI think schema is what I am talking about, as being able to impart some meaning to the aggregate#2019-10-0718:20favilainstead of being ā€œmerelyā€ a keyset#2019-10-0718:21Alex Miller (Clojure team)the intent here is not that attributes can be assigned many different meanings, but that unqualified keys are missing the link to attribute semantics#2019-10-0718:21Alex Miller (Clojure team)it's still just a collection of attributes though#2019-10-0718:27favilayes it does not solve that problem#2019-10-0808:42mishayeah, I felt tension speccing pull-patterns vs pulled data too, but I think pull-pattern should not be specced with s/keys, which kinda resolves the situation#2019-10-0809:29sgerguriIn my case, we only do two layers of transformation before passing the data out. Input is big, mangled and flaky. Given all that it felt the domain - if there is one indeed - was rather thin, and thus there were few benefits to be gained in namespacing everything. But then we're using it for data engineering over Kafka, so we're basically talking about transit transformations, rather than domain manipulations.#2019-10-0809:36ikitommi> spec 2 select just says that the notion of required/optional keys is contextual ... but there is no explicit syntax of marking some keys as optional in select? Rather implicit "If the key if not part of the select, it's optional", right?#2019-10-0810:07valeraukothere's no equivalent for the opt declaration in spec 1? "if present, it has to be this shape" thing#2019-10-0812:08Alex Miller (Clojure team)Schemas in spec 2 are all optional#2019-10-0812:10Alex Miller (Clojure team)Select specifies whatā€™s required in a particular context#2019-10-0821:25rapskalianI have a question that I think is related to the conversation above. If I have the following datomic-style schema:
{:db/ident       :user/email
    :db/unique      :db.unique/identity
    :db/valueType   :db.type/string
    :db/cardinality :db.cardinality/many}
How would one spec :user/email? The difficulty stems from the fact that I can transact {:user/email "myemail"} or {:user/email ["myemail"]}, but when I query the db, I would always receive {:user/email ["myemail"]}. Perhaps itā€™s recommended to define contextual specs:
(s/def :email/address ::non-empty-string)
(s/def :user/email (s/coll-of :email/address))    ;; matches the schema in name and shape
(s/def :user.input/email (s/or :one :email/address :many :user/email))  ;; matches what the boundaries of the system would accept
Iā€™m especially interested in the use-case of integrating specs with datomicā€™s :db/ensure feature in order to validate entities before they enter the db.
#2019-10-0821:26Alex Miller (Clojure team)with specs it's always best to try to "state the truth" so I'd go with something that took either#2019-10-0908:08dominicmI think the truth is that these are semantically different uses of the same key. One is a DSL, and one is data. Requiring all my functions to handle a collection or singular because spec doesn't allow me to distinguish the contexts makes pointless additional work for me. You could move the spec to the aggregate, applying additional constraints. I'm not sure how well that would work on a conventional spec 2 system.#2019-10-0821:27Alex Miller (Clojure team)for the last question, might be best to ask that in #datomic, I'm not well-versed enough to have a good answer#2019-10-0821:28rapskalianThat makes sense. The truth here being that, indeed, the input is a different thing than what comes out of the database?#2019-10-0821:29rapskalianI suppose I could restate the question as: if I want my specs to ā€œfollowā€ the schema, should I spec what comes out, or what goes in?#2019-10-0821:29rapskalianBy ā€œfollowā€ I mean ā€œpossess the same semanticsā€#2019-10-0821:34rapskalianI think I may have answered my own questionā€¦Iā€™m realizing that specā€™ing input is much more useful than having a truthful ā€œspec+schemaā€ (the latter of which is likely redundant)#2019-10-0821:35rapskalianIn other words, specā€™ing what comes out of the database is pointlessā€¦at that point itā€™s too late. Spec efforts should be focused at the input boundaries of the system.#2019-10-0821:36rapskalianInterestingly tho this makes spec and the :db/ensure feature of datomic feel a bit incompatible, because predicate functions given to :db/ensure have the signature (defn my-pred [db eid])ā€¦and so they can only validate what comes out#2019-10-0822:19favilaI think itā€™s better to think of the transaction as a DSL. You can spec a dslā€™s grammar, but not its meaning#2019-10-0822:20favilaso you canā€™t spec :user/email specifically as input to d/transact, only as an opaque key in a transaction map#2019-10-0822:22favilayou should either spec a level higher, e.g. some function that takes well-formed input and produces a set of transaction assertions/retractions/maps that do what you want; or a level lower with data-aware assertions, e.g. perhaps the DSL produces an ast which is easier to analyze, or in the case of datomic :db/ensure to guarantee your invariants.#2019-10-0822:23favilaitā€™s a little less obvious how to leverage spec in the ā€œlevel lowerā€ cases#2019-10-0822:24favilacombining spec with datomic, Iā€™ve had more luck specing what you would want to see out of d/pull, because thatā€™s whatā€™s flowing through the functions in an application#2019-10-0822:24favilathat feels like where spec shines the most#2019-10-0822:25favilaand Iā€™ve just accepted thereā€™s no silver bullet for the cases where the data approaches the edges, e.g. to serialize/deserialize or to transact to the db#2019-10-0900:54rapskalianThanks @favila thatā€™s very helpful >combining spec with datomic, Iā€™ve had more luck specing what you would want to see out of d/pull, because thatā€™s whatā€™s flowing through the functions in an application Iā€™m reaching this conclusion as well. It actually works very well for specā€™ing at that ā€œlower levelā€ you mentioned, because :db/ensure can easily make use of pull results. #2019-10-0918:05rapskalianThinking some more on this. I gave up on trying to use spec ā€œnearā€ the database. The schema is sufficient for specifying data at rest, coupled with some basic structural constraints (e.g. required keys, or in my case, ensuring no cycles are formed). Structural meaning only the relationships are constrained, not their actual values. I think this is what you mean by opaque @favila. With that in mind, spec seems to really flex its muscle upon data in motion; the transitions between valid data-at-rest so to speakā€¦ Thinking out loud šŸ™‚#2019-10-0918:06favilaPerhaps. I was thinking of it more in terms of speccing the grammar vs analysing the result#2019-10-0918:07favilafor a tx, [{:my-map myval}] the spec is really not anything about :my-map, but a vector of things, some of which may be maps, the keys of which are arguments to some transformation#2019-10-0918:08favilaso the key is ā€œopaqueā€ in that sense, itā€™s not specced as itself; map values are independent (gramatically) from the keys#2019-10-0918:09favilathink about what this would conform to: it would produce some kind of normalized ast with {:attribute-key :my-attr :attribute-scalar-value myval} for example#2019-10-0918:10favilaitā€™s hard to spec ā€œbothā€ at once, i.e. both the generic grammar and also some specific constraints on terms in it (constraints which would be enforced by an analyzer stage in a typical compiler)#2019-10-0918:19rapskalianI see. So to apply your compiler analogy to an ā€œapplicationā€, youā€™re referring to the fundamental difference between, say, the description of some mutation (as data), and the new database that results (assuming db-as-a-value semantics). The keys in both could also be said to be grammatically separate (?)#2019-10-0918:23favilayeah#2019-10-0918:24favilayour key inside the transaction mini-language means something different than your key in a map with your data in it#2019-10-0918:25favilain the first, itā€™s a parameter to some invisible astā€™s map key, in the later itā€™s the data itself#2019-10-0918:25favilamaybe map-as-syntax vs map-as-data#2019-10-0918:26favilayou canā€™t s/keys the keys inside a map-as-syntax, only s/map-of them#2019-10-0918:30rapskalianThatā€™s where my mapping of these concepts to spec was failing. s/keys seems to imply that a key must mean the same thing everywhere itā€™s used, but thereā€™s an assumption that youā€™ll be checking against map-as-data. It is inappropriate for map-as-snytax. s/map-of was the šŸ’” for me, the data there is purely syntax.#2019-10-0918:31rapskalianFor some reason I seem to have this desire to write ā€œthe base specsā€ of a system, but specs really donā€™t make sense without a contextā€¦#2019-10-0918:36favilayeah, for me that hurts most when the context is the type of record itā€™s in#2019-10-0918:37favilaI keep running into cases where a key has a wider domain, but can be narrowed/refined contextually by the type of its record.#2019-10-0918:37favilathis is a common feature of type systems inspired by class based inheritence or xsds#2019-10-0919:04rapskalianInteresting, I havenā€™t run into that yet. How have you been handling it? Seems you would need some ability to compose spec registries into hierarchies.#2019-10-0919:07rapskalianOr perhaps express that inheritance in the key itself, eg :slack.user/email#2019-10-0919:21favilaI handled it with this horrible macro: https://gist.github.com/favila/ab03ba63e6854a449d64d509aae74618#2019-10-0919:21favilaI think with spec2 schema I may not need it, I may be able to refine the spec inline#2019-10-0919:22favilabut Iā€™m not sure, I havenā€™t tried spec2 at all#2019-10-0919:21favila#2019-10-1120:47mattparkerWhile playing around with spec2 I noticed that I can select keys that are not in the schema:
(s/def ::first-name string?)
(s/def ::last-name string?)

(s/def ::user (s/schema [::first-name ::last-name]))

(s/select ::user [:first-name])
Is this expected behavior? From reading the docs on select I expected it to select from the schema and provide a more restrictive spec. Is this the wrong way of thinking about it?
#2019-10-1121:02seancorfield@mattparker I think it's a known issue (I think I remember Alex mentioning this).#2019-10-1121:02mattparkerah ok. Thanks!#2019-10-1121:02seancorfieldIt would probably be nice for the above to throw an exception (that :first-name isn't present in the schema).#2019-10-1121:26Alex Miller (Clojure team)schemas and selects are both open, so this is the expected behavior#2019-10-1121:27Alex Miller (Clojure team)if doing closed spec checking, it would make sense for that to error (I'm not at a repl but I suspect it doesn't right now due to some lingering work there)#2019-10-1121:27seancorfieldAh, that might be what I'm thinking of...#2019-10-1121:27Alex Miller (Clojure team)select is about requirements#2019-10-1121:27Alex Miller (Clojure team)you must supply ... , not you can only supply#2019-10-1121:28seancorfield@mattparker
user=> (s/explain (s/select ::user [:first-name]) {::first-name "Sean"})
#:user{:first-name "Sean"} - failed: (fn [m] (contains? m :first-name))
nil
user=> 
#2019-10-1121:28seancorfield(so select can force key presence for things that are not aligned with specs for their values)#2019-10-1121:29mattparkerwouldn't that easily be achieved by expanding the schema with union?#2019-10-1121:30mattparkerthat would make the first arg what's possible and then the second arg the restrictions#2019-10-1121:30Alex Miller (Clojure team)that's not what select is#2019-10-1121:31Alex Miller (Clojure team)select tells you what is required in a context#2019-10-1121:31Alex Miller (Clojure team)"restrictions" are about negation / closed world#2019-10-1121:32mattparkerwhat got me thinking that was in the wiki s/select is a spec op that uses a schema to define the world of possible keys#2019-10-1121:33mattparkerin this case :first-name is in the world of possible#2019-10-1121:33Alex Miller (Clojure team)well, there's a separate wrinkle here of qualified vs unqualified keys#2019-10-1121:33mattparkerbut I guess what this means is schema is a subset of what's possible#2019-10-1121:34Alex Miller (Clojure team)probably this is mostly that I could be better at writing words :)#2019-10-1121:34mattparkerit's a wiki for an alpha. I definitely don't take it as a contract haha#2019-10-1121:34Alex Miller (Clojure team)anything on that wiki is me writing, and not vetted by Rich#2019-10-1121:37mattparkerthanks for helping me understand!#2019-10-1122:18Steven ZhouA question around clojure.spec.test.check/check for its num-tests opt. Iā€™m trying to check how many times a snippet has run based on the print values as below.
(s/fdef foo :args (s/cat :s string?) :ret string?)
(defn foo [s] (prn "s") s)

(s/fdef bar
  :args (s/cat :x string?)
  :ret (s/fspec :args (s/cat :x string?)
                :ret string?))
(defn bar [x]
  (prn "x")
  (fn [y] (prn "y") y))

(alias 'stc 'clojure.spec.test.check)

(st/check `foo {::stc/opts {:num-tests 1}})
(st/check `bar {::stc/opts {:num-tests 1}})

s => printed 1 time
x => printed 1 time
y => printed 21 times
Could anyone give any insights here that why y has been printed out 21 times? And is there a way to control the number of tests run for a s/fspec block?
#2019-10-1122:25gfredericksfspecs are validated using their own test.check run#2019-10-1122:25gfredericksI doubt it's configurable#2019-10-1122:29Steven ZhouThanks for the quick response! That makes sense, will you able to point where that's configured in the source code? It will be really nice if I could configure that, since one of my fspec is running super slow and I don't want it to eat up all my resources.#2019-10-1122:50seancorfield@stevenz You can re-bind this dynamic var https://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/alpha.clj#L25-L27#2019-10-1123:21wilkerlucio@alexmiller hello, I just watched your video from the clojuretree. nice talk! I noticed one thing about the new schema / select thing, in the current spec1, its valid to create anonymous selections, like calling (s/keys) directly as a parameter or a return value, this possibility seems to be vanished with the requirement to name the things with s/schema before doing the structural selection with s/select, did I got that right? if not, is there gonna be a way to define nested selections without having to name it?#2019-10-1123:23Alex Miller (Clojure team)Thereā€™s no requirement to name either#2019-10-1123:24wilkerluciook, so the idea is to use like: (s/select [] [{::my/selection [::deep-value]}]), like this?#2019-10-1123:25Alex Miller (Clojure team)well, vector there, but yes#2019-10-1123:25wilkerluciofixed šŸ™‚#2019-10-1123:25Alex Miller (Clojure team)yep#2019-10-1123:25wilkerluciocool, thanks for the clarification#2019-10-1123:26Alex Miller (Clojure team)schemas can be put in the registry to reuse them, which we think is useful, but not required#2019-10-1123:26Alex Miller (Clojure team)selects are probably less useful to put in the registry#2019-10-1123:26wilkerlucioyeah, what makes it important to me, is that more and more I'm going in a modeling direction that I don't name groups of things most of the time#2019-10-1123:27wilkerlucioinstead, just say what "shape" a fn needs directly, so I can get very specific every time#2019-10-1123:27wilkerluciodo you think a select with arity 1 makies sense? so it would use empty keyset#2019-10-1123:50Alex Miller (Clojure team)Dunno#2019-10-1123:52kennyI have a map with a ::type key and a ::tags key. ::tags is a map. I want the keys in the ::tags map to depend on the ::type of the "top-level" map. Is this possible to spec? e.g., Where the ::tags map is spec'ed as (s/keys :req [::a]) for ::type ::a.
{::type ::a
 ::tags {::a 1}}
#2019-10-1123:56kennyIt seems like the options are either: 1) restructure the map so the type and tags are at the same level (this won't work because there needs to be a way to get the tags map easily) 2) Copy the ::type key into the ::tags map & use multi-spec (this isn't great because the data is redundant) 3) Create a s/and predicate on the top-level map that checks the ::tags using multi-spec after assoc'ing the ::type onto the ::tags map. (this feels icky but may be the best way) 4) Unqualify ::tags (I don't really like this because every other key in the map is qualified. The key :tags may be used all over the place so finding usages of the key becomes harder therefore making refactoring harder.)#2019-10-1203:15Alex Miller (Clojure team)well I would prioritize how you're going to use it as data over how to spec it - that's worth 10x more thought than how to spec it#2019-10-1203:16Alex Miller (Clojure team)and multi-spec does not require that you use a top-level key - the dispatch function can be anything#2019-10-1215:23kennyThereā€™s not way to change the spec for the nested ::tags map with multi-spec though. #2019-10-1215:24kennyThe data definitely belongs in a nested map like above. #2019-10-1221:16tianshuHow about the current status about spec2? Will it replace the spec1?#2019-10-1222:15seancorfield@doglooksgood That is my understanding, yes. Spec 2 will probably just become clojure.spec once it moves out of alpha and Spec 1 will be left as-is.#2019-10-1222:34Alex Miller (Clojure team)They may coexist for a while#2019-10-1222:49seancorfieldSpec 1 will "never go away" completely because a lot of code out there depends on it, so it depends on your definition of "coexist".#2019-10-1222:49seancorfield(Maven Central is forever šŸ™‚ )#2019-10-1223:27Alex Miller (Clojure team)By coexist, I mean that core continues to support both in critical touch points (error handling, registry, doc)#2019-10-1412:01yuhanIs there a regex op which expresses "n to m repetitions of a spec"?#2019-10-1412:29vlaaad@qythium &:
(s/valid? (s/cat :3-to-5-evens (s/& (s/* even?) #(<= 3 (count %) 5)))
          [2 2 2 2 2]) 
=> true
#2019-10-1412:36yuhanhmm, what if the sub-specs are variable length regex ops themselves?#2019-10-1412:39yuhansomething like: (re-find #"^(?:a+|b+){3,5}$" "aababbbbaa")#2019-10-1412:51jaihindhreddyI believe that cannot be done with a regular language. You can do (s/coll-of ::s :min-length n :max-length m) though.#2019-10-1414:21yuhanhmm, I'm not so familiar with automata theory, but isn't the above string-based regex constructed using a regular language?#2019-10-1414:22yuhanGood to know that I'm not missing anything obvious built-in though
#2019-10-1415:49andy.fingerhutSomething like a{1,2} can be hand-expanded into what it means, which is a|aa. The expansions can get pretty long, of course.#2019-10-1414:00Alex WhittHi! I have some s/keys specs in which the keys' specs use s/or, and the map is only valid for certain combinations of s/or paths. I ended up writing a function, keys-conform?, which allows me to give a map of keywords to spec path, and check those spec paths against the conformed values:
(s/def ::foo (s/or :int    int?
                   :string string?))
(s/def ::bar (s/or :branching ::foo
                   :kw        #{:a :b}))
(s/def ::map (s/keys :req [::foo ::bar]))

(deftest keys-conform?
  (are [?conform-test ?input ?result]
      (= ?result (keys-conform? ?conform-test (s/conform ::map ?input)))
    {::foo :int ::bar :kw}               {::foo 1 ::bar :a}            true
    {::foo :int ::bar :kw}               {::foo "hi" ::bar :a}         false
    {::foo :int ::bar :branching}        {::foo 1 ::bar "hi"}          true
    {::foo :int ::bar [:branching :int]} {::foo 1 ::bar :a}            false
    {::foo :int ::bar [:branching :int]} {::foo 1 ::bar 1}             true))

My typical use-case involves using keys-conform? in other specs:
(s/def ::specific-map
  (s/and ::map
         (s/or :both-int
               #(keys-conform? {::foo :int ::bar [:branching :int]} %)

               :string-and-kw
               #(keys-conform? {::foo :string ::bar :kw} %))))

(s/valid? ::specific-map {::foo 1 ::bar 1})
;=> true

(s/valid? ::specific-map {::foo 1 ::bar "hi"})
;=> false

(s/valid? ::specific-map {::foo "hi" ::bar :a})
;=> true

(s/conform ::specific-map {::foo "hi" ::bar :a})
;=> [:string-and-kw #:user{:foo [:string "hi"], :bar [:kw :a]}]
This works, but as I've been using it, I'm finding I also want access to more subtle conditions for valid paths, like "or" and "not", and the logic is getting more complex. So I'm wondering if there's already a built-in way to do what I'm doing, or if someone has written a library like this? I can't think of a good way to use multi-specs for this, but maybe someone else can? (Also, I'll preempt any possible objections that ::foo and ::bar should be split up into different keys. In my use case, they really do have to be the same key with different possible paths.)
#2019-10-1420:04seancorfield@alex.joseph.whitt Is there any sort of disciminant you could use to turn this into a multi-spec?#2019-10-1420:06Alex WhittThe problem seems to be that the branching at the level of ::specific-map is sort of multi-dimensional. So there's not necessarily one keyword that changes everything. The options are sort of mix-and-matchable. Do you think there might still be a way?#2019-10-1420:08Alex WhittThe domain in question is parsing a rather nuanced binary protocol called BACnet. The packets have a number of valid shapes, but just as many invalid combinations of fields.#2019-10-1420:08Alex WhittAnd the fields are obnoxiously polyvalent#2019-10-1420:12Alex WhittPerhaps there could be a solution in nested multi-specs? Like if the flag in the header says this is a foo-packet, check foo-spec, and foo-spec has a number of other options under it based on the given values?#2019-10-1420:22Alex WhittWell actually, it seems like multi-specs would just be replacing the use of s/or in ::specific-map... I don't think that would change the use of keys-conform?...#2019-10-1420:44misha@alex.joseph.whitt do you have specification document url?#2019-10-1420:45Alex WhittDo you mean you want to see my real use case?#2019-10-1420:45mishayes#2019-10-1420:47mishabut I think the issue here is your attribute tries to be too much different things. I'd try to model it with many attributes instead, e.g.
(s/def : int?)
(s/def :bac.str/foo string?)
,,,
#2019-10-1420:48mishamight save you some ifs down the road#2019-10-1420:50mishaanother question is: "why exactly do you want to spec that?"#2019-10-1420:50mishaand "how many valid combinations there are vs. how many possible ones?"#2019-10-1420:55mishalol, it's 150$ for pdf#2019-10-1420:58Alex WhittI can put a portion of my code up in a gist in a bit. I have a few main reasons why I want to spec it: 1. To validate packets and my parsing logic 2. To have the ability to generate random but valid packets for automated test purposes (testing an embedded device that supports the BACnet protocol). 3. Generative testing for code that touches BACnet How many valid combinations: Depends. For most parts that would use this pattern, probably less that 6 for each level of abstraction. The possible combinations are probably 2-3 times that number. Heh, yeah it's a closed standard. As for splitting the keys, that's a pattern I can and do utilize, but at some point it does boil down to one key representing multiple possibilities due to the nature of the fixed positions in the binary protocol. I'll keep noodling on it though.#2019-10-1421:00misha> BACnet currently defines 35 message types, or "services," that are divided into 5 classes. sound like you can put em into multispec#2019-10-1421:04mishaif messages are sequential, did you try to write reg-ex specs for them, instead of map ones? https://clojure.org/guides/spec#_sequences#2019-10-1421:06Alex WhittI'm working at a different level than the message types at the moment. At the message type level, absolutely a multi-spec works (and that's what I use.) Here's my code for the generic header: https://gist.github.com/WhittlesJr/4571fed1596400e242724972ee39b2d4#2019-10-1421:06mishaso the general advice is to reduce combinatorial explosion, and spec only things you need to spec. how exactly ā€“ depends on that protocol, and what exactly you are doing with it.#2019-10-1421:07mishacan you show raw header/message/whatever?#2019-10-1421:07Alex Whittyeah, one sec#2019-10-1421:08mishaI think this
(s/def ::class
  (s/or :application #{:application}
        :contextual  #{:contextual}))
can be just
(s/def ::class #{:application :contextual})
#2019-10-1421:09mishasorry, Sean kappa#2019-10-1421:10Alex WhittHere's an example capture of BACnet packets. Open in Wireshark and apply the bacnet filter. You'll probably also need to add UDP port 47824 as a BVLC decoding. (Analyze -> Decode as... -> +)#2019-10-1421:11Alex WhittI used the s/or there for testing branching in ::attributes#2019-10-1421:12Alex WhittAlternatively I suppose you could use a multi-spec, but seems to be 6 of one 1/2 dozen of the other#2019-10-1421:21mishawell, looking at raw data did not help opieop#2019-10-1421:22Alex WhittMerp. The BACnet spec is really dense and nuanced, takes a ton of contextualizing to grok the issues involved.#2019-10-1421:25mishathis is what I dislike about it: context. spec works great when attributes are the same regardless of context. and it (still) seems to me, flattening out amount of valid attribute combinations with different attribute names would be simpler solution, than trying to replicate 1to1 all the branchyness in specs (don't forget to write custom generators hehe)#2019-10-1421:27mishalooking at spec names (all ::/) namespaces of attributes don't mean much for you, so I'd try to encode types in those, to keep attribute name "polymorphic" (if you need for something)#2019-10-1421:28Alex WhittYeah, I have to wrestle with the high-context nature of the spec when writing the parsing logic. It's horrible. I'll look into breaking it up, but something tells me that it won't be a fully satisfying solution. Part of the purpose of my code is to represent the packet fully, much like Wireshark does. It will make it less obvious what the actual binary fields are saying if I have three different optional keys rather than one key that can have three different types. But that's more of a human-level issue rather than a code-level issue.#2019-10-1421:30mishawhat are you parsing exactly? what is your input to spec?#2019-10-1421:33Alex WhittIf you want to look at the capture, check out packet 337 -> "Building Automation and Control Network APDU" -> "list of Values:" -> "{[4]". You should be able to recognize the header in there, which is the part of the spec that my gist is dealing with.#2019-10-1421:34Alex WhittEverything under the "list of Values:" has one of those headers (or is a standalone "header" acting as an opening or closing tag)#2019-10-1421:38Alex WhittAnyway, the approach I'm taking is working, I was just curious if there was already a solution out there or if I was coming at this from the wrong angle. Depending on how this goes, I may just make a library out of keys-conform? with some well-defined syntax. If my employer will let me open source anything, that is... :c#2019-10-1421:40mishadoes it have generator tho? :)#2019-10-1421:43Alex WhittYeah, actually I'm doing pretty well with generators, surprisingly. I'm finding that the coercion logic a la spec-coerce is synergizing really well with generators.#2019-10-1421:44Alex WhittEspecially as you climb up the abstraction chain and you get more and more fields that are interdependent, the coercion approach is helping to keep generation sane#2019-10-1421:45Alex WhittBut maybe I have yet to hit a real barrier and it's lurking somewhere in the thickets, waiting to spring an ambuscade on my productivity#2019-10-1421:46Alex Whitt::attributes in the gist generates out of the box#2019-10-1511:52mpenetis there a way to express negation of other specs in a regex spec other than something like (s/+ #(not (some (fn [spec] (s/valid? spec %) [::foo ::bar])))#2019-10-1511:52mpenetI am sure I am missing the obvious#2019-10-1512:23Alex Miller (Clojure team)Not missing anything#2019-10-1512:23Alex Miller (Clojure team)Negation is fairly unusual in specs #2019-10-1512:24mpenetyeah it doesn't even work for my use case actually, I am better of with a simple predicate#2019-10-1514:06misha@mpenet is that pseudocode or actual code?#2019-10-1514:13mpenetpseudo#2019-10-1514:13mpenetI went with something else btw. I was specing a form similar to try/catch/finally#2019-10-1514:14mpenetgetting the body accurately was the target#2019-10-1617:22rickmoynihanIn spec 1 keywords are privileged with regards to s/keys. Iā€™d love for spec 2 to let me define other types of values to name key specsā€¦ e.g. (s/defkey "foo" integer?) (s/keys :req-key ["foo"]) Iā€™m not actually so bothered by string keys, as what Iā€™d really like to do is use URIā€™s as keys. My use case is that we work with RDF, and itā€™s really annoying having to convert between URI predicates and clojure keywordsā€¦ Iā€™d like to essentially do: (def rdfs:label (URI. "http://,,,,#label")) (s/defkey rdfs:label (s/or :xsdstring string? :langstring langstring?) (s/def ::resource (s/keys :req-key [rdfs:label])) Any ideas on whether this sort of thing might be possible in spec2?#2019-10-1617:23Alex Miller (Clojure team)no#2019-10-1617:24rickmoynihanimplementing my own s/keys seems like itā€™ll be hard work#2019-10-1617:25rickmoynihanmight there be an easy way to do it that can leverage stuff in spec?#2019-10-1617:25Alex Miller (Clojure team)the idea of keywords as names for specs is deeply embedded#2019-10-1617:25Alex Miller (Clojure team)not just s/keys but every spec impl relies on that#2019-10-1617:25rickmoynihanI wouldnā€™t mind having a layer of indirection between the name of the spec and the URI/key#2019-10-1617:25Alex Miller (Clojure team)we have no plans to make that#2019-10-1617:28rickmoynihanso would doing: (s/def :rdfs/label (URI. ""http://,,,,#label")) (s/def ::resource (rdf/keys :req [:rdfs/label])) not be feasible?#2019-10-1617:29Alex Miller (Clojure team)no#2019-10-1617:30Alex Miller (Clojure team)it is possible that we might allow arbitrary metadata for specs. if we did that (and I'm not committing to it yet), then you could hang your mapping to a uri on that.#2019-10-1617:35Alex Miller (Clojure team)in the meantime you could create such a mapping outside spec if desired#2019-10-1707:17vlaaad@alexmiller can you share rationale behind decision to limit spec names to keywords? (and symbols, right? for fdefs...)#2019-10-1713:53jaihindhreddyI believe, it's more like: spec allows us to spec data and functions. We spec functions by their name and pieces of data by their name (keywords), symbols are already used to name vars in the context of the language, which is why symbols can't be used. All other stuff (strings, characters, aggregates, etc) is not namespace qualified and hence those cannot be "precise" names. That causes collisions and is a recipe for brittleness IMHO.#2019-10-1708:04rickmoynihan@vlaaad: Though Iā€™m asking whether itā€™s possible or suggestions on how to do it; I totally understand the reason to name specs with keywords. I suspect the main reason is that they support namespaces; and can therefore be unambiguous and named in a global context, but donā€™t require a dependency on being loaded like vars. Allowing the use of a key to be decoupled from its spec/definition, whilst also allowing the same key to have multiple specs. Like with RDF it means itā€™s open for extension, anyone can spec anything etc. The reason Iā€™d like to do it, is because when I convert RDF into a clojure structure, itā€™d be nice not to have too coerce URIā€™s (which are used to identify properties & classes in the same way spec uses keywords) into keywords. However itā€™s probably not so bad, maintaining that mapping. I was just hoping itā€™d be possible to write my own version of s/keys that used URIā€™s directly.#2019-10-1714:45mpenetit's super easy to make your own in the meantime. Just use atom as metadata registry + write few fns (alter-meta, doc, meta, etc). then it's possible to chain (-> (s/def ::foo string?) (stg/with-doc "bla bla bla")) since specs returns their key.#2019-10-1718:48eskemojoe007I'm trying to create a spec around a time characterstic in a cljc file and getting confused. I generally try to use tick to manage time within cljs and clj. But their api doesn't have any tests for type of time that I'd like to put into the spec. What do others do for cljc spec that involve times?#2019-10-1803:59jjttjjHave you seen https://github.com/henryw374/time-specs#2019-10-1720:23dominicmDoes inst? work?#2019-10-1817:54agwhatā€™s the idiomatic way to assert results of stest/check?#2019-10-1818:07taylornot sure how idiomatic this is but
(deftest foo-test
  (is (= 1 (-> (st/check `foo)
               (st/summarize-results)
               :check-passed))))
#2019-10-1818:09agokay, but letā€™s say it returns nil and I want to print that out with expound, I guess Iā€™m gonna have to run st/check one more time?#2019-10-1818:10taylornot sure about the expound part#2019-10-1818:12ag
(expound/explain-results (stest/check `foo))
would print out the reason in human readable form, right? But it doesnā€™t return anything, so no way to assert it. and I canā€™t do something like:
(let [chk (stest/check `foo] ,,,
to re-use it for both
#2019-10-1818:13agfor assertion and for human-readable output#2019-10-1818:13taylorIā€™ve never tried to use expound with test.check output#2019-10-1818:13taylorwhich part doesnā€™t return anything?#2019-10-1818:14taylorexplain-results? you could use
(doto (stest/check `foo) expound/explain-results)
if so
#2019-10-1818:19agah yeah, nifty doto. Cool, thanks!#2019-10-1818:41agah, the problem with doto approach though - stest/check throws on failureā€¦ damn#2019-10-1818:52taylorhmm really? maybe Iā€™m thinking of something else but I thought it returned a map with useful failure info#2019-10-1818:52tayloror maybe thatā€™s in the exception#2019-10-1818:06agcan someone show me an example of deftest where combination of stest/check used with expound#2019-10-1818:13bbrinckhttps://gist.github.com/kennyjwilli/8bf30478b8a2762d2d09baabc17e2f10#gistcomment-2682609#2019-10-1818:14bbrinck@ag -^#2019-10-1818:16agOhā€¦ wowā€¦ this is pretty cool.#2019-10-1818:16bbrinckAlso apparently Kaocha can automate some of this https://cljdoc.org/d/lambdaisland/kaocha/0.0-554/doc/automatic-spec-test-check-generation#2019-10-1818:16bbrinckHavenā€™t tried it yet, but itā€™s on my list. #2019-10-1818:21agReally awesome ideas, thank you guys!#2019-10-1818:36kszabowe use Kaochaā€™s feature, it works very well#2019-10-2012:48abdullahibraHell everyone,#2019-10-2013:08abdullahibracan i pass the spec to function? (defn validate [data spec]) ?, or this isn't right, because the spec is globally defined so i can use it inside the function directly ?#2019-10-2015:07rascioyes you can, it is what the default validate does. specs are just values if you want to do a function that can work with any kind of spec you should do it.#2019-10-2418:48agdo we have any libraries that can generate specs based on DB schema, e.g.: postgress DDLs?#2019-10-2420:27aghas anyone dealt with type of a problem where you need a spec for a map with underscored keys (for data coming from DB) and absolutely same spec with the only difference being that keys are hyphenated? Maybe someone wrote a macro?#2019-10-2420:37ikitommiThere is no way to close a Spec2 Select? e.g. dontā€™ allow any extra keys, right?#2019-10-2420:43ikitommiSo, if i have s Schema with ::id, ::name, ::description and ::secret keys and I would like to make an api where you must send ::id & ::name, optionally ::description, but not ::secret, I would first have to make an (api) Schema with just ::id, ::name and ::description close that (so the ::secret becomes illegal) and make a select for that schema with just ::id and ::name defined so the ::description becomes optional. Right?#2019-10-2420:55seancorfieldSpec 2 uses an option hash map to specify which specs are closed -- at checking time (`s/valid?` s/conform etc)#2019-10-2420:56seancorfieldselect is always open. It just says what is required.#2019-10-2421:02ikitommiThanks, thatā€™s my understanding too. So, to support a use case that one canā€™t send the ::secret, I need to create a concrete new Schema to be able to mark is as closed.#2019-10-2421:03seancorfieldThe checking code can determine which specs are treated as closed -- independent of schema / select.#2019-10-2421:04seancorfieldhttps://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha#closed-spec-checking#2019-10-2421:08ikitommi
(s/def ::id int?)
(s/def ::name string?)
(s/def ::description string?)
(s/def ::secret boolean?)

;; the internal schema
(s/def ::user (s/schema [::id ::name ::description ::secret]))

;; the api schema
(s/def ::user2 (s/schema [::id ::name ::description]))

;; the api select (id & name mandatory, description optional)
(s/def ::user3 (s/select ::user2 [::id ::name]))

;; closed validation
(s/valid? ::user3 {::id 1, ::name "kikka"} {:closed #{::user2}})
#2019-10-2421:28vlaaadi wonder why closed specs are implemented as options instead of as another type of spec#2019-10-2421:38seancorfield@vlaaad Alex talks about that on the Inside Clojure blog as Spec 2 went down the "closed spec" path first and then changed to "closed checking".#2019-10-2421:41vlaaadyou mean this http://insideclojure.org/2019/04/19/journal/ ?#2019-10-2421:42vlaaad(s/close-specs ::s) this is some mutability thing, I was talking about having closed specs as different specs, no mutability attached#2019-10-2421:42vlaaadlike that:
(s/def ::open-keys (s/keys :req-un [::some-key]))

(s/def ::closed-keys (s/closed ::open-keys #{::open-keys}))
#2019-10-2421:45vlaaadI think that makes sense: being able to refer to your data specification by name, instead of by combining a name and an arguments to s/valid?#2019-10-2421:47seancorfieldI'll leave it up to @alexmiller to respond. I prefer the path they've taken where "closedness" is just an option on checking.#2019-10-2421:48vlaaadIf you have it as another spec it's still opt-in, but now it's also composable#2019-10-2421:50Alex Miller (Clojure team)Not going to do that#2019-10-2421:51Alex Miller (Clojure team)Itā€™s not composable - negation is inherently problematic as specs evolve#2019-10-2507:20vlaaadI don't get it, can you explain? I thought about closed specs for a bit, and the way I see them, they can be just a s/select counterpart, where s/select describes lower bound of data shape ("be at least that"), while s/closed describes upper bound ("be at most that"). By composable I mean using same specs both as open and as closed in same s/valid? etc. checks. I think it might make sense, for example, if I were to spec a function that cleans user input before putting it into db, it's spec would look like that:
(s/fspec
  :args (s/cat :user ::user)
  :ret (s/closed ::user))
Speaking of specs evolution, do you mean being able to tell if change of spec from a to b is accretive or not? I don't see problems here as well: - changing (s/closed (s/keys [::name])) to (s/closed (s/keys [::name ::age])) in return position provides more ā€” it returned ::name, now it also can return ::age; - changing (s/closed (s/keys [::name])) to (s/closed (s/keys [::name ::age])) in argument position requires less ā€” it didn't accept ::age, now it accepts it; - changing (s/closed (s/keys [::name])) to (s/keys [::name]) in return position provides more ā€” it used to return only ::name, now it may return more.
#2019-10-2512:39Alex Miller (Clojure team)those words do not match what you're saying in the specs. the closed version says "MUST NOT contain ..." 1/3. is probably ok from the consumers point of view but is contextual to use 2. if you provide age on the first one, then forbid on the second one, you've broken users that previously passed age#2019-10-2512:42Alex Miller (Clojure team)this notion of context is critical and is driving the changes in schema/select/fdef, so I wouldn't rule out some way to make a more restrictive statement at the point of use, although we currently think of that again as more of a "check", not part of a spec#2019-10-2513:04vlaaad> those words do not match what you're saying in the specs what words?#2019-10-2513:07vlaaadmaybe we have different definitions of closed? My (s/closed [::name ::address]) says that map may be empty, or contain ::name, or contain ::address, or contain both, but no extra keys are allowed#2019-10-2513:11vlaaadlike select, that requires at least this and allows more, but in different direction: requires at most this and allows less. in different contexts select, closed and a combination of both might be useful#2019-10-2513:12vlaaadso in "2" you could not provide ::age before: this is invalid, and version after allows it#2019-10-2816:56vlaaad@alexmiller bump#2019-10-2421:57ikitommiWhat if they were part of the select?#2019-10-2422:32Alex Miller (Clojure team)well, again the principle here is that "closedness" is part of the checking, never part of the spec#2019-10-2504:43ikitommiI agree on that, but have thought that select is a utility of making checking context (values!) from schema. It would allow one place & syntax to easily close any of the nested maps too. The current syntax of closing specs by name in s/valid? with different (nested maps) syntax adds more things for the developer to keep in his/her head. Need to jump to both schema and select definitions to see what subkeys are including in the select and need to be closed. Also, if schemas & selects get refactored, the closed options might get out-of-sync.#2019-10-2421:58ikitommi
(s/select ::user [::id ::name s/closed])
#2019-10-2422:01ikitommiAlso:
(s/select ::user [::id ::name [s/optional ::description] s/closed])
#2019-10-2422:05ikitommiIn the code I pasted above, keysets need to be introduced in three places: the root schema, the api schema and the select. Also, partially in the s/valid? call to make it closed.#2019-10-2422:15seancorfieldI was a bit surprised by this:
user=> (require '[clojure.spec-alpha2 :as s])
nil
user=> (s/def ::thing (s/schema [::a ::b ::c]))
:user/thing
user=> (s/valid? (s/select ::thing [::a ::b]) {::a 1 ::b 2 ::c 3} {:closed #{::thing}})
true ; fine -- ::c is optional but present
user=> (s/valid? (s/select ::thing [::a ::b]) {::a 1 ::b 2} {:closed #{::thing}})
true ; fine -- ::c is optional and omitted
user=> (s/valid? (s/select ::thing [::a ::b]) {::a 1 ::b 2 ::d 4} {:closed #{::thing}})
true ; huh? -- ::d is not in ::thing -- why doesn't this fail?
user=> (s/valid? (s/select ::thing [::a ::b]) {::a 1 ::d 4} {:closed #{::thing}})
false ; because ::b is required but omitted... see next...
user=> (s/explain-str (s/select ::thing [::a ::b]) {::a 1 ::d 4} {:closed #{::thing}})
"#:user{:a 1, :d 4} - failed: (fn [m] (contains? m :user/b))\n"
user=> 
@alexmiller is that expected or a bug?
#2019-10-2422:17seancorfield
user=> (s/def ::sub-thing (s/select ::thing [::a ::b]))
:user/sub-thing
user=> (s/valid? ::sub-thing {::a 1 ::b 2 ::d 4} {:closed #{::sub-thing}})
true
user=> (s/valid? ::sub-thing {::a 1 ::b 2 ::d 4} {:closed #{::thing}})
true
(still puzzled)
#2019-10-2422:29Alex Miller (Clojure team)there's some lingering tech debt in the schema/select code where some things are duplicated rather than having select lean on schema's validation code. I think once that's cleaned up, these will fail as you expect#2019-10-2723:01kennyIs there a way to write s/ands without having the conformed value passed to the predicate? All of our s/and predicates call s/unform on the value before calling the actual predicate function. This seems unnecessary. For example:
(s/def ::base-metric
  (s/and
    ::metric/metric
    #(metric-bounds-lower-less-than-or-equal-to-upper?
       (s/unform ::base-metric %))
    #(metric-default-value-within-bounds?
       (s/unform ::base-metric %))))
#2019-10-2800:15Alex Miller (Clojure team)currently, no. we will probably have both flowing and unflowing forms in spec 2#2019-10-2807:53mishayou might get away with moving spec to the last position in the s/and, depending on resilience of your predicates to random input. @kenny or, if this is the only place you use predicates ā€“ make them expect conformed value#2019-10-2816:05kennyHmm. Can't do the latter because the predicates are used on the regular form of the data too.#2019-10-2816:19Alex Miller (Clojure team)you could wrap s/nonconforming around each pred#2019-10-2816:20Alex Miller (Clojure team)
(s/def ::base-metric
  (s/and
    (s/nonconforming ::metric/metric)
    (s/nonconforming metric-bounds-lower-less-than-or-equal-to-upper?)
    (s/nonconforming metric-default-value-within-bounds?)))
#2019-10-2816:20Alex Miller (Clojure team)something like that#2019-10-2816:24kennyAh, that's a lot nicer.#2019-10-2816:26kennyThat doesn't appear to work for all cases.#2019-10-2816:27kennyOh, I missed the first one.#2019-10-2816:31kennyThis is going to make things so much cleaner! Thanks @alexmiller šŸ˜ƒ
(defmacro and2
  [& pred-forms]
  (let [pred-forms (map (fn [pred-form]
                          `(s/nonconforming ~pred-form)) pred-forms)]
    `(s/and 
#2019-10-2817:15ataggartPer the description of every, the :kind pred "must generate in order for every to generate". I see that preds like set? can generate, e.g., (gen/sample (s/gen (s/spec set?))), but I'm at a loss as to how to provide generators for custom preds.#2019-10-2822:51Alex Miller (Clojure team)there is no open mechanism to bind generators to custom preds currently#2019-10-2822:53Alex Miller (Clojure team)you can provide the :kind pred as a separate spec outside s/every too (s/and (s/with-gen my-pred ...) (s/every ...))#2019-10-2822:54Alex Miller (Clojure team)that said, what kind of pred are you using that's not a core pred?#2019-10-2915:32ataggartThe specific case of wanting to use every over a priority queue.#2019-10-2915:51Alex Miller (Clojure team)prob easier to attach a generator to the overall spec#2019-10-2916:30ataggartYup#2019-10-2919:59ataggartI assume the same goes for predicates like uuid?#2019-10-2920:00ataggartIn that we can't provide generators directly for custom predicates#2019-10-2920:00Alex Miller (Clojure team)I think uuid? has a generator? or maybe that's just one we've talked about#2019-10-2920:01Alex Miller (Clojure team)yeah, uuid? gens#2019-10-2920:01Alex Miller (Clojure team)you can generally wrap specs with custom generators around preds with s/with-gen or s/spec#2019-10-2920:01Alex Miller (Clojure team)the particular case of :kind in s/every is a place where you don't have that opportunity#2019-10-2920:46ataggartI've been fighting to successfully use with-gen for about an hour now#2019-10-2920:47ataggartAnd I just figured it out.#2019-10-2920:48ataggartspec.gen wants fns for everything, but I'm used to test.check which doesn't.#2019-10-2920:48ataggartI'm good now, thanks for your help#2019-10-2920:51Alex Miller (Clojure team)yep, it's thunked to make the load step dynamic and optional#2019-10-2920:54ataggartYeah, makes sense#2019-10-2920:55ataggartIt's a one-time problem#2019-10-2920:55ataggartNow I know#2019-10-2907:46jumarLooking at spectrum, I can't figure out how to use it: https://github.com/arohner/spectrum I know it's (still) not recommended for any kind of use, but I was curious about what it can do. However, I don't even know what kind of output can I expect from it. Trying the most basic example doesn't produce anything useful:
(defn foo [i]
  (inc i))

(f/infer-var #'foo)
;;=> couldn't find analysis for #'clojure-experiments.spectrum/foo

;; throws AssertionError
#_(f/infer-form '(foo 3))
Using spectrum 0.2.5; code is here: https://github.com/jumarko/clojure-experiments/blob/master/src/clojure_experiments/spectrum.clj#L11
#2019-10-2907:58jumarIs anybody using this tool and if so how?#2019-10-2909:11danielnealas far as I know itā€™s still all in development and not usable yet#2019-10-2921:36arohnerItā€™s not ready yet. master has much newer code, but is also not ready.#2019-10-3005:13jumarHmm. looking forward to hearing more in the future then#2019-10-2921:34ataggartI'm digging into it now, but does anyone have a guess as to what might be causing this?
user=> (require '[clojure.tools.logging :as log])
Syntax error macroexpanding clojure.core/defn at (clojure/tools/logging/impl.clj:241:1).
THIS - failed: vector? at: [:fn-tail :arity-1 :params] spec: :clojure.core.specs.alpha/param-list
THIS - failed: (or (nil? %) (sequential? %)) at: [:fn-tail :arity-n :bodies] spec: :clojure.core.specs.alpha/params+body
The line of code referenced above: https://github.com/clojure/tools.logging/blob/tools.logging-0.5.0/src/main/clojure/clojure/tools/logging/impl.clj#L241
#2019-10-2921:34ataggartI'm not yet sufficiently fluent in spec-error-ese.#2019-10-2921:48andy.fingerhut@ataggart Most likely an older version of clojure.tools.logging from before Clojure 1.9 was released, that contains some old syntax that Clojure 1.9's and later's stricter checking rejects, and someone has since fixed with a later tools.logging release.#2019-10-2921:49ataggartI'm using 0.5.0, which is the latest version. And that "someone" would be me. šŸ˜›#2019-10-2921:49ataggartdeps hell once again#2019-10-2921:49andy.fingerhutSorry, I was going for an answer appropriate for 99% of similar questions. Looking at the line of code now...#2019-10-2921:50ataggartlol np#2019-10-2921:50seancorfield
(! 775)-> clojure -Sdeps '{:deps {org.clojure/tools.logging {:mvn/version "0.5.0"}}}'
Clojure 1.10.1
user=> (require '[clojure.tools.logging :as log])
nil
user=> 
Something in your environment @ataggart?
#2019-10-2921:51seancorfield(and I did wonder if a version conflict was bringing in an older version)#2019-10-2921:52seancorfieldFWIW, we use tools.logging 0.5.0 at work with Clojure 1.10.1 -- which includes all the Spec checking on defn -- and we don't see any problems.#2019-10-2921:52seancorfieldWhat other dependencies have you got in play?#2019-10-2921:53ataggartYep, I'm confident it's a deps issue. I'll keep digging. Aside, any guess on what that error means?#2019-10-2921:54seancorfieldIt's a badly formed defn. Expecting an argument vector or sequence of pairs of arg vectors / bodies.#2019-10-2921:54andy.fingerhutIt mentions param-list, so my first guess would be it thinks the defn parameter list is not a vector#2019-10-2921:54ataggartweird#2019-10-2921:56seancorfieldAre you using lein/boot/clj? What does your deps tree show? Could it be a badly-behaved plugin patching stuff?#2019-10-2921:56ataggartI'll find out and report back.#2019-10-2922:03ataggartSo, I changed the version to 0.4.1, same error. Changed it back to 0.5.0, and the error is gone....#2019-10-2922:04andy.fingerhutPlease keep your gremlins away from my computer.#2019-10-2922:05ataggartThe vision of a world of spec'd libraries without versions can't come soon enough.#2019-10-2922:06ataggartBack to work....#2019-10-2923:05Alex Miller (Clojure team)maybe a cached classpath? changing the deps file would invalidate the cache#2019-10-2923:05Alex Miller (Clojure team)so any change would have forced an update#2019-10-2923:06Alex Miller (Clojure team)if you're ever not sure with clojure, adding an -Sforce can force that regardless#2019-10-3013:26ataggartIs there a canonical way to generate only a subset of the ā€œtypesā€ from a multi-spec?#2019-10-3013:33sgerguriYou can force a multimethod clause to be selected with the following:
(s/gen (<multispec> <dispatch-arg>)))
#2019-10-3013:34sgerguriTo pick a subset of the multispecs you can make a collection of the valid dispatch args, then randomly sample that.#2019-10-3013:35ataggartThanks!#2019-10-3014:22ataggartHmm... doing that is yielding maps that have the correct shape for the type, but the value of the dispatching type key is random, not the dispatch value.#2019-10-3014:24ataggartI'm guessing it's because the result of (s/gen ...) isn't flowing through the multi-spec's retag.#2019-10-3014:25sgerguriThat all depends on how tight your multispec clauses are. You can override what you get with fmap like this:
(clojure.test.check.generators/fmap #(assoc % ...) (s/gen (<multispec> <dispatch-arg>)))
#2019-10-3014:26ataggart>That all depends on how tight your multispec clauses are. What do you mean?#2019-10-3014:28sgerguriFor example, I have a multispec where I dispatch on two keys in the map I generate. The keys are unqualified, meaning I can strictly enforce that for a particular clause, I always get a particular value. Like this:
(s/def ::type #{:foo})

(defmethod multispec-fn :foo
  [_] (s/keys :req-un [::type ...]))
#2019-10-3014:29sgerguriSo even if the s/keys is part of a multispec for a dispatch value that involves :foo, because the dispatch is done based on the value of a function applied to the data this way I make sure that when the above clause is selected, :foo will be the value of the key :type.#2019-10-3014:29sgerguriIn general though, if you do the fmap above it should be fine.#2019-10-3014:30sgerguriIrrespective of whether you do something similar to what I just described or not.#2019-10-3014:33ataggartUsing the code here: https://clojure.org/guides/spec#_multi_spec
user=> (map :event/type (gen/sample (s/gen :event/event)))
(:event/error :event/search :event/search :event/search :event/error :event/search :event/error :event/search :event/search :event/error)

user=> (map :event/type (gen/sample (s/gen (event-type {:event/type :event/search}))))
(:+/- :*/. :*/gq :IZ/K :./.35 :YT/* :d/B :*Pk/q2L6 :Lf/J! :+/M-)
#2019-10-3015:27ataggart@U064X3EF3 Continuing the multi-spec example from the official guide, does this seem reasonable for the simple case of a kw retag?#2019-10-3014:33ataggartThe intention of the second one is to only yield :event/search.#2019-10-3014:35ataggartThat's why I think just calling the multi-method isn't enough. Generating via the multi-spec would flow through retag, but the code above doesn't.#2019-10-3014:37ataggartThe master approaches...#2019-10-3014:37Alex Miller (Clojure team)I think going back to your original question, the answer is no, there is no way to (easily) generate a subset of the multispec values right now#2019-10-3014:38sgerguriIf you fmap, however, you should still get the right thing.#2019-10-3014:38ataggartMan, I was all braced to receive your wisdom, Alex.#2019-10-3014:38Alex Miller (Clojure team)there are a variety of hacks :)#2019-10-3014:38ataggartYeah, I can manually retag#2019-10-3014:39Alex Miller (Clojure team)you can choose a particular one and retag, or you can dive into the guts of the multimethod dispatch value map (which is what the multispec generator does)#2019-10-3014:39sgerguriAlso :event/type is defined as a keyword in the example, whereas in mine above I restrict that particular spec to a single value.#2019-10-3014:39Alex Miller (Clojure team)or you could s/and and only take events of a particular type#2019-10-3014:39sgerguriIt just so happens that I have the same unqualified key in all the multispecs defined difrerently per multispec.#2019-10-3014:40ataggartEvery time I try to use s/and for this issue I end up with such-that errors#2019-10-3014:40sgerguris/with-gen is your friend there.#2019-10-3014:40sgerguriDon't be afraid to use it - I do in strategic places, typically around complex s/and cases.#2019-10-3014:40ataggartproblem is I can't tell what's causing the such-that error#2019-10-3014:41ataggartWell, I can eventually, but it's not obvious when it happens#2019-10-3014:41ataggartok, I'm going to beat my head on this a little more#2019-10-3014:41sgerguriWell if it's an s/and then simply provide your own generator by wrapping that in s/with-gen.#2019-10-3014:41ataggartThe issue tends to be using s/and when I should use s/merge#2019-10-3014:42sgerguriPossibly. Though if you rely on custom conformers, s/and is a very frequent thing to end up using.#2019-10-3014:43ataggartyep, just wasn't helping in this case#2019-10-3014:44ataggartretagging via fmap over the generated values from the multimethod is working though.#2019-10-3014:46ataggartFound the such-that:
user=> (s/gen :event/event)
#clojure.test.check.generators.Generator{:gen #function[clojure.test.check.generators/such-that/fn--6656]}
#2019-10-3014:46ataggartšŸ™‚#2019-10-3015:06orestisI know it's a vague question, but what's the upgrade path from spec -> spec2 look like? I know there's changes coming in s/keys and I guess the functions that validate/conform/explain might change, but I was wondering how much of the spec definitions will change?#2019-10-3015:08orestisI'm trying to figure out if we should start with spec2 directly, and deal with the breakage as it comes (I guess it's pre-alpha), or start with spec and deal with the upgrade breakage when it comes.#2019-10-3017:05seancorfieldWe've been trying to keep a branch of our codebase at work up to date on Spec 2 but it's required quite a few changes to some of our Spec 1 code and at this point, we've essentially "given up" on a straightforward migration. We plan to adopt Spec 2 for new code once it is out of pre-alpha status and run a mixed Spec 1 / Spec 2 codebase for a good long while.#2019-10-3015:16Alex Miller (Clojure team)I would use spec right now#2019-10-3015:17Alex Miller (Clojure team)the api functions - validate/conform/explain etc have gained some capabilities, but I expect will continue to work in spec 2 as is#2019-10-3015:19mpenetany news on eta? we're also waiting on v2 release for some new features (looking in particular at new data spec format for "interpretation" of specs)#2019-10-3015:19Alex Miller (Clojure team)the spec op forms will mostly be the same, although keys -> schema/select will be a significant change and there are some differences around the use of sets, preds, functions, etc at thee top level#2019-10-3015:20Alex Miller (Clojure team)I don't have an eta but I'd guess it's on the order of months#2019-10-3015:32Alex Miller (Clojure team)I guess?#2019-10-3015:34ataggartheh, ok, thanks#2019-10-3016:59orestisThanks for your reply Alex, much appreciated. #2019-10-3017:07seancorfield@orestis As Alex says, use Spec 1 for the time being. Conversion to Spec 2 may be easy or hard depending on how you use Spec 1.#2019-10-3017:17orestisThe thing is I need to do closed map checking, recursively. I think that it can be hacked on top of spec1, but not sure if itā€™s already done and if not, if worth the effort to do myself. #2019-10-3017:22seancorfieldIs the project a production-level one, or likely to be one within the next couple of months?#2019-10-3017:22seancorfieldIf not, I'd say give Spec 2 a go. Otherwise, stick to Spec 1.#2019-10-3017:23Alex Miller (Clojure team)I wouldn't, I'd say stick to spec 1 :)#2019-10-3017:25Alex Miller (Clojure team)there is no release of spec 2 available, because I do not think it is release-worthy yet. until then, I would not recommend for anything other than tire-kicking and feedback (which is very welcome)#2019-10-3017:28seancorfieldRight, that's what I meant about not production-level -- if this is just an exercise project, go kick the tires. Otherwise, stick to Spec 1.#2019-10-3017:28seancorfieldI really like where Spec 2 is going but it's a different beast in many ways from Spec 1...#2019-10-3018:26orestisProduction-level. Ok, Iā€™m hearing you loud and clear :)#2019-10-3020:59mloughlinThe book Secure By Design from Manning writes about using Java classes with immutable properties and data contracts in the constructor, then only using this defined class (instead of using, say, a string for a bank account number) in order to prevent business logic bugs. The nice thing about this is the type system enforces a degree of defensive programming. My question: is there an clojure.spec analogue, or convention for this?#2019-10-3021:29Alex Miller (Clojure team)mostly I think that's trying to patch problems that are endemic to use of Java, rather than an actually good idea. however, I'd say the closest thing is to separate specs into two levels - domain definitions (::domain/bank-acct-no) and uses in specific contexts that alias those domain definitions#2019-10-3021:30Alex Miller (Clojure team)but this is really mostly just making complicated definitions once rather than repeating them#2019-10-3022:40mloughlinwould you validate as a pre-condition every time you were to use ::domain/bank-acct-no? I suppose it's super cheap computationally#2019-10-3022:41Alex Miller (Clojure team)No, I would not#2019-10-3022:41Alex Miller (Clojure team)I would use instrument to check during dev/test #2019-10-3022:43mloughlinthanks for clarifying#2019-10-3022:43mloughlinšŸ™‚#2019-11-0109:51Nazralhi#2019-11-0109:52NazralI'm trying to write a macro that among other things needs to generate new specs by wrapping or merging existing ones, and it doesn't work. I made the simple following example
(defmacro wrapspec
  [base-name base-spec]
  (let [n-name `(keyword (str ":wrapped-" '~base-name))
        new-spec (s/coll-of base-spec)]
    `(s/def ~n-name ~new-spec)))
I'm not sure why doesn't this work? I tried with a symbol instead of a keyword too, to no avail
#2019-11-0110:12NazralI also tried with
n-name `(symbol (str (.name *ns*)) '~base-name)
to no avail
#2019-11-0112:51Alex Miller (Clojure team)did you look at the expansion?#2019-11-0112:52Alex Miller (Clojure team)just as a general technique it's important to (pprint (macroexpand-1 '(wrapspec 'abc int?))) or whatever to see if it's doing what you want#2019-11-0112:53Alex Miller (Clojure team)here s/coll-of is going to be a spec object, which is not what you want#2019-11-0112:53Alex Miller (Clojure team)
new-spec `(s/coll-of ~base-spec)
#2019-11-0112:54Alex Miller (Clojure team)probably closer but I'm just eyeballing this#2019-11-0121:25FloAnyone aware of issues with clojure.spec.alpha/gen in cljs (using shadow-cljs)? This snippet works well in clj repl:
(s/def ::foo string?)
(s/gen ::foo)
but fails in cljs with: Var clojure.test.check.generators/simple-type-printable does not exist, clojure.test.check.generators never required at eval (/js/compiled/cljs-runtime/cljs.spec.gen.alpha.js:1854:8)
#2019-11-0121:26FloMy project.clj has a dependency entry for [org.clojure/test.check "0.10.0"]#2019-11-0121:28FloNevermind, after I explicitly required [clojure.test.check.generators] in the same file I'm (s/gen ..)ing, it works.#2019-11-0208:35kurt-o-sysI'm having a pretty extensive spec, with quite some s/or parts. When any data structure doesn't conform, it shows the issues, as expected. Conforming structures show that they conform... so all is well so far. However, what I'd like to have, is to know which 'path' has been taken to conform:
(s/def ::spec (s/and string?
                     (s/or :path-A ::path-A
                           :path-B ::path-B)))
Is there a way to know, if a structure conforms, to know if it took path-A or path-B?
#2019-11-0209:07vlaaadIsn't s/conform showing it?#2019-11-0209:11kurt-o-sysoh yeah, it is... thx.#2019-11-0209:11kurt-o-systhat was easy šŸ™‚#2019-11-0218:49coder46334Hi everyone! Let's say I have a map like this
(def example-map {0  {:children [1 22 55] :parent nil}
                               22 {:children [14] :parent 0}
                               14 {:children [] :parent 22}})
How can I ensure with spec that a) a :parent is never the same key as the current line or ideally b) that :parent is a valid key in the map? I read through the website and some tutorials and I don't know if that's even possible with spec. Any advise would be super helpful
#2019-11-0220:02andy.fingerhutYou can write arbitrary functions that return true or false for a given value, and use those in your specs, so if you can write a normal Clojure function to do it, you can write a spec to do it. Some of those reduce the amount of help you get from random generation of values, etc.#2019-11-0220:04coder46334This sounds useful, can you hint me what this is called in the docs so I can look it up? Also, I thought it would be cool to have the test cases auto-generated, so that won't work?#2019-11-0220:05andy.fingerhutThis property of functions is mentioned very early in the spec guide article here: https://clojure.org/guides/spec under the heading "Predicates"#2019-11-0220:07andy.fingerhutI am not a knowledgeable enough user of spec to give you the best answer to how to auto-generate things with custom predicates, but there are places in the docs that describe how to create custom random value generators and use those instead of the default built-in ones.#2019-11-0220:10coder46334I guess where I'm stuck at is how I bring that into the s/def of a map-of. I can't put it onto the smaller pieces, because they'd have to reference each other#2019-11-0220:11coder46334Ok, I thought of looking into the generators, but was hoping there is an easier solution to define it on the specs. There are not really a lot of examples out there that are more complex than one-liners. Thanks for your help!#2019-11-0220:12coder46334And if anyone has an idea about how to link it with the s/def of the map-of I'm still curious. Will be back at the computer later and will try to find out#2019-11-0220:14andy.fingerhutI believe s/and can combine arbitrary specs into one?#2019-11-0220:17coder46334I'll try later - thanks!#2019-11-0221:57coder46334Eventually I found some good examples here: https://clojuredocs.org/clojure.spec.alpha/map-of.#2019-11-0318:03alex-dixonAnyone interested in spec as a parser combinator library?#2019-11-0410:06acronIs there a preference to using either s/* or s/coll-of when we just want 'a collection of x`? I realise coll-of has more tuning, but besides that, is there difference between (s/* ::foo) and (s/coll-of ::foo) ?#2019-11-0410:15jumarProbably the most visible difference will be when you're nesting them - the regex based specs will make it work like it was "flattened" => I'd normally use coll-of; see also https://stackoverflow.com/questions/58636813/i-cant-understand-the-following-clojure-spec-error/58637063#2019-11-0410:17acronThanks, I think for sanity sake then I will ask for coll-of in these situations#2019-11-0412:57Alex Miller (Clojure team)Yes, should use coll-of by default#2019-11-0420:43manutter51Hmm, I thought this would work:
(s/fdef page-header-ribbon
        :args (s/cat :title string? :extra (s/* any?))
        :ret vector?)

(page-header-ribbon "Manage Users") 
but I get
Call to #'page-header-ribbon did not conform to spec:ā†µ
val: "Manage Users" fails at: [:args] predicate: (cat :title string? :extra (* any?))...
#2019-11-0420:44manutter51What am I doing wrong?#2019-11-0420:46Alex Miller (Clojure team)is there more to that ... ?#2019-11-0420:46manutter51Just the rest of the stack dump stuff, the JS objects etc#2019-11-0420:47manutter51
Call to #'page-header-ribbon did not conform to spec:ā†µ
val: "Manage Users" fails at: [:args] predicate: (cat :title string? :extra (* any?))ā†µ
:cljs.spec.alpha/spec  #object[cljs.spec.alpha.t_cljs$spec$alpha44661]ā†µ
:cljs.spec.alpha/value  "Manage Users"ā†µ
:cljs.spec.alpha/args  "Manage Users"ā†µ
:cljs.spec.alpha/failure  :instrument
#2019-11-0420:48Alex Miller (Clojure team)oh, cljs#2019-11-0420:49Alex Miller (Clojure team)at a glance it doesn't look wrong to me#2019-11-0420:50manutter51Ok, good, Iā€™m not nuts then.#2019-11-0420:50manutter51Probably some kind of dependency conflict or something.#2019-11-0420:56Alex Miller (Clojure team)I couldn't repro in either clj or cljs. maybe old repl statee?
#2019-11-0420:58manutter51Itā€™s surviving multiple shutdown/clean/restart cycles. :shrug:#2019-11-0420:59Alex Miller (Clojure team)#2019-11-0421:00Alex Miller (Clojure team)in case that's different than what you're doing...#2019-11-0421:02manutter51Not [cljs.spec.alpha :as s]?#2019-11-0421:15Alex Miller (Clojure team)either should work#2019-11-0421:15Alex Miller (Clojure team)the clojure one aliases to the cljs one#2019-11-0506:54murtaza52I am trying to specify that atleast one key should be present, ex -
(def ::city string?)
(der ::name string?)

(def my-input (s/keys :opt-un [::city ::state])
The above makes both of them optional, I want to specify that atleast one should be present in the input.
#2019-11-0510:00misha
(s/def ::a string?)
(s/def ::b string?)
(s/def ::m (s/keys :req-un [(or ::a ::b)]))

(s/explain ::m {:a "1" :b "2"})
;; Success!

(s/explain ::m {:a "1" :b 2})
;; 2 - failed: string? in: [:a] at: [:b] spec: :user/b

(s/explain ::m {:a "1"})
;; Success!

(s/explain ::m {})
;; {} - failed: (or (contains? % :a) (contains? % :b)) spec: :user/m
#2019-11-0704:57murtaza52thanks @U051HUZLD#2019-11-0513:41jeremysHey guys I ran into something Iā€™d like to ask you about. I wanted to ā€œmergeā€ to s/keys specs using s/and. It didnā€™t work as I thought it would. Here is an example showing what I wanted to do:
(ns example
  (:require [clojure.spec.alpha :as s]))

(s/def ::type (s/or :s string? :i int?))

(s/def ::a ::type)
(s/def ::b ::type)

(s/def ::t1 (s/keys :req [::a]))
(s/def ::t2 (s/keys :req [::b]))

(s/def ::v1 (s/and ::t1 ::t2))
(s/def ::v2 (s/and #(s/valid? ::t1 %)
                   #(s/valid? ::t2 %)))

(def example {::a "1" ::b 2})
(s/valid? ::v1 example)
;=> false
(s/valid? ::v2 example)
;=> true
(s/explain ::v1 example)
;[:s "1"] - failed: string? in: [:example/a] at: [:example/a :s] spec: :example/type
;[:s "1"] - failed: int? in: [:example/a] at: [:example/a :i] spec: :example/type
;[:i 2] - failed: string? in: [:example/b] at: [:example/b :s] spec: :example/type
;[:i 2] - failed: int? in: [:example/b] at: [:example/b :i] spec: :example/type
;=> nil
Do I use s/and in an unintended way? Is ::v2 the correct way to do it? Did I miss something in specā€™s api?
#2019-11-0513:48sgerguri@jeremys Sounds to me like you want s/merge.#2019-11-0513:57jeremys@sgerguri as usual I missed something in the api... Thanks a lot mate#2019-11-0514:07jeremysItā€™s so logical also, as you merge maps, you merge specs about maps...#2019-11-0517:46colinkahnIs it a good idea to define your generators in the same file as your specs? Sometimes I have generators that need my specs and vise-versa. Couldnā€™t tell if it was good practice to keep them together though.#2019-11-0602:02sgerguriI (and my colleagues) tend to do that, then things are nicely collocated. Doubly useful when your spec is a complicated s/and - if you wrap with s/with-gen it's all neatly in one place.#2019-11-0621:21telekidIs there a functional difference between (s/spec some-pred :gen some-gen) and (s/with-gen some-pred (constantly some-gen))?#2019-11-0701:52Alex Miller (Clojure team)No#2019-11-0721:30pabloreis there an expected date for spec 1.0.0 ?#2019-11-0721:32Alex Miller (Clojure team)no#2019-11-0721:39vlaaadwill there be spec 1.0.0?#2019-11-0721:39vlaaadand tools.deps without alpha#2019-11-0721:42Alex Miller (Clojure team)yes#2019-11-0722:18seancorfieldspec 1.0.0 will be Spec 2 when it's "baked", right @alexmiller ?#2019-11-0722:36Alex Miller (Clojure team)most likely#2019-11-0722:37Alex Miller (Clojure team)I don't know that it will be 1.0.0 exactly but it will be some non-alpha version#2019-11-0800:45pablorewhat are the main differences between spec 2 and alpha?#2019-11-0800:48kszabomainly how they handle aggregates of attributes. You only specify required/optional attributes at the context of the usage site#2019-11-0800:48kszaboā€˜bagsā€™ of attributes are now the primary way of mapping domain entities in spec2, they donā€™t talk about requirements#2019-11-0800:49kszaboalso some programmatic convenience improvements#2019-11-0800:49kszaboof course I am oversimplifying#2019-11-0803:57Alex Miller (Clojure team)https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha#2019-11-0803:57Alex Miller (Clojure team)and https://github.com/clojure/spec-alpha2/wiki/Schema-and-select#2019-11-0816:23wilkerluciohello, with spec1, I'm having issues to define a spec that is not loaded, I get Unable to resolve spec: ..., consider this is file B, the spec in question is defined by A, but A requires B, so I can't load A before loading B (circular reference), with that constraint, is there a way to tell spec to ignore that the spec is not defined at that time?#2019-11-0816:26Alex Miller (Clojure team)most spec types delay resolving references so don't run into this, but there are a few that don't#2019-11-0816:26Alex Miller (Clojure team)for those that don't, you'll need to break that cycle somehow#2019-11-0816:43wilkerluciothanks, I got rid of the cycle by moving those spec definitions to a new namespace (since I we can define things from other namespaces, so still have the same name, just defined in a different file)#2019-11-0821:01didibusWhat's the best way to restrict a string to a min/max length, so that the generator still works for it ?#2019-11-0821:04Alex Miller (Clojure team)I don't think there is a general way to do that without writing a generator#2019-11-0821:06Alex Miller (Clojure team)(and to do that, I would generate strings of length up to (max-min) and a string of length min, then concat)#2019-11-0821:06didibusHum, alright, I'll use a custom generator then#2019-11-0821:06didibusthx#2019-11-0917:17cddrI made a thing to capture one side of that constraint as part of a toolkit for generating test-data to match a SQL schema....
(defn string-up-to [max-len]
  (s/with-gen string?
    #(gen/fmap (fn [x] (apply str x))
               (gen/bind (s/gen (s/int-in 0 max-len))
                         (fn [size]
                           (gen/vector (gen/char-alpha) size))))))
#2019-11-0920:18bortexzIs there any way to add a docstring to specs?#2019-11-0920:56Alex Miller (Clojure team)no, not currently#2019-11-0923:51bortexzAre there any plans to add it? I think it could be interesting to have docstring when using spec to model the domain of an application#2019-11-1000:28Alex Miller (Clojure team)yes, it is the highest voted ticket in the tracker :)#2019-11-1000:29bortexzgreat, thanks :+1:#2019-11-1000:30Alex Miller (Clojure team)it's on the list for spec 2, but we haven't gotten to it yet#2019-11-1200:49didibusFYI, its pretty easy to generate a string generator of min/max length using: #(sgen/fmap str/join (sgen/vector (sgen/char-alphanumeric) min max))#2019-11-1214:18coder46334Hi! When I start shadow-cljs with :browser-test , insead of :app, my spec instrumentation doesn't work. The specs work with (exercise fully-qualified-ns/spec), but I can't instrument functions either through code or repl. Any ideas?#2019-11-1214:22aisamuIs something from the test namespaces requiring the files containing the specs?#2019-11-1214:24coder46334Yes. I also have an instrument command in that namespace and also tried it through REPL while in that namespace#2019-11-1214:26coder46334I can even copy the specs into REPL, run the instrument command there and it's not working#2019-11-1214:47aisamuThis is probably something with your code. We have specs on test ns's running with :browser-test, and there's nothing special about the invocation/setup:
(stest/instrument `ns/component)
#2019-11-1214:49aisamuIt's worth noting that the instrument call must happen after the fdef, so mind the namespaces loading order#2019-11-1215:00coder46334I just drop-in replaced it with orchestra-cljs.spec.test and it works#2019-11-1215:01coder46334yes, the instrument calls are after the fdefs#2019-11-1215:02coder46334So with orchestra it works, but no idea why it doesn't with normal spec. It's hard to debug when you don't get an error, it just doesn't work šŸ˜ž#2019-11-1222:51Oliver GeorgeI find myself wringing my hands when writing specs in my CLJS code. Particularly choosing namespaces for my specs. I'd like something like (s/def ::list-options/options (s/coll-of ::option)) to mean (s/def :current-ns.list-options/options (s/coll-of ::option)) where the alias isn't defined in the current ns.#2019-11-1222:51Oliver GeorgeI find myself wringing my hands when writing specs in my CLJS code. Particularly choosing namespaces for my specs. I'd like something like (s/def ::list-options/options (s/coll-of ::option)) to mean (s/def :current-ns.list-options/options (s/coll-of ::option)) where the alias isn't defined in the current ns.#2019-11-1301:12ataggartI've found it convenient to always define specs as ::foo, and reference specs with ::foo or ::x/foo. It eliminates a lot of bikeshedding and subtle bugs. I want it to fail when the ns alias isn't defined.#2019-11-1301:21ataggartOh, I see what you mean. You could just move the dot segments to the other side of the slash, e.g., ::list-options.options.#2019-11-1304:58Oliver GeorgeThat works sometimes but for s/keys you need the name part to match so it's the namespace part which needs to be adaptable/unique.#2019-11-1315:46ataggartAh, yeah, if you're using unnamespaced keywords#2019-11-1505:00valeraukoi wanted a similar thing for db enums so i made it.
(defn enum
  [value]
  (let [key-ns (namespace value)
        last-dot (.lastIndexOf key-ns ".")
        model (subs key-ns 0 last-dot)
        spec-name (subs key-ns (inc last-dot))
        type-name (str/replace (str model "-" spec-name) "-" "_")
        spec-kw (keyword (str "my-app.spec." model "s") spec-name)
        enum-value (name value)]
    (if (s/valid? spec-kw enum-value)
      (convert-enum type-name enum-value)
      (throw (ex-info "invalid enum value"
                      {:type ::invalid-enum
                       :data (s/explain-data spec-kw enum-value)})))))
this validates keywords like :my-model.status/ok against ::my-model/status to see if "ok" is a valid value
#2019-11-1222:51Oliver GeorgeOr something similar and less complected of course.#2019-11-1222:53Oliver GeorgePerhaps something like clojure.core/alias for CLJS would solve the problem.#2019-11-1404:10danielcomptonIs there a ticket for spec to bring in the rest of the generators in clojure.test.check.generators?#2019-11-1405:05Alex Miller (Clojure team)we did not have a general goal to replicate all aspects of test.check.generators in spec.gen#2019-11-1405:06Alex Miller (Clojure team)so, no as we do not intend to do so#2019-11-1405:06Alex Miller (Clojure team)you can just use them from test.check if you need them?#2019-11-1416:33kennyThe problem is that you want the lazy loading spec.gen ns provides.#2019-11-1417:02Alex Miller (Clojure team)is there something specific you're looking for?#2019-11-1417:24kennyNo runtime dependency on test.check. #2019-11-1417:25Alex Miller (Clojure team)I mean which generator functions?#2019-11-1417:41kennyThe ones that we have copied over are: generator?, let, vector-distinct-by. I believe there may be some others scattered throughout the code though.#2019-11-1417:42Alex Miller (Clojure team)is the dynaload stuff in gen exposed enough to just use?#2019-11-1417:44kennyWe had to copy over lazy-combinator macro because dynaload was private.#2019-11-1417:44Alex Miller (Clojure team)lazy-combinator is public though, right?#2019-11-1417:44Alex Miller (Clojure team)ah, so you can't get to the expansion, right#2019-11-1417:44kennyRight#2019-11-1417:47Alex Miller (Clojure team)let's a macro - so I presume that's not something you can dynaload?#2019-11-1417:47Alex Miller (Clojure team)there actually is a ticket about that one in particular#2019-11-1417:47kennyYes. But let is used a ton so having that in the spec gen ns would be very helpful.#2019-11-1417:47Alex Miller (Clojure team)vector-distinct-by could be done through (s/gen (s/coll-of ::foo :distinct true :into [])) ?#2019-11-1417:48Alex Miller (Clojure team)oh, distinct-by#2019-11-1417:48Alex Miller (Clojure team)nvm#2019-11-1417:49Alex Miller (Clojure team)generator? is prob weird given the thunking done in gen isn't it?#2019-11-1417:50kennyThis is what we are using:
(def generator? #'gen/generator?)
#2019-11-1417:50kennywhere gen is clojure.spec.gen.alpha#2019-11-1417:52Alex Miller (Clojure team)so these each seem like distinct issues#2019-11-1417:52Alex Miller (Clojure team)there's a ticket for let already, generator? could just be made public, and vector-distinct-by could be added to the dynaload list#2019-11-1417:54kennyYep. @U051KLSJF may be interested in some other missing ones. I'm pretty sure there's a few others I've run into.#2019-11-1418:02Alex Miller (Clojure team)I've made the generator? and vector-distinct-by changes in spec-alpha2#2019-11-1418:03kennyThanks!#2019-11-1420:04danielcomptonI was missing byte which prompted me to ask, there are also more generators around ints which arenā€™t exposed #2019-11-1415:31ataggartI find that the things in clojure.spec.gen.alpha are sufficient to define generators for specs, and I only need the things in clojure.test.check.generators for generating "interesting" data for tests.#2019-11-1416:03Alex Miller (Clojure team)you can get pretty far by using (s/gen <some-spec>) too#2019-11-1416:03Alex Miller (Clojure team)often I find that's easier that constructing the generator from scratch#2019-11-1518:14dpkpHow do folks usually choose between req and req-un ? Namespaces are great for organizing, but do folks usually drop them after validation w req-un? Or maybe coerce to namespaced keys? Or require upstream users to pass namespaced keys?#2019-11-1519:27seancorfield@dpkp "It Depends". We tend to use :req-un for input validation specs (coming from JSON or URL/form input) but :req for quite a bit of internal stuff, including database-related data (where we either use clojure.java.jdbc's :qualifier option to add a generic "namespace" to all columns, or next.jdbc's default table-qualified column names).#2019-11-1519:29seancorfieldIt does mean that at system boundaries you have to decide where (or even whether) to switch from unqualified keys to qualified keys -- but I think that's reasonable since I would expect input data to be fairly "flat" and domain data to be a more interesting shape, i.e., I would expect a mapping from input -> domain to exist anyway in your code, even in the absence of Spec.#2019-11-1522:12cjmurphyWhen you believe a spec is not being picked up for whatever reason (I'm using guardrails/ghostwheel - therefore expound and spec are picked up), is there a way to clear the registry, just to test the assumption?#2019-11-1522:18ghadiwhat do you mean by "not being picked up"?#2019-11-1522:20kszabojust use: https://clojuredocs.org/clojure.spec.alpha/form to verify#2019-11-1522:50cjmurphyThanks that helps. But I just verified they are as they are written in the file that defines them. The problem is unlikely to be with spec itself. But if I could somehow 'invalidate the cache' I'd be able to force the other tools to recalculate, hopefully...#2019-11-1522:33cjmurphy@ghadi 'not being picked up": I have a spec for essential-rule and another for rule. One of the function parameters has been changed from rule to essential-rule, yet there's still a spec failure and the expound message thinks that the function parameter is rule rather than the new and more lax essential-rule. I've touched various files, restarted the JVM/REPL. But still somehow 'it' still thinks the function is different to the way it is written.#2019-11-1522:52kszaboif you restarted the JVM then itā€™s not a cached thing#2019-11-1523:02cjmurphyI worked it out, and its not the first time I've fallen for this šŸ˜ž. The function in question is multi-arity and I was changing one of the arities and not the other. Thanks for your help kszabo and ghadi.#2019-11-1713:37tianshuin spec2, should {:closed true} be available for schema spec?
(s/def ::user (s/schema [::name ::age]))

;;; contains? not supported on type: java.lang.Boolean
(s/valid? ::user {::name "cat" ::age 10} {:closed true})

(s/def ::complete-user (s/select ::user [*]))

;;; ok
(s/valid? ::complete-user {::name "dog" ::age 15} {:closed true})
#2019-11-1715:40Alex Miller (Clojure team)Itā€™s a set of schema names, not a boolean#2019-11-1722:28coder46334Hi! Is there a way to spec a return type of Promise<something> ? I would need this in quite a few places using cljs#2019-11-1723:19Alex Miller (Clojure team)Not usefully, no#2019-11-1723:22coder46334Thanks!#2019-11-1804:28tianshu@alexmiller thanks!#2019-11-1821:05colinkahnIā€™ve used with-redefs around an st/check in ClojureScript code and it has worked fine, but trying in Clojure code it doesnā€™t. I had to redefine a my-function-with-redefs and spec that with the redefs inside the body. Is that expected? Iā€™m assuming itā€™s something to do with multiple threads vs single thread in JS#2019-11-1821:24Alex Miller (Clojure team)might be - st/check uses pmap which is multi-threaded in Clojure (or possibly it's really the laziness of pmap)#2019-11-1821:25Alex Miller (Clojure team)make sure you realize the result of pmap with doall or something inside the lexical scope of the with-redefs#2019-11-1916:41colinkahnAwesome! That worked#2019-11-1917:09Lukasz HallData generation question - I have a top-level (map) spec composed of many nested specs and their associated generators at a leaf level. Some of the leaves of the nested structures use the same spec as other leaves, and I need those that share a spec to produce the same values when they generate. Is there any way to have this happen without explicitly writing a generator for the top level spec?#2019-11-1917:37Alex Miller (Clojure team)No#2019-11-1917:37Alex Miller (Clojure team)To guarantee properties at the aggregate level, you need to control generation at the aggregate, as a general principle #2019-11-1920:21telekidIā€™m curious how people integrate instrumentation with testing. Do they create an instrumentation fixture and call it in a :once block at the top of every test ns? If so, how do you restore instrumentation to its initial state at the end of that fixture?#2019-11-1920:39Alex Miller (Clojure team)unstrument exists to do that#2019-11-1921:43seancorfield@jake142 I instrument directly at the top of a test file (after the ns form) and don't bother undoing it. I do it separately from my test fixtures because those are often functions I reuse across multiple test namespaces, whereas the instrumentation that I do (if any) is usually limited to the functions under test in a specific namespace.#2019-11-2010:21abdullahibrahello everyone#2019-11-2010:21abdullahibrawhat is the spec for checking file data type ?#2019-11-2010:53vlaaadyou mean like that #(instance? java.io.File %)?#2019-11-2013:33DaoudaHey Folks, I am learning spec and need help to understand the registry parte. They say that it help to avoid name collision, but since we already have namespaces, they aren't enough to avoid that? I don't get to see the real advantage of using :: in spec declaration. Can someone help me to understand please with example will be wonderful#2019-11-2013:58Alex Miller (Clojure team)::keywords are ā€œauto-resolvedā€ to have the current namespace so itā€™s just a useful tool for succinctly making qualified keywords#2019-11-2013:59Alex Miller (Clojure team)They create the exact same result as using :my.ns/keyword so use that if you prefer!#2019-11-2014:00Alex Miller (Clojure team)Youā€™ll see them in a lot of examples as it just makes the example shorter#2019-11-2016:13DaoudaThank you very much @alexmiller#2019-11-2017:00DaoudaI think I`ve got the big picture now. When you want to use a spec/def, first spec look in the spec-registry if exist then take from there. Else it create it and store it in the spec-registry. A very naive description of course hehehe#2019-11-2019:33DaoudaHey Folks, when should I use qualified or unqualified keys in clojure spec?#2019-11-2019:43Alex Miller (Clojure team)qualified#2019-11-2019:44Alex Miller (Clojure team)unless you are working with existing data, don't want to transform it, and it has unqualified keys#2019-11-2111:12DaoudaHey folks, can you help me to understand why I can't gen/generateor gen/sample:
(def email-regex #"^[a-zA-Z0-9._%+-]
#2019-11-2111:18guyHave you tried exercising your specs?#2019-11-2111:18guyCouldn't satisfy such-that predicate after 100 tries#2019-11-2111:18guyThats the key part to the error message#2019-11-2111:18guyI would just go through each def and see if they can produce a result#2019-11-2111:19vlaaadmy bet is on email and phone#2019-11-2111:19guyyes i would agree#2019-11-2111:19guyIf the predicate is too hard to er fufill? based on random string input, don't you usually just create a generator for those complicated predicates?#2019-11-2111:20guyalso i'm not all that sure how (spec/def ::phone #"^\d{12}$") works#2019-11-2111:26guyIf you exercised ::phone what would you get? thats a pattern isnt it šŸ¤”#2019-11-2111:28guyUnable to construct gen at: []#2019-11-2111:28guyThats what i get#2019-11-2111:28guyfor (s/exercise ::phone)#2019-11-2111:29DaoudaJust saw now, the answers, will be right back after doing what you said šŸ˜„#2019-11-2111:29guySo then I think for the email, its just as simple as, the string generator can't create a random entry that matches your regex#2019-11-2111:29guywhich makes sense right? as even after 100 times its going to be difficult to randomly create an email i think haha#2019-11-2111:30guyhttps://clojure.org/guides/spec#_custom_generators#2019-11-2111:32DaoudaAfter trying to exercise each attribute, it appears that ::phone and ::email-type can't be exercised#2019-11-2111:35acron@quieterkali Use https://clojuredocs.org/clojure.spec.alpha/with-gen to provide alternative generators#2019-11-2111:36acronAlso useful for composing generators: https://clojure.github.io/test.check/clojure.test.check.generators.html#2019-11-2111:43guyahh thanks that was the link i was looking for. šŸ‘#2019-11-2117:19seancorfieldhttps://github.com/gfredericks/test.chuck has a generator for regex strings that I've found very useful.#2019-11-2122:18sgerguriI am trialling offloading some extra logic onto s/or and would like some collective opinion, if y'all were so kind. Suppose I have a spec defined like this:
(s/or :x :x/item
      :y :y/item)
Now suppose I add x1 and want to combine it with x at some point; so let's say I start making the semantic distinction between what this thing is and what it can be combined into:
(s/or [:additive :x] :x/item
      [:additive :x1] :x1/item
      [:multiplicative :y] :y/item)
Now, suppose further that I want to encode the order in which the respective operation should be defined. Now I'm thinking it would be really useful to be a bit more descriptive in my tags, so why not use a map?
(s/or {:type :additive
       :operand-type :x
       :precedence 1}
      :x/item

      {:type :additive
       :operand-type :x1
       :precedence 2}
      :x1/item

      {:type :multiplicative
       :operand-type :y}
      :y/item)
Now I'm thinking, should I simply add this via a conformer, keeping the tags simple (and thus transforming them at a higher level elsewhere in the spec) or do I keep this? This might all seem like a contrived example - and it is, because my use case isn't algebraic operations, but rather data merging, and thus it's really important to define what order each data types should be merged in, and what the parent type is, so that the correct merge strategy is applied. My question is - am I really using s/or in a way I shouldn't be or would you still consider this ok?
#2019-11-2206:27jaihindhreddyMy 2c is that: this is trying to do too much with spec. I'd keep the tags simple keywords, and do the manipulation later, using spec only to suss out the implicit ordering or types into explicit data (via cat and or respectively).#2019-11-2209:59sgerguriI'm always torn between that and doing more. Spec feels like it's one step away from automatic transformations, with conformers being the key enabling element for this.#2019-11-2210:42jaihindhreddyI can sympathise 100%#2019-11-2210:43jaihindhreddyBut spec's neither a kitchen sink, nor a silver bullet šŸ˜€#2019-11-2211:06sgerguriI think if specs supported metadata one could use simple tags and just indicate code-level stuff there. Perhaps we'll get that functionality in version 2.#2019-11-2211:38jaihindhreddyI dunno if spec wants to do that. Because the awesome thing in Clojure is that, we already have namespaced keywords, which are (when correctly used) globally precise names. And this means just like spec maintains a global registry of specs attached to these names for the purpose of specification, we can write our own things which maintain global registries of other stuff attached to the same names, for use in other contexts. I think this is one of those instances where making things global makes sense, and alleviates parochiality. For this kind of a thing to be realistic though, I feel like the idea of providing and requiring contexts needs to be first class and written down in code, so that automatic verification of breakage can be detected and avoided, making way for an ecosystem that can integrate various things seamlessly while still being dynamic and "growable"#2019-11-2213:48Alex Miller (Clojure team)conform is designed to tell you why something was validated and separate data into parsed components#2019-11-2214:00Alex Miller (Clojure team)It is not a general ā€œdata transform meat grinderā€ and youā€™ll be disappointed if you try to use it that way#2019-11-2516:58vemvGiven a ns that goes like this:
(defmulti event-type ...)

(spec/def :some-event (spec/multi-spec event-type ::type))

(defmethod event-type :some-dispatch-value [_]
  (spec/keys :req-un [::login ::email]))

;; ...many other defmethods...
...how to generate :some-events only with :some-dispatch-value type? i.e., exclude all other dispatch values (i.e. the specs coming from other defmethods)
#2019-11-2523:28sgerguri(g/generate (s/gen event-type {::type :some-dispatch-value}))#2019-11-2600:36vemvawesomesauce! thanks#2019-11-2711:34doesntmeananythingwhat's the best way to spec a nested map? Say, I have a customer map that looks something like this:
{:email "
how would you go about validating these nested fields? Simply write a couple of spec registries and then collect them in an entity map? So you'd do:
(s/def ::customer (s/keys :req-un [::email ::password ::first-name ::last-name ::organization]))
and ::organization would be specced to have the namespaced nested data? I'm a bit unsure on how to proceed since the official guide doesn't really have explicit examples of nested data as far as I can see
#2019-11-2711:51vemvI'd say you're going in the right direction try the pattern you have posted, and verify if you are getting the desired validations. Else post the exact failed attempt here#2019-11-2712:08vlaaadhttps://clojure.org/guides/spec#_a_game_of_cards#2019-11-2712:09vlaaadthis is an example of nested data structures: players/deck in a game#2019-11-2712:09doesntmeananythingcool, thanks for the help, guys#2019-11-2722:39bbloommini-survey: are folks using https://github.com/danielcompton/defn-spec or something similar? how do you like/dislike it?#2019-11-2800:24vemvI authored https://github.com/nedap/speced.def and we're happy with it at work. Since it compiles to a clojure/script defn it has zero limitations or new syntax, guaranteed. it doesn't preclude creating generic/reusable specs: you can author those and still use them here. It has no :fn equivalent (only :`args` :ret ones), but honestly I have yet to see how that is not equivalently achieved with the occasional :pre condition.#2019-11-2800:25vemvI haven't campaigned much about it so far, but I plan to post something to #news-and-articles later this week :)#2019-11-2722:56seancorfield@bbloom I don't use it -- I often put fdefs in a separate namespace in libraries so they are optional and the code can then run on older Clojure versions.#2019-11-2722:57seancorfield(and I don't like that syntax so I wouldn't want to use it even when I would put fdefs above the function for which they exist)#2019-11-2722:58seancorfieldSeems like a lot of limitations as well https://github.com/danielcompton/defn-spec#limitations#2019-11-2722:58bbloomthanks for the thoughts - iā€™m not using anything like it yet b/c i like to kinda use the primitives until i grok them fully and then and only then abstract more#2019-11-2722:58seancorfieldThose limitations alone would make it non-viable for most of my fdef use cases I think.#2019-11-2722:59bbloomiā€™m still trying to figure out where/when to utilize spec#2019-11-2722:59seancorfieldhttps://corfield.org/blog/2019/09/13/using-spec/ is about how we use it at World Singles Networks#2019-11-2722:59bbloomiā€™ve been doing a lot of TypeScript for work lately, and all the inline annotations are simultaneously: laborious, helpful, and hurtful#2019-11-2723:07bbloomthanks @seancorfield, that all makes sense to me - to some extent iā€™m trying to decide how far to do w/ development time checking#2019-11-2723:09bbloommy current thinking is basically just-in-time specing, where i spec stuff as part of my debugging process and leave the specs around for future use#2019-11-2723:13didibusI've got a library like that in the works, probably 80% done at this point. Don't know if or when I'll finish it. But my focus is on fn specs, since those are the painful ones to write in my opinion. As well as multi-arity, var-args, et all. Those are the hard ones to write, so my DSL intends to make these way quicker to spec, as to encourage more spec writing. Right now the machinery attaches itself as metadata on defns, so it also is compatible with older Clojure versions.#2019-11-2723:14didibusSome edge cases can be tricky, which is why it's taking me longer than I initially thought to put together.#2019-11-2723:15didibusBut I do think I'd write more fn specs if defn had a easier syntax for writing them. So that's what I'm trying to achieve. I also recognize it is hard to come up with one that is simpler but not limited. Which is why there probably isn't an official one.#2019-11-2800:29Alex Miller (Clojure team)Thatā€™s on the table for spec 2#2019-11-2800:29Alex Miller (Clojure team)But they are likely to be significantly different than spec 1#2019-11-2802:01bbloom@alexmiller Could you please clarify: ā€œThatā€™sā€ == what exactly?#2019-11-2802:04Alex Miller (Clojure team)specs integrated into defn#2019-11-2802:05bbloomah, ok, just wanted to be sure thatā€™s what you meant šŸ™‚ very cool. looking forward to seeing what you folks come up with#2019-11-2802:34didibusWhy is there no support for and/or inside of s/keys :opt ?#2019-11-2802:35didibus
(s/keys :req-un [::a]
        :opt-un [(and ::b ::c)])
#2019-11-2802:35didibusIn my case, ::b and ::c are optional, but if anyone of them is there, the other has to be as well#2019-11-2802:45Alex Miller (Clojure team)Because optional things are ... optional#2019-11-2802:45Alex Miller (Clojure team)Youā€™ll need to s/and another predicate if you have additional constraints#2019-11-2802:46didibusHum.. its just more annoying, because the combinations to list out with s/and can quickly grow#2019-11-2802:46didibusbut that's what I'll do for now#2019-11-2802:48didibusI guess I need s/or#2019-11-2802:49didibus
s/or (s/keys :req-un [::a]) (s/keys :req-un [::a ::b ::c])
#2019-11-2803:02didibusHum... its a lot trickier then I thought#2019-11-2803:02didibusOr I'm tired#2019-11-2803:03didibus
(s/or :one (s/keys :req-un [::a]) :two (s/keys :req-un [::a (and ::b ::c)])
This means that the following is valid as well:
{:a :b}
Because the first s/keys is open.
#2019-11-2816:14gwsYou could do something like this, which doesn't require you to list all possible optional combinations#2019-12-0200:16fredericHi. Do you have guidelines, or rules of thumb for naming specs in the following two situations 1. do you rather choose a name that describes the role that the data plays in the enclosing structure (say, ::confirmation-url), or a name that describes the intrinsic kind of data that key holds (say, ::url). I tend towards the former, because I think the more descriptive keys read better, but I'm wondering if by doing so I'm not inadvertantly creating private DSL (fragmenting keys when I could have one large overarching key instead and maximise the number of functions that can operate on the data) 2. when do you use symbols rather than keywords to name specs? I tend to use symbols as a kind of zero-arg spec-op, specs that I think of as building blocks, reserving keywords for specs that I would expect to actually appear in maps. Is that a personal quirk of mine?#2019-12-0200:21Alex Miller (Clojure team)1. I would say both. Name the generic thing, then have the specific one refer to the generic one.#2019-12-0200:22Alex Miller (Clojure team)2. You should never name specs with symbols (those are reserved for function specs)#2019-12-0200:22Alex Miller (Clojure team)Lots of places in spec make that assumption and you are likely to be burned by using symbols.#2019-12-0200:27fredericThank you! That answer was fast and very clear cut, I appreciate that šŸ™‚#2019-12-0202:09favilaBy ā€œsymbolsā€ do you mean vars with def or literally (s/def ā€˜symbol ...)?#2019-12-0202:11favilaI go back and forth on whether I should register specs or just def the spec object. Iā€™m on a ā€œdonā€™t registerā€ swing now#2019-12-0202:23seancorfield@favila (s/def ::keyword ...) vs (s/def 'symbol ...) -- the code assumes that only function specs use symbols (s/fdef 'foo ...)#2019-12-0202:26favilaI know this, I am asking OP what he meant#2019-12-0202:31seancorfieldBoth Alex and I made the same assumption and the OP didn't correct Alex so... ĀÆ\(惄)/ĀÆ#2019-12-0211:12fredericYou were correct about what I meant, @seancorfield. Thanks for suggesting using a plain def when there's no need to s/def a keyword, @favila.#2019-12-0309:13QuestAnyone have a convenient solution for defining clojure specs at runtime? Channel logs say either wrap with another macro OR to use eval. Macro approach requires everything else to become a macro & still doesn't allow full runtime extension. Eval works but creates more complexity to get it working in CLJS.#2019-12-0309:18QuestInterestingly, I can simply copy my own version of res (private FN from clojure.spec.alpha) & suddenly the below works.
(s/def-impl k (res v) v)
While hackish it proves that dynamic definition works just fine, but seems to have been intentionally disallowed? Though I still prefer over the eval nonsense. (Which for completeness sake is included below:)
(eval (list 's/def k v))
#2019-12-0309:31QuestLooks like CLJS still hiccups on the def-impl approach due this a difference in how clojure.core/resolve works. It's a macro in CLJS but a FN in Clojure? (I'm curious how the impl differs under the hood if anyone knows...) The offending line:
(symbol? form) (c/or (-> form resolve ->sym) form)
In my case I'm just passing namespaced keywords at runtime, so this part is unnecessary for me. Simply commenting it out obviates the issue. I'm sure this breaks many best practice recommendations, but if this is the most direct solution then I'm happy with it. I understand that spec2 should support this more easily & development is focused there, meaning that spec1 is unlikely to change & break this approach.
#2019-12-0310:33Alex Miller (Clojure team)Correct. There are multiple non-macro ways to make this work now in spec 2#2019-12-0413:38Vincent CantinHi, I am trying to define a cross-recursive spec (i.e. 2 specs defined using the other one) using spec-alpha2. How to do that?#2019-12-0413:40Vincent CantinMy repl complains on the first definition, saying that it does not find the second one (which is defined after)#2019-12-0413:57Alex Miller (Clojure team)That wonā€™t work at the moment#2019-12-0413:58Alex Miller (Clojure team)Thereā€™s a pass needed to ensure every spec properly delays resolution of named specs#2019-12-0414:11vlaaadwill it work in spec-alpha2?#2019-12-0414:20Alex Miller (Clojure team)yes#2019-12-0414:20Alex Miller (Clojure team)it's just work I haven't done yet#2019-12-0414:20Alex Miller (Clojure team)it (mostly) works in spec 1#2019-12-0502:08Lukasz HallHow fully is support for unqualified keywords baked into spec2? (Last example of https://github.com/clojure/spec-alpha2/wiki/Schema-and-select#unqualified-keys-1 fails with latest codebase)
(gen/sample (s/gen (s/select {:a int? :b keyword?} [:a])) 5)
Error printing return value (ExceptionInfo) at clojure.test.check.generators/such-that-helper (generators.cljc:320).
Couldn't satisfy such-that predicate after 100 tries.
#2019-12-0503:22seancorfieldI expect that's just a bug in master -- or a doc bug.#2019-12-0503:23seancorfield@lukaszkorecki Spec2 is very much a work-in-progress right now.#2019-12-0503:24seancorfield@hall ^#2019-12-0509:35QuestTook a closer look at spec2. Really like ease of programmatic definition & similar API is a plus. Good vision & look forward to moving to it after CLJS support šŸ™‚šŸ‘#2019-12-0510:21rickmoynihanIn lieu of spec 2, is there any advice on how to structure spec 1 specs, so that theyā€™re more likely to be easily portable to spec 2's schema / style?#2019-12-0510:21rickmoynihanFor example, defining essentially the schema (leaf) keys, and then essentially defining what would be the select ions on the fdefs.#2019-12-0510:21rickmoynihanObviously that can lead to some duplicationā€¦ is there a way in spec 1 to dissoc an s/key :req from an s/keys spec?#2019-12-0510:22rickmoynihanI canā€™t see such a thing; so Iā€™m thinking itā€™s better to build the composites from multiple s/keys at the fdefsites with s/merge?#2019-12-0510:28rickmoynihani.e. it seems you can kind of simulate spec2's idiomatic structure with spec1#2019-12-0512:52kszaboI had this discussion with Alex before, your ā€˜record typesā€™ should all use :opt/`opt-un` , at usage sites (endpoints/fdefs) you can tighten them down by (s/merge ::user (s/keys :req [:user/name :user/age]))#2019-12-0513:16rickmoynihan@thenonameguy: great, thatā€™s what Iā€™ve been doing. Thanks for the clarification. Is ā€˜record typesā€™ a phrase Alex mentioned, or is it one youā€™ve coined (hence the bunnies?) I ask because I wouldā€™ve thought he wouldā€™ve said ā€˜schema typesā€™ or just ā€˜schemasā€™, given that is the terminology used in spec2.#2019-12-0513:17rickmoynihanOr is there some more nuance to that?#2019-12-0513:24kszaboI just came up with it, case class/record type/schema, there are lots of names for arbitrary groupings of attributes. That is my conclusion from these spec2 developments: Aggregates are incidental and are context-dependent.#2019-12-0513:33kszaboAnd you can see the industry trend towards systems that are attribute-focused and not aggregate focused. Think of SQL vs Datomic for instance, folks appreciate the multi-dimensional flexibility of EAV* stores vs predetermined tables. There is also REST vs GraphQL where the predetermined ā€˜resourceā€™ is inferior to getting just the attributes that you want.#2019-12-0513:34rickmoynihanYeah, I do a lot of work with RDF; which is pretty much the poster child for property-based open-world thinking.#2019-12-0514:08Alex Miller (Clojure team)and the inspiration for parts of spec#2019-12-0514:24rickmoynihanyeah and it really showsā€¦ same for clojure too actually šŸ‘#2019-12-0514:31rickmoynihanIncidentally Iā€™m not sure if you or Rich had seen SHACL or SHEX; but theyā€™re remarkably similar to spec (but for RDF): https://www.w3.org/TR/shacl/ https://shex.io/ Iā€™ve often wondered if they were part of the inspiration behind spec, or whether spec and SHACL/SHEX just ended up in the same sort of place because of the nature of RDF / OWL. Either explanation seem plausible to me.#2019-12-0514:34rickmoynihanIncidentally this (free) online book describes both of them and their usage, and their differences: https://book.validatingrdf.com/#2019-12-0513:35kszaboAlthough you can take one more step and go to Pathom which is even more attribute-focused, there are no predetermined ā€˜typesā€™ at all, very RDF-like#2019-12-0513:37rickmoynihanI should check out pathomā€¦ itā€™s been on my radar for a while ā€” but Iā€™ve not really dug into it, so thanks for the pointer.#2019-12-0513:37kszabohttps://www.youtube.com/watch?v=IS3i3DTUnAI highly recommended#2019-12-0513:38kszaboWe are in the process of adding Pathom to all of our APIs so that we can later have a unified ā€˜maximalā€™ graph, it has been great so far#2019-12-0514:58eggsyntaxI thought there was a way to say in a map spec that a key could be either qualifed or unqualified. I misremembered that :req-un behaved that way, but apparently not:
all-test> (s/def :foo/bar string?)
:foo/bar
all-test> (s/def ::baz (s/keys :req-un [:foo/bar]))
:dw-domain-specs.specs.all-test/baz
all-test> (s/valid? ::baz {:bar "hi"})
true
all-test> (s/valid? ::baz {:foo/bar "hi"})
false
Is there any good way to say that either qualified or unqualified is fine? I'd like to be able to reuse the same spec both in internal contexts where keys are qualified and at the system boundary where keys come in unqualified. The simplest thing I've thought of is to use :req-un and always strip namespaces before checking for validity. But I thought that spec itself had a way to express that...
#2019-12-0515:04Alex Miller (Clojure team)it doesn't#2019-12-0515:05eggsyntaxOh well. Thanks, Alex! Might be an interesting feature to consider at some point, it'd make spec reuse easier and encourage people to use qualified keys when possible rather than defaulting to the least common denominator of unqualified keys (in cases where they have to deal with unqualified keys somewhere in their system).#2019-12-0515:31Alex Miller (Clojure team)the intent with spec is to strongly encourage you to use qualified keys#2019-12-0515:32Alex Miller (Clojure team)spec 2 has more support for working with unqual in schemas though#2019-12-0515:32Alex Miller (Clojure team)unqualified that are not tied by name to qualified, but just directly to specs#2019-12-0516:23eggsyntaxMeaning something like this?
(s/def :foo/bar string?)
(s/select {:bar :foo/bar} [:bar])
#2019-12-0516:24Alex Miller (Clojure team)yeah#2019-12-0517:46eggsyntaxIn a spirit of spec 2 brainstorming -- it seems like this case is similar to closed spec checking: something that it's better to avoid, but in some specific contexts turns out to be necessary. It would be cool if rather than write the boilerplate I gave above (which would be tedious and noisy in the case of an edge function that takes a map with a lot of unqualified keys) you could use an :unqualified setting with conforming API calls (just like with :closed):
(s/valid? ::foo {:bar "a"} {:unqualified true})
This also somewhat parallels the use of :keys in map destructuring for the common case where you want the binding names to be the same as the key names -- ie it handles the common case where the unqualified keys are the same as the names of the qualified keys. Just a thought šŸ™‚
#2019-12-0518:05Alex Miller (Clojure team)again, trying to encourage qualified keys#2019-12-0518:06eggsyntaxFair enough šŸ™‚#2019-12-0518:06Alex Miller (Clojure team)the settings are really for conform ops (valid? / conform / explain)#2019-12-0518:07Alex Miller (Clojure team)unqualified would also need to be in play for gen#2019-12-0518:10eggsyntaxThat makes sense. I was only suggesting it for valid? / conform / explain, but I see how gen breaks the parallelism with :closed in an important way.#2019-12-0516:19eggsyntax> the intent with spec is to strongly encourage you to use qualified keys For sure, and I'm 100% bought into that idea. But sometimes at the edges you have to accept unqualified keys, and it'd be nice to be able to say, even on a case-by-case basis, "I'm looking for qualified keys, but I'm willing to accept unqualified" rather than (as currently) either a) making a separate unqualified spec for the same sort of data structure (eg a user), for use at the edges; or b) using a :req-un spec across the board and internally stripping namespaces from keys before you check for validity. What would you say is the best way to handle that situation?#2019-12-0516:23Alex Miller (Clojure team)I think I'd either do a) or go the other way and add a pre-transformation to qualify the attributes so they match the qualified specs#2019-12-0516:23Alex Miller (Clojure team)b is stripping good information so I don't think I'd do that#2019-12-0516:24Alex Miller (Clojure team)if you're going to change the data, at least make it better :)#2019-12-0516:24eggsyntaxGood point, and I should have thought of that since I do have places where I already do that šŸ˜†#2019-12-0611:55FloHow can I check whether the (optional) keys are within a set? e.g. valid keys are #{:foo :bar :qux} and would (s/valid? :myspec {:foo true})#2019-12-0611:58Floah! map-of is my friend#2019-12-0614:45bortexzWhen using spec in the context of domain modelling, sometimes I see that some of the require I do are only for specs, as some entities would have properties from a different entity (sometimes to define relationships). This makes it very easy to end up with circular dependencies, is there any way on the namespace declaration to have only ns aliases, without requiring them to avoid circular dependencies?#2019-12-0614:46Alex Miller (Clojure team)no, but you don't need to require them at all to use keywords with a certain namespace#2019-12-0614:46Alex Miller (Clojure team)you just have to write the full namespace out w/o aliases#2019-12-0614:48bortexzOk, I thought about that, I wanted to know if there was any way to avoid having to write the full namespace#2019-12-0614:49Alex Miller (Clojure team)this is an area where we may add something in the future (https://ask.clojure.org/index.php/2817/lighter-weight-aliasing-for-keywords)#2019-12-0618:34telekidWeā€™ve been using this pattern at Flexport: https://ask.clojure.org/index.php/2817/lighter-weight-aliasing-for-keywords?show=8918#a8918 Not sure if itā€™s a good idea necessarily, but it works for us.#2019-12-0618:59seancorfieldThat looks a lot more work and complexity than a simple alias/`create-ns` like I pasted in the main channel. Could you explain what your macros are trying to do, beyond just the alias/`create-ns` call I showed?#2019-12-0620:43telekidYeah, I think my def-synthetic-ns fn is overly complex, probably because of a combination of ā€œmy first macroā€ confusion and not noticing that create-ns returns the ns#2019-12-0614:50bortexzNice, thanks @alexmiller !#2019-12-0617:14vemvnot sure if I already mentioned this one to you last time we talked, but I tend to use a namespace called kws.clj (for "keywords") which contains only specs, and zero implementation details (other than very simple predicates) since kws definitionally depend on nothing, you can always depend on a "kws" ns, and no circular deps will arise#2019-12-0617:59bortexzThat could be useful, but I prefer to have different namespaces as I want to keep ns keywords on the namespace that contains function that operate on them, for now I am using keywords under namespaces that do not correspond to the files, like :product/id instead of ::product/id , even if I define itā€™s spec in ::product namespace. I prefer this to writing the full namespace, for that Iā€™ll probably wait until there is a better way of doing so#2019-12-0618:09vemvI see! In case it wasn't clear, I actually use one kws ns per model (or 'module', depending on the chosen arch) e.g. product.clj + product/kws.clj#2019-12-0618:18bortexzOh ok, I thought you meant one big kws. Although this still does not solve interdependency between models where you want to reference each other through the specs, as you might need to import to define specs from another place. Image orders from a product, you might have ::product/id inside an order, and have ::orders referencing orders from product#2019-12-0618:24vemvthat design sounds problematic per se tbh: the DB design might not be sound, and in any case the generative part of spec might choke on the mutual recursion#2019-12-0618:30bortexzBut databases have relationships of this kind, by the product_id of an order you are able to get all orders from a product (::orders) . It would be the case when you want to hydrate a one-to-many relationship, the many has id of the one , and the one has an array of the many#2019-12-0618:33bortexzI am not very familiar with the generative part of spec, but a problem I can see in generative is that it wouldnā€™t generate the same :product/id for all orders when generating a set of orders, aside from that the specs per se are not recursive, no spec end up calling itself, but both reference things from the other#2019-12-0618:36vemvone-to-many is easily modeled as:
product/kws.clj # zero references to orders; since products exist independently from orders
order/kws.clj # references product specs; this is a simple (non-cyclic) dependency
and even a fancy many-to-many relationship can be modeled as:
product/kws.clj # zero references to orders
order/kws.clj # zero references to products
order_product/kws.clj # depends on orders and products
#2019-12-0618:38vemvI wouldn't let a trivial concern like ids (normally ids have the same type across all SQL tables) introduce cyclical deps IMO that's true regardless of the way you decide to use spec#2019-12-0618:55bortexzI still think could be useful to have some product info on the order itself (without directly referencing a product). Maybe this scenario is not the ideal one for this use case, but I still think you might want to reference inside your schema specs from other ns, I donā€™t think itā€™s bad to have cyclical namespaced keywords inside specs, as itā€™s not a dependency itself, and allows a lot of flexibility for how you retrieve your domain entities. In my case the spec are not directly translated to database tables or columns (at least not a database I control), and sometimes properties from a parent entity are hydrated directly on the child entity, not always referencing by id#2019-12-0618:58bortexz> I wouldnā€™t let a trivial concern like ids (normally ids have the same type across all SQL tables) introduce cyclical deps Namespaced keywords do not need a require, so itā€™s not strictly a cyclical dep, thatā€™s only for aliasing them, and I think it is better to have the flexibility of being able to do that (or at least as it is now just using full namespace)#2019-12-0619:00vemvCould be, I'd have to check it out carefully Still, keep in mind that specs are composable. so you can have non-cyclical, discrete product and order specs, and then completely aside, a hydrated-order spec that composes both. keeps things non-cyclical, without compromising DRY either :)#2019-12-0619:02vemvalso keep in mind that at write time, order should only have thin references to products. so a full-blown spec could be an impediment, as it could require more product keys to be present. the hypothetical hydrated-order spec keeps read/write concerns separate
#2019-12-0619:04bortexzschema/select allow to handle those contextual dependencies#2019-12-0619:05bortexzBasically I am going for these specs with schema like ā€œThis is everything an order might containā€, then by context I could select some properties#2019-12-0619:05vemvyeah. it's in the radar for spec 2 as you probably know#2019-12-0619:06vemvmust be said, considering graphql-style responses, a monolithic spec can end up being problematic. there can be N "read models" of a given thing#2019-12-0619:06bortexzYeah, as this is a personal project I am already using it, so my concern was on that context#2019-12-0619:07bortexz> must be said, considering graphql-style responses, a monolithic spec can end up being problematic. there can be N ā€œread modelsā€ of a given thing I am not familiar with graphql, what do you mean N read models?#2019-12-0619:09vemvthere are N possible schemas ("models") in which a client might want a response e.g. [[1 2] [3 4]] or {1 2 3 4} (same data, different representation) a monolithic spec cannot reasonably define all schemas that client could request#2019-12-0619:16bortexzWhy not? And what do you mean by monolithic schema?#2019-12-0619:18vemvmonolithic: a single spec that can be used for describing both reads and writes, relying on select for flexiblity#2019-12-0619:19vemvproblem with that: different clients (e.g. web, android, ios) might want different data shapes, pagination, camelCase, referenced models, etc probably a single spec defining all possible combinations of those would not be maintainable#2019-12-0619:55bortexzWhat would be best in that case, one per read model and one per write?#2019-12-0619:56bortexzI think I would have 1 spec for the read you retrieve from the DB, and then map it to whatever the client expects, probably not event having a schema for the specific data shape of the client, or if you want, then a spec for each of those too#2019-12-0619:59bortexzItā€™s far from my use case though#2019-12-0703:29vemv> What would be best in that case, one per read model and one per write? at work I advocate the following: * there's one write model (since there should be only 1 acceptable way of persisting things) * by default, that also serves as the read model (since one always can read a subset of what he wrote) * there also can be N additional read models, as your needs grow so, rule of thumb is 1:N#2019-12-0703:31vemv> Itā€™s far from my use case though (edited) yeah I don't even particularly like graphql, and in a personal project you likely won't have 10 different clients ;p but I like finding patterns that scale, that are universal. that way I don't have to worry later, or have different patterns per-project#2019-12-0708:45bortexz> or have different patterns per-project I disagree here, depending on the scope of the project this could make sense#2019-12-0708:56vemvmorning :) sure thing. Personally I enjoy having a minimal amount of patterns in my head. The tradeoff being that sometimes 'scalable' solutions can certainly feel as overkill#2019-12-0709:13bortexzMorning šŸ‘‹ I think though that scalable could mean different things on different projects, to justify different patterns to achieve such scalability#2019-12-0618:12seancorfieldYou can also do this to get a local alias without needing the full namespace to actually exist as code:
(alias 'm  (create-ns 'ws.domain.member))
#2019-12-0619:03bortexzThanks! This looks really good and something that solves my use case šŸ™‚#2019-12-0618:12seancorfieldThen ::m/country will expand to :ws.domain.member/country for the spec name.#2019-12-0618:13seancorfield(in this particular case, that ns does exist, but we don't need to require it everywhere we use the specs)#2019-12-0703:49vemvGenuine question (and not an exercise in snobbery ;p), I wonder if many people's seeming disfavour for ns-qualified keywords comes from lack of (properly setup or understood?) IDE tooling. With cljr-slash (https://github.com/clojure-emacs/clj-refactor.el/blob/50d2d8aad5e0bd8002173b300f8419d72ceab7af/clj-refactor.el#L1968 ), I type ::foo/ and the right :require will be inserted in my ns declaration, and the requiring will be actually performed into the runtime that's an instantaneous op, and after that, tab completions for ::foo/ will be 100% accurate I can imagine that if lacking this tooling, using ns-qualified kws could be more tedious.#2019-12-0703:53eggsyntaxOooh, I hadnā€™t stumbled on cljr-slash before, thanks for pointing that out!#2019-12-0703:54eggsyntaxI can say that for me personally, I found it annoying using the more verbose namespaced keys, until I started to feel the benefits directly. Now Iā€™m the person whoā€™s pushing people toward namespacing keywords everywhereā€¦#2019-12-0703:54seancorfieldI don't find auto-completion/hinting to be any obstacle at all to using ns-qualified keywords, to be honest. My set up is Atom/Chlorine with its bare-bones completion (without Compliment). Stu Halloway says he doesn't use auto-complete/hinting at all in his editor setup -- and he seems to use ns-qualified keywords just fine šŸ™‚#2019-12-0703:57seancorfieldAs I'm converting code at work from clojure.java.jdbc to next.jdbc I'm working more and more with (table)-qualified keywords and I like it. I'm even changing some old code that still uses c.j.j to use the :qualifier option to add a table-qualifier to the column keywords šŸ™‚#2019-12-0703:59vemvinteresting design, thanks for sharing!#2019-12-0704:01seancorfieldIt was this that caused me to update my CFML/Clojure bridge library to support namespace-qualified keys in hash maps for interop, BTW šŸ™‚#2019-12-0822:25fredericIs this supposed to work in spec 2?
(s/def ::some-keyword ::some-other-keyword)
When I try to validate against ::some-keyword I get this exception:
java.lang.IllegalArgumentException: No implementation of method: :conform* of protocol: #'clojure.spec-alpha2.protocols/Spec found for class: clojure.lang.Keyword
I can fix the symptoms by doing this instead of s/def, but the fact that I have to do a special dance makes me suspect that I might be working against the tool's grain
(s/register ::some-keyword (s/resolve-spec ::some-other-keyword))
#2019-12-0822:34fredericI get that spec2 is stricter than spec1 in terms of separating symbolic specs from spec objects. From the documentation (https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha) (emphasis mine) > s/def is a helpful wrapper macro for s/register that understands how to interpret all kinds of symbolic specs (not just spec forms), including symbols and sets, which will be wrapped in the helper s/spec spec op. I was hoping that ::some-other-keyword would qualify as a symbolic spec, but maybe not.#2019-12-0822:34seancorfieldI believe aliased Specs are supposed to work but there are situations where you can forward reference in Spec 1 but you cannot yet forward reference in Spec 2.#2019-12-0822:34seancorfieldI assume ::some-other-keyword is defined after ::some-keyword here?#2019-12-0822:35fredericActually, it's imported from another ns in my real code, shouldn't have tried to simplify that away#2019-12-0822:35seancorfield
user=> (require '[clojure.spec-alpha2 :as s])
nil
user=> (s/def ::one pos-int?)
:user/one
user=> (s/def ::two ::one)
:user/two
user=> (s/valid? ::two -1)
false
user=> (s/valid? ::two 1)
true
user=> 
#2019-12-0822:35frederichmm#2019-12-0822:35fredericinteresting#2019-12-0822:35fredericThere must be something else going on then#2019-12-0822:36fredericLet's see if I can build a minimal example that reproduces the behaviour I observe#2019-12-0822:36seancorfieldIn that simple case, it does let you forward reference FYI -- I tried it in a fresh REPL session with the defs of ::one and ::two swapped and it still worked.#2019-12-0822:40fredericInteresting. Thank you for the handholding šŸ˜Š, will try to figure out what I'm doing differently and report back#2019-12-0822:52seancorfieldWith the caveat, again, that there are numerous bugs in Spec2 still at this point. And it sounds like function specs will change dramatically before they're done, according to Alex's latest blog point (and s/and may become non-flowing -- which would be another departure from Spec1). While I was working against the Spec2 branch for a while, things would break between various updates on master, sometimes intentionally but often unintentionally. It's very much a moving target right now.#2019-12-0822:54seancorfieldI had hoped to migrate our (large) codebase from Spec1 to Spec2 but, in reality, I think we're "stuck" with Spec1 in our existing code at this point and we'll just adopt Spec2 for new code (depending on how Spec1 and Spec2 interop -- because they don't, right now).#2019-12-0822:54seancorfieldEventually, we'll convert everything over but I suspect we'll do it piecemeal over time.#2019-12-0822:58fredericMine is just a hobby project, but yes, I might have been over eager there šŸ™‚#2019-12-0823:00fredericMaybe I should just switch back to spec 1, and follow the advice above (<https://clojurians.slack.com/archives/C1B1BB2Q3/p1575541261064300>) to make all my specs :opt rather than :req, merging in :req at the last minute to emulate schema and select.#2019-12-0823:00fredericThat way I'd also get orchestra and expound, which would be nice#2019-12-0823:01fredericAnd probably other libraries and tooling integration that I'm not aware of yet.#2019-12-0823:04eggsyntax> make all my specs :opt rather than :req, merging in :req at the last minute to emulate schema and select. I hadnā€™t seen the discussion of that above, but thatā€™s pretty much what Iā€™ve come to independently for a large-ish set of domain specs Iā€™ve been working on. Seems to be a decent approach.#2019-12-0823:06eggsyntaxAlong with a couple of helper functions to eg build a selection from a schema less verbosely by passing in the base schema and the list of attributes that should be required (in the simple cases), and by passing in the base schema and a full tree of requiredness (for complex cases).#2019-12-0901:51fredericConverting to spec 1 fixed my problem. I must have run into some corner case that wasn't covered yet.#2019-12-0915:52arohnerI have a multi-spec which works fine, but for a test I want to gen messages of a single type. If I do (defmethod my-spec ::foo [_] ::foo), then the messages are missing retag. If I include the retag key in ::foo, the keys are auto-generated and donā€™t correspond to the message type. Whatā€™s the best way to make the correct retag show up in the individual messages?#2019-12-0917:48Alex Miller (Clojure team)if you hit the backchannel here, several people have walked through this recently - zulip has the archive#2019-12-0917:49Alex Miller (Clojure team)https://clojurians.zulipchat.com/#narrow/stream/180378-slack-archive/topic/clojure-spec is the link there I think#2019-12-1009:33bortexzIn spec2, can you create a subschema of a schema, like doing select-keys on schema keeping the optionality?#2019-12-1010:21vlaaadisn't it what select is for?#2019-12-1010:32bortexzselect returns the required keys from a schema on a specific context, I am looking for something to generate schemas of optionals from a subset of keys of another schema#2019-12-1013:44Alex Miller (Clojure team)Schema keys are all optional. I donā€™t understand what youā€™re trying to do.#2019-12-1014:05bortexzImagine I have an entity, for which I have a spec/schema with a set of keys that it can have. Now, I would like to spec the request parameters to modify the given entity. You can only modify half of the parameters of that entity, although each of them is still optional. How would I approach this case? Does it make sense?#2019-12-1014:16Alex Miller (Clojure team)a select is exactly the case of specifying a more constrained structure based on a schema, intended specifically for "in a context", like request parameters#2019-12-1014:17Alex Miller (Clojure team)if the keys are optional, then the existing schema suffices#2019-12-1014:41bortexzI see, in terms of validation and instrumentation it makes sense, just use the ā€œbig schemaā€ with all possible keys and select if there is any requirement. I guess what I was trying to achieve was more aligned with a ā€œspecification / documentationā€ use of schema, so I could tell which of the keys are supported by a specific function, rather tan required#2019-12-1014:52bortexzIs there in spec any concept or would it make sense to have something to specify that a property is supported? with a meaning of The property will be used if it exists, but it's not required , or is it something that should be left to documentation / outside of the code specification?#2019-12-1014:59Alex Miller (Clojure team)nothing like that in spec, no#2019-12-1015:12bortexzThanks alex. Would it make sense in the context of spec? I think it could be something very close to the concept of select . As I imagine it, would be: ā€¢ schema: All keys that travel together, outside of function context (all are optional, although wouldnā€™t optional also be contextual? optional meaning supported or used) ā€¢ select: All keys that are used in a specific context (With the slight change that it would allow to select optional and required keys for the case mentioned above) Is this something that makes sense? Am I missing something in how I conceive spec?#2019-12-1015:17Alex Miller (Clojure team)schemas are all optional, select is all about what's required#2019-12-1015:58bortexzI (think I) understand, what I am trying to achieve maybe fits more into the metadata space (saying what is used if specified, for auto generated doc or generative test) rather than in spec. I am trying to wrap my mind around the philosophy behind the schema/select approach, I already find it quite useful how it is now, although sometimes I find myself in a 3-concepts space rather than two (schema/select) when thinking about the information model: 1. What travels together (big schema with all related kw, e.g. domain entity) 2. What a context uses (subset of kw used by the fn. Not everything used is required, but itā€™s not ignored) 3. What a context requires to work properly (required set of kw for the fn to work) Maybe it makes sense to have smaller schemas in 2. with shared kw with 1. , or to have 2. just as metadata of the fn outisde spec. Any ideas on this ? Seems a good time to re-watch Maybe not , I might be missing something about the philosophy of the tool#2019-12-1016:01Alex Miller (Clojure team)On 2, you don't have to use a predefined schema, you can just restate specifically what attrs are allowed, which may be a smaller set#2019-12-1016:02Alex Miller (Clojure team)or you can just make a schema that is a smaller set#2019-12-1016:17bortexz> or you can just make a schema that is a smaller set Iā€™ll go for that. Which brings me to the beginning, create schema from subset of another schema, (or join schemaā€™s together). Probably something that can be done as tooling outside spec2. Looking forward for a final release šŸ™Œ cheers!#2019-12-1017:28seancorfields/merge could merge smaller schemas to make a bigger schema, yes?#2019-12-1017:31bortexz^ Indeed !#2019-12-1017:33bortexzBy <https://github.com/clojure/spec-alpha2/wiki/Schema-and-select> I think s/union would be the one to use on schemas#2019-12-1017:45bortexzThinking about my previous comment of generating schema from subset of schemas, it doesnā€™t make any sense :man-facepalming: Schemaā€™s syntax is just the set of keys (when ns-kw at least)#2019-12-1017:46Alex Miller (Clojure team)We may actually re-purpose ā€œunionā€ for nonflowing s/and so donā€™t get too attached to that name :)#2019-12-1018:07seancorfieldUsual caveat: Spec2 is a moving target whose function names may change and/or whose semantics may change, potentially several times šŸ™‚#2019-12-1018:26bortexzYeah, luckily it is a personal project with a small codebase, refactors arenā€™t very painful (I do lots of them to try new approaches). Having that into account, I was avoiding to go much further than defā€™ing schemas, which it feels already useful to define my domain entities and properties (and understand them) as a common ground between a set of external services that all of them use similar entities but each one of them with different set of properties and property names and formats#2019-12-1011:12mpenetfound a bug ? https://gist.github.com/mpenet/fa611105b4a0a38126c5ba8ffe60d8a7#2019-12-1011:13mpenet#2019-12-1011:13mpenetbasically spec will happily validate against stuff that was never declared, just checking for presence of the key as if any?#2019-12-1011:15mpenetthe docstring is not really clear about that one#2019-12-1013:45Alex Miller (Clojure team)s/keys checks that required keys exists and that all registered keys have values that conform#2019-12-1013:47Alex Miller (Clojure team)So, not a bug. I think youā€™ll get exceptions if you try to gen though#2019-12-1014:03mpenetyes, it throws#2019-12-1020:35danielcomptonIs there a way to validate a "closed select"? I can see how to check a schema with closed settings, but it doesn't seem like that works for selects?
(s/def ::a (fn [x] 
             (Thread/sleep 2000)
             true))
(s/def ::b string?)
(s/def ::b-sch (s/schema [::b]))
(s/def ::b-sel (s/select ::b-sch [::b]))

(comment
  (s/valid? ::b-sel {::b "a" ::a nil})
  ; => <waits> true
  
  (s/valid? ::b-sch {::b "a" ::a nil} {:closed #{::b-sch}})
  ; => <immediately> false

  (s/valid? ::b-sel {::b "a" ::a nil} {:closed #{::b-sch}})
  ; => <waits> true

  (s/valid? ::b-sel {::b "a" ::a nil} {:closed #{::b-sel}})
  ; => <waits> true
  )
#2019-12-1100:44Alex Miller (Clojure team)I think the select linkage is not fully implemented yet, long story, wip#2019-12-1020:36danielcomptonThe use case is for API boundaries, where you want to make a selection of keys that you will accept, fail if extra keys are provided, and especially to not validate the extra keys before failing#2019-12-1102:56bbrinckSlightly off-topic, but I have a question about what people would prefer in their error messages. Iā€™d appreciate a vote if you have a minute https://twitter.com/bbrinck/status/1204595098207444993#2019-12-1122:43kennyDoes anyone have a keys spec but for a map with string keys?#2019-12-1123:00seancorfield@kenny Not sure how you'd do that -- s/keys requires keywords I believe? You could use s/map-of for string keys (and add some custom predicates depending on how much checking you need)...#2019-12-1123:00kennyYeah that's what I have. Was hoping someone had written something already that did that & generated valid values.#2019-12-1123:17Alex Miller (Clojure team)not that I'm aware of#2019-12-1123:17Alex Miller (Clojure team)you can get some of what you want by combining s/and and an s/conformer that keywordizes keys#2019-12-1123:18Alex Miller (Clojure team)not sure it works on all the api operations the way you want though#2019-12-1201:05Oliver GeorgeHi Alex, Iā€™m super excited about the new spec. Just trying to moderate my expectationsā€¦ Do you think it will be a complex port to clojurescript? From memory the first version took a while (months?) to come out. #2019-12-1201:10Alex Miller (Clojure team)ĀÆ\(惄)/ĀÆ#2019-12-1201:12Alex Miller (Clojure team)I think there were a couple tricky parts to the cljs port in gen/instrument/check - I suspect those tricky bits aren't going to change, at least in the tricky parts. And I think the increased clarity around symbols vs objects is probably likely to make the cljs code larger but simpler#2019-12-1201:13Alex Miller (Clojure team)so I don't see any reason it should be particularly hard, but there is a lot of internal structural change, enough that if it were me doing it, I'd probably start fresh and pull from the old code as needed#2019-12-1201:14Alex Miller (Clojure team)that said, we're not done yet :)#2019-12-1201:24Oliver GeorgeThanks #2019-12-1201:25Oliver GeorgeWhile Iā€™ve got you can I ask about the idea of including specs in defn. Do you think we will see :args and :ret keys added to the pre/post condition map?#2019-12-1203:11Alex Miller (Clojure team)Havenā€™t looked at that yet#2019-12-1203:12Alex Miller (Clojure team)Ret and fn specs are likely going to change a lot#2019-12-1203:20Oliver GeorgeFair enough. Thanks.#2019-12-1216:13gariepyalexI am trying without success to generate clj-time dates using specs in clj-time.spec . (s/exercise ::spec/date-time) always returns dates close to January 1rst 2011 by a few millis. The spec in the lib is using:
(spec/def ::past-and-future (spec/int-in (to-long (date-time 2011 1 1 00 00 00))
                                         (to-long (date-time 2030 12 31 23 59 59))))
or simplified
(s/exercise (s/int-in 1293840000000 1924991999000))
which always returns numbers close to 1293840000000. Under the hood, it looks like (gen/large-integer* {:min 1293840000000 :max 1924991999000}) . When I run this directly, I have the same issue. Is there a way to generate dates that are more spread from 2011 to 2030?
#2019-12-1216:22ghadiare you sure that it doesn't start near 2011, then move away progressively? @gariepyalex#2019-12-1216:22ghadiif you sample more values#2019-12-1216:28gariepyalexYou are right. With more samples some are different. Still, most date are very close to the minimum value. This is expected behavior? If I want something else, I need to write a custom generator?#2019-12-1216:30ghadiThis is expected.#2019-12-1216:30ghadithere's a control -- I don't have a link handy, but see the test.check wiki I think#2019-12-1216:31ghadihttps://clojure.github.io/test.check/growth-and-shrinking.html#2019-12-1216:32gfredericks@gariepyalex the expected behavior of large-integer is that many values are close to 0#2019-12-1216:33gfredericksSo any date/time generator naively implemented with large-integer will have the sort of behavior you're describing#2019-12-1216:33gfredericksThere's no datetime generator that doesn't privilege some span of time#2019-12-1216:35gariepyalex> From https://clojure.github.io/test.check/growth-and-shrinking.html > gen/sample starts with very small sizes in the same way that the quick-check function does. This can be misleading to users who donā€™t expect that and take the first ten results from gen/sample to be representative of the distribution of a generator. Using gen/generate with an explicit size argument can be a better way of learning about the distribution of a generator. Thanks @ghadi and @gfredericks!#2019-12-1216:36gfredericks:+1:#2019-12-1320:32kvltIs it not possible to use with-redefs inside of defspec? I have a relatively simple test that seems to suggest that it's not working: The function I'm testing is here:
(defn send-email2
  [domain]
  (gen-uri domain "/messages"))
The test using deftest:
(deftest abc-test
  (with-redefs [sut/gen-uri (constantly "")]
    (is (= ""
           (sut/send-email2 "String is ignored because of the redef")))))

 (abc-test) => nil
The test using defspec:
(defspec send-email-test 1
  (with-redefs [sut/gen-uri (constantly "")]
    (prop/for-all [domain (s/gen string?)]
                  (is (= ""
                         (sut/send-email2 domain))))))

(send-email-test) =>

FAIL in () (email_test.clj:24)
expected: ""
  actual: ("")
{:result false,
 :seed 1576269139182,
 :failing-size 0,
 :num-tests 1,
 :fail [""],
 :shrunk {:total-nodes-visited 0, :depth 0, :result false, :smallest [""]}}
#2019-12-1320:42gfredericksNot to override generators like that, no#2019-12-1320:43gfredericksI'm assuming gen-uri is a generator#2019-12-1320:44kvltgen-uri is a function that generates a url to a service I'm using to send email:
(defn gen-uri
  [domain route]
  (-> domain
      base-url
      (str route)))
#2019-12-1320:44kvltI see how it was poorly named for this example, sorry for that#2019-12-1320:44kvltWhat I really want to mock is the http call to the service that sends the email. BUt wanted to keep things simple#2019-12-1320:45gfredericksThen try reversing the order of prop and with-redefs#2019-12-1320:45kvltOh I feel silly#2019-12-1320:45kvltI didn't expect that to throw things off#2019-12-1320:46kvltThanky ou#2019-12-1320:47gfredericksThe prop call is essentially creating a function that gets called later, at which point your with-redefs has already exited#2019-12-1320:47gfredericksSimilar to if you put it around a defn#2019-12-1320:47kvltThat makes sense.#2019-12-1320:47kvltThank you again!#2019-12-1321:40Alex Miller (Clojure team)I don't know if you've looked at it, but clojure.spec.test.alpha/instrument has the ability to mock and stub#2019-12-1411:32kwrooijenHey, I was wondering if there was a shorthand for creating fdefs , For example I have:
(s/fdef new-person
  :args (s/cat :name :person/name
               :age :person/age)
  :ret :app/person)
(defn new-person [name age]
  {:person/name name
   :person/age age})
Which works, but itā€™s quite a lot noise (seeing that the fdef is larger than the function itself I know I shouldnā€™t be comparing spec to type signatures, but thatā€™s basically what Iā€™m doing here. Something like this would be pretty useful for me:
(s/sig new-person [:person/name :person/age] :app/person)
(defn new-person [name age]
  {:person/name name
   :person/age age})
#2019-12-1413:27taylorI think Iā€™ve seen a few libraries that have macros combining defn and fn specs#2019-12-1413:29taylorhereā€™s one https://github.com/danielcompton/defn-spec#2019-12-1413:29taylorhttps://github.com/danielcompton/defn-spec#alternatives#2019-12-1414:19kwrooijenThanks, these sort of do what I want. But they replace defn which I donā€™t like. I couldnā€™t find anything so I ended up writing a quick macro:
(defmacro defsig
  [fname fargs fret]
  `(clojure.spec.alpha/fdef ~fname
     :args (clojure.spec.alpha/cat 
#2019-12-1413:53Alex Miller (Clojure team)Stay tuned for spec 2.... :)#2019-12-1414:21kwrooijenLooking forward to that!#2019-12-1416:02Alex Miller (Clojure team)Rich is working on the design for this right now and itā€™s starting to look pretty good #2019-12-1514:14valeraukoI'd be interested to hear what your workflow (feedback loop?) is for developing the language. Are there any blogs about that?#2019-12-1514:48Alex Miller (Clojure team)I did a talk at Clojutre that talks about it some#2019-12-1806:30valeraukoin your Nice and Accurate Counsel?#2019-12-1416:23kwrooijenThat would be such an amazing improvement for me. Iā€™d honestly want to spec every function at my day job (I guess thatā€™s the static typing mindset in my head talking), but itā€™s just too verbose at the moment#2019-12-1416:24kwrooijenitā€™s also difficult to convince my colleagues šŸ˜„#2019-12-1613:46kwrooijenWrote a small library which does what I need: https://github.com/kwrooijen/spec-signature Hopefully spec2 will make this obsolete, but for now this works šŸ™‚#2019-12-1716:24vemv> Hopefully spec2 will make this obsolete I also hope the current alternative landscape goes mostly obsolete - obviously not good to have N competing solutions, or syntaxes. If browsing a given general-purpose library (e.g. a file unzipper, whatever), casual readers shouldn't struggle to understand its defns. It goes in the opposite direction of having a common language that we all speak :) There are some interesting ideas around though, like ghostwheel's. https://github.com/nedap/speced.def which I authored is quite minimalistic, in that regard. Although it has advanced features like spec -> type hint inference, which at this point would be painful to stop benefiting from (at work we have essentially zero reflection warnings, other than those coming from external libs. Our lib makes this easy and meaningful)#2019-12-1718:44zane> obviously not good to have N competing solutions Rich said something interesting about this a few months ago: https://www.reddit.com/r/Clojure/comments/crnq9f/joy_clark_interviews_rich_hickey_problem_solving/ex7fitv/#2019-12-1719:01vemvThe money quote being, I presume: > I'm not gonna go code up this big thing, because you know, some of the things that you're talking about as being standard, they also have a bunch of known shortcomings Fully agreed. I don't look forward to a big solution that solves everyone's problems But I do advocate small-yet-full solutions that actually solve one specific problem. Else the landscape you have is a Lisp Curse http://www.winestockwebdesign.com/Essays/Lisp_Curse.html . There's actually a pretty strong analogy between Common Lispers doing OOP and Clojurists wrapping spec :)#2019-12-1821:46aghow can I ns-resolve symbol a spec sitting in a different namespace based on a given string?#2019-12-1821:47Alex Miller (Clojure team)can you more clearly state inputs and output?#2019-12-1821:49agI want something like:
(ns-resolve
 'my-specs
 (symbol "foo"))
but for a spec. How can I resolve ::my-specs/foo when given two strings ā€œmy-specsā€ and ā€œfooā€
#2019-12-1821:49Alex Miller (Clojure team)resolve to what?#2019-12-1821:50Alex Miller (Clojure team)a spec object?#2019-12-1821:50Alex Miller (Clojure team)a fq keyword?#2019-12-1821:51agyeah, letā€™s say I want to validate a spec, but instead of spec I have its name as a string#2019-12-1821:51Alex Miller (Clojure team)my-specs is an alias only meaningful in the context of a namespace. is that the current namespace or some other one?#2019-12-1821:54agso letā€™s say I have a bunch of fields that I extracted from e.g. SQL query, itā€™s a vector of ["foo" "bar"]. I have a namespace my-specs with two specs in it: (s/def ::foo string?) and (s/def ::bar boolean?)ā€¦ now I want to validate data, or generate data, basically I need to ā€œgetā€ those specs#2019-12-1821:55agall that programmatically#2019-12-1821:55Alex Miller (Clojure team)do you know that all of these specs are in my-specs?#2019-12-1821:57agsureā€¦ letā€™s assume they are 100% there#2019-12-1821:57Alex Miller (Clojure team)if so (s/get-spec (keyword "my-specs" "foo")) should work?#2019-12-1821:58agOMGā€¦ thereā€™s a literally a function called get-specā€¦ LOL#2019-12-1821:58agThank you Alexā€¦ sorry for being so lame at explaining#2019-12-1821:58Alex Miller (Clojure team)np, just trying to impedance match :)#2019-12-1821:59aghoweverā€¦ what if I want to check if the spec is indeed there? šŸ™‚#2019-12-1821:59Alex Miller (Clojure team)well if you get nil, it's not there :)#2019-12-1821:59Alex Miller (Clojure team)you can also call (s/registry) to just get the full map too#2019-12-1821:59agahā€¦ okayā€¦ awesomeā€¦ exactly what I needed. Thanks again!#2019-12-1822:00seancorfield@ag expectations.clojure.test does a dynamic lookup like that to let you "expect" values conform to specs: https://github.com/clojure-expectations/clojure-test/blob/master/src/expectations/clojure/test.clj#L34-L40#2019-12-1822:01seancorfield(the dynamic require/resolve is so the code can run on Clojure 1.8)#2019-12-1822:01agOhā€¦ thatā€™s nice. Iā€™ll give a gander as well. Thank you Sean!#2019-12-1822:48aghey friendsā€¦ another dynamic lookup related question: if I have a (s/keys) spec and string representation of one of the fields how do I get-spec of that? e.g: I have: ::foo-spec/foo defined as:
(s/def ::foo 
  (s/keys :req-un [::bar-spec/bar]))
and in bar-spec ns I have:
(s/def ::bar (s/keys :req-un [::name]))
and I have strings ā€œfoo-specā€, ā€œfooā€ and ā€œbar.nameā€ whatā€™s the best way to get-spec of ::bar/name ? How can I make it work for multiple levels of nesting?
#2019-12-1822:53agmeaning I can get-spec/foo but now I need to ā€œanalyzeā€ its :bar field, I have no idea where it sits, is there a way to find it out?#2019-12-1822:53agdynamically?#2019-12-1822:54Alex Miller (Clojure team)there are multiple independent questions here#2019-12-1822:55Alex Miller (Clojure team)re understanding the structure of a keys spec, you can use s/form to get a fully resolved form representing the spec (so you'd see (clojure.spec.alpha/keys :req-un [:bar-spec/name]) )#2019-12-1822:57Alex Miller (Clojure team)finding the subspecs inside that spec is a matter of fishing for it (made somewhat complicated by the and / or support in s/keys). There are a variety of ways to tackle that, all a little meh, that's something we're looking at having better answers for in spec 2.#2019-12-1822:58Alex Miller (Clojure team)if you go that path, you have only fully-qualified subspecs, so you can just pass them to s/get-spec#2019-12-1822:58Alex Miller (Clojure team)and re nesting, there is no such thing as ::bar/name in this case - you've just smooshed together two independent things there#2019-12-1822:59ag> youā€™ve just smooshed together two independent things there ehmmā€¦ Iā€™m just trying to illustrate that I needed nested lookupā€¦#2019-12-1823:01Alex Miller (Clojure team)just saying that the specs themselves are not "nested" and have no naming relationship. spec references are always via fully qualified keywords (even if you use autoresolved names to specify them)#2019-12-1823:02agyeah, it seems this isnā€™t as simple as I thought it would beā€¦ so basically I wanted to figure out specs for a vector of strings like:
["account.id" "account.contact.first-name" "account.contact.address.city"]
given that all specs are there with the relations set between them
#2019-12-1823:07Alex Miller (Clojure team)what do those strings mean?#2019-12-1823:08Alex Miller (Clojure team)those are nested keys in map data or something?#2019-12-1823:10agso thereā€™s:
(s/def account (s/keys :req-un [::id ::contact])
(s/def contact (s/keys :req-un [::first-name ::address])
(s/def address (s/keys :req-un [::city])
#2019-12-1823:11agand they all may be sitting in different namespaces#2019-12-1823:13agnow without knowing the ns of city or contact but knowing where account resides and given a string ā€œaccount.contact.address.cityā€ I want to get-spec of ::address/city#2019-12-1823:20agoh.. well, it gets a tad bit crazier when some fields are s/nilable#2019-12-1823:30Alex Miller (Clojure team)the whole point of having namespaces is being able to differentiate things. seems like you're working really hard to rebuild what that gives you.#2019-12-1823:31Alex Miller (Clojure team)the whole idea behind spec is that attributes are the main source of semantics and maps are weaker aggregations of those#2019-12-1823:31Alex Miller (Clojure team)you are working significantly at odds with that idea#2019-12-1823:33agSo if you want a spec that describes a relational data, how would you do it?#2019-12-1823:35Alex Miller (Clojure team)generically, relational data is sets of maps#2019-12-1823:37seancorfield@ag it would probably help you to remember to use explicit qualifiers (on keywords) in your spec definition and stop using :: at least until you get your head around this...#2019-12-1823:37agso the above snippet with account -> contact -> address is okay?#2019-12-1823:38seancorfield
(s/def :person/account (s/keys :req-un [:account/id :account/contact])
(s/def :account/contact (s/keys :req-un [:contact/first-name :contact/address])
(s/def :contact/address (s/keys :req-un [:address/city])
for example ^
#2019-12-1823:39seancorfieldThat also gets you close to the "relational" model if you look at something like next.jdbc querying a database since it will use qualified keywords for column names, based on the table they are in...#2019-12-1823:40seancorfield(although there you have to reify the primary keys so you'd most likely have :account/id :account/contact_id and the latter would be the FK into the :contact table and its :contact/id column etc)#2019-12-1823:43seancorfieldIf you did the join with next.jdbc/execute! you'd end up with a flat map containing
{:account/id 123, :contact/first-name "Ag", :address/city "Wherever"}
#2019-12-1823:43agyeah, I see how this can simplify a few things.#2019-12-1916:23valeraukoare there any tools to use spec for static checks?#2019-12-1916:50Alex Miller (Clojure team)there's https://github.com/arohner/spectrum#2019-12-1916:51Alex Miller (Clojure team)it's something that's inherently going to have limits, but I was surprised at how far it went#2019-12-1916:53valeraukoawesome! and so many stars too! gonna dive in during the holidays#2019-12-1916:54Alex Miller (Clojure team)there's a talk about it from Conj a couple years ago too#2019-12-1916:55Alex Miller (Clojure team)https://www.youtube.com/watch?v=hzV7dFYmbAs#2019-12-1918:06joefromctif i just type into a repl a fn and a fdef, is there any reason it wouldn't be enforced for a given project? I have an older project I'm jumping into and can't understand why my fdef's aren't throwing errors... I'm sure check-asserts is true although i didn't think that would even matter.#2019-12-1918:22Alex Miller (Clojure team)fdef on macro or function?#2019-12-1918:22Alex Miller (Clojure team)if function, you have to instrument it for it to do anything#2019-12-1918:24joefromctok, so runtime checks have to be inside the form... either with an assertion, :pre check, or conform. I guess i missed that in the gude... I think i'll plug in an assertion so i can turn it off when i'm done with my test/dev/exploration on trying to figure out these data structures.#2019-12-1918:24joefromct^ fdef on a function#2019-12-1918:51Alex Miller (Clojure team)note that there is a s/assert#2019-12-1918:54joefromctthanks as always for your help Alex#2019-12-2612:52norton@alexmiller There appears to be a typo on line 16 and line 17 in this commit https://github.com/clojure/spec-alpha2/commit/7c708d063b6ea925fd406f87e08f508b7ed8c91d#diff-04c6e90faac2675aa89e2176d2eec7d8R16#2019-12-2613:55Alex Miller (Clojure team)Can you be more specific?#2019-12-2613:56Vincent Cantinon line 17, a i instead of [#2019-12-2613:58Vincent Cantin@alexmiller on line 16, .gen is missing#2019-12-2614:00Alex Miller (Clojure team)Fixed, thx#2019-12-2717:15Alexis Vincentwhy does the last example not work?
(def s-id (s/or :uuid uuid? :string string? :keyword keyword? :num number?))
(gen/generate (s/gen s-id)) ;; => works

(s/def ::id (s/or :uuid uuid? :string string? :keyword keyword? :num number?))
(gen/generate (s/gen ::id)) ;; => works

(s/def ::id s-id)
(gen/generate (s/gen ::id)) ;; => doesn't work
#2019-12-2717:16Alexis Vincentare these not equivalent?
(s/def ::id (s/or :uuid uuid? :string string? :keyword keyword? :num number?))
;; and
(def s-id (s/or :uuid uuid? :string string? :keyword keyword? :num number?))
(s/def ::id s-id)
#2019-12-2717:18Alexis VincentDoes s/def do some macro magic?#2019-12-2717:18Alexis VincentThis is on the latest spec2 btw#2019-12-2717:22seancorfields/def converts the symbolic form to an actual Spec expression and then registers the spec name. So in (s/def ::id s-id) you do not have a symbolic form of a spec. It probably should give an error.#2019-12-2717:23Alexis Vincentthanks! so would I then rather do (s/def ::id (s/spec s-id)) or something similar?#2019-12-2717:33Alex Miller (Clojure team)Use s/register instead of s/def#2019-12-2717:33Alexis Vincentawesome! thanks!#2019-12-2717:33Alex Miller (Clojure team)s/register is a function and takes a spec object #2019-12-2717:36Alexis VincentIf I have an s/cat expression but would like to generate/validate vectors not arbitry seqā€™s, how would I express it?#2019-12-2717:37Alexis Vincent
(s/def :order/pair
  (s/with-gen
    (s/cat :base ::asset :counter ::asset)
    #(vec
      (gen/generate
       (s/gen
        (s/cat :base ::asset :counter ::asset))))))
This worksish. But obviously is not ideal
#2019-12-2717:42Alex Miller (Clojure team)If youā€™re on spec 2, you can use the new s/catv #2019-12-2717:43Alex Miller (Clojure team)Itā€™s difficult to do easily on spec 1#2019-12-2717:43Alexis VincentPerfect, thanks šŸ™‚#2019-12-2717:55seancorfield@alexmiller Should that (s/def ::id s-id) give an error? Or is it valid but just does something unexpected?#2019-12-2718:05Alex Miller (Clojure team)Prob error but Iā€™ll look at it next week#2019-12-2718:20Alexis VincentThis is probably the same issue as before, but, are these not equiv in spec 2? The first one works nicely, but the second one blows up when generating maps from schemas that contain these keys, but not when generating them directly
(s/def :order/id (s/or :uuid uuid? :string string? :keyword keyword? :num number?))
;; vs 
(s/def ::id (s/or :uuid uuid? :string string? :keyword keyword? :num number?))
(s/def :order/id ::id)
#2019-12-2718:33seancorfield@mail024 That should probably work but there are bugs in Spec 2 that affect aliasing of specs like that, I believe.#2019-12-2720:17Alexis Vincent@seancorfield ok thanks!#2019-12-2721:25Alexis VincentIf I have a schema with a qualified key Iā€™m trying to override inline, it doesnt seem to want to use my override. It just picks the spec from the registry
(s/union
   (s/schema {:event/type #{:event.type/snapshot}})
   (s/schema
    [:event/nanotime]))
Since event/type already exists in the registry and since (I assume) it is qualified. It picks the registry definition not the one iā€™m declaring inline
#2019-12-2721:31seancorfieldYou can only use the inline specs for unqualified keys.#2019-12-2721:32seancorfieldQualified keys are intended to be globally unique so overriding them doesn't make sense in that context (since their meaning is supposed to be globally fixed).#2019-12-2721:33Alexis Vincentfine. Although in this case itā€™s the generator I want to override. Would s/with-gen work for this?#2019-12-2721:33seancorfieldYes, possibly. Depending on exactly what you are trying to do.#2019-12-2721:34Alexis VincentI have a set of valid event types. Defined as a spec set. I want to define a particular message. Which is some schema, alongside a specific item from the set#2019-12-2721:34seancorfieldIf you want the keys in a hash map to "depend on" the value of a particular key, it sounds like you want multi-specs#2019-12-2721:35seancorfieldIf you define this as a multi-spec, it should generate correctly automatically.#2019-12-2721:35Alexis VincentThanks šŸ™‚ Ill check them out.#2019-12-2721:51Alexis VincentPerfect thanks. Working well#2020-12-2907:17Ho0manHi, everyone is using spec/assert in production a good practice ? I want to produce detailed exceptions like spec/assert does for mismatches but do not want to allow some erroneous code to be able to disable the asserts by calling spec/check-asserts . Am I getting something wrong ? Thanks a alot#2020-12-2908:25vlaaadI use spec/assert in dev and custom function which is a mostly a valid? + throw in production#2020-12-2911:01Ho0manHi @vlaaad but this way I won't have descriptive exceptions like those thrown by spec/assert. Am I right ?#2020-12-2911:13vlaaad@ho0man that's configurable by you. when I throw, I store spec/explain-data as error data and use spec/explain-str as error message ā€” all the bits are there#2020-12-2911:18Ho0manThanks a lot @vlaaad, I didnā€™t know about them Thanks#2020-12-2914:34unbalancedCurious if there's a way in spec to do the following:
{:a (s/coll-of int?)
 :b (s/coll-of int? :count <3 times (count (:a this))>)}
or for that matter is it even possible to specify multiples instead of :count in coll-of?
#2020-12-2914:36unbalanced(need 3 coordinates per vertex ID, ideally, is what I'm going for)#2020-12-2915:38codonnell@goomba Any predicate function is a spec, so you could use s/and to combine a spec like you typed above (but without the bit about counts) with a predicate function checking that the counts are valid. If you want generation to work, you'll likely want to provide a custom generator that does something like generate a map with the proper types and then truncate things to make the counts work out.#2020-12-2915:45unbalancedahhh okay thank you! I'll look into this.#2020-01-0203:16EddieI know that you can pull the :ret and :arg spec from a function spec.
(def my-fn-spec
    (s/fspec :args (s/cat :x int?)
             :ret int?))

(s/valid? (:args my-fn-spec) [5])  ;; true
(s/valid? (:ret my-fn-spec) 5)     ;; true
Is there a way to decompose/query other kinds of specs? For example, I would like to do something like the following to get the spec for the individual fn argument named x.
(:x (:args my-fn-spec))
;; or maybe
(first (:args my-fn-spec))
#2020-01-0203:20seancorfieldI think the TL;DR is not easily with Spec 1 @erp12 but Spec 2 offers more facilities for taking specs apart and programmatically building them.#2020-01-0203:25seancorfieldIn Spec 1, you can get the form of a Spec and break it apart, but it isn't easy to turn that back into Spec objects that you can use tho'...#2020-01-0203:42Eddie@seancorfield Good to know, thank you! Based on that, would you agree that currently the best option would be to keep the sub-specs in a map and use some utility functions to "materialize" real specs from them. Just spitballin' here but something like ...
(s/def ::spec (s/spec s/spec?))

; Deconstructed Function Spec
(s/def ::arg-specs (s/coll-of ::spec))
(s/def ::ret-spec ::spec)
(s/def ::d-fn-spec (s/keys :req [::arg-specs ::ret-spec]))

; Deconstructed Collection Spec
(s/def ::coll-kind ::spec)
(s/def ::element-spec ::spec)
(s/def ::d-coll-spec (s/keys :req [::coll-kind ::element-spec]))

; Deconstructed Map Spec
(s/def ::key-spec ::spec)
(s/def ::value-spec ::spec)
(s/def ::d-map-spec (s/keys :req [::key-spec ::value-spec]))

(defn construct-spec 
  [m] 
  ...)
#2020-01-0203:43EddieOr do you know of any other pattens followed by the community for stuff like this?#2020-01-0203:59seancorfieldI don't really understand what problem you are trying to solve here... It doesn't look like the sort of thing I've seen anyone trying to do with Spec.#2020-01-0204:00seancorfieldHave you looked at Spec 2? That's much more amenable to programmatic manipulation of specs...#2020-01-0220:10rafaelHi. I'm struggling with generating data from a simple (s/schema) use case.
(s/def ::x int?)
  (s/def ::baz (s/schema [::x]))
  (s/def ::bar ::baz)
  (s/def ::foo (s/schema [::bar]))
  (gen/sample (s/gen (s/spec ::foo)))
#2020-01-0220:10rafaelThe call to (gen/sample) throws a No implementation of method: :conform* of protocol: #'clojure.alpha.spec.protocols/Spec found for class: clojure.lang.Keyword exception.#2020-01-0220:12rafaelWhile a straightforward translation to s/keys seems to work fine:
(s/def ::x int?)
  (s/def ::baz (s/keys :opt [::x]))
  (s/def ::bar ::baz)
  (s/def ::foo (s/keys :opt [::bar]))

  (gen/sample (s/gen (s/spec ::foo)))
I'm probably getting something wrong in my schema definitions, but I can't figure out the problem.
#2020-01-0220:32Alex Miller (Clojure team)code looks fine, prob just a bug#2020-01-0220:32Alex Miller (Clojure team)in spec#2020-01-0220:38rafaelCool, I'll stick with the (s/keys ..) version for a while.#2020-01-0220:43rafaelChanging the :bar definition to
(s/register ::bar (s/resolve-spec ::baz))
appears to work around the issue.
#2020-01-0220:48Alex Miller (Clojure team)that makes sense - you're basically copying the spec object rather than relying on resolving through the alias#2020-01-0220:48Alex Miller (Clojure team)I have a pretty good hunch on where that bug is#2020-01-0220:57rafaelAwesome, thanks!#2020-01-0303:42Eddie@seancorfield I spent some time today with spec 2. It seems like it's design goals are exactly what I need. I attempted to migrate and it was pretty rough (not complaining, I totally understand that it is still in development and also a new major version so I expected many breaking changes). I am not sure if we will be jumping on board with spec 2 yet, but thank you for the pointer!#2020-01-0303:46EddieThere are a few things that we are doing in spec 1 that I cannot figure out how to recreate in spec 2, even though it smells like it should be possible. For example, I would like to implement a custom generator that simply pulls a random element from a set. I can get this to work if the set satisfies (every? constant-val? ...) but otherwise, I cannot.#2020-01-0303:53Alex Miller (Clojure team)are you using a set spec?#2020-01-0303:56Alex Miller (Clojure team)
user=> (s/def ::s #{1 2 3})
:user/s
user=> (gen/sample (s/gen ::s))
(2 1 3 2 2 1 1 1 2 2)
#2020-01-0303:58Alex Miller (Clojure team)if you're trying to do this outside s/def (which has some magic), you can do something like
user=> (gen/sample (s/gen (s/resolve-spec #{1 2 3})))
(1 2 1 1 3 1 3 3 1 1)
#2020-01-0304:00Alex Miller (Clojure team)one of the caveats you might be seeing is that set specs are constrained to constant values only which is part of the new symbolic spec direction#2020-01-0304:01Alex Miller (Clojure team)there are a couple of options but you can just use the built-in generators directly if you have something else#2020-01-0317:08EddieThank you! gen/elements is exactly what I was looking for. However I am running into some strange behavior on the with-gen .
user=> (s/with-gen (s/spec int?)
                   (fn [] (gen/elements [-1 0 1])))
Execution error (IllegalArgumentException) at clojure.alpha.spec.protocols/eval190$fn$G (protocols.clj:11).
No implementation of method: :with-gen* of protocol: #'clojure.alpha.spec.protocols/Spec found for class: clojure.lang.PersistentList

user=> (s/exercise (s/with-gen int? (fn [] (gen/elements [-1 0 1])))
            10)
([-1 -1] [0 0] [1 1] [-1 -1] [-1 -1] [-1 -1] [0 0] [1 1] [-1 -1] [1 1])
Neither of those seem right to me. Am I missing a change from Spec 1?
#2020-01-0317:14Alex Miller (Clojure team)yeah, seems fishy. I'd expect the first one to work#2020-01-0304:02Alex Miller (Clojure team)
user=> (def a 1) (def b 2)
#'user/a
#'user/b
user=> (gen/sample (gen/elements [a b]))
(1 2 2 2 1 2 2 2 2 2)
#2020-01-0316:40A.J. Gardnerhello! it seems like @rafael and I ran into the same issue. I posted about it in the google group: https://groups.google.com/d/topic/clojure/rcuWmqyGWzs/discussion my specs:
(s/def ::id int?)
(s/def ::tag-id ::id)
(s/def ::child-tag ::tag-id)
(s/conform ::child-tag 22)
;; 22
(s/conform ::child-tag "a")
;; :clojure.alpha.spec/invalid
(s/conform (s/schema [::child-tag]) {::child-tag 22})
;; Execution error (IllegalArgumentException) at clojure.alpha.spec.protocols/eval1458$fn$G (protocols.clj:11).
;; No implementation of method: :conform* of protocol: #'clojure.alpha.spec.protocols/Spec found for class: clojure.lang.Keyword
(s/conform (s/schema [::tag-id]) {::tag-id 22})
;; Execution error (IllegalArgumentException) at clojure.alpha.spec.protocols/eval1458$fn$G (protocols.clj:11).
;; No implementation of method: :conform* of protocol: #'clojure.alpha.spec.protocols/Spec found for class: clojure.lang.Keyword
(s/conform (s/schema [::id]) {::id 22})
;; #:foo{:id 22}
#2020-01-0316:44A.J. Gardnerand the stack trace, just in case thatā€™s useful:
(clojure.repl/pst)
IllegalArgumentException No implementation of method: :conform* of protocol: #'clojure.alpha.spec.protocols/Spec found for class: clojure.lang.Keyword
	clojure.core/-cache-protocol-fn (core_deftype.clj:583)
	clojure.core/-cache-protocol-fn (core_deftype.clj:575)
	clojure.alpha.spec.protocols/eval1458/fn--1556/G--1439--1567 (protocols.clj:11)
	clojure.alpha.spec.impl/schema-impl/reify--2753 (impl.clj:435)
	clojure.alpha.spec/conform (spec.clj:245)
	clojure.alpha.spec/conform (spec.clj:237)
	clojure.alpha.spec/conform (spec.clj:241)
	clojure.alpha.spec/conform (spec.clj:237)
	foo/eval6028 (form-init13722384443750232197.clj:1)
	foo/eval6028 (form-init13722384443750232197.clj:1)
	clojure.lang.Compiler.eval (Compiler.java:7177)
	clojure.lang.Compiler.eval (Compiler.java:7132)
#2020-01-0316:50Alex Miller (Clojure team)yeah, this is just a bug in spec#2020-01-0317:10A.J. Gardnerwhew#2020-01-0615:52borkdudeI'm in the process of writing a version of spec(1) that can be used with babashka. For now it's an interpreted lib:
$ export BABASHKA_CLASSPATH="$(clojure -Sdeps '{:deps {spartan.spec {:git/url "" :sha "104129aae9eab6dd4622937d0f46abeed9c9c537"}}}' -Spath)"
$ bb -e "(require '[spartan.spec :as s]) (s/explain (s/cat :i int? :j string?) [\"foo\" 1])"
"foo" - failed: int? in: [0] at: [:i]
I'm unsure what parts I could be adding to babashka as a built-in. Are there any assurances what parts of the API will be unchanged, and what the future namespace of spec will be ?
#2020-01-0706:15Jakub HolĆ½ (HolyJak)Why not spec 2? Though not finished, I guess it bears completion (just the last 10% that use 90% of the time :rolling_on_the_floor_laughing:)...#2020-01-0615:52borkdudeIf I have some more assurances, I could adopt parts of this lib as built-ins which would make it much faster.#2020-01-0615:55Alex Miller (Clojure team)I can assure you that the future namespace will be different :) The current spec 2 is clojure.alpha.spec, clojure.alpha.spec.gen, etc. We're expecting this to eventually be clojure.spec, clojure.spec.gen#2020-01-0615:56Alex Miller (Clojure team)most of the existing spec forms will probably be the same, with some additions and some fine details changing around a few things (and deprecation/removal of s/keys)#2020-01-0615:57Alex Miller (Clojure team)most of the spec operations will be the same although we have added arities in all of the conforming ops (valid?, conform, explain, etc) to support spec checking modes#2020-01-0616:04borkdudethanks#2020-01-0617:51seancorfield@alexmiller Any sense yet of whether there will a migration path/tooling at launch of clojure.spec?#2020-01-0617:54Alex Miller (Clojure team)as long as there are no follow-up questions, yes :)#2020-01-0623:27lgesslerhi, I want to use s/fdef with protocol functions. In one namespace, I'm calling fdef and defining and implementing a protocol, but the protocol's methods appear to be unaffected by fdef. If I understand [this issue](https://clojure.atlassian.net/browse/CLJ-2109) right, this is expected behavior. Is that all right, and if so, is there a recommended workaround for speccing protocol functions?#2020-01-0623:33Alex Miller (Clojure team)Thatā€™s correct - fn specs donā€™t work with protocol methods. Only workaround Iā€™d suggest is wrapping the call to the protocol function in a method (which I often do as a matter of practice anyways)#2020-01-0718:39telekidIā€™ve got a spec(ish) design question that Iā€™ve been thinking about for a while. I finally decided to throw it in a gist: https://gist.github.com/telekid/0f276bfa3b3f0d395a7a71158adbb35d Iā€™m curious to know how other people approach this problem?#2020-01-0720:14vemvIn a given (ns models.geography.continent), I tend to solve this with:
(spec/def :models.geography.continent.penguins/animals ...)

(spec/def :models.geography.continent.lions/animals ...)
i.e. I don't use :: syntax for these cases, and I don't create additional Clojure namespaces (files). I call it the "synthetic ns pattern" It's a bit verbose, and losing the 1:1 mapping between cljs namespaces and spec namespaces isn't ideal either. But it's fairly occasional so I've been happy to use these for a couple years
#2020-01-0720:15vemvbtw, not sure if this solution is already described in your gist. sometimes I'm impatient šŸ™ƒ#2020-01-0720:48telekidyeah, I like that#2020-01-0720:49telekidIā€™ve experimented with that a bit in a few places (but I forgot about it when I was writing the gist - thanks for bringing it up)#2020-01-0720:56vemvāœŒļø! I realised, a more accurate nickname would be "synthetic sub-ns pattern". so, while it's not a 1:1 mapping, one is still reasonably close to that ideal#2020-01-0721:08telekidfunnily enough, Iā€™ve been using the same name: https://ask.clojure.org/index.php/2817/lighter-weight-aliasing-for-keywords?show=8918#a8918#2020-01-0718:51Eddie@jake142 If EDIT: Gist has been updated. Disregard! :)#2020-01-0718:51Eddie@jake142 If EDIT: Gist has been updated. Disregard! :)#2020-01-0718:52telekidOh, I just realized that I made a typo#2020-01-0718:52telekidfacepalm#2020-01-0718:52telekidone moment#2020-01-0718:52telekidL20 how has ::animals/lions#2020-01-0718:54EddieI see. Thanks for the clarification.#2020-01-0719:05telekidSorry for the confusion!#2020-01-0721:37kennyWhy were multi-specs implemented using multi-methods? I have never found the dynamism of multimethods useful in defining multi-specs. In fact, I think I'd prefer all of the multi-spec types to be declared statically.#2020-01-0721:44ghadi@kenny so that they're open to extension#2020-01-0721:44ghadiin fact I can't think of anything in clojure that's closed#2020-01-0721:45kennyRight. At least interally, I've never found that useful. All type->keys are declared upfront. Multimethods make it hard to determine the input data.#2020-01-0721:45ghadi> All type->keys are declared upfront. ?#2020-01-0721:46kennyDispatch functions#2020-01-0721:47ghadiI'm sorry I still don't understand#2020-01-0721:49kennyFor a particular entity, I know it has types A, B, C. It can never and should never be extended except directly in the definition.#2020-01-0721:49kennyAll the variants of an entity are known upfront.#2020-01-0721:50ghadiok that's your use case#2020-01-0721:50Joe LaneYou might not be the one defining the multispecs, the users of your library may be.#2020-01-0721:50ghadi^^#2020-01-0721:50ghadihttps://github.com/clojure/tools.deps.alpha/blob/master/src/main/clojure/clojure/tools/deps/alpha/extensions.clj look at t.d.a.extensions ^#2020-01-0721:50ghadiand all the extensions in the extensions folder#2020-01-0721:53kennyOh right, I see the use for libs. This internal use case is much different. Perhaps some sort of new macro is what we need:
(multi-spec2 {:type1 (s/keys :opt [])
              :type2 (s/keys :opt [])})
#2020-01-0722:32kennyFor those interested...
(defmacro closed-multi-spec
  [name dispatch-key dispatch->spec]
  (let [defmethods (map (fn [[dispatch spec-form]]
                          `(defmethod ~name ~dispatch
                             [~'_]
                             ~spec-form))
                        dispatch->spec)]
    `(do
       (defmulti ~name ~dispatch-key)
       
Don't know if this is where it'll land but it's a start.
#2020-01-0721:54ghadiwouldn't it be amazing if you can write that macro today?#2020-01-0721:54ghadišŸ˜„#2020-01-0800:19kennyHas anyone written a version of spec2 select that works with spec1? Deciding to have keys required or optional for all use cases is so painful šŸ˜£#2020-01-0800:45seancorfield@kenny Not sure what you mean by "works with spec1"? The two libraries use different registries so you cannot combine them (at least, not easily)#2020-01-0800:45kennyI mean a macro for spec1 that imitates spec2's select.#2020-01-0800:47kennyWhen using nested maps, spec1 doesn't let you change which keys in the nested map are required.#2020-01-0800:47seancorfieldThat's exactly why Spec2 is coming -- that required/optional thing in Spec1 is painful šŸ™‚#2020-01-0800:47seancorfieldIn Spec1 that is complected and baked into how s/keys works. In Spec2 the set of keys and the requiredness are decomplected.#2020-01-1316:31aviYou know, it just occurred to me ā€”Ā this is going to make it hard to create an interop schema using spec ā€” in other words, to fully describe a data structure that a system expects/requires, including the keys that are required in each mapā€¦ I suppose we might need to write some new plumbing to create a schema based on some spec2 specs, with additional annotation, expectations, etc. šŸ¤”#2020-01-1316:49seancorfieldI'm not sure I follow. Spec 2 still let's you specify which keys are required, including nested keys. It just teases the two concepts apart. #2020-01-1317:20aviHmm for some reason I was thinking that the new schema function was designed solely for the case of defining function specsā€¦ but I havenā€™t looked super closely šŸ˜¬#2020-01-1317:20aviis it schema? or maybe Iā€™m thinking about select ā€”Ā I should just review the docs šŸ™„ sorry#2020-01-1317:43seancorfieldschema defines the possible set of keys. select defines the required subset of a schema.#2020-01-1317:44seancorfieldHence, decomplecting specifying requiredness from specifying possible šŸ™‚#2020-01-1317:44aviSounds pretty exciting, honestly!#2020-01-1317:45aviI just did a walkthrough of a pretty involved schema using spec1 with some members of my team who arenā€™t familiar with it, and they asked a bunch of questions that I answered with ā€œnot right now, but thatā€™s coming in spec2ā€#2020-01-1318:10seancorfieldYup, I'm looking forward to Spec2 becoming non-alpha and the "standard" way to work with Spec.#2020-01-1318:12seancorfieldFor a while, I had a branch at work tracking Spec2 but with the number of bugs and changes as it has evolved, it was proving a bit much trying to keep our codebase current on it... so I gave up a while back and we'll take another look once it is stable. I doubt we'll migrate, but we'll probably start using Spec2 for new code (and may convert old code over as we maintain it over time).#2020-01-0800:48kennyYa. Was hoping for a workaround for now. It seems potentially possible to get it to work.#2020-01-0800:49seancorfieldI don't know if s/merge will let you override an all-optional s/keys with a subset of keys that are now marked required. Doesn't help you much with nesting I suspect.#2020-01-0800:50seancorfieldYou could just start using Spec2 I suppose šŸ™‚ But not for any production work as it's still changing a lot.#2020-01-0800:53kennyIt's gotta be production worthy unfortunately#2020-01-1001:52telekidWe ran into a question about how to spec multimethods while maintaining their open nature. There wasnā€™t much good documentation on this topic floating around the internet, so I went down the rabbit hole a bit and came back up with this: https://gist.github.com/telekid/f2e588718dbdfe48306d64e5388bdc15#2020-01-1411:57mpingis there a way to spec a map according to its key? example: if key is a number, value should be a number, if key is something else then value should be a string#2020-01-1413:34Alex Miller (Clojure team)Yes, although itā€™s a little complicated#2020-01-1413:36Alex Miller (Clojure team)The structure is to spec it as a s/coll-of s/tuples, where each tuple is an s/or of whatever key/value is allowed#2020-01-1413:36Alex Miller (Clojure team)The coll-of should also have an :into {}#2020-01-1413:47favilasomething like
(s/coll-of (s/or :num-val (s/tuple number? number?)
                 :string-val (s/tuple (s/and any? (complement number?)) string?))
           :into {})
#2020-01-1413:57Alex Miller (Clojure team)yep#2020-01-1416:59mpinggot it, tks!#2020-01-1417:03colinkahnI ran across something unexpected when using spec alpha v1:
(s/def ::foo number?)
(s/def ::bar-ret ::foo)
(gen/generate (s/gen ::bar-ret {::bar-ret #(gen/return 100)}))
;; => -2123123
this seems to be because when overrides are applied it determines the key to pull from the overrides map using something like this:
(let [s (@#'s/specize ::bar-ret)]
  (@#'s/spec-name s))
;; => ::foo
and like that shows ::bar-ret returns ::foo. My question is whether this is by design or perhaps a bug, and is it too hacky using something like (s/def ::bar-ret (s/spec ::foo)) to get this to work?
#2020-01-1417:41Alex Miller (Clojure team)this is a known bug (there's a ticket for it) and we plan to fix in spec 2#2020-01-1417:41Alex Miller (Clojure team)your workaround is fine#2020-01-1417:41colinkahnAwesome, thanks!#2020-01-1518:54nullptri have a dumb spec question that unfortunately is in the "difficult to google" category - say i had a sequence with the following structure: [1 "a" 2 "b" "c" 0 3 "d" "e" "f"] the pattern here is a number which indicates how many following items there are directly in the sequence -- i adjusted spacing above to hopefully make that clearer this is of crouse trivial to express if the following items are in another sequence, but scanning the docs i can't see a way to spec the flat version of this -- note that this isn't motivated by any important use case, more just part of me trying to learn spec so feel free to treat this as pure curiosity#2020-01-1519:16Alex Miller (Clojure team)regex specs combine to spec one flat sequence#2020-01-1519:20Alex Miller (Clojure team)so you could do something like (s/* (s/& (s/cat :c nat-int? :s (s/* string?)) #(= (:c %) (count (:s %)))))#2020-01-1519:23nullptrthanks! that is extremely straightforward -- i feel like i had something similar but surely missed something basic and made a wrong assumption. will compare and learn -- thanks again.#2020-01-1519:24Alex Miller (Clojure team)it's very important to use s/&, not s/and there#2020-01-1519:24Alex Miller (Clojure team)s/and is not a regex spec and will expect a new nested collection boundary#2020-01-1519:24nullptryeah i had the & but i think i had a surrounding cat there that was throwing it all off#2020-01-1714:33yuhanIs there any way of relating a neighboring spec's value into the :count option of s/every ?#2020-01-1714:34yuhaneg. I have a map spec with keys :width and :things#2020-01-1714:35yuhanand I want to ensure that things has width number of elements#2020-01-1714:35Alex Miller (Clojure team)you need to s/and with a predicate that can do that at the containing level#2020-01-1714:36yuhanokay, I have something like this
(s/def :ctx/region
  (s/and (s/keys :req [:ctx/width
                       :ctx/slots])
    #(= (count (:ctx/slots %))
        (:ctx/width %))))
#2020-01-1714:37Alex Miller (Clojure team)yep#2020-01-1714:37yuhanso I'll have to write a custom generator if I want to gen examples of the spec?#2020-01-1715:01Alex Miller (Clojure team)yes#2020-01-1715:03Alex Miller (Clojure team)this is true of any data with internal constraints like this. the general strategy is to use gen the width first, then use a combination of fmap and bind to generate the slots and assemble into the map#2020-01-1715:04Alex Miller (Clojure team)you might also ask what the width is buying you in this data structure when it is the same as the count of the slots#2020-01-1715:04Alex Miller (Clojure team)(and as an aside, I find that pushing on data that is hard to spec often improves the shape of the data and the code that uses it)#2020-01-1715:15yuhanMy idea was to have a degree of "memoization" or redundancy baked into the data structure - in this case slots may be a lazy sequence that is generated on demand, so I don't want to keep calling count on it unnecessarily#2020-01-1715:16yuhanof course this introduces the risk of width getting out of sync with the data, which is why I'm using spec to assert the constraints during dev time#2020-01-1715:18Alex Miller (Clojure team)fair enough#2020-01-1715:19yuhanthere are a few other "derived" keys similar to this that I'm storing in the same data structure, just wondering if this isn't considered a code smell?#2020-01-1715:22Alex Miller (Clojure team)it really depends#2020-01-1715:31yuhanas with so many things šŸ™‚#2020-01-1715:33yuhanHere's my best attempt at that generator:
(s/def :ctx/width (s/int-in 1 11))
(s/def :ctx/slot string?)
(s/def :ctx/slots (s/coll-of :ctx/slot))
(s/def  :ctx/region
  (s/and (s/keys :req [:ctx/width
                       :ctx/slots]
           :gen #(let [w (gen/generate (s/gen :ctx/width))]
                   (gen/hash-map
                     :ctx/width (s/gen #{w})
                     :ctx/slots (s/gen (s/coll-of :ctx/slot
                                         :count w)))))
    #(= (count (:ctx/slots %))
        (:ctx/width %))))
#2020-01-1715:51Alex Miller (Clojure team)you don't want to use gen/generate in there - that will foil the test.check shrinking mechanisms#2020-01-1715:52Alex Miller (Clojure team)instead, use gen/bind with (s/gen :ctx/width) and the gen/hashmap you have#2020-01-1715:52Alex Miller (Clojure team)gen/bind lets you make a generator on a generator#2020-01-1718:06yuhangot it, thanks so much for the help!
#(gen/bind (s/gen :ctx/width)
   (fn [w]
     (gen/hash-map
       :ctx/width (gen/return w)
       :ctx/slots (s/gen (s/coll-of :ctx/slot :count w)))))
#2020-01-1722:32zaneLetā€™s say I have a Datomic query and I want to find all the variables defined in that query. Would spec be a good tool to bring to bear on that problem?#2020-01-1722:50seancorfield@zane I'm not saying it couldn't be done but I certainly wouldn't expect to use Spec for such a problem.#2020-01-1722:51seancorfieldEven if you wrote a complete Spec of Datomic queries, if you s/conform it so it "identifies" the ? variables, you'd still have to walk the resulting data structure to extract them all -- you might just as well walk the original Datomic query.#2020-01-1723:15zaneYeah, I had imagined doing something clever with s/conform and clojure.walk.#2020-01-1723:15zaneBut I see what you mean.#2020-01-1722:57Alex Miller (Clojure team)I believe there are specs of Datomic's query syntax out there btw, don't have any links handy (but I agree that I probably wouldn't pick that as the first approach)#2020-01-1723:15zaneGood to know!#2020-01-1723:20zaneIs it fair to say that conform is not really suited to these kinds of problems in the general case?#2020-01-1723:22seancorfieldconform is not designed for parsing or general transformation, if that's what you're asking?#2020-01-1723:23seancorfieldconform is intended to "tag" the resulting data with "how" the data conformed to the Spec, so downstream code can behave accordingly.#2020-01-1723:36zaneIsnā€™t that kind of tagging exactly what Iā€™m after in this case? Iā€™m not sure Iā€™m understanding the details here.#2020-01-1723:56seancorfieldYou'd still have to walk the result to find the tagged values tho#2020-01-1800:03zaneTrue.#2020-01-1800:04zaneI guess Iā€™m still struggling with which kinds of ā€œparsingā€ are appropriate to do with spec and which arenā€™t.#2020-01-1800:05zaneDifferentiating between different shapes of data seems like something you could achieve with it via s/or and s/conform.#2020-01-1800:05zaneBut you could also do that kind of thing with match, or with plain old Clojure.#2020-01-1723:24seancorfieldThat's not to say that some people don't (ab)use Spec to do some amount of parsing and transformation... cough ...but that's not what it was designed for: coercion is somewhat of a "side-effect" of conforming, you might say šŸ™‚#2020-01-1723:34zaneThat makes sense.#2020-01-1723:34zaneI find myself wanting something like Instaparse, but for EDN rather than strings, often.#2020-01-1814:45dharriganIs there an example of how I might use spec to ensure that a field is in OffsetDateTime format?#2020-01-1815:07Alex Miller (Clojure team)#(instance? OffsetDateTime %) ?#2020-01-1815:10dharriganoooh#2020-01-1815:10dharriganwill try that#2020-01-1815:20dharriganSorta gets me there, the field is a string, so I think I'll have to do some extra magic#2020-01-1815:33Alex Miller (Clojure team)probably need to use a parser then#2020-01-1815:34Alex Miller (Clojure team)which is part of the java time library#2020-01-1815:34dharrigankk#2020-01-1815:34dharriganthanks for the pointer šŸ™‚#2020-01-1913:01Philipp SiegmantelHello Everybody, what test runner/library do you use with clojure.spec.test.alpha/check? I found https://gist.github.com/jmglov/30571ede32d34208d77bebe51bb64f29 for integration with clojure.test but the errors it prints aren't verry informative. Is there something better?#2020-01-1915:03dominicmClojure core uses something custom#2020-01-1916:07Philipp SiegmantelDo you have a link?#2020-01-1917:58respatializedhas anyone written an unofficial spec for EDN? I'm looking for a programmatic way of distinguishing between Clojure forms and plain EDN forms.#2020-01-1918:45seancorfield@afoltzm This is the definitive word on EDN I believe https://github.com/edn-format/edn#2020-01-1918:53respatializedyeah, I figured I'd probably have to create one myself on the basis of the format's description that suits my purposes, which also serves as a useful exercise for improving my understanding of spec itself.#2020-01-1921:57andy.fingerhutClojure core has an implementation of reading arbitrary Clojure forms, via clojure.core/read, and the EDN subset, via clojure.edn/read#2020-01-1921:58andy.fingerhutThose implementations in Clojure itself are written in Java. The tools.reader contrib library also has implementations written in Clojure.#2020-01-2101:14Vincent CantinIn spec2, is it possible to use s/conform on a vector parsed via a sequence matcher like vcat and then getting back a vector via s/unform?#2020-01-2101:16Vincent CantinFrom the documentation, it does not look like it would work.#2020-01-2102:55Alex Miller (Clojure team)Yes, it works now#2020-01-2102:56Alex Miller (Clojure team)With s/catv#2020-01-2102:59Alex Miller (Clojure team)What documentation are you referring to?#2020-01-2113:05Vincent Cantin@alexmiller I had this impression after reading those 2 places: ā€¢ https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha#scat which hints that vcat uses s/and- , ā€¢ and https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha#nonflowing-sand--new which does not say that s/unform validates other preds in a s/and-.#2020-01-2113:09Vincent CantinThe example about s/vcat may potentially confuse people w.r.t. s/catv .#2020-01-2113:24Alex Miller (Clojure team)Thx, Iā€™ll update that #2020-01-2115:32Filipe Silvaheya#2020-01-2115:32Filipe Silvaare there any breaking changes one should be aware of between 0.1x and 0.2.x for spec?#2020-01-2115:33Filipe SilvaI'm updating a project's cljs 1.10.520 toĀ `1.10.597` , and my production build now fails when loading something spec related#2020-01-2115:33Filipe Silva
| TypeError: Cannot read property 'prototype' of ...
                                                V
$APP.$cljs$spec$alpha$t_cljs$0spec$0alpha62946$$.prototype.$cljs$spec$alpha$Spec$gen_STAR_$arity$4$ = $JSCompiler_unstubMethod$$(6, function($_$jscomp$256$$, $overrides$jscomp$11$$, $path$jscomp$62$$, $rmap$jscomp$11$$) {
  return $APP.$cljs$core$truth_$$(this.$gfn$) ? this.$gfn$.$cljs$core$IFn$_invoke$arity$0$ ? this.$gfn$.$cljs$core$IFn$_invoke$arity$0$() : this.$gfn$.call(null) : $APP.$cljs$spec$alpha$re_gen$$(this.$re$, $overrides$jscomp$11$$, $path$jscomp$62$$, $rmap$jscomp$11$$, $APP.$cljs$spec$alpha$op_describe$$(this.$re$));
});
#2020-01-2115:34Filipe Silvadoesn't seem to affect the normal build though#2020-01-2116:49GHave folks ever used spec to define a transformation between namespaced, clojure maps to another format? My use case is creating maps in clojure but transforming them to JSON, and I don't want for there to be coupling between my clojure key names (ie. namespaced keywords) and my JSON maps (which may even contain keys that aren't representable as clojure keywords)#2020-01-2121:17rapskalian@U3G51H7NY you can have a look at this little lib I wrote a while ago for a similar use case https://github.com/cjsauer/disqualified#2020-01-2121:18rapskalianMight not be exactly what youā€™re needing but could serve as a possible reference #2020-01-2121:18rapskalianItā€™s less than 50 lines of code#2020-01-2121:23Git's not exactly what I'm looking for right now (for instance, my un-namespace keys have conflicting names, although the full paths are distinct), but this is definitely useful! I've had to implement something very similar for another project, so will keep this in mind in the future šŸ™‚#2020-01-2116:53GI was hoping to have the resulting key name as part of the spec so as to keep all of the relevant information about that attribute in one place, but perhaps this is best done outside of spec?#2020-01-2116:55ghadiyou can store a lookup from qualifiedkey -> key in a sibling database, doesn't have to be in the spec registry#2020-01-2216:31kennyI really like how succinct spec-tools data-specs are. It takes 17 lines worth of specs for a map that has a key that has a collection of maps down to 3 lines. Will Spec2 be able to do this as well? The syntax described here https://github.com/clojure/spec-alpha2/wiki/Schema-and-select#unqualified-keys looks quite similar. The description makes it sound like that syntax will only be available for unqualified keys though.#2020-01-2216:32Alex Miller (Clojure team)Thatā€™s correct#2020-01-2216:32kennyWhy not support qualified keys with that syntax?#2020-01-2216:35kenny
(s/def ::error-code string?)

(s/def ::error-message string?)

(s/def ::id uuid?)

(s/def ::failed-request
  (s/keys :req [::error-code ::error-message ::id]))

(s/def ::failed-requests
  (s/coll-of ::failed-request))

(s/def ::result
  (s/keys :req [::failed-requests]))
versus
(s/def ::result
  {::failed-requests [{::error-code    string?
                       ::error-message string?
                       ::id            uuid?}]})
The latter seems so much easier to read.
#2020-01-2216:36Alex Miller (Clojure team)https://clojure.org/about/spec#_global_namespaced_names_are_more_important#2020-01-2216:37kennySorry, I don't follow. The latter could easily expand to do exactly as that doc describes.#2020-01-2216:37kennyWell, I suppose the nested map wouldn't have a name.#2020-01-2216:37Alex Miller (Clojure team)the objective here is not just to validate but to build a library of reusable (ie named + registered) specifications#2020-01-2216:42kennySo Spec1 forces you to do that with unqualified keys. Why has that changed with Spec2?#2020-01-2216:43Alex Miller (Clojure team)not sure I understand the question#2020-01-2216:44kennyIf I'm understanding the syntax here https://github.com/clojure/spec-alpha2/wiki/Schema-and-select#unqualified-keys, you are no longer creating a reusable specification for the nested map.#2020-01-2216:46kennyWell, that I guess that applies for key & value in that ::order spec. Nothing is reusable.#2020-01-2216:46Alex Miller (Clojure team)well, you are in the schema, but not in the sub levels. as unqualified names, they often are ambiguous#2020-01-2216:47Alex Miller (Clojure team)you still can do like you did in spec 1 (relate unqualified keys to qualified specs) but it's not constrained to the unqualified name match and it's optional#2020-01-2216:51kennyTo be clear, the issue with something like this
(s/def ::result
  {::failed-requests [{::error-code    string?
                       ::error-message string?
                       ::id            uuid?}]})
Is that the nested map is not named?
#2020-01-2216:52Alex Miller (Clojure team)you're inventing syntax here, it can't be just considered in isolation#2020-01-2216:53kennyFair enough. I mean like this
(s/def ::order 
  (s/schema {:purchaser string? 
             :due-date inst?
             :line-items (s/coll-of (s/schema {:item-id pos-int? 
                                               :quantity nat-int?})
                           :kind vector?
                           :min-count 1
                           :gen-max 3)}))
but with every key qualified instead of unqualified.
#2020-01-2216:53Alex Miller (Clojure team)this now occupies the "map" slot in the language of specs, which may have other uses#2020-01-2216:55Alex Miller (Clojure team)certainly, this goes against the grain of naming and registering the specifications. names have multiple utility in spec - they are used for spec reuse, but also in things like explain reporting, and I don't remember what else#2020-01-2216:56Alex Miller (Clojure team)none of this rules out the possibility of adding more support for concise definition of multiple specs in tandem#2020-01-2216:56Alex Miller (Clojure team)or even extension to support this in the future, but it's not our top concern#2020-01-2216:57Alex Miller (Clojure team)and if we were going to do so, it would be driven by a top level problem, not just "it's a thing we can do".#2020-01-2216:58kennyUnderstood. I don't think it necessarily goes "against the grain of naming and registering".
(s/def ::order
  (s/schema {::purchaser  string?
             ::due-date   inst?}))
That would register the ::purchaser spec as string? , ::due-date as inst? and ::order as "s/keys".
#2020-01-2216:38Filipe Silva@alexmiller heya, if you're around can I ping you about https://clojurians.slack.com/archives/C1B1BB2Q3/p1579620754050400?#2020-01-2216:39Alex Miller (Clojure team)what do you mean by 0.1.x and 0.2.x?#2020-01-2216:39Alex Miller (Clojure team)are you talking specifically about spec.alpha?#2020-01-2216:40Alex Miller (Clojure team)https://github.com/clojure/spec.alpha/blob/master/CHANGES.md#2020-01-2216:40Alex Miller (Clojure team)0.1.x is Clojure 1.9 era and 0.2.x is Clojure 1.10 era#2020-01-2216:41Alex Miller (Clojure team)I don't think there were any breaking changes between those#2020-01-2216:42Filipe Silvayes that is what I meant#2020-01-2216:43Filipe Silvatrying to update from cljs 1.10.520 toĀ `1.10.597` causes my app to have a runtime error for optimized builds on what seems to be something spec related, so I thought there might be a breaking change#2020-01-2216:43Alex Miller (Clojure team)what are 1.10.520 and 1.10.597 applicable to?#2020-01-2216:44Alex Miller (Clojure team)oh cljs, sorry that got lost#2020-01-2216:44Alex Miller (Clojure team)spec.alpha is CLJ only#2020-01-2216:44Alex Miller (Clojure team)the cljs spec impl is inside ClojureScript so the stuff above does not directly correlate#2020-01-2216:45Filipe Silvaah I see... so maybe I should ask about cljs.spec.alpha in #clojurescript instead#2020-01-2216:47Alex Miller (Clojure team)yeah#2020-01-2216:47Alex Miller (Clojure team)I don't know anything on that#2020-01-2216:48Filipe Silvacoolio, thanks for the direction!#2020-01-2417:57lilactownfor direction/discussion w.r.t. cljs.spec.alpha, should I lobby that here or in #cljs-dev?#2020-01-2418:11Alex Miller (Clojure team)cljs-dev#2020-01-2417:58lilactownIā€™m interested in exploring (and advocating for support for) the capability in CLJS to remove specs when building for deployment#2020-01-2418:05Matti UusitaloPut them in a separate source folder which isn't part of the production build?#2020-01-2418:09lilactownthey need to show up in the namespace dependency graph of the application in order to be included at dev time and be rebuilt when the implementations change#2020-01-2418:09lilactownI could manually write a namespace that includes all specs I use in my app, and add that as a preload. but that is tedious#2020-01-2612:11vemvHow about a my.spec/def macro which expands to sth like when js/goog.asserts.ENABLE_ASSERTS (spec/def ...)?#2020-01-2721:19favilais there any advice for speccing containers, e.g. atoms or delays?#2020-01-2816:38vemvSay you have a check! function. I normally hook up the :validator to it, but only on *assert*#2020-01-2817:05favilaI went with an fspec instead#2020-01-2817:05favilaMost ref types have validators, I forgot about those. you could use s/assert in there#2020-01-2817:06faviladoesnā€™t help with generation, but thereā€™s manual intervention for generation all the time anyway#2020-01-2817:06faviladelays however donā€™t have validators#2020-01-2817:18vemv> doesnā€™t help with generation, yeah it's a tradeoff :) > delays however donā€™t have validators yup, as you may know though a custom IDeref impl can be quite thin#2020-01-2721:19favilaIā€™d like to say ā€œtakes a delay that returns a thing satisfying some predicate when derefedā€#2020-01-2721:42seancorfield@favila There's no way to do that. You can spec around the code that processes what's inside the container, but you can't spec the container itself.#2020-01-2721:44lilactownatoms do have validator functions that you can pass in on instantiation I think?#2020-01-2721:49Alex Miller (Clojure team)they do#2020-01-2721:49Alex Miller (Clojure team)but carefully consider the costs there#2020-01-2902:55agI need a spec for a map like {:a :b} where :b has to be a required key, but only when value of :a is not nil. Can someone help me with that?#2020-01-2903:27seancorfield
user=> (s/def ::ab (s/and (s/keys :opt-un [::a ::b]) #(or (nil? (:a %)) (contains? % :b))))
:user/ab
user=> (s/valid? ::ab {:x 1})
true
user=> (s/valid? ::ab {:a nil})
true
user=> (s/valid? ::ab {:a 1})
false
user=> (s/valid? ::ab {:a 1 :b 2})
true
@ag how about that?
#2020-01-2903:28seancorfieldIf :a should be required, use (s/keys :req-un [::a] :opt-un [::b]) I guess.#2020-01-2903:33agah, I was playing around, come up with something like this:
(s/def ::a string?)
(s/def ::b string?)
(s/def ::base-map (s/keys :req-un [::a ::b]))

(s/def
  :foo/bar
  (s/merge
   ::base-map
   (s/and
    (fn [m]
      (if (= (m :a) "bobo-included")
       (contains? m :bobo)
        true
        )))))
thereā€™s probably better way of doing this Basically now :foo/bar is a spec for a map that must have :a and :b, keys, but when value of :a is equal to ā€œbobo-includedā€, it must have :bobo, key, otherwise, :bobo key is optional
#2020-01-2903:37seancorfields/merge is intended for two key specs -- I don't think you should use it for a key spec and a predicate.#2020-01-2903:37seancorfieldAnd it looks like you have s/and with just a single predicate there?#2020-01-2903:38seancorfieldNow that you've restated the problem, it sounds like you want to look at multi-specs.#2020-01-2903:39agahā€¦ yeah. let me try digging in there#2020-01-2903:39seancorfieldThose are intended to select different specs based on some function of a value, in your case, the value of the :a key.#2020-01-2903:40agwell, yeah, I guess I didnā€™t exactly follow my own requirement, but the approach you showed me probably would work for me. Iā€™m gonna refresh my memory on multi-specs anyway.#2020-01-2903:40agThank you Sean!#2020-01-2910:18lambdamHello, I was playing a bit with spec 2 on an Advent of Code exercise and bumped into this case:
(def dam
  {:firstname "Dam"
   :age 35})

(s/def ::user (s/schema {:firstname string?
                         :lastname string?}))

(s/explain (s/select ::user [*])
           dam)
;; => {:firstname "Dam", :age 35} - failed: (fn [m] (contains? m :lastname))

(s/explain (s/select {:firstname string?
                      :lastname string?}
                     [*])
           dam)
;; => Success!

(s/explain (s/select [{:firstname string?
                       :lastname string?}]
                     [*])
           dam)
;; => {:firstname "Dam", :age 35} - failed: (fn [m] (contains? m :lastname))

(s/explain (s/select [{:firstname string?}
                      {:lastname string?}]
                     [*])
           dam)
;; => {:firstname "Dam", :age 35} - failed: (fn [m] (contains? m :lastname))
The second explain is weird (considering https://github.com/clojure/spec-alpha2/wiki/Schema-and-select#literal-schemas). Is there a subtle detail that I'm missing? (ping @alexmiller) Thanks
#2020-01-2913:38Alex Miller (Clojure team)Dunno, probably a bug#2020-01-2910:37lambdamAlso, is there a specific reason not to instrument :ret and :fn in fdef (even optionally) in spec 2? I use https://github.com/jeaye/orchestra with spec 1 and find it very useful.#2020-01-2913:29Alex Miller (Clojure team)https://clojure.org/guides/faq#instrument_ret#2020-01-2914:45lambdamAh ok. Thanks.#2020-01-2911:33lambdamAlso
(s/valid?
  (s/schema {:foo string?})
  {:foo 1})
=> false

(s/assert
  (s/schema {:foo string?})
  {:foo 1})
=> {:foo 1}
... weird
#2020-01-2915:22zcljI get an exception when using a schema referring a spec with an indirection to another spec. It works as expected if I just use the spec. Am I doing anything wrong here or is it a bug?
(s/def ::thing string?)
(s/def ::other-thing ::thing)
(gen/sample (s/gen ::other-thing)) ;; => works as expected

(s/def ::foo (s/schema [::thing]))
(s/def ::bar (s/schema [::other-thing]))
(gen/sample (s/gen ::foo)) ;; => works as expected
(gen/sample (s/gen ::bar)) ;; => Exception below
  
  No implementation of method: :conform* of protocol:
   #'clojure.alpha.spec.protocols/Spec found for class:
   clojure.lang.Keyword
#2020-01-2915:34Alex Miller (Clojure team)this is a bug in spec 2, so won't work yet#2020-01-2920:13kennyDoes spec2 let you write a spec for a qualified keyword such that its definition changes depending on the context? For example, if I were to spec Datomic's :db/id, it would only ever be a nat-int? when part of the result is from a pull query. When transacting to Datomic, :db/id could be a lookup ref (e.g., (s/tuple keyword? some?) ), a nat-int? , or a string?. One could spec :db/id using s/or but that means everything that takes a db id as an input needs to handle all of those cases, which does not always make sense.#2020-01-2920:52Alex Miller (Clojure team)in short, no#2020-01-2920:52Alex Miller (Clojure team)in long, the general recommendation is to try to give attributes specs that truly reflect the data#2020-01-2920:53Alex Miller (Clojure team)within certain contexts, you can add on additional specs that narrow the scope if needed#2020-01-2921:00kennyBy this do you mean you can add additional predicates to the :db/id spec in different places?#2020-01-2921:05Alex Miller (Clojure team)yes, you could s/valid while s/and'ing in an additional narrower predicate for example#2020-01-2921:05kennyOh cool. Is there an example of what that looks like?#2020-01-2921:19Alex Miller (Clojure team)just what I said?#2020-01-2921:21kennyOh - that works if the :db/id is taken as an argument itself. I was imagining a function that takes a map (perhaps nested) that has :db/ids on the maps.#2020-01-2921:21kennyOr do you mean needing to write a predicate that walks that structure validating db/id against this new predicate?#2020-01-2921:39Alex Miller (Clojure team)you can still s/and a predicate to a map that checks a value in the map#2020-01-2921:39Alex Miller (Clojure team)it's also an option to just not spec :db/id#2020-02-0122:28kennyShould spec2's s/select work for collections of collections? I would think this would return false.
(s2/def ::coll-next (s2/coll-of (s2/coll-of (s2/schema [::a]))))
(s2/def ::map (s2/schema [::coll-next]))
(s2/valid?
    (s2/select ::map [::coll-next {::coll-next [::a]}])
    {::coll-next [[]]})
=> true
#2020-02-0123:15seancorfieldGiven that ::coll-next is a collection, not a schema, I'm not sure how that should work...?#2020-02-0220:21kennyHi all. I wrote a library that adds support for Spec2's schema & select to Spec1. My company is working on a new product and thus a new data model. Defining the data model using Spec1's :req feels so wrong now that we see Spec2's direction. We intend to use this library to bridge the gap until Spec2 is out or official support for some sort of Spec1 schema/select is released. It's still very early but the few examples I've worked through all work as expected! https://github.com/ComputeSoftware/spec1-select#2020-02-0400:09kenny... now with CLJS support šŸ™‚#2020-02-0422:56kennyWill Spec2 provide a way to select the required keys of a nested Spec based on a dispatch value? For example, say you have a map that has a key ::items . The required keys for each map within the items list will vary based on the :type key on each item map. I could write add a top-level predicate to the ::items spec via s/and that would check the validity of each map in the items list against another defined spec. That would result in poor error messages though.#2020-02-0423:06Alex Miller (Clojure team)Tbd#2020-02-0423:11kennyGot it. You probably have a good idea of what it may look like already. In case it may be of any use, this is what I'm thinking we'll end up implementing internally for this problem.
(s/select
  ::report-input-data-schema
  [::instances
   {;; map of dispatch value to required keys
    ::instances {:default         []
                 :instance.type/a [:instance.a/prop1]
                 :instance.type/b [:instance.b/prop1]}
    ;; required keys for all maps
    :common     [:instance/status
                 :instance/name]
    ;; dispatch fn
    :dispatch   :type}])
#2020-02-0506:45colinkahn@kenny couldn't you do this with a multi-spec defmethods returning select specs?#2020-02-0515:55kennyNot when it's a nested collection.#2020-02-0516:02colinkahnAh right, agreed this would be great to have built in#2020-02-0516:08kennyDefinitely agree. Fortunately it's not that hard to build in yourself if it doesn't include it. It really does seem like something it should support by default though.#2020-02-0517:13colinkahnRight, I built a spec that did something like this for spec-1, basically ā€œclampedā€ nested multi-specs. It was a bit unwieldy because the definitions had to be so long.#2020-02-0514:42Wilson Velezhi, when to use :req or :req-un? Iā€™ve used :req-un always#2020-02-0515:44orestisIf you have maps with namespaced keywords, use :req#2020-02-0516:30Wilson Velez:+1:#2020-02-0601:06kennyI just spend a solid 2 hours trying to figure out how to make a schema -> select work in Spec1. Started diving deeper into the Spec2 code to land on this https://github.com/clojure/spec-alpha2/blob/495e5ac3238be002b4de72d1c48479f6bec06bb3/src/main/clojure/clojure/alpha/spec/impl.clj#L421. I think this makes sense. I'm curious if there is more reasoning behind it since it seems like there isn't a technical reason as to why you can't have a select within a schema.#2020-02-0601:15kennyI guess it's more of a fundamental thing -- a schema is always a set of optional attributes. If attributes become required, it cannot be a schema.#2020-02-0603:50Alex Miller (Clojure team)yes, that#2020-02-0616:57Ben HammondI'm working on generators for a spec like
(spec/keys :req-un [::interval/start
                    ::interval/end])
but I want to create linkage between the start/end dates. I've written a generator `
(defn sqlinterval-generator
to give me the interval boundaries that I want, but what is the best way to poke them in to the overal keys structure ? In the past I've used a bind and fmap to overwrite the interval boundaries AFTER the default generator has given me random ones; seems a bit non-intentional though; is there a better way?
#2020-02-0617:04Alex Miller (Clojure team)go the other way#2020-02-0617:05Alex Miller (Clojure team)I guess you kind of are#2020-02-0617:05Alex Miller (Clojure team)I don't think what you're doing is bad necessarily#2020-02-0617:05Alex Miller (Clojure team)and it might be the easiest way in this situation#2020-02-0617:06Ben Hammondso I have this to daisy-chain the generators
(defn generator-decorate-sqlinterval
  "overwrites the gen-in values with values from the interval generator"
  [gen-in start-key end-key interval-generator]
  (test.gen/bind gen-in
    (fn [o] (test.gen/fmap
              (fn [[start end]] (assoc o start-key start
                                         end-key end))
              interval-generator))))
#2020-02-0617:07Ben HammondI guess I'm asking if there is something like (test.gen/tuple but that will give me a map instead of a vector#2020-02-0617:08Ben Hammondnah that wouldn't be any easier would it?#2020-02-0617:08Ben HammondI'll keep hacking away at it then#2020-02-0617:08Ben Hammondthanks#2020-02-0619:09waffletowerDoes clojure.spec.alpha support defining a spec for a map where keys are not known in advance, but a spec exists for their values?#2020-02-0619:14seancorfield@waffletower For qualified keys, yes, if I'm understanding you correctly. Not for unqualified keys.#2020-02-0619:15seancorfield
user=> (require '[clojure.spec.alpha :as s])
nil
user=> (s/def :my/key int?)
:my/key
user=> (s/def ::bag (s/keys))
:user/bag
user=> (s/valid? ::bag {})
true
user=> (s/valid? ::bag {:my/key "a"})
false
user=> (s/valid? ::bag {:my/key 1})
true
user=> 
#2020-02-0619:16seancorfieldSpec checks any (qualified) keys' values against existing specs, even if s/keys doesn't list them. But if you generate/exercise ::bag, you will only get empty maps.#2020-02-0619:27waffletowerThanks Sean, I need something like a wildcard key, such that every key in a map would validate against a specific spec.#2020-02-0619:33waffletower
{:unknown 1
 :mystery 4
 :unspecified 7} etc.
#2020-02-0619:34waffletowerAnd somehow link any key to be validated against (s/def ::wildcard-key int?)#2020-02-0619:35seancorfield(s/map-of keyword? ::wildcard-key)#2020-02-0619:35seancorfield(instead of s/keys)#2020-02-0619:36waffletowerlet me test that out, thanks#2020-02-0619:43waffletower
(s/def ::wildcard-key int?)
(s/valid?
 (s/map-of keyword? ::wildcard-key)
 {:unknown 1
  :mystery 4
  :unspecified 7})
true
#2020-02-0619:43waffletowernice, many thanks!#2020-02-0715:47achikinHow do I check for exact value?#2020-02-0715:47Alex Miller (Clojure team)#{:foo} is a valid spec#2020-02-0715:47Alex Miller (Clojure team)a set containing one value#2020-02-0715:48achikinThank you!#2020-02-0715:48achikinAs far as I can understand if itā€™s a boolean I can do (s/def :mykey true?) right?#2020-02-0716:03Alex Miller (Clojure team)prob better to use boolean?#2020-02-0716:15denikI'm trying to get the spec (or any spec) from cljs.core.specs.alpha but
(ns my-ns
  (:require [clojure.spec.alpha :as s]
            [cljs.core.specs.alpha :as cljs-specs]))

(s/form ::cljs-specs/local-name)
throws
Error: Unable to resolve spec: :cljs.core.specs.alpha/local-name
Why this is happening? the spec is defined: https://github.com/clojure/clojurescript/blob/ef32778989f7ba2311a1e8a5d99c30e6805f5719/src/main/cljs/cljs/core/specs/alpha.cljc#L16
#2020-02-1006:29jrwdunhamI want a spec that accepts nilable values but does not generate them. So far I'm going with a macro like the following. Is there a better way to do this? Is wanting to do this an indicator that I'm making a bad design decision? The motivation is that I'm working with a db schema that allows NULL but my validation on data coming from users will prevent incoming nil values. I want my generators to simulate user-supplied data going into the DB and not data coming out of the existing DB.#2020-02-1013:44Alex Miller (Clojure team)seems ok to me#2020-02-1006:29jrwdunham
(defmacro def-spec-nilable-non-nilable-generator
  [kw spec]
  `(s/def ~kw
    (s/with-gen
      (s/nilable ~spec)
      (constantly (s/gen ~spec)))))
#2020-02-1012:54thomDoes anybody know of anything resembling linear temporal logic (or something weaker but still useful for describing sequences) for spec?#2020-02-1021:20andy.fingerhutI do not know if this is anywhere near what you are looking for, but a Google search for the terms "clojure linear temporal logic" turned up this project: https://github.com/esb-lwb/lwb#2020-02-1021:21andy.fingerhutSpec I am sure has nothing built in for this, except the general purpose escape hatch of letting you use arbitrary functions you write that return true/false as specs -- which is a huge enabler for anything you can write code for.#2020-02-1021:21andy.fingerhutIt won't necessarily help you generate random instances satisfying that function -- that is separate work you would need to do if you wanted that capability.#2020-02-1223:25telekidInstrumentation / generation question. In the following gist, an outer function calls an inner function twice with the same value (`i`). I want to write an fdef for the outer function, using a :fn predicate to assert that the return from the two calls to the inner fn are identical. However, when I try to stub the call to inner with instrumentation, the generated values returned by innerā€™s stub are (sensibly) different. Is there any way to express on the fdef of inner that the generated values should be some known function of the input? gist here: https://gist.github.com/telekid/01b0aed73511167c337794bc286908cf#2020-02-1304:50colinkahnWould using :replace instead of :stub work in this case?#2020-02-1320:45telekidOh, interesting! Iā€™ll look into that, cool idea#2020-02-1313:37cddrHey folks, Looking for a bit of advice about adding specs to an open source project. We have a "command syntax" where commands are basically vectors. A few examples are provided below but essentially each command's first item is an op-code and subsequent items configure the operation in an op-specific way....
[:do some-side-effect-fn]
[:write! :foo {:id 1 :payload "yolo"} optional-args]
[:watch some-pred optional-args]
Seems like there's a few ways this could be spec'd and I was just wondering if anyone had any thoughts about how I should structure it. There are some additional commands not shown here but not more than 10 overall and I don't anticipate too many more being added in the future.
#2020-02-1313:48cddrAh I've just seen how crux does it. https://github.com/juxt/crux/blob/b7fdb38357adb85627c3be5710d0aff26a796f19/crux-core/src/crux/tx/event.clj I think our commands follow the same sort of pattern as their transactions so can probably just use that. I didn't realize that multi-spec could be used with non-map shaped structures.#2020-02-1317:41colinkahnIf you look in the spec 2 repo currently there is still the keys spec, will that stay? If so, with schemas/select is there a suggested limited set of usecases for keys now?#2020-02-1317:59seancorfield@colinkahn I have seen a couple of mentions from @alexmiller indicating that keys may not stay in the final Spec 2 version -- but Spec 2 is still very much alpha and in flux. Also the fdef stuff is likely getting a substantial overhaul.#2020-02-1318:01seancorfieldI spent quite a bit of time tracking Spec 2 last year and had a branch of our system at work that passed all tests using Spec 2 instead of Spec 1 -- but still using keys and there were quite a lot of changes just to get to that point (and a number of bugs and workarounds I had to deal with). I gave up in early October and decided that we wouldn't attempt a direct migration from Spec 1 to Spec 2: instead we'll adopt Spec 2 for new code and perhaps incrementally replace our Spec 1 usage over time.#2020-02-1318:02seancorfieldOne thing that needs to be figured out for a mixed Spec 1/2 codebase is how the spec systems interact, because Clojure itself uses Spec 1 (1.9, 1.10) and so you can't (easily) mix core specs with your own Spec 2 stuff right now.
#2020-02-1318:08colinkahnThanks for the insight @seancorfield. In your experience do you think your refactor would have been easier if you were already using https://github.com/ComputeSoftware/spec1-select for you spec 1 code? Iā€™m looking to port some of my code over to it. Regarding keys I was curious if the (and) and (or) syntax used within :req/opt(-un) would exist in some form in spec 2.#2020-02-1319:28Alex Miller (Clojure team)Re and/or - no, probably not#2020-02-1319:29Alex Miller (Clojure team)Schemas support unqualified keys in spec 2#2020-02-1406:41colinkahnThanks!#2020-02-1318:09seancorfieldOur use of Spec dates back to early alphas of Clojure 1.9 so that library is moot for us.#2020-02-1318:10colinkahnRight, not saying for your specific case but Iā€™m evaluating using it to make our transition to spec 2 eventually smoother#2020-02-1318:11seancorfieldGiven how much Spec 2 is in flux, I don't know that such a library would necessarily be a useful starting point. Most of the work for us in attempting to get our Spec 1 codebase running on Spec 2 was around the much stricter delineation of symbolic specs vs spec objects etc.#2020-02-1318:11seancorfieldWe had a lot of Spec 1 specs that simply weren't legal in Spec 2 and we had to rewrite them to use defop and various other constructs.#2020-02-1318:13seancorfieldRemember: we stayed with keys when we attempted that migration -- it wasn't about switching to schema/`select`... that was considered "future work".#2020-02-1318:18colinkahnGood to know, I havenā€™t dived into the defop stuff in spec 2 yet so didnā€™t consider that. Besides future compatability I definitely feel the pain points that schema/select are meant to solve, which is why Iā€™m seeing that as a valuable first step.#2020-02-1319:47seancorfieldSure, they are a big improvement on Spec 1's keys -- but I would be wary of relying on a 3rd party lib that tries to backport Spec 2 features onto Spec 1 when a) Spec 2 is still changing so much and b) Spec 2 already has stricter rules about "what is a spec" than Spec 1.#2020-02-1319:48seancorfieldI don't consider a lib like that to be a useful "future proofing" option...#2020-02-1319:50seancorfield(it's an interesting exercise, to be sure, and if folks want some of what Spec 2 offers while staying on Spec 1 for an unspecified future time period then maybe it's of some use -- but I really doubt it will make the "migration" from Spec 1 to Spec 2 any easier)#2020-02-1423:21jasonjcknA way to attach meta data to a spec or something similar enough ? e.g.
(s/def ^:property1 ::myspec ...)
#2020-02-1423:30seancorfield@jasonjckn You can't add metadata to a keyword.#2020-02-1423:34jasonjcknyah I'm aware; thanks for the reply#2020-02-1423:35jasonjcknhttps://github.com/mpenet/spex is a 3rd party lib that offers meta data ; but unlikely to be maintained i'd guess#2020-02-1511:21mpenetYou'd be surprised ;)#2020-02-1511:22mpenetIn any case, it's just an atom and a couple of functions, easy to make a metadata registry of your own until core implements one#2020-02-1507:45ikitommiHere issue related to metadata: https://clojure.atlassian.net/plugins/servlet/mobile?originPath=%2Fbrowse%2FCLJ-1965#issue/CLJ-1965#2020-02-1507:46ikitommiThere is also spec-tools, which has a wrapper record for specs, which can hold extra (meta-)data.#2020-02-1507:46ikitommihttps://cljdoc.org/d/metosin/spec-tools/0.10.1/doc/spec-records#2020-02-1520:08EddieDoes anybody know of a java-time LocalDate generator? My search came up empty.. I am fine with implementing my own, but it seems like something that probably exists.#2020-02-1521:18dominicmI used a few number generators.#2020-02-1521:18dominicmUsing gen/let#2020-02-1718:24EddieThanks, thats what I ended up doing. šŸ‘#2020-02-1608:46abdullahibraHi everyone,#2020-02-1608:47abdullahibrais there any simple example for howto modify the error message produced by clojure.spec to be more readable ?#2020-02-1618:22seancorfield@abdullahibra You could look at Expound and Phrase, and there was a new one announced recently (which I think extends Phrase).#2020-02-1716:35ikitommiIā€™m trying to understand best practices on using spec2 schema, select and and qualified-keys in general. Is this idiomatic use of Spec2? https://gist.github.com/ikitommi/87f9b9136167c6be2e4b9f315a212046#2020-02-1717:03ikitommioh, the closed spec donā€™t even work there. Can one close selects (or subslects)?#2020-02-1717:04ikitommiclosing the schema seems to work#2020-02-1717:04ikitommiā€¦ but canā€™t make the one field optional just with schema.#2020-02-1717:12Alex Miller (Clojure team)you can only close schemas, but I think there are some impl gaps in properly validating use of a select over a closed schema#2020-02-1717:12Alex Miller (Clojure team)but otherwise, sure? I don't consider any of the schema/select stuff done, so hard to suggest "best" practices :)#2020-02-1717:14ikitommithanks. would be great to able to say, ā€œclose all schemas/selectsā€ in the future.#2020-02-1717:15Alex Miller (Clojure team)that's something we've discussed#2020-02-1717:15Alex Miller (Clojure team)so tbd#2020-02-1717:15ikitommi:+1:#2020-02-1805:19bmaddyIs it possible to parse streams or lazy seqs with spec? Specifically, I'm interested in a stream where trying to consume too much data would cause it to hang forever.#2020-02-1805:21bmaddyTo clarify that a little, there would be a number at the beginning that tells me how many more numbers to consume.#2020-02-1805:21seancorfieldSpec is not designed to be used for parsing. I suspect you already know that but it's worth repeating.#2020-02-1805:24bmaddyHuh, I guess I'd forgotten that. So definitely worth repeating! Thanks!#2020-02-1805:26seancorfield(you can abuse Spec for parsing but it will often be suboptimal... at best)#2020-02-1808:44jeroenvandijk@seancorfield I understand that Spec is not meant for high performance parsing. I do find it useful and use it sometimes to parse datastructures. Do you know of a nice alternative that is performant?#2020-02-1808:58delaguardohttps://github.com/youngnh/parsatron this library could be a good starting point but it was designed to parse strings instead of data so might need some tweaks#2020-02-1809:02jeroenvandijkThat's useful as well as alternative to instaparse. Thank you#2020-02-1809:18delaguardoinstaparse can not be turned into the parser of data structures unfortunately. Only strings are allowed and this is deeply hidden inside of implementation details#2020-02-1809:19jeroenvandijkYeah I might have combined instaparse with clojure.spec. Not sure actually#2020-02-1809:39pithylesshttps://github.com/metosin/malli and https://github.com/noprompt/meander come to mind. They attack different problems, but are both useful when working in the domain of "I need to interpret some data structures".#2020-02-1810:59mmeixI just read this article: https://juxt.pro/blog/posts/parsing-with-clojure-spec.html (a small parser for ical-entries) and watched this video: https://www.youtube.com/watch?v=xLB8Xgt8lpA (a simple hiccup-parser). These seemed (to me, as a spec beginner) reasonable applications of spec - maybe, because, they are quite simple? Would there be a problem for using spec at this scale?#2020-02-1811:54mmeixAdditional question: is it considered idiomatic to dispatch a multimethod on the result of a s/conform? Are there drawbacks?#2020-02-1812:20jeroenvandijkI have done that. Cannot think of real drawbacks expect for that if your spec changes you have to update the multimethods#2020-02-1812:20jeroenvandijk(Changing) dispatch functions of multimethods are a bit of a hassle in a repl environment as it is hard to undefine (for good reasons). But to make this work in the repl you need to use some tricks#2020-02-1813:39mmeixok, thanks!#2020-02-1814:01dergutemoritzTemporarily placing (def my-multimethod nil) in front of the defmulti and then recompiling the namespace is an easy way to do it!#2020-02-1815:41jeroenvandijk@U06GVE6NR This is true, but you have to reload all the namespace or the method is not extended the way you expect. My trick is to define a dispatch function and define the method as (defmulti my-method #'dispatch-my-method). This has been the most reliable way for me when working in the repl for a long time#2020-02-1815:41dergutemoritzAh right, if you have method implementations in other namespaces, then this trick won't suffice, good point šŸ™‚#2020-02-1815:41dergutemoritzGood idea with the var#2020-02-2018:21rickmoynihanhmmā€¦ Iā€™m wanting to spec some data that is coming from a 3rd party library, the Ataraxy router. It always uses the namespaced keyword :ataraxy/result to contain the parsed parameters for the route; however because ataraxy is configured on a route by route basis; the exact data structure differs, yet the ring request map always contains the same namespaced keyword. It strikes me that this is an unfortunate mistake; in that the same global keyword is being used to name different data. What is the best way to spec such a thing? Iā€™m thinking a multispec is really the only option, that could dispatch on the first argument. :ataraxy/result is a vector of the form [:route/id ,,,route-params,,,]#2020-02-2018:22rickmoynihanor I suppose I could coerce it into something else first šŸ˜ž#2020-02-2018:26rickmoynihanactually thinking itā€™s better to just spec the functions after it, that donā€™t depend on the :ataraxy/result key.#2020-02-2018:45Alex Miller (Clojure team)you could just do something like (s/cat :route-id ::route-id :params (s/* any))#2020-02-2018:45Alex Miller (Clojure team)but it depends a lot what route-params is#2020-02-2018:45Alex Miller (Clojure team)if it's kwargs, then keys* would work great#2020-02-2021:47jasonjcknmy data looks like {::type "foobar" ::a ... ::b ... ::c ...} I want different s/keys validation based on the value of ::type whats the feature i'm looking for#2020-02-2021:47jasonjcknit's like ~ conditional spec validation polymorphic to field value#2020-02-2021:47jasonjckni guess some sort of mulitmethod on the spec#2020-02-2021:48jasonjcknreading about multi-spec now, this looks like it#2020-02-2021:57jasonjcknfor the :default case#2020-02-2021:57jasonjcknwhat would be a canonical way to fail spec validation#2020-02-2021:57jasonjckn
defmethod tagmm :default
#2020-02-2021:57jasonjcknhttps://clojuredocs.org/clojure.spec.alpha/multi-spec#2020-02-2021:58Alex Miller (Clojure team)(s/keys*) would automatically handle all registered specs for the tail of that#2020-02-2021:58jasonjckni want default case to fail#2020-02-2021:58jasonjcknvalidation#2020-02-2021:59Alex Miller (Clojure team)you can definitely use multi-spec for this if that makes sense#2020-02-2021:59jasonjcknwhat is a canonical way to implement :default#2020-02-2021:59jasonjcknbesides a conformer that always throws exceptions#2020-02-2021:59Alex Miller (Clojure team)I don't understand enough what you're doing#2020-02-2021:59Alex Miller (Clojure team)what does :default mean?#2020-02-2022:00jasonjckn
(s/def ::tag #{:a :b :c :d})
(s/def ::example-key keyword?)
(s/def ::different-key keyword?)

(defmulti tagmm :tag)
(defmethod tagmm :a [_] (s/keys :req-un [::tag ::example-key]))
(defmethod tagmm :default [_] (s/keys :req-un [::tag ::different-key]))

(s/def ::example (s/multi-spec tagmm :tag))
(gen/sample (s/gen ::example))
here's the example
#2020-02-2022:00jasonjckncan I do :default ::s/invalid ?#2020-02-2022:00Alex Miller (Clojure team)oh, you mean multi-method default#2020-02-2022:00jasonjckn
(defmethod tagmm :default [_] ::s/invalid) 
will test that out
#2020-02-2022:00Alex Miller (Clojure team)the multimethod returns specs so you need to return a spec that always fails#2020-02-2022:01Alex Miller (Clojure team)so probably more (constantly false) would work#2020-02-2022:01jasonjcknok great#2020-02-2022:02Alex Miller (Clojure team)or #{}#2020-02-2022:02jasonjcknthat seems more canonical#2020-02-2022:02Alex Miller (Clojure team)a set of no values :)#2020-02-2022:02jasonjcknparticular in the spec explanations#2020-02-2022:02jasonjcknit'd be nice to get something that made sense#2020-02-2023:45vemva q out of curiosity more than anything else: (spec/valid? (spec/coll-of char? :min-count 1 :max-count 20) "abc") doesn't work (and a straightforward impl would underperform anyway?) are there spec-related libraries offering a coll-of-like API that efficiently works on strings? I just like coll-of's API, in that it doesn't resemble regex#2020-02-2100:08seancorfield@vemv The standard answer is that Spec isn't designed for parsing strings šŸ™‚#2020-02-2100:09seancorfieldMaybe Instaparse?#2020-02-2100:28vemvI don't want to parse them (e.g. do sth useful with them) - just validate them#2020-02-2421:53gariepyalexWhen conforming a sequence spec, can we guaranty key order? For instance:
(s/def ::a neg-int?)
(s/def ::b zero?)
(s/def ::c pos-int?)
(s/def ::sequence (s/cat :a ::a, :(s/? ::b), :c ::c))
I would like to use conform to know if the second element of the sequence is :b or :c (labels given in cat). conform returns a clojure.lang.PersistentHashMap, which does not key the keys in the same order as in the sequence.
#2020-02-2422:49Alex Miller (Clojure team)s/tuple is a better match#2020-02-2421:57seancorfields/conform just tells you how it matched. If you call s/valid? and get true then the input sequence is valid as is.#2020-02-2421:59gariepyalexI would have liked to use conform to parse the sequence and end up with {:a -2, :c 10}#2020-02-2422:02seancorfieldSpec isn't really a "parser" but if you know your elements should be in a given order, you can easily manipulate the data you get from s/conform to produce a more appropriate structure.#2020-02-2422:03seancorfield(you know you're going to get :a first and :c last so the only question is (contains? conformed-data :b) to see if a zero is present...#2020-02-2422:04gariepyalexok thanks for your help!#2020-02-2514:12Filipe Silvaheya#2020-02-2514:12Filipe Silvais it possible to rename a key with s/keys ?#2020-02-2514:13Filipe SilvaI'd like to do something like this
(s/def ::msg (s/and string? #(< 20 (count %))))
;; TODO: declaring a spec for a limited :db/id doesn't sound right. Figure out a better way.
(s/def :db/id (into #{} (range 1 10)))
(s/def ::map-datom (s/keys :req [:db/id ::msg]))
(s/def ::simple-tx (s/coll-of ::map-datom :kind vector? :min-count 0 :max-count 3))
#2020-02-2514:13Filipe Silvabut instead of (s/def :db/id, I'd like to do (s/def ::id#2020-02-2514:14Filipe Silvaand then on s/keys say that ::id is required but should show up as :db/id#2020-02-2514:15Filipe SilvaI'd like to do this because I don't want to be declaring the spec for the real :db/id, but would like to have an alternative version of it for testing#2020-02-2514:27Alex Miller (Clojure team)in short, no - the whole idea with spec is that attributes are primary and have meaning and maps are just aggregations of attributes#2020-02-2514:28Alex Miller (Clojure team)if you want a broader definition of ::db/id, then your spec should reflect that#2020-02-2514:32Filipe Silvaok, thank you for explaining it to me#2020-02-2514:32Filipe Silvaif I want to just generate test data that is a subset of all possible data, should I be using something else?#2020-02-2514:34rickmoynihan@filipematossilva: It sounds like you might be complecting generating data with specing. If in your tests you want to generate known idā€™s that are a subset of valid idā€™s, you can probably just pass a generator for that set where you need it.#2020-02-2514:35Filipe Silvayes, I think I am#2020-02-2514:35Filipe Silvabut s/keys does not support inline value specs (by design, according to the docs), so I don't think I can do that#2020-02-2514:36Filipe Silva(unless, of course, I am misunderstanding what you propose)#2020-02-2514:36Filipe Silvaan alternative is to generate the test data with exercise, and then map over the results to change the key#2020-02-2514:36Filipe Silvawhich sounds fair enough, but I wanted to inquire first#2020-02-2514:36Alex Miller (Clojure team)you can pass generator overrides when doing most of the functions that generate in spec, but you still have to generate data that conforms to the spec#2020-02-2514:39Filipe Silvaah, then this was just something I did not know existed#2020-02-2514:39Filipe Silvais it https://clojuredocs.org/clojure.spec.alpha/gen?#2020-02-2514:39Filipe Silvaor rather https://clojuredocs.org/clojure.spec.alpha/with-gen, as that seems to have examples#2020-02-2514:40rickmoynihanTotally! :-) When I first started with spec I made the mistake of over speccing to get spec to try and generate test data of the right shapeā€¦ but now prefer the alternative of specing a little more broadly but using :gen ā€™s to generate the more precise data I want. Obviously itā€™s all trade offs. Anyway I was just suspecting Filipe might be falling into this same trap.#2020-02-2514:42Filipe Silvayes, I was šŸ™‚#2020-02-2514:45Filipe Silvathank you#2020-02-2514:45Filipe SilvaI'll try to fiddle with the :gen option on s/keys and see where that takes me#2020-02-2515:00rickmoynihanšŸ‘ hope it helps#2020-02-2515:11Filipe Silva@rickmoynihan I'm trying to use the :gen key on s/keys, but I don't quite follow how I can use it to replace a single def#2020-02-2515:11Filipe Silvais that possible? or does :gen need to override how everything is generated#2020-02-2515:11Filipe Silvaif you have an example of using it with s/keys it would be super useful#2020-02-2515:23Filipe Silvaoh, I think this is the idea?#2020-02-2515:23Filipe Silva
(s/def :db/id pos-int?)
(s/def ::small-pos-int? (into #{} (range 1 10)))
(s/def ::map-datom (s/keys :req [:db/id]))
(def small-pos-int-gen (s/gen ::map-datom {:db/id #(s/gen ::small-pos-int?)}))
(s/def ::small-id-map-datom (s/keys :req [:db/id]
                                    :gen (fn [] small-pos-int-gen)))

(s/exercise ::small-id-map-datom 10)
;; => ([{:db/id 5} {:db/id 5}] [{:db/id 3} {:db/id 3}] [{:db/id 1} {:db/id 1}] [{:db/id 1} {:db/id 1}] [{:db/id 7} {:db/id 7}] [{:db/id 8} {:db/id 8}] [{:db/id 6} {:db/id 6}] [{:db/id 8} {:db/id 8}] [{:db/id 6} {:db/id 6}] [{:db/id 2} {:db/id 2}])
#2020-02-2515:23Filipe Silvamaking the real spec, then making another spec that has a generator that takes the real spec generator and adds an override#2020-02-2515:26rickmoynihanThatā€™s not quite what I was suggestingā€¦#2020-02-2515:31rickmoynihan
(s/def ::foo integer?)

  (s/def ::foo-map (s/keys :req-un [::foo]))

 ;; The general generator (generates negative integers too)
  (s/exercise ::foo-map);; => ([{:foo -1} {:foo -1}] [{:foo 0} {:foo 0}] [{:foo 0} {:foo 0}] [{:foo 0} {:foo 0}] [{:foo 3} {:foo 3}] [{:foo -2} {:foo -2}] [{:foo -7} {:foo -7}] [{:foo -37} {:foo -37}] [{:foo -2} {:foo -2}] [{:foo -3} {:foo -3}])

;; An overriden tighter generator generating just 1 and 2 values for :football: 

  (s/exercise ::foo-map 10 {::foo-map (fn [] (s/gen #{{:foo 1} {:foo 2}}))});; => ([{:foo 1} {:foo 1}] [{:foo 2} {:foo 2}] [{:foo 1} {:foo 1}] [{:foo 2} {:foo 2}] [{:foo 2} {:foo 2}] [{:foo 1} {:foo 1}] [{:foo 1} {:foo 1}] [{:foo 2} {:foo 2}] [{:foo 2} {:foo 2}] [{:foo 2} {:foo 2}])
#2020-02-2515:33rickmoynihan@filipematossilva: You can override on s/exercise, or st/check as well, so you can tailor your generation there rather than in the specs if you want.#2020-02-2515:33Filipe Silvaah on exercise itself#2020-02-2515:33Filipe SilvaI see now#2020-02-2515:33Filipe Silva
(s/def :db/id pos-int?)
(s/def ::small-pos-int? (into #{} (range 1 10)))
(s/exercise :db/id 10 {:db/id #(s/gen ::small-pos-int?)})
#2020-02-2515:34Filipe SilvaI understand now#2020-02-2515:34Filipe Silvathank you for taking the time to make the examples for me#2020-02-2515:34rickmoynihanšŸ‘#2020-02-2515:34Filipe Silvait was very helpful#2020-02-2515:36rickmoynihanessentially s/exercise / st/check are the testing entry points to spec; so you can split the concerns of specing and overriding generation/testing at those entry pointsā€¦ whilst s/conform s/valid? and s/explain-xxx are the checking side of spec ā€” and consequently donā€™t support generators at all (because theyā€™re a different concern)#2020-02-2515:36rickmoynihanat least thatā€™s how I think of it#2020-02-2515:37rickmoynihanThere are probably more nuanced explanations šŸ™‚#2020-02-2515:39Filipe Silvathat sounds much more sensible than I what was attempting#2020-02-2515:39rickmoynihanits ok been there, done it, got burned! šŸ™‚#2020-02-2604:48yuhanI was playing around with spec2 and noticed that it doesn't allow for using local bindings in definitions#2020-02-2604:48yuhanie. this worked in spec.alpha:
(let [max-v 10]
  (s/def ::foo (s/int-in 1 (inc max-v)))
  (s/valid? ::foo 20))
#2020-02-2604:48yuhanbut not in spec2, where it would throw an error "Unable to resolve symbol max-v"#2020-02-2604:50yuhanIs this a deliberate design decision or was it ever officially supported in spec.alpha in the first place?#2020-02-2605:41seancorfield@qythium Spec 2 draws a much sharper line between symbolic specs and spec objects. Spec 2 has new facilities for creating specs programmatically that allow you to create such a spec, but in a different way.#2020-02-2605:46seancorfieldYou can read about the design decisions here https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha @qythium#2020-02-2605:59yuhanhmm.. that's slowly starting to make sense#2020-02-2605:59yuhanSo I would do this instead?
(let [max-v 10]
  (s2/register ::foo
    (s2/resolve-spec `(s2/int-in 1 (inc ~max-v))))

  (s2/valid? ::foo 20))
#2020-02-2606:05seancorfieldYup, that should work.#2020-02-2606:08yuhanaha, so it's roughly like (s2/def kwd expr) is macro sugar for (s2/register kwd (s2/resolve-spec expr))`#2020-02-2606:11seancorfieldYes, if you look at the source https://github.com/clojure/spec-alpha2/blob/master/src/main/clojure/clojure/alpha/spec.clj you'll see def calls register and several macros use resolve-spec to produce spec objects from symbolic forms.#2020-02-2606:12seancorfieldI don't have any code handy but defop is also handy for building new spec predicates.#2020-02-2606:13yuhanThat actually makes a lot more sense than the nested macros in the original Spec šŸ™‚#2020-02-2606:14yuhanwhich I recall macroexpanding to trace the logic and eventually just treated as magic#2020-02-2606:14yuhanthanks!#2020-02-2618:13benIf I have written a s/fdef for a function, is there an easy way to generate property tests for that function (as part of a test suite), without having to manually (re)write all the generators?#2020-02-2618:23dchelimskyHave you looked at https://clojure.github.io/spec.alpha/clojure.spec.test.alpha-api.html#clojure.spec.test.alpha/check-fn?#2020-02-2618:28benI saw this, but my understanding is that is runs the tests (like test.check/quick-check) rather than creating a test (`deftest`). Have I misunderstood?#2020-02-2619:33rapskalian@UFUDSN6BE maybe youā€™re looking for something like defspec https://github.com/clojure/test.check/blob/master/doc/intro.md#clojuretest-integration#2020-02-2620:26kennyAlthough some don't recommend it, we do run spec's "check" in deftests using an "internal" (but public) library: https://github.com/Provisdom/test/blob/a61266ab281580af88fd394e9896cb85332cc5d7/src/provisdom/test/core.clj#L330 This will let you write something like this
(deftest my-fn-gen-test
  (is (spec-check `user/my-fn)))
which will run & report st/check errors.
#2020-02-2709:59benThanks @U6GFE9HS7 - I am aware of test.check, but as I understand it, I will have to write my own generators again, rather than using fdefā€™s :args to check the function#2020-02-2710:00benThanks @U083D6HK9 - this looks like what I was imagining. Why do some not recommend it?#2020-02-2716:01kennyI think the argument is that they should run at a different time than regular unit tests because they are different. I don't have a problem with that -- run them in whatever way fits your company's workflow. These sort of tests are invaluable though. You should certainly run them before deployment. Having one test runner (https://github.com/lambdaisland/kaocha for us) is quite nice. We have a large codebase with thousands of tests. Running gen tests with our unit tests has not caused us any substantial pain. They typically take 1s or less to run. The more complex functions with custom gens can take longer. The results of a gen test do not fit well into the expected/actual framework clojure.test uses though. That is certainly another downside.#2020-02-2718:57robertfw@UFUDSN6BE re: generating args. I use the following code to generate args from an fdef:
(defn arg-gen
  "Creates an argument generator for a given function symbol"
  [sym]
  (-> sym s/get-spec :args s/gen))
#2020-02-2618:14benLike stest/check but as part of my tests#2020-02-2711:40rickmoynihan@ben Kaocha has support for suites of fdefs#2020-02-2800:41robertfwI'm trying to figure out how the overrides map for (s/gen) works, namely for overriding a generator that's down the tree a bit. That is, I have a top level spec ::data which has a key ::things which is a collection of ::thing, which has a key ::properties . I'm generating the top level - ::data - and wanting to override both ::things so I get only 1 thing, and ::properties so I get an empty list. using (s/gen ::data {::things #(s/gen (s/coll-of ::thing :min-count 1 :max-count 1))}), I'm able to force ::things to only have 1 ::thing, but I haven't figured out the incantation needed to have ::properties be empty.#2020-02-2800:50robertfwThe docstring talks about providing a vector of keyword paths, but I haven't been able to find any more detail on what those paths should look like. I've been trying various combinations#2020-02-2801:29robertfwI suspect the issue is in using (s/coll-of) - for now I'm using an spec.gen/fmap call to remove any extra data I don't want, but would prefer to be able to just generate only the data needed#2020-02-2801:32seancorfieldYour s/gen override for ::things could have a third argument which is the override for ::properties#2020-02-2801:32seancorfieldA bit like this
(s/exercise ::things 10 {::things #(s/gen (s/coll-of ::thing :min-count 1 :max-count 1) {::properties (fn [] (s/gen (s/coll-of string? :min-count 0 :max-count 0)))})})
#2020-02-2801:33robertfwgotcha. that makes sense#2020-02-2801:33seancorfieldI would have expected to be able to use [::properties] as a key in the top-level overrides but that doesn't seem to work.#2020-02-2801:33seancorfieldI can't figure it out from the code šŸ˜•#2020-02-2801:34robertfwYeah I spent some time walking through some spec guts but couldn't unravel it#2020-02-2801:35seancorfields/keys and several others build a vector of keys to index into the overrides but I got lost trying to trace through multiple layers#2020-02-2802:54colinkahnI think it's because the nested s/gen creates a different context that doesn't know about the overrides in the top level s/exercise#2020-02-2803:01Alex Miller (Clojure team)^^ this, there's actually a bug around this iirc#2020-03-0119:42bbrinckIn spec2, how would I use a set spec to match against a symbol that happens to be the name of a macro?
> (s/def ::is-foo #{foo})
:expound.alpha2.core-test/is-foo
> (s/form ::is-foo)
#{foo}
> (s/explain ::is-foo 'foo)
Success!
nil
> (s/def ::is-or #{or})
:expound.alpha2.core-test/is-or
> (s/form ::is-or)
#{clojure.core/or}
> (s/explain ::is-or 'or)
or - failed: #{clojure.core/or} spec: :expound.alpha2.core-test/is-or
nil
Or should I avoid using set specs in this case?
#2020-03-0120:08Alex Miller (Clojure team)there is actually a known issue around sets of symbols (kind of a collision with symbol as function reference, which need qualification)#2020-03-0120:13Alex Miller (Clojure team)a workaround for the moment is (s/register ::is-or (s/resolve-spec #{'or}))#2020-03-0121:06bbrinckPerhaps just a different view on the same underlying issue, but it looks like spec2 is using clojure.core/and instead of the and symbol for combining specs in :req?
> (s/def :keys-spec/name string?)
> (s/def :keys-spec/age int?)
> (s/def :key-spec/state string?)
> (s/def :key-spec/city string?)
> (s/def :key-spec/zip pos-int?)
> (s/def :keys-spec/user2 (s/keys :req [(and :keys-spec/name
                                           :keys-spec/age)]
                                :req-un [(or
                                          :key-spec/zip
                                          (and
                                           :key-spec/state
                                           :key-spec/city))]))
> (s/form :keys-spec/user2)
(clojure.alpha.spec/keys :req [(clojure.core/and :keys-spec/name :keys-spec/age)] :req-un [(clojure.core/or :key-spec/zip (clojure.core/and :key-spec/state :key-spec/city))])
#2020-03-0121:09bbrinckI would have expected :req [(and :keys-spec/name :keys-spec/age)]#2020-03-0122:35favilaIā€™m pretty sure spec1 does the same? I think the form (in spec1) gets rewritten to a predicate fn in a trivial way. Iirc you can even use your own predicates instead of only and/or#2020-03-0202:36Alex Miller (Clojure team)technically, it is the clojure.core/and function in both spec 1 and 2#2020-03-0202:41Alex Miller (Clojure team)not really going to do anything with it as the plan is for s/keys and and/or to go away in spec 2 anyways#2020-03-0215:30bbrinckAh, thatā€™s good to know. Thanks!#2020-03-0215:14norton@alexmiller I tried the following example from spec 2 wiki page but it fails with ā€œCouldnā€™t satisfy such-that predicate after 100 triesā€. Is this a known issue?
(gen/sample (s/gen (s/select {:a int? :keyword?} [:a])) 5)
#2020-03-0215:14norton@alexmiller I tried the following example from spec 2 wiki page but it fails with ā€œCouldnā€™t satisfy such-that predicate after 100 triesā€. Is this a known issue?
(gen/sample (s/gen (s/select {:a int? :keyword?} [:a])) 5)
#2020-03-0215:19nortonIā€™m using this version:
{:git/url ""
         :sha "8498f9cb352135579b6d3a0a5d15c40e5c2647ce"}
#2020-03-0215:26nortonI would like to use select with unqualified keys for schema name and select keys. Something like the following:
(s/def :foo/bar
    (s/schema {:a int? :b keyword?}))

(gen/sample (s/gen (s/select :foo/bar [:a])) 5)
#2020-03-0216:18Alex Miller (Clojure team)used to work, prob broken by later changes#2020-03-0216:20Alex Miller (Clojure team)(gen/sample (s/gen (s/select [{:a int? :b keyword?}] [:a])) 5) should work as an alternate right now#2020-03-0216:48nortonThat works. Thank you. Any suggestions for this? It fails with ā€œCouldnā€™t satisfy such-that predicate after 100 triesā€.
(s/def :foo/bar
  (s/schema {:a/b string? :b keyword?}))

(gen/sample (s/gen (s/select :foo/bar [:a/b])))
#2020-03-0217:07Alex Miller (Clojure team)sorry, not sure off the top of my head#2020-03-0300:25norton@alexmiller I was able to find the reported error (outside of the generator). The last s/def below fails with the following error:
1. Unhandled java.lang.AssertionError
   Assert failed: (every? (fn* [p1__18963#] (not (select?
   p1__18963#))) unq-specs)

                  impl.clj:  423  clojure.alpha.spec.impl/schema-impl
(s/def :foo/bar
  (s/schema {:a/b string? :b keyword?}))

(s/def :foo/baz
  (s/select :foo/bar [:a/b]))

(s/def :foo/buz
  (s/schema
   {:z :foo/baz}))
#2020-03-0716:18norton@alexmiller I tried to find a smaller failing case.
(require '[clojure.alpha.spec :as s] '[clojure.alpha.spec.gen :as gen])
;; => nil

(s/def :wf/S
  (s/schema {:s/i-001 string?
             :s/i-002 string?}))
;; => :wf/S

(s/def :wf/S1
  (s/select :wf/S [:s/i-001]))
;; => :wf/S1

(s/valid? :wf/S {:s/i-001 "1"})
;; => true

(s/valid? :wf/S {:s/i-001 1})
;; => false

(s/valid? :wf/S1 {})
;; => false

(s/valid? :wf/S1 {:s/i-002 "1"})
;; => false

(s/valid? :wf/S1 {:s/i-001 "1"})
;; => true

(s/valid? :wf/S1 {:s/i-001 1})
;; => true

(gen/sample (s/gen :wf/S1) 1)
;; Caused by clojure.lang.ExceptionInfo
;; Couldn't satisfy such-that predicate after 100 tries.)
The result of the last 2 expressions is not expected (to me).
#2020-03-0403:53jrwdunhamAre there best practices for how specs are intended to be identified with one another? I have the situation where I want to define spec ::data via (s/def ::data ::specs/jsonb) (where (s/def ::jsonb map?)). I want to define a map (`spec-casters`) from specs to the functions that can be executed to transform their values (for db insertion). The spec-casters-YES-PLEASE function (which I want to use) seems to do what I want, but I have already run into situations in development where (get spec-casters-YES-PLEASE (s/spec ::other-ns/data)) ;; => nil. I could use spec-casters-NO-THANKS but I would rather avoid that verbosity. This makes me wonder consider that there might be something inadvisable about this approach.
(def spec-casters-YES-PLEASE
  {(s/spec ::specs/jsonb) pg-cast-jsonb})
(def spec-casters-NO-THANKS
  {::specs/jsonb pg-cast-jsonb
   ::other-ns/data pg-cast-jsonb})
#2020-03-0404:01Alex Miller (Clojure team)I only understand about 30% of what you're trying to do, but (s/spec ::specs/jsonb) is a spec object without well-defined equality semantics so it seems like a bad choice for a key#2020-03-0404:09jrwdunhamOk, the fact that specs lack well-defined equality semantics suggests that I should just be more explicit and use the NO-THANKS approach where the keys are just the namespaced keywords.#2020-03-0404:10jrwdunhamHowever, that seems to violate the intention of a key spec though, namely that its value should conform to it ...#2020-03-0404:11Alex Miller (Clojure team)but you're not conforming#2020-03-0404:14Alex Miller (Clojure team)you're just matching spec objects#2020-03-0404:15Alex Miller (Clojure team)not using them for anything#2020-03-0404:17jrwdunhamright, that's true.#2020-03-0522:03jrwdunham(btw, thanks for taking the time to discuss my question Alex. I do appreciate it.)#2020-03-0800:12agDo we have a repo with a collection of arbitrary specs for all sorts different cases? I often need to create a spec for something so trivial but end up spending more time than intended, only to remember later that I did it already in some other Clojure project. Sometimes I just need to perform Github search and find some gems. Would be nice to have a community driven repo of good examples.#2020-03-0800:16seancorfieldHmm, I see Specs as being tied to domains rather than being that sort of reusable -- what sort of things do you have in mind @ag?#2020-03-0800:28agthings like various datetime, timestamps, comma separated lists of currencies or/and US states and US-state abbrevs, etc.#2020-03-0800:29agYou right though, maybe not a collection, but perhaps something like a cookbook#2020-03-0800:53seancorfieldYou're talking about strings? Parsing strings? That's not a good use for Spec tho'...#2020-03-1307:59ro6I went down that road a bit once too, mostly for want of the automatic error descriptions Spec gives, but extending down into a String (eg "That's not a valid email because X, Y, Z"). Spec definitely does a good job of that for aggregates/shapes. I don't recall finding a satisfactory solution at the time.#2020-03-0800:57seancorfieldSpecs for datetime/timestamps -- inst?, s/inst-in.#2020-03-0801:01seancorfieldLocales, country codes, currency codes -- all easily available from Java classes but I guess ready made Specs for sets of those things would save you writing a single line of Clojure?#2020-03-0801:04seancorfieldUS states -- that's domain-specific: do you just want the 50 states, plus DC? Plus the various US territories/islands? But stuff like that belongs in databases and you can construct a set spec from that.#2020-03-0816:06derpocioushi, I am wondering if it is possible to instrument the return value of a function?#2020-03-0816:59EddieI like to use post-assertions combined with spec. They look like this.
(defn foo
	[x y]
	{:post [(s/valid? ::my-spec %)]}
	(do-something x y))
#2020-03-0817:01EddieYou can also validate your arguments similarly using a pre-assertion.
(defn foo
	[x y]
	{:pre  [(s/valid? ::x-spec x)
	        (s/valid? ::y-spec y)]
	 :post [(s/valid? ::my-spec %)]}
	(do-something x y))
#2020-03-0907:10andy.fingerhutI have not used it before, but the orchestra library aims to do this: https://github.com/jeaye/orchestra. It is not included in spec because it is instead recommended to only do checking of return values during testing, e.g. property-based testing using randomly generated arguments.#2020-03-0918:55Gustavo GoretkinHi. Total beginner here. I've seen mention of this global registry of specs. Where can I see which specs are defined there? Also, are there any writings about spec and other format-specific Schema definitions (like XML Schema (.xsd) or JSON schema things)?#2020-03-0918:58favilahttps://clojure.github.io/spec.alpha/clojure.spec.alpha-api.html#clojure.spec.alpha/registry to get the registry#2020-03-0918:59favilaitā€™s global--to your process, not the world#2020-03-0919:00Gustavo GoretkinI see. Thanks! I think I misinterpreted. Are there clojure libraries/packages that primarily define specs?#2020-03-0919:05favilaI only know of https://github.com/cognitect-labs/anomalies Sometimes a library will include specs for its api or extension points. Iā€™m not aware of any library that just shares spec objects the way one would share utility functions#2020-03-1011:54kszabohttps://github.com/edn-query-language/eql/blob/master/src/edn_query_language/core.cljc is like 50% specs, 50% code#2020-03-1020:35Gustavo GoretkinThanks for the pointers. I was expecting more packages like this, I think, to capture "ontologies", the same way there are schemas for e.g. geometry data, like https://github.com/mapbox/geobuf , http://wiki.ros.org/geometry_msgs Is that kind of the spirit of spec at all?#2020-03-1021:33Gustavo GoretkinOr something like this: https://clojureverse.org/t/experiment-a-spec-for-maven-poms-generated-from-xml-schema/2594#2020-03-1012:06ataggartAnyone have insight into why the such-that is used here? https://github.com/clojure/spec.alpha/blob/spec.alpha-0.2.176/src/main/clojure/clojure/spec/gen/alpha.clj#L187#2020-03-1012:09ataggartOur tests are occationally failing with "Couldn't satisfy such-that predicate after 10 tries.", and the associated pred is :pred #object[clojure.core$ratio_QMARK_ 0x62fb2f00 ".#2020-03-1012:13sogaiuthere is some mention of such-that failures in gary fredericks' generators video at around 16:16 -- https://www.youtube.com/watch?v=F4VZPxLZUdA#2020-03-1012:25gfredericksratios are conceptually goofy because you may or may not want to consider integers to be ratios in a given context it's useful to think of integers as special kinds of ratios, because almost any use case for ratios should almost certainly apply to integers as well. It's hard to do arithmetic with ratios without bumping into integers inadvertently at least (e.g., (* 3/4 4/3)) but arguably it would surprise people if (ratio? 42) returned true, so it doesn't; maybe there are other good reasons for this too beyond just surprise in any case, there's a mismatch because test.check takes the first approach, but ratio? wearing its spec hat obviously has the same semantics as ratio? wearing its predicate hat, and so when you generate from it you don't want to generate integers I think (such-that ratio? (ratio)) could definitely be improved; setting a higher retry threshold would stop the exceptions, but a custom generator would be the best; something like (gen/let [[n d] (gen/tuple gen/large-integer gen/large-integer)] (let [r (/ n d)] (if (ratio? r) r (/ (inc n) d)))) there might be some arithmetical edge case there I'm not thinking of, but I think that works#2020-03-1013:25ataggartThanks for that! I thought I was losing my mind.#2020-03-1013:28ataggartPerhaps spec'ing on rational? is the way I should go.#2020-03-1012:27gfredericksalso I forgot to handle (zero? d)#2020-03-1123:06aghow can I ā€œunaliasā€ an fdef spec?#2020-03-1123:13Alex Miller (Clojure team)Fdef to nil#2020-03-2401:33QuestSpecific question about the "data specs" feature in spec-tools: https://github.com/metosin/spec-tools/blob/master/docs/02_data_specs.md I want to specify a vector of elements where the vector's count = 2. Is this possible to specify using data-specs?#2020-03-2401:44seancorfield@quest You can certainly specify that with clojure.spec. No idea about spec-tools.#2020-03-2401:46QuestYeah, the :min-count option works fine in normal specs... I'm trying to cast to clojure specs from an EDN business schema I have, proving difficult#2020-03-2401:46QuestI know spec2 has much improved support for runtime specs, but I need the CLJS side unfortunately#2020-03-2401:47seancorfieldAh, so you can't eval either...#2020-03-2502:35QuestJust to follow up on what I ended up doing: Creating a separate "pre-runtime" step which generates a file of clojure.spec definitions from the data model. I expect this can be eliminated once spec2 is usable everywhere.#2020-03-2502:46seancorfieldSounds like a good compromise for now!#2020-03-2401:48QuestYeah. I had some success by doing funky things using the spec private "impl" functions, but it completely mangles the error messages when data fails validation#2020-03-2418:03yonatanelIs it possible to spec a map where either a namespaced key or another unnamespaced key is required? e.g {:my/a 1} ok, {:my-a 1} ok, {:my/a 1, :my-a 2} should also be fine, {:b 2} invalid.#2020-03-2420:00colinkahnSpec V1 doesnā€™t really have a way to say ā€œthis map canā€™t contain :bā€. For aliasing :my-a to :my/a you could choose a single ā€œcanonicalā€ version of your map and use a conformer when you expect there to be variations of it.
(s/def :my/a number?)
(s/def ::m (s/keys :req [:my/a]))

(s/def ::->normalize-keys (s/conformer #(clojure.set/rename-keys % {:my-a :my/a})))

(s/valid? (s/and ::->normalize-keys ::m) {:my-a 1})   ;-> true
(s/valid? (s/and ::->normalize-keys ::m) {:my/a 1})   ;-> true
(s/valid? (s/and ::->normalize-keys ::m) {:my-a "1"}) ;-> false
(s/valid? (s/and ::->normalize-keys ::m) {:my/a "1"}) ;-> false
Depending on what your needs are for generation though this can be troublesome.
#2020-03-2420:07yonatanelYeah, itā€™s nice, but I wanted (s/explain) to mention the unnamespaced key if both are missing#2020-03-2420:19colinkahn@U1B0DFD25 this is kind of convoluted but:
(s/def :my/a number?)
(s/def ::my-a :my/a)

(s/def ::m (s/merge (s/nonconforming (s/or :v1 (s/keys :req [:my/a])
                                           :v2 (s/keys :req-un [::my-a])))
                    (s/keys :opt [:my/a]
                            :opt-un [::my-a])))

(s/valid? ::m {:my/a 1}) ; true
(s/valid? ::m {:my-a 1}) ; true
(s/valid? ::m {:my/a 1 :my-a 2}) ; true
(s/valid? ::m {:b 2}) ; false

(s/explain-data ::m {:b 2})
(s/exercise ::m)
#2020-03-2420:58yonatanel@U0CLLU3QT Thanks. I wasnā€™t aware of nonconforming. Gave me some options anyway.#2020-03-2418:40kszabonot directly#2020-03-2418:40kszaboyou have to spec that with additional predicates. (s/keys) doesnā€™t have support for that AFAIK across req/req-un groups#2020-03-2421:08aghow do I make a (s/coll-off ::foo), making sure that the elements are distinct by one given field? e.g.: if (s/def ::foo (s/keys :req-un [:name])), I want that all the names in the generated collection are unique.#2020-03-2421:11agnevermindā€¦ figure it outā€¦ used: (s/and (s/coll-of ,,,, ) #(apply distinct? (mapv :name %)),,,#2020-03-2511:05Janne SauvalaWhat is your recommendation using spec2 in a new project? My observation is that many are using spec1, Plumatic/Prismatic Schema or metosin malli and not migrating to spec2 yet.#2020-03-2512:17Alex Miller (Clojure team)Spec 2 is not ready for use yet#2020-03-2513:06Janne SauvalaThanks, good to know :+1::skin-tone-2:#2020-03-2606:14Ben SlessI have a bit of a philosophical question: I watched several talks about spec and it's pretty clear how you'd use it for domain modelling when starting from scratch, but several mentioned its benefits in speccing an existing codebase. Any tips on where and how to start with speccing an existing code base?#2020-03-2606:34robertfwI would start by adding fdefs for key functions#2020-03-2614:32Timur LatypoffMaybe it's also a good idea to spec places where pieces of code, written by different people in the team, meet. To set expectations and settle future disputes.#2020-03-2607:12sogaiuthere was this bit from stuart halloway: http://blog.cognitect.com/blog/2017/6/19/improving-on-types-specing-a-java-library -- may be that's of some use?#2020-03-2622:02bortexzHi, does clojure-spec provide or will provide a way to convert schema maps (unqualified) without explicitly specifying spec on every val? e.g.
From (spec/schema {:type #{:open :closed}}) -> (spec/schema {:type (spec/spec #{:open :closed})})
Seems like a map traversal, is it something that can already be done with normal clojure on the context of spec and schemas?
#2020-03-2623:39Alex Miller (Clojure team)I dont think you should even need that?#2020-03-2700:00bortexzyeah on a second thought itā€™s not very useful#2020-03-2700:26bortexzAlso because it would only apply to spec from simple predicates, not other types of specs#2020-03-2700:34Alex Miller (Clojure team)it already does what you're asking, unless I misunderstand what you're asking#2020-03-2710:41bortexzTrue, I was confused by this in the schema and select wiki [::a ::b {:c (s/spec int?)}] , but in the unqualified keys section is shown as I wanted#2020-03-2721:59hiredmanhttps://gist.github.com/hiredman/60e5e6f0025cb38c8a7739d21f5a8ce6 is a little thing I've been toying with for using spec to define and check stateful protocols, meaning it defines how a client and server communicate, and certain client requests are only valid when the server is in certain states. given such a definition you stick it between your client and server and it checks the communication back and forth and gives errors if either side violates the protocol.#2020-03-3020:21robertfwI'm experimenting augmenting some of our fdefs with some additional keys, to make it easier to write fdefs that test async functions, or fns that have some stateful dependencies injected in the args. I thought I would ask in here to see if there is any strong reason not to do this, or any related changes coming in spec2 that I should be aware of#2020-03-3020:22Alex Miller (Clojure team)none that you can prepare for :)#2020-03-3020:22Alex Miller (Clojure team)fdefs will probably be changing significantly in spec 2#2020-03-3020:23robertfwah. any dev logs or documents that discuss possible directions they will be taking?#2020-03-3020:24Alex Miller (Clojure team)no, still in ideation#2020-03-3020:36robertfwgotcha. thanks šŸ™‚#2020-03-3118:10arohnerIs there a way to get there-exists behavior to compliment for-all? Context: I have a protocol, and a bunch of implementations (30+), and a test suite that is a series of test.check properties. A broken implementation slipped through, because it returned a pathological response: i.e. (s/nilable ::foo) always returned nil. Iā€™d like to assert during the test, ā€œthis can return nil or ::foo, but the test must return ::foo at least onceā€#2020-03-3118:11gfredericksYou could generate 10 or 50 calls or something#2020-03-3118:12gfredericksSomething large enough that it should never fail#2020-03-3118:12arohnertrue#2020-03-3118:12gfredericksIt does seem like that would be a useful feature though#2020-03-3118:13gfredericksBecause then you could test things of that sort without paying for 50 trials each time#2020-03-3118:13gfredericksA similar structure is using such-that and just generating a single thing#2020-03-3118:18arohnerI think Iā€™ve got it using such-that, thanks#2020-03-3118:18gfredericksThere's a bunch of dumb tests in t.c itself that use the 50 things approach; it's slow#2020-04-0115:34abdullahibraHi everyone,#2020-04-0115:34abdullahibrais there a way to convert json schema into clojure spec in dynamic way ?#2020-04-0115:35abdullahibrawhat i mean by dynamic here is: i read json schema files and convert them into equivalent clojure spec which could be used#2020-04-0120:11dominicmI would make a single spec per schema which was a predicate which validated the data using the schema?#2020-04-0115:45EddieDepending the exact specs you need, you might be able to write a utility using spec-tools to do accomplish your goal. For examples, https://github.com/metosin/spec-tools/blob/master/docs/02_data_specs.md.#2020-04-0316:22Ben GrabowSo far I've been using spec mostly for validating data structures "in-place", meaning I have an x, at the front door of my API I check if it's a valid format, and if it is I continue using x inside my API. There's another way of using spec that I don't really understand yet so I'm looking for good resources on it: I have an x, and at the front door of my API I check if it's a valid format and destructure/reformat it to the format expected internally in my API, y. y might have all the same data as x but perhaps arranged in a different nested structure or with different key names. Are there any good guides to using spec for this kind of thing? I guess this is using spec as a parser, rather than just a validator. This use case is hinted at (https://clojure.org/about/spec#_conform) but I haven't found a good set of examples yet.#2020-04-0316:25Alex Miller (Clojure team)Thatā€™s what conform is for#2020-04-0316:25Alex Miller (Clojure team)ā€œIs this valid, and if so, why?ā€
#2020-04-0316:26Alex Miller (Clojure team)With the caveat that it is NOT for ā€œperform arbitrary transformation to other structureā€#2020-04-0316:27Ben GrabowI'm trying to understand the situations it's good for. e.g. https://juxt.pro/blog/posts/parsing-with-clojure-spec.html#2020-04-0316:27Alex Miller (Clojure team)If you want that, use conform + standard Clojure data transformation#2020-04-0316:27Ben GrabowIn that example, a sequence of characters is being parsed into a map that describes the components.#2020-04-0316:27Ben GrabowI have one map with a nested structure that I want to transform to another map with a flat structure.#2020-04-0316:27Alex Miller (Clojure team)Yeah, dont do that#2020-04-0316:28Alex Miller (Clojure team)(That applies to the string case)#2020-04-0316:28Alex Miller (Clojure team)If you want to transform maps, use Clojure stuff to do that#2020-04-0316:28Alex Miller (Clojure team)If you want to validate, use valid?#2020-04-0316:29Alex Miller (Clojure team)If you want to know why something is valid wrt options and alternatives, use conform#2020-04-0316:29Ben GrabowIt's that last part I'm looking for more content on.#2020-04-0316:30Ben GrabowI don't quite understand when I'm dealing with a situation that's good for conform and when I'm not.#2020-04-0316:33Ben GrabowI remember in one of Rich's talks he mentioned David Nolen rewriting part of the cljs parser in spec. Is that code publicly available somewhere? It sounds like a powerful use case.#2020-04-0316:41Ben Grabowhttp://blog.cognitect.com/blog/2017/1/3/spec-destructuring#2020-04-0316:42Alex Miller (Clojure team)Well thatā€™s mine :)#2020-04-0316:42Alex Miller (Clojure team)Using it to parse the input to a complex macro is a good fit#2020-04-0316:42Alex Miller (Clojure team)Itā€™s particularly good for ā€œsyntaxā€ stuff#2020-04-0316:43Alex Miller (Clojure team)If your input is mostly maps and stuff, probably not as useful#2020-04-0316:45Ben GrabowThe difference in the way my input map is structured vs how my output map is structured feels like syntax to me so I wonder where the distinction lies#2020-04-0316:47Ben GrabowThere's the idea that a map is just a seq of kv pairs. What if the spec is an s/cat of kv-pairs instead of an s/keys?#2020-04-0316:49Alex Miller (Clojure team)Syntax is positional (this is kind of present in the Greek roots of the word even)#2020-04-0316:49Alex Miller (Clojure team)Maps are not positional#2020-04-0316:50Alex Miller (Clojure team)Position in syntax is implicit#2020-04-0316:50Alex Miller (Clojure team)Spec regex ops describe that, and the conformed values tell you how it was interpreted#2020-04-0316:51Alex Miller (Clojure team)Must of the map specs essentially return the map if itā€™s valid - there was no thing to figure out and tell you about#2020-04-0316:53Alex Miller (Clojure team)Specing a map as a seq of kv pairs is not the same thing as maps are inherently unordered#2020-04-0316:53Alex Miller (Clojure team)You canā€™t rely on position to describe the structure of the map#2020-04-0316:54Ben Grabowhmm, that's a good point#2020-04-0316:54Alex Miller (Clojure team)The regex specs like cat will even error if you try to do this, for this reason#2020-04-0316:56Ben GrabowI had in mind using a big s/or to describe all the possible keys. Which brings up an interesting question, how does the positionality of syntax impact s/or? I usually reach for s/or when I have a couple different possibilities for my input, like a union type, and I'm usually frustrated that I'm pushed towards destructuring and naming the options instead of conforming to just the value.#2020-04-0316:56Ben Grabow(s/or :even even? :small #(< % 42)) from the docstring as an example.#2020-04-0316:57Ben GrabowI wish the API were (s/or even? #(< % 42)) for the use I have in mind#2020-04-0316:57Ben GrabowI feel like there's something I don't "get" in the reasons why s/or asks for keywords to name the options.#2020-04-0316:59Alex Miller (Clojure team)or gives you back a map entry of tag (key) and value#2020-04-0317:02Ben GrabowSure, I get that. I'm wondering why it does that when it doesn't seem to be about processing the position of a thing. Instead it seems to be about processing the alternatives of a single thing.#2020-04-0317:27Alex Miller (Clojure team)well the idea is the same - tell me what choice you made when there was an alternative#2020-04-0317:28Alex Miller (Clojure team)in practice, it is often a source of issues in deeply nested data where that's the only conformed thing so I kind of would like to have some more options in spec 2#2020-04-0317:28Alex Miller (Clojure team)whether that's an or variant, or a s/nonconforming, or something else#2020-04-0317:29Alex Miller (Clojure team)but that's down the list a bit and we haven't talked about it#2020-04-0317:33Ben GrabowGlad to hear the sentiment is shared.#2020-04-0317:35Ben GrabowI often run into situations where a map is mostly just a set of specific things so s/keys works, until we add uuids into the mix and they could be strings or uuid objects. I'm using a custom conformer to get around the s/or destructuring for that case but it would be nice to have a more general solution.#2020-04-0317:39Alex Miller (Clojure team)one workaround right now is to wrap those s/or's in s/nonconforming#2020-04-0317:39Alex Miller (Clojure team)which is not documented, but exists#2020-04-0317:49Ben GrabowNice!#2020-04-0317:49Ben Grabow
(comment
  (s/conform (s/nonconforming (s/or :x uuid? :x ::sc/uuid-string)) (UUID/randomUUID))
  (s/conform (s/nonconforming (s/or :x uuid? :x ::sc/uuid-string)) (str (UUID/randomUUID)))
  (gen/generate (s/gen (s/nonconforming (s/or :x uuid? :x ::sc/uuid-string)))))
All these work as expected. The first two produce just the value, without the keyword tag. The third generates both strings and uuids.
#2020-04-0317:51Ben GrabowMy ::uuid-string spec is still a little complicated to make the generator work but I think it could be simplified.#2020-04-0819:06dspiteselfI see spec alpha 2 adds select. Will select have a special knowledge of coll-of so that you could select a map of a vector of maps?#2020-04-0819:07dspiteselfLike pull does across datomic cardinality many attributes.#2020-04-0819:07Alex Miller (Clojure team)Currently, no, but thatā€™s under discussion#2020-04-0820:21dspiteselfThanks I really like what is going on over there. That is the only problem left I see that limits our uses.#2020-04-0820:29Alex Miller (Clojure team)certainly there are a lot of similarities to pull#2020-04-1113:46credulousHi! This is definitely a novice question, but Iā€™ve struggled for a day or so trying to figure out the best way to use spec in my re-frame application. My problems stem from not grokking spec I think.#2020-04-1113:46credulousIā€™m using my app db to store form information under a :form key. Forms are stored by form-id.#2020-04-1113:48credulous#2020-04-1113:52credulousIn each field the ā€œerrorā€ is set based on a spec that gets checked on each edit:
(s/def ::id (s/and string? #(> (count %) 0)) )
(s/def ::team (s/and string? #(> (count %) 0)) )
(s/def ::nickname (s/and string? #(> (count %) 0)))
(s/def ::password (s/and string? #(> (count %) 7)) )
#2020-04-1113:52credulous(Lengths are set to zero but will change)#2020-04-1113:54credulousBut I also want to define a spec for the form as a whole, so I can pass the ā€œsignupā€ form above to (s/valid?). But I canā€™t figure out how to define the structure above, where signup has four keys, each of which have the same structure#2020-04-1113:55credulousin :signup :team for example, I want to say signup has a map that requires a :team key, and the team key is a map that has a :value key that is defined by ::team in the snippet pasted above#2020-04-1116:22shooitIā€™m not sure that I understand 100% but you might need to put each :value key into a different namespace so that they can be a distinct spec e.g :foo/value, :bar/value. You could also check out multi-spec for multimethod like dispatch on the submaps #2020-04-1113:56credulousI canā€™t define a ::value spec to include in (s/keys) for :team - because what :value is is different for each sibling key in :signup#2020-04-1116:26Alex Miller (Clojure team)spec is primarily designed to associate semantic specs with namespaced keys, so by using :value in each of these places (instead of :id/value or whatever), you are outside the sweet spot#2020-04-1116:27Alex Miller (Clojure team)you can do this with s/keys and :req-un for unqualified keys though, but you'll need to match up the name parts of the specs#2020-04-1116:32Alex Miller (Clojure team)
(s/def :team/value ::team)  ;; same for the others
(s/def :team/team (s/keys :req-un [::team/value])) ;; same for the others
(s/def ::signup (s/keys :req-un [:team/team ..etc..]))
#2020-04-1116:33Alex Miller (Clojure team)as I said, this is outside the expected pattern so it's more work for sure#2020-04-1118:08credulousThanks Alex. What is the preferred way to satisfy the goal I have here, which is to independently validate the form contents and the form data as a whole? Is the correct answer to my confusion ā€œread moreā€? šŸ™‚#2020-04-1119:17Alex Miller (Clojure team)the preferred way is to have namespaced attributes with one meaning#2020-04-1308:21AronHi, I would like to do some "spec driven development" in cljs, where should I start? Is it the same for clojure and clojurescript?#2020-04-1413:56colinkahnI think in general itā€™s similar. For UI applications the boundries that you want to focus your specs on are different of course. I consider anything coming over http and websockets a good candidate for checking with spec and all of my component props as well. If youā€™re using Reagent the later is easy because your components are already functions and you can just us s/fdef . Another consideration is I like to keep my specs separate from my other code on the F/E. Instead I write separate spec files that require the namespaces that Iā€™m specing. Then you can make a preloads file that imports those spec files and runs st/instrument to get development-time feedback.#2020-04-1414:07AronI am not using reagent because it has no support of modern react, I am using helix.#2020-04-1414:09AronThanks for the note on 'separate', I was just thinking about that, I saw a video with @U064X3EF3 where it was noted to not use these in production and I am not quite sure how to do that. I thought that the compiler would remove these when I build for production and I will not have to maintain two versions of my code#2020-04-1414:18colinkahnIt looks like the defnc macro in helix eventually produces a function with [props ?ref] that you can spec like a Reagent component: https://github.com/Lokeh/helix/blob/master/src/helix/core.clj#L80-L87#2020-04-1414:19AronHow many years before I can do what you just did @U0CLLU3QT ? : D#2020-04-1414:20AronI can read the code, but all these ^s and ~ confuse the hell out of me.#2020-04-1414:21Aronand yeah, I am fairly sure I can spec it, it's more about the context/reducer situation that I am somewhat unsure. I think I can do state transition functions and spec those and then I can create spec for the state as input and output before and after the transition.#2020-04-1414:24colinkahnThe ā€œseparateā€ approach for me works because I agree these are more development-time tools and donā€™t really belong in my production source code. The compiler I believe will remove most of it (Iā€™ve seen tricks where frameworks using Google Closure features to help with this as well) but I think certain parts of the spec library itself cannot be dead code eliminated (though donā€™t quote me on that, and probably better to try it yourself). For having two versions, this isnā€™t necessary. For example: app.components.cljs
(ns app.components)

(defn my-component [props]
  ...stuff)
app.components.specs.cljs
(ns app.components.specs
 (:require [app.components :as c]))

(s/fdef c/my-component
  :args (s/cat ...))
#2020-04-1414:25colinkahnhttps://clojurians.slack.com/archives/C1B1BB2Q3/p1586873997105600?thread_ts=1586766083.099100&amp;cid=C1B1BB2Q3 You have to go through your ā€œmacrosā€ phase šŸ˜‰#2020-04-1414:26AronI haven't even started that... : )#2020-04-1414:29colinkahnThis is the preloads feature I was referring too: https://clojurescript.org/reference/compiler-options#preloads Once you have your *.specs namespaces I make a single specs.preload or something that is roughly:
(ns app.specs.preload
  (:require [clojure.spec.test.alpha :as st]
            ; all your spec namespaces here
            [app.component.specs]))

(st/instrument)
#2020-04-1506:53AronThis is somewhat different for shadow-cljs, right?#2020-04-1515:12colinkahnItā€™s pretty much the same, youā€™d specify the preloads in your build defined in shadow-cljs.edn#2020-04-1515:13colinkahnhttps://shadow-cljs.github.io/docs/UsersGuide.html#_preloads#2020-04-1515:14colinkahnActually yeah, itā€™s under a different key#2020-04-1410:00Jim NewtonI've never used spec before. Can someone help me with a recipe ? for a given integerĀ `n`Ā  and a given sequence, possibly heterogeneous, whether the sequence is of the formĀ `n`Ā many optional integers followed by exactlyĀ `n`Ā integers?Ā Ā  some analogous expression to "a?^8a^8" which I can substitute any fixed integer for 8 ?#2020-04-1411:29gfredericksis that the same thing as "between 8 and 16 integers"?#2020-04-1418:15zaneHow common is it to use spec (specifically s/or + conform + (`case` / multimethod / ā€¦)) to define polymorphic functions, and when would / wouldn't doing so be appropriate?#2020-04-1418:31Alex Miller (Clojure team)I have no way to judge frequency, but seems ok to me. Itā€™s probably not the fastest option.#2020-04-1509:28flowthingI often test my specs in the REPL by doing something like (clojure.test.check.generators/generate (clojure.spec.alpha/gen ::foo)). Sometimes I mess up the spec such that the generator can't generate a value from my spec, and I get Couldn't satisfy such-that predicate after 100 tries.. Is there an easier way to debug cases like this than process of elimination? If the spec is fairly large, it's can be pretty time-consuming and difficult to find the offending spec. The exception and the stack trace aren't of much help.#2020-04-1512:38Aron'test in the REPL' means you type it in? #2020-04-1512:41flowthingWell, I evaluate the forms in my editor, but yeah.#2020-04-1512:50Aronare you by chance using neovim and shadowcljs?#2020-04-1512:50flowthingCursive, but I have dabbled with Conjure a bit.#2020-04-1512:52Aronhttps://github.com/thheller/shadow-cljs/issues/508#issuecomment-565021177 YAY. so i can use conjure finaly#2020-04-1512:52Aronshould've paid attention, it's been MONTHS#2020-04-1512:52Aronthanks flowthing! : )#2020-04-1515:12Aronwell, nope.#2020-04-1512:56gfrederickssomebody needs to write up a summary of what the such-that error means for spec users#2020-04-1513:07potetm@gfredericks It looks like @flowthing knows what it means. The question is: is there an easier way to find it than manual searching?#2020-04-1513:07potetm(Iā€™m curious your answer to that as well.)#2020-04-1513:08gfredericksI don't think so. There's a feature added to such-that that enables spec to give better errors, but that was never implemented in spec#2020-04-1513:23Alex Miller (Clojure team)s/was never/has not yet been/#2020-04-1513:25gfredericksI was uncertain whether these situation with spec2 was similar or not#2020-04-1513:48Alex Miller (Clojure team)no changes for this yet#2020-04-1513:48Alex Miller (Clojure team)I did a little spike of it a while back but it was less obvious than I expected#2020-04-1513:48mpenetsorry to ask again: any eta for spec2 ?#2020-04-1513:59Alex Miller (Clojure team)no, kind of been dormant recently but we'll be back to it soon I hope#2020-04-1517:02seancorfieldRe: such-that -- about the only thing I've found that helps avoid that is relentlessly running s/exercise on each spec as I write it, so that I trip over a "Couldn't satisfy" error at the earliest possible point and therefore have a really small surface of possibilities to investigate/debug. It's not perfect but it helps me, and it fits naturally in with a very tight RDD workflow that has you evaluating every piece of code as you write it.#2020-04-1517:18Frank HenardHey Everyone, I would love your input on my SO question: https://stackoverflow.com/q/61234691/59439#2020-04-1518:36Alex Miller (Clojure team)they both ultimately call the same function so I would expect perf to be pretty similar right now. we have considered fast-pathing a separate valid? path in spec 2 though and I would expect that to be the faster option.#2020-04-1519:08Frank HenardGot it, thanks!#2020-04-1519:11Alex Miller (Clojure team)the big differences are that a) with valid?, you can fail fast and b) with explain, you have to track a lot of additional info and accumulate it for reporting#2020-04-1517:43vlaaadIsn't it sort of obvious what specs might throw such-that errors? The ones that use s/and with second condition rejecting most values from the set specified by first condition?#2020-04-1606:49flowthingYes, it's often obvious, but not always, if the spec is large. An error message that points to the location where it originates would certainly be helpful. Anyway, I think the approach @U04V70XH6 proposed is the way to go for the time being.#2020-04-1518:18zaneI think it's a good idea to avoid using the word "obvious" in this kind of context. Spec is pretty complicated, and people come to it from a variety of backgrounds.#2020-04-1518:42bbucciantiCould be possible to use spec to validate lambda expressions?#2020-04-1518:42bbucciantiI mean, I'm building a lambda calculus reducer#2020-04-1518:43bbucciantiAnd I'm thinking if would be possible to use property based testing in order to see if it's reducing properly#2020-04-1518:43bbucciantiBut I never used spec and I don't know (yet) if that would be possible#2020-04-1518:48Alex Miller (Clojure team)s/fspec is the tool provided, but it's often more trouble than it's worth when using gen#2020-04-1518:51bbucciantiDo you mean than it doesn't ve of any value building such a thing?#2020-04-1518:51bbuccianti*be#2020-04-1518:59Alex Miller (Clojure team)spec is not particularly useful for checking functions as args in a deep way#2020-04-1519:01Aronout of curiosity, is there anything that can do that? Sounds like something that I call the halting problem?#2020-04-1519:14nwjsmith@U0VQ4N5EE there are several property-based testing tools that can do this, they generate functions that satisfy the function's type#2020-04-1519:20AronThen I misunderstood what functions as args in a deep way means, sorry.#2020-04-1519:18ikitommifast-path to valid? is a good idea. Any other new ideas for Spec2? New syntax for fn specs still wip?#2020-04-1521:17Alex Miller (Clojure team)too many ideas :) fn specs dormant atm, working on other stuff, but we'll get back to it#2020-04-1521:47kennyIs this a known bug?
(s/def ::foo
  (s/with-gen #(string? %) #(s/gen string?)))
(s/form ::foo)
=> :clojure.spec.alpha/unknown
#2020-04-1521:52Alex Miller (Clojure team)Yes#2020-04-1521:52Alex Miller (Clojure team)Not going to be fixed, but not an issue in spec 2#2020-04-1521:53kennyThis appears to be an ok workaround:
(s/form (s/spec #(string? %) :gen #(s/gen string?)))
=> (clojure.core/fn [%] (clojure.core/string? %))
#2020-04-1521:54seancorfield@kenny Why use #(string? %) instead of just string??#2020-04-1521:55kennyFor demonstrating the bug, nothing else šŸ™‚#2020-04-1522:37seancorfieldAh, so it works just fine with string??#2020-04-1522:38kennyYep#2020-04-1522:38kennyI encountered this with a spec defined with a #(re-find ...)#2020-04-1812:23JoeIs this the correct way to spec a non-empty collection?
(s/def ::things (s/coll-of ::thing :min-count 1))
#2020-04-1813:34shooitThat would do it. The other thing that some to mind is from the sequential specs: s/+. It depends on context which is preferred. Just a non-empty collection of things? Your s/coll-of solution is it. A non-empty sequence of things as part of a larger pattern? Look to the sequential specs for their regex like functionality. #2020-04-1817:32prncSpec rationale quote Iā€™m pondering > Invariably, people will try to use a specification system to detail implementation decisions, but they do so to their detriment. The best and most useful specs (and interfaces) are related to purely information aspects. https://clojure.org/about/spec#_informational_vs_implementational Trying to get clear on the distinction ā€” what would be the canonical example of the offending type of use and how far you can push the boundary so to speak? Is this mostly about trying to enforce implementation a la types (an thus cussedness/rigidity etc.) ?#2020-04-1818:00Alex Miller (Clojure team)Clojure core functions are a good example#2020-04-1818:01Alex Miller (Clojure team)You can spec a function as talking PersistentVector (bad, concrete, may change) or as satisfying vector? or seq? or coll? or sequential? (depends on the fn of course)#2020-04-1818:02Alex Miller (Clojure team)And itā€™s common to try to spec what exactly it does now vs leaving open for future#2020-04-1901:47salam@alexmiller It seems like the Clojure Spec 1 alpha JAR is compiled against Java 1.5 (class file version 49). Is that intentional? It seemed a bit odd now that Clojure itself targets Java 1.8 (class file version 52).#2020-04-1901:48Alex Miller (Clojure team)Which version in particular?#2020-04-1901:50salamthis is what's on my class path: spec.alpha-0.2.176.jar#2020-04-1901:51salami took a look at the pom.xml (https://github.com/clojure/spec.alpha/blob/master/pom.xml) file for spec.alpha and the absence of <target.version>1.8</target.version> in the <properties> section seems to confirm that.#2020-04-1901:52Alex Miller (Clojure team)Iā€™m not at a computer but certainly older versions of the parent pom could have produced that#2020-04-1901:55salamI see. I was experimenting with https://openjdk.java.net/jeps/310 and noticed the following warning and figured I should at least let you know:
OpenJDK 64-Bit Server VM warning: Pre JDK 6 class not supported by CDS: 49.0 clojure/spec/alpha/Spec
Preload Warning: Cannot find clojure/spec/alpha/Spec
#2020-04-1901:58Alex Miller (Clojure team)There is a newer version that I suspect is newer bytecode. - 0.2.187#2020-04-1902:00Alex Miller (Clojure team)Spec was introduced in the same release that bumped to java 1.8 bytecode and it took a while for the contrib libs to catch up#2020-04-1902:04salamah, i see. it appears that the release version 1.10.1 of the clojure.jar still depends on the version that i have: https://search.maven.org/artifact/org.clojure/clojure/1.10.1/jar#2020-04-1902:33Alex Miller (Clojure team)You can add an explicit dependency on the newer spec.alpha version to override #2020-04-1902:35salamyep, i'll go with that. thank you.#2020-04-1903:17salamok, reporting back here. it turns out [org.clojure/spec.alpha "0.2.187"] is compiled against java 1.5 as well.#2020-04-1904:27seancorfieldConfirmed, if I'm reading this right. class version 49 for spec.alpha 0.2.187, but class version 52 for clojure itself. The latter is Java 8.#2020-04-1908:13practicalli-johnspec version 1 or version 2, that is the question šŸ™‚ My default assuming is spec 1 I am starting a new project, which I will mainly use spec for data structures I design and pass to functions as arguments. Its a data science visualisation project so there is a lot of data. I will also write specs for the functions that for the API for each namespace. Just checking that I am not missing an obvious benefit of using spec 2. Thank you.#2020-04-1908:38mpenet1 for sure. v2 seems still wip, especially fn annotations which apparently will be improved/modified#2020-04-1908:38didibusSpec 2 is experimental and subject to breaking change#2020-04-1908:38didibusSo unless you are doing this for fun, go with Spec 1#2020-04-2021:42Aronso I am reading docs and I am sure I will figure this out too, but someone could help me here a lot with a shortcut to speccing a hashmap that has 3 levels, the first is a fixed set of keys, second also a fixed set of keys, and the third is depending on the key either a string or a vector of keywors. so a valid example would be {:firstlevelkey {:label "string" :fields [:fieldone :fieldtwo]}} and I just want to say that on the first level only certain keys are allowed( one predefined set of keywords), second level always have to be both a :label and :fields and label always have to be string and the :fields always needs to be a vector of keywords from another set of predefined keys#2020-04-2021:42Aroni don't need a full solution, just tell me please what to use so I can read up on those specifics because it's taking forever to sieve through šŸ™‚#2020-04-2021:43AronI am trying to google > clojure spec "-un" and it's not going well#2020-04-2021:53Aronthe more I read the less I understand.#2020-04-2022:06seancorfield> I just want to say that on the first level only certain keys are allowed This isn't something Spec favors -- it follows the concept of "open for extension". So you will need additional predicates with s/and to restrict keys (and remember that it's not really "idiomatic" to do that).#2020-04-2022:07AronOk, good to know, so I won't force it if it's not expected#2020-04-2022:11seancorfieldThe whole spec is going to end up looking something like this:
(s/def :key/label string?)
(s/def :field/keys #{:fieldone :fieldtwo ,,,}) ; your set of predefined keys allowed
(s/def :key/fields (s/coll-of :field/keys :kind vector?)) ; maybe you want min and/or max counts too?
(s/def :second/level (s/keys :req-un [:key/label :key/fields]))
(s/def :top/keys #{:firstlevelkey ,,,}) ; available top-level keys
(s/def :top/level (s/map-of :top/keys :second/level))
#2020-04-2022:12seancorfieldIf your set of keys isn't known at spec-time, it's a bit harder but that's probably a good first cut outline to start with.#2020-04-2022:12Aronit's known šŸ™‚ luckily#2020-04-2022:12seancorfield(with better names, of course)#2020-04-2022:13seancorfieldThat actually solved the restricted set of top-level keys, but at the expense of not requiring a minimum set either.#2020-04-2022:13Aronthe naming of keywords is arbitrary, right? so I could do just :label instead of :key/label, it just pays off to have very specific names of symbols so there is no collision?#2020-04-2022:14seancorfieldRight. Qualified names help avoid collisions.#2020-04-2022:14seancorfieldOnly s/keys cares -- :req-un treats the qualified names as just their unqualified part (`:req` would pay attention to the qualifier as well).#2020-04-2022:15Aronit's quite hard to grasp the pattern#2020-04-2022:15AronI need to experiment with this, thank you very much#2020-04-2022:17Aronone more question that I think I know the answer for but still, if I want to spec anything, I always have to give a name to both the thing I want to spec and either use the definitions in place or create a named spec, right? So there is no way to write specs that would fit existing names automatically.#2020-04-2022:21seancorfieldI'm not quite sure I'm following your question but I think the answer is "no, no way to do that in Spec 1".#2020-04-2022:22seancorfieldSpec 2 does allow unqualified keys in a schema to have inline specs (predicates).#2020-04-2022:22Aronin app namespace I define something like (def whatever "string") and then in app.specs namespace I say (s/def whatever string?) and there is some magic to match them#2020-04-2022:23Aronprobably a bad idea#2020-04-2022:25seancorfieldSpecs are in a completely separate "space" from regular Var names so you need to associate them yourself explicitly.#2020-04-2022:26seancorfieldWith functions, that's s/fdef but there's nothing for other Vars.#2020-04-2022:26seancorfieldYou need to use s/valid? or s/conform to "apply" a spec to a value at runtime.#2020-04-2022:26seancorfield(does that answer your question?)#2020-04-2022:27Aronyes : )#2020-04-2101:12AlexI'm having some trouble with adding a spec for a keyword which is supposed to represent a function. I tried this
(s/fdef ::on-change
  :args (s/cat :value :option/id))

(s/def ::props (s/keys :req-un [::options
                                ::value]
                       :opt-un [::class
                                ::centered?
                                ::on-change
                                ::variant]))

(s/fdef tabs
  :args (s/cat :props ::props))
#2020-04-2101:12Alex> Var clojure.test.check.properties/for-all* does not exist, clojure.test.check.properties never required#2020-04-2101:12Alexfrom trying to pass ::on-change to fdef#2020-04-2101:14AlexAfter reading the API docs, it seems fdef can only take a symbol naming a function. My question is... how can I create a spec for the function that is passed as :on-change?#2020-04-2101:17robertfwIt's funny you ask, I was just arriving to ask something similar. I have a spec describing a map, of which some keys are functions. I'd love to be able to specify that those functions should conform to a given fdef, but not sure how to accomplish that. As is, I just have the keys referring to the functions specced using fn?#2020-04-2101:19AlexNow I don't feel too silly asking šŸ™‚#2020-04-2101:26Alex Miller (Clojure team)use ifn? not fn?#2020-04-2101:27Alex Miller (Clojure team)you can use fspec to do this as well, but there are some big caveats for generation#2020-04-2101:27Alex Miller (Clojure team)personally, I have not found it to be worth doing fspec over ifn?#2020-04-2101:32Alex@alexmiller Does ifn? allow you to specify the shape of the arguments that are passed in? I can't seem to find any examples doing so#2020-04-2101:37Alex Miller (Clojure team)no#2020-04-2101:38Alex Miller (Clojure team)you can do that with fspec (same args as fdef)#2020-04-2101:45AlexTried a simple fspec but that doesn't seem to compile. Hmm
(s/fspec ::on-change
         :args (s/cat :value number?)
         :ret any?)
#2020-04-2102:37kennyRemove the first arg:
(s/fspec :args (s/cat :value number?)
         :ret any?)
#2020-04-2103:06AlexThanks! How do I associate it to the keyword which will use the spec?#2020-04-2111:40sgepigon
(s/def ::on-change
  (s/fspec :args (s/cat :value number?)
           :ret any?))
#2020-04-2114:09AlexI get this error when I try that. Is this expected? > Uncaught Error: Var clojure.test.check.properties/for-all* does not exist, clojure.test.check.properties never required#2020-04-2114:09kennyfspec uses test.check to ensure the correctness of the spec'ed input.#2020-04-2123:10AlexThank you!#2020-04-2107:23Aronhow can I spec transient things like values of let bindings?#2020-04-2109:06Ben SlessIt might be terrible but I recently had a similar issue, went for this solution:
(let [x (expr ...)]
  (eval
   `(s/def ::my-spec
      (s/keys
       :req-un
       [~x]))))
#2020-04-2109:06AronšŸ˜„#2020-04-2109:07Aroni am not sure about terrible but it's scary#2020-04-2109:07Ben Slessit is. If you can convince yourself the expr doesn't do any IO then you can sleep well imo#2020-04-2109:09Ben SlessIn my case I had to create a qualified keyword who's name starts with a number. seems like :1foo is a valid kw but :user/1foo can't be read by the reader. (keyword "user" "1foo") works#2020-04-2109:12Aronthis is supposed to run in a browser šŸ™‚#2020-04-2109:13Ben Slessah#2020-04-2107:33AronI see the suggestion to use s/assert in docs, but isn't that also something that I should only do in development and not make it part of the code that might be shipped eventually?#2020-04-2112:33Alex Miller (Clojure team)Spec asserts can be turned off and even compiled out in prod code#2020-04-2112:47Aronthanks, I realized there has to be some solution because there were blogposts about how to use it, but now I know where to look for it.#2020-04-2109:02Ben SlessIs it possible to spec/validate java collections? this naive example doesn't work
(s/def ::foo int?)
(s/def ::bar string?)
(s/def ::m (s/keys :req [::foo ::bar]))

user=> (s/valid? ::m {::foo 1 ::bar "2"})
true
user=> (s/valid? ::m (java.util.HashMap. {::foo 1 ::bar "2"}))
false
#2020-04-2112:32Alex Miller (Clojure team)Spec does not cover Java collections so youā€™d have to pour one into a Clojure collection first#2020-04-2117:35Ben SlessThis is probably a terrible idea, but I guess everything is possible if you try hard enough#2020-04-2117:52Ben SlessIt would be nice, maybe in spec2, if the meanings of maps and sequences were relaxed a bit. But I'm coming at it from an esoteric use case#2020-04-2117:52Alex Miller (Clojure team)that seems way harder than converting the HashMap into a Clojure map#2020-04-2117:53Alex Miller (Clojure team)we do not have plans to support Java colls in spec 2#2020-04-2117:54Alex Miller (Clojure team)(s/valid? ::m (into {} (java.util.HashMap. {::foo 1 ::bar "2"})))#2020-04-2117:56Ben SlessThat's the trivial case, in reality I might be dealing with an arbitrarily nested java collection šŸ˜ž#2020-04-2118:05Ben SlessIt's also unfortunate because it creates a bit of a mismatch between the validation mechanism and the applied predicates. In this example, get works on java.util.Map, so does seq, so conform* can do its work. The only "hurdle" is map?, which is a pretty stringent requirement. Why not expand spec a bit more towards reflecting intent and less implementation?#2020-04-2118:15Alex Miller (Clojure team)the intent is to validate clojure data#2020-04-2118:16Alex Miller (Clojure team)validating java colls was out of scope for us#2020-04-2116:23eval2020What spec could validate that a map contains at least one true value?#2020-04-2116:25Alex Miller (Clojure team)any function that takes a value can be a spec#2020-04-2116:25Alex Miller (Clojure team)so rephrasing - what function could validate that a map contains at least one true value?#2020-04-2116:25Alex Miller (Clojure team)write that function and you're done#2020-04-2116:28eval2020Ah, I was overthinking this - thanks#2020-04-2201:08Aronshould spec have their own ns an import app ns? or should the app import the spec ns? I am having this issue that circular dependencies seem unavoidable#2020-04-2201:12seancorfield@ashnur The answer is "it depends". You should be able to find a partial ordering of dependencies that would allow you to split things into different namespaces -- probably three, so you have an "app" ns, a "spec" ns, and a "predicate" or "utility" ns that contains things the specs depend on, and the app can also depend on. But it may not be worth doing that analysis so having the specs in the same ns as your app code may just be easier.#2020-04-2201:13seancorfieldIf you're writing code that you want to be able to run on older versions of Clojure(Script) that don't include spec, you have to break them out into optional namespaces so that users can opt in if they want the specs -- but that's really only for library writers.#2020-04-2201:14AronI am running shadow-cljs#2020-04-2201:14seancorfieldAt work, I try to have the data specs in a separate namespace (and do the work to untangle any problematic dependencies) but the function specs will generally go in the same namespace as the functions they are for.#2020-04-2201:14AronI initially imagined having files for data/specs/components/react-app all separated#2020-04-2201:15Aroncan I have multiple files all in the same namespace?#2020-04-2201:21seancorfieldThe simple answer to that question is no. The complex answer is there are ways to spread a namespace across multiple files but I don't think it will solve the problem you have here.#2020-04-2201:22Aronok, no it is šŸ™‚ I want to keep it simple#2020-04-2201:22AronI must be doing something silly again because I am unable to refer to my s/defs from the spec file/ns#2020-04-2201:24seancorfieldCan you share some code to illustrate the problem? Is this in a project up on GitHub?#2020-04-2201:29Aronin spec file I wanted to do something like (s/def :my.specs/one-spec identity) then in app file ns require [my.specs :as myspecs]#2020-04-2201:31Aronit's closed source but I can make a new repo with completely new code, it might just take a couple of weeks before i have the time#2020-04-2201:31AronšŸ˜„ only half kidding though#2020-04-2202:00seancorfieldOK. And what can't you do in the app file ns?#2020-04-2202:01seancorfield(sorry for the delay in responding -- was dealing with an issue someone just opened on clj-new that I needed to repro!)#2020-04-2202:04seancorfieldIn you app file ns, after that require, you should be able to reference that spec either as :my.specs/one-spec (i.e., its fully-qualified name) or ::myspecs/one-spec using the auto-resolve syntax which will expand ::myspecs to :my.specs using the alias of the namespace... the latter requires that the actual spec name really matches the spec ns in the qualifier. (and you could just (s/def ::one-spec identity) in the spec ns to have the spec automatically match the ns of the namespace it is defined in).#2020-04-2202:04AronI feel bad if you apologize for such stuff, I realize people have lives beyond what is visible here : )#2020-04-2202:04AronI had to do (def (s/def ... and it worked#2020-04-2202:04Aronfor some reason I thought just (s/def would be enough#2020-04-2202:05AronI am also in utter confusion about how keywords work, not just in spec but other places. I know it's a symbol that retuns itself, but apparently it also can hold specs#2020-04-2202:06seancorfields/def updates a registry behind the scenes which is essentially a map from keywords to spec objects. The keywords don't "hold" specs -- the association is done separately.#2020-04-2202:07Arongotcha#2020-04-2202:08seancorfieldI have no idea what (def (s/def ...)) would do... but it's definitely not the right thing to do.#2020-04-2202:10Aronsorry, (def name-of-spec (s/def#2020-04-2202:10seancorfieldThat's what I assumed -- not the right thing to do.#2020-04-2202:12Aronso much black magic šŸ˜ž#2020-04-2202:12seancorfieldThe "name-of-spec" is the keyword. That's how you refer to it in code that calls s/valid? or something.#2020-04-2202:12Aronso specs can only be registered to keywords?#2020-04-2202:13seancorfieldYes.#2020-04-2202:13Arondocs said resolvable symbol k#2020-04-2202:13AronI am confused, that's all#2020-04-2202:14seancorfieldLink? So I can see what the context is.#2020-04-2202:14Aronhttps://clojuredocs.org/clojure.spec.alpha/def#2020-04-2202:15Aronhow can I use :require :refer to import a spec definition?#2020-04-2202:16seancorfieldAh, probably because of function specs: s/fdef uses the fully-qualified function name (symbol) so that's probably why s/def will accept a symbol.#2020-04-2202:16seancorfieldYou can't refer in a single spec.#2020-04-2202:16seancorfieldOnce you've loaded the ns in which specs are defined, they are globally available.#2020-04-2202:16seancorfieldThey're not like Vars or functions.#2020-04-2202:17seancorfieldSo you could require the specs ns in whatever you app entry point is and you would have access to those specs (as keywords) everywhere in your program.#2020-04-2202:21AronI can't even require them at this point#2020-04-2202:21Aronhow to "load the ns", I usually just do :refer or :as, neither which seems to work here#2020-04-2202:22seancorfieldJust require the namespace like any other. Then the specs are in the registry and available globally.#2020-04-2202:22Aronoh ok, so i can just write the ns :as, but then i have to completely ignore it#2020-04-2202:23Aronand instead always have to read the file of the spec, because ns-publics don't list it#2020-04-2202:24seancorfieldYeah, if you're not using the :: auto-resolve syntax, you don't even need :as#2020-04-2202:24Aronand it's available via some (black, that is, unobservable) magic#2020-04-2202:24seancorfield(:require ... [my.specs] ...)#2020-04-2202:24Aronbut how? just (:require [my.specs])?#2020-04-2202:24Aronok šŸ™‚#2020-04-2202:24Aronthanks again#2020-04-2202:25Aronwhen I think I understand something, it turns out that thing is specific to that particular area, and something else using the same syntax behaves completely differently#2020-04-2202:25seancorfieldHere's where the docs talk about the registry https://clojure.org/guides/spec#_registry -- don't know if that'll help?#2020-04-2202:27Aronit will help šŸ™‚#2020-04-2202:28Aronthe problem, which you have to realize is a problem, that I've had this page open for days and I have read it through from top to bottom and bottom to up several times#2020-04-2202:29seancorfieldDon't worry, sometimes it taking a lot of reading the Clojure docs before it actually makes sense and you get to see a big picture!#2020-04-2202:30AronI see the big picture, I just don't see the details.#2020-04-2202:30AronšŸ™‚ i've been trying to use clojure for 8 years at least#2020-04-2202:31Aronthis time I will succeed because finaly I can do stuff like this, waking up at 1:30 am, then suffering for an hour before you come to help šŸ™‚#2020-04-2202:31Aronoh and because I can use shadow-cljs#2020-04-2202:31Aronso the tooling is solved more or less#2020-04-2202:33seancorfieldI hear great things about shadow-cljs -- when (if) I ever go back to trying to do ClojureScript, that's the path I'll take.#2020-04-2213:58lambdamHello everyone, I am defining some specs in a .cljc file, some only exist in a #?(:clj (do ...)) , others in a #?(:cljs (do ...)) I get an error on a keyword at parsing time:
2. Unhandled clojure.lang.ExceptionInfo
   failed compiling
   file:/.../foo.cljc
   {:file
    #object[.File 0x5b1b23e7 "/.../foo.cljc"],
    :clojure.error/phase :compilation}
             compiler.cljc: 1717  cljs.compiler$compile_file$fn__3955/invoke

1. Caused by clojure.lang.ExceptionInfo
   /.../foo.cljc
   [line 52, col 41] Invalid keyword: ::const/env.
   {:type :reader-exception,
    :ex-kind :reader-error,
    :file
    "/.../foo.cljc",
    :line 52,
    :col 41} 
Here the ::const namespace is declared only for .clj parts of code (:require ... #?(:clj [... :as const])) When I split this .cljc into two files .clj and .cljs, there is no error. Would it be related to spec or is it a more general error? It is the first time that it happens. I really don't have any clue. Thanks
#2020-04-2214:04Alex Miller (Clojure team)it's more general - ::const relies on having a clojure runtime namespace context to resolve it - I'm not sure what the exact problem is without more context but generally you'll want to avoid these in reader conditionals#2020-04-2214:05Alex Miller (Clojure team)you should be able to just use :foo/const or whatever instead#2020-04-2214:41lambdamThanks @alexmiller it works. It's not as convenient as the shortcut version but it's still better than two .clj and cljs files. If you want more info, don't hesitate to ask for. I'll send in a private message.#2020-04-2317:07friczeIā€™m curious about specing functions. Thereā€™s probably something I donā€™t understand, but why function specs made with fdef are not reusable? I can easily imagine, and even find in my codebase, few functions that share the same spec, but still fdef have to be called with all the arguments for each function. If anyone has some insight into this decision, Iā€™d love to hear something šŸ™‚ thanks#2020-04-2317:15kennyNot sure if I'm following what you mean but you can do this:
(s/fdef example
  :args (s/cat :x int?)
  :ret int?)
=> user/example

(s/valid? (:args (s/get-spec `example)) (list 1))
=> true
#2020-04-2317:17kennyYou can also
(s/def ::example-args (s/cat :x int?))

(s/fdef example
  :args ::example-args
  :ret int?)
#2020-04-2317:53friczeyeah, I can do that but can I do something like
(s/def ::example-fn-spec 
  {:arg (s/cat :x int?)
   :ret int?})

(s/fdef example-fn ::example-fn-spec)
(s/fdef example-fn-2 ::example-fn-spec)
?
#2020-04-2317:54kennyNo#2020-04-2317:54friczelikeā€¦ most things in Clojure are easily movable and spec is not always like that. Iā€™m curious why?#2020-04-2317:55friczeit seems to work a bit opposed to rest of Clojure design#2020-04-2319:42nikolavojicicI think in spec2 it will be (or it is already?) possible to transform spec -> map and vice versa.#2020-04-2323:49pbrownIn spec1 s/def can refer to another spec by fully qualified symbol too, so this should work:
(s/fdef f1 ...)
(s/def f2 `f1)
#2020-04-2702:03arnaud_bosThere's an example in spec-alpha2's wiki to illustrate select from a literal schema with unqualified keys:
(gen/sample (s/gen (s/select {:a int? :b keyword?} [:a])) 5)
;;=> ({:b :C, :a -1}
;;    {:b :f_/s!, :a -1}
;;    {:b :J8.M+/+88, :a -1}
;;    {:a -2}
;;    {:b :IEcw.l?/X, :a -1})
But at the REPL this is what I get:
(gen/sample (s/gen (s/select {:a int? :b keyword?} [:a])) 5)
Error printing return value (ExceptionInfo) at clojure.test.check.generators/fn (generators.cljc:435).
Couldn't satisfy such-that predicate after 100 tries.
I'm on 8498f9cb352135579b6d3a0a5d15c40e5c2647ce , which seems to be the latest commit on master. Sorry if it's already been reported before. I don't really know where to look for stuff like this.
#2020-04-2702:11Alex Miller (Clojure team)yeah, there's a bug in this area#2020-04-2714:28dspiteselfI am sure you have already seen this, but unqualified keys are not being looked at for closed specs in spec-alpha2
8498f9cb352135579b6d3a0a5d15c40e5c2647ce
(s/register
  :blah/name2
  (s/resolve-spec 'clojure.core/string?))
(s/register
  :blah/Object2
  (s/resolve-spec
    `(s/schema {:name :blah/name2})))
(s/explain :blah/Object2 {:name "hi"} )
;-> Success!
(s/explain :blah/Object2 {:name "hi"} {:closed #{:blah/Object2}})
;-> {:name "hi"} - failed: (subset? (set (keys %)) #{}) spec: :blah/Object2
https://github.com/clojure/spec-alpha2/blob/master/src/main/clojure/clojure/alpha/spec/impl.clj#L452
#2020-04-2714:32dspiteselfCreating Specs Programmatically is so pleasant. I have really been enjoying spec-alpha2.#2020-04-2714:42Alex Miller (Clojure team)I'm sure there are any number of bugs in the newer stuff right now, it's not done#2020-04-2714:50dspiteselfyep I was just checking if it would be valuable to report it. Thanks for the awesome work.#2020-04-2809:23arnaud_bosThere's this line in spec-alpha2's wiki page on "Schema and select" (emphasis mine): > General form: (s/select schema selection) > ā€¢ schema (required) - can be a registered schema name, schema form (like s/union), or literal schema But I can't get the "schema form" to work.
(in-ns 'user)
=> #object[clojure.lang.Namespace 0x54029c23 "user"]

(s/def ::a int?)
=> :user/a

(s/def ::b keyword?)
=> :user/b

;; Literal schema: OK
(gen/sample
    (s/gen
      (s/select
        [::a ::b]
        [*]))
    2)
=>
(#:user{:a -1, :b :B/d}
 #:user{:a -1, :b :Q/r})

;; Schema name: OK
(s/register ::ab (s/schema [::a ::b]))
=> :user/ab

(gen/sample
    (s/gen
      (s/select
        ::ab
        [*]))
    2)
=>
(#:user{:a -1, :b :B/_}
 {})


;; Union name: OK
(s/register ::ba (s/union [::a] [::b]))
=> :user/ba

(gen/sample
    (s/gen
      (s/select
        ::ba
        [*]))
    2)
=>
(#:user{:b :./l?}
 #:user{:a -3})

;; s/schema form: Err
(gen/sample
    (s/gen
      (s/select
        (s/schema [::a ::b])
        [*])))
Execution error (IllegalArgumentException) at clojure.alpha.spec.protocols/eval1814$fn$G (protocols.clj:20).
No implementation of method: :keyspecs* of protocol: #'clojure.alpha.spec.protocols/Schema found for class: clojure.lang.PersistentList

;; s/union form: Err
(gen/sample
    (s/gen
      (s/select
        (s/union [::a] [::b])
        [*])))
Execution error (IllegalArgumentException) at clojure.alpha.spec.protocols/eval1814$fn$G (protocols.clj:20).
No implementation of method: :keyspecs* of protocol: #'clojure.alpha.spec.protocols/Schema found for class: clojure.lang.PersistentList
Looks like I've messed something up wrt "symbolic" vs "object"?
#2020-04-2812:52Alex Miller (Clojure team)No, just bugs I think#2020-04-2812:53Alex Miller (Clojure team)Spec 2 is not ready for use#2020-04-2814:32arnaud_bosYeah, sorry to make you feel like you have to repeat it again and again. I'm just toying around šŸ™‚#2020-04-2814:32Alex Miller (Clojure team)the whole schema/select impl is due for a rewrite, just don't have time to work on it right now#2020-04-2814:37arnaud_bosI see, thank you šŸ™‚#2020-04-2814:35flipmokidHi all, I'm playing around with the lastest spec alpha 2. I'm not sure if I'm doing something wrong but:
(spec/def ::tag (spec/and string? (spec/conformer clojure.edn/read-string str)))
(spec/def ::value string?)
(spec/def ::kv (spec/cat :tag ::tag :value ::value))
(spec/def ::fix-message (spec/and string?
                                  (spec/conformer (fn [x] (map #(clojure.string/split % #"=") (clojure.string/split x #"\01"))))
                                  (spec/+ ::kv)))
Should work with a FIX input string (a bunch of k=v pairs joined by ASCII code 01) but currently fails with
["8" "FIX.4.4"] - failed: string? in: [0] at: [:tag] spec: :cme-fix.spec/tag
However, adding an additional predicate into ::kv makes it work:
(spec/def ::tag (spec/and string? (spec/conformer clojure.edn/read-string str)))
(spec/def ::value string?)
(spec/def ::kv (spec/and (constantly true) (spec/cat :tag ::tag :value ::value)))
(spec/def ::fix-message (spec/and string?
                                  (spec/conformer (fn [x] (map #(clojure.string/split % #"=") (clojure.string/split x #"\01"))))
                                  (spec/+ ::kv)))
Am I doing something obviously wrong?
#2020-04-2818:31FranklinHey guys šŸ‘‹ , I'm tring to get started on using clojure.spec and I hit an unexpected/frustrating roadblock...#2020-04-2818:31Franklin
user=> (s/exercise false?)

Execution error (FileNotFoundException) at user/eval1428 (form-init5827540222428447763.clj:1).
Could not locate clojure/test/check/generators__init.class, clojure/test/check/generators.clj or clojure/test/check/generators.cljc on classpath.
#2020-04-2818:32FranklinThis error is thrown for any spec for which I call s/exercise#2020-04-2818:32FranklinAny help with be appreciated#2020-04-2818:33jaihindhreddyYou need to add test.check library as a dependency to your classpath#2020-04-2818:33Franklincool... I'll go ahead and do that#2020-04-2818:33jaihindhreddyspec uses it for generation#2020-04-2818:37Alex Miller (Clojure team)this is covered in the spec guide btw https://clojure.org/guides/spec#2020-04-2818:37Franklinoh... cool.. I didn't actually read that.. I'm following some tutorials... will go through that..#2020-04-2818:37Franklin@jaihindhreddy That worked šŸ˜ƒ šŸ‘#2020-04-2819:19kennyIs there a spec collection macro that works with eduction?#2020-04-2819:20kennye.g., something that returns true here:
(s/valid? (s/coll-of int?) (eduction (map inc) (range 10)))
=> false
#2020-04-2819:25shooitI donā€™t know if there is a spec that you can use but you could alternatively call seq on your eduction to turn it into a LazySeq which would then pass the spec#2020-04-2819:29dominicm@kenny I'm not sure, but I don't think eduction returns a collection, it returns a reducible :). That doesn't help but it does indicate that coll-of probably isn't the right tool.#2020-04-2819:30kennyRight. I'm curious what folks use to validate against it. Hmm, I could do that @shewitt!#2020-04-2819:32Alex Miller (Clojure team)eductions are suspended computations so there is no data to validate without forcing it#2020-04-2819:34kennyThis use case is for in a test so I think @shewitt's idea solves it. In general, it probably wouldn't make sense to call valid? on an eduction for that reason, right? Even if checked with s/every, it'd still need to recompute *coll-check-limit* when you finally use the eduction elsewhere.#2020-04-2819:42Alex Miller (Clojure team)Right#2020-04-3002:24bbrinckI noticed that instrumentation doesnā€™t work for certain core functions. Is this related to the inline metadata?
(require '[clojure.spec.alpha :as s])
(require '[clojure.spec.test.alpha :as st])
(s/fdef clojure.core/count :args (s/cat :coll coll?))
(s/fdef clojure.core/reverse :args (s/cat :coll coll?))
(st/instrument ['clojure.core/count 'clojure.core/reverse])
    
(reverse 1) ; gives spec error
(count 1) ; gives non-spec error
#2020-04-3003:52Alex Miller (Clojure team)yes#2020-04-3003:54Alex Miller (Clojure team)you will also find that adding specs to core functions will not be checked in calls from one core function to another as core is compiled with direct linking#2020-04-3015:16bbrinckThanks! Thatā€™s good to know. #2020-04-3015:20bbrinckThe reason why the ā€œinlineā€ limitation comes up is Iā€™m thinking about ways to use instrumentation to help provide some different error messages when using core functions, but this means that some functions cannot be instrumented. #2020-04-3015:22bbrinckI wonder if thereā€™s a way around this. I can also see how many functions are actually inlined. Maybe itā€™s not a huge amount in practice.#2020-04-3015:23bbrinckI may be misremembering, but I seem to remember some mention of spec2 instrumentation changing things. Do you have any idea if this inline limitation will remain in the new instrumentation?#2020-04-3015:23Alex Miller (Clojure team)no difference in how instrumentation is implemented#2020-04-3015:33bbrinckOK, thanks. A quick search in clojure.core indications about 85 cases of :inline. Maybe itā€™s not a huge deal to just skip those functions or maybe thereā€™s a non-instrumentation-based way to help with errors. Iā€™ll think more.#2020-04-3015:39Alex Miller (Clojure team)the biggest thing to be concerned about is probably performance. Some of the tests they did with https://github.com/borkdude/speculative made instrumented core pretty much unusable.#2020-04-3017:58bbrinckAgreed. I tried instrumentation on my test suite and it was way too slow. I have an idea about a different way to instrument that might be better wrt performance (basically by only doing spec checking if there is an error) but the inline limitation will require some additional thinking #2020-04-3018:16Alex Miller (Clojure team)Sounds like a good idea#2020-04-3016:18Frank HenardTo use spec.alpha2, do I need to clone the repo? I don't see it in maven-central#2020-04-3016:20Alex Miller (Clojure team)it has not yet been released#2020-04-3016:20Alex Miller (Clojure team)there are instructions in the readme on how to use it as a git dep#2020-04-3016:21Alex Miller (Clojure team)it is still a wip, actively changing, and buggy. you have been warned. :)#2020-04-3016:21Frank Henardunderstood, thanks!#2020-05-0320:02oskarkvI tried one of the examples on http://clojure.org, namely this one:
(defn adder [x] #(+ 1 x %))

(s/fdef adder
  :args (s/cat :x number?)
  :ret (s/fspec :args (s/cat :y number?)
                :ret number?)
  :fn #(= (-> % :args :x) ((:ret %) 0)))
And I did did
(st/instrument `adder)
If I pass in a string, I get a complaint. But as you can see, I added a 1 in the body of adder so that the :fn check wouldn't hold. But nothing happens when I call adder. What's going on?
#2020-05-0320:16Alex Miller (Clojure team)https://clojure.org/guides/faq#instrument_ret#2020-05-0322:45oskarkvThanks!#2020-05-0722:43dvingois there any way to abstract a spec?
(defmacro to-many-ref-type
 [spec]
 `(s/coll-of (s/or :id u/id? :ident ::ident ~(keyword (name spec)) ~spec) :type vector?)))

(s/def :habit/tasks (to-many-ref-type ::task))
i'm trying this, but I think the macroexpansion of s/def doesn't like it
#2020-05-0722:45dvingoactually, i think it's working - my issue is it's not working in cljs, I'll dig into it#2020-05-0919:17aisamuCLJS macros can be a bit finicky! Check https://code.thheller.com/blog/shadow-cljs/2019/10/12/clojurescript-macros.html#2020-05-0919:22dvingoThat was my issue! I was trying to use a cljc file and updated them to separate cljs and clj files. it's working now šŸ™‚#2020-05-1019:49Amir EldorHello, rather new to this. How would you instrument specs only in development? I'm using Leiningen, I guess there's some way to check what profile we're running? If so, would you do this (if) at the bottom of each namespace with specs??#2020-05-1021:37seancorfield@amir In several of my projects, I have the tests run with instrumentation in place, so each test file has a test fixture that turns instrumentation on (or just a top-level form that does it as the test file is loaded). Since I tend to be loading/running tests a lot during development, that usually means I have instrumentation turned on during development -- but it's easy enough to just eval an instrument call from a comment in your source file.#2020-05-1021:38seancorfieldNot sure what profiles have to do with that. If I'm developing, I'm working with a REPL. If I'm testing, I can have the tests turn instrumentation on. Otherwise the code won't turn instrumentation on by default, which is the behavior I want.#2020-05-1021:39seancorfieldWe use Specs heavily in production as well -- see https://corfield.org/blog/2019/09/13/using-spec/#2020-05-1417:10dvingothis is super useful, thanks for the post! I have a use case where I want to generate some crud operations as well for some map specs - do you have any pointers for functions you used to parse the specs? I see s/describe and s/form did you use those or something else?#2020-05-1417:38seancorfieldJust those. To get at the primary list of required/optional keys.#2020-05-1417:41dvingoperfect, thanks!#2020-05-1109:50Petrus TheronHow do I use Spec with core.async? E.g. how do I spec a function that returns a channel where I'll put! another spec on? Or do I need to wrap it in some kind of record for this?#2020-05-1116:45jkxyzyou can spec that something is a Channel, but thatā€™s not very useful, so you probably want to spec the values at the points where you put/take them#2020-05-1620:19Petrus TheronYes, I realized that I could spec the channel transducers that interpret incoming values on put!.#2020-05-1122:44rafaelI'm seeing strange behavior in spec alpha2 (latest github commit). If I define a schema using the map form, to specify non-namespaced keywords, refer to it from another schema, and later try to generate data from a selection of latter schema, I get an "Unable to resolve spec" exception:
; Context
  (s/def ::foo int?)
  (s/def ::inner (s/schema {:foo ::foo}))
  (s/def ::outer (s/schema [::inner]))

  ; Works as expected:
  (s/exercise ::outer)
  (s/exercise (s/select ::inner [*]))


  ; Fails with "Unable to resolve spec: :foo"
  (s/exercise (s/select ::outer [*]))
#2020-05-1122:44rafaelIs this expected, should I change something in my code, or is it a bug that I should report somewhere?#2020-05-1123:15seancorfieldSpec 2 is very much a work in progress and isn't ready for folks to use -- and, yes, it has numerous bugs in it @rafael#2020-05-1123:25rafaelYeah, so far I'm finding workarounds. I think I can work around this one as well with s/keys#2020-05-1202:09seancorfieldAs long as you're not building anything with it that is more than a personal toy project @rafael...#2020-05-1209:22fmnoisehi everyone, is it possible to add meta to spec?
(s/def ::my-custom-int ^{:error-fmt #(str % " is not valid int")} int?)
when I call (meta (s/get-spec ::my-custom-int)) it doesn't contain my meta
#2020-05-1209:26hkjels@fmnoise I donā€™t know the answer, but Iā€™m pretty sure youā€™re checking the wrong thing. Try (meta ::my-custom-int)#2020-05-1209:31vlaaad
(s/def ::my-int (with-meta (s/spec int?) {:err-format #(str % " is not a valid int")}))
#2020-05-1209:32vlaaad@fmnoise then (meta (sp/get-spec ::my-int)) will work#2020-05-1209:56fmnoisethanks @vlaaad, this works!#2020-05-1212:45Alex Miller (Clojure team)While this may work, please that it wonā€™t work on all specs and some spec transformations like with-gen will not preserve meta. This is more accidental than intentional#2020-05-1212:49Alex Miller (Clojure team)Adding meta and doc string support to specs is the most requested thing in the tracker and I expect we will add it in spec 2#2020-05-1218:16sgerguriI have a design question - I have a Kafka Streams topology where I use conformance to automatically sanitise input, then split the stream into two - those where conformance resulted in ::s/invalid and a happy path. In the happy path, I call a pure function that transforms the conformed message. This transformer has a function spec attached to it, but because I have already conformed the input according to the arg spec before calling the function, it will fail the conformance check.#2020-05-1218:17sgerguriI seem to be facing two options - either remove the function spec, or create yet another spec for the already-conformed input, a "derived spec" of sorts. Neither really feels satisfactory.#2020-05-1218:18sgerguriYet another alternative would be to simply validate with s/valid? earlier in the stream, then conform inside the pure function - which feels like the right thing to do, but it also feels like I am validating the data twice effectively. What would be the cleanest solution here?#2020-05-1218:59seancorfield@sgerguri The approach we've taken at work is to treat that initial Spec layer as a boundary and assume it maps from "external data format" to "internal data format" and is applied consistently. Thus everything "inside" is Spec'd in terms of the (already conformed) data produced at the boundary.#2020-05-1219:00sgerguriSo if I understand it correctly you have a second spec layer for the internal representation, that represents things that have been conformed, correct?#2020-05-1219:07seancorfieldYeah, an API Spec, a Domain Spec, and actually a Persistence Spec. The API Spec is conforming, from external data (often strings) to internal data (numeric, Boolean, date, etc). The Domain Spec describes the data formats used inside the API. The Persistence Spec describes the data that goes into the database (flat hash maps with JDBC-compatible types). We don't use all three in all cases, but it seems to have become a useful way to split things up -- and it clearly identifies two boundaries (API input and database output).#2020-05-1219:09sgerguriThat's excellent, thanks. I have used a similar approach in another service, the only thing I'm struggling with in this one is the fact that the internal layer is very thin and quite close to the input layer, thus it feels like duplicating the specs, but might be worth it for the greater clarity.#2020-05-1219:10mpenetspec-coerce is quite good for this too#2020-05-1219:11seancorfieldI would caution against leaning too heavily on conforming with Spec -- I don't really like what spec-coerce enables.#2020-05-1219:12mpenetBy default it will do the right thing to coerce x to your spec expeceted values, and the other way around (serializing to db type, ex keyword -> string) just requires to override those cases#2020-05-1219:12mpenetNo spec duplication that way#2020-05-1219:12mpenetNo polluted specs either (no conform abuse)#2020-05-1219:13sgerguriI have used it for some fairly tricky transformations in the past, but I've found it really works well for emulating pattern matching along with multimethods by tagging individual cases through an s/or. In this particular example I only wanted to trim whitespace but for some reason am having trouble fitting it neatly in without overhauling one part of the service or another.#2020-05-1219:15sgerguriThis particular situation also left me wondering whether conformance should either be the final station in your transformation/validation stack or whether it should yield something that also has a spec for it (without further conformers), like @seancorfield described.#2020-05-1219:15sgerguriIn the grand scheme of things I could just trim the fields I need to but with spec around that just feels like the wrong approach.#2020-05-1219:16seancorfieldThere are folks on both sides of that decision šŸ™‚#2020-05-1219:17seancorfieldSome feel that input should be cleaned and parsed first, then checked for validity with Spec (and therefore next to no conforming). Others feel that input conformance to valid data is a reasonable use of Spec.#2020-05-1219:17mpenetI used to have dual specs like you mention, but it requires more work and is more brittle imho#2020-05-1219:18mpenetThen as you say ymmv#2020-05-1219:18seancorfieldI am slightly on the conforming side of center. spec-coerce is quite a lot further out on the conforming side. I think Alex (and maybe others at Cognitect) are on the non-conforming side of center.#2020-05-1219:18Alex Miller (Clojure team)I actually like spec-coerce :)#2020-05-1219:19seancorfieldReally? I was sure you had complained about it in principle in the past? šŸ™‚#2020-05-1219:19Alex Miller (Clojure team)I don't like spec-tools#2020-05-1219:21sgerguriFood for thought. Thank you everyone, always a pleasure to come here for another opinion on how folk do things. šŸ™‡#2020-05-1219:26Alex Miller (Clojure team)spec-coerce builds a separate registry of coercions that leverages specs, which I think is a good approach#2020-05-1219:27Alex Miller (Clojure team)I haven't used it in anger but that seems in line with how I would approach it#2020-05-1219:29mpenetIt's gotten better lately, it was missing multi-specs, merge & tuple until recently but now it's quite feature complete#2020-05-1219:30mpenetIt's also 300ish lines of code, easy to modify, fit to your taste if needed#2020-05-1219:43seancorfield@alexmiller Ah, thanks for the clarification. Then I suspect I'm getting the two libraries confused and maybe I should look at spec-coerce in more detail? šŸ™‚#2020-05-1220:06mpenetYou contributed to it a few years ago https://github.com/wilkerlucio/spec-coerce/pull/11 :)#2020-05-1221:40seancorfieldWow, I don't even remember that. Looks like I'd started to look at spec-coerce as a possible alternative to our existing "web specs" at World Singles Networks -- and I guess by the time that PR got accepted I'd decided not to go down that particular path for some reason...#2020-05-1221:45seancorfieldI've made a note to revisit it, but here's what we ended up doing at work https://github.com/worldsingles/web-specs#2020-05-1220:31Amir EldorHello, I hope it's ok to post some code. I'm trying to spec a function that has ehmm, this optional arguments thing? I'm not sure how it's called in Clojure. I'm having some trouble with the spec for it. Right now I specifically use (st/instrument) in a test as suggested and the following code breaks with the following exceptions: (ship (:id p)) ; (:id p) is surely a uuid || :cause "class java.util.UUID cannot be cast to class java.lang.Number (java.util.UUID and java.lang.Number are in module java.base of loader 'bootstrap')"#2020-05-1220:32Amir EldorIt seems like it's related to the ::speed thing, being an integer. I must have made something funny in the spec, if someone can notice. Thanks!#2020-05-1220:46Amir EldorAh yes, thet exception I gave is invoked on the s/def of ::speed šŸ˜•#2020-05-1221:35fmnoisewhat is (random-id) ?#2020-05-1221:40Amir EldorIt's my own function, which calls (UUID/randomUUID) wth java.util.UUID#2020-05-1221:43Amir EldorThe code itself works without the spec, I must be defining something badly#2020-05-1221:45fmnoisewhat about default-ship-speed?#2020-05-1221:46Amir Eldor(defonce default-ship-speed 20)#2020-05-1221:51seancorfieldWhat is DateTime?#2020-05-1221:51Amir Eldor
(:import [org.joda.time DateTime]
           [java.util UUID]))
From Java, too
#2020-05-1221:51Amir EldorThough I use clj-time, so I might be wrong here when I think of it#2020-05-1221:52seancorfield'k... Joda Time is deprecated. If you're on Java 8 or later, you should use Java Time really.#2020-05-1221:52seancorfieldclj-time is also deprecated.#2020-05-1221:52seancorfield(for the same reason)#2020-05-1221:52Amir EldorOh, thanks for letting me know. I'll google for Java Time#2020-05-1221:53fmnoise(s/def ::speed (fn [v] (and (int? v) (> v 0)))) this definition works#2020-05-1221:54fmnoisebut dunno why it assumes uuid there by default when doing coercion#2020-05-1221:57seancorfieldThe problem is that all three optional arguments are optional independently#2020-05-1221:58seancorfieldSo dest-planet-id could be omitted and the speed and departure-time could still be provided.#2020-05-1221:59seancorfieldIn other words, it tries to check (ship src-planet-id src-planet-id) against a signature that is essentially [uuid? #(> % 0)] and that's causing the exception.#2020-05-1222:00seancorfieldAnd that's why adding (and (int? %) ...) into ::speed "works" -- it guards the > operation from being called on non-numeric values.#2020-05-1222:01seancorfieldBecause then the ::speed spec successfully fails to match (instead of blowing up) and Spec goes on to try the other options.#2020-05-1222:01seancorfieldSo...#2020-05-1222:02seancorfieldI'd suggest spec'ing ship a bit differently, perhaps using s/alt over the four arities, each spec'd with no optional parameters.#2020-05-1222:03seancorfieldOr at least across the first three and leave just :departure-time as s/?#2020-05-1222:03Amir EldorThanks a lot! However I don't fully understand how this happens: > dest-planet-idĀ could be omitted and theĀ `speed`Ā andĀ `departure-time`Ā could still be provided.#2020-05-1222:05Amir EldorI'm sorry I have to go now as it's getting late. I hope it's ok if I ping you tomorrow in this thread if I still have some trouble. Thanks!#2020-05-1222:09seancorfieldYou have src dest? speed? time? -- each of those three are optional, so each could be omitted while the others are passed.#2020-05-1222:09seancorfieldso src speed is "valid", as is src time or src dest time or src speed time.#2020-05-1222:09seancorfieldThat's what your fdef says.#2020-05-1222:10seancorfieldSo when Spec sees (ship src-planet-id src-planet-id) it's going to try src speed, src time, and src dest in some random order.#2020-05-1222:11seancorfieldSince your speed spec just tries to compare the value > it will throw an exception if passed a non-number: which a UUID is.#2020-05-1222:11seancorfieldBecause Spec encounters an exception, it won't try the other options -- it just propagates the exception.#2020-05-1222:15seancorfield
(! 629)-> clj -A:test -Sdeps '{:deps {clj-time {:mvn/version "RELEASE"}}}'
Clojure 1.10.1
user=> (require '[clojure.spec.alpha :as s] '[clojure.spec.test.alpha :as st])
nil
user=> (import '(org.joda.time DateTime) '(java.util UUID))
java.util.UUID
user=> (defonce default-ship-speed 20)
#'user/default-ship-speed
user=> (defn random-id [] (UUID/randomUUID))
#'user/random-id
user=> (s/def ::id uuid?)
:user/id
user=> (s/def ::src-planet-id uuid?)
:user/src-planet-id
user=> (s/def ::dest-planet-id uuid?)
:user/dest-planet-id
user=> (s/def ::speed #(> % 0))
:user/speed
user=> (s/def ::departure-time #(or (nil? %) (instance? DateTime %)))
:user/departure-time
user=> (s/def ::resources #(>= % 0))
:user/resources
user=> (s/def ::ship (s/keys :req-un [::id ::src-planet-id ::dest-planet-id ::speed ::departure-time ::resources]))
:user/ship
user=> (defrecord Ship [id src-planet-id dest-planet-id speed departure-time resources])
user.Ship
user=> (s/fdef ship
        :args (s/alt :arity1 (s/cat :src-planet-id ::src-planet-id)
                     :arity2 (s/cat :src-planet-id ::src-planet-id :dest-planet-id ::dest-planet-id)
                     :arityN (s/cat :src-planet-id ::src-planet-id :dest-planet-id ::dest-planet-id :ship-speed ::speed :departure-time (s/? ::departure-time)))
        :ret ::ship)
user/ship
user=> (defn ship
  ([src-planet-id]
   (ship src-planet-id src-planet-id))

  ([src-planet-id dest-planet-id]
   (ship src-planet-id dest-planet-id default-ship-speed))

  ([src-planet-id dest-planet-id ship-speed]
   (ship src-planet-id dest-planet-id ship-speed nil))

  ([src-planet-id dest-planet-id ship-speed departure-time]
   (->Ship (random-id) src-planet-id dest-planet-id (if (nil? ship-speed) default-ship-speed ship-speed) departure-time 0)))
#'user/ship
user=> (st/instrument)
[user/ship]
user=> (ship (random-id))
#user.Ship{:id #uuid "3ad92050-2b9b-499e-b7e3-f933053d7b1c", :src-planet-id #uuid "79cd0367-d78d-4774-9f5f-3a29a8c77851", :dest-planet-id #uuid "79cd0367-d78d-4774-9f5f-3a29a8c77851", :speed 20, :departure-time nil, :resources 0}
user=> 
#2020-05-1222:15seancorfield^ That shows the s/alt structure that would work @U0140AKS332#2020-05-1306:38Amir EldorThanks @seancorfield for the detailed answer and thanks @fmnoise for helping out too!#2020-05-1310:36fmnoisethat was interesting case, thanks @U0140AKS332 and @seancorfield#2020-05-1314:09oskarkv
(s/def ::fact
  (s/cat :var (s/? ::?symbol)
         :type (s/or :simple symbol? :acc ::acc-vec)
         :destruct (s/? vector?)
         :exprs (s/* list?)))

(s/def ::fact-without-var
  (s/cat :type (s/or :simple symbol? :acc ::acc-vec)
         :destruct (s/? vector?)
         :exprs (s/* list?)))
Is there a way to get rid of the repetition here without introducing more nesting and keywords in the conformed data?
#2020-05-1314:12oskarkvThere are always macros of course but I was hoping for something easier.#2020-05-1314:13Alex Miller (Clojure team)doesn't the first one include the second?#2020-05-1314:16oskarkvBut I want a spec for data where a var is forbidden. And one where it's optional.#2020-05-1314:16Alex Miller (Clojure team)ah#2020-05-1314:16oskarkvI'm not sure I understand what you meant.#2020-05-1314:17Alex Miller (Clojure team)so, you can create a new spec that is an s/cat that just contains the common portions, then reference it in both ::fact specs#2020-05-1314:18Alex Miller (Clojure team)
(s/def ::fact-without-var
  (s/cat :type (s/or :simple symbol? :acc ::acc-vec)
         :destruct (s/? vector?)
         :exprs (s/* list?)))

(s/def ::fact
  (s/cat :var (s/? ::?symbol)
         :tail ::fact-without-var))
#2020-05-1314:19Alex Miller (Clojure team)that does introduce nesting in the conformed data so is not everything you want#2020-05-1314:19oskarkvYeah šŸ™‚#2020-05-1314:20Alex Miller (Clojure team)another way would be to restrict the bigger one#2020-05-1314:21Alex Miller (Clojure team)maybe something like
(s/def ::fact
  (s/cat :var (s/? ::?symbol)
         :type (s/or :simple symbol? :acc ::acc-vec)
         :destruct (s/? vector?)
         :exprs (s/* list?)))

(s/def ::fact-without-var
  (s/& ::fact #(nil? (:var %))))
#2020-05-1314:22Alex Miller (Clojure team)can't say I've done this before, might need to tweak that predicate more#2020-05-1314:23oskarkvHm, gotta read up on &. Thanks!#2020-05-1314:24Alex Miller (Clojure team)& just lets you and arbitrary predicates into the regex spec#2020-05-1314:24oskarkvAh, I see. Could work. Thanks!#2020-05-1419:08Amir EldorHow do you define a map spec with keys of similar spec? e.g.:
(s/def potato-name string?)
(s/def potato-happiness number?)

;; This is a potato:
{:name "potato name"
 :happiness-1 12
 :happiness-2 54}
Similarly, I see that (s/keys) requires a fully qualified name for each key and it's strange to me. What if I got two specs with the same key but different specs?
#2020-05-1419:49Alex Miller (Clojure team)the idea behind spec is to give attributes global semantic meaning, by using namespaced attributes to differentiate and not do that#2020-05-1518:54Amir EldorSo it's not intended, for example, to use things like ::mygameobject-id on other namespaces of specific 'gameobject' each? Among the things I'm playing around with is having a ::gameobject-id defined in some-ns.game and use that in some-ns.game.dog and some-ns.game.bat, each of which has an :id which is a gameobject-id. Am I doing things in a non-Clojure way?#2020-05-1519:06Alex Miller (Clojure team):: means to auto-resolve the keyword in the context of the current namespace#2020-05-1519:07Alex Miller (Clojure team)so if you're using ::id in different namespaces, you'll get different keywords specific to each namespace, and there is no conflicting name#2020-05-1519:08Amir Eldorand if I want to use the same spec definition for each? just use a common function?#2020-05-1519:08Alex Miller (Clojure team)you can (s/def ::id ::common/gameobject-id) in each namespace#2020-05-1519:08Alex Miller (Clojure team)so have one common spec and then make namespaced specs that refer to it#2020-05-1519:10Alex Miller (Clojure team)removing the potentially confusing auto-resolving keywords, something like:
(s/def :common/gameobject-id string?)
(s/def :some-ns.game.dog/id :common/gameobject-id)
(s/def :some-ns.game.bit/id :common/gameobject-id)
#2020-05-1519:12Amir EldorI was trying to do just that and it didn't work, and not it works. Great. Maybe I had a funky repl state before :man-shrugging: Thanks!#2020-05-1519:12Alex Miller (Clojure team)good!#2020-05-1519:14Amir EldorI see that when I do something like
(s/def ::thingy (s/keys :req-un [:common/gameobject-id]))
#2020-05-1519:14Amir EldorThen the thingy gets an :id property, and not { :common/gameobject-id "whatever" }#2020-05-1519:14Amir EldorWhich is good for me#2020-05-1519:15Amir Eldor(or maybe :gameobject-id, I am mixing up some namings between this chat and my actual code)#2020-05-1519:16Alex Miller (Clojure team)it doesn't "get" a :gameobject-id property, it describes a map expected to have a :gameobject-id key which matches the :common/gameobject-id spec#2020-05-1519:21Amir EldorI meant that when I conform something to ::thingy, the keyword in the conformed map is { :gameobject-id "something" } and not { :common/gameobject-id "something" }. I guess it omits the namespace from the keyword?#2020-05-1519:21Alex Miller (Clojure team):req-un means "required unqualified key"#2020-05-1519:22Alex Miller (Clojure team)(as opposed to :req)#2020-05-1519:22Alex Miller (Clojure team)so more importantly it's validating the incoming map expecting an unqualified key#2020-05-1519:22Alex Miller (Clojure team)conforming just follow that#2020-05-1519:23Amir EldorI seeeee I made a thingy2 now with :req and it wants the fully qualified key#2020-05-1519:23Amir EldorInteresting and cool.#2020-05-1519:40Amir EldorDo I have to require common for using :common/gameobject-id or something?#2020-05-1519:43Amir EldorOh nevermind.#2020-05-1519:52Alex Miller (Clojure team)no#2020-05-1601:02Kyle BrodieWhats the best way to spec numbers as strings so something like "12.53" will be generated?#2020-05-1602:21seancorfield@kyle Take a look at https://github.com/worldsingles/web-specs#2020-05-1602:22seancorfieldIt has specs that accept strings or longs and generate as strings that are parseable as longs. Should give you something to work from.#2020-05-1602:49Kyle Brodie@seancorfield Thank you! That library looks very helpful. I might have to write my own date though because mine are "YYYY-MM-DD". Unless "yyyy-M-d" is the right format for zero padded in java-time#2020-05-1603:09seancorfieldYeah, it'll do zero-padded. And you want yyyy and d lowercase for dates.#2020-05-1611:41Kira McLeanHi there.. Iā€™m just getting started with spec, have a somewhat noob question that Iā€™m having trouble finding an answer to: Is it possible to have two different specs for the value of a given un-qualified keyword? I.e. I have a map and I know that the values might have different requirements in different contexts, but I would like to keep re-using the same key. So I have specs like this (`fs` is [datoteka.core :as fs] ):
(s/def ::path fs/path?)
(s/def ::html-path (s/and ::path (partial matches-ext "html")))
(s/def ::content string?)

;; This one is great -- checks my maps like {:path ,,, :content ,,,}
(s/def ::page (s/keys :req-un [::path ::content]))

;; Here is the issue: I want to check a map that also has the keys [:path 
;; :content], not one with an html-path key
(s/def ::html-page (s/keys :req-un [:path ::html-path ::content])) 
In the ::html-page spec I want :path to match the html-path spec but for the key to still be called :path . Is this possible?
#2020-05-1611:51ghadiIf you define two specs, :a.b.c/path and :x.y.z/path you can use them to spec unqualified keys in different maps#2020-05-1616:14Kira McLeanMakes perfect sense, thanks.#2020-05-1618:17David PhamAre there good library to use with clojure.spec? šŸ™‚#2020-05-1618:49salam@neo2551 you can find spec libraries (along with other libraries) here: https://www.clojure-toolbox.com/#2020-05-1620:57Kira McLeanCan you write a function spec for a multi-method? If so, any chance anyone can share an example of this somewhere?#2020-05-1621:31favilaYou can spec the method, but afaik not the implementations#2020-05-1621:32favilaCaveat with fspec on mm: instrumentation changes the type to a fn, so future defmethods will fail with a type error#2020-05-1621:41favilaYou need to unstrument before evaluating more defmethods #2020-05-1623:29Kira McLeanInteresting.. good to know. Yeah Iā€™m curious about whether itā€™s possible to spec the implementations of a multimethod. I couldnā€™t find any examples online, but so far Iā€™ve found with clojure that doesnā€™t always mean itā€™s not possible.#2020-05-1623:41Alex Miller (Clojure team)You can spec the dispatch method used in the mm if thatā€™s useful#2020-05-1623:42Alex Miller (Clojure team)But mms themselves, no#2020-05-1623:42Kira McLeanThatā€™s something! But good to know, thanks!#2020-05-1900:04telekidIā€™m not sure if this is useful or relevant to your goals, but: https://gist.github.com/telekid/f2e588718dbdfe48306d64e5388bdc15#2020-05-2121:33Kira McLeanAh this is really cool! Never thought of separating out the args and mm to spec separately. Perhaps a little unorthodox or is this normal? But definitely interesting. Thanks!#2020-05-1812:41FranklinHow can I fix this:#2020-05-1812:41Franklin
Error Error: Require clojure.test.check and clojure.test.check.properties before calling check.
#2020-05-1812:42FranklinI get this when I call (stest/check `filter-list)#2020-05-1812:45Franklinpicard-facepalm never mind, I should have just read the error messsage#2020-05-1813:09mpenet@seancorfield @alexmiller @sgerguri fyi we released our internal fork of spec-coerce https://github.com/exoscale/coax#2020-05-1813:10sgerguriNice, I'll have a look today - appreciate the heads up!#2020-05-1818:07Felipe MarquesHi, there, I'm using spec-tools with a multi-spec, and I get the following error when trying to coerce some data:
TypeError: Cannot read property 'cljs$core$IFn$_invoke$arity$2' of null
    at type_priority (/js/cljs-runtime/spec_tools.parse.js:270:22)
    at eval (/js/cljs-runtime/cljs.core.js:7726:102)
    at eval (/js/cljs-runtime/cljs.core.js:7638:85)
    at stableCompareFn (/js/cljs-runtime/goog.array.array.js:640:12)
    at Array.sort (<anonymous>)
    at Object.goog.array.sort (/js/cljs-runtime/goog.array.array.js:626:7)
    at Object.goog.array.stableSort (/js/cljs-runtime/goog.array.array.js:642:14)
    at Function.eval [as cljs$core$IFn$_invoke$arity$2] (/js/cljs-runtime/cljs.core.js:7686:12)
    at Function.eval [as cljs$core$IFn$_invoke$arity$3] (/js/cljs-runtime/cljs.core.js:7724:23)
    at Function.eval [as cljs$core$IFn$_invoke$arity$2] (/js/cljs-runtime/cljs.core.js:7720:26)
Does anyone have a similar problem?
#2020-05-1906:01joefromcthi, quick question, given a spec ::my-map based on req and opt how could i get a list of the keys that this spec pertains to? Should i be parsing s/form or is there an easier way?#2020-05-1908:56ikitommi@marques.goncalves.fel havenā€™t seen that. please write an issue to spec-tools and you could check also spec-coerce or coax - many libs doing mostly/exactly the same.#2020-05-1909:26mpenetšŸ… vs šŸ…#2020-05-1909:28mpenetI haven't tested extensively the cljs part personally, tests pass but that's it for now#2020-05-1912:01jrychterThis just bit me while writing :postconditions, where I wanted to assert that the return value was (among other things) a "status", but did not want to assert anything else: this is at an edge API where data has been transformed into external representations (timestamps to ISO8601 date strings, for example):
(s/def ::a string?)
(s/def ::b string?)

(s/def ::foo (s/keys :req [::a ::b]))
;; Let's define ::bar, which is not concerned with ::b at all:
(s/def ::bar (s/keys :req [::a]))

;; This is expected:
(s/valid? ::foo {::a "a" ::b "b"}) => true
(s/valid? ::foo {::a "a" ::b 1}) => false

;; But even though ::bar is not concerned with ::b, validation fails. I was hoping to only check if something
;; is a valid ::bar, without asserting anything about keys that are not in :req or :opt in ::bar.
(s/valid? ::bar {::a "a" ::b 1}) => false
I expected the last form to return true, to this day I did not realize that spec is eagerly checking all keys, not just those listed by s/keys.
#2020-05-1921:06EddieGood to know! Thanks for mentioning it. It seems odd to me that we would associate a value that is not a valid ::b with the very specific qualified keyword of ::b in a map. Perhaps this indicates that in the :bar maps should be using a different key, like :b.#2020-05-1913:06Alex Miller (Clojure team)all keys are checked by s/keys (even (s/keys) is a valid and useful spec)#2020-05-1913:07Alex Miller (Clojure team)if you want to limit, you could select-keys on the data first, or validate the attributes specifically#2020-05-1920:40jrychterThat was unexpected, I somehow got used to reading s/valid? as "tell me if this thing is a valid ::bar". Especially since it takes a parameter ā€” (s/validate {:data :foo}) is intuitively different from (s/valid? ::foo {:data :foo}).#2020-05-1920:42jrychterPerhaps I'm using it wrong, but I would find the ability to check just "is this a valid ::bar ?" very useful.#2020-05-1921:09seancorfield@jrychter That makes sense for unqualified keys but part of the idea behind Spec for qualified keys is that they are globally unique (within your program) and therefore if :foo/bar is present in a hash map, it should conform to the Spec for :foo/bar if any. s/keys is more about required/optional anyway (and that's emphasized in Spec 2 via schema and select).#2020-05-1921:28Alex Miller (Clojure team)the big idea is that spec gives named attributes global semantics#2020-05-1921:28Alex Miller (Clojure team)if you have global semantics, it's ok (and good) to validate them when you see them#2020-05-1921:29Alex Miller (Clojure team)if that's bad then either the spec is bad (b/c it doesn't fully encompass what that attribute can mean), or your data is bad (b/c it doesn't conform to the global semantics) or your version of "valid" does not match spec's semantics :)#2020-05-2005:12David PhamIs it possible to share spec over wire? Or to serialize them?#2020-05-2005:14seancorfield@neo2551 You can get the code form of a spec with s/form -- and encode that send it over the wire -- but any predicates are going to depend on the code itself (anything beyond simple core predicates).#2020-05-2005:14seancorfieldI'm not sure how easy it is in Spec 1 to reconstitute a spec from its form tho'. That's something that is easier with Spec 2.#2020-05-2012:49Alex Miller (Clojure team)Just eval the form you get#2020-05-2005:15David PhamGreat! Thanks a lot!#2020-05-2007:24jrychterHmm. I think I understand the "global semantics" goals, but what about the specific case I have now? I think I am falling into answer (3) in Alex's list above. I am preparing API responses and I do want to preserve the same key names, as they will be sent to the API callers somewhere down the way. However, those responses fall outside of my app semantics boundary, and some fields will contain data in a different form (like timestamps). That is OK, but I would still like to verify some aspect of those responses ā€” whether they are a valid ::status/status, for example. I think there is also a conflict with the "open maps" idea that Rich has been talking about. I would like to be able to validate data as being valid in terms of the supplied spec, and whatever else is in the map should not matter. I do understand the global semantics idea, just pointing out that these two might sometimes clash. I would find the ability to check only "is this a valid ::bar ?" very useful. Anyway, just wanted to describe the use case, as material for thinking.#2020-05-2018:03seancorfield@jrychter FWIW, Spec 2 has support for checking specs with closed maps as an option (it's a check-time option, not a spec definition time option).#2020-05-2018:04seancorfieldAlso, if it helps: what we do at work is have different (but related) sets of specs for domain, API, and persistence -- since API and persistence are constrained by external systems whereas domain is not.#2020-05-2018:04seancorfieldThen we transform data at the boundaries between those layers as needed.#2020-05-2022:35Kyle BrodieI am trying to spec CSV rows that are parsed into Clojure maps but I'm hitting StackOverflowError in (s/keys :req-un [::kw1 ... ::kw156]) and right now the keywords are all (s/def ::kwN string?). I mostly want this for generating examples while developing so I don't have to use spec. It works if I give Java more stack space (I'm giving it -Xss512k to match Heroku's smallest dynos) but I think I'm using spec wrong#2020-05-2022:42Kyle BrodieI think the overflow is in expanding the and macro for keys-pred at https://github.com/clojure/spec.alpha/blob/5a4834eeb2967b2eca382efe0afcb3f590bdb3c2/src/main/clojure/clojure/spec/alpha.clj#L466#2020-05-2022:46seancorfield@kyle That's certainly a lot of keys in a hash map to require all be present... I assume they have more meaningful names than kw1, kw2, etc?#2020-05-2022:50Kyle BrodieYeah its because the CSVs are denormalized so the columns are always there even if there is no data. I can get example data and make them into Clojure maps for my development. Just thought it would be cool to test out my functions with generated data#2020-05-2022:51Kyle BrodieThe data will be reduced to less keys after a few steps so those I should be able to spec then#2020-05-2022:51seancorfieldIf you wanted random key/value rows with random keywords and random strings, just for testing stuff, you could use
(s/def ::csv (s/map-of keyword? string? :min-count 156 :max-count 156))
#2020-05-2022:52seancorfieldI guess you could make a set of all the known keys and replace keyword? there with the set.#2020-05-2022:53seancorfield
(s/def ::csv (s/map-of #{:kw1 :kw2 ,,, :kw156} string? :min-count 156 :max-count 156))
I think that would work (you could make the set of keys into a separate spec)
#2020-05-2023:01Kyle Brodie(s/def ::simple-row (s/map-of (set columns) string? :min-count 156 :max-count 156)) (s/exercise ::simple-row 1)
Error printing return value (ExceptionInfo) at clojure.test.check.generators/fn (generators.cljc:435).
Couldn't satisfy such-that predicate after 100 tries.
class clojure.lang.ExceptionInfo
#2020-05-2023:53seancorfield@kyle Ah, yeah, that's because it is trying to randomly generate elements from the full set for each key but each key in the generated map needs to be distinct šŸ˜ž#2020-05-2023:53seancorfieldYou'd have to wrap the key part spec with a generator that produces the full set (in some random order, if you cared).#2020-05-2023:55seancorfieldI suspect it would be easier to wrap the whole s/map-of spec in a custom generator since all it really needs to do is zip together that set of keys with a bunch of random strings...#2020-05-2100:01Kyle BrodieSo I'd do (s/def (with-gen (my-spec-here) (fn [] (something-using-test-check-functions))))#2020-05-2100:13seancorfield(s/def ::simple-row (with-gen (s/map-of ,,,) (fn [] (...)))) yup#2020-05-2212:46practicalli-johnI am assuming that if I am using spec keys macro, I do not also need to check the data is a hash-map. So in the following code, the map? predicate would be redundant. Can someone confirm?
(spec/def ::customer-details
  (spec/and
    map?
    (spec/keys
      :req [::first-name ::last-name ::email-address ::home-address ::social-secuirty-id])))
#2020-05-2212:47minimalyes itā€™s redundant#2020-05-2212:54Alex Miller (Clojure team)s/keys does a map? check#2020-05-2220:08David PhamHello everyone. I really like Clojure.spec, it is really fun to use. I am currently wondering if there was a way to extract the default generator of a specs when using with-gen? The reason I am asking is I wish to provide more specific example than the default generator, but would still like to use the randomness of the default one. I know we can use gen/one-of, or gen/frequencies, but can we define the specific generator directly with with-gen? If not what would be a good/concise alternative to my problem, which is to define a spec with a custom generator which combine the default generator as well?#2020-05-2220:21kenny@neo2551 If I understand you correctly, we do this sort of thing a lot. Typically it's because of having a s/and. Here's an example of what we often have:
(s/def ::base-thing (s/keys :req [::a]))

(s/def ::thing
  (s/with-gen (s/and ::base-thing pred1? pred2?) #(gen/fmap (fn [base-thing] ) (s/gen ::base-thing))))
#2020-05-2221:33David Pham@kenny It is more if you define the ::color spec as keyword? and provide the #{:blue :red} as examples? In this case the base gen is concise, but in a more general case, I would need to copy/double the definition of my spec, will I?#2020-05-2221:35David PhamI would still like to retain the ability to generate random keywords, while using my blue and red examples. One solution is gen/one-of, but I donā€™t see how I can get the random one without copying the base spec again.#2020-05-2221:39kenny
(def colors #{:blue :red})
(s/def ::color (s/with-gen keyword? #(gen/one-of [(s/gen colors) (s/gen keyword?)])))
?
#2020-05-2221:41David PhamWhat if keyword? was some really long spec?#2020-05-2221:42kenny
(s/def ::color (s/or :predefined #{:blue :red}
                     :other keyword?))
#2020-05-2221:42kennyI'm not really sure I understand your problem šŸ™‚#2020-05-2307:18David PhamIn your answer with with gen/one-of, you repeated keyword? which was the main predicate. In that case you repeated the main predicate twice, once for defining the validity, once in the definition of the generator. Can we avoid this repetition?#2020-05-2307:19David PhamI think it is a common pattern to give some examples to a generator for documentation, but it reduce the power for testing.#2020-05-2321:13niclasnilssonRecursive specs? Iā€™ve tried all old examples Iā€™ve found, but they donā€™t seem to work (any longer?). In spec or spec2, are recursive specs possible, or is eager lookup making it impossible? I mean things like this:
(def ex '(+ (+ 1 (* 2 3)) 200))

(defn operator? [x] (#{'+ '- '* '/} x))

(s/def ::operator operator?)
(s/def ::operand int?)

;; I've tried many different ways, but fail to get something like this working
(s/def ::expr
  (s/or :operand ::operand
        :expr (s/cat :operator ::operator
                     :left ::expr
                     :right ::expr))
#2020-05-2413:56Charles Fourdrignier@niclasnilsson You need to define a reference ::expr before the recursive one.
(s/def ::expr any?)
#2020-05-2506:32niclasnilsson@charles.fourdrignier, well, yes and no. It compiles then, but it still doesnā€™t work. The following compiles, but the expected output is not useful, and not whatā€™s expected from a recursive spec. (see output from explain and conform)
(require '[clojure.alpha.spec :as s])

(def expr '(+ (+ 1 (* 2 "three")) 200))

(defn operator? [x] (#{'+ '- '* '/} x))

(s/def ::operator operator?)
(s/def ::operand int?)

(s/def ::expr any?)
(s/def ::expr
  (s/or :operand ::operand
        :expr (s/cat :operator ::operator
                     :left ::expr
                     :right ::expr)))

(s/explain ::expr expr) ;; success, but shouldn't be if recursive specs works.

(s/conform ::expr expr)
;; result 
[:expr {:left (+ 1 (* 2 "three")) :operator + :right 200}]
#2020-05-2507:43valeraukoi haven't tried this, but would an alias work? eg.
(s/def ::expression ::expr)
(s/def ::expr
  ; ...
  :left ::expression
  :right ::expresssion)))
#2020-05-2508:20Charles FourdrignierWhich Clojure did you use ? Clojure (from 1.9.0) includes clojure.spec and the namespace is different.
(require '[clojure.spec.alpha :as s])
With this namespace, it works for me (got an invalid for your expr ). Maybe a bugfix done on clojure repository and not in the clojure/spec.alpha ?
#2020-05-2511:10niclasnilsson@UAEH11THP, nope, unforturnately doesnā€™t work.#2020-05-2511:11niclasnilsson@charles.fourdrignier, this is spec 2, but Iā€™ve tried previously on the spec thatā€™s in the box with 1.10 as well.#2020-05-2511:31Charles FourdrignierSorry, I don't get you were using Spec 2.#2020-05-2511:40niclasnilssonBut it seems to work in spec 1, even without the any? trick. Very odd?#2020-05-2511:44Charles FourdrignierYes. I use the any? trick a couple of months ago in Spec 1. Pretty sure some error push me to use it... Very strange.#2020-05-2511:28niclasnilssonHmmā€¦ But this actually seems to work in ā€œspec 1ā€, (clojure 1.10.1) but not in spec 2.
(require '[clojure.spec.alpha :as s])

(def bad-expr '(+ (+ 1 (* 2 "three")) 200))
(def expr '(+ (+ 1 (* 2 3)) 200))

(defn operator? [x] (#{'+ '- '* '/} x))

(s/def ::operator operator?)
(s/def ::operand int?)

(s/def ::expr
  (s/or :operand ::operand
        :expr (s/cat :operator ::operator
                     :left ::expr
                     :right ::expr)))

(s/valid? ::expr bad-expr)  ; false
(s/explain ::expr bad-expr) ;
(s/conform ::expr bad-expr) ; :clojure.spec.alpha/invalid

(s/valid? ::expr expr)   ; true
(s/explain ::expr expr)  ; success
(s/conform ::expr expr)
#2020-05-2512:45Alex Miller (Clojure team)There are some known issues with eager spec dereferencing in spec 2#2020-05-2513:20niclasnilssonAh, ok. Thanks @alexmiller.#2020-05-2619:37wagjoHi, using latest spec2 alpha, I get an exception, is this a known issue or am I doing something wrong?
(s/def :foo/a string?)
(s/def :foo/b :foo/a)
(s/def :foo/m (s/schema [:foo/a :foo/b]))
(s/valid? :foo/m {:foo/a "s" :foo/b "d"})
Unhandled java.lang.IllegalArgumentException
   No implementation of method: :conform* of protocol:
   #'clojure.alpha.spec.protocols/Spec found for class:
   clojure.lang.Keyword
#2020-05-2619:44Alex Miller (Clojure team)it's a known issue for aliased specs#2020-05-2619:45Alex Miller (Clojure team)in that they don't work right#2020-05-2619:47wagjothanks, I'll use (s/def :foo/b (s/and :foo/a)) as a workaround for now#2020-05-2820:32David PhamSaying you a spec ::a , you can get back the definition with s/form. Is it possible to extract a generator from the s/form?#2020-05-2820:46Alex Miller (Clojure team)sure, just eval to to get a spec object, then s/gen#2020-05-2820:47Alex Miller (Clojure team)but if you have ::a, you're already there, so not sure what you're driving at#2020-05-2820:48Alex Miller (Clojure team)one big caveat is that that s/form does not currently capture when a generator is overridden with like s/with-gen#2020-05-2904:29David PhamHow do You eval? I actually want the default generator because I override it, but for testing i would also like to test the generic one.#2020-05-2913:14Alex Miller (Clojure team)with the eval function?#2020-05-2913:15Alex Miller (Clojure team)(s/gen (eval (s/form ::a)))#2020-05-2919:30jacklombardHello, I want to introduce clojure spec to validate API endpoints. The parsed response naturally does not have namespaced keywords. How do I go about validating the data? Should I simply use unqualified keys? Where should I keep my spec?#2020-05-2919:31jacklombardI know it is a very common question of where to put spec but can't find the right answer#2020-05-2919:34Alex Miller (Clojure team)I answered these in #beginners - in the future, it's best to put questions in just one channel#2020-05-2919:55David PhamIs it not a bad thing to use the eval function?#2020-05-2920:16Alex Miller (Clojure team)it's not inherently bad, it's just a tool#2020-05-2920:17Alex Miller (Clojure team)Clojure evals all of your expressions after all
#2020-05-2920:58practicalli-johnI have a simple function and an associated fdef specification, but am not getting an error when calling the function with incorrect arguments. The function should take a map that is a ::customer specification. Have I misunderstood something?
(defn customer-fullname
  "Return customer full name from customer details"
  [customer-details]
  (str (::first-name customer-details)
       "_"
       (::last-name customer-details)))

(spec/fdef customer-fullname
  :args (spec/cat :customer ::customer)
  :ret string?
  :fn #(= (:ret %)
          (str (::first-name :args) " " (::last-name :args))))

(spec/def ::first-name string?)
(spec/def ::last-name string?)
(spec/def ::email-address
  (spec/and string?
            #(re-matches #"^[a-zA-Z0-9._%+-]
I would have expected a call to the customer-fullname to fail when the wrong kind of arguments are passed
(customer-fullname "customer")
#2020-05-2921:01practicalli-johnThe ::customer spec works with spec/valid? and spec/assert when I use them with :pre and :post conditions in a function definition, but seem to be missing something when using spec/fdef.#2020-05-2921:25practicalli-johnAh, I have now learned about instrumenting the fdef specifications, and now it works, well fails when I expect it too...
(spec-test/instrument `customer-fullname)
#2020-05-2921:31practicalli-johnI am not clear on if it is valuable to use fdef without instrumenting them. Some discussions suggest is it but I have not really understood why as yet. I am still not that clear on the :fn aspect of fdef so will be on the look out for more examples. I still have a lot to understand about spec.#2020-05-2921:34Alex Miller (Clojure team)you can see the specs in (doc customer-fullname)#2020-05-2921:35seancorfield@jr0cket You can also spec-test/check them, separate from instrumentation.#2020-05-2921:35Alex Miller (Clojure team)^^#2020-05-2921:35Alex Miller (Clojure team)those are the main benefits#2020-05-2921:36seancorfieldIn particular, instrumentation checks :args passed in are correct (and ignores :ret and :fn). Check is generative and passes in conforming random arguments (per the :args spec) and then checks :ret and :fn are satisfied.#2020-05-2921:38practicalli-johnI can see the specs in the docs, yes that is very useful (had forgotten that in my frustration to get the code to fail). Will try out spec-test/check in the morning, sound promising. Thank you both.#2020-05-3111:07plexusI'm trying to figure out how to prevent my recursive spec-based generators from StackOverflowing. From what I can tell the checks that are built in based on *recursion-limit* are moot as soon as you use with-gen.#2020-06-0713:02Vincent CantinThatā€™s really a coincidence, because I am currently working days and days on overcoming this problem.#2020-06-0713:02Vincent CantinYou might want to give Minimallist a try.#2020-06-0713:04Vincent Cantin@U07FP7QJ0 let me know if you want to try the pre-release repo.#2020-06-0810:27plexuswhat's your sales pitch? why should I consider Minimalist over Malli (my preferred solution nowadays), and does it solve the problem I linked to?#2020-05-3111:09plexusI currently have two places where I'm using with-gen that cause recursion, https://github.com/lambdaisland/regal/blob/master/src/lambdaisland/regal/spec_alpha.cljc#L44-L47#2020-05-3111:09plexushttps://github.com/lambdaisland/regal/blob/master/src/lambdaisland/regal/spec_alpha.cljc#L85-L95#2020-05-3111:11plexusThe second one I could get away without the custom generator, it'll just do some extra work until it finds a valid value, but the first one is annoying. The spec boils down to (s/and (s/cat ...) vector?), but (s/cat ...) never generates vectors so when generating this tries 100 times and gives up#2020-05-3111:11plexusmaybe there's a better way to write that? can I use regex specs but still convey that I really only want to match/generate vectors?#2020-05-3114:48Alex Miller (Clojure team)Not easily (this is something weā€™ve added in spec 2)#2020-05-3115:41claytonHello! I've recently done some reading on clojure spec and have added it to some side projects of mine, but I'm a little lost on how/when to use instrumentation (or orchestra). I've spec'd multiple functions in my project, but realize those are useless (aside from readability) without running some sort of instrumentation. Are there any resources explaining how to integrate instrumentation in a clojure project? I don't want to permanently leave a call to instrument inside my clojure files, and only want them to really be run during development phase. Aside from just running instrumentation in the REPL during the dev process, are there any best practices for instrumenting in my project? Thanks šŸ™‚#2020-05-3116:48Alex Miller (Clojure team)most useful in dev at the repl#2020-05-3116:49Alex Miller (Clojure team)but could also be useful to turn on/off in a fixture around tests#2020-05-3116:55claytonThanks! I've only ever used instrumentation in tests or the repl, but was thinking I was missing something. Back to the specs I go šŸƒ#2020-05-3118:10seancorfield@UB4EZH42F In the tests for seancorfield/next-jdbc I turn on instrumentation as part of test setup in several of the test namespaces.#2020-05-3118:11claytonwill check out the repo for examples - thanks!#2020-05-3118:11seancorfieldI talk about how we use Spec at work in https://corfield.org/blog/2019/09/13/using-spec/ if that helps.#2020-06-0101:21Eccentric JA friend shared this with me https://gizmo385.github.io/clojure/programming/2015/08/11/adts-in-clojure.html which ended with a macro for (data Tree = Empty | Leaf value | Node left right). I'm trying to write the spec for something like that just for learning purposes. I got to (s/def ::tree (s/or empty?)) before getting a bit stuck šŸ˜…#2020-06-0101:24Eccentric JMy first question is can you compose specs together like (def ::leaf ...) (def ::node ...) (def ::tree (s/or empty? ::leaf :;node))?#2020-06-0101:30seancorfieldI think you'd need "tagged" data structures and multi-spec for that @jayzawrotny#2020-06-0101:31Eccentric JAh thanks for the pointer!#2020-06-0101:54Eccentric JLooks like someone had started down that path https://github.com/bluesxman/adt-spec/blob/master/src/adt_spec/core.clj#2020-06-0102:32Eccentric JFound from https://gist.github.com/Olical/f2b934873a49c0638ca673ab764a0131
(s/def ::element (s/or :string string?
                       :element (s/cat :name keyword?
                                       :attrs (s/? (s/map-of keyword? any?))
                                       :children (s/* ::element))))
#2020-06-0102:33Eccentric JThat's really close I think!#2020-06-0102:44Eccentric J
(s/def ::tree (s/or :empty empty?
                    :node (s/tuple (s/? ::tree) (s/? ::tree))
                    :leaf number?))
#2020-06-0102:45Eccentric JIt's getting [] => :empty, but failing on anything else#2020-06-0102:56Eccentric J
(ns spec-intro.core
  (:require
   [clojure.spec.alpha :as s]))


(s/def ::tree (s/or
               :empty (s/and coll? empty?)
               :node (s/cat :left (s/? ::tree)
                            :right (s/? ::tree))
               :leaf number?))

(def tree [0 [1 [2 3]]])

[(s/conform ::tree tree)
 (s/conform ::tree [1 0])
 (s/conform ::tree [])
 (s/conform ::tree 1)]
#2020-06-0102:57Eccentric JGot it!#2020-06-0114:55plexusregarding my question from yesterday re. infinite recursion in generators, I managed to work around it with size/`resize`, basically making sure that nested generators get increasingly smaller size parameters, and always emitting terminal tokens at small sizes https://github.com/lambdaisland/regal/blob/master/src/lambdaisland/regal/spec_alpha.cljc#L21-L33#2020-06-0114:56plexusmaybe that's useful to someone šŸ™‚#2020-06-0308:20MorongƖaI want to use phrase to produce human friendly errors messages. Has anyone used this before? What are your comments?#2020-06-0402:20nmkipI have a map that has this shape {"account" {"active" true "balance" 30}} , what's the best way to spec the "account" key? I've tried these and I'm using the first one.
(s/def ::account-key (s/with-gen (s/and string?
                                        #(= "account" %))
                       #(gen/return "account"))) 
(s/def ::account-key (s/with-gen (s/and string?
                                        #{"account"})
                       #(gen/return "account"))) 
#2020-06-0402:20seancorfieldYou can use #{"account"} to spec a known set of string values.#2020-06-0402:21seancorfield(and that will also generate correctly)#2020-06-0402:21nmkipjust #{"account"} that makes sense#2020-06-0402:21seancorfield(s/def ::account-key #{"account"})#2020-06-0402:21nmkipyes#2020-06-0402:22nmkipgreat, thanks! šŸ™‚ I'll use that instead, much shorter#2020-06-0402:23nmkipI was being redundant here:
(s/def ::account-key (s/with-gen (s/and string?
                                        #{"account"})
                       #(gen/return "account"))) 
#2020-06-0402:24seancorfieldInventive... but certainly "overkill" šŸ™‚#2020-06-0402:24nmkipindeed#2020-06-0413:05dev.4openIDLearner. Maybe a dumb question. I have built up a detailed spec using s/def s/valid? s/conform. It works fine. Following best practices all my keywords use :: (e.g. ::timestamp). However, my data covered JSON to map, all keywords are single colon (e.g. :timestamps) When I construct the data with two colons the valid? and conform has no problem whereas the mapped data with single colon fails. I was understanding the :: indicates the keyword in this namesapace so having :: in my spec vs. the single : in may data should not matter. Any clarifications on this? šŸ¤” <script src="https://gist.github.com/dev4openid/1becc5be61b764e330edc82d16f99eb6.js"></script>#2020-06-0413:05dev.4openIDLearner. Maybe a dumb question. I have built up a detailed spec using s/def s/valid? s/conform. It works fine. Following best practices all my keywords use :: (e.g. ::timestamp). However, my data covered JSON to map, all keywords are single colon (e.g. :timestamps) When I construct the data with two colons the valid? and conform has no problem whereas the mapped data with single colon fails. I was understanding the :: indicates the keyword in this namesapace so having :: in my spec vs. the single : in may data should not matter. Any clarifications on this? šŸ¤” <script src="https://gist.github.com/dev4openid/1becc5be61b764e330edc82d16f99eb6.js"></script>#2020-06-0413:12aisamuThe conversion from JSON will probably give you unqualified keywords, since no such concept exists there. You have to either qualify your keywords on a separate post-processing step, or make a spec for the payload that takes unqualified keys (e.g. :req-un instead of :req)#2020-06-0413:23dev.4openID@U3UFFB420 Thanks for the explanation. But..... why is the :timestamp not found in the namespace or is it global? If global than how do I relate it to my incoming data - as it were. Because the spec is not of use right now. Obviously I do not see the connection#2020-06-0413:24dev.4openIDSo that is the purpose of req-un then? Does that make the :keyword global?#2020-06-0413:30franquitoMaybe you are mixing concepts. Keywords are just keywords, they can be qualified (`:test/timestamp`) or unqualified (`:timestamp`). :: It's a shortcut to write a qualified keyword with the same namespace as the current file.#2020-06-0413:31aisamuKeywords can be qualified or unqualified. Unqualified can be thought of as "global", but it's better to stick with "lacking a namespace". When you type in ::timestamp, Clojure uses the current namespace and creates a qualified keyword. req-un means that the spec will match unqualified keys to the qualified keys, comparing just the name (i.e. discarding the namespace)#2020-06-0413:32dev.4openIDSo in a more complex map structure, where I have several collections do I have to req-un for all definitions or only at the top level and it is inferred?#2020-06-0413:33dev.4openID'Cos I req-un all my defs and now it does not work#2020-06-0413:42aisamuYes, you'd have to req-un all of them. Try breaking it into smaller pieces to find where it's failing#2020-06-0413:42dev.4openIDhere is the gist <script src="https://gist.github.com/dev4openid/1becc5be61b764e330edc82d16f99eb6.js"></script>#2020-06-0413:42dev.4openIDIt is detailled#2020-06-0413:44aisamuThe spec for ::loca seems a bit off#2020-06-0413:45dev.4openIDseems to work.... what do you mean?#2020-06-0413:46dev.4openIDIf it is all :req for keywords exampl2 works fine and conforms. So I am confused here?#2020-06-0413:46aisamuUnless this is spec2, I don't think this is a valid definition: (s/def ::loca (s/keys :req-un {::addr [::country]}))#2020-06-0413:46aisamuIIRC, req & req-un take a vector of keys, not a map#2020-06-0413:47aisamuMight have worked by accident, or because you had a correct definition earlier that was stored in the registry#2020-06-0413:48dev.4openIDwould (s/def ::loca :req {::addr [::country]}) do it? no it does not#2020-06-0413:49dev.4openIDSurely the country spec forces a s/keys?#2020-06-0413:52aisamuI've never seen that syntax (feeding a map to :req). Does it work on a simple case?#2020-06-0413:53dev.4openID2 rows above and it works -> (s/def ::addr (s/keys :req [::country])) (s/conform ::addr {::country "USA"}) ;; => #:myproj.trial.spec{:country "USA"}#2020-06-0413:53dev.4openIDIt is effectively a map of map, no?#2020-06-0413:55dev.4openIDand :loca is the map of the above. But if I do not have s/keys it seems now not to work#2020-06-0413:56aisamuThese two are not the same:
(s/def ::addr (s/keys :req-un [::country]))
(s/def ::loca (s/keys :req-un {::addr [::country]}))
You're feeding a vector to the first req, but a map to the second
#2020-06-0413:57dev.4openIDAgreed but I tried ::loca {:: addr {::country and it will not accept it#2020-06-0414:06dev.4openID@U1UQEM078 BTW what do you mean as spec2? I am using clojure.spec.alpha latest version#2020-06-0415:00dev.4openID@U3UFFB420 I agree with you, how does one relate the one to the other? I changed my spec to have :reg-un at the highest level and it fails spec is at <script src="https://gist.github.com/dev4openid/1becc5be61b764e330edc82d16f99eb6.js"></script>#2020-06-0415:06franquitoTry first working with simpler examples, try to create a spec for a map with unqualified keys#2020-06-0415:07dev.4openIDIf you look at the gist that Is what I am doing? No?#2020-06-0415:10dev.4openIDWhen all the keys are NOT reg-un then all my test cases pass#2020-06-0415:10franquitoWell, https://gist.github.com/dev4openid/1becc5be61b764e330edc82d16f99eb6#file-myproj-trial-spc-clj-L19 you need to change it to (s/def ::loca (s/keys :req-un [::addr]))#2020-06-0415:10dev.4openIDand so my spec seems OK ??#2020-06-0415:11dev.4openIDOK how? I know it is indicating a map but if I do not use s/keys it fails#2020-06-0415:11dev.4openIDhttps://app.slack.com/team/UEGT2541JĀ Ā [2:57 PM] Agreed but I tried ::loca {:: addr {::countryĀ Ā  and it will not accept#2020-06-0415:12dev.4openIDEven though I initially figured it wasa map of map of map#2020-06-0415:14dev.4openIDOh oh!!!! I see it now!#2020-06-0415:15franquitoI see several places you use :req-un instead of :req, which is the correct way for conforming/validating these examples#2020-06-0415:16franquitoAs I said, try creating a simpler spec for a non nested map and then try validating maps#2020-06-0415:17dev.4openIDOk. But how do I use the spec against the data map (ex JSON) which is :keyword and not ::timeline (IYKWIM)#2020-06-0415:20franquitoHere's an example for a qualified map
(ns myproj.trial.spec)

(s/def ::country string?)
(s/def ::street string?)

(s/def ::addr (s/keys :req [::country ::street]))

(s/valid? ::addr {::country "USA"
                  ::street "A street"})
;; => true

(s/valid? ::addr {:country "USA"
                  :street "A street"})
;; => false

(s/valid? ::addr {:myproj.trial.spec/country "USA"
                  :myproj.trial.spec/street "A street"})
;; => true
#2020-06-0415:23dev.4openIDyes, without changing the above, how do you apply it to a map ex JSON) {:addr {:country "USA :street "A street"}}#2020-06-0415:23franquitoAs aisamu said before, if you are working with JSON you loose the namespace of the keywords, so you probably want to use :req-un in your specs.
(ns myproj.trial.spec)

(s/def ::country string?)
(s/def ::street string?)

(s/def ::addr (s/keys :req-un [::country ::street]))

(s/valid? ::addr {:country "USA"
                  :street "A street"})
;; => true
#2020-06-0415:24dev.4openIDI was advise to use :req-un to allow for unqualif namesapces#2020-06-0415:25franquitoRight, but then when you use valid? or conform you also have to use unqualified keywords for the map https://gist.github.com/dev4openid/1becc5be61b764e330edc82d16f99eb6#file-myproj-trial-spc-clj-L69-L81#2020-06-0415:40dev.4openID(s/def ::country string?) (s/def ::addr (s/keys :req-un [::country])) (s/def ::loca (s/keys :req-un [::addr ::country ])) (s/conform ::country "USA") ;; => "USA" (s/conform ::addr {:country "USA"}) ;; => {:country "USA"} (s/conform ::loca {:addr {:country "USA"}}) ;; => #:myproj.trial.spec{:addr #:myproj.trial.spec{:country "USA"}} (s/explain-str ::loca {:addr {:country "USA"}}) ;; => "{:addr {:country \"USA\"}} - failed: (contains? % trial.spec/addr) spec: :trial.spec/loca\n{:addr {:country \"USA\"}} - failed: (contains? % trail.spec/country) spec: :parceltrax.tracking.spec/loca\n" ??why this??#2020-06-0415:47dev.4openIDI updated the gist <script src="https://gist.github.com/dev4openid/1becc5be61b764e330edc82d16f99eb6.js"></script> If you look at line 16 When I look sat it it is a map of map#2020-06-0415:53franquitoThis line is wrong https://gist.github.com/dev4openid/1becc5be61b764e330edc82d16f99eb6#file-myproj-trial-spc-clj-L19#2020-06-0415:53franquitoTry conforming this
(s/conform ::loca {:addr {:country "USA"}
                   :country "OTHER"})
#2020-06-0415:55dev.4openIDYes, in theory that will work, however the data looks like this :pieceID ["ABC1" "DEF2" "GHI3"]}]}) (def exampl2 {::abc [{::loca {::addr {::country "USA"}} ::dets {::some{::timestamp "2020-04-09T15:27:00" ::Url "https://mywebpage.com/answer"#2020-06-0415:57dev.4openIDLook at line 1 of the gist#2020-06-0415:57dev.4openIDso loca is a map of map of map#2020-06-0415:58dev.4openIDThe trouble I a,mm having is the compounded map that does not seems to play nicely#2020-06-0416:06dev.4openID
OK - it does conform to the definition Thx.  However it is not as per line 1 - which is how the data is meant to be.   So extracting from line 1:  :loca {:addr {:country "USA"}}
#2020-06-0416:09franquitoYes, you need to modify ::loca so it looks like this (s/def ::loca (s/keys :req-un [::addr]))#2020-06-0416:09dev.4openID@U3UFFB420 OK !!!! I think I have it ! I must take out the ::country in the s/def as the ::country is already defined (s/def ::loca (s/keys :req-un [::addr]))#2020-06-0416:09dev.4openIDI think that is what you were trying to get me to see#2020-06-0416:10franquitoYes, exactly! šŸ™‚#2020-06-0416:10dev.4openIDAppreciated#2020-06-0416:26dev.4openID@U3UFFB420 Thanks for your patience and help. I got it now!! my complete script now works! Kudos for you šŸ™‚#2020-06-0413:12franquitoThe difference is that :: expands to the current namespace. So if you're working in a ns called my.test then ::timestamp expands to :my.test/timestamp. The spec fails with :timestamp because it's also expecting the namespace.#2020-06-0419:30dev-hartmannHi fellow clojurians, is anyone aware of a way to parse a swagger json to valid specs that can be used to generate test data?#2020-06-0910:40elimisteveThat's a great idea and something I'm interested in, too#2020-06-0911:04dev.4openIDThis is what I did to learn spec. From this point on one can gen data. What you see is a map converted from a JSON structure. Now this is not a "take a swagger spec and convert" but is the basis of how to clojure spec based on a swagger that I did.
(def exampl {:abc [{:loca {:addr {:country "USA"}}
                    :dets {:some{:timestamp "2020-04-09T15:27:00"
                                 :Url ""
                                 :prod {:name "this product"}}}
                    :totalNumber 399
                    :pieceID ["ABC1" "DEF2" "GHI3"]}]})


(s/def ::country string?)
(s/def ::addr (s/keys :req-un [::country]))
(s/def ::loca (s/keys :req-un [::addr]))

(s/conform ::country "USA") ;; => "USA"
(s/conform ::addr {:country "USA"}) ;; => {:country "USA"}
(s/conform ::loca  {:addr {:country "USA"}}) ;; => {:addr {:country "USA"}}
(s/valid? ::loca  {:addr {:country "USA"}})  ;; => true
(s/check-asserts?)
(s/check-asserts true)                       ;; by default false!!
(s/assert ::loca  {:addr {:country "USA"}})  ;; => {:addr {:country "USA"}}
(s/assert ::loca  {:addr {:countryx "SA"}})  ;; note the exception error BUT look at the repl

(s/def ::timestamp (and not nil?
                        string?))     ;; this needs work for T-time issue
(s/def ::Url string?)                 ;; need to validate URLs - technique
(s/def ::name string?)
(s/def ::prod (s/keys :req-un [::name]))
(s/def ::some (s/keys :req-un [::timestamp ::Url ::prod]))
(s/def ::dets (s/keys :req-un [::some]))

(s/conform ::timestamp "2020-04-09T15:27:00")
(s/conform ::Url "")
(s/conform ::name "this product")
(s/conform ::prod {:name "this product"})
(s/conform ::some {:timestamp "2020-04-09T15:27:00"
                   :Url ""
                   :prod {:name "this product"}})
(s/conform ::dets {:some {:timestamp "2020-04-09T15:27:00"
                           :Url ""
                           :prod {:name "this product"}}})

(s/def ::totalNumber (and pos? int?))
(s/def ::pieceID (and (s/coll-of string?)
                      vector?))
(s/conform ::totalNumber 399) ;; => 399
(s/conform ::pieceID ["ABC1" "DEF2"]) ;; => ["ABC1" "DEF2"]

(s/def ::abc (s/coll-of (s/keys :req-un [::loca ::dets ::totalNumber ::pieceID])))
(s/conform ::abc [{:loca {:addr {:country "USA"}}
                   :dets {:some{:timestamp "2020-04-09T15:27:00"
                                :Url ""
                                :prod {:name "this product"}}}
                   :totalNumber 399
                   :pieceID ["ABC1" "DEF2" "GHI3"]}])

;; now ex is an map of a single instance)
ex
;; => {:abc [{:loca {:addr {:country "USA"}}, :dets {:some {:timestamp "2020-04-09T15:27:00", :Url "", :prod {:name "this product"}}}, :totalNumber 399, :pieceID ["ABC1" "DEF2" "GHI3"]}]}

(s/def ::an_instance (s/keys :req-un [::abc]))
(s/conform ::an_instance {:abc [{:loca {:addr {:country "USA"}}
                                 :dets {:some{:timestamp "2020-04-09T15:27:00"
                                              :Url ""
                                              :prod {:name "this product"}}}
                                 :totalNumber 399
                                 :pieceID ["ABC1" "DEF2" "GHI3"]}]})

(s/conform ::an_instance exampl)
(s/valid? ::an_instance exampl)
(s/explain-str ::an_instance exampl)
Also look at https://www.youtube.com/watch?v=f2hNQdS2VxQ to watch a conversion - the nearest I have found. Good luck with this šŸ˜‰
#2020-06-0914:19dev-hartmannhey @UEGT2541J thanks for joining in. actually the challenge is not to create a clojure map from the json and then writing the spec. but parsing the json into specs . the difference being that I don't want to write them manually. but you are absolutely right, the chain is: json -> clojure-map -> transform clojure map entries to spec entries and compose them to upper level spec. then we can use the gen functions to create test data#2020-06-0914:21dev-hartmannif we have a working parser for that, we can just call the endpoint on webservices, slurp the swagger json and create specs for data models#2020-06-0914:22dev-hartmannwhich is a handy little tool šŸ˜„#2020-06-0914:26dev-hartmannI'm not an expert for specs, but searching a swagger yaml/ json for the definitions#2020-06-0914:27dev-hartmannand then using the type information there and maping those and the field names to specs sounds doable šŸ˜„#2020-06-0914:27dev-hartmann
definitions:
  Order:
    type: "object"
    properties:
      id:
        type: "integer"
        format: "int64"
      petId:
        type: "integer"
        format: "int64"
      quantity:
        type: "integer"
        format: "int32"
      shipDate:
        type: "string"
        format: "date-time"
      status:
        type: "string"
        description: "Order Status"
        enum:
        - "placed"
        - "approved"
        - "delivered"
      complete:
        type: "boolean"
        default: false
    xml:
      name: "Order"
#2020-06-0914:27dev-hartmanni like structured data šŸ˜„#2020-06-0914:28dev-hartmannI'll work on that a bit an will put a link to a gist here if I can create something useful#2020-06-0917:44flyboarder@U0J8XN37Y im really interested in this, we just rolled out swagger to our APIā€™s both internally and externally, I would love to be able to have our apps handle specs for us#2020-06-0917:46dev-hartmannwill keep you posted too.#2020-06-0917:47dev-hartmannI think it's easier than I thought.#2020-06-0917:47dev-hartmann.. I mean until I hit the brick wall šŸ˜„#2020-06-0917:55dev-hartmannat least the basics. translating the conditions on properties can take some time šŸ˜„#2020-06-0920:29dev.4openIDOK, similar to what you are stsing then: look st the library serene - It autoigenerates the specs for REST and graphql APIs.#2020-06-0920:30dev.4openIDI'll shut up now šŸ™‚#2020-06-0922:05dev-hartmannno, please don't. never heard of that library and it looks like they are doing the same thing, but for a different format. šŸ‘#2020-06-0514:09Felipe Marques@ikitommi Hi, a few days ago you mentioned some other libs to perform spec coercion, but I forgot to take notes on its names to check it later. Could you share the names again? Also, I'm having some trouble with coercing some data using spec-tools. I'm not sure if I got the logic right. Do you have any resource explaining how the coercion works in spec-tools? Thanks very much!#2020-06-0514:41ikitommiHi @marques.goncalves.fel! The slack history (& my message) is here: https://clojurians-log.clojureverse.org/clojure-spec/2020-05-19. I believe all spec-coercion libs are doing it in the same way, : given a spec, it's form is parsed and for all different specs (`or`, and, keys, integer?, any? etc.) a transforming function is selected and applied to the value. Corcion is recursive and best effort. With spec-tools:
(require '[clojure.spec.alpha :as s])
(require '[spec-tools.core :as st])

(s/def ::spec (s/nilable
                (s/nilable
                  (s/map-of
                    keyword?
                    (s/or :keys (s/keys :req-un [::c1])
                          :ks (s/coll-of (s/and int?) :into #{}))))))

(def value {"keys" {:c1 "1" ::c2 "kikka"}
            "keys2" {:c1 true}
            "ints" [1 "1" "invalid" "3"]})

(st/coerce ::spec value st/string-transformer)
;{:keys {:c1 "1", :test/c2 "kikka"}
; :keys2 {:c1 true}
; :ints #{1 "invalid" 3}}

(st/coerce ::spec value st/json-transformer)
;{:keys {:c1 "1", :test/c2 "kikka"}
; :keys2 {:c1 true}
; :ints #{"3" 1 "invalid" "1"}}
#2020-06-0514:41ikitommiHi @marques.goncalves.fel! The slack history (& my message) is here: https://clojurians-log.clojureverse.org/clojure-spec/2020-05-19. I believe all spec-coercion libs are doing it in the same way, : given a spec, it's form is parsed and for all different specs (`or`, and, keys, integer?, any? etc.) a transforming function is selected and applied to the value. Corcion is recursive and best effort. With spec-tools:
(require '[clojure.spec.alpha :as s])
(require '[spec-tools.core :as st])

(s/def ::spec (s/nilable
                (s/nilable
                  (s/map-of
                    keyword?
                    (s/or :keys (s/keys :req-un [::c1])
                          :ks (s/coll-of (s/and int?) :into #{}))))))

(def value {"keys" {:c1 "1" ::c2 "kikka"}
            "keys2" {:c1 true}
            "ints" [1 "1" "invalid" "3"]})

(st/coerce ::spec value st/string-transformer)
;{:keys {:c1 "1", :test/c2 "kikka"}
; :keys2 {:c1 true}
; :ints #{1 "invalid" 3}}

(st/coerce ::spec value st/json-transformer)
;{:keys {:c1 "1", :test/c2 "kikka"}
; :keys2 {:c1 true}
; :ints #{"3" 1 "invalid" "1"}}
#2020-06-0514:43ikitommiwrote few years back: https://www.metosin.fi/blog/spec-transformers/#2020-06-0514:52Felipe MarquesThanks for the links and explanation. Sorry for not finding the history. I looked only in slack! šŸ˜…#2020-06-0520:53dev.4openIDLearner here. Dumb Question maybe, maybe a mindset I have a map (ex JSON) in it exists the following: {:status {:timestamp "2020-04-09T15:27:00" :status "abc"} Yes, I suppose I could change the data so that the 1st :status has a different name but that is not very elegant. If the structure remains the same, I assume you cannot redefine the :status again within the spec Is there a way int spec (some technique) that deals with this type of anomaly? I have a data set exactly like that. Any thoughts?#2020-06-0521:08Alex Miller (Clojure team)s/keys with :req-un will match the short name of the provided spec but can use different qualified specs#2020-06-0521:10Alex Miller (Clojure team)
(s/def :inner/status string?)
(s/def :inner/timestamp string?)
(s/def :outer/status (s/keys :req-un [:inner/status :inner/timestamp]))
(s/def :outer/map (s/keys :req-un [:outer/status]))
#2020-06-0521:26dev.4openID(s/def :inner/status string?) (s/def :inner/timestamp string?) (s/def :outer/status (s/keys :req-un [:inner/timestamp :inner/status])) (s/def :outer/map (s/keys :req-un [:inner/status])) (s/valid? :outer/map {:status {:timestamp "2020-09" :status "abc" }}) Hi, @alexmillerseems to fail against the data. Perhaps I miss it? (s/explain-str :outer/map {:status {:timestamp "2020-09" :status "abc" }}) ;; => "{:timestamp \"2020-09\", :status \"abc\"} - failed: string? in: [:status] at: [:status] spec: :inner/status\n"#2020-06-0521:47seancorfield@dev.4openid That works for me:
user=> (require '[clojure.spec.alpha :as s])
nil
user=> (s/def :inner/status string?)
:inner/status
user=> (s/def :inner/timestamp string?)
:inner/timestamp
user=> (s/def :outer/status (s/keys :req-un [:inner/status :inner/timestamp]))
:outer/status
user=> (s/def :outer/map (s/keys :req-un [:outer/status]))
:outer/map
(s/valid? :outer/map {:status {:timestamp "2020-09"
                                                    :status "abc" }})
true
(s/explain-str :outer/map {:status {:timestamp "2020-09"
                               :status "abc" }})
"Success!\n"
user=> 
(you can use triple-backticks around your code to format it and make it easier to read)
#2020-06-0521:59dev.4openIDHi @alexmiller and @seancorfield - thanks for the help - I am wrong: my fat fingers and compiling versions of the trial and error - I screwed up! šŸ˜­šŸ˜€ - along day!#2020-06-0607:05practicalli-johnQuestion about spec keys. It seems I can use either qualified keys with :req or unqualified keys with :req-un . I cannot use qualified keys with :req-un. So if I have keys that could be qualified or unqualified, would I have req and req-un sections in the keys
(spec/def ::customer-details
  (spec/keys
    :req [::first-name ::last-name ::email-address ::residential-address ::social-security-id]
    :req-un [::first-name ::last-name ::email-address ::residential-address ::social-security-id]))
Or I assume I should just choose to use one or the other. I could just add the #:: auto-resolve macro to hash-maps with unqualified keys...
#2020-06-0607:14practicalli-johnAdding a specific namepace in front of the hash-map, #:practicalli.bank-account-spec seems to work too and feels like a better approach
(spec/explain :practicalli.bank-account-spec/customer-details
              #:practicalli.bank-account-spec
              {:first-name          "Jenny"
               :last-name           "Jetpack"
               :email-address       "
#2020-06-0607:26AronI haven't really had time to watch all the videos, is there one where the organization of the namespaces (especially in relation with models, specs, entities)? I am coming from js where circular dependencies are allowed in certain circumstances and I find it a bit too much work to create new files to create new namespaces (note, it's not the problem that I have to create files, I love lots of tiny files, I just don't like the now additional work that means I have to constantly rewrite all my files if I want to reorganize just a couple.)#2020-06-0607:28practicalli-john@ashnur I will be covering a bit of organisation in the broadcast today in about 30 minutes https://youtu.be/jUgm4zh-vF4 - I create a src, test and spec namespace and have a design-journal namespace to show my working šŸ™‚ It still fairly basic, so you dont need to watch the other videos#2020-06-0607:29Aronthanks, I will be watching#2020-06-0611:36dev.4openIDThere seems some confusion about :req and :req-un in regards to specs. In regards to a s/valid? or s/explain-str a lot of error info is generated. So you s/def uses :: and can compose other s/defs with :: However when using it with s/explain-str and it has a :req means (e.g. (s/explain-str ::myspec {:test1 "a" :test2 "b"))} it will generate errors versus (e.g. (s/explain-str ::myspec {::test1 "a" ::test2 "b"})) - assuming test1 and test2 are s/def somewhere. At the moment the workaround is just use req-un and that gets past the problem. However, that seems a hack. Anyone with a good explanation anywhere as to the appropriate use of :req and :req-un. This seems more complicated to the eyeball than "it's good enough". I see similar being asked above#2020-06-0611:36dev.4openIDThere seems some confusion about :req and :req-un in regards to specs. In regards to a s/valid? or s/explain-str a lot of error info is generated. So you s/def uses :: and can compose other s/defs with :: However when using it with s/explain-str and it has a :req means (e.g. (s/explain-str ::myspec {:test1 "a" :test2 "b"))} it will generate errors versus (e.g. (s/explain-str ::myspec {::test1 "a" ::test2 "b"})) - assuming test1 and test2 are s/def somewhere. At the moment the workaround is just use req-un and that gets past the problem. However, that seems a hack. Anyone with a good explanation anywhere as to the appropriate use of :req and :req-un. This seems more complicated to the eyeball than "it's good enough". I see similar being asked above#2020-06-0614:00Alex Miller (Clojure team)First, spec know nothing about :: - this is just a Clojure shorthand to fully resolve a keyword within the current namespace at read time. Spec will just see the full kw as if you had typed :user/myspec. s/keys will always validate all fully qualified keys in the map with their associated registered specs. It will check that the map has all of the specified :req keys (match fully qualified) or :req-un keys (match only the unqualified name). And specially for :req-un it will verify the values validate against the spec for the qualified key.#2020-06-0720:59dev.4openID@U064X3EF3 Hi Alex
(s/def ::timestamp string?)  ;; => :parceltrax.tracking.spec/timestamp
(s/def ::status string?) ;; => :parceltrax.tracking.spec/status
(s/def :o/status (s/keys :req-un [::timestamp ::status])) ;; => :o/status
(s/valid? :o/status {:timestamp "2020-09"
                     :status "abc" }) ;; => true
(s/explain-str :o/status {:timestamp "2020-09"
                          :status "abc" }) ;; => "Success!\n"

(s/def ::mapx (s/keys :req-un [:o/status])) ;; => :parceltrax.tracking.spec/mapx
(s/valid? ::mapx {:status {:timestamp "2020-09"
                           :status "abc" }}) ;; => true
(s/explain-str ::mapx {:status {:timestamp "2020-09"
                                :status "abc" }}) ;; => "Success!\n"
I accept is works and maybe I'm being a little thick here: The solution depends of the "temp" creation of a namespace as a workaround. In this case a :o/status in order to differentiate from the actual :req-un namespace of :status. My perception is this in not very elegant or consistent with the spec model. As is encouraged by the clojure team, developers are to use :: more often. In the simple example provided - the code is not "portable" as such. I recognize it may seem may seem nitpicking - perhaps I have missed the understanding or this is just accept "it works this way" In the real workplace there are many occasions of "recursive dup fields in JSON data. Could you provide a comment or two on this? Much appreciate your efforts
#2020-06-0721:29Alex Miller (Clojure team)Iā€™m not trying to encourage more use of :: and itā€™s typically not what you need (but itā€™s very convenient for examples)#2020-06-0721:30Alex Miller (Clojure team)Personally I most often use full namespaces for data specs#2020-06-0809:32dev.4openIDHi, I was not stating you were the advocate - the spec literature does - just to be clear Any thought about my point of "making up kw names" in conjunction with the point of recursive/nested JSON map structures that employ a keword such as my example :status - is there no recursive strategy for example? I don't know thus I ask.#2020-06-0809:32dev.4openIDThanks for your patience#2020-06-0613:18tianshuIs it expected if :a/b is not defined in this case?
(s/def ::c
  (s/keys :req [:a/b]))


(s/valid? ::c {:a/b {}})
;; => true
#2020-06-0613:18tianshuIs it expected if :a/b is not defined in this case?
(s/def ::c
  (s/keys :req [:a/b]))


(s/valid? ::c {:a/b {}})
;; => true
#2020-06-0613:43dev.4openIDNo. Imagine they are defined It's not about a/b nut when :req or :req-un is appropriate#2020-06-0613:52Alex Miller (Clojure team)I would say yes, this is expected. :req means itā€™s required (and itā€™s there). If there are specs for any keys, they must be followed.#2020-06-0703:29tianshuFor the case (s/def ::a ::b), ::b must be declared before use, but in this case it is not. Should it be unified? (I mean always define before use or always check at runtime)#2020-06-0713:33Alex Miller (Clojure team)Yes, in general resolution should be delayed for all specs and itā€™s not entirely true right now. Not going to make any updates on this in spec 1 though, will work on it in spec 2#2020-06-0613:34hadilsI am having a very difficult time organizing my specs with my code. This is for code that interfaces to an external API. I have a single namespace for the code and an specs.namespace for the specs. The problem is that I have to rename fields in different args and returns for the API code so they won't collide with different uses. Help!!!#2020-06-0613:34hadilsI am having a very difficult time organizing my specs with my code. This is for code that interfaces to an external API. I have a single namespace for the code and an specs.namespace for the specs. The problem is that I have to rename fields in different args and returns for the API code so they won't collide with different uses. Help!!!#2020-06-0613:45dev.4openIDI have just done this and it is OK for code and spec to be in different files I assume you have converted the JSON to a structure. Now define the spec "structure" accordingly Apply s/conform ::spec "JSON structure" - note the spec has :: and your JSON map has : for all elements - in spec make sure all :req are :req-un and it will work#2020-06-0613:50dev.4openIDin my case the file my namespace is ns: myproj.trail.clj and myproj.trial.spec.clj#2020-06-0613:52dev.4openIDLesson learnt model the API JSON map example in spec file fisrst and all becomes more clear. I started from leaves to trunk and not trunk down as seemingly many do#2020-06-0613:53hadilsThanks! @UEGT2541J#2020-06-0613:54hadilsWhat about for multiple JSON outputs in the same spec file? How do I do that?#2020-06-0613:57hadilsThat's the real problem I'm having.#2020-06-0613:57dev.4openIDIf you break down the JSON maps into components and composed elements that are reusable. You then compose complete "trees" for all the responses for all the API calls. There are commonalities in the APIs returns and it goes faster in the compositions as you do more. i.e do 1 at a time. I went with the most complex one (in order to lean spec) and it just became easier to gen the rest.#2020-06-0613:58hadilsHow do you qualify your spec keys?#2020-06-0614:00dev.4openIDIt becopmes more and more obvious. Just build the leaves and branches slowly - use s/valid? and s/explain-str against sample data - extracted from the JSON map eventually build more complex branches and tada! you will compose the trunk. It seem tedious at first (as I was leaning) but it become faster#2020-06-0614:03hadilsI am hung up on the qualified keys like ::foo instead of using :node/foo. The latter seems more appropriate for my needs. How should I approach this?#2020-06-0614:04dev.4openIDlook at the toy example I made first here <script src="https://gist.github.com/dev4openid/5c9320fb8ef2f8f5383836f379ff507e.js"></script> It is tedious at first BUT it gave me insights on how to do things I will leave it up for a while#2020-06-0614:05dev.4openIDin the spec file use :: format for your definitions - saves typing#2020-06-0614:05hadilsThanks!#2020-06-0614:09dev.4openIDnote the :reg-un !!#2020-06-0614:25hadilsYour Gist does not show up properly on my browser.#2020-06-0614:30dev.4openID
(def exampl {:abc [{:loca {:addr {:country "USA"}}
                    :dets {:some{:timestamp "2020-04-09T15:27:00"
                                 :Url ""
                                 :prod {:name "this product"}}}
                    :totalNumber 399
                    :pieceID ["ABC1" "DEF2" "GHI3"]}]})


(s/def ::country string?)
(s/def ::addr (s/keys :req-un [::country]))
(s/def ::loca (s/keys :req-un [::addr]))

(s/conform ::country "USA") ;; => "USA"
(s/conform ::addr {:country "USA"}) ;; => {:country "USA"}
(s/conform ::loca  {:addr {:country "USA"}}) ;; => {:addr {:country "USA"}}
(s/valid? ::loca  {:addr {:country "USA"}})  ;; => true
(s/check-asserts?)
(s/check-asserts true)                       ;; by default false!!
(s/assert ::loca  {:addr {:country "USA"}})  ;; => {:addr {:country "USA"}}
(s/assert ::loca  {:addr {:countryx "SA"}})  ;; note the exception error BUT look at the repl

(s/def ::timestamp (and not nil?
                        string?))     ;; this needs work for T-time issue
(s/def ::Url string?)                 ;; need to validate URLs - technique
(s/def ::name string?)
(s/def ::prod (s/keys :req-un [::name]))
(s/def ::some (s/keys :req-un [::timestamp ::Url ::prod]))
(s/def ::dets (s/keys :req-un [::some]))

(s/conform ::timestamp "2020-04-09T15:27:00")
(s/conform ::Url "")
(s/conform ::name "this product")
(s/conform ::prod {:name "this product"})
(s/conform ::some {:timestamp "2020-04-09T15:27:00"
                   :Url ""
                   :prod {:name "this product"}})
(s/conform ::dets {:some {:timestamp "2020-04-09T15:27:00"
                           :Url ""
                           :prod {:name "this product"}}})

(s/def ::totalNumber (and pos? int?))
(s/def ::pieceID (and (s/coll-of string?)
                      vector?))
(s/conform ::totalNumber 399) ;; => 399
(s/conform ::pieceID ["ABC1" "DEF2"]) ;; => ["ABC1" "DEF2"]

(s/def ::abc (s/coll-of (s/keys :req-un [::loca ::dets ::totalNumber ::pieceID])))
(s/conform ::abc [{:loca {:addr {:country "USA"}}
                   :dets {:some{:timestamp "2020-04-09T15:27:00"
                                :Url ""
                                :prod {:name "this product"}}}
                   :totalNumber 399
                   :pieceID ["ABC1" "DEF2" "GHI3"]}])

;; now ex is an map of a single instance)
ex
;; => {:abc [{:loca {:addr {:country "USA"}}, :dets {:some {:timestamp "2020-04-09T15:27:00", :Url "", :prod {:name "this product"}}}, :totalNumber 399, :pieceID ["ABC1" "DEF2" "GHI3"]}]}

(s/def ::an_instance (s/keys :req-un [::abc]))
(s/conform ::an_instance {:abc [{:loca {:addr {:country "USA"}}
                                 :dets {:some{:timestamp "2020-04-09T15:27:00"
                                              :Url ""
                                              :prod {:name "this product"}}}
                                 :totalNumber 399
                                 :pieceID ["ABC1" "DEF2" "GHI3"]}]})

(s/conform ::an_instance exampl)
(s/valid? ::an_instance exampl)
(s/explain-str ::an_instance exampl)
#2020-06-0614:31dev.4openIDstrange#2020-06-0614:35hadilsThanks a lot @UEGT2541J!!!!#2020-06-0616:22hadils@UEGT2541J what do you do when you are trying to spec 2 different JSON outputs with same names -- some of the fields are identical and some are not, but have the same name?#2020-06-0616:23hadilsDo I need to split them out into separate files?#2020-06-0716:45dev.4openID@UGNMGFJG3 Do you mean 2 fields in the JSON structure, where 1 is subordinate to another? {:name {:person "abc" :name}} or do you mean recurring pasterns that are in a map?#2020-06-0716:45dev.4openIDFor the former: (s/def :inner/status string?) ;; => :inner/status (s/def :inner/timestamp string?) ;; => :inner/timestamp (s/def :outer/status (s/keys :req-un [:inner/timestamp :inner/status])) ;; => :outer/status (s/def :outer/map (s/keys :req-un [:outer/status])) ;; => :outer/map (s/valid? :outer/map {:status {:timestamp "2020-09" :status "abc" }}) ;; => true (s/explain-str :outer/map {:status {:timestamp "2020-09" :status "abc" }}) ;; => "Success!\n"#2020-06-0801:53practicalli-johnAfter quite bit of experimentation, I refactored my specs, code and tests for a bank account and learnt a bit about qualified keys and auto-resolve macro. I have specs that I can use with my clojure.test unit tests I updated the journal to show how I would create the specs, unit tests and code now https://github.com/practicalli/leveraging-spec/blob/master/src/practicalli/bank_account_design_journal.clj And the specs are in their own namespace, using a Clojure Common extension, .cljc (I believe all the specs are host neutral) https://github.com/practicalli/leveraging-spec/blob/master/test/practicalli/bank_account_spec.cljc#2020-06-0801:53practicalli-johnAfter quite bit of experimentation, I refactored my specs, code and tests for a bank account and learnt a bit about qualified keys and auto-resolve macro. I have specs that I can use with my clojure.test unit tests I updated the journal to show how I would create the specs, unit tests and code now https://github.com/practicalli/leveraging-spec/blob/master/src/practicalli/bank_account_design_journal.clj And the specs are in their own namespace, using a Clojure Common extension, .cljc (I believe all the specs are host neutral) https://github.com/practicalli/leveraging-spec/blob/master/test/practicalli/bank_account_spec.cljc#2020-06-0803:01seancorfield@U05254DQM Is this a deliberate bug to fall out of testing later? https://github.com/practicalli/leveraging-spec/blob/master/src/practicalli/bank_account_design_journal.clj#L56#2020-06-0803:10seancorfield:args should be a sequence spec, using s/cat: https://github.com/practicalli/leveraging-spec/blob/master/src/practicalli/bank_account_design_journal.clj#L422#2020-06-0803:11seancorfieldDo you understand why this doesn't work? https://github.com/practicalli/leveraging-spec/blob/master/src/practicalli/bank_account_design_journal.clj#L337-L344#2020-06-0803:13seancorfields/or requires labels for the alternatives: (s/or :qualified (s/keys :req [...]) :unqualified (s/keys :req-un [...]))#2020-06-0807:10practicalli-johnOh yes, labels. At 2.30am I tend to forget things :white_frowning_face: I find fdef a bit opaque, some more reading and experimenting to do. Thanks for spotting the bug in last-name, not intentional. Thanks for the review#2020-06-0813:53practicalli-johnI got the spec/or expression to wrap the :req and :req-un versions working. I have my fdef :args validating after instrumenting. Still working on testing the :ret in the fdef...#2020-06-0816:46seancorfieldRight, :args is for instrument -- checking that your code is calling the function correctly -- and :ret/`:fn` are for generative testing, to make sure your function behaves correctly.#2020-06-0816:47seancorfieldWith instrument, :ret and :fn are ignored. With check, :args is used to generate conforming arguments and then the function is called and the :ret and :fn specs are checked against the return value and against the arguments/return value respectively.#2020-06-0816:48seancorfieldA lot of people don't feel the docs are clear enough about :args/`instrument` and :ret/`:fn`/`check`#2020-06-0817:50practicalli-johnIā€™m comfortable with instrument for now. I donā€™t seem to have something right with check yet, but will look again tomorrow. Thanks.#2020-06-0915:28dev-hartmannhey folks, excuse my noobiness, but is it possible to return a s/def from a function?#2020-06-0915:28dev-hartmannI'm trying to build them dynamically from a parsed json which is working okish to the point where I try to return them#2020-06-0915:35Alex Miller (Clojure team)s/def is a function which I don't think you mean. do you mean a spec form or a spec object?#2020-06-0917:21dev-hartmannAh, yes. Sry, thatā€™s what I mean. I want to create them dynamically, save their names to an atom and then do an s/def#2020-06-0918:09Alex Miller (Clojure team)one way you could do this is to directly manipulate the registry (as s/def's impl does). in spec 2 we've pulled out a non macro function s/register for this purpose.#2020-06-0921:23dev.4openID
(defn len [n] (and (< (count n) 6)
                   string?))

(s/def ::piece-IDs (and vector?
                       (s/coll-of len into [])))

(s/valid? ::piece-IDs ["92257" "12" "01234"] );; => true
(s/conform ::piece-IDs ["92257" "12" "01234"]) ;; => ["92257" "12" "01234"]
(s/explain-str ::piece-IDs ["92257" "12" "01234"]) ;; => "Success!\n"

In the above code the s/def is valid, however it depends on the defn provided - seems untidy and not best practice


Can this not be changed to incorporate the count condition soemwhat as below?
(obvs. the coud does not work as the count does not apply to the strings)

(s/def ::piece-IDs (and vector?
                        (s/coll-of #(and (< count 0)
                                         string?) into [])))
Any ideas to eliminate the defn in the above case?
#2020-06-0921:46Alex Miller (Clojure team)you don't need a custom predicate for that at all#2020-06-0921:46vlaaad
(s/valid? (s/coll-of string? :kind vector? :max-count 5) ["1" "2" "3" "4" "5" "6"])
#2020-06-0921:46Alex Miller (Clojure team)coll-of has a :max-count modifier#2020-06-0921:48Alex Miller (Clojure team)also note that in spec 2, there is now a catv too that would slim this down to just (s/catv string? :max-count 5)#2020-06-0922:24dev.4openIDHi, @vlaaad and @alexmiller (s/def ::pieceIds (s/coll-of string? :kind vector? :max-count 5)) (s/valid? ::pieceIds ["JD014600007821392257" "12" "01234567890123456789" "777777777" "66666666666666666666666666666" "77"]) ;; => false it is counting 6 strings in vector (s/conform ::pieceIds ["JD014600007821392257" "12" "11111111111111112222222222222222222y"]) ;; => ["JD014600007821392257" "12" "11111111111111112222222222222222222y"] (s/explain-str ::pieceIds ["JD014600007821392257" "12" "11111111111111112222222222222222222y"]) ;; => "Success!\n" it is not applying the string length limit of 5 chars produces the count how many string there are; whereas the code I presented determines whether every string length is less than 6 (this being the challenge I find myself in) Subtle difference to what I suggested. Any ideas? BTW Alex: I am using leiningen so I am not sure how to exactly refer spec 2; i.e. I do not use an edn file per se. Perhaps some guidance on that would be helpful - It refers to git etc. so I am unsure#2020-06-0922:49seancorfield@dev.4openid Do you want a :min-count as well as a :max-count?#2020-06-0922:50seancorfieldOh, you want the strings themselves to be checked for length, not the vector?#2020-06-0922:50dev.4openIDYes#2020-06-0922:51seancorfieldSo you want (s/def ::short-string (s/and string? #(>= 5 (count %)))) and then (s/coll-of ::short-string ...)#2020-06-0922:52seancorfield:min-count and :max-count apply to s/coll-of, not to things inside the collection.#2020-06-0922:53dev.4openIDWell, it looks like I had the general idea right šŸ˜‰ but not the implementation. I will try now#2020-06-0923:05dev.4openID@seancorfield No it must be missing something
(s/def ::short-string (and #(>= 5 (count %))
                    string?))

(s/def ::piece-IDs (s/coll-of ::short-string into []))

(s/valid? ::piece-IDs ["92257" "12" "012344444"] );; => true NOT
(s/conform ::piece-IDs ["926257" "12" "01234"]) ;; => ["92257" "12" "01234"] 
(s/explain-str ::piece-IDs ["9992257" "12" "01234"]) ;; => "Success!\n"
#2020-06-0923:07seancorfieldRead what I suggested for ::short-string and compare it to what you wrote.#2020-06-0923:09seancorfield(I just edited mine BTW)#2020-06-0923:09seancorfield(and #(..) string?) is truthy#2020-06-0923:10seancorfield(s/and #(..) string?) is a spec#2020-06-0923:11seancorfieldAlso, it's safer to add the type predicate (e.g., string?) first before applying count otherwise this will blow up (s/valid? ::piece-IDs [42]) because it will try to call (count 42) before testing it is a string.#2020-06-0923:14dev.4openIDYep, I missed it šŸ˜  now it works and I have to discipline myself on the s/. Bit of improvement work!! @seancorfield thanks for the help. Will use string (type checks first) Thx#2020-06-0923:15seancorfield(somewhat ironically, my incorrect version -- (and string? #(>= 5 (count %))) -- worked by accident because it evaluated to just #(>= 5 (count %)) because (and truthy x) => x for any x#2020-06-0923:16seancorfieldI only realized my version was broken when I tried the 42 example and mine blew up, even tho' it had correctly rejected your test example with "012344444"#2020-06-0923:29dev.4openIDI am learning this language ad enjoying it. this is after the last serious coding I did 30 years ago. It is a steep curve, it feels like doing "maths" all over again - strict discipline and be aware of the subtilties . There are so many variations and tweaks I sometimes get lost! Too easy too make mistakes. Worth learning as it is pretty powerful. Thx for your help#2020-06-0923:40seancorfield@dev.4openid The REPL is your friend here. If you try out every single function as you write it, you might have caught the problem with your len function sooner:
user=> (defn len [n] (and (< (count n) 6)
                   string?))
#'user/len
user=> (len "123")
#object[clojure.core$string_QMARK___5410 0x3bc735b3 "
#2020-06-0923:41seancorfieldand:
user=> (len 42)
Execution error (UnsupportedOperationException) at user/len (REPL:1).
count not supported on this type: Long
user=>
#2020-06-1014:49adamI am trying to validate a server-side form with spec (so Clojure not ClojureScript) and I am a little lost about where to start to formulate human readable error messages. Is https://github.com/alexanderkiel/phrase the de-facto library for that?#2020-06-1014:49adamI am trying to validate a server-side form with spec (so Clojure not ClojureScript) and I am a little lost about where to start to formulate human readable error messages. Is https://github.com/alexanderkiel/phrase the de-facto library for that?#2020-06-1114:10aviIā€™ve always used expound for that#2020-06-1109:17jacklombardDo you all unit test specs?#2020-06-1109:17jacklombardDo you all unit test specs?#2020-06-1113:10Jivago AlvesNever unit tested specs. But I'm using it as part of the test. I'd test it if I was going to release it as a library.#2020-06-1113:32jacklombardfor complex specs that i use to validate data, would like to know if the spec is right#2020-06-1114:11aviThatā€™sā€¦ interesting! Because I think I already think of specs as, more or less, akin to tests themselves.#2020-06-1114:12aviI do of course sometimes find that a spec Iā€™ve written turns out to not quite capture what I had in mind, or a requirement, etcā€¦ but Iā€™ve done the same with ā€œtraditional unit testsā€ (i.e. example tests) ā€¦#2020-06-1116:08Jivago AlvesI use spec to help me to parse some data but I'm testing the function that does that. The fact I'm using a spec is just a implementation detail. At least, that was most of my use cases.#2020-06-1116:46aviAh, I see, yeah. That makes sense.#2020-06-1116:46aviIn that case, yes, I would probably write some example tests.#2020-06-1118:28seancorfieldMy approach to developing complex Specs is to have a (comment ,,,) form under the Specs with expressions in it that I use to test validation, conformance, and generation.#2020-06-1118:28seancorfieldI also make sure to "test" each individual part of a Spec (so I can catch generation problems early -- debugging generation problems in a large Spec is no fun!).#2020-06-1115:53mishacan I express "this key is optional, but if is in map - that key has to be there too, " as s/keys out of the box? with some combination of :opt :req and or like (s/keys :req [(or :a/foo :a/bar)]) or something? Or should I s/and it?:
(s/and
      (s/keys :opt [:a/foo :a/bar])
      (fn [m]
        (let [present (partial contains? m)
              missing (complement present)
              ks [:a/foo :a/bar]]
          (or 
            (every? present ks)
            (every? missing ks)))))
#2020-06-1115:55Alex Miller (Clojure team)s/and'ing a predicate is prob best#2020-06-1115:56mishais it s/and in spec2 for the same use case?#2020-06-1115:57mishathank you#2020-06-1115:59Alex Miller (Clojure team)yeah, I would do the same in spec 2#2020-06-1115:59Alex Miller (Clojure team)or rather I would probably make 2 different s/selects for the same s/schema#2020-06-1116:00Alex Miller (Clojure team)s/keys is probably going away in spec 2#2020-06-1117:58dev.4openID@alexmiller Over what time-frame are you anticipating spec 2 to come out?#2020-06-1117:58dev.4openID@alexmiller Over what time-frame are you anticipating spec 2 to come out?#2020-06-1118:38Alex Miller (Clojure team)Iā€™m deep in something else right now, but hoping to pop the stack back to that soon. Not sure, prob mostly gated in rework of function specs which rich has been hammocking on#2020-06-1117:59dev.4openIDI am assuming it will be spec.alpha2 right?#2020-06-1117:59dev.4openIDI am assuming it will be spec.alpha2 right?#2020-06-1118:39Alex Miller (Clojure team)Iā€™m hoping it will just be clojure.spec#2020-06-1118:25seancorfield@dev.4openid I'm not Alex but my understanding is that Spec 2 will eventually become just clojure.spec when it is ready to come out of alpha -- and there's no timeframe for it yet, based on what I've seen/heard, since a lot of things are still being designed/revised.#2020-06-1118:25seancorfieldThe repo is here https://github.com/clojure/spec-alpha2#2020-06-1118:28seancorfieldFor a while, I kept a branch of our codebase current against that repo but it involved quite a few changes (and it had to keep changing as Spec 2 changed). I stopped tracking it back in ... September/October I think? Our approach going forward -- once Spec comes out of alpha -- will be to use the new Spec for new code we write and slowly, over time, migrate our Spec 1 code to "Spec 2" as needed. Spec 1 will stay available as-is "forever" so folks can stay on the old version if they don't want to migrate.#2020-06-1118:35dev.4openIDThx#2020-06-1121:06plinshow can I create a generator from an arbitrary function? hereā€™s a silly example but It would be something like (s/def ::name (s/spec string? :gen (constantly "Margaret"))) not sure if possible but id like to combine it with a lib like https://github.com/paraseba/faker to generate test data#2020-06-1121:29Alex Miller (Clojure team)s/with-gen#2020-06-1121:29Alex Miller (Clojure team)s/with-gen#2020-06-1203:14practicalli-johnHow can I specify the number of tests run when using clojure.spec.test.alpha/check ? By default its running 1000 tests for a check against 1 spec (the playing cards example from https://clojure.org/guides/spec#_a_game_of_cards ) and takes around 80 seconds to complete. The docs mention :num-tests within clojure.spec.test.check/opts but either I have the syntax wrong or missing something
;; runs 1000 tests
(spec-test/check `deal-cards
                 {:num-tests 1})

;; java.lang.RuntimeException
;; Invalid token: ::clojure.spec-test-check/opts
(spec-test/check `deal-cards
                 {::clojure.spec-test-check/opts {:num-tests 1}})
Apart from Clojure 1.10.1, the project includes the dependency :extra-deps {org.clojure/test.check {:mvn/version "1.0.0"}} Requiring clojure.spec.test.check generates an error when the namespace is evaluated
java.io.FileNotFoundException
   Could not locate clojure/spec/test/check__init.class,
   clojure/spec/test/check.clj or clojure/spec/test/check.cljc on classpath.
Project code is at https://github.com/practicalli/spec-generative-testing if it helps...
#2020-06-1204:06seancorfield@jr0cket There's no such namespace: https://github.com/practicalli/spec-generative-testing/blob/prime/src/practicalli/spec_generative_testing.clj#L6#2020-06-1204:07seancorfieldThe option should be a qualified keyword -- but that doesn't mean a namespace exists.#2020-06-1204:09seancorfield:clojure.spec.test.check/opts#2020-06-1204:11seancorfieldIf you want to use ::stc you can introduce an alias:
user=> (alias 'stc (create-ns 'clojure.spec.test.check))
nil
user=> ::stc/opts
:clojure.spec.test.check/opts
user=>
#2020-06-1204:12seancorfieldYour ::clojure.spec-test-check/opts is going to fail because :: will try to auto-resolve clojure.spec-test-check which is not an alias.#2020-06-1204:12seancorfieldYour ::clojure.spec-test-check/opts is going to fail because :: will try to auto-resolve clojure.spec-test-check which is not an alias.#2020-06-1205:08David PhamSpec2 seems so cool. Increased programmability is such a good advantage. I have been using custom macros to generate spec, and it was a bit odd.#2020-06-1207:18AronSorry if this question has an easy answer discoverable that I missed, didn't do a deep dive. I see that https://github.com/clojure/spec-alpha2/wiki/Schema-and-select#select is available in alpha2 but not in alpha. Obviously lot of people use spec already, but which version?#2020-06-1207:22mpenet"spec1" (spec.alpha)#2020-06-1207:23mpenetit's not really clear what will/wont change in spec2 and there are also a few bugs#2020-06-1207:31AronThanks! is there something similar to spec/select already existing, perhaps written by someone else, published under a different name?#2020-06-1215:42seancorfield@ashnur Spec 2 will eventually become the official clojure.spec. It's just not ready for use yet as it is still being actively designed and changed. Spec 1 will remain available for everyone already using it. I don't think anyone has tried to copy Spec 2 -- because it is not yet complete (and why would anyone try to recreate an official part of Clojure when Rich himself hasn't fully figured out parts of the design?).#2020-06-1216:03AronI am just curious if I want to use select or something that does similar stuff to select, what are my best options currently.#2020-06-1216:11seancorfield@ashnur If you're just building toy stuff or learning/experimenting, you could use Spec 2. It's just not ready for production use.#2020-06-1216:13seancorfieldWe were tracking it at work, with a branch of our (95k lines) codebase, and I really like the changes in Spec 2 -- above and beyond the schema/`select` stuff -- but there's been a lot of churn in Spec 2 and Alex has said that Rich will likely overhaul s/fdef completely before it is released (and it may drop s/keys completely as well), so we stopped tracking it months ago.#2020-06-1216:13seancorfieldWe're just going to wait for it to be "fully baked" at this point.#2020-06-1216:14seancorfieldOnce Alex signifies that it is stable and just needs testing to help iron out the bugs, we'll pick it up again.#2020-06-1216:14Alex Miller (Clojure team)I will signify that by making a release :)#2020-06-1216:22Aronso, if I understand that correctly, there is nothing else that targets the same problem domain to be used in production in the interim?#2020-06-1216:26seancorfieldclojure.spec.alpha targets that domain and can be used in production. Spec 2 is the "next generation" of that and will be the preferred solution when it becomes ready.#2020-06-1216:27seancorfieldSpec 2 is definitely "better" than Spec 1 -- because it's designed to incorporate lessons learned from the first version. But Spec 1 definitely has value in production.#2020-06-1216:36AronI like what s/select does and would like to use something like it in production, even if it's not necessarily exactly the kind of s/select that is planned by Rich, since as I understand it, there are still months, maybe years until that version will be ready.#2020-06-1216:47Alex Miller (Clojure team)god I hope it's not years :)#2020-06-1217:27mishahow can not be expressed in spec? to maximize built-in generators reuse and composability (really just ability to wrap any spec in "it" without looking at spec/form I am wrapping)#2020-06-1217:28misha(before you look at me funny, I am writing a translator from json-schema to clojure-spec, particularly https://json-schema.org/understanding-json-schema/reference/combining.html#not)#2020-06-1217:33mishaanything better than this?
(do
  (s/def ::foo string?)
  (s/def ::bar (s/with-gen
                 (complement (partial s/valid? ::foo)) 
                 #(s/gen any?)))
  (s/exercise ::bar))
#2020-06-1217:33Alex Miller (Clojure team)not really#2020-06-1217:33Alex Miller (Clojure team)not is weird and I would generally avoid doing it :)#2020-06-1217:35mishaat this time, I am trying to generate spec as close to schema as possible, as code you than paste into file, and then might chose to change#2020-06-1217:37mishaAlex, is there an (out the box) way to conform unqualified map and get qualified conformed map back?#2020-06-1217:40misha
(do
  (s/def :my/foo string?)
  (s/def ::map (s/keys :req-un [:my/foo]))
  (s/magic-conform ::map {:foo "x"})  #_=> {:my/foo "x"})
#2020-06-1218:21seancorfieldI'm not Alex @misha but I can't think of any easy way to do that. You'd probably have to derive the (qualified) keys from the s/form of the Spec, and then zipmap with a version of those keys that had been mapped to unqualified keys, and then use clojure.set/rename-keys on your validated data.#2020-06-1218:22seancorfield(but that won't work with nested data structures/specs or anything more complex than just s/keys)#2020-06-1218:25Alex Miller (Clojure team)You could unform#2020-06-1218:25Alex Miller (Clojure team)I guess you still wouldnā€™t get unqual#2020-06-1218:26Alex Miller (Clojure team)So Iā€™ll go with no :)#2020-06-1218:57mishaI thought about just including both :req and :req-un sets of keys, but it does not solve "I have a map from example page, show me the specs it uses", and screws up the generators, which you probably want to generate either entirely qualified or entirely unqualified deep tree.#2020-06-1219:03seancorfieldIf we were starting again from scratch with Spec available, and using next.jdbc instead of clojure.java.jdbc, I think we would only have unqualified keys at the boundary of our system: either as API input or user input (forms, URLs), and at outgoing boundaries for JSON-based systems. So our use of :req-un/`:opt-un` would be a lot smaller, and we'd explicitly transform validated input into a domain model that always used qualified keys. Interacting with JDBC via next.jdbc means you can use qualified keys going out to the DB and you would get qualified keys coming in from the DB as well, automatically.#2020-06-1219:04Joshua SuskaloI'm playing around with custom generators, since there's a type which it seems like spec is having a hard time generating for some tests.
(s/def ::value pos-int?)
(s/def ::name keyword?)
(s/def ::symbol (s/keys :req [::value ::name]))
(s/def ::symbols (s/coll-of ::symbol :kind set?))
(s/def ::rows pos-int?)
(s/def ::columns (s/and pos-int?
                        #(>= % 3)))
(def machine-gen
  (gen/let [machine (gen/fmap
                     (fn [[cols rows]]
                       {::rows rows ::columns cols})
                     (gen/tuple (gen/fmap (partial + 3) gen/nat)
                                (gen/fmap inc gen/nat)))
            symbols (gen/vector-distinct (s/gen ::symbol)
                                         {:min-elements (inc (::rows machine))})]
    (assoc machine ::symbols symbols)))
(s/def ::machine (s/with-gen
                   (s/and (s/keys :req [::symbols ::rows ::columns])
                          #(> (count (::symbols %)) (::rows %)))
                   (constantly machine-gen)))
The problem is that whenever I try to sample the machine-gen, it works fine, but if I try to sample the result of (s/gen ::machine) it always says that a such-that isn't met after 100 tries. What would be the cause of this?
#2020-06-1219:04Joshua SuskaloI'm playing around with custom generators, since there's a type which it seems like spec is having a hard time generating for some tests.
(s/def ::value pos-int?)
(s/def ::name keyword?)
(s/def ::symbol (s/keys :req [::value ::name]))
(s/def ::symbols (s/coll-of ::symbol :kind set?))
(s/def ::rows pos-int?)
(s/def ::columns (s/and pos-int?
                        #(>= % 3)))
(def machine-gen
  (gen/let [machine (gen/fmap
                     (fn [[cols rows]]
                       {::rows rows ::columns cols})
                     (gen/tuple (gen/fmap (partial + 3) gen/nat)
                                (gen/fmap inc gen/nat)))
            symbols (gen/vector-distinct (s/gen ::symbol)
                                         {:min-elements (inc (::rows machine))})]
    (assoc machine ::symbols symbols)))
(s/def ::machine (s/with-gen
                   (s/and (s/keys :req [::symbols ::rows ::columns])
                          #(> (count (::symbols %)) (::rows %)))
                   (constantly machine-gen)))
The problem is that whenever I try to sample the machine-gen, it works fine, but if I try to sample the result of (s/gen ::machine) it always says that a such-that isn't met after 100 tries. What would be the cause of this?
#2020-06-1219:22mishaSean, my initial motivation is exploration, specifically of https://vega.github.io/ So I want to generate spec from schema (which is 9999km long), then take an example json, and with magic-qualify-conform see, which spec is that, and then navigate through keywords and specs in my IDE, instead trying to find things in huge json schema: https://vega.github.io/schema/vega-lite/v4.json or https://vega.github.io/schema/vega/v5.json This, and, the usual spec goods: exercise, etc.#2020-06-1219:23mishaso it seems I'd have to come up with "qualiform" too.#2020-06-1219:34seancorfieldYeah, I can definitely see the utility of this and it would be nice as an option in s/conform.#2020-06-1219:35seancorfieldI think it's interesting that Spec 2 takes a different approach, where you can specify unqualified keys inline in a hash map spec or else qualified keys in a schema, to be select'ed#2020-06-1220:26Joshua SuskaloSearch is being very unhelpful for this problem, seems like few people run into it. @seancorfield would you happen to know of anything I could do to debug this issue or to alter the generator so that it'll work?#2020-06-1220:30seancorfield@suskeyhose what is gen/ in your code above? I gather it's not clojure.spec.gen.alpha...?#2020-06-1220:32Joshua SuskaloIt's clojure.test.check.generators#2020-06-1220:35Joshua SuskaloMy understanding was that clojure.spec.gen.alpha was just a namespace that re-exposed some of the test.check vars.#2020-06-1220:37Joshua SuskaloWhich seems to be true looking at the source.#2020-06-1220:41seancorfieldHahaha... OK, it took me a while... What is ::symbols? What does it generate?#2020-06-1220:41seancorfieldAnd then in machine-gen, what type is symbols?#2020-06-1220:42Joshua Suskalosymbols is just a set of ::symbol, which are just maps with ::name and ::value#2020-06-1220:43seancorfield::symbols is a set. symbols in machine-gen is a vector.#2020-06-1220:43Joshua SuskaloOh boy, of course that's it#2020-06-1220:44seancorfieldThat was a nice Friday afternoon debugging diversion -- thank you! šŸ™‚#2020-06-1220:44Joshua SuskaloThanks for helping me out! I feel so dumb when I just get my types misaligned like that šŸ™ƒ#2020-06-1220:46seancorfieldNo worries. I couldn't see it either. And I was simplifying the ::machine spec trying to figure out what the problem was... I was quite bewildered by it!#2020-06-1422:35mishais there a sensible way to specify min/max items count for s/? s/* other than function predicate, like?:
(s/and ::my-cat-spec #(< 5 (count (s/unform ::my-cat-spec %))))
#2020-06-1422:38sgepigon@misha If youā€™re using regex specs, you probably want s/& instead of s/and#2020-06-1422:39sgepigonso (s/& ::my-cat-spec #(< 5 (count %)))#2020-06-1422:42mishayou are probably right about s/&, but I still will have to s/unform#2020-06-1422:44mishaactually, since I apply custom pred to the top spec, it does not seem to matter whether I use s/and or s/&#2020-06-1422:46mishathe reason I hope there is another way, is because I'd like to avoid unform#2020-06-1422:54sgepigonHmm I guess Iā€™m still unsure why you need s/unform. Perhaps thereā€™s something about ::my-cat-spec Iā€™m missing:
$ clj
Clojure 1.10.1
user=> (require '[clojure.spec.alpha :as s])
nil
user=> (s/def ::points (s/+ int?))
:user/points
user=> (s/conform (s/& ::points #(< 5 (count %))) [0 1 2 3 4 5])
[0 1 2 3 4 5]
#2020-06-1422:56misha
(s/def :foo/bar (s/cat :1 (s/+ (s/or :a int?))))
(s/explain 
    (s/& :foo/bar #(->> % (s/unform :foo/bar) count (= 3)))
    [1 2 3])

;Success!
=> nil
(s/explain 
    (s/& :foo/bar #(->> % count (= 3)))
    [1 2 3])

;{:1 [[:a 1] [:a 2] [:a 3]]} - failed: (->> % count (= 3))
=> nil
#2020-06-1423:00mishain your example conformed int it just int, so unforming is identity. but if you have some branchy spec (s/or, s/alt, s/cat) - it will change the shape of the data, so in custom predicates for s/and and s/& you either have to unform, or validate conformed value (but this requires you knowing exactly what kind of spec is as the first arg to s/and s/&)#2020-06-1423:02mishafor the homogeneous collections I can use s/coll and it has :max-count and :min-count options. I was hoping s/cat has something similar#2020-06-1423:03sgepigonI see. Thanks for providing an example.#2020-06-1423:04sgepigonI believe in spec2 this is ā€œnon-flowing s/andā€#2020-06-1423:04mishayeah, should have mentioned: need this for spec1 opieop#2020-06-1423:04sgepigonYou can modify the latter one to go into the map and count from there if you really want to avoid s/unform#2020-06-1423:04sgepigon
(s/explain 
 (s/& :foo/bar #(->> % :1 count (= 3)))
 [1 2 3])
#2020-06-1423:05sgepigonbecause you end up putting all the conformed values under the :1 key.#2020-06-1423:05mishawhat about
(s/cat :1 (s/+ (s/or :a int?))) :2 string?)
kappa
#2020-06-1423:07mishathe thing is: this is for translating json-schema to clojure spec, so all specs are dynamic, and I don't do this manually. So I need to find the most sane lowest common denominator#2020-06-1423:07mishathe thing is: this is for translating json-schema to clojure spec, so all specs are dynamic, and I don't do this manually. So I need to find the most sane lowest common denominator#2020-06-1506:55ikitommiinteresting. any code to share?#2020-06-1508:29Aronsecond that. I will have to do something similar soonish, and I am not even sure where I will begin : )#2020-06-1509:34mishawithin this week, I hope#2020-06-1423:18sgepigon
(defn conformed-count
  [conformed]
  (->> conformed
       vals
       (reduce (fn [acc x]
                 (if (coll? x)
                   (+ acc (count x))
                   (inc acc)))
               0)))

(s/conform
 (s/& :foo/bar #(= 4 (conformed-count %)))
           [1 2 3 "blah"])
#2020-06-1423:18sgepigon
(defn conformed-count
  [conformed]
  (->> conformed
       vals
       (reduce (fn [acc x]
                 (if (coll? x)
                   (+ acc (count x))
                   (inc acc)))
               0)))

(s/conform
 (s/& :foo/bar #(= 4 (conformed-count %)))
           [1 2 3 "blah"])
#2020-06-1423:22mishabut what if it is just int? instead of s/cat ? troll#2020-06-1423:22sgepigonlol yeah#2020-06-1423:24sgepigonFWIW hereā€™s the non-flowing s/and I mentioned: https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha#nonflowing-sand--new#2020-06-1423:33mishathanks!#2020-06-1423:19sgepigonYeah I donā€™t think thereā€™s a great solution for this now in spec1.#2020-06-1423:20sgepigons/unform might be your best bet.#2020-06-1423:22mishaso far, it seems very much like it, yes#2020-06-1707:08tianshuI'm trying to use s/coll-of ... :kind map? and s/or
(s/def ::k-str (s/tuple keyword? string?))
(s/def ::k-num (s/tuple keyword? number?))
(s/def ::kv (s/or
             :k-str ::k-str
             :k-num ::k-num))

(s/conform (s/coll-of ::kv :kind map?) {:a 1 :b "2"})
;; => {:a [:a 1], :b [:b "2"]}
I have no idea what the conform result should be, but this one looks not right
#2020-06-1707:28vlaaad@doglooksgood what about map-of?
sandbox=> (s/conform (s/map-of keyword? (s/or ::string string? ::number number?)) {:a 1 :b "2"})
{:a [:sandbox/number 1], :b [:sandbox/string "2"]}
#2020-06-1708:23tianshu@vlaaad Thanks for the advice! But I want care about both key and value. You remind me, since there's a map-of , so coll-of is only designed to work with sequences?#2020-06-1708:29vlaaad@doglooksgood figured it out after reading coll-of docstring:
(s/conform (s/coll-of (s/or
                        :kw->num (s/tuple keyword? number?)
                        :kw->str (s/tuple keyword? string?))
             :kind map?
             :into [])
  {:a 1 :b "2" :c 1})
=> [[:kw->num [:a 1]] [:kw->str [:b "2"]] [:kw->num [:c 1]]]
#2020-06-1708:30vlaaadby default it conforms to coll of the same type as input coll. :into overrides this#2020-06-1708:41tianshu@vlaaad thanks! don't know there's a into option.#2020-06-1708:41tianshuthis is what I'm looking for#2020-06-1712:14borkdudeRe: god I hope it's not years šŸ™‚ (https://clojurians.slack.com/archives/C1B1BB2Q3/p1591980460308000) Reminded me of: > We're hoping to have something releasable by Conj or the end of this year Source: https://youtu.be/KeZNRypKVa4?t=1201 Approaching one year away from that talk soon šŸ˜‰#2020-06-1712:59Alex Miller (Clojure team)"soon" is still many months :)#2020-06-2008:16Vincent Cantinwhat are the most difficult parts? design or implementation?#2020-06-1713:00Alex Miller (Clojure team)I didn't say which Conj#2020-06-1713:08borkdudelol šŸ™‚#2020-06-1713:48AronA school administrator in a speech said once (and I heard it with my own ears), "in 2000, by 9 o'clock, we will finish the new school building!"#2020-06-1803:36jumar@scott.silver asked this in #cider: https://clojurians.slack.com/archives/C0617A8PQ/p1592435925498900 I'm also curious what's your approach with stest/instrument and evaluating a clojure buffer since it must be a problem with other editors too. In short, after you evaluate a ns buffer all the instrumentation is gone and you need to call instrument again manually which is cumbersome and easy to forget. Developers then commit spec errors which are only found much later, etc.#2020-06-1803:56seancorfieldIf I'm using instrument with dev/test, I have a call to instrument as part of each test namespace. See https://github.com/seancorfield/next-jdbc/blob/develop/test/next/jdbc_test.clj#L21 (and similar lines in every test ns in next.jdbc).#2020-06-1803:58seancorfieldYou could make it part of you fixtures if you wanted to ensure it was added for every test you ran via the REPL from your editor but I think this is enough to ensure a full test run uses specs.#2020-06-1804:29jumarInteresting, but I'm more worried about real "src" namespaces, rather than tests... do you have instrument calls there as well?#2020-06-1804:37seancorfieldNo. Why would you?#2020-06-1804:37seancorfieldYou want instrumentation in place for testing, not production.#2020-06-1817:57Scott SilverI agree that you donā€™t want it in production, but itā€™s very useful in dev, not just in test, yeah?#2020-06-1818:03seancorfieldI run tests during dev so... šŸ™‚#2020-06-1818:05seancorfield(also, I only spec API boundaries in general, or functions that have some unusual aspects -- we have over 600 specs in our code but only 30-some function specs)#2020-06-1818:06Scott Silverdo you run tests manually while developing? or do you use something like lein-test-refresh to run the test suite when you make changes?#2020-06-1818:09seancorfieldI have hot keys bound to run: a) all tests in the current namespace b) all tests in a test namespace that corresponds to the current namespace c) the test under the cursor#2020-06-1818:10seancorfieldSo it is manual but very easy. I don't use lein at all and I don't like watcher-based tasks or any sort of reload/refresh workflow.#2020-06-1805:38jumarWe enable instrumentation for all fdef-s in dev mode (when running in REPL) to catch issues early - it's not enabled in production#2020-06-1813:11practicalli-johnI am assuming the following ways to define a spec are equivalent ? The seem to provide the same results with confirm/valid? etc.
(def suit? #{:clubs :diamonds :hearts :spades})
(spec/def ::suit #{:clubs :diamonds :hearts :spades})
Is it overkill to define a spec for a set as it already acts as predicate function? Or does it make little or no difference?
#2020-06-1813:29Alex Miller (Clojure team)either is fine. s/form probably works better with sets#2020-06-1913:50David PhamI have a tricky bug: it says that I have an undefined variable in cljs.alpha.spec at line 479 at the gen* method. However, I never referece cljs.spec.alpha (I use clojure.spec.alpha).#2020-06-1913:50David PhamAnyone encountered something similar?#2020-06-1913:52David PhamThis behavior only manifest itself when I am suing advanced compilation#2020-06-1913:59David PhamOkay, solve my problem by putting my specs namespace into the main module.#2020-06-2001:29practicalli-johnAny suggestions on how to create a custom generator for this specification for an email address (or an alternative specification for an email address that can be used - I could just use string until I get chance to think about it properly...)
(def email-regex #"^[a-zA-Z0-9._%+-]
When I use clojure.spec.gen.alpha/generate on the returned generator from (clojure.spec.alpha/gen ::email-type) I get an exception (my other specs will all generate data, just not this. So I am assuming I need a custom generator (it will be my first) The error I get from generate is
Unhandled clojure.lang.ExceptionInfo
   Couldn't satisfy such-that predicate after 100 tries.
   {:pred #function[clojure.spec.alpha/gensub/fn--1876],
    :gen {:gen #function[clojure.test.check.generators/such-that/fn--6372]},
    :max-tries 100}
#2020-06-2513:23djtango@jr0cket sorry to be that guy John but https://stackoverflow.com/questions/201323/how-to-validate-an-email-address-using-a-regular-expression#2020-06-2513:23djtangothough if you're just doing this for fun, then w/e#2020-06-2513:44practicalli-johnEverything I do is for fun, unless someone is paying me a 6 figure sum to do otherwise (although I did still have some fun working at the bank).#2020-06-2814:02djtangošŸ™‚#2020-06-2002:03gfrederickscheck the test.chuck (no typo) library#2020-06-2002:06seancorfieldYeah, a big +1 for test.chuck -- we use the regex generator a lot!#2020-06-2008:44borkdude@jr0cket Additionally, lambdaisland/regal also has generators for regexes#2020-06-2215:29buttergunsIs this the right place to ask about spec-tools ? If so: I'm trying out Swagger2 generation. I'd love to make use of https://swagger.io/specification/v2/#referenceObject to avoid duplicating the same definitions all over my json doc. Does spec-tools support this?#2020-06-2218:07ikitommi@mattmorten currently, no.. I think it has been in ring-swagger/plumatic schema for years, just not ported into spec-tools. PR welcome#2020-06-2218:08buttergunsThanks for the response @U055NJ5CC, I'll look into it#2020-06-2423:12misha#2020-06-2508:36ikitommiHow complete are you aiming this to be? Could it generate something else than spec syntax? There is work ongoing in making JSON Schema -> #malli and would like to see this for #schema too#2020-06-2508:38mishaI am doing this just to get familiar with vega enough to implement few charts for my projects troll#2020-06-2508:39mishaso for now I am going to finish spec1alpha, and then (but don't know when) - spec2#2020-06-2508:44mishaI need to finish few combinator use cases, such as oneOf , and see what can be done for string spec (all the regexes, etc.), then I'll package it up as a lib. The main purpose would be to generate spec forms, and it will be up to user what to do with them: paste into file, etc. Now, I am not aiming for it to be fast, just maintainable.#2020-06-2508:49mishaI am not very familiar with plumatic-schema, and only have seen few slides of mali, but I think it would be even easier to translate to those, given: that would be second/third implementation, and those support nested inline anonymous declarations.#2020-06-2516:16denikanyone have an idea why this returns invalid?
(s/conform (s/cat
             :kvs (s/and (s/* any?) (s/conformer identity)))
           [:foo :bar])
=> :clojure.spec.alpha/invalid
#2020-06-2521:00borkdude@denik
(s/conform (s/cat
             :kvs (s/& (s/* any?) (s/conformer identity)))
           [:foo :bar])
#2020-06-2708:40Eccentric JDoes anyone have or know of any examples of using spec with reagent views?#2020-07-0615:08Joe LaneHey @jayzawrotny, I'm a little late to the party, but I found this blog post to be very apropos https://juxt.pro/blog/cljs-apps#2020-07-0615:10Eccentric JThanks @U0CJ19XAM #2020-06-2709:18borkdude@jayzawrotny I do have one example from a commercial app I've worked on for the last few years:
(s/fdef dre.components.widget/widget
  :args
  (s/and
   (s/cat
    :opts
    (s/keys
     :req-un [::title ;; title in header
              ::content ;; Contents to show in widget. It must be a Hiccup
              ;; vector. Widget does not support varying arguments in
              ;; content, they are for initialization only.
              ]
     :opt-un [::id ;; unique identifier for widget (page wide, not app
              ;; wide)
              ::icon ;; icon to show in header
              ::widget-class ;; CSS class
              ::collapsable? ;; If it can be collapsed
              ::init-collapsed? ;; initialize the widget in a collapsed
              ;; state?
              ::help ;; help contents which are shown in modal
              ::controls ;; Arbitrary component rendered in the
              ;; header. What it does is up to you.
              ::selections ;; coll of dropdown/tabs
              ::dropdown ;; convenience option, singular version of
              ;; selections with
              ;; {:type :dropdown, :id :dropdown} by default
              ::tabs ;; convenience option, singular version of selections
              ;; with {:type :tabs, :id :tabs}
              ::loading? ;; Whether or not to display a loading indicator
              ::locked? ;; Whether or not content modifications are possible
              ::menu ;; menu
              ::on-will-unmount]))
   (fn [{:keys [opts]}]
     (if-let [unexpected
              (seq (apply dissoc opts expected-keys))]
       (do
         (warn "Unexpected options" unexpected)
         false)
       true))))
#2020-06-2709:20borkdude@jayzawrotny This is about the only Reagent component I've spec-ed because it has so many options#2020-06-2709:22borkdudeNot sure if it's useful, but this is the app: http://covid19.doctorevidence.com/ (this is a free preview of it)#2020-06-2718:29Eccentric JThanks @borkdude this really helps!#2020-06-2815:52sveriWould it make sense to incorporate alpha 2 into a new project? All the documentation I can find right now is the differences to alpha 1. Is it enough to read the docs for alpha 1 and the differences page?#2020-06-2815:53Alex Miller (Clojure team)No, you should not use it yet#2020-06-2815:53Alex Miller (Clojure team)Unless you enjoy bugs and breaking changes #2020-06-2815:55sveriOk, good to know, thank you @alexmiller#2020-06-2815:57Alex Miller (Clojure team)There are docs at https://clojure.github.io/spec-alpha2/ btw#2020-06-2816:00sveriHonestly, when looking at a library I always look at a high level doc first instead of api documentation. On the other hand, that api doc is very exhaustive šŸ™‚#2020-06-2816:00sveriSo thanks for the pointer#2020-07-0113:16mishaharold
:1
=> :1

(keyword "foo" "1")
=> :foo/1

:foo/1
Syntax error reading source at (REPL:1:1).
Invalid token: :foo/1
#2020-07-0113:17Alex Miller (Clojure team)there's a lot of tedious history here but keywords with numbers as the name part are technically not valid in the reader#2020-07-0113:18Alex Miller (Clojure team)due to a long-standing bug in the regex for keywords, things like :1 are actually read however and we've decided not to make them invalid#2020-07-0113:18mishabackward compatibility is tough#2020-07-0113:19Alex Miller (Clojure team)and just generally, there are a lot of keywords you can make programatically that are not print/read roundtrippable (https://clojure.org/guides/faq#unreadable_keywords)#2020-07-0113:21mishathis is I am aware of, yes. trying to choose generated keyword format, suitable for all of: item spec (:foo/i1) cat/or branch spec for that item (:i1), conformed value walking (idx) etc.#2020-07-0113:25misha@alexmiller is it possible to get quoted form of lambda verbatim as edn? e.g. #(+ % 2) so it could be printed out exactly like this #(+ % 2) ?#2020-07-0113:25misha
'#(+ % 2)
=> (fn* [p1__7217#] (+ p1__7217# 2))
#2020-07-0113:25Alex Miller (Clojure team)no?#2020-07-0113:25mishaok :(#2020-07-0113:26Alex Miller (Clojure team)if you're worried about the gensyms, you can transform to (fn [%] (+ % 2)) - spec does this for forms#2020-07-0113:28Alex Miller (Clojure team)(as an aside, there is a ticket and some work on better controlling gensym construction - this is often a tricky case when trying to symbolically compare macroexpanded code chunks)#2020-07-0113:29mishaI am generating lambda forms for generated specs, and often fn one is way too long, so I might as well just defn it:
(defn <=10? [x] (<= x 10))
(defn >=5? [x] (>= x 5))
(s/def :user/root (s/and number? >=5? <=10?))
#2020-07-0113:30mishawhich is, arguably, better, than
(s/def :user/root (s/and number? (fn [x] (>= x 5)) (fn [x] (<= x 10))))
#2020-07-0123:55Jan KYou could also do #(<= 5 % 10)#2020-07-0208:36mishanot gonna: ā€¢ complicates lib code ā€¢ less granular spec errors ā€¢ less similar to source json-schema, so when(if) you gonna debug things ā€“ you will have to recalculate another transformation in your mind #2020-07-0219:03vemvhttps://clojuredocs.org/clojure.spec.alpha/int-in#2020-07-0315:39misha> range from start (inclusive) to end (exclusive). - only for ints - (ex)Inclusiveness is baked in - requires both limits - specifically range spec was not the point, but readability of inline functions was#2020-07-0113:32mishabut then, there are things like ratios, decimals, etc opieop#2020-07-0320:27mishafwiw https://github.com/akovantsev/json-schema-to-clojure-spec ~80% core functionality complete#2020-07-0420:01adamHow do I refer a spec defined in another namespace? The usual :refer throws an exception#2020-07-0420:03Alex Miller (Clojure team)you don't need to refer it, just use the fq name#2020-07-0420:03Alex Miller (Clojure team)you do need to load the namespace defining the spec (w/ require)#2020-07-0420:09adamGot it, thanks. Ringā€™s wrap-reload middleware got me confused because it is picking it up even without referring the namespace IF I make some changes to my spec file... it suddenly becomes available everywhere.#2020-07-0420:10Alex Miller (Clojure team)specs are loaded into a global registry#2020-07-0606:01tianshuI want to use s/multi-spec to validate a map that has a few different structures. And I'm using s/conform to test if the map is satisfy the spec, but I can't find a way to figure out what branch it is matched since the conform on a multi-spec just simply return a map without more information. how can I know which branch it matched, like in s/or .#2020-07-0612:36Alex Miller (Clojure team)I assume you're not just checking a keyword in the map, it's something more complicated#2020-07-0612:38Alex Miller (Clojure team)I don't think you're going to automatically get this, but you could add a conformer to the different cases to "tag" the map with a branch identifier#2020-07-0614:34tianshu@alexmiller I end up call that dispatch function again to get the branch identifier.#2020-07-0807:21rivalHi everyone šŸ‘‹ to get my head around spec, I decided to just pick the FIX protocol and spec it. The result is my first Clojure lib https://github.com/rvalentini/fix.core. However the way I used spec felt a little bit "forced" here and there. At some point, I had the feeling that I was wrapping my validation logic into a thin spec wrapper definition for the sake of using spec, without much benefits. I couldn't find a clean/nice way how to build the validation logic as composition of individual specs. To illustrate what I mean: in https://github.com/rvalentini/fix.core/blob/master/src/fix/spec/primitives_spec.clj it felt for me like an "intended" usage of spec, where the individual spec definitions compose quite nicely. But here https://github.com/rvalentini/fix.core/blob/master/src/fix/spec/message_spec.clj it felt more like I misused spec, since the spec definition is just like a facade. Any advice on how I could improve this is much appreciated!#2020-07-0815:05misha@riccardovalentini1854 Spec's strengths are granular errors, parsing, generators. Since you basically use/provide neither of those, and just give "yes/no" answer to "valid?" ā€“ I think, such validation would be "lighter" with instaparse, or just giant regex. What can be alternatively useful, though, is a speced (with generators) edn DSL + an encode/decode functions to/from that edn data structure to FIX string messages. (pretty much what I'm doing for graphviz dot files, right now)#2020-07-0907:04rival@U051HUZLD Thanks a lot for your feedback! As a next step I planned to improve the error reporting for the client, so that in case the validation fails, the client receives some message indicating which part of the FIX message is invalid. I could make use of spec's granular error reporting there I think. I will also have a look at custom generators and think about how I could apply them to the internal edn representation of the parsed FIX messages. A nice encoder to produce FIX messages also sounds like a good idea! :+1:#2020-07-0815:13mishauser then can explore "WTH is FIX message, anyway?" with s/exercise, and assemble messages as familiar maps/vectors with core functions, validate, get granular errors, etc, and then convert them into message strings.#2020-07-1022:36walterlHi everyone. I'm still pretty new to spec, and can't figure out what's wrong with this simple spec with a generator fn:
(s/def :offset-date-time
  (s/with-gen (partial instance? OffsetDateTime)
    #(OffsetDateTime/now)))
(s/gen :offset-date-time) fails with
Execution error (AssertionError) at clojure.test.check.generators/such-that (generators.cljc:346).
Assert failed: Second arg to such-that must be a generator
(generator? gen)
Any ideas?
#2020-07-1022:37gfredericksThat's not a generator#2020-07-1022:37gfredericksGenerators are special objects from spec's generators namespace#2020-07-1022:38walterlAh!#2020-07-1022:38walterlThanks, @gfredericks#2020-07-1217:11cobyHey all, I want to check my intuition about something I'm trying to model. I need to design a small API for taking calendar availabilities (start/end pairs of date-times) and returning available appointment windows (also as start/end pairs, and also taking existing appointment times into account). Spec provides inst-in which is very nice, but I need to do a bunch of date comparisons, for which it's much nicer (IMHO) to use clojure.java-time. So would the recommended approach be to convert all datetimes to/from java-time at the edges of the API, and take/return insts?#2020-07-1219:24Alex Miller (Clojure team)inst is backed by a protocol so you donā€™t actually need to convert, just ensure the type youā€™re using extends Inst #2020-07-1512:27tikotusOnce in a while I'm toying with Clojure, currently I'm trying to learn how to benefit from specs. I just re-watched Hickey's "Maybe Not" talk to support some thoughts running through my head. I ended up wondering about a thing in the proposed schema solution: In the selection we give ::user [::first ::last] etc. Now what's the point of ::user? Why not just {::first ::last}? Just for communication? I'm not sure of the value it adds to communication. One thought I had was that it's there to tell us that we might be using other stuff from the ::user too, like ::addr, we just don't require it. But isn't this then just a briefer, less informative way of saying {:req [::first ::last] :opt [::addr]}? Now we're just one step away from the old spec :keys. Where did ::user go, what was the value of having a schema? Anyways, are there any news on the next take on spec that I'm missing? Couldn't find anything...#2020-07-1518:44jaihindhreddywith schema you can describe trees and not just one-level. For example, we can say, address is optional, but when you do give me an address, I need the zip code and the street. Before schema and select, this kind of a thing would result in an ad-hoc spec to be written, and we end up with a parochiality problem.#2020-07-1522:03tikotusI still have two questions: 1. What's the use of ::user at the root in this case? 2. Why is schema needed for a nested select? I don't need a schema for ::address to say that if ::address exists in the map, I require it to contain ::zip and ::street.#2020-07-1612:16jaihindhreddyLet's say we're dealing with a scenario where we're dealing with a seller and a buyer, and we need the address of both, then ::seller and ::buyer would be useful. With just ::user, it's not very useful, but if we don't have the ::user in there, that means it's assumed that the ::address is that of the user implicitly, making it context-sensitive. With ::user, we're making it clear that it is user information.#2020-07-1513:07Alex Miller (Clojure team)work has been ongoing on the next release at https://github.com/clojure/spec-alpha2#2020-07-1513:08Alex Miller (Clojure team)https://github.com/clojure/spec-alpha2/wiki/Schema-and-select is current state#2020-07-1513:08Alex Miller (Clojure team)and other differences at https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha#2020-07-1521:00mikebCool. What's best way to begin trying out spec2, or is that discouraged until it's further along?#2020-07-1521:05seancorfield@mikeb I'd say it is discouraged at this point. We were tracking it closely at work for several months last year but it's still very fluid and very buggy and there's clearly still some "hammock time" needed. After talking to Alex in September/October about it, we decided to wait until he says it is "baked" before we try it again.#2020-07-1521:06Alex Miller (Clojure team)instructions are in the readme if you want to try it as a git dep#2020-07-1521:06seancorfieldI think it's going to be a big improvement over Spec 1, based on our foray into it last year, but be prepared for it to change/break quite a bit as it continues to evolve šŸ™‚#2020-07-1616:12kennyAre there some cases where the generator override passed to st/check will not actually be the generator used? I seem to be hitting a case like this. I do see this note in the docstring for s/gen "Note that parent generator (in the spec or overrides map) will supersede those of any subtrees." I'm not sure if this is relevant here. It's also not totally clear what "parent generator" means in this case.#2020-07-1616:14Alex Miller (Clojure team)I believe there are some cases where this doesn't work. the note is talking about recursive cases I think#2020-07-1616:15Alex Miller (Clojure team)iirc one common issue in this area is with aliased specs#2020-07-1616:15kennyAliased specs?#2020-07-1616:16kennyi.e., (s/def ::parent ::child) ?#2020-07-1616:16ghadiyeah#2020-07-1616:17kennyYep, this is definitely an aliased spec#2020-07-1616:17kennyGuessing this fix will be to use the alias.#2020-07-1616:18kennyNot exactly a fit but I think it'll work in my case.#2020-07-1616:19kennyfwiw, my usage here is really a workaround to selecting which (deeply nested) keys are required for a test šŸ™‚#2020-07-1710:58Charlie Briggshello, weā€™ve recently started using specs and have a few questions regarding how to load them correctly say we have a namespace identity-provider and wish to have specs for identity-providers in a s_eparate file_, how would we go about doing that? Weā€™re looking at putting the actual specs into a separate namespace as otherwise weā€™re having cycling import issues if we put our specs in a separate file/namespace (say specs.core, and named them as :core.identity-provider/id, :core/identity-provider would we need to then require :refer :all in consumer namespaces to ensure theyā€™re loaded? Would it be more sensible to have a separate file for each entity specs, e.g. specs.identity-providers ? How would the ā€˜rootā€™ object for the identity-provider look in that case (rather than using the name :core/identity-provider )#2020-07-1715:41seancorfieldYou don't need to refer specs. Just requiring the namespace they're in will cause them all to be registered.#2020-07-1715:43seancorfieldIt's common to put data-describing specs in a separate namespace from code. As long as you require that spec namespace somewhere (anywhere, early in your execution path), those specs will then become globally available.#2020-07-2008:31Charlie Briggsthanks for your response, so would it be common practise to load all specs in the server entrypoint, or a user.clj dev-namespaced file for use in the repl, rather than loading them in the code which is using them?#2020-07-1715:43seancorfield@charliebriggs ^#2020-07-1715:52Daniel StephensI don't know if this is at all good practice, but we have a ns per 'entity' for our specs to keep things readable, specs in one namespace can refer to others outside of their namespace by using fully qualified keywords (as normal), and then we have one ns which just requires them all. Those are all package together as one dependency, then anything that wants to use them just imports the dependency and requires that one namespace.#2020-07-1915:38mishaBy ā€œreadableā€ you mean ::?#2020-07-2008:22Daniel StephensAssuming this is related to my response above I just meant readable as in 'more easily read (by humans)', I prefer reading a file with 10 specs in all relating to the same entity than trying to find what I want in a file of hundreds.#2020-07-2106:49tikotusIs utilising specs in linters technically possible, or is it too slow? Is there any tool working on this?#2020-07-2211:25denis_krivosheevHello everyone. I want to write some tests for my library but I really donā€™t want to write example based tests. I thought spec can be an instrument for property based testing, but couldnā€™t find a way to transform spec (or generator) into a test. Can someone please help me with that?#2020-07-2212:43Alex Miller (Clojure team)With spec, the main way you do it is by writing function specs and running ā€œcheckā€#2020-07-2212:45Alex Miller (Clojure team)Spec generators are also test.check generators so you can use test check property testing too#2020-07-2218:43denis_krivosheevOh this should work. Thank you very much #2020-07-2218:47Alex Miller (Clojure team)in fact, I find using (s/gen a-spec) to often be easier than using test.check to create generators directly#2020-07-2212:46Alex Miller (Clojure team)https://clojure.org/guides/spec covers using check #2020-07-2213:56borkdude@jahvenni https://github.com/arohner/spectrum is one attempt at this. I'm not sure if it's actively being worked on.#2020-07-2306:12tikotusThanks @borkdude, that's what I was looking for. But doesn't seem very active. Is it just me or wouldn't such a tool be simply amazing? It would make transitioning from statically typed languages feel a lot safer.#2020-07-2314:46seancorfield@jahvenni I think its very misleading to think of anything Spec-related as some sort of analogy to static typing...#2020-07-2314:53borkdudeYes. spec is a runtime validation/conformation/generation library. And considering how complex core.typed gets, I don't think getting leverage out of specs at compile time is trivial. Using a tool for something it's not designed for in general is ... hard.#2020-07-2314:55borkdudeI do think you can get <some> leverage out of it. E.g. a tool could inspect specs at runtime, parse out the trivial stuff like, is the first argument an int?? Then something like https://github.com/borkdude/clj-kondo/blob/master/doc/types.md can be generated and you will get <some> static analysis benefits for clj-kondo, for example.#2020-07-2314:57borkdudeMalli / @ikitommi has demoed a similar approach at ClojureD.#2020-07-2316:27tikotusYeah, figuring out the trivial stuff is what I had in mind mostly. I can see many trivial cases where a linter utilising specs could easily provide value. Things like order of parameters, names of keyword arguments or tricky get-in's are mentally really heavy for someone who is used to have these things given by autofill. To me these sound like trivial things to check in most cases if spec is provided. It would be a much softer landing to a dynamic language.#2020-07-2316:32borkdude@jahvenni The approached I outlined is open for anyone to implement or experiment with šŸ™‚#2020-07-2316:32borkdudeThis comment specifically: https://clojurians.slack.com/archives/C1B1BB2Q3/p1595516128028300#2020-07-3117:15johanatanHave you considered this: https://github.com/arohner/spectrum ?#2020-07-3117:28borkdudeThat was mentioned as part of that conversation#2020-07-3117:30johanatanOh, sorry. Didnā€™t scroll back far enough I guess. #2020-08-0612:49arohnerspectrum is no where near working, and Iā€™m not sure whether I will have the time to finish it#2020-07-2318:12tikotusYeah, thanks! Thatā€™s exactly the kind of info I was looking for :)#2020-07-2422:07seancorfieldI don't know how many folks here follow http://ask.clojure.org but it would be nice for someone to go and answer this https://ask.clojure.org/index.php/9486/how-to-consume-the-chan-return-by-go-block-in-cljs#2020-07-2423:47kenny
(s/valid?
    (s/keys)
    report-data)
=> false
(s/explain-data
    (s/keys)
    report-data)
=> nil
(s/valid? (s/keys) (apply hash-map report-data))
=> true
(type report-data)
=> clojure.lang.PersistentArrayMap
#2020-07-2423:47kennyAny idea what's going on there?#2020-07-2423:56seancorfield@kenny I can't repro with Spec 1. Is that Spec 1 or Spec 2 you're using?#2020-07-2423:56kennySpec 1#2020-07-2423:56seancorfield
user=> (require '[clojure.spec.alpha :as s])
nil
user=> (def report-data {:a 1 :b 2})
#'user/report-data
user=> (type report-data)
clojure.lang.PersistentArrayMap
user=> (s/valid? (s/keys) report-data)
true
user=> (s/explain-data
    (s/keys)
    report-data)
nil
user=> (s/valid? (s/keys) (apply hash-map report-data))
true
user=> 
#2020-07-2423:57seancorfieldWhat sort of keys are in your report-data map?#2020-07-2423:58kennyKeywords#2020-07-2423:59kennyDid you mean something else?#2020-07-2500:01kennyI can't create a repro locally either. It's something about report-data itself.#2020-07-2711:11Aronwhere should I look for docs/guide for how to test sideeffects on the frontend with specs?#2020-07-2715:03vemvdo you want to test that side-effects happen? or to avoid them, so that things are more cleanly testable?#2020-07-2715:05Arontest them#2020-07-2715:05Aronsadly, no one pays me for code that is pure šŸ™‚#2020-07-2715:08vemvwhat do you use for frontend? with re-frame you should have effect segregation for free#2020-07-2715:08Aronthe state is handled by react hooks locally in react components so it's not easy to test state transitions. There are specific situations that I want to avoid, so I was thinking I could run the app with generated data and test that specific situations don't occur. Like there is always a button to click or always a feedback message is shown#2020-07-2715:09Aronthe time I started this project re-frame didn't support hooks, which were required by my boss and colleagues#2020-07-2715:09Aronso I use helix#2020-07-2715:19vemvnot sure if I'd run Spec's generative testing (specifically, spec.test.alpha/check) with side-effectful functions. IME it's the kind of thing that is painful/slow to implement, and later to debug It might be cleaner to generate data with Spec, and pass said data to a vanilla integration test (whether a purely frontend-bound one, or a full-stack one aided by Webdriver)#2020-07-2715:21vemvFor completeness, with re-frame I think it's feasible to craft a useful generative test that asserts that interactions always yield a valid application state e.g. click X + click Y, in any order, generate events a, b, c all of which are valid and form a valid new app state All this side-effect free (thanks to the indirection bought by events)#2020-07-2715:46AronThe first suggestion with the integration test makes sense, it's what direction I was going, but I wasn't sure if there is no other way.#2020-07-2715:46AronThe second part with re-frame I don't understand, or maybe I understand just don't see how it is relevant?#2020-07-2720:05robertfwI do stateful generative testing but not on the frontend, so I can't help with specifics there. But I can confirm what @U45T93RA6 says - it is a bit painful and slow to implement. I've gone through a few iterations and am finally at a spot where I think we are taming the beast, but it's taken some work getting there and we have some work to go to get it running in a more reasonable amount of time.#2020-07-2720:10vemv> The second part with re-frame I don't understand, or maybe I understand just don't see how it is relevant? it's relevant because it attempts to show how one could write a generative, integration-ish side-effect-free test using a specific framework. Don't know helix myself so yeah it may have limited utility :)#2020-07-2720:25AronI still don't understand, are you saying I should use a different library? Or framework, although I do not use frameworks so I hesitate to use the word framework.#2020-07-2721:01vemvno, I'm stating a technique that may be translated to other (but not all) libraries/frameworks#2020-07-2721:02AronWouldn't that mean that I would have to start over and write the whole thing in a different style?#2020-07-2721:02Aronso are you saying this for consideration for future projects?#2020-07-2721:08vemvIDK, I don't know helix so I don't know how practical the described techique could be for you. In the end it's just food for thought :)#2020-07-3010:24arohnerAre there any libraries for parsing swagger/openAPI into specs? I see https://github.com/metosin/spec-tools, but that appears to only do generation, not parsing#2020-07-3010:36misha@arohner this might be useful to you: https://github.com/akovantsev/json-schema-to-clojure-spec#2020-07-3010:37arohnerthanks#2020-08-0602:10Oliver GeorgeI keep wanting ::x/blah to expand to :current.namespace$x/blah when there's no x alias. It'd open up "private namespaces" for spec use.#2020-08-0602:10Oliver GeorgeMy use case is CLJS function arg specs. I have many fns in a ns. They expect different things from a :db key passed in as context. (re-frame handlers)#2020-08-0602:13Oliver GeorgeWith the proposed change I could do things like this
(ns app.events (:require [clojure.spec.alpha :as s]))
(defn event1 [{:keys [db]}] ...)
(defn event2 [{:keys [db]}] ...)
(s/fdef event1 :args (s/cat :ctx (s/keys :req-un [::event1/db]))
(s/def ::event1/db (s/keys :req-un [::ddb])))
(s/fdef event2 :args (s/cat :ctx (s/keys :req-un [::event2/db]))
(s/def ::event2/db (s/keys :req-un [::form])))
#2020-08-0602:18Oliver GeorgePerhaps a macro would do the job so could add to replace keyword namespaces in a form
(rename-keyword-ns {*ns* :app.events$event1)
  (s/fdef event1 (s/cat :ctx (s/keys :req-un [::db]))
  (s/def ::db (s/keys :req-un [::ddb]))))
#2020-08-0602:45seancorfieldI suspect Spec 2 (a.k.a. what will eventually become just clojure.spec šŸ™‚ ) will help you here because you can define a schema will all the possible keys with their specs, all in one place -- and then you can select the specific keys that are required for each individual handler.#2020-08-0604:29Oliver GeorgeGood point. If/when that lands it'll be an improvement.#2020-08-0604:29Oliver GeorgeI guess it'd look something like this...
(ns app.events (:require [clojure.spec2 :as s]))
(defn event1 [{:keys [db]}] ...)
(defn event2 [{:keys [db]}] ...)
(s/def ::ctx (s/schema {:db ::db}))
(s/def ::db (s/schema {:ddb ::ddb :form ::form}))
(s/fdef event1 :args (s/cat :ctx (s/select ::ctx [:db {:db [:ddb]}]))
(s/fdef event2 :args (s/cat :ctx (s/select ::ctx [:db {:db [:form]}]))
#2020-08-0604:31Oliver George(Guessing at the unqualified key bits)#2020-08-0617:50johanatanhi, does clojure.spec.test.alpha/check run its test cases in sequence or in parallel ?#2020-08-0618:00Alex Miller (Clojure team)parallel via pmap#2020-08-0618:00johanatanany way to tell it to do sequential? because my code under test is already parallel and is kicking off a bunch of promises#2020-08-0618:01johanatanand the combination of the two is overwhelming the browser i'm afraid#2020-08-0618:10Alex Miller (Clojure team)not currently#2020-08-0618:14johanatanok, no worries. i can just limit the num-tests to a small amount for now#2020-08-0922:19johanatanhi, i have a spec'd function that is taking as input a collection of data where the individual elements have attached metadata via a with-gen for their spec. however, because stest/check conforms these inputs to their constituent components (specified via s/cat) the metadata which was attached to the outer element is lost. what is the recommended way around this? perhaps a way to override "conform" or otherwise propagate the metadata? something else?#2020-08-0922:21johanatani could manually attach the metadata rather than to the outer value to one of its constituents but this doesn't feel like the proper data modeling for the actual situation. i could also add an extra component to the s/alt but that isn't desirable as it would expose "test-only" data to the code under test (and also generally violate one's tastes with respect to the actual modeling).#2020-08-0922:25seancorfield@johanatan Are you saying that stest/check is passing conformed data into the function under test? That doesn't seem right since the conformed shape won't necessarily match the input shape...#2020-08-0922:25johanatanno, it's passing it into the :fn verifier#2020-08-0922:26johanatansorry wasn't clear. stest/check is passing the generated data to the actual function#2020-08-0922:26johanatanbut that data is slightly deformed via conform to the actual :fn checker#2020-08-0922:26seancorfieldOh, right. So you're trying to rely on input metadata in the :fn verifier?#2020-08-0922:26johanatan(per the labeling)#2020-08-0922:26johanatanyea#2020-08-0922:27seancorfieldI wouldn't expect metadata to be preserved through that. It doesn't seem like something you should be relying on in :fn.#2020-08-0922:27johanatanit would be very useful for testing purposes#2020-08-0922:27johanatanthe generators can then inform the tests what the expected behavior should be#2020-08-0922:27johanatanI can't imagine any better way to accomplish that in fact#2020-08-0922:28johanatanand as long as the generators don't use s/alt s/cat and the like, this is entirely possible#2020-08-0922:28johanatan(i've done it previously)#2020-08-0922:29johanatani have to step out now so will respond back later this evening. thx#2020-08-0922:31seancorfieldCould you use s/conformer in the spec to thread the metadata into the conformed result I wonder?#2020-08-1002:44johanatan@seancorfield i'm not sure how to force s/check to use my conformer but yes that would seem to work if it were possible to do so.#2020-08-1002:45johanatani wonder if there is a "conformer registry" i could write into ?#2020-08-1002:48seancorfieldNo, I mean directly in your Spec. So it always runs when a Spec is conformed.#2020-08-1002:54johanatansuppose i have a spec defined like so:
(s/def ::an-entity
  (s/with-gen
    ::a-base-entity
    #(gen/let [...] (with-meta a-base-entity-conforming-structure {:some :metadata}))))
are you suggesting to use s/and to weave in a conformer like so: (s/and (s/conformer ...) ::a-base-entity) ?? or something else?
#2020-08-1002:55johanatan@seancorfield ^#2020-08-1002:55johanatanthe base entity in this case looks like this:
(s/def ::re-frame-event
  (s/and vector?
         (s/cat :event-name qualified-keyword?
                :params (s/* any?))))
#2020-08-1002:56johanatanactually i can also just share the derived event:
(s/def ::callback-func-event
  (s/with-gen
    ::re-frame-event
    #(gen/let [params (s/gen (s/coll-of any?))
               triggers-failure? (gen/frequency [[1 (gen/return true)] [9 (gen/return false)]])
               event-name (s/gen qualified-keyword?)]
       (with-meta
         (concatv [event-name] params)
         {:triggers-failure? triggers-failure?}))))
#2020-08-1002:57seancorfieldPut s/conformer in the second/last slot of s/and#2020-08-1002:58seancorfieldThat allows you to modify the (conformed) value as part of the conform process without affecting the validation.#2020-08-1002:59johanatanah, ok#2020-08-1002:59seancorfield(s/and ... (s/conformer #(with-meta % {:what :ever}))) -- untested but that's what I had in mind.#2020-08-1003:01johanatanhmm, ok. let me try that#2020-08-1003:02johanatanthe problem with that is that the meta depends on a generated value#2020-08-1003:02johanatanand i need to somehow "attach" that value to the generated structure#2020-08-1003:02johanatanwhile allowing it to survive the first "conform" in the s/and#2020-08-1003:03johanatani.e., the following doesn't work because ::re-frame-event has already stripped away the meta and destructured into labeled data#2020-08-1003:03johanatan
(s/def ::callback-func-event
  (s/with-gen
    (s/and ::re-frame-event (s/conformer identity))
    #(gen/let [params (s/gen (s/coll-of any?))
               triggers-failure? (gen/frequency [[1 (gen/return true)] [9 (gen/return false)]])
               event-name (s/gen qualified-keyword?)]
       (with-meta
         (concatv [event-name] params)
         {:triggers-failure? triggers-failure?}))))
#2020-08-1003:04johanatani could "cheat" and stuff it into the last "param" and allow my conform to strip it out of there and put it in the meta#2020-08-1003:04johanatanbut that kind of violates a sense of purity šŸ™‚#2020-08-1003:05johanatanand would also only work for some specific subset of cases; i.e., we're lucky in the sense that params is an s/* of s/any which could accommodate the data#2020-08-1003:06johanatanperhaps i just make the single conformer the "spec" and forget about using s/and ?#2020-08-1003:07johanatanthen do the base conform within that conform#2020-08-1003:07seancorfieldTrue, you could have a completely custom "predicate" that conforms as a spec...#2020-08-1003:07johanatanyea, i think i'll go that route. probably the path of least resistance at this point#2020-08-1003:13johanatanthis works:
(defn metadata-preserving [spec]
  (s/conformer
   (fn [v]
     (let [m (meta v)
           res (s/conform spec v)]
       (if (= res ::s/invalid)
         res
         (with-meta res m))))))
#2020-08-1003:13johanatanused like so:
(s/def ::a-spec
  (with-gen
     (metadata-preserving ::a-base-spec)
     #(with-meta ( ... generation here ) {:some :meta})))
#2020-08-1003:15johanatanthanks!#2020-08-1004:10seancorfieldNice!#2020-08-1305:16ikitommia common question for spec-tools has been how to create custom specs that coerce correctly and emit valid JSON Schema (for web-stuff). I though of pasting a sample here too for adding support for ZonedDataTime#2020-08-1305:17ikitommi#2020-08-1305:23ikitommiwould be happy to throw away the custom wrappers for adding meta-data for specs in favour of core spec supporting that.#2020-08-1313:12respatializedcan I re-use a matched value within the body of spec/cat for further pattern matching later on by referring to its key? I'm trying something like:
(def lookup {:a 1 :b 2 :c 3})
(s/cat :key keyword?
       :contents (spec/* (spec/or :string string? :matched (get lookup :key))))
Is this possible?
#2020-08-1313:13Alex Miller (Clojure team)no
#2020-08-1313:14Alex Miller (Clojure team)you can s/& at the top level and check constraints with any arbitrary predicate inside the container#2020-08-1313:18respatializedok, thanks! I think the space of possible keys in the lookup map I'd be relying on is small enough to fully enumerate using individual specs, so I think I'll just do that instead (probably what I should have been doing from the start, but I wanted to see if I could save myself some typing šŸ˜‚)#2020-08-1317:35johanatanis it possible to define a spec for a function in a namespace that one doesn't control?#2020-08-1317:35Alex Miller (Clojure team)yes#2020-08-1317:36johanatanhow?#2020-08-1317:36Alex Miller (Clojure team)the same way?#2020-08-1317:36johanatanso like
(s/fdef the-ns/the-func ...)  ?
#2020-08-1317:36Alex Miller (Clojure team)yep#2020-08-1317:37johanatanah cool. thx!#2020-08-1317:37Alex Miller (Clojure team)fdef always takes fq symbols and those are just keys in the registry#2020-08-1318:15johanatan:+1:#2020-08-1322:19apbleonardAnyone know whether spec2 is informed by SHACL? https://youtu.be/apG5K3zc4V0#2020-08-1908:08rickmoynihanFYI there is also a #rdf channel on here, it has infrequent but good discussions on it when they occur#2020-08-1908:11apbleonardThanks. Good to know :)#2020-08-1322:30Alex Miller (Clojure team)while I find it's usually a bad bet to say that Rich is not aware of something, as far as I know he is not aware of that. certainly a lot of the design work precedes that talk#2020-08-1322:38Alex Miller (Clojure team)from a quick skim of the video, both Rich and are I well-versed in RDF and OWL and that inspired many things in the design of both Clojure and Datomic (this is covered in some more detail in the History of Clojure paper https://clojure.org/about/history). I'm not sure that SHACL has anything particularly novel in it and it also makes a closed world assumption that's actually intentionally absent from spec to a large degree#2020-08-1322:39Alex Miller (Clojure team)I actually started using Clojure at the semweb startup Revelytix about 10 years ago and wrote a commercial federated SPARQL engine there#2020-08-1322:40Alex Miller (Clojure team)all in Clojure, sadly, no code survives from the company dying :(#2020-08-1418:58apbleonardVery cool you started Clojure in semweb :) I should have asked whether SHACL informed spec2/select as IIUC its all about defining different shapes of data (including closed world for sure) that can be drawn from the same (open world) OWL ontology for different specific use cases - a bit like how select will (I think) fix a shape suitable for e.g. a given function, whereas specs focus on the unchanging characteristics of attribute values in all use cases? (Hope I haven't mangled spec2 there!) I don't think it's novel - I mean it's a web standard ... Thought the specification of a data shape using SHACL as a graph itself was cool.#2020-08-1419:01apbleonardSHACL actually stands for "Shapes Constraint Language" which I hadn't realised :)#2020-08-1811:57rickmoynihan@apbleonard: Iā€™ve noted this similarity before too, but I think the similarity comes more from the property oriented starting point inherent in RDF and spec. i.e. spec and SHACL are similar because they both come from the same starting point, that properties/keywords are first class and more important primitives than the entities on which they exist. @alexmiller As I understand it SHACL is also open by default; though you can close shapes with sh:closed true. Itā€™s true though that I think one of the big usecases for SHACL is in being able to close the world; in particular setting the min/max cardinality of a shape to 1, though I think the excitement around SHACL is more that it helps a long-standing area of frustration in RDF, which is that in RDF essentially every property/join can be many->many. Which has historically made it somewhat harder to validate and build user interfaces against it. In this regard datomic/clojure is I think arguably less open-world, but more pragmatic, in that it closes the world on some properties by allowing essentially cardinality of 1. Sure, OWL does let you sort of do this; by falling back on the non-unique-names assumption, but relying on reasoning everywhere can be impractical.#2020-08-1914:24bhaim123Hi, Would appreciate some help šŸ™‚ What am I doing wrong? I have a spec for an empty map: `
(s/def ::empty_map (s/and empty? map?))
and then I build: `
(s/def ::name string?)
(s/def ::someone1 string?)
(s/def ::someone2 string?)
(s/def ::someone_big (s/or
                       ::empty_map
                       (s/keys :req-un [::someone1 ::someone2])))
(s/def ::final (s/keys :req-un [::name ::someone_big]))
The idea here that I would be able to pass:
{:name "abc" :someone_big {}}
And it wold pass, since it is an empty map. But what I get is an error on the required params:
(s/explain ::final {:name "abc" :someone_big {}})
{} - failed: (contains? % :someone1) in: [:someone_big] at: [:someone_big :<NS>empty_map] spec: :<NS>/someone_big
{} - failed: (contains? % :someone2) in: [:someone_big] at: [:someone_big :<NS>/empty_map] spec: :<NS>/someone_big
Any help would be appreciated šŸ™‚
#2020-08-1914:48sgepigon@bhaim123 You need to label each branch of the s/or e.g.
(s/def ::someone_big (s/or :empty-map ::empty_map
                             :someone (s/keys :req-un [::someone1 ::someone2])))
user=> (s/conform ::final {:name "abc" :someone_big {}})
{:name "abc", :someone_big [:empty-map {}]}
#2020-08-1915:14bhaim123Thanks, but I donā€™t want to send the keyword empty map, the someone_big should be either an empty map. or a map of 2 keys#2020-08-1915:25sgepigonIā€™m not sure I follow. The spec does capture that itā€™s either an empty map or a map of two keys. Youā€™re not ā€œsendingā€ any keywords, youā€™re labelling alternatives. Both (s/conform ::final :empty-map) and (s/conform ::final :someone) are invalid for this spec.#2020-08-1915:26sgepigonThe original ::someone_big spec you posted was a s/or with 1 branch: it could only be a map with two keys and it was labelled ::empty_map. If you reversed the order it might be clearer why itā€™s wrong:
(s/def ::someone_big (s/or
                       (s/keys :req-un [::someone1 ::someone2])
                       ::empty_map))
Unexpected error (AssertionError) macroexpanding s/or at (REPL:1:22).
Assert failed: spec/or expects k1 p1 k2 p2..., where ks are keywords
(c/and (even? (count key-pred-forms)) (every? keyword? keys))
#2020-08-1915:27sgepigonNotice the path from the s/explain you posted:
at: [:someone_big :<NS>empty_map]
#2020-08-1918:33bhaim123Thank you very much @U56NS7GMQ#2020-08-1915:37Alex Miller (Clojure team)as an aside, (s/def ::empty_map (s/and map? empty?)) would be better than what you have as it should gen when the other would not#2020-08-2019:01johanatanHi, I'm running into a crash in some substrate of my testing environment. When calling stest/check on a particular function, if I increase the number of tests beyond a point or increase the size of the collections involved, i get the following (unfortunately not very illuminating output):
> clojure -A:test
2020-08-20 11:45:17.836:INFO::main: Logging initialized @5575ms to org.eclipse.jetty.util.log.StdErrLog
[Figwheel] Validating figwheel-main.edn
[Figwheel] figwheel-main.edn is valid \(惄)/
[Figwheel] Compiling build test to "target/public/cljs-out/test-main.js"
[Figwheel] Successfully compiled build test to "target/public/cljs-out/test-main.js" in 5.052 seconds.
Launching Javascript environment with script:  "./scripts/launch_headless.sh"
Environment output being logged to: target/public/cljs-out/test/js-environment.log
#error {:message "ClojureScript test run failed", :data {:type :end-run-tests, :fail 1, :error 0, :pass 18, :test 4}}
Error: ClojureScript test run failed
    at new cljs$core$ExceptionInfo ()
    at Function.cljs$core$IFn$_invoke$arity$3 ()
    at Function.cljs$core$IFn$_invoke$arity$2 ()
    at cljs$core$ex_info ()
    at Function.eval [as cljs$core$IFn$_invoke$arity$variadic] (eval at figwheel$repl$eval_javascript_STAR__STAR_ (), <anonymous>:64:25)
    at redacted$test_runner$_main (eval at figwheel$repl$eval_javascript_STAR__STAR_ (), <anonymous>:18:33)
    at eval (eval at figwheel$repl$eval_javascript_STAR__STAR_ (), <anonymous>:1:26)
    at figwheel$repl$eval_javascript_STAR__STAR_ ()
    at 
    at Object.G__12816__2 ()
Execution error (ExceptionInfo) at cljs.repl/evaluate-form (repl.cljc:578).
#error {:message "ClojureScript test run failed", :data {:type :end-run-tests, :fail 1, :error 0, :pass 18, :test 4}}
Full report at:
/var/folders/jz/920rhb8h8xj864001s7_grh80000gn/T/clojure-8321569605138574991.edn
Neither the full report nor the js environment log contains anything of interest. Just stack traces in figwheel main.
(s/def ::any-coll
  (s/with-gen
    (s/coll-of any?)
    #(s/gen (s/coll-of any? :max-count 5))))

(s/fdef concatv
        :args (s/cat :x ::any-coll :rest (s/* ::any-coll))
        :fn #(let [r (:ret %1)
                   x (-> %1 :args :x)
                   rest (-> %1 :args :rest)]
               (println (count r) (count x) (count rest))
               (= r (apply (partial concat x) rest)))
        :ret (s/coll-of any? :kind vector?))

(defn concatv
  "Strict version of concat."
  [x & rest]
  (case (count rest)
    0 x
    1 (into x (first rest))
    (into x (apply concatv rest))))
I can currently execute fine with num tests of 20 but when increasing to 100 or more, the crash occurs.
#2020-08-2019:02johanatanThe crash only seems to occur under headless Chrome; when using figwheel-extra-main/auto-testing in the actual browser, it doesn't occur.#2020-08-2019:04johanatanUnfortunately I don't get access to the "seed" when the problem occurs and my printlns do not get executed either so actually diagnosing is difficult.#2020-08-2019:22seancorfield@johanatan At a guess, since it seems both environment-specific and size-specific, I wonder if your (apply (partial concat x) rest) in the :fn spec is causing a stack overflow due to a buildup of lazy concat calls?#2020-08-2019:23johanatanoooh! that's possible. in fact that was the issue that spurred the creation of concatv to begin with#2020-08-2019:23johanatanwould (mapcat identity colls) be a better impl ?#2020-08-2019:24seancorfieldI think I would approach testing this via test.check and properties/generators, rather than trying to use fdef with :fn.#2020-08-2019:25seancorfieldAfter all, your :fn is pretty much re-implementing the function being tested, but using lazy concat instead.#2020-08-2019:25johanatanthis does use test.check and generators though. yes, but that's pretty much how all :fn s end up#2020-08-2019:26seancorfieldYou're misunderstanding my point I think.#2020-08-2019:26seancorfieldYou're basically testing a vector-based concat implementation against concat itself -- so your test is going to run into the same problems as concat.#2020-08-2019:27johanatanah, true. that's why i mentioned (mapcat identity ...) though. so, basically what we need are "two strict implementations of concat". one to test the other#2020-08-2019:27seancorfieldWhat you should be testing are properties of the result, e.g., count of the result is sum of count of each argument.#2020-08-2019:27seancorfieldSet of values of result is union of set of values in each argument.#2020-08-2019:28seancorfield(I'm currently working through Eric Normand's three courses on Property-Based testing so this is top of my mind right now)#2020-08-2019:29seancorfieldIn the beginner course, he has a specific example of testing some properties of concat.#2020-08-2019:29johanatanwell i agree that testing the actual sets is better. that supercedes the individual properties#2020-08-2019:30johanatani think there is a difference of philosophy here. if you test for actual equality then you don't need to test for the properties#2020-08-2019:30johanatanthis is what i typically do with my :fn tests#2020-08-2019:30seancorfieldThe key is to not re-implement the function under test though.#2020-08-2019:30johanatani see the other sort of property-based tests as strictly less powerful / more limited / less robust#2020-08-2019:30johanatanno, re implementation is fine#2020-08-2019:31seancorfieldClearly it isn't šŸ™‚ That's what is causing this problem.#2020-08-2019:31johanatanas long as the implementation is in a completely different fashion#2020-08-2019:31johanatanwell laziness is (likely) what's causing this problem#2020-08-2019:31seancorfieldAnd also, you may end up with bugs in your function under test because you accidentally replicated them in your :fn re-implementation of it.#2020-08-2019:31johanatanso if we have two completely separate strict implementations that both produce the same result, then we're good#2020-08-2019:32johanatansure, but i don't do that šŸ™‚ or i typically keep iterating until i eliminate those#2020-08-2019:32seancorfieldYou would probably change your position on this if you took Eric's course.#2020-08-2019:33johanatani find a lot of his material to be a bit more "entry level" than fits my needs. (just in general, having seen a few of his other videos. there are definitely philosophical differences here and his audience is, like you said, beginner-esque)#2020-08-2019:34johanatani've done property-based testing for a while now. it's not like my thoughts on this just formed recently.#2020-08-2019:35johanatani do agree that there are times where checking properties is fine: the canonical example being checking the correctness of an NP-complete problem's solution#2020-08-2019:35johanatanbut a lot of problems in practice do not have "easy checks for correctness"#2020-08-2019:36johanatanbut yea i'll give it a listen#2020-08-2019:37johanatanstill yet, and we're assuming that laziness is the problem here (which it very well could be), there would seem to be a "bug" in the way the failure is presented if you will. i.e., there is literally no trace of helpful information in this case in the actual output šŸ™‚#2020-08-2019:48johanatanand actually just realized mapcat won't help here since it is a thin wrapper around apply concat#2020-08-2019:50johanatanalso, btw, nothing about :fn says that you must do a full reimplementation. you can still do "property" checks in there. of course with the downside being perhaps less readable output upon a subexpression failure. i'm willing to make that tradeoff though for the streamlined / inline specification#2020-08-2019:54johanatani think what i'll do in this case since a second "strict" version of concat seems elusive is a simple pairwise traversal comparing elements along the way#2020-08-2020:20seancorfield> nothing aboutĀ :fnĀ says that you must do a full reimplementation. you can still do "property" checks in there I would say property checks are better than a re-implementation.#2020-08-2020:22seancorfield> i find a lot of his material to be a bit more "entry level" than fits my needs I've been doing Clojure in production for a decade and I'm still picking up new ideas from even his "entry level" courses -- but I agree that his style is aimed at beginners in many courses. He has three PBT courses: beginner, intermediate, and advanced -- so I figured since I have a subscription, I might as well watch all of them (I run them at 1.5x speed).#2020-08-2020:22johanatancool, sounds good. i'll give it a look. thanks for your help on this!#2020-08-2020:27seancorfieldIt sounds like fdef is going to get substantially reworked in Spec 2, based on what Alex has been saying about that. But it's still all in "design mode" right now.#2020-08-2021:01johanatanhm, in what way? btw, which properties would you test here if you were going the "property route" ?#2020-08-2021:01johanataneverything i'm trying to do is hitting the same problem / crash#2020-08-2021:01johanatanlike sum of counts, sum of hashes etc#2020-08-2021:02johanatane.g., this crashes:
(= (count r) (apply (partial + (count x)) (mapv count rest)))
#2020-08-2021:05johanatanperhaps the s/* is causing this to blow up really large. i've read somewhere that it can happen and that it's hard to tweak as "recursion depth" is not all that precise of a control on it#2020-08-2021:05seancorfield> (`fdef`) hm, in what way? Alex hasn't said... just that it's going to be substantially reworked.#2020-08-2021:15Alex Miller (Clojure team)Rich is working on it, there have been many ideas explored, not sure where it's going to end up#2020-08-2103:37johanatanbtw i think i found the problem
(apply concatv rest)
can blow up the stack / is not in tail position.
#2020-08-2103:40seancorfieldOh, interesting... Does that introduce laziness somehow?#2020-08-2103:40johanatan(at least that is "one" problem)#2020-08-2103:40johanatanwell, it's a recursive call#2020-08-2103:41seancorfieldOH! Yeah, that sounds so obvious once you say it out loud! šŸ™‚#2020-08-2103:42johanatanlet me explain. sorry, i noticed that even if I remove the :fn I still get the crash with the original impl of concatv. but with this loop/recur one (inspired by the Stuart Sierra blog on concat) that crash won't occur:
(defn concatv
  "Strict version of concat."
  [head & tail]
  (loop [seqs (cons head tail) acc []]
    (if (empty? seqs)
      acc
      (recur (rest seqs) (into acc (first seqs))))))
#2020-08-2103:43seancorfieldSeems weird to have head & tail when you only cons them together and never use them again.#2020-08-2103:44johanatanoh, could just be [& args] ?#2020-08-2103:44johanatangood point#2020-08-2103:44seancorfieldYou might want to consider whether (concatv) should work and what it should produce (I'd say yes and [] respectively).#2020-08-2103:45johanatani'm just trying to match behavior of the original concat (which seems to agree w/ you) šŸ™‚
cljs.user=> (concat)
()
#2020-08-2103:45johanatanand yea that concatv does similarly:
cljs.user=> (concatv)
[]
#2020-08-2103:46seancorfieldI just tried it in Clojure and got an error#2020-08-2103:48seancorfieldOh, it works after changing to & args and (loop [seqs args ...#2020-08-2103:48johanatanhmm, mine was from lumo#2020-08-2103:48johanatanyea, sorry#2020-08-2103:48johanatandidn't paste in here#2020-08-2103:48johanatan
(defn concatv
  "Strict version of concat."
  [& args]
  (loop [seqs args acc []]
    (if (empty? seqs)
      acc
      (recur (rest seqs) (into acc (first seqs))))))
#2020-08-2116:45petterikDid you consider (reduce into [] args)?#2020-08-2120:51johanatanNope, does it work?#2020-08-2120:51johanatanAlso, how much memory will it use? the loop/recur one is tail optimized so should use a constant amount of memory#2020-08-2103:48seancorfieldThe version with [head & tail] requires 1+ arguments šŸ™‚#2020-08-2103:49johanatanhaha, yea 2+ or 0+ makes more sense than 1+#2020-08-2104:01johanatanhere is a "final" version that works (tested up to 75 iterations at a time):
(s/def ::any-coll
  (s/with-gen
    (s/coll-of any?)
    #(s/gen (s/coll-of any? :max-count 125))))

(s/fdef concatv
  :args (s/cat :args (s/* ::any-coll))
  :fn #(let [r (:ret %1)
             args (-> %1 :args :args)
             sumhash (fn [c] (apply + (mapv hash c)))]
         (and
          (= (count r) (apply + (mapv count args)))
          (= (sumhash r) (apply + (mapv sumhash args)))))
  :ret (s/coll-of any? :kind vector?))

(defn concatv
  "Strict version of concat."
  [& args]
  (loop [seqs args acc []]
    (if (empty? seqs)
      acc
      (recur (rest seqs) (into acc (first seqs))))))
#2020-08-2104:06johanatanso yea i think i was kinda confounded earlier by the fact that there was a potential stack overflow in the code under test. so even when i got the test code "right" the code under test could screw me#2020-08-2104:07johanatanbut it was somewhat non-deterministic on both sides. bit of an entangled heisenbug#2020-08-2104:07johanatanšŸ™‚#2020-08-2104:08johanatantwo entangled heisenbugs rather#2020-08-2104:08seancorfieldComplected, even šŸ™‚#2020-08-2104:08johanatanhaha exactly#2020-08-2119:04johanatansmall update: so ... there turned out to be 3 problems. my hunch about s/* was also correct. some of the crashes were emanating from it as well. here's a "more final" version (at least until I get another random, intermittent failure some time in the future):
(defn arbitrarily-partition [coll freq]
  (let [signals (take (count coll) (cycle (cons true (repeat freq false))))
        shuffled (shuffle signals)
        zipped (map vector shuffled coll)
        partitioned (partition-by first zipped)]
    (for [c partitioned]
      (map second c))))

(s/def ::coll-of-colls
  (s/coll-of (s/coll-of any?)))

(s/def ::distinct-coll-of-colls
  (s/with-gen
    ::coll-of-colls
    #(gen/let [elems (gen/set (s/gen any?) {:max-elements 150})
               freq (s/gen (s/int-in 1 10))]
       (arbitrarily-partition elems freq))))

(s/fdef concatenate
        :args (s/cat :args ::distinct-coll-of-colls)
        :fn #(let [r (:ret %1)
                   args (-> %1 :args :args)
                   sumhash (fn [c] (apply + (mapv hash c)))]
               (and
                (= (count r) (apply + (mapv count args)))
                (= (sumhash r) (apply + (mapv sumhash args)))))
        :ret (s/coll-of any? :kind vector?))

(defn- concatenate
  "Strict version of concat."
  [args]
  (loop [seqs args acc []]
    (if (empty? seqs)
      acc
      (recur (rest seqs) (into acc (first seqs))))))

(defn concatv
  "Strict version of concat."
  [& args]
  (concatenate args))
#2020-08-2308:23mishaI did not read entire discussion, so it might be irrelevant, but: there is a :distinct option in s/coll-of
(s/coll-of int? :distinct true) 
#2020-08-2308:25mishaor is it supposed to be "distinct for all 1st and 2nd level items"?#2020-08-2419:40Lennart BuitSo spec (1) has a list of predicates it can create generators for, so (s/gen int?) works. I kinda forgot, is it possible to extend this on the predicate level, e.g. can I register a predicate so that this works (s/gen my-predicate?)?#2020-08-2419:46seancorfield@lennart.buit You'd have to write your own generator (and then supply it when you define the spec based on your predicate).#2020-08-2419:47Lennart BuitYah right, so I can only define a spec with s/def and attach a generator that way#2020-08-2419:47Lennart BuitI canā€™t extend like this list: https://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/gen/alpha.clj#L146#2020-08-2419:47seancorfieldhttps://clojure.org/guides/spec#_custom_generators#2020-08-2419:53Lennart BuitYeah right, reason Iā€™m asking, we have quite a lot of specs that use a (non-spec aware) predicate, e.g. (s/def ::my-spec my-predicate?). So I was kinda hoping I wouldnā€™t have to do like:
(s/def ::my-predicate-spec (s/with-gen my-predicate? (fn [] ...))
(s/def ::my-spec ::my-predicate-spec)
But what Iā€™m finding in the source, I appear to be out of luck
#2020-08-2419:55Alex Miller (Clojure team)yeah, this is not currently an open set#2020-08-2419:57Alex Miller (Clojure team)I think the issue is that if you could extend it, then predicates may gen in one place but not in another if you didn't extend it the same way (somewhat reminiscent of reader macros).#2020-08-2420:01Lennart BuitRight ā€” because you have this global idea of what generators exists for what predicates and that could clash, or not be loaded or ā€¦ Yeah I can see that as causing pains#2020-08-2505:11stuartrexkingIs it possible to add a req-un to an existing s/def to create a new s/def. I have a request map with a number of keys, and a response map with all the keys in the request, plus an additional id key. Is there a nice way to do this?#2020-08-2505:13seancorfield@stuartrexking you can use s/merge to merge two s/keys specs together.#2020-08-2505:14seancorfield(s/merge ::my-spec (s/keys :req-un [::some-new-key]))#2020-08-2505:15stuartrexking@seancorfield Thank you.#2020-08-2510:54jacklombardIs there a way to get the results of s/valid? and s/expalin-str in one call?#2020-08-2510:55sogaiuyou don't mean like by using juxt or something right?#2020-08-2510:56jacklombardNope, want spec itself to return it#2020-08-2510:56jacklombardso that I can destructure say the validity and the error in the same call#2020-08-2511:01jacklombardI am using spec for api validation where I check whether the body or the params are valid, if not I return 400 with error message as s/explain-str against the same spec. Of course the error message is not humanized, but we can deal with it. Donā€™t want spec to run its validation twice since payloads can be huge#2020-08-2512:10Alex Miller (Clojure team)You could run s/explain-data, check that, then run the function to produce the string from the data#2020-08-2608:43AronWhat's the proper way to generate random emails for testing?#2020-08-2608:46dharriganIf you're using spec you could use a generator#2020-08-2608:46dharrigan shows one such approach#2020-08-2608:47dharrigan another#2020-08-2608:51AronYes, I am trying to learn spec to use it : ) Would be a stretch that I am using it now : )#2020-08-2608:53AronI see, so these are valid emails but they only generate a relatively small subset of possible email formats#2020-08-2613:49Joe Lane@ashnur Check out https://github.com/miner/strgen , ignoring that it still uses clojure.spec in it's examples, I think everything else should work.#2020-08-2613:50AronI was just about to try test.chuck#2020-08-2614:48plexusor (shameless plug): Regal https://lambdaisland.github.io/regal-playground/#2020-08-2717:53AronI tried out regal, and I like it, but I am not entirely sure I understand what is happening.I tried out regal, and I like it, but I am not entirely sure I understand what is happening.#2020-08-2615:14Aronthat link is broken šŸ˜ž#2020-08-2615:15AronI am having an extremely frustrating experience while trying out test chuck. Even though the namespace is included, the example code doesn't work, methods like string-from-regexp are nil. I am not even sure in this situation where should I go to ask for help, probably #beginners ?#2020-08-2615:30Alex Miller (Clojure team)here's probably fine - can you share a snippet of what you're doing?#2020-08-2615:31Alex Miller (Clojure team)https://lambdaisland.github.io/land-of-regal/ looks to be the right link#2020-08-2615:37Alex Miller (Clojure team)
% clj -Sdeps '{:deps {com.gfredericks/test.chuck {:mvn/version "0.2.10"}}}'
Clojure 1.10.1
user=> (require '[com.gfredericks.test.chuck.generators :as gen'])
nil
user=> (require '[clojure.test.check.generators :as gen])
nil
user=> (gen/sample (gen'/string-from-regex #"([ā˜ƒ-ā™„]{3}|B(A|OO)M)*"))
("" "" "BAMBOOM" "ā™›ā™œā˜–ā˜ā™¢ā˜“ā™ā™”ā™" "ā˜¹ā™ā™ˆā˜ˆā˜ ā™‡BOOM" "BAMBAMā™„ā™žā˜¬ā™—ā˜’ā˜„" "" "BOOM" "ā™¤ā˜ŗā˜µā˜—ā˜ƒā˜»BAMā˜Øā™„ā™—ā™•ā™„ā˜ˆBOOMBOOMā™Šā™…ā™" "ā˜Ŗā˜“ā˜‰ā˜ ā˜“ā˜”BOOM")
#2020-08-2615:47Aronso this only works with java I am guessing#2020-08-2615:48Aronhttps://gist.github.com/ashnur/3f5b7fd20d80a3831fe5a1a4c45bb55a#2020-08-2615:53AronProbably because java regex is not the same as js regex. But then I am probably better off with one of the other options that were mentioned.#2020-08-2615:56Alex Miller (Clojure team)it should work with both#2020-08-2615:57Alex Miller (Clojure team)did you (require '[clojure.test.check.generators :as gen]) ?#2020-08-2615:57Alex Miller (Clojure team)test.chuck builds on top of test.check, so you need to include both of the generator namespaces#2020-08-2615:58Aronhmm#2020-08-2615:58AronI did, but this gives me an idea#2020-08-2616:00Alex Miller (Clojure team)it seems from those errors that you have issues with both the gen and gen' namespaces#2020-08-2616:12hadilsI need help with multi-spec:
(def a #{1 2 3})
(def b #{4 5 6})
(s/def ::type #{:foo :bar})
;;; This is pseudocode
(if (= ::type :foo)
   ; I want ::new-sym to be defined as a
  else (= ::type :bar)
    ; I want ::new-sym to be defined as b
; ::type and ::new-sym appear in the same map
I am pretty sure this is done with multi-spec but I am stumped as to how to make it work...
#2020-08-2616:23gfrederickstest.chuck's regex generator is jvm-only#2020-08-2616:36Alex Miller (Clojure team)ah, sorry#2020-08-2616:38Alex Miller (Clojure team)@hadilsabbagh18 s/multi-spec is for the case where you can look at the data and return different specs based on the data#2020-08-2616:38Alex Miller (Clojure team)what is your actual data?#2020-08-2616:40Alex Miller (Clojure team)in general, spec is designed primarily for the case where attributes always have the same spec so you are probably going against that grain a bit.#2020-08-2616:41Alex Miller (Clojure team)if the attribute is in a map, you could create (s/and (s/keys :req-un [::new-sym]) arbitrary-condition)) to tighten a more general spec#2020-08-2616:41Alex Miller (Clojure team)and then use an s/multi-spec to choose which of those applies#2020-08-2616:50hadilsThank you @alexmiller! Here is the actual snippet:
(s/def :budget-item/main-category #{"Household" "Personal" "Savings" "Investments" "Other"})

(defn budget-categories
  [main-category]
  (into #{}
    (mapv first
      (d/q '[:find ?cat :in $ ?main :where
             [?e :expense/budget-category ?main]
             [?e :expense/categories ?cat]] (db) main-category))))

(defmulti ordered-categories :budget-item/main-category)
(defmethod ordered-categories "Household" [_]
  (budget-categories "Household"))
(defmethod ordered-categories "Personal" [_]
  (budget-categories "Personal"))
(defmethod ordered-categories "Savings" [_]
  (budget-categories "Savings"))
(defmethod ordered-categories "Investments" [_]
  (budget-categories "Investments"))
(defmethod ordered-categories "Other" [_]
  (budget-categories "Other"))
(s/def :budget-item/ordered-categories (s/multi-spec ordered-categories ,,,))
I want to make :budget-item/ordered-categories dependent on what :budget-item/main-category is set to. They will be in a map together...
#2020-08-2617:12Alex Miller (Clojure team)yeah, I don't think you should do this in a spec#2020-08-2617:13Alex Miller (Clojure team)or just make it a valid-category? predicate#2020-08-2617:15Alex Miller (Clojure team)just spec ordered-categories as a string? or keyword? or whatever, then s/and the whole map predicate with a custom predicate that validates that extracts ordered-categories and main-category, and validates that one is valid according to the other#2020-08-2617:15hadilsOk, I'll do that. Thanks a lot for your help @alexmiller! I am really grateful.#2020-08-2617:16Alex Miller (Clojure team)dependent stuff is hard to do well - it's easiest to just specify that constraint at the level that contains all of the data#2020-08-2710:43AronSo, about this email generation thing. I know I am asking for a lot šŸ™‚. But this is clojurescript, so my expectations are high because of the hype. However, the task to put together a generator for this http://emailregex.com/ in either of the suggested DSLs seem at least as daunting as just doing it from scratch, using built-ins.#2020-08-2710:46AronBasically, I am blocked because it's such a huge task, I don't even want to start, I can write all the necessary tests by not generating emails at all. Some of the hand written email address generators that I have seen suggested or used in libraries are positively naive compared to what is out in the wild. : )#2020-08-2711:17Lennart BuitThe pragmatic approach to validating emails: Check they contain an @ , and send an email to confirm šŸ˜‰.#2020-08-2717:55AronNow, what is the pragmatic approach to generate emails in clojurescript? šŸ™‚#2020-08-2717:56Aron
(def em-chr [:not "<" ">" "(" ")" "[" "]" "\\" "." "," ";" ":" "@" :whitespace :newline "\""])
(def em-id-wrd [:cat [:+ em-chr] [:* em-chr]])
(def em-str [:cat '\" :any '\"])
(def em-id [:alt em-id-wrd em-str])
(def em-domain [:cat [:+ [:class [\a \z] [\A \Z] :digit "-" "." ]] [:repeat [:class [\a \z] [\A \Z]] 2 63]])
(def email [:cat em-id "@" em-domain])
(prn (gen/generate (regal-gen/gen email)))
#2020-08-2717:57Aronthis is regal, but I am not sure if it will work#2020-08-2718:15vlaaadWhat is the problem you are solving @ashnur?#2020-08-2718:22AronWhich one? šŸ™‚#2020-08-2718:22AronI am writing automated integration tests.#2020-08-2718:59vlaaadfor system that validates emails? šŸ™‚#2020-08-2719:10AronI am not sure why validation is coming up. I didn't speak about validation at all.#2020-08-2719:12vlaaadSorry, too snarky... You need to decide what properties of strings representing emails matter for your tests involving emails. Nothing really matters, it's just stored? Use generator #" , who cares. Test needs to successfully send an email, but still has to generate different emails? Use my.email+<random-stuff-here>@gmail.com .#2020-08-2719:18vlaaadbtw your domain generator implies 2-nd level domains like {:tag :mailto:mefoo.commefoo.com, :attrs nil, :content nil} , but email hosts might be IP addresses (e.g. {:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "016c64413033322f3033322f3033322f31"}, :content ("[emailĀ protected]")}). If your program runs in a corporate network, sysadmins might configure it to have internal resources as 1st-level domain (e.g. just {:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "d9b4bc99bab6aba9"}, :content ("[emailĀ protected]")}).#2020-08-2723:56pandais there a really easy way to make human name generators with spec?#2020-08-2801:53Alex Miller (Clojure team)A set is a valid spec that gens. So override the generator to gen from a known set of names#2020-08-2800:02pandaalso is there a way to get the generators for different items in a nested map to be consistent? i.e. for instance if i had a map like:
(s/def ::begin-date inst?)
(s/def ::end-date inst?)

(s/def ::dates (s/keys :req-un [::begin-date ::end-date]))
how can i get begin-date to be less than end-date in the generated output? sorry if this is documented somewhere šŸ™‚
#2020-08-2800:02panda^ appreciate the help in advance šŸ™ šŸ˜„#2020-08-2800:29seancorfield@lpanda2014 You can wrap the s/keys part with s/and and add a predicate to check that #(some-date-lib/after? (::begin-date %) (::end-date %))#2020-08-2800:45pandathanks! for some reason that didnā€™t work though šŸ˜ž had to change it to ints since i dont have a datetime library in my repl but the below code gave me a null ptr when i went to generate. is my syntax off?
(s/def ::begin-date int?)
  (s/def ::end-date int?)
  (s/def ::dates (s/and
                  (s/keys :req-un [::begin-date ::end-date])
                  #(> (::end-date %) (::begin-date %))))

(gen/generate (s/gen ::dates)) ;; broken
#2020-08-2800:50pandaactually nvm figured it out - the second predicate canā€™t be :: šŸ™‚ thanks for your help!#2020-08-2801:06seancorfieldhttps://github.com/stackoverflow/date-clj is a simple library for manipulating dates, that has before/after comparisons. We use that at work.#2020-08-2800:30seancorfield(most of the date utility libraries have a simple function to check if one date is after another -- don't know what or if you're using)#2020-08-2800:31kennyor built in to Date: #(.after (::end-date %) (::begin-date %))#2020-08-2800:35kennyNeeds a custom gen too. Something like (gen/fmap #(zipmap [:begin-date :end-date] (sort %)) (s/gen (s/tuple inst? inst?)))#2020-08-2800:41pandathanks! this worked like a charm
(s/def ::dates (s/with-gen
                  (s/keys :req-un [::begin-date ::end-date])
                   #(gen/fmap (fn [d] (zipmap [:begin-date :end-date] (sort d))) (s/gen (s/tuple inst? inst?)))))
#2020-08-2800:42kennyYou'll still (probably) want the predicate added with s/and#2020-08-2800:46pandahm i couldnā€™t get the s/and part to work actually#2020-08-2800:46kenny(s/and ::dates #(.after (::end-date %) (::begin-date %)))#2020-08-2800:50pandathanks !#2020-08-2801:08seancorfieldAh, good point. I couldn't remember whether java.util.Date had comparison built-in (and was too lazy to check the Java API docs šŸ™‚ )#2020-08-2919:06Lennart BuitHow would you go about specā€™ing a (http) patch like api using nil as a sentinel for retraction. Letā€™s say you have an entity that has an optional attribute, Iā€™d hate to say that this attribute is s/nilable in its global definition just because there is a patch endpoint using nil to signal retraction#2020-08-2919:06Lennart BuitHow would you go about specā€™ing a (http) patch like api using nil as a sentinel for retraction. Letā€™s say you have an entity that has an optional attribute, Iā€™d hate to say that this attribute is s/nilable in its global definition just because there is a patch endpoint using nil to signal retraction#2020-08-2919:08Lennart BuitLike ā€” if I send you this attribute, it will either not be there, or it will have a non-nil value.#2020-08-2919:19Lennart BuitI donā€™t know, it feels asymmetrical: Iā€™ll always promise you either no value at all, or some value satisfying a spec, but you can provide me no value, a value satisfying a spec or nil, but only in this specific case. I guess it doesnā€™t help me that I can have only one definition for a namespaced keyword (here).#2020-08-2919:56seancorfieldIsn't this just a non-required key in a spec?#2020-08-2919:56seancorfieldIt's either present (and conforms) or it is not present.#2020-08-2920:19Lennart BuitYes, outgoing I would agree: Iā€™ll (= server) never send you a nil for a key I have no value for and instead omit it. The problem is in how you (= client) tell me that you donā€™t want this value anymore: If you omit it in your (partial) update request, do you mean to retract it, or to retain it? So some rest endpoints allow you to patch an entity, but with nil as value, meaning ā€˜letā€™s get rid of this value for that attributeā€™. So now we have a bit of asymmetry: Iā€™ll promise you to never send a nil value, but you can send me a nil to indicate retraction on this patch endpoint.#2020-08-2920:20Lennart BuitI donā€™t know how to express, in spec, that I as a server make a stronger guarantee than you as a client have to when patch ā€™ing entities. If that makes any sense šŸ™‚.#2020-08-2920:34Lennart Buit(Or more specifically, I donā€™t know how to do that when my keys are namespace qualified. If I were to use s/keys with :req-un I could maintain two specs.)#2020-08-2920:41seancorfieldThat's just two different specs (perhaps with reuse on the common stuff). A spec for the result of a call (where the key is optional but spec'd to be non-nil). A spec for the input value (where the key is nilable).#2020-08-2920:43seancorfieldIf it's an API spec, it's going to be for unqualified keys, surely? Since it will be a wire spec, e.g., JSON.#2020-08-2920:49Lennart BuitRight, we may have extended this nilability (or the spec, for that matter) too far into our system. We have a JSON handler that accepts this nil and is spec checked, that is unqualified, but then we have an update method shared between this JSON endpoint and other places that is also checked, but has qualified keys. Thats where friction occurs: the keys are qualified, but their semantics differ between what you supply and what you get.#2020-08-2920:55seancorfieldWell, if you have an internal (qualified) name for that attribute, either it should be optional and non-`nil`, or it should be nilable -- in all cases. And if that's not possible, then the two semantics should have different names.#2020-08-2921:04Lennart BuitSorry for the kinda fuzzy descriptionā€¦ Right so in the latter of your options you say to extend the notion of valid values for this attribute to be a superset of all values it can take in all contexts. That means that consumers I could have promised no nils (because they consume the ā€˜outgoingā€™ format), have to consider nils because thats what I could promises them in my spec, right? I find that kinda sad.#2020-08-2921:07seancorfieldI'm saying use different specs as needed and transform the data to match as you cross boundaries.#2020-08-2921:13seancorfieldAnother option is to chose a specific, unique, representation for a retraction (and, again, map from the inbound retraction to that representation).#2020-08-2921:15seancorfield(we had exactly this situation and we tried to blur the lines with nilable and optionality and it was a mess so we mapped it to a different representation altogether)#2020-08-2921:15Lennart BuitRight that makes sense#2020-08-2921:19Lennart BuitYeah, I think we are going wrong in a similar way: have a single namespace qualified keyword for an attribute in all its contexts, but using it in different ways. Iā€™ll put this in the hammock, thank you for the input, as always šŸ™‚!#2020-08-3010:17borkdudeI'm running a poll about including clojure.spec inĀ babashkaĀ here: https://twitter.com/borkdude/status/1300428421252747266#2020-08-3014:36borkdudeWill a next Clojure include spec2 and drop spec1?#2020-08-3014:36Alex Miller (Clojure team)tbd#2020-08-3019:56Aronno one knows anything, it's so exciting! šŸ˜„#2020-08-3101:16hadilshttps://clojurians.slack.com/archives/C03S1KBA2/p1598836570469900#2020-08-3102:26rapskalianThe fact that youā€™re destructuring the key doesnā€™t change the fact that budget-item-type simply accepts a map. Assuming you have a spec for :transaction/amount, your args spec would be something like (s/cat (s/keys :req-un [:transaction/amount]))#2020-08-3102:27seancorfield@U6GFE9HS7 That's the answer they got in the other channel.#2020-08-3102:27seancorfieldPlease don't encourage people to cross-post questions here.#2020-08-3102:27rapskalianOops, didnā€™t see that. Noted @seancorfield #2020-08-3102:23seancorfield@hadilsabbagh18 It's a good idea to not cross-post questions here.#2020-08-3102:23seancorfieldYou've already gotten a response in another channel on this.#2020-08-3102:26hadils@seancorfield Sorry my bad#2020-08-3113:09Petrus TheronHow to constrain the size of two dependent coll-of specs, A & B so that (<= (count (concat A B)) 40)? AFAICT, I need to use g/bind to constrain the :max-count of B's sample based on the length of A, but I'm having some trouble getting this to work in Spec2. @alexmiller you mentioned you are considering basic logic resolution. It would be awesome if it was possible to do something like:
(let [a (s/coll-of digit? :max-count 30)
      b (s/coll-of digit? :max-count (- 40 (s/count a))]
  (g/fmap (fn [a b] (apply str a ":" b) [(s/gen a) (s/gen b)]))
#2020-09-0204:30johanatanI always use gen/let for this sort of thing. You can use it just like a normal let except with generators on the rhs and generated values bound to the lhs. if you need a regular value you can use a gen/return to effectively raise them to that level such that both regular values and generated values can be bound in the same gen/let.#2020-09-0204:31johanatanin this particular case, you could generate an int with int-in 1 40 and then compute a second int-in 40 - the first. then use the two ints as exact size parameters to your collection generators.#2020-09-0204:16johanatandoes anyone see why the following wouldn't work? I'm getting an error: sym not defined on the stest/check line.
(defn filtered-check [sym opts]
  (let [filter (or (.get (js/URLSearchParams. js/window.location.search) "filter") "")
        matches? (fn [s] (or (= filter "all")
                             (clojure.string/includes? (name s) filter)))]
    (when (matches? sym)
      (stest/check sym opts))))
#2020-09-0204:25johanatanthis is the exact error:
Unable to resolve symbol: sym in this context

  114  (defn filtered-check [sym opts]
  115    (let [filter (or (.get (js/URLSearchParams. js/window.location.search) "filter") "")
  116          matches? (fn [s] (or (= filter "all")
  117                               (clojure.string/includes? (name s) filter)))]
  118      (when (matches? sym)
  119        (stest/check sym opts))))
                          ^---
#2020-09-0204:25johanatanit seems that stest/check may be a macro (I got "can't take value of macro" previously) but in the spec.alpha code it is defined as a function: https://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/test/alpha.clj#L373#2020-09-0204:39seancorfieldPerhaps it is a macro in ClojureScript?#2020-09-0204:40seancorfieldYup: https://cljs.github.io/api/cljs.spec.test.alpha/check it's a function in Clojure and a macro in ClojureScript.#2020-09-0204:43seancorfield@johanatan You might have to ask in #clojurescript -- that seems an unfortunate and gratuitous difference in the implementations.#2020-09-0212:54borkdudeI've also ran into this a couple of times#2020-09-0213:00borkdudeI think the root of this lies in dynaload which is a macro in CLJS because requires are only possible at compile time (when not using a REPL). This implies that library code around check yields another macro. E.g. https://github.com/borkdude/respeced/blob/f5ff67aa78f588e7bad2a1b86dd1a646d3fdab3d/src/respeced/test.cljc#L81 and https://github.com/borkdude/respeced/blob/f5ff67aa78f588e7bad2a1b86dd1a646d3fdab3d/src/respeced/impl.cljc#L71.#2020-09-0216:54johanatanI suppose my best course of action is to succumb to macro contagiousness and make this a macro ?#2020-09-0217:44borkdudemacros beget macros yeah#2020-09-0219:03johanatanšŸ™‚#2020-09-0221:17johanatanthis is what i ended up coming up with (for anyone following along):
(defmacro filtered-check [sym opts]
  `(let [fltr# (or (.get (js/URLSearchParams. js/window.location.search) "filter") "")
         check# (fn [res#]
                    (cljs.test/is (= true (-> res# :clojure.spec.test.check/ret :result))
                        (goog.string/format "spec check failure:\r\n%s" (with-out-str (cljs.pprint/pprint res#)))))]
     (when (or (= fltr# "all") (clojure.string/includes? (name ~sym) fltr#))
       (let [check-res# (clojure.spec.test.alpha/check ~sym ~opts)]
         (clojure.spec.test.alpha/summarize-results check-res#)
         (if (nil? check-res#)
           (cljs.test/is false "stest/check result was nil. did you pass it any valid symbols?")
           (doall (map check# check-res#)))))))
you can use it with figwheel main to filter your stest/check calls:
location/figwheel-extra-main/auto-testing?filter=the_filter
and with your calls to the macro such as:
(macros/filtered-check ns/sym {:clojure.spec.test.check/opts {:num-tests N})
#2020-09-0221:18johanatan[massive time saver once you start to accumulate many of these]#2020-09-0301:59kennytiltonIs clojure.spec a timid retreat from dynamism? In part bowing before the sacred cow of testing? "tests are essential to quality" --https://clojure.org/about/spec It seems like Clojure ventured bravely forth from the suffocating safety of Java's strong static typing, caught one gust of untyped wind, and scurried back to the shelter of Java's skirts. Or was this just a pragmatic choice to make Clojure more palatable in tall buildings?#2020-09-0302:00kennytiltonNot trolling, just trying to decide whether to buy into the burden of spec.#2020-09-0302:08seancorfieldYou certainly sound like you're trolling @hiskennyness but I know enough of your posts to know that's just kind of your "teasing" style šŸ™‚#2020-09-0302:09seancorfieldWe use Spec very heavily at work, in production code, to do data validation. It is not any sort of type system so, no, it is not any sort of "scurrying back to Java's skirts" šŸ™‚#2020-09-0302:10seancorfieldHave a read of https://corfield.org/blog/2019/09/13/using-spec/ and see the many ways we're using Spec at work -- none of which are in any way as a substitute for types.#2020-09-0302:13seancorfieldWe do find Spec to be very helpful in testing (and to some extent in development). We use specs to generate test data, to help support property-based testing, to valid test results...#2020-09-0302:13kennytiltonThx! I'll check it out. But reading that intro I kept hearing "OK, we should have objects with defined properties; maps are too loosey-goosey". Lemme see what HisCorfieldNess is up to...#2020-09-0302:17seancorfield(I'm going off to make dinner but I'm certainly happy to discuss Spec in great depth when I get back -- or tomorrow šŸ™‚ )#2020-09-0302:38kennytiltonOK, nice, short and sweet. Actually readable! But we likely disagree on testing, so no progress can be made there. As for runtime parameter checking, spec could be a handy way to code up what folks do anyway, but then that could be trivial little DSL I knock off in an hour then evolve easily over time. spec seems to be a handful at times, with people asking for help expressing different things. Sounds like a classic case of a good idea run amok. As for code generation from spec, now you are scaring me. šŸ™€ We have macros for DSLs, and I write the macro that parses a tree of symbols whose structure can be as user-friendly as I like? Think "CL loop" šŸ™‚. Why parse a spec definition, whose structure I cannot completely control? Even if I can make the latter work, it now feels as if I am in places coding my app in spec so it can be generated, when I already have a fine language to work with (Clojure!). Maybe this is why people get stuck using spec, because they ineluctably get drawn into more and more intense usage trying to program with it. So how widely used is spec?#2020-09-0303:53seancorfieldBack from dinner now @hiskennyness... I'd say Spec is pretty widely used. Lots of libraries provide optional Specs in a separate namespace so folks on Clojure 1.8 or earlier can avoid it and anyone on Clojure 1.9 or later can opt in if they wish.#2020-09-0303:55seancorfieldAs for generating code from Specs, that's about having one "source of truth" and it might as well be the Spec (esp. since mostly it's macros in Spec 1 and they're hard to deal with programmatically -- that will change in Spec 2).#2020-09-0303:55seancorfieldAnd we generate code from Specs we write.#2020-09-0303:58seancorfieldSpec has a lot of different uses. Some people do try to use it like a type system (bad idea, IMO, and that's not what it is designed for). Given your skepticism, I can only assume you've not watched any of the (many) talks about Spec over the years? Spec has been available for about four, four and a half years, since 1.9.0-alpha1 in May 2016.#2020-09-0312:18kennytiltonI'll google up some vids. Yes, well aware of spec for a while. Just ran into it at work for the first time, so it is time to get to know it. Thx for the feedback!#2020-09-0316:47seancorfieldFeel free to ping with any Qs if you want šŸ™‚#2020-09-0309:33borkdude> Lots of libraries provide optional Specs in a separate namespace I think this is good practice also for programs written for > 1.9#2020-09-0718:06Ho0manHi everyone, Can I write a multi-spec for the component/config segment of this data based on the component/type which is not part of component/config's corresponding value? :
{:component/type   [:instance-manager :v-2.0.0]
 ;;-----------------------------------------;;
 :component/config {,,,
                    }
 ;;-----------------------------------------;;
 :component/deps   {}}
(I'm using spec1) Thanks a lot
#2020-09-0718:07Ho0manthe component/config part has to be extensible ... so different components of the system across different namespaces define their own internal config map#2020-09-0718:31borkdudeIs it possible to inspect an evaluated regex spec?
(def x int?)
(s/fdef foo :args (s/cat :x int? :y x))
(prn (::s/op (s/cat :x int?))) ;; :clojure.spec.alpa/pcat
(prn (::s/op (:args (s/get-spec 'user/foo)))) ;; nil
Use case: I'd like to know what the x resolved to. Just inspecting the form won't tell me that.
#2020-09-0718:52borkdudeI hoped I could get more into the structure of the regex, but it seems to be an opaque thing#2020-09-0718:54borkdudeOtherwise formulated: it is possible to get back to the int? function once it's wrapped inside this reified spec object?
(s/spec int?)
#2020-09-0720:10borkdudeMaybe I'm looking for datafy on specs#2020-09-0720:28tekacs@borkdude (s/form (s/spec int?)) => 'clojure.core/int?#2020-09-0720:29tekacsis I think what youā€™re looking for#2020-09-0720:35borkdude@tekacs That's already better!
(clojure.spec.alpha/cat :x clojure.core/int? :y user/x)
#2020-09-0720:36tekacs@borkdude
(let [{:keys [x]} (rest (s/form (:args (s/get-spec 'user/foo))))] x)
=> clojure.core/int?
#2020-09-0720:37tekacsdefinitely not ideal, but that works?#2020-09-0720:37borkdudeI kinda wish that I could have user/x resolved to clojure.core/int? as well, since this information is available in the spec itself#2020-09-0720:38tekacsright ā€”Ā though you wouldnā€™t usually use def to define a sub-spec#2020-09-0720:38tekacsyouā€™d usually have done:
(s/def ::x int?)
(s/fdef foo :args (s/cat :x int? :y ::x))
#2020-09-0720:38borkdudeyeah, I guess using this for linter stuff is alright, but then I might as well just inspect the raw sexpr directly#2020-09-1816:46souenzzosee also https://github.com/wilkerlucio/spec-inspec/blob/master/test/com/wsscode/spec_inspec_test.clj#L10#2020-09-1817:35borkdudeThanks. Added link to https://github.com/clj-kondo/inspector/issues/4#2020-09-0720:39borkdudeyes, but in that case you still get :user/x#2020-09-0720:39tekacsright, but
(s/def ::x int?)
=> :user/x
(s/form (s/spec ::x))
=> clojure.core/int?
#2020-09-0720:39tekacsif you pass those through s/spec it resolves them#2020-09-0720:40borkduderight, you can recursively call that on qualified keywords#2020-09-0720:40tekacsright, precisely#2020-09-0720:40borkdudecool, thanks!#2020-09-0720:40tekacsindeed you can safely pass most things through (s/form (s/spec ā€¦)) I believe#2020-09-0720:40borkdude@tekacs purpose of this exercise: https://gist.github.com/borkdude/c0987707ed0a1de0ab3a4c27c3affb03#2020-09-0720:41tekacsIā€™ve since moved to malli but Iā€™m excited for this! šŸ™‚#2020-09-0720:41tekacsI saw that you posted in #malli too šŸ™‚#2020-09-0720:42tekacsI also forked aave for malli to add CLJS support and some additional features I havenā€™t seen elsewhere (https://github.com/tekacs/aave), so I should probably pay attention to the clj-kondo type spec šŸ™‚#2020-09-0720:43borkdude@tekacs That's cool, you can use the same kind of trick indeed#2020-09-0720:52borkdude@tekacs#2020-09-0720:56kennyThis is awesome. Support for s/keys would be a huge productivity win. We have many functions that take large maps as params and it's so easy to forget or misspell one.#2020-09-0720:56kennyThank you for working on this @borkdude šŸ˜€#2020-09-0721:24borkdude:)#2020-09-0721:25kennyOMG#2020-09-0721:26kennySo freaking cool. An optional toggle for something like https://github.com/bhauman/spell-spec would epic (I could see some people not liking it).#2020-09-0721:27kennyI wonder how many places in our code base we have a :req key and aren't actually passing it but it happens to magically work in prod šŸ˜…#2020-09-0721:28borkdudeNote that clj-kondo can only reliably detect this right now in places where literal maps are passed#2020-09-0721:28borkdudethere are some attempts around assoc et al, but not fully worked out#2020-09-0721:31borkdudebut I guess something is better than nothing :)
#2020-09-0721:31kennyAh, right - makes sense. And totally!#2020-09-0721:32kennyWonder how far you could go with it... e.g.,
(defn do-stuff
  [x])

(s/fdef do-stuff
  :args (s/cat :x int?)
  :ret (s/keys :req [::a]))

(defn do-moar-stuff
  [m]
  )

(s/fdef do-moar-stuff
  :args (s/cat :m (s/keys :req [::a ::b]))
  :ret int?)

(-> 1 
    (do-stuff)
    (do-moar-stuff))
Technically seems like you could infer that do-moar-stuff will (probably) not get passed the right stuff, assuming the specs are correct.
#2020-09-0721:35borkdudedo-stuff can still required a map which key ::b, although it's not required, so I don't see anything wrong here#2020-09-0721:35borkdudewhat we could however do is, when do-stuff has ret int? and do-moar-stuff expects a map, give an error#2020-09-0721:36borkdudeclj-kondo already has support for this, it just needs to be mapped from spec to the right format.#2020-09-0721:36kennyThe specs for do-stuff and do-moar stuff don't align and they should. By happenstance, do-stuff could always return ::b but that's just luck.#2020-09-0721:37kennyA future refactoring to do-stuff could result in ::b getting removed which will cause downstream do-moar-stuff to break.#2020-09-0721:40borkdudethe clj-kondo "type system" will only complain if it's sure something is wrong, so in this case it would not complain I think#2020-09-0721:41kennyI think that particular scenario would fall under the warning category.#2020-09-0721:41borkdudetype-wise, but maybe it could have a spec linter that gave a warning about this#2020-09-0721:42borkdudenot sure if I want to go very deep into s/keys, since spec2 will have different ways of doing#2020-09-0721:42borkdudebut basic type warnings as you type are already a free win#2020-09-0720:54borkdude(code updated here: https://gist.github.com/borkdude/c0987707ed0a1de0ab3a4c27c3affb03)#2020-09-0806:25ikitommi@borkdude if thatā€™s of any value, there is one decent spec form parser in spec-tools. Doesnā€™t cover regexs, but lot of other dirty details. There is also a comprehensive regression & progression test suite. e.g. testing that the form bugs are not fixed, together with link to the JIRA issue. Feel free to c&p any parts that you find useful.#2020-09-0806:25ikitommithe parser: https://github.com/metosin/spec-tools/blob/master/src/spec_tools/parse.cljc#2020-09-0806:25ikitommispec visiting tests: https://github.com/metosin/spec-tools/blob/master/test/cljc/spec_tools/visitor_all_test.cljc#2020-09-0806:31borkdudeInteresting, Iā€™ll have a look#2020-09-0919:40borkdudePushed the experiment to a repo: https://github.com/clj-kondo/inspector Contributions welcome.#2020-09-0919:49borkdudeHmm, I realize that this is just valid Clojure syntax:
(defn foo [^:int x ^:keyword y ^:string z]
  [x y z])

(prn (foo 1 2 3))
Could be a way to integrate clojure spec with existing defns without the need for writing another macro.
#2020-09-0919:49borkdudeNon-namespaced keyword: refers to a predicate, namespaced keyword: refers to a spec#2020-09-0919:51borkdudee.g.:
(s/def ::s string?)

(defn foo [^:int x ^:keyword y ^:ss z]
  [x y z])
#2020-09-0919:52Alex Miller (Clojure team)we're not going to do that, but there are some other ideas percolating#2020-09-0919:55borkdudeUnambiguous:
user=> (alias 'c 'clojure.core)
nil
user=> ::c/int?
:clojure.core/int?
user=> (defn foo [^::c/int? x])
#2020-09-0919:56borkdudeBit ugly, but one of those things that could have been deemed: yeah, Rich intended it like this all along, it was all from a design originating back to 2002 ;)#2020-09-0919:58Alex Miller (Clojure team)except it's the opposite of what Rich intends :)#2020-09-0919:58borkdudetell me more#2020-09-0919:59Alex Miller (Clojure team)this just plays to the familiarity from static typing langs, which is just not what Rich is trying to do with spec#2020-09-0920:01borkdudeThis would not be limited to simple annotations. You could do:
user=> (s/def ::foo (s/keys :req-un [::x]))
:user/foo
user=> (defn f [^::foo m])
so I don't get the static typing remark.
#2020-09-0920:04Alex Miller (Clojure team)it looks like static typed function signatures, but it's not#2020-09-0920:05borkdudeso?#2020-09-0920:05Alex Miller (Clojure team)so making things that are different look the same causes confusion#2020-09-0920:07Alex Miller (Clojure team)there are other considerations as well - compilation, evaluation, reuse across arities, combination with other features that don't exist yet, etc#2020-09-0920:07Alex Miller (Clojure team)Rich is looking at all that and considering a wide variety of options including the one above#2020-09-0920:07borkdudeI bet#2020-09-0920:08Alex Miller (Clojure team)I just think this one is probably unlikely based on what I've heard#2020-09-1417:42tvalerioHi! I have this spec in a namespace and a test so I can validate it:
(ns develop.rates)

(defn- valid-rate-intervals?
  [rate]
  false)

(s/def ::rates (valid-rate-intervals?))
(ns develop.rates-test
   (:use clojure.test)
   (:require
      [clojure.spec.alpha :as s]
      [develop.rates :refer :all])

(deftest should-test-a-rate
  (testing "Test a rate"
    (let [result (s/explain-data :develop.rates/rates {:rate "test"})]
      (prn "result: " result))))
Why canā€™t I use the spec like ::rates instead of the example in the code? If I try yo use it, I get an unable to find spec error. Also, when I watch the result of explain-data I can see that it begins with:
#:clojure.spec.alpha{:problems...
And not like:
#:rates{:problems...
As I saw in the examples. Iā€™m trying to verify which function got error to return a proper error message but Iā€™m getting a hard time to do so. Trying Spec for the first time. How could I do that?
#2020-09-1509:28anthony-galea@tvalerio See the following under the Literals section at https://clojure.org/reference/reader > A keyword that begins with two colons is auto-resolved in the current namespace to a qualified keyword So when you use ::rates inside rates.clj itā€™s as if youā€™ve written :develop.rates/rates but when you do the same inside rates-test.clj you get develop.rates-test/rates and not :develop.rates/rates#2020-09-1512:01tvalerioOk, I understood. Thanks @U446AB17F! =D#2020-09-1517:35kennyIt seems the conformed result of the pred passed to s/coll-of is getting disregarded. Is this behavior of expected?
(s/conform
    (s/and (s/tuple #{"a"} boolean?)
           (s/conformer (fn [[_ v]] [:a v])))
    ["a" true])
=> [:a true]

(s/conform
  (s/coll-of
    (s/and (s/tuple #{"a"} boolean?)
           (s/conformer (fn [[_ v]] [:a v])))
    :kind map?)
  {"a" true})
=> {"a" true}
#2020-09-1517:59seancorfield(the API URL was incorrect)#2020-09-1518:05Alex Miller (Clojure team)@kenny I think because you are using :kind map? you're into the map-specific conforming logic which (by default) does not conform keys#2020-09-1518:07Alex Miller (Clojure team)You can use :conform-keys to change that default:
(s/conform
  (s/coll-of
    (s/and (s/tuple #{"a"} boolean?)
           (s/conformer (fn [[_ v]] [:a v])))
    :kind map?
    :conform-keys true)
  {"a" true})

{:a true}
#2020-09-1518:37seancorfield(that's a more useful URL, sorry for the noise)#2020-09-1520:25kennyGreat, that works perfect. Thank you!#2020-09-1617:47kennyIs there a way to not include a key when using s/coll-of + kind map? + conform-keys true? For example,
(s/conform
  (s/coll-of
    (s/and (s/tuple #{"a"} boolean?)
           (s/conformer (fn [[_ v]] nil)))
    :kind map?
    :conform-keys true)
  {"a" true})
=> {nil nil}
That behavior seems correct. Curious if there's a way to indicate to not include the key though. Perhaps only way is to s/and in a conformer that removes nil keys?
#2020-09-1617:52Alex Miller (Clojure team)no way to do that#2020-09-1617:53Alex Miller (Clojure team)in general, use of conformers to do arbitrary transforms is considered an anti-pattern (conformer really exists for building new spec types primarily)#2020-09-1617:55kennyYep - definitely in a gray area. Thanks.#2020-09-2119:27johanatanhi, is there any way to override the comparator used for :distinct calculations for s/coll-of ?#2020-09-2119:27johanatanor is the best bet just to use s/and with a custom predicate ?#2020-09-2119:40Alex Miller (Clojure team)no way to do that currently#2020-09-2119:41Alex Miller (Clojure team)but s/and custom pred would work#2020-09-2119:41johanatan:+1:#2020-09-2120:46adamfreyis there a way to do something like :gen-max on a regex s/cat spec?#2020-09-2121:10Alex Miller (Clojure team)not currently#2020-09-2121:11Alex Miller (Clojure team)you can s/& a predicate to only accept smaller colls and that will have the same effect, but it's still producing the larger coll and that at least has memory implications (and may prevent such-that from succeeding at all)#2020-09-2415:53Jeff EvansFrom the guide: https://clojure.org/guides/spec#_custom_generators Iā€™m trying to figure out whether itā€™s possible to refer to kw-gen (which was just defined in the previous section) here, but I canā€™t quite figure out the right syntax. For instance, this doesnā€™t work:
(s/def ::kws (s/with-gen (s/and keyword? #(= (namespace %) "my.domain")) #(kw-gen)))
#2020-09-2416:38Alex Miller (Clojure team)
user=> (def kw-gen (s/gen #{:my.domain/name :my.domain/occupation :my.domain/id}))
#'user/kw-gen
user=> (s/def ::kws (s/with-gen (s/and keyword? #(= (namespace %) "my.domain")) (fn [] kw-gen)))
:user/kws
user=> (gen/sample (s/gen ::kws))
(:my.domain/name :my.domain/id :my.domain/name :my.domain/name :my.domain/name :my.domain/occupation :my.domain/occupation :my.domain/occupation :my.domain/id :my.domain/occupation)
#2020-09-2416:39Alex Miller (Clojure team)kw-gen here is a generator so you need to wrap it in a no-arg function that returns it (not invokes it)#2020-09-2417:30Jeff Evansthanks! I misunderstood something fundamental, since I thought #(kw-gen) was equivalent to (fn [] kw-gen)#2020-09-2417:34Alex Miller (Clojure team)the former is the same as (fn [] (kw-gen))#2020-09-2417:35Alex Miller (Clojure team)I guess you could also do (constantly kw-gen)#2020-09-2602:39johanatanhas anyone given much thought to (for the purposes of using multi-spec liberally across many namespaces) dynamically constructing "qualified keywords" at reader / compile time ?#2020-09-2602:39johanatan(in clojurescript)#2020-09-2602:40johanatani've attempted both a plain macro and a tagged literal reader#2020-09-2602:40johanatanthis is the macro:
(defmacro ns-grouped-keyword [group keyword]
  `(keyword (clojure.string/join "/" [~(str *ns*) ~group]) ~keyword))
#2020-09-2602:40johanatanand this is the reader:
(defn read-ns-grouped-keyword [s]
  `(let [[group# kwd#] (clojure.string/split ~s #"/")]
    (keyword (clojure.string/join "/" [~(str *ns*) group#]) kwd#)))
#2020-09-2602:41johanataneven the latter apparently only gets expanded and not eval'd at reader time#2020-09-2602:42johanatansuch that: (s/keys :opt-un [#my/reader "blah/blaz"]) will not work#2020-09-2602:43johanatanwhere as I had expected it to get eval'd to: (s/keys :opt-un [:the-current-namespace/blah/blaz]) before keys expansion time#2020-09-2615:24uditI am trying to use clojure.spec for validating the inputs to my API handlers. Whatā€™s the idiomatic pattern for usage here? I was thinking of creating a generic fn which takes in spec and the data to be validated. This fn can throw an exception when the validation fails. This fn would be called as the very first thing in the handlers, thus if the data is not valid the exception will act like an early exit. But this is using exceptions for control flow. The other approach I can think of is wrapping all my handlers with an if-else block that do this validation. This is fine, except it sort of makes my handlers less readable. Whatā€™s a clean approach that people have discovered for validating (and coercing) data using spec.#2020-09-2615:45Jeff EvansCanā€™t you have a single wrapper fn that basically does
(let [res (s/conform input)]
  (if (:s/invalid? res)
    (build-error-reponse)
    (real-handler input) ; real-handler could be an input to this fn
  )
)
Then the real-handler doesnā€™t have to worry about whether the input is valid, and your outermost layer (i.e. the one that first receives the request) can call this instead of real-handler directly. And if you want more details out of build-error-response you could use explain-str or explain-data instead of conform. Or perhaps Iā€™m misunderstanding the situation.
#2020-09-2616:45uditThis works @jeffrey.wayne.evans! Thanks! Although I would like to have different spec for different handlers, but I presume that can be passed in as a parameter to this fn a well, right?#2020-09-2616:54Jeff EvansSure! You would still have to somehow organize the specs per handler. Not sure if there is an established pattern for that but you could always have a keyword based map for them (handler function name as key?)#2020-09-2616:56ikitommireitit has a spec-module, you can declare request & response specs to endpoints, and there is a set of predefined middleware / interceptors to use them for validation, also coercion. Example here: https://github.com/metosin/reitit/blob/master/examples/ring-spec-swagger/src/example/server.clj#2020-09-2813:11arohnerWhen using stest/instrument in tests, how do you clean up after a test? e.g. ` (deftest foo (stest/instrument foo {:stub foo})#2020-09-2813:11Alex Miller (Clojure team)unstrument#2020-09-2813:11arohnerright, but you donā€™t know what the previous state was#2020-09-2813:12arohnerwas it instrumented, or stub+ instrumented?#2020-09-2813:13Alex Miller (Clojure team)Sorry, not understanding the question#2020-09-2813:14arohnerWe have some code that is always instrumented. During a test, if I call (instrument foo {:stub foo}), that overwrites the existing instrument with a stub+instrument. Now if I call unstrument at the end, the next test will run with foo un instrumented#2020-09-2813:16Alex Miller (Clojure team)Instrument remembers the old var and unstrument replaces it iirc#2020-09-2821:11borkdude@arohner I have some code to deal with that: https://github.com/borkdude/respeced#with-instrumentation#2020-09-2908:39Eugenhi, is there some tooling to generate spec code from XSD schema files ? I would like to validate with clojurescript (if possible) some OFX XML I generate#2020-09-2914:16fadrianI tried both of the XSD parsers I found using Google search (which looks like they came from the same root). Neither of them seemed to work. clj-xml will parse an XSD, but moving on from the parsed XML to a spec seems to still be a work-in-progress.#2020-09-2916:13mpenetI have a case where I generate specs, and sometimes I have fwd declarations in spec alises: (s/def ::foo ::bar) where ::bar is not declared/speced yet, so it blows up of course. A dirty way to do that is to do (s/and ::bar) as spec for instance but I am sure there's a better way (short of building a dep graph and specing in order)?#2020-09-2916:15Alex Miller (Clojure team)Forward references are actually intended to be ok in specs - the spec alias case (s/def ::foo ::bar) is one known exception to that (there is a ticket for this)#2020-09-2916:15Alex Miller (Clojure team)not sure if you've tried (s/spec ::bar) - that might also work#2020-09-2916:15Alex Miller (Clojure team)would be slightly less dirty if so#2020-09-2916:16mpenetI tried s/spec, doesn't work#2020-09-2916:16Alex Miller (Clojure team)well, can't say I have a better alternative for you then#2020-09-2916:16mpenet"unable to resolve spec ::bar "#2020-09-2916:17mpenetalright, thanks for the info. If it's a bug I'll take the s/and trick as acceptable for now#2020-09-2916:19mpenetI cannot find the jira for it, but I am not a good at jira'ing#2020-09-2916:52Alex Miller (Clojure team)https://clojure.atlassian.net/browse/CLJ-2067#2020-09-2916:53Alex Miller (Clojure team)or https://ask.clojure.org/index.php/3321/s-def-a-b-throws-unable-to-resolve-error-if-b-is-not-defined?show=3321#q3321 if you want to vote#2020-09-2917:05mpenetThanks!#2020-10-0209:08Petrus TheronWhere do I file Spec2 bug reports? (let [charset #{\a \b \c}] (s/+ charset)) throws :
Execution error (IllegalArgumentException) at clojure.alpha.spec/resolve-spec (spec.clj:219).
Symbolic spec must be fully-qualified: charset
However moving the set to a separate namespace or passing it in directly works, e.g.: (s/+ #{\a \b \c}) .
#2020-10-0212:20Alex Miller (Clojure team)This is expected behavior - there are some changes due to symbolic specs#2020-10-0212:22Alex Miller (Clojure team)https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha#symbolic-specs#2020-10-0215:10AronHow to have a generator that generates a constant string?#2020-10-0215:16Alex Miller (Clojure team)assuming you're talking spec 1, you can pull a generator from a set of one thing#2020-10-0215:17Alex Miller (Clojure team)(gen/sample (s/gen #{"a string"}))#2020-10-0215:23Aronthanks. Yeah, I was told not to use spec2 in production by someone#2020-10-0507:58Jim NewtonI'm not a spec user, but I have a few questions about the API. Is it true that spec designates certain symbols as patterns? How can I know whether a given symbol designates a spec pattern? As I understand, given such a symbol, I can call (s/valid? pattern-designator some-value) to determine whether a given value conforms to the pattern. Are such pattern designators always available globally? or can the be defined lexically and thus be invisible to other applications?#2020-10-0508:01Jim NewtonIs s/valid? side-effect free assuming the given pattern is side-effect free?#2020-10-0508:02Jim NewtonUnder which situations might s/valid? throw an exception? Do I need to wrap calls in try/catch ?#2020-10-0513:05Alex Miller (Clojure team)By ā€œpatternā€, I assume you mean ā€œsymbolic specā€. Specs have both a spec form (comprised of keyword names, list forms, sets, functions, etc) and a compiled instance form. Spec forms refer to macros that expand to code that emits spec instances. The details of this are quite a bit different in spec 1 and 2. But in both cases the spec forms are available globally as they are defined by macros. s/valid? does not have side effects of its own. s/valid? itself wonā€™t throw but if it includes a predicate, that might throw on a bad input#2020-10-0513:17Jim NewtonIs there a predicate I can use to know whether a given form is a valid "symbolic spec" ?#2020-10-0513:21Jim NewtonAm I correct that s/valid? will throw an error if you give it an invalid symbolic spec?#2020-10-0513:43Alex Miller (Clojure team)I think s/spec will either give you a spec instance or an error#2020-10-0513:43Alex Miller (Clojure team)s/valid? should do the same#2020-10-0513:44Jim Newtonthat's reasonable, yes. So It could be nice if my application could ask spec whether the given spec instance is valid or not, before putting it in a place where at some time in the future when s/valid? is eventually called, spec will throw an error.#2020-10-0513:45Jim NewtonOH, am I correct in assuming that s/valid? checks whether the given data matches the spec? i.e., it does not validate the spec, but rather validates the data...#2020-10-0514:52Alex Miller (Clojure team)yes#2020-10-0515:32Jim Newtonso which function should I use to detect whether a candidate given to my by the function who called me, is a valid symbolic spec?#2020-10-0515:39Alex Miller (Clojure team)spec 2 is a bit better in this area (but is not ready for use yet)#2020-10-0515:41vlaaad#(contains? (clojure.spec.alpha/registry) %)#2020-10-0515:49Alex Miller (Clojure team)that's not what he's asking for afaict#2020-10-0515:59Jim NewtonWithout knowing spec, it is hard to know really what I'm asking for. But Id like to know whether what I have in variable x is something I can pass as the first argument of s/valid?.#2020-10-0515:59Jim NewtonI suppose my first implementation doesn't to need to verify its values, just trust the caller, and let spec sort it out.#2020-10-0516:02Jim Newtonlooks like everything in that registry is either symbol or a keyword. (distinct( map (fn [x] (or (symbol? x) (keyword? x))) (keys (clojure.spec.alpha/registry)))) returns (true)#2020-10-0516:02borkdudesymbols are used for fdefs#2020-10-0516:59Alex Miller (Clojure team)s/valid? takes spec names, spec forms, and spec objects - there is no method that validates all of those#2020-10-0621:34Michael Stokleyis there a good way to spec out just some of the arguments in s/fdef's :args? like :args (s/cat :first-param (constantly true) :second-param int?)?#2020-10-0621:35ghadiany?#2020-10-0621:35Michael Stokleythanks!#2020-10-1223:27johnjelinekis there a way to make a spec validate a reference to another key's value? let's say you have a (s/coll double?) and you want to make a key where the predicate is something like (s/and double? #(= % (apply + ,,,)) where the ,,, is that (s/coll double?)#2020-10-1223:28johnjelinekmaybe your schema is like (s/schema [::double-list ::sum])#2020-10-1223:28johnjelinekI'm prolly not doing a great job of explaining this, does it vaguely make sense?#2020-10-1223:28Alex Miller (Clojure team)Can you give an example of valid data?#2020-10-1223:29Alex Miller (Clojure team)The example you gave should basically work so Iā€™m trying to make sure I understand what you mean #2020-10-1223:29Alex Miller (Clojure team)Are you talking about in a map?#2020-10-1223:30johnjelinekya#2020-10-1223:30Alex Miller (Clojure team)You can s/and a predicate that validates whatever you want#2020-10-1223:31johnjelinekok, so let's say concretely, I've got something like: {::prices '(1 2 3 4) ::total 10} and so total's predicate has to match the collection of prices (except doubles instead of ints)#2020-10-1223:33johnjelinekso, how would a (s/def ::total)'s predicates (apply + on ::prices?#2020-10-1223:37Alex Miller (Clojure team)(s/def ::m (s/and (s/keys ...) (fn [{::keys [total prices]}] (= total (reduce + prices))))#2020-10-1223:38Alex Miller (Clojure team)Just spec ::total as an int? or whatever#2020-10-1223:38johnjelinekoh!#2020-10-1223:38johnjelinekok great!#2020-10-1223:38Alex Miller (Clojure team)The attribute by itself does not have this constraint#2020-10-1223:39Alex Miller (Clojure team)It only has that constraint when combined with prices in a map#2020-10-1223:40johnjelinekI'm not sure I grok that yet#2020-10-1223:40Alex Miller (Clojure team)The total=prices is a property of the map, not of the total#2020-10-1223:42johnjelinekah, right#2020-10-1223:50johnjelinek
(s/def ::total int?)
  (s/def ::prices (s/coll-of int?))
  (s/def ::m (s/and (s/keys :req [::total ::prices])
                    (fn [{::keys [total prices]}] (= total (reduce + prices)))))
  (s/valid? ::m {::prices '(1 2 3 4) ::total 10})
#2020-10-1223:50johnjelinekthis spits out an java.lang.IndexOutOfBoundsException#2020-10-1223:54johnjelinek
(let [{::keys [total prices]} {::prices '(1 2 3 4) ::total 10}]
    (= total (reduce + prices))) ;=> true
((fn [{::keys [total prices]}]
     (= total (reduce + prices))) {::prices '(1 2 3 4) ::total 10}) ;=> true
(s/def ::m (fn [{::keys [total prices]}]
                 (= total (reduce + prices)))) ;=> java.lang.IndexOutOfBoundsException
(s/def ::m (s/and (s/keys :req [::total ::prices])
                    (fn [{::keys [total prices]}]
                      (= total (reduce + prices))))) ;=> :scratch/m
(s/valid? ::m {::prices '(1 2 3 4) ::total 10}) ;=> java.lang.IndexOutOfBoundsException
#2020-10-1300:19johnjelinekoh, hmm:
; (#:scratch{:keys [scratch/total prices]}) - failed: Extra input at: [:fn-tail :arity-1 :params] spec: :clojure.core.specs.alpha/param-list
#:scratch{:keys [scratch/total prices]} - failed: vector? at: [:fn-tail :arity-n :params] spec: :clojure.core.specs.alpha/param-list
#2020-10-1303:05Michael Stokleyis there an fdef that works for multi-methods?#2020-10-1303:49Alex Miller (Clojure team)no, but you can pull the dispatch function out and spec that#2020-10-1314:21Michael Stokleythank you!#2020-10-1303:13johnjelinekya, dunno what was up with that destructuring, but this works:
(s/def ::m (s/and (s/keys :req [::total ::prices])
                    #(= (::total %) (reduce + (::prices %)))))
#2020-10-1303:33johnjelinekah-ha! I think I got it! thx @alexmiller!#2020-10-1303:35johnjelinekI did something like this:
(s/def ::m (s/and ::total=prices (s/schema [::prices ::total])))
#2020-10-1303:50Alex Miller (Clojure team)just as a heads up, I do not consider spec 2 to be ready for use and the api may change before release#2020-10-1321:11vlaaadI have the following spec:
(s/def ::grid
  (-> (s/keys :opt-un [::grid])
      (s/coll-of :kind vector? :min-count 1)
      (s/coll-of :kind vector? :min-count 1)))
How do I do generative testing with this kind of spec? any attempt to generate an example of ::grid makes my OS super laggy until I kill the JVM process...
#2020-10-1321:37Alex Miller (Clojure team)that's not a valid spec? what are you expecting -> to do there?#2020-10-1405:14vlaaadOh, sorry, I added threading in the message for readability, actual spec doesn't have it#2020-10-1321:39Alex Miller (Clojure team)I guess maybe it would work in the spec 1 macrology, but I find that very confusing to parse (and it won't in spec 2 I think)#2020-10-1321:39Alex Miller (Clojure team)in general, setting :gen-max is useful for coll specs though to bound nested coll size#2020-10-1405:33vlaaadThanks, I'll try that#2020-10-1407:04vlaaadIt did help, thank you!#2020-10-1322:27kennyWe often build our own spec "types" by doing something like this.
(defmacro my-custom-map
  [kpred vpred]
  (let [form `(s/map-of ~kpred ~vpred)]
    `(s/with-gen ~form #(gen/fmap identity (s/gen ~form)))))

(s/def ::k string?)
(s/def ::v string?)

(s/def ::my-custom-map (my-custom-map ::k ::v))
This, however, does not play nice with :gen overrides.
(gen/generate (s/gen ::my-custom-map {::k #(s/gen #{"a" "b"})}))
=>
{"3" "b4tq9FeW6Gt",
 "8h" "o47F763P0v5B63sA52c4xu1FN",
...}
Is there a better way of doing this that works with :gen overrides?
#2020-10-1322:35kenny#2020-10-1323:09kennyThis seems to be a common pattern: https://stackoverflow.com/questions/43230546/a-clojure-spec-that-matches-and-generates-an-ordered-vector-of-variable-length#2020-10-1323:15kennyIt's like with-gen needs to take an argument with :gen overrides or implicitly use a dynamic variable.#2020-10-1323:41kennyCan't figure out any way around this other than to override the top level spec (in my example ::my-custom-map). That can be quite painful with complex specs.#2020-10-1416:07kennyšŸ˜¢#2020-10-1416:07kennyhttps://clojure.atlassian.net/browse/CLJ-2095#2020-10-1402:56johnjelinekquestion regarding preference: do you add your specs to an existing namespace of a domain model or do you have a separate spec namespace to share your specs around?#2020-10-1403:07seancorfield@johnjelinek Here's how we organize the different types of specs we write https://corfield.org/blog/2019/09/13/using-spec/#2020-10-1415:49Ivan Fedorovso, how safe is it to use s/conform as a regex matcher? As in https://juxt.pro/blog/parsing-with-clojure-spec
(s/def ::contentline
  (s/cat
    :name ::iana-token
    :params (s/*
              (s/cat
                :semi #{\;}
                :param-name ::iana-token
                :equals #{\=}
                :param-value ::iana-token))
    :colon #{\:}
    :value ::iana-token))

(s/conform
  ::contentline
  (seq "DTSTART;TZID=US-EAST:20180116T140000"))
=>
{:name [\D \T \S \T \A \R \T],
 :params
 [{:semi \;,
   :param-name [\T \Z \I \D],
   :equals \=,
   :param-value [\U \S \- \E \A \S \T]}],
 :colon \:,
 :value [\2 \0 \1 \8 \0 \1 \1 \6 \T \1 \4 \0 \0 \0 \0]}

(apply str (get-in ā€¦ [:params 0 :param-value]))
=> "US-EAST"
#2020-10-1416:02uwospec isn't designed for string parsing -- I've seen people recommend instaparse: (alex miller on the topic) https://www.reddit.com/r/Clojure/comments/fx5ko1/parser_libraries/fmttmzy/?utm_source=amp&amp;utm_medium=#2020-10-1416:04Ivan FedorovThanks @U09QBCNBY!#2020-10-1416:10Ivan FedorovI think Iā€™m not that interested in performance here, more interested in how stable the implementation will be.#2020-10-1415:51Ivan FedorovThinking about using it to parse recurrence rule from RFC5545 https://icalendar.org/iCalendar-RFC-5545/3-3-10-recurrence-rule.html#2020-10-1417:14Alex Miller (Clojure team)we do not recommend using regex specs to parse strings#2020-10-1505:35Ivan Fedorovack, thanks!#2020-10-1810:16mike_ananev@alexmiller do you have any plans to release spec2? The winter is coming, and I'll write to Santa my wishes: new Clojure release and Spec2 release in new year. šŸ˜†#2020-10-1812:46Alex Miller (Clojure team)Sorry, no plans either way#2020-10-1817:39thomIs there some list of milestones that need to be achieved before we ever see a non alpha Spec? Anything the community should be helping with?#2020-10-1818:02Alex Miller (Clojure team)no, it is mostly design work that Rich is working through at the moment#2020-10-1818:22vlaaadI'm curious what's missing in current spec vision that requires more design work...#2020-10-1818:36Alex Miller (Clojure team)redoing function specs#2020-10-1818:36Alex Miller (Clojure team)there are other things on the list but that's the big one atm#2020-10-2417:16wcalderipehey, is there a way to reuse the same spec for different map keys without having to redefine them over and over?
(s/def ::address (some-pred-to-check-ethereum-addr))

(s/def ::tx (s/keys :req-un [:sender ::address  ;; apply address spec to :sender 
                             :recipient ::address])) ;; apply address spec to :recipient 
#2020-10-2417:38Alex Miller (Clojure team)In short, no#2020-10-2418:36borkdude@wcalderipe you can do this:
(def spec (s/....)
(s/def ::a spec)
(s/def ::b spec)
#2020-10-2418:37borkdudeNote sure if that also works in spec2 - I completely lost track of that#2020-10-2418:41Alex Miller (Clojure team)No, it wonā€™t like that but you could set it quoted and make it work via s/register#2020-10-2418:42Alex Miller (Clojure team)But I donā€™t think that works the way itā€™s requested with req-un in either case#2020-10-2418:47wcalderipethanks @alexmiller and @borkdude#2020-10-2418:47borkdudeI think select might work better here right in spec2?#2020-10-2418:49Alex Miller (Clojure team)Yes, but will still have repetition#2020-10-2418:48wcalderipes/keys is the best way to check the shape of a map?#2020-10-2418:49Alex Miller (Clojure team)For stuff like above, yes#2020-10-2620:28waffletowerIs there a way to limit recursion depth for clojure.test.check.generators/recursive-gen?
(require '[clojure.test.check.generators :as g])
(binding [s/*recursion-limit* 1]
        (g/generate (g/recursive-gen #(g/vector % 1 2) g/boolean)))
#2020-10-2620:28waffletowerSeeing results like:
[[[false]] [[true] [true]]]
#2020-10-2620:30gfredericksNo. It's your example result a problem?#2020-10-2620:30waffletowerYes, this is a contrived example, trying to limit stack overflows with a recursive spec#2020-10-2620:31gfredericksI think recursive specs are generated in a different way and have their own depth control. Not 100% sure#2020-10-2620:33waffletowerWrapping the collections within my component specs using recursive-gen has made stack overflows statistically less likely than not using that wrapping. So I thought they might be helpful if they could be further controlled#2020-10-2620:35waffletowerAre there other depth controls beside s/*recursion-limi*t?#2020-10-2620:40Alex Miller (Clojure team)no, that's it - there are a few places where they are not being properly checked I think (b/c the gen code wrongly doesn't go through the place it's checked)#2020-10-2620:40Alex Miller (Clojure team)that's just a suspicion right now but seems like I've seen that before#2020-10-2620:41Alex Miller (Clojure team)and that's only for spec generators, won't affect directly constructed test.check generators#2020-10-2810:04borkdudeGrep Clojure code using clojure.spec: https://gist.github.com/borkdude/a391146ad81a06c28fb97ccdc1f64d44#2020-10-2810:04borkdudeGrep Clojure code using clojure.spec: https://gist.github.com/borkdude/a391146ad81a06c28fb97ccdc1f64d44#2020-10-2816:49borkdudeAlright, let's ship it: https://github.com/borkdude/grasp#2020-10-3020:54teodorluGood evening! I've written specs to validate a small math language with addition and multiplication. Now I want to generate values! But I'm hitting Stack Overflow. I assume that's because my specs/generators are mutually recursive. Any tips for writing mutually recursive spec generators that don't blow the stack? Code attached.#2020-10-3022:35teodorluIdea: write generators as functions rather than directly on specs. Argument: recursion depth. Call generator functions from the spec.#2020-10-3022:51teodorluFollowed through on a generator namespace with recursion limit. That fixed my StackOverflow. Sample for add and multiply:
(defn add [depth]
  (gen/fmap to-list
            (gen/tuple (gen/return '+) (expression (dec depth)) (expression (dec depth)))))

(defn expression [depth]
  (if-not (pos? depth)
    (number)
    (gen/one-of [(number)
                 (multiply depth)
                 (add depth)])))
#2020-10-3113:48teodorluNote: Tree depth can also be controlled probabilistically. Make it more probable to take the non-recursive path than the recursive path. Example from the docs:
(gen/frequency [[9 gen/small-integer] [1 (gen/return nil)]])
The probability of the non-recursive path would have to counterbalance the fanout for the recursive paths.
#2020-10-3114:23teodorluFrom reading test.check source, I see that many generators accept a size parameter. I suspect that I could have used that instead of inventing my own depth.#2020-10-3022:52borkdudeGrasp now supports finding keywords using clojure specs while also preserving location metadata! https://github.com/borkdude/grasp#finding-keywords This should come in handy for finding re-frame events and subscriptions in CLJS apps. I also made a #grasp channel#2020-10-3114:02borkdudeThe ::invalid keyword occurs 67 times in the spec repo:
$ ./grasp ~/git/spec.alpha/src -w -e "(fn [k] (= :clojure.spec.alpha/invalid (unwrap k)))" | wc -l
      67
#2020-10-3114:26teodorluAnd you're finding both fully qualified :clojure.spec.alpha/invalid and shorthand ::invalid / ::s/invalid uses? Nice! I'm curious about use-cases for grasp in tooling for refactoring.#2020-10-3115:01borkdudeyes#2020-10-3116:55kirill.salykinHi please advice, how I can write a spec for :fileds
(s/explain-data ::fields [{:name "aaa" :type "bbbb"} {:name 1 :type "cccc"}])
so it checks that fields/name is unique - fails on every non unique entry and the spec also contains path of the non unique row Thanks
#2020-10-3118:59borkdude@kirill.salykin The uniqueness of the name is a property of the collection, not of one item
user=> (s/def ::unique-names (fn [xs] (= (map :name xs) (distinct (map :name xs)))))
:user/unique-names
user=> (s/explain ::unique-names [{:name :foo} {:name :bar}])
Success!
nil
user=> (s/explain ::unique-names [{:name :foo} {:name :foo}])
[{:name :foo} {:name :foo}] - failed: (fn [xs] (= (map :name xs) (distinct (map :name xs)))) spec: :user/unique-names
#2020-10-3119:00kirill.salykinI understand and agree thought there is a dark magic to get indecies of non-unique elements, but apparently no thanks for the answer#2020-10-3119:01borkdudeThere might be, but I don't know if it's pretty#2020-10-3119:03kirill.salykinmaybe there is a way shape spec output in a way as it reports per failed element (eg with s/problems, path (`:in []`) and predicate?)#2020-10-3119:10borkdude@kirill.salykin Maybe something like this:
user=> (s/def ::unique-names (s/and #(group-by :name %) (s/every-kv any? #(= 1 (count %)))))
:user/unique-names
user=> (s/explain ::unique-names [{:name :foo} {:name :foo}])
Execution error (UnsupportedOperationException) at user/fn (REPL:1).
nth not supported on this type: PersistentArrayMap
but I'm getting an exception
#2020-10-3119:11kirill.salykininterestingā€¦ so (s/and ) behaves like ->?#2020-10-3119:11Dmytro Buninno afaik#2020-10-3119:16borkdudeIt needed s/conformer around the first spec#2020-10-3119:15borkdude@kirill.salykin Ah, I needed s/conformer:
user=> (s/def ::unique-names (s/and (s/conformer #(group-by :name %)) (s/every-kv any? #(= 1 (count %)))))
:user/unique-names
user=> (s/explain ::unique-names [{:name :foo} {:name :foo}])
[{:name :foo} {:name :foo}] - failed: (= 1 (count %)) in: [:foo 1] at: [1] spec: :user/unique-names
#2020-10-3119:18Dmytro Bunintil, thanks šŸ™‚#2020-10-3119:16kirill.salykinyou are magician! thank you so much!#2020-10-3120:23borkdudeI've made binary versions of grasp available in the #grasp channel now. Clojure spec on the command line, yeah :)#2020-11-0312:39borkdudeIs there any existing work combining clojure-spec conform + core.match or datalog? Like: I want ?x and ?y from (s/conform ...)? I could see this being useful for grasp#2020-11-0312:54delaguardotake a look at mine matchete https://github.com/xapix-io/matchete if meander is not a good fit#2020-11-0312:58borkdudethanks. can you describe the difference between meander and yours?#2020-11-0313:01delaguardoā€¢ no macro ā€¢ less power -> cleaner pattern syntax (this is just my opinion, btw) ā€¢ much smaller codebase -> much easier to read through and explore#2020-11-0313:02borkdudešŸ‘#2020-11-0314:34borkdude@U04V4KLKC How do I express this meander usage in your lib?
(m/find (first conformed) {:clauses {:clause {:sym !interface} :clauses [{:sym !interface} ...]}} !interface)
#2020-11-0314:35borkdudeThe output should be something like:
[clojure.lang.IDeref clojure.lang.IBlockingDeref clojure.lang.IPending java.util.concurrent.Future]
#2020-11-0314:35borkdudeAnd the input:
{:reify reify, :clauses {:clause {:sym clojure.lang.IDeref, :lists [(deref [_] (deref-future fut))]}, :clauses [{:sym clojure.lang.IBlockingDeref, :lists [(deref [_ timeout-ms timeout-val] (deref-future fut timeout-ms timeout-val))]} {:sym clojure.lang.IPending, :lists [(isRealized [_] (.isDone fut))]} {:sym java.util.concurrent.Future, :lists [(get [_] (.get fut)) (get [_ timeout unit] (.get fut timeout unit)) (isCancelled [_] (.isCancelled fut)) (isDone [_] (.isDone fut)) (cancel [_ interrupt?] (.cancel fut interrupt?))]}]}}
#2020-11-0314:52borkdudeHere's the full example: https://github.com/borkdude/grasp#meander#2020-11-0315:49delaguardoalmost the same for that particular case
(require '[matchete.core :as mc])

(def data
  '{:reify reify
    :clauses {:clause {:sym clojure.lang.IDeref
                       :lists [(deref [_] (deref-future fut))]}
              :clauses [{:sym clojure.lang.IBlockingDeref
                         :lists [(deref [_ timeout-ms timeout-val] (deref-future fut timeout-ms timeout-val))]}
                        {:sym clojure.lang.IPending
                         :lists [(isRealized [_] (.isDone fut))]}
                        {:sym java.util.concurrent.Future
                         :lists [(get [_] (.get fut)) (get [_ timeout unit] (.get fut timeout unit)) (isCancelled [_] (.isCancelled fut)) (isDone [_] (.isDone fut)) (cancel [_ interrupt?] (.cancel fut interrupt?))]}]}})

(def pattern
  {:clauses
   {:clause {:sym '!interface}
    :clauses (mc/scan {:sym '!interface})}})

(mc/matches pattern data)
#2020-11-0315:57borkdudeThis returns:
({!interface [clojure.lang.IDeref clojure.lang.IBlockingDeref]} {!interface [clojure.lang.IDeref clojure.lang.IPending]} {!interface [clojure.lang.IDeref java.util.concurrent.Future]})
#2020-11-0315:57borkdudeIs there a way to get a list of only the symbols like with meander?#2020-11-0316:00delaguardosorry, I was too fast copypasting ) instead of mc/scan please use mc/each#2020-11-0316:00delaguardoscan is for inspecting items in collection individually (in separate decision branches) each is for walking over a collection#2020-11-0316:01delaguardo
(def pattern
  {:clauses
   {:clause {:sym '!interface}
    :clauses (mc/each {:sym '!interface})}})
this pattern should work
#2020-11-0316:01borkdudeI get:
No such var: mc/each
#2020-11-0316:04delaguardoyou are using experimental 2.0.0 version check out 1.2.0#2020-11-0316:05borkdudeno, I was using the version that is suggested in the README
$ clj  -Sdeps '{:deps {io.xapix/matchete {:mvn/version "1.1.0"}}}'
:)
#2020-11-0316:05delaguardoargh, cljdoc badge got updated for master branch as well,#2020-11-0316:06delaguardothen this is my bad, I should update it there#2020-11-0316:06delaguardosorry#2020-11-0316:07borkdudeno problem!#2020-11-0316:55borkdudeExcellent, it works now!
user=> (def conformed (map #(s/conform ::reify %) matches))
#'user/conformed
(def pattern
  {:clauses
   {:clause {:sym '!interface}
    :clauses (mc/each {:sym '!interface})}})
#'user/pattern
user=> (mc/matches pattern (first conformed))
({!interface [clojure.lang.IDeref clojure.lang.IBlockingDeref clojure.lang.IPending java.util.concurrent.Future]})
#2020-11-0317:05borkdude@U04V4KLKC https://github.com/borkdude/grasp#matchete#2020-11-0317:18borkdudeNice, it seems your lib also works with babashka:
$ bb -cp "$(clojure -Spath)" -e "(require '[matchete.core :as mc]) (mc/matches '{:x ?x} {:x 1})"
({?x 1})
#2020-11-0317:50delaguardošŸŽ†#2020-11-0317:51delaguardoanother pusher to polish documentation šŸ˜‰#2020-11-0312:41borkdudeMaybe meander is a good match for this#2020-11-0315:27Jim NewtonI'm starting looking at clojure.spec.alpha can someone explain the different between the s/valid? function and the s/spec? function? Their docstrings look pretty similar.#2020-11-0315:27borkdude@jimka.issy spec? is more like an instance check, valid? validates a value against a spec#2020-11-0315:28borkdudein fact, it is an instance check :)#2020-11-0315:33Jim NewtonAs I understand the normal use case is the user associates certain tags, normally starting with :: with some DSL object which tells how to check validity? Does spec? check whether the symbol has been associated with a spec object, or does it check whether this is such an object?#2020-11-0315:34Jim NewtonHow can I know whether a symbol such as ::xyzzy designates a spec or not?#2020-11-0315:34borkdude
user=> (s/def ::foo int?)
:user/foo
user=> (s/spec? ::foo)
nil
user=> (s/spec? (s/get-spec ::foo))
#object[clojure.spec.alpha$spec_impl$reify__2059 0x4d48bd85 "
#2020-11-0315:35borkdudeSo get-spec returns the spec for a keyword or a symbol (in case of fdef)#2020-11-0315:38borkdude@jimka.issy There is also s/spec:
user=> (s/spec ::foox)
Execution error at user/eval162 (REPL:1).
Unable to resolve spec: :user/foox
user=> (s/spec ::foo)
#object[clojure.spec.alpha$spec_impl$reify__2059 0x4d48bd85 "
#2020-11-0315:38borkdudefwiw, I don't use these functions a lot, except maybe when I'm writing tools around spec. Not in normal daily spec usage#2020-11-0315:39borkdudes/def, s/valid? and s/conform are probably the most common ones I use#2020-11-0316:01Jim NewtonAfter something like the following:
(s/def ::big-even (s/and int? even? #(> % 1000)))
can I call some function on ::big-even to get back (s/and int? even? #(> % 1000) ?
#2020-11-0316:06borkdudealmost:
user=> (s/def ::big-even (s/and int? even? #(> % 1000)))
:user/big-even
user=> (s/form ::big-even)
(clojure.spec.alpha/and clojure.core/int? clojure.core/even? (clojure.core/fn [%] (clojure.core/> % 1000)))
#2020-11-0316:09Alex Miller (Clojure team)s/describe returns a shorter form, useful for printing (but not fully resolved so not too useful as data)#2020-11-0316:10Jim Newtonseems like s/form does what I want. But it's not certain.#2020-11-0316:11Alex Miller (Clojure team)research question - if you are using specs with aliases, do you tend to use one alias, a few, or many in any given namespace?#2020-11-0316:12borkdudewhat do you mean by an alias?#2020-11-0316:12borkdudelike (alias 'foo 'bar) and then ::bar/spec ?#2020-11-0316:13borkdude@alexmiller I think grasp could be used to find this out#2020-11-0316:13Lennart Buitone reason for me to use many aliases would be if many keys in a s/keys require a same generator. For one, we have a present string spec, with corresponding generator that we alias a lot#2020-11-0316:17Lennart Buitah nevermind then, I interpreted this as an alias:
(s/def ::my-spec ...)
(s/def ::key-2 ::my-spec)
(s/def ::key-1 ::my-spec)
(s/def ::map (s/keys :req [::key-2 ::key-2]))
#2020-11-0316:14borkdudehmm, never mind, grasp returns the fully qualified keyword, so you can't see if it was aliased or not#2020-11-0316:15Alex Miller (Clojure team)@borkdude yes, that's what I mean#2020-11-0316:16Alex Miller (Clojure team)https://grep.app/search?q=%28alias%20%27&amp;filter[lang][0]=Clojure gets me some idea at least#2020-11-0316:17Alex Miller (Clojure team)but really wondering what people are doing in private code bases#2020-11-0316:17borkdudewe usually don't use alias for this, since a lot of our specs are in CLJS. We create namespaces on disk for this#2020-11-0316:21borkdude@alexmiller typically we have this in our backend, using on disk namespaces:
(s/def ::system
  (s/merge ::dict/system
           ::adis/system
           ::annotator/system
           (s/keys :req [::database/db ::cache/cache])))
#2020-11-0317:23Jim NewtonMore spec questions. When I define ::big-even as in https://clojure.org/guides/spec.
(s/def ::big-even (s/and int? even? #(> % 1000)))
I'm guessing that the function #(> % 1000) is a closure in its current lexical context. Right? So later when I call s/form I don't get back that object, but some new text, which doesn't know the original lexical context. Right?
(s/form ::big-even)
 ==> (clojure.spec.alpha/and
 clojure.core/int?
 clojure.core/even?
 (clojure.core/fn [%] (clojure.core/> % 1000)))
#2020-11-0317:26borkdude@jimka.issy The text #(...) is not representable as an s-expression, it's something built into the reader which expands to (fn [...] ...)#2020-11-0317:27borkdudeWhat simply happens is that s/def is a macro which captures the input and by that time the s-expression has already expanded to the fn form#2020-11-0317:36Alex Miller (Clojure team)the spec function will capture the lexically scoped values at the point of definition (but the form will not - it will have the names)#2020-11-0317:36Alex Miller (Clojure team)
user=> (let [x 100] (s/def ::foo (s/and int? #(= % x))))
:user/foo
user=> (s/valid? ::foo 100)
true
user=> (s/valid? ::foo 99)
false
user=> (s/form ::foo)
(clojure.spec.alpha/and clojure.core/int? (clojure.core/fn [%] (clojure.core/= % x)))
#2020-11-0317:36Alex Miller (Clojure team)^^ form is broken there#2020-11-0317:42Alex Miller (Clojure team)this is all somewhat more consistent in spec 2 (namely, this will not work at all as the divide between symbolic specs and spec objects is clearer) but there is built-in support for creating parameterized specs (like the one above) if needed#2020-11-0408:51Jim Newton@alexmiller, thanks for the explanation. however, i'd like to get the actual function object which spec would use to validate with. For example given something like (s/and int? #= % x)), I can use int? to get the int? function, but I can't use the s-expression form of the closure to recuperate the closure. However, I'm pretty sure spec has the closure somewhere, otherwise s/valid? would not work.#2020-11-0408:58Jim NewtonWhat I'm trying to do is figure out which spec incantations can be expressed in a simple type system which I have implemented, called genus , normally abbreviated gns/. For example, a naĆÆve first attempt is able to translation ::big-even as follows
clojure-rte.rte-core> (require '[clojure.spec.alpha :as s])
nil

clojure-rte.rte-core> (s/def ::big-even (s/and int? even? #(> % 1000)))
:clojure-rte.rte-core/big-even

clojure-rte.rte-core> (s/get-spec ::big-even)
#object[clojure.spec.alpha$and_spec_impl$reify__2183 0x61966631 "
The important result being the next step:
clojure-rte.rte-core> (gns/canonicalize-type '(spec ::big-even))
(and
 (or Long Integer Short Byte)
 (satisfies clojure.core/even?)
 (spec (clojure.core/fn [%] (clojure.core/> % 1000))))
So genus can unwind the spec into types and predicates, if the predicates are symbolic. But what's left (clojure.core/fn [%] (clojure.core/> % 1000)) is not something which is useable.
#2020-11-0409:00Jim NewtonI could try to write a translator/compiler for expressions such as (clojure.core/fn [%] (clojure.core/> % 1000)) which try to convert them back to a closure, in the case that they don't have any free variables. But that work seems redundant since spec already has the closure in its internal data structure.#2020-11-0409:09Jim NewtonI just realized something interesting. If I eval the list (clojure.core/fn [%] (clojure.core/> % 1000)) then I get a function object which seems to have the same semantics as the original function, in case there are no free variables.
clojure-rte.rte-core> ((eval '(clojure.core/fn [%] (clojure.core/> % 1000)))  12)
false
clojure-rte.rte-core> ((eval '(clojure.core/fn [%] (clojure.core/> % 1000)))  100000)
true
clojure-rte.rte-core> 
Perhaps that is good enough for my proof-of-concept.
#2020-11-0409:10Jim NewtonAnd if there's a free variable, eval throws an exception
clojure-rte.rte-core> (eval '(clojure.core/fn [%] (clojure.core/> x 1000)))
Syntax error compiling at (clojure-rte:localhost:51477(clj)*:45:51).
Unable to resolve symbol: x in this context
#2020-11-0414:11Alex Miller (Clojure team)Well eval is literally the function that takes a form and returns a function
#2020-11-0317:42Alex Miller (Clojure team)
user=> (s/defop narrow-int [x] (s/and int? #(= % x)))
#'user/narrow-int
user=> (s/def ::foo (narrow-int 100))
:user/foo
user=> (s/form ::foo)
(user/narrow-int 100)
user=> (s/valid? ::foo 100)
true
user=> (s/valid? ::foo 99)
false
#2020-11-0410:05Jim NewtonYet another question about spec. This time about the semantics of the sequence designators such as s/alt, s/+, s/* etc. an element such as (s/+ even?) means (if I understand correctly, a sequence of 1 or more objects for which the even? predicate is true. but I can substitute a spec name for even? and say (s/+ ::big-even) and that will be a sequence of 1 or more objects each of which validates the ::big-even spec. However, what does (s/+ (s/alt :x even? :y prime?)) mean. I would guess that it means a sequence of 1 or more elements each of which either satisfy even? or satisfy prime? But if I define
(s/def ::even-or-prime (s/alt :x even? :y prime?)) 
what does this mean? (s/+ ::even-or-prime) It is a sequence of 1 or more sequences (each of which contains elements which are either even or prime), or is it a sequence of 1 or more integers each of which is either even or prime?
#2020-11-0410:06Jim NewtonI'm pretty sure spec has a way to designate both, as they are both possible things the user might want to express. right?#2020-11-0410:07borkdude@jimka.issy s/alt is for regexes, use s/or for normal predicates#2020-11-0410:08borkdude(s/+ (s/alt ....)) means a sequence of one or more alternatives#2020-11-0410:09borkdudeit doesn't mean each, this can be expressed with s/every or s/coll#2020-11-0410:11Jim NewtonSorry, my question too convoluted. Let me ask simpler. If ::foo is a spec defined somewhere using s/def , what does (s/+ ::foo) mean? Does it mean a sequence of 1 or more objects which each validate ::foo ?#2020-11-0410:11borkdudeyes#2020-11-0410:12Jim NewtonSo it is not referntially transparent?#2020-11-0410:13Jim NewtonI.e. substituting the expression for ::foo in place, changes the meaning?#2020-11-0410:13borkdudeI don't see why that would be the case?#2020-11-0410:17Jim Newtonsuppose
(s/def ::foo (s/alt :x even? :y prime?))
Now (s/+ (s/alt :x even? :y prime?)) matches a sequence of 1 or more integers each of which are either even or prime, right? but (s/+ ::foo) matches a sequence of 1 or more objects which each satisfy :foo , i.e., a sequence of sequences each of which contain even or prime integers.
#2020-11-0410:20borkdude@jimka.issy Please give a working example. E.g.:
user=> (s/def ::foo (s/or :x even? :y neg?))
:user/foo
user=> (s/conform (s/+ ::foo) [2 4 -11])
[[:x 2] [:x 4] [:y -11]]
user=> (s/conform (s/+ (s/or :x even? :y neg?)) [2 4 -11])
[[:x 2] [:x 4] [:y -11]]
#2020-11-0410:23borkdudeA spec is only applied to one thing, not to all elements of a collection, unless you use s/every or s/coll-of.#2020-11-0410:24borkdude
user=> (s/def ::nums-or-strings (s/or :nums (s/every number?) :strings (s/every string?)))
:user/nums-or-strings
user=> (s/conform ::nums-or-strings [1 2 3])
[:nums [1 2 3]]
user=> (s/conform ::nums-or-strings ["foo" "bar"])
[:strings ["foo" "bar"]]
#2020-11-0410:24Jim Newtonyes, but you answer avoids my question. I'm not asking how to write a good spec. I'm asking about the semantics.#2020-11-0410:25borkdudeAnd I asked you about a counter-example of referential transparency. A working one without imaginary functions like prime? which does not exist in core.#2020-11-0410:25Jim NewtonOk here is a very simple one.#2020-11-0410:26vlaaadI mean you donā€™t really need prime? source to know its semantics#2020-11-0410:26Jim Newton(s/+ ::foo) matches a sequence for which all the elements are either even or negative.#2020-11-0410:26vlaaadlooks right#2020-11-0410:27borkdudeYour words are ambiguous to me. Do you mean: every element is a ..., or on a case by case basis?#2020-11-0410:27Jim Newtonlets stick with integers and sequences thereof.#2020-11-0410:28Jim NewtonNow define
(s/def ::foo2 (s/+ ::foo))
#2020-11-0410:29Jim Newtonwhat does (s/+ ::foo2) mean. It does not mean a sequence of 1 or more objects which of which match ::foo2#2020-11-0410:30borkdudeWhat does ::foo mean in your code here?#2020-11-0410:30borkdudesome kind of integer that's either this or that?#2020-11-0410:30Jim Newton
(s/def ::foo (s/or :x even? :y neg?))
#2020-11-0410:31borkdudethen ::foo2 means multiple even or negs, but they can be mixed. it's not all or nothing#2020-11-0410:31vlaaadI would. say (s/+ ::foo2) is a seq of 1 or more objects each of which is is ::foo2, why would you say it isnā€™t that?#2020-11-0410:37Jim Newton
(s/def ::foo (s/or :x even? :y neg?))
(s/def ::foo2 (s/+ ::foo))
#2020-11-0410:38Jim NewtonNow ::foo matches even and negative integers#2020-11-0410:38Jim Newton(s/+ ::foo) matches a non-empty sequence of even and negative integers#2020-11-0410:39vlaaadsounds right#2020-11-0410:39Jim Newtonbut (s/+ ::foo2)does not match a non-empty sequence of objects which match ::foo2#2020-11-0410:40vlaaadah, I think thatā€™s because of how regex ops nest#2020-11-0410:40Jim Newton#2020-11-0410:41Jim Newtonyess!!!!!!!!#2020-11-0410:41Jim Newtonthis was my original question (s/+ x) does not match a non empty sequence of things which match x#2020-11-0410:41borkdude@jimka.issy If you want to do this, you need to use s/spec around the ::foo2#2020-11-0410:42borkdudeThis has to do with regex nesting indeed, not with referential transparency. Read the spec guide, which explains this#2020-11-0410:42borkdude> When regex ops are combined, they describe a single sequence. If you need to spec a nested sequential collection, you must use an explicit call to spec to start a new nested regex context.#2020-11-0410:43vlaaad(s/valid? (s/+ (s/spec ::foo2)) [[2 -3 4] [2 -3 4] [2 -3 4]]) => true#2020-11-0410:43Jim NewtonI have read down https://clojure.org/guides/spec#_sequences.#2020-11-0410:44Jim NewtonDid I miss the paragraph about when regex ops are combined.#2020-11-0410:44borkdudeI assume you know how Ctrl-f works ;)#2020-11-0410:44Jim NewtonBesides, I'm confused about what "regex" operator means.#2020-11-0410:44borkduderegex operators are things like s/+, s/?#2020-11-0410:44Jim Newtonsometimes it means string regexps. and some times it means sequence operations.#2020-11-0410:45borkdudein the context of spec, regex means a spec which describes a sequence of things#2020-11-0410:46borkdudeThe idea of spec is based on this paper: http://matt.might.net/papers/might2011derivatives.pdf where that terminology comes from#2020-11-0410:47Jim Newtonso does this mean that if a sequence operator is directly within another sequence operator it has this special modal/merging meaning. otherwise it has its normal meaning?#2020-11-0410:47borkdude(or maybe not that one, but the paper where that paper is based on)#2020-11-0410:48borkdude> When regex ops are combined, they describe a single sequence. If you need to spec a nested sequential collection, you must use an explicit call to spec to start a new nested regex context.#2020-11-0410:48borkdudes/alt returns a regex op, s/or is for combining predicates#2020-11-0410:48borkdudeso s/or is not a regex op.#2020-11-0410:49Jim Newtonhave you seen this paper? https://www.lrde.epita.fr/dload/papers/newton.18.meta.pdf#2020-11-0410:50borkdudeI haven't#2020-11-0410:51Jim Newtonand this one, where I introduced the derivate for implementing something quite similar to spec in common lisp https://www.lrde.epita.fr/dload/papers/newton.16.els.pdf#2020-11-0410:52Jim Newtonthe system I devised (in 2016) and after, is called RTE (regular type expressions).#2020-11-0410:53borkdudecool!#2020-11-0410:53Jim NewtonWhat i'm trying to do now is figure how similar or different it is from spec at a theoretical leve.#2020-11-0410:53Jim NewtonSo I'm trying to see if I can "compile" a spec into an RTE.#2020-11-0410:54Jim Newtonif so, then an RTE can me validated in linear time with no backtracking.#2020-11-0410:54Jim Newtonas I understand it, a spec takes exponential time worst case, so users avoid worst cases.#2020-11-0410:55Jim Newtonhowever, since I have found no academic papers on spec, and I am not an expert of spec, I'm just guessing. Maybe I'm completely wrong about my supposition#2020-11-0410:55borkdude@jimka.issy I'm interested in that for clj-kondo as well. Clj-kondo has a tiny type system in which I try to avoid backtracking of type checking arguments. Currently by just not allowing sophisticated constructs :)#2020-11-0410:55borkdude(https://github.com/borkdude/clj-kondo/blob/master/doc/types.md)#2020-11-0410:55Jim Newtoninteresting*#2020-11-0410:56Jim NewtonI submitted a paper recently to the Dynamic Languages Symposium where I introduced a Simple Type System for use in dynamic languages. The paper was rejected. One of the reasons for rejection was the revewers didn't see practical applications.#2020-11-0410:57Jim Newtona simple type system allows you to reason about types (where type is defined as any set of values) . Furthermore intersections, unions, and complements of types are also types.#2020-11-0410:58Jim NewtonIf your type logic can conclude that a type is empty, then you've usually found a bug in the user's code.#2020-11-0410:58borkdudeIt sounds like it could be very useful for clojure. Are you aware of the work done in core.typed?#2020-11-0410:58Jim NewtonThats Ambrose?#2020-11-0410:58borkdudeyes#2020-11-0410:59Jim NewtonHe's looked at my work and is interested in helping out. but everyone is busy.#2020-11-0410:59Jim NewtonI'd also love to have a student researcher interested in working with clojure.#2020-11-0410:59borkdudeIt looks like your work could help improve clj-kondo's type system since linear checking is necessary for performance#2020-11-0411:00Jim Newtonwhat's clj-kondo written in? is it written in clojure?#2020-11-0411:00borkdudeyes#2020-11-0411:01Jim Newtonare you more of a hacker, or more of an academic?#2020-11-0411:01Jim Newtonor more of an engineer? I don't mean that in any way insulting. sorry if it sounds as such....#2020-11-0411:01borkdudehaha. I do have a CS master degree, but not a Phd in type systems ;)#2020-11-0411:02borkdudeclj-kondo focuses on usefulness though, whereas something like core.typed is maybe more an academic project#2020-11-0411:02borkdudeI don't care about publishing papers, I want the tool to help me#2020-11-0411:04Jim NewtonI'm not sure why my next best step should be. For the moment I have an implementation of genus, (the simple type system in clojure) and RTE (regular type expressions) which allows genus to represent sets of sequences which match regular expressions in terms of types. Genus claims to be extensible. For the moment my experimental task is see if I can indeed extend genus (in user space) to incorporate the spec type. I.e., given a spec, the type will be the set of all objects which validate the spec.#2020-11-0411:05Jim Newtonspec types will be treated as opaque types such as arbitrary predicate. except in the case where I can compile a spec into more fundamental types in which case the system can reason about them.#2020-11-0411:05borkdudeAnother tool I made recently also uses clojure.spec for searching code: https://github.com/borkdude/grasp This could potentially made faster Another project which tries to use spec at compile time: https://github.com/arohner/spectrum It's experimental in nature.#2020-11-0411:06Jim Newtonfor example it will be able to decide whether the set of objects satisfying spec1 and also spec2 is empty. or decide whether two given specs are equivelent.#2020-11-0411:07Jim Newtonin order for RTE to represent the regular type expression as a deterministic finite automaton, it needs to be able to determine whether two given types are disjoint. This is currently not possible with spec in the most general case, but could be possible in many particular cases.#2020-11-0411:08Jim NewtonAs I understand spec's internal data structure uses non-deterministic finite automata, thus the exponential behavior in the worst case.#2020-11-0411:09Jim NewtonANYWAY, maybe now you understand a bit better the reason for my strange beginner questions about the semantics of spec ???#2020-11-0411:09borkdudeyes, thanks for explaining!#2020-11-0411:10borkdudeThe authors of spec usually emphasise that spec is not a replacement for a type system, but a runtime validation system.#2020-11-0411:10Jim Newtonin my opinion there is no difference#2020-11-0411:10borkdudeWhat might be of interest, I have a collection of specs for core functions here: https://github.com/borkdude/speculative#2020-11-0411:11borkdudeAs in, spec could be expressed as a case of dependent typing?#2020-11-0411:12Jim Newtonwhat is speculative? I didn't immediately understand by reading the intro.#2020-11-0411:13borkdude> a collection of specs for core functions#2020-11-0411:13Jim NewtonFor me a type is a set of values. And a type system allows programmers to designate and reason about certain types.#2020-11-0411:13Jim Newtonthat's what spec does.#2020-11-0411:14borkdudee.g. a spec for clojure.core/cons: https://github.com/borkdude/speculative/blob/4e773794a4065a84bdadd997516e52c76ab51b1f/src/speculative/core.cljc#L17#2020-11-0411:14borkdudeyes, if you define it like that, it makes sense. but this set of values can often not be known at compile time, which makes the difference between static compile time checking#2020-11-0411:15Jim Newtonwhat is s/fdef ?#2020-11-0411:16borkdudeclojure.spec supports defining specs for checking function arguments and return values using fdef#2020-11-0411:17Jim Newtonyes many times you can indeed know a lot about a type at compile time. in those cases, compilers like the SBCL common lisp compiler can create more efficient code, or tell the user about errors or unreachable code.#2020-11-0411:18Jim Newtonso an fdef decribes what a valid function call site looks like?#2020-11-0411:18borkdudeyes#2020-11-0411:19Jim NewtonI can see how the predicate based assumption of spec, could be a bottle neck for a code analyzer like kondo.#2020-11-0411:20borkdudeI initially used spec in clj-kondo, but I found the performance not good enough#2020-11-0411:20Jim Newtonone think that I've noticed is that many many many of these predicates can be rewritten as a simple type check. And I have code which automates this.#2020-11-0411:21Jim Newtonahhh, so you have abandoned that piste now?#2020-11-0411:22borkdudethe format now uses something more explicit: you provide the seq of types per arity. so you don't have to do any backtracking to to match a seq based on some s/alt expression. which makes more sense to me considering the structure of multi-arity functions.#2020-11-0411:22borkdudeand a seq of types is usually a fixed number of things or a fixed number of thing + a variable amount of things of the same type#2020-11-0411:22borkdudeI think this matches 99% of cases how people call functions in clojure#2020-11-0411:23Jim Newtonfor example
(gns/canonicalize-type '(satisfies int?))
  ==> (or Long Integer Short Byte)
(or ...) is a union type. when intersected and unioned with other types, it can be arranged so that redundant checks are eliminated.
#2020-11-0411:23borkdudeclj-kondo also supports union types, it's just #{:int :string}#2020-11-0411:24borkdudeit also tries to infer returns types and threads types through let expressions#2020-11-0411:24borkdudeExample:#2020-11-0411:25Jim Newtonso it has type inferencing?#2020-11-0411:25borkdudeIt has inferencing yes, but limited to the things it knows, else it's any? and it won't complain#2020-11-0411:26Jim NewtonI wrote something pretty similar some years ago for SKILL++ which was a proprietary lisp used by the company I worked for for many years.#2020-11-0411:28Jim Newtonin my system, called "loathing" it would see that foo returns type string (because of the definition), and the call site inc expects an integer. So the type at that point is (and integer string) which is the empty-type.#2020-11-0411:29borkdudeclj-kondo does something similar. but it just has a simple graph of things that are reachable or not: you can never go from string to int, so this is an error#2020-11-0411:30borkdudebut you can go from any to int, or any to string, so this is not an error#2020-11-0411:30Jim Newtonhowever, if inc had input type (or integer string), then the intersection would be (and string (or string integer)) = string != empty-set#2020-11-0411:30borkdudeyep, same thing in kondo#2020-11-0411:31borkdudee.g.: this will not give a warning:
(defn foo [x]
  (if (int? x)
    x
    (str x)))

(inc (foo 1))
#2020-11-0411:32Jim NewtonYou mentioned that you are not interested in publications. But I need to make publications. It's part of my job.#2020-11-0411:32borkdudeWell, I meant that this is not the goal of clj-kondo.#2020-11-0411:32Jim NewtonI need to get several publications out of this work.#2020-11-0411:32borkdudeIt's not the product of academic research#2020-11-0411:32borkdudeI have nothing against making publications#2020-11-0411:33borkdudeI do have my name on one publication ;)#2020-11-0411:33Jim Newtonand I think there are several interesting results to publish. But i'm a very poor publicist. I don't do well at convincing people of the interest or novelty of my work.#2020-11-0411:34borkdudeIf you could get practical benefit out of your work by e.g. improving clj-kondo or a similar tool, I'm sure it will convince people#2020-11-0411:35borkdude@jimka.issy I'm sharing this for fun, but it's also related: https://borkdude.github.io/re-find.web/ Find functions based on example in and outputs#2020-11-0411:35borkdudeThis uses the speculative specs to search for matches#2020-11-0411:37Jim Newtonthe clojure community is friendly for the most part. although I do see some snide comments now and then.#2020-11-0411:37Jim NewtonSo it is usally pretty easy to get questions answered.#2020-11-0411:37Jim NewtonI would love to have some spec examples for test cases.#2020-11-0411:37borkdudeI would say the clojure community is better in that respect than some others#2020-11-0411:38Jim NewtonCould I solicit you to give me some spec examples, simple and complex. some which are trivial, and some which are very slow to validate?#2020-11-0411:38Jim NewtonFor the moment I'm just worried about sequences, not functions or maps or data structures. My code is not advanced enough to handle those.#2020-11-0411:39Jim NewtonI want to try to compile these to RTE and see if I can come up with some cases where RTE is faster.#2020-11-0411:39Jim Newtonif RTE is never faster, then i'm in trouble#2020-11-0411:39borkdudeYou could take a look at speculative, which has more than hundred specs#2020-11-0411:40borkdudeI don't have a particular example of a slow spec, but I guess you can fabricate one based one which does backtracking#2020-11-0411:41Jim Newtonwhen analyzing code statically, i suppose you are usually looking at nested sequences, not really at maps and such.#2020-11-0411:41borkdudeThis tool: https://github.com/borkdude/grasp searches through code. It applies the spec at every sub-tree of an s-expression#2020-11-0411:42borkdudeSo performance improvements would be really noticeable there#2020-11-0411:42borkdudeI have to go now. speak to you later#2020-11-0411:45Jim Newtonok, thanks. talk later#2020-11-0411:45Jim NewtonBTW i filed an issue for kondo, was it helpful?#2020-11-0411:48borkdudeYes, thank you. I'm not sure if clj-kondo will be able to support this, as it sees one file as a standalone unit and in-ns doesn't really match with this model. So either using in-ns with :refer on the original namespace requires you to help clj-kondo with some annotations or config, or clj-kondo has to make non-trivial changes.#2020-11-0411:48borkdudeI will think about this for a while#2020-11-0411:49Jim Newtongocha#2020-11-0413:35Jim NewtonI don't think s/& is a regular expression operation. It doesn't seem regular. as far as I can tell, it requires the implementation to remember the sequence being tested. a regular expression demands that every decision be based only one the current object of the sequence, and a state. whereas only a finite number of states are allowed. In this case if there are N states, you cannot test a sequence of length N+1.#2020-11-0410:31Jim NewtonWait. I think I'm confused. Let me work out a more consistent example. And come back to you in 5 minutes. OK?#2020-11-0410:31borkdudeok#2020-11-0410:35Jim Newtontaking this conversation to a thread. I hope everyone follows.#2020-11-0411:47Jim Newtonanother spec question. Are all spec definitions global? If I'm writing test cases, I'd love to introduce local definitions that don't pollute the global space.#2020-11-0411:54borkdude@jimka.issy Yes, spec has one global registry (compare spec keywords to RDF uris for example). Malli (#malli) which aims to be an alternative schema library has local registries as well.#2020-11-0411:55Jim NewtonI suppose for test cases I can use ::xyzzy to keep the symbols in my local namespace.#2020-11-0411:59Jim NewtonDo I understand correctly that there is no intersection equivalent of s/alt for regex semantics. I.e., s/or is to s/alt as s/and is to what?#2020-11-0411:59borkdudes/&#2020-11-0412:05Jim Newtonahh, it's not included in https://clojure.org/guides/spec#_sequences, at least not in the table with the others#2020-11-0412:08Jim Newtonsyntactically does s/& work like s/alt and s/cat in that I need to tag the components? or like s/and where no tags are used?#2020-11-0412:09borkdude@jimka.issy In the guide, search for: > Spec also defines one additional regex operator, &, which takes a regex operator and constrains it with one or more additional predicates. to see an example#2020-11-0412:19Jim Newtonahh so s/& is not really analogous to s/alt it is yet a third syntax and semantic.#2020-11-0412:19borkdudeI thought you were asking analogous to s/and. > I.e.,Ā s/orĀ is toĀ s/altĀ asĀ s/andĀ is to what?#2020-11-0412:25borkdudes/alt and s/& are supposed to be used to build regexes, s/and and s/or are just general ways of combining predicates#2020-11-0412:26borkdudebut sometimes they do the same thing#2020-11-0412:26Jim NewtonIt looks (from the documentation) that s/& is not analgous to s/and in the same way that s/alt is analogous to s/or. if they were analogous I'd expect s/& to work like this.
(s/* (s/alt :x (s/cat :a neg? :b even?) :y (s/cat :c odd? :d pos?)))
(s/* (s/& (s/cat :a neg? :b even?) (s/cat :c odd? :d pos)))
but instead it works like this
(s/* (s/& (s/cat :a neg? :b even?) #(even? (count %))))
#2020-11-0412:27borkdudeSee the difference here:
user=> (s/conform (s/alt :foo int? :bar string?) [1])
[:foo 1]

user=> (s/conform (s/or :foo int? :bar string?) [1])
:clojure.spec.alpha/invalid
user=> (s/conform (s/or :foo int? :bar string?) 1)
[:foo 1]
#2020-11-0412:29Jim Newtonyes s/or matches an object, and s/alt matches a sequence of objects#2020-11-0412:35Jim NewtonSo why doesn't this return true?
(s/valid? (s/* (s/&  (s/cat :a neg? :b even?)  
                     (s/cat :c odd? :d pos?))) 
          [-3 4 -5 2])
it is both a sequence of neg even occurring 0 or more times, and also a sequence of odd pos occuring 0 or more times
#2020-11-0412:36Jim NewtonIf I replace s/& with s/alt (and insert the required keys), it works as expected.
(s/valid? (s/* (s/alt :x  (s/cat :a neg? :b even?)  
                      :y  (s/cat :c odd? :d pos?))) 
          [-3 4 -5 2])
#2020-11-0412:49Jim NewtonIt looks to me (on first and second reading of the specification) that (s/& pattern predicate) means that the subsequence which matches the pattern must as a sequence match the predicate. So the following
(s/cat :a (s/& (s/* string?) #(even? (count %)))
       :b (s/& (s/+ int?)    #(odd? (count %))))
matches a sequence which begins with an even number of strings and ends with an odd number of integers. Is my interpretation correct? I don't think these are really regular expressions any more. I have to think about it, but my suspicion is that this cannot be solved without using a stack.
#2020-11-0414:00Jim NewtonHow can I recognize an object such as the one returned by
(s/* (s/alt :x  (s/cat :a neg? :b even?)  
            :y  (s/cat :c odd? :d pos?)))
s/spec? returns false . type returns clojure.lang.PersistentArrayMap . s/get-spec returns nil.
#2020-11-0414:13Alex Miller (Clojure team)isnā€™t there a regex?#2020-11-0414:15Jim Newtonso regex? returns non-nil. is that one I should use?#2020-11-0414:16Alex Miller (Clojure team)Yeah#2020-11-0414:18Jim Newtonseems to be any object for which ::op returns boolean true.#2020-11-0414:19Jim Newtonwhere ::op is :clojure-spec-alpha/op#2020-11-0414:19Jim Newtonfair enough#2020-11-0517:02Jim NewtonI just discovered that the following doesnt work.
(ns some-ns
  (:require [clojure.spec.alpha :as s]))

(defn foo []
  (= (resolve 's/*) (resolve 'clojure.spec.alpha/*)))
because when foo gets called by the application, the name space might not have required [clojure.spec.alpha :as s] Instead, I did the following and it seems to work.
(ns some-ns
  (:require [clojure.spec.alpha :as s]))

(defn foo []
  (binding [*ns* (find-ns 'some-ns)]
    (= (resolve 's/*) (resolve 'clojure.spec.alpha/*))))
#2020-11-0517:02Jim NewtonI just discovered that the following doesnt work.
(ns some-ns
  (:require [clojure.spec.alpha :as s]))

(defn foo []
  (= (resolve 's/*) (resolve 'clojure.spec.alpha/*)))
because when foo gets called by the application, the name space might not have required [clojure.spec.alpha :as s] Instead, I did the following and it seems to work.
(ns some-ns
  (:require [clojure.spec.alpha :as s]))

(defn foo []
  (binding [*ns* (find-ns 'some-ns)]
    (= (resolve 's/*) (resolve 'clojure.spec.alpha/*))))
#2020-11-0517:05andy.fingerhutYou get different results than this in a REPL?
user=> (require '[clojure.spec.alpha :as s])
nil
user=> (resolve 's/*)
#'clojure.spec.alpha/*
user=> (resolve 'clojure.spec.alpha/*)
#'clojure.spec.alpha/*
user=> (= (resolve 's/*) (resolve 'clojure.spec.alpha/*))
true
#2020-11-0517:06andy.fingerhutNote that you have no * after clojure.spec.alpha/ I do not know if that is a typo in chat, or reflects your actual code.#2020-11-0517:06Jim Newtonyes a copy/paste error. corrected, should be /* everywhere#2020-11-0517:07andy.fingerhutNow I see clojure.spec.alpha. Did you intend to use clojure.spec.alpha/* instead?#2020-11-0517:08andy.fingerhutdefn foo is in the namespace some-ns ? If so, how does some part of your code call foo without first require'ing some-ns ?#2020-11-0517:09Jim Newtonimportant observation is that the behavior of resolve depends on the value of *ns* when foo is called, not when foo is defined.#2020-11-0517:09Jim Newtonthe caller of foo has required some-ns, but he might not have aliased closure.spec.alpha to s#2020-11-0517:10andy.fingerhutGot it. Yes, that is true, and resolution of aliases like s definitely depends upon the namespace in which they are resolved.#2020-11-0517:11Jim Newtoni found out because I ran my tests from the repl where I had required spec and aliased to s. But when I tested from lien test at the comand line, it ran the tests with the user namespace, which I never tested in.#2020-11-0517:11andy.fingerhutAlias s could be not an alias in some namespace, be used for clojure.spec.alpha in another namespace, and be used for clojure.set in yet another namespace. Not recommended for developer sanity to use the same alias for different namespaces, but Clojure allows it.#2020-11-0517:12Jim Newtonbesides, when I develop the code, I use some convention. But I don't impose that convention on my customers who might have very different programming styles. or they theoretically might even be calling my code from java or some such#2020-11-0517:12andy.fingerhutThe doc string for resolve gives a pretty strong hint about this dependence upon the current value of *ns*#2020-11-0517:14Jim NewtonI think all calls to resolve or suspect within code.. I'm thinking of refactoring every call in my program and replacing with ns-resolve so I'll have to think about which ns to use. Then test from repl and also from lein command line as a general pratice#2020-11-0517:14Jim Newtonit's on my to-do list.#2020-11-0517:15andy.fingerhutUsing ns-resolve instead of resolve sounds like a good idea, if you want to force yourself to be explicit about in which namespace the resolving is performed.#2020-11-0517:17Jim Newtonof course nothing prohibits me from using (ns-resolve *ns* foo) if that's the appropriate one to use.#2020-11-0720:39borkdudeWhat was the take on spec and coercions again - spec isn't meant to do coercions of values (during conformation), rather it only validates or conforms values, right? Then what is the opinion on a library like this one? https://github.com/exoscale/coax#2020-11-0721:01seancorfieldMy reading of comments by Alex, at least, is that Specs should not do coercions, but libraries that infer coercions from Specs, and do those coercions "outside" Spec are a good way to do stuff.#2020-11-0721:03seancorfieldOur "web specs" library does the coercions internally in the Specs ("bad") but I've been looking at moving to spec-coerce instead (or maybe coax) and deprecating our library @borkdude#2020-11-0721:08borkdudeNice, thanks. Coax looks pretty cool#2020-11-0806:10mpenetcoax is used heavily in production, so far so good :) we fixed a number of bugs from the original spec coerce fork and made it more flexible and performant. I am not sure I would still call coax a fork, the source is now totally different.#2020-11-0806:17mpenetspec-coerce outlined some good ideas but it felt a bit like a POC, I am not sure it is/was used seriously.#2020-11-0914:53Jim Newtonhow can I recognize an object such as the one returned from (s/or :1 int? :2 number?) . I see that s/regex? , s/get-spec, and s/spec? all return nil. I notice however that the class of the object has (type (s/or :1 int? :2 number?)) in its ancestors?
(ancestors (type (s/or :1 int? :2 number?)))
  ==> #{clojure.spec.alpha.Spec clojure.lang.IObj clojure.lang.IMeta
  java.lang.Object clojure.spec.alpha.Specize}
So I can use the following ugly idiom:
(instance? clojure.spec.alpha.Spec (s/or :1 int? :2 number?))
isn't there a better way?
#2020-11-0914:56Jim NewtonNext question, how can I extract int? and number? from the object? I see that (keys (s/or :1 int? :2 number?)) throws an Exception.
(keys (s/or :1 int? :2 number?))
Execution error (IllegalArgumentException) at clojure-rte.genus-spec-test/eval17912 (form-init5322407590801644377.clj:30310).
Don't know how to create ISeq from: clojure.spec.alpha$or_spec_impl$reify__2118
I see that (s/describe (s/or :1 int? :2 number?)) returns the list (or :1 int? :2 number?) . However, to recognize the object as one which has a describe method.
#2020-11-0915:07Alex Miller (Clojure team)it's important to be clear in talking about the two worlds of spec - spec forms and spec objects. (s/or :1 int? :2 number?) is a spec form, which when evaluated, returns a spec object (something that satisfies the spec protocol). Trying to nail down concrete types for the latter part is going to be a bad time - the important thing is the protocol which is inherently polymorphic.#2020-11-0915:08Jim Newtonso how do I know whether I have an object which satisfies the spec protocol?#2020-11-0915:09Alex Miller (Clojure team)spec 1 does not have great answers for extracting information from either spec objects (which are opaque) or spec forms. One path to this is to write specs for spec forms and then use s/conform to "parse" the forms. A significant stab at this exists in https://clojure.atlassian.net/browse/CLJ-2112 but I think the future direction is really making a more data-centric representation which is one thing we are doing in spec 2 (but that's still very much a work in progress)#2020-11-0915:11Alex Miller (Clojure team)spec? is the predicate for that#2020-11-0915:11Jim Newtonyes but spec? returns nil.#2020-11-0915:11Alex Miller (Clojure team)for what?#2020-11-0915:12Alex Miller (Clojure team)
user=> (s/spec? (s/or :1 int? :2 number?))
#object[clojure.spec.alpha$or_spec_impl$reify__2118 0x51ec2df1 "
#2020-11-0915:13Jim Newtonaaaaaaaaahhhhhhhhh!!!!!! yikes. s/spec? returns non-nil. That's EXCELLENT I have a local function named gs/spec? which just asks whether it is a sequence whose first element is spec. spec? was my first guess. just was shadowed by a local function. Cool. thanks.#2020-11-0915:13Alex Miller (Clojure team)yeah, your predicate is asking a symbolic spec form question#2020-11-0915:13Jim Newtonand I just typed spec? at the REPL.#2020-11-0915:13Jim NewtonREPL issue.#2020-11-0915:17Jim Newtonis there a way to undef a function. Sometimes I rename a function while debugging, and the old function (of course) is still defined in vm, and if my code accidentally calls it because I didn't find and change all references, that introduces bugs which are hard to find.#2020-11-0915:26borkdude@jimka.issy You can use (s/fdef foo/bar nil)#2020-11-0915:26borkdude(I assume, I know (s/def ::foo nil) at least works)#2020-11-0915:29Alex Miller (Clojure team)yes, s/def to nil#2020-11-0915:29Jim Newtonwhat is s/ ?#2020-11-0915:29Alex Miller (Clojure team)clojure.spec.alpha#2020-11-0915:29Jim Newtonyou mean use spec to redefine a function?#2020-11-0915:29Alex Miller (Clojure team)are you asking about function specs or functions?#2020-11-0915:30Alex Miller (Clojure team)I think @borkdude and I assumed you meant specs since we're in the spec channel here#2020-11-0915:30Jim Newtonfuntions. Sorry if I mistyped before?#2020-11-0915:30Jim Newtonsorry, its probably my fault.#2020-11-0915:30Alex Miller (Clojure team)you can use ns-unmap#2020-11-0915:30Jim Newton(def name nil) works of course.#2020-11-0915:31Alex Miller (Clojure team)^^ that doesn't unmap, just redefines#2020-11-0915:31Jim Newtonahhh ns-unmap. thats the cleaner way.#2020-11-0915:31borkdude@jimka.issy
user=> (def x 1)
#'user/x
user=> (ns-unmap *ns* 'x)
nil
user=> x
Syntax error compiling at (REPL:0:0).
Unable to resolve symbol: x in this context
user=> user/x
Syntax error compiling at (REPL:0:0).
No such var: user/x
#2020-11-0915:32borkdudeI recently discovered ns-unmap also works for imported classes. I need to fix a bug in sci which doesn't support that yet :/ ;)#2020-11-0915:32Jim NewtonI have several functions which are memoized. it speeds up my program 1000 fold. But when debugging, it can be a source of errors, because I forget that the function might not really be called.#2020-11-0915:32Alex Miller (Clojure team)the double-edged blade of memoization :)#2020-11-0915:33Jim Newtonindeed. So sometimes I really want to set-the-d*mn-function to undefined#2020-11-0915:33Jim Newtonto make sure it's not a problem of memoization#2020-11-0915:33borkdude@jimka.issy A solution to that problem might be to call the original function foo* and the memoized one foo#2020-11-0915:34Jim NewtonHere's the macro I'm using.
(defmacro defn-memoized
  [[public-name internal-name] docstring & body]
  (assert (string? docstring))
  `(let []
     (declare ~public-name) ;; so that the internal function can call the public function if necessary
     (defn ~internal-name 
This allows my to locally rebind the name such as
(binding [my-function (memoize -internal-name)]
   ...)
which I often do in test suits
#2020-11-0915:35borkdudemakes sense#2020-11-0915:35borkdudefor tests you can also use with-redefs which doesn't require your vars to be dynamic#2020-11-0915:36Jim Newtonnever heard of with-redefs. what's that.#2020-11-0915:36borkdude(doc with-redefs)#2020-11-0915:38Jim Newtonindeed. so that just saves needing to declare them as dynamic? or does it do something else?#2020-11-0915:38borkdudedynamic bindings aren't visible to all threads, with-redefs changes the root binding of vars temporarily#2020-11-0915:39Alex Miller (Clojure team)(note that there are issues using with-redefs with concurrency, including future etc)#2020-11-0915:40Jim Newton@U064X3EF3 are these the same issues with any dynamic variable?#2020-11-0915:40borkdudeyeah, when running tests concurrently this may bite you#2020-11-0915:41Jim NewtonAren't dynamic variables thread-local in clojure?#2020-11-0915:41borkdudeyes, but with-redefs doesn't use thread-local, it's just a global mutation that gets restored afterwards#2020-11-0915:42Jim Newtonah ha.#2020-11-0915:42borkdudePersonally I don't have any test suites that run into problems with this, but then again, I hardly use with-redefs at all#2020-11-0915:43borkdudeThere could be a performance penalty to marking all your vars dynamic, but not if they haven't been dynamically re-bound yet since there's a pretty efficient check for that happy path#2020-11-0916:00Alex Miller (Clojure team)we regularly see people run into failing test suites when using with-redefs#2020-11-0916:00Alex Miller (Clojure team)because they don't understand what it does, which is why I mention this#2020-11-0916:01borkdudešŸ‘#2020-11-0916:01Jim Newton:+1::skin-tone-2:#2020-11-0915:52Jim Newtonwhy doe s/describe return lists using and and or rather than clojure.spec.alpha/and and clojure.spec.alpha/or ? that makes programmatic usage more difficult.#2020-11-0915:53borkdudes/describe isn't intended for programmatic usage. use s/form rather than that#2020-11-0915:54Jim NewtonExcellent. that does the trick. much more program friendly#2020-11-0916:01Alex Miller (Clojure team)s/describe is primarily useful for printing shorter specs for humans (used in doc for example)#2020-11-0916:01Jim Newtonmakes sense.#2020-11-0920:48amorokhI just gotta ask, is there a way to create a shorthand alias for a namespace to use in fully-qualified keys without actually creating that namespace (using the ns form)?#2020-11-0920:51borkdude@amorokh I think this is a pretty common pattern:
user=> (require '[clojure.spec.alpha :as s])
nil
user=> (alias 'foo (create-ns 'foo))
nil
user=> (s/def ::foo/bar int?)
:foo/bar
Rumor has it that core team is working on making this easier
#2020-11-0920:53amorokh@borkdude thanks, not sure I want to do that but at least I know about that possibility#2020-11-1016:35Jim NewtonI'm getting an error from spec that I don't understand. Maybe someone can help me. The following example, which I've based on an example in https://clojure.org/guides/spec, works fine, and valid? returns true.
(s/valid? (s/keys :req [::first-name ::last-name ::email]
                  :opt [::phone])
          {::first-name "Bugs"
           ::last-name "Bunny"
           ::email "
However, when I've programmatically generated the following list
(let [sss '(clojure.spec.alpha/keys
            :req [:clojure-rte.genus-spec-test/first-name
                  :clojure-rte.genus-spec-test/last-name
                  :clojure-rte.genus-spec-test/email]
            :opt [:clojure-rte.genus-spec-test/phone])]
  (s/valid? sss
          {::first-name "Bugs"
           ::last-name "Bunny"
           ::email "
I get an uninformative exception
Execution error (ClassCastException) at (REPL:1).
null
is there a way to all s/valid? with a programmatically generated spec?
#2020-11-1016:37borkdude@jimka.issy An s-expression is not yet a spec. I would assume most people would write a macro for creating specs programmatically#2020-11-1016:37Alex Miller (Clojure team)Yes, but you need to understand more about how spec works#2020-11-1016:37Alex Miller (Clojure team)valid? works on spec objects#2020-11-1016:38Alex Miller (Clojure team)Spec forms need to be evaluated into spec objects first#2020-11-1016:38Alex Miller (Clojure team)In the case above, you could do that with eval#2020-11-1016:39borkdudeor generate the form with a macro#2020-11-1016:39Alex Miller (Clojure team)Even so, youā€™re still leaning on eval#2020-11-1016:40borkdudeyep#2020-11-1016:41borkdudeMade some fun macros the other day:
(def kws (map keyword (repeatedly gensym)))

(defmacro cat [& preds]
  `(clojure.spec.alpha/cat 
This allows you to write (g/cat int? string?), when you're not interested in the conformed value
#2020-11-1016:43Alex Miller (Clojure team)Spec 2 has a lot more tools for all this stuff#2020-11-1016:43borkdudedefop right?#2020-11-1016:43Alex Miller (Clojure team)Thatā€™s one#2020-11-1016:43borkdudeany docs I could read yet?#2020-11-1016:43Alex Miller (Clojure team)The wiki pages if you havenā€™t seen those#2020-11-1016:44Alex Miller (Clojure team)But not everything is in there and itā€™s missing a lot of the internal design #2020-11-1016:44borkdudeI'll just wait, no hurry#2020-11-1016:44Alex Miller (Clojure team)There is now an intermediate map data form now that you can work on directly#2020-11-1016:45Alex Miller (Clojure team)But will almost certainly change#2020-11-1016:45borkdudeWorking with maps is a lot easier than writing macros (since macros beget more macros)#2020-11-1016:46Alex Miller (Clojure team)Some day rich or I will do a talk about the internals - there are several interesting aspects of it#2020-11-1016:47borkdudeLooking forward#2020-11-1016:52Jim Newtonmacros wont help in this case. the origin of my spec is not sitting in an evaluation position.#2020-11-1017:04Alex Miller (Clojure team)thus, eval#2020-11-1017:04Alex Miller (Clojure team)evaluating a spec form gives you a spec object#2020-11-1117:10Jakub HolĆ½ (HolyJak)What is the state of / plan for Spec 2? Or is there I can check for the answers? šŸ™#2020-11-1117:23seancorfield@holyjak It's still in the design phase -- Rich is rethinking the whole function spec thing, as far as I can tell from comments Alex has made.#2020-11-1117:23seancorfieldSo it's still very alpha and subject to (a lot of) change.
#2020-11-1117:24Jakub HolĆ½ (HolyJak)thank you!#2020-11-1117:24seancorfieldI'm chomping at the bit to use it -- for a while I had a branch of our code at work tracking Spec 2 but it just had too much churn and too many bugs to keep up with it, so I abandoned that branch after several months.#2020-11-1117:24seancorfieldIt definitely has some nice usability improvements in it, and the whole schema/`select` thing is much better than keys#2020-11-1118:20Alex Miller (Clojure team)sorry, don't have any concrete plan#2020-11-1118:21Alex Miller (Clojure team)it continues to get slices of attention is about all I can tell you :)#2020-11-1118:42borkdudeWithout encouraging more delay, I must say I kind of envy the patience that Rich has before releasing anything ;)#2020-11-1220:39Ronny LiHi, does anyone have advice for how to read the following error message from plumatic/schema? My understanding is that the arity doesn't conform to spec but where did the error occur? (`(demo/data.clj:77:1)` is simply where I defined the function)
Error refreshing environment: Syntax error macroexpanding clojure.core/defn at (demo/data.clj:77:1).
Call to clojure.core/defn did not conform to spec. clojure.lang.ExceptionInfo: Call to clojure.core/defn did not conform to spec. #:clojure.spec.alpha{:problems ({:path [:fn-tail :arity-1 :params], :pred clojure.core/vector?, :val :-, :via [:clojure.core.specs.alpha/defn-args :clojure.core.specs.alpha/params+body :clojure.core.specs.alpha/param-list :clojure.core.specs.alpha/param-list], :in [1]} {:path [:fn-tail :arity-n :bodies], :pred (clojure.core/fn [%] (clojure.core/or (clojure.core/nil? %) (clojure.core/sequential? %))), :val :-, :via [:clojure.core.specs.alpha/defn-args :clojure.core.specs.alpha/params+body :clojure.core.specs.alpha/params+body], :in [1]}), :spec #object[clojure.spec.alpha$regex_spec_impl$reify__2509 0x406684fa "
#2020-11-1220:42borkdudeMaybe you could post the function itself#2020-11-1220:53seancorfieldSounds like a defn that has :- instead of an argument vector. I'm guessing you used defn (from core) with Schema syntax instead of Schema's own variant of defn? @ronny463#2020-11-1220:54Ronny Lithat's exactly what happened, thank you šŸ™‚#2020-11-1220:54seancorfield(based on :pred clojure.core/vector?, :val :- in the first line of the exception)#2020-11-1316:15borkdudeWhat is the reason spec chose a custom deftype LazyVar over the built-in delay in its implementation?#2020-11-1316:17Alex Miller (Clojure team)for which?#2020-11-1316:17borkdudeI'm trying to come up with something that would work in Clojure for this case:
(def input (LazyVar. (fn [] (line-seq (java.io.BufferedReader. (java.io.StringReader. "1\n2\n3\n")))) nil))
such that you don't have to write @input but just input so the first time it's derefed by Clojure it starts its realization. Just a naked line-seq doesn't work, since that starts reading once it's def-ed.
#2020-11-1316:17borkdude@alexmiller in the gen namespace#2020-11-1316:19borkdudeI think in Clojure I would have to write a subclass of clojure.lang.Var to make that work.#2020-11-1316:19Alex Miller (Clojure team)sorry, I don't actually see what you're talking about#2020-11-1316:20borkdudeAnd maybe I should just go with @input and not do any magic... This lead to the question: why is gen/LazyVar not just a delay but a custom data structure. Ah right.. sorry, the LazyVar is something in CLJS, not CLJ. facepalm#2020-11-1316:21borkdudehttps://github.com/clojure/clojurescript/blob/5e88d3383e0f950c4de410d3d6ee11769f3714f4/src/main/cljs/cljs/spec/gen/alpha.cljc#L15#2020-11-1316:22borkdudeI'll ask over there in #clojurescript#2020-11-1316:23Alex Miller (Clojure team)ah#2020-11-1318:39socksyI am confused about the syntax for (or) and (and) in s/keys. My understanding of it was that anything that is passed to or could be correct, and anything that passed into and has to be present. Did I misunderstand? My intuition does not work for the following example:
(s/def :foo/foo #{:foo})
(s/def :bar/foo #{:bar})
(s/def ::an-int int?)

(s/def ::baz (s/keys :req-un [(or :foo/foo
                                  (and :bar/foo
                                       ::an-int))]))

(s/valid? ::baz {:foo :foo})
;; => false
(s/valid? ::baz {:foo :bar})
;; => true
#2020-11-1318:51socksyok it looks like when you have the same naked keyword (not sure of the right terminology, keyword without the NS) then it will always take the last one defined. That's a huge bummer, since I wanted to be able to spec something like "In this case, do this, in this other case, do this + another piece of data", and the way that was being specced before was by storing everything as a tuple#2020-11-1318:52borkdude@socksy can't you use s/or for either case?#2020-11-1318:52borkdudemore verbose#2020-11-1318:53seancorfield@socksy You'll need to wrap s/keys with s/and and add your rules via a predicate -- or use s/or around s/keys as @borkdude suggests.#2020-11-1318:53socksyyes I think it might be possible. I'll go hit my head on it#2020-11-1318:53socksyi am thinking my own predicate will probably be the most readable but let's see#2020-11-1318:54borkdudemaybe spec2 has a better answer to this... although I'm not sure if s/select can solve this case#2020-11-1318:54seancorfieldAnother option would be a multi-spec I think?#2020-11-1318:57socksy
(s/def ::baz (s/or :foo-case (s/keys :req-un [:foo/foo])
                   :bar-case (s/keys :req-un [:bar/foo ::an-int])))
is not too bad actually
#2020-11-1318:59borkdudeNow I wonder how you would use this in spec2. I think you would define ::baz as the thing with all possible keys and then use s/select for either case?#2020-11-1319:02borkdude
(s/def ::baz (s/schema [:foo/foo :bar/foo ::an-int]))
(s/def ::foo-case (s/select [:foo/foo]))
(s/def ::boo-case (s/select [:bar/foo ::an-int]))
Something like this?
#2020-11-1408:25Rob Hanlonhey all, iā€™m trying to write a spec for a map using s/keys that may contain keys that are namespaced or unnamespaced:
(s/keys :opt [::foo ::bar ::baz] :opt-un [:foo ::bar ::baz])
alright, thatā€™s all well and good! but now iā€™d like to reduce the duplication (in my real-life application, the vectors have ~10 items each)
(def ks [::foo ::bar ::baz])
(s/keys :opt ks :opt-un ks)
this doesnā€™t work, because ks is a symbol within the keys macro. iā€™ve tried unquoting:
(def ks [::foo ::bar ::baz])
(s/keys :opt ~ks :opt-un ~ks)
but this doesnā€™t work eitherā€”it fails the namespace-qualification assertion within the keys macro. any pointers on how to reuse these keys? thanks šŸ™‚
#2020-11-1409:01borkdudeThat doesn't work indeed. You will need to write macros to accomplish this because s/keys itself is a macro#2020-11-1414:53dvingo@robhanlon I came up with this helper for composing keys:
(defmacro def-domain-map
  "Return s/keys :req for fields supports passing in symbols for keys"
  ([spec required]
   (let [req (eval required)]
     `(s/def ~spec (s/keys :req ~req))))
  ([spec required opt]
   (let [req        (eval required)
         opt        (eval opt)
         global-opt (eval global-keys)]
     `(s/def ~spec (s/keys :req ~req :opt ~(into opt global-opt))))))
It works from clj and cljs:
(>def :work-log/id fuc/id?)
(>def :work-log/description string?)
(>def :work-log/begin tu/date-time?)
(>def :work-log/end tu/date-time?)

(def required-work-log-keys
  [:work-log/id :work-log/description :work-log/begin :work-log/end])
(def optional-work-log-keys [])

(su/def-domain-map ::work-log required-work-log-keys optional-work-log-keys)
#2020-11-1414:55dvingoI wanted the required and optional keys to be reused in different parts of the app without needing to maintain two lists of these keys#2020-11-1415:29Alex Miller (Clojure team)at some point you have to ask yourself - is this easier than just saying the keys twice?#2020-11-1420:04Rob HanlonI think I have my answer hereā€”Iā€™ll just repeat the keys. Thank you šŸ™ #2020-11-1815:11Dmytro BuninI stumbled upon this behavior, which I found a bit weird.
(s/def :sequence-list/eid number?)
(s/valid?
 (s/keys :opt []) ;; ā‡  no reference to :sequence-list/eid
 (merge #:entity {:label "foo"}
        #:sequence-list {:eid "sample"}))
=> false
does the s/keys look at global registry in some way?
#2020-11-1815:11Alex Miller (Clojure team)yes#2020-11-1815:11Alex Miller (Clojure team)s/keys will check all registered keys in the map#2020-11-1815:12Alex Miller (Clojure team)so even (s/keys) is a spec that does things#2020-11-1815:13Dmytro Bunininteresting, thanks šŸ™‚#2020-11-1815:13Alex Miller (Clojure team)this is covered if you (doc s/keys) or look at the spec guide https://clojure.org/guides/spec#2020-11-1815:13Dmytro Bunin> > In addition, the values of all namespace-qualified keys will be validated > (and possibly destructured) by any registered specs. Note: there is > no support for inline value specification, by design.#2020-11-1815:13Alex Miller (Clojure team)^^#2020-11-1815:13Dmytro Buninyeah says so in docstring šŸ™‚#2020-11-1906:52lassemaattaA question concerning multi-spec : Suppose I have a spec similar to the one given in the clojure spec guide (ie. :event/event), where different events contain a :event/type key. How can I generate sample events of a particular type, for example :event/error events? Or refer to the spec of a particular event somewhere (e.g. in a s/fdef). What I've tried so far: If I replace the spec from the corresponding defmethod call into a s/def definition, I can obtain the generator for it and I get all the fields I expect but the :event/type key will be a random keyword. Another hack was to generate events from :event/event and then use s/and + predicate to only accepts events which have the correct type, but this "lacks elegance" šŸ™‚#2020-11-1907:01lassemaattaOne might argue that it reflects poor design if one has to refer to particular type like this, but still it might be nice to know if & how this is possible :thinking_face:#2020-11-1914:23Alex Miller (Clojure team)I think something like what you describe is how most people are approaching it right now#2020-11-2019:53neilyioCould anyone give me a hand with this error? Var clojure.test.check.generators/simple-type-printable does not exist, clojure.test.check.generators never requiredĀ happens every time I callĀ `(spec/explain ::my-spec my-val)`Ā in ClojureScript. I don't have instrumentation turned on. I've followed the advice atĀ https://stackoverflow.com/questions/57877004/how-to-fix-clojure-test-check-generators-never-required-when-exercising-a-funcĀ  and addedĀ `org.clojure/test.check`Ā to myĀ `deps.edn`Ā dependencies, but it hasn't fixed the problem.#2020-11-2019:55borkdudewhen you see this message, in CLJS you should require the ns yourself.#2020-11-2019:55borkdudebut it surprises me that s/explain needs test.check, that should not be the case I think#2020-11-2019:58borkdude@neil.hansen.31 which version of CLJS are you using?
{:tag :a, :attrs {:href "/cdn-cgi/l/email-protection", :class "__cf_email__", :data-cfemail "22404d504946574647626f60721012131b"}, :content ("[emailĀ protected]")}
#2020-11-2020:45neilyio@borkdude I'm on 1.10.773. I'm trying it on an empty REPL now and it's working as your example is... I guess something is going on in my Figwheel build that I'll have to figure out...#2020-11-2020:55neilyioActually, this is happening only on a specific spec.#2020-11-2020:55neilyioWill debug.#2020-11-2020:56borkdudeMaybe you are using with-gen in this spec?#2020-11-2021:25kennyCould potentially be an fspec#2020-11-2021:49neilyioYes, it's definitely something to do with fspec. It also seems to act strange in this case:
(spec/fdef foo
  :args (spec/fspec :args (spec/cat)))
#2020-11-2021:50kennyFspec uses gen testing during explain. #2020-11-2021:52neilyioWhen I instrument these functions:
(defn foo []
    nil)

(defn bar []
   nil)
I get a "spec failed... should satisfy ifn?" expound error with this: (foo bar)
#2020-11-2021:52neilyioI would expect that to Success!. Am I misunderstanding something?#2020-11-2021:55borkdude@neil.hansen.31 Why are you using fspec here?#2020-11-2021:56borkdudeI would have expected:
(spec/fdef foo
  :args (spec/cat))
#2020-11-2021:57neilyioYup, I'm starting to realize I'm doing something seriously wrong here šŸ˜›#2020-11-2021:59neilyioI made an assumption from the https://clojure.org/guides/spec#_higher_order_functions that fspec would work for :args, but it seems like it's only for :ret. I was hoping that spec could check that the arguments to my function would receive only certain arguments.#2020-11-2022:00borkdudespec can check that. it does when you instrument the function#2020-11-2022:01neilyioOh, also my example for foo was wrong up above, sorry, it should have been something like:
(defn foo [f]
    (f))
(defn bar []
   nil)
(foo bar)
#2020-11-2022:02borkdudeThen your spec may have been correct, and the error message makes sense. Foo wasn't passed an IFn#2020-11-2022:03borkdudeI think it should be something like:
(spec/fdef foo
  :args (spec/cat :f (spec/fspec :args (spec/cat))))
#2020-11-2022:05borkdudeIn Clojure:
user=> (require '[clojure.spec.alpha :as spec])
nil
user=> (spec/fdef foo
  #_=>   :args (spec/cat :f (spec/fspec :args (spec/cat))))
user/foo
user=> (defn foo [f] (f))
#'user/foo
user=> (stest/instrument 'user/foo)
[user/foo]
user=> (foo identity)
Execution error (FileNotFoundException) at user/eval35926 (form-init3141792093562271177.clj:1).
Could not locate clojure/test/check/generators__init.class, clojure/test/check/generators.clj or clojure/test/check/generators.cljc on classpath.

user=> (foo (fn []))
Execution error (FileNotFoundException) at user/eval35926 (form-init3141792093562271177.clj:1).
Could not locate clojure/test/check/generators__init.class, clojure/test/check/generators.clj or clojure/test/check/generators.cljc on classpath.
#2020-11-2022:06neilyioOh my goodness, you're right, thank you so much. The :args always have to bee a tuple don't they.#2020-11-2022:06borkdudea sequential#2020-11-2022:06borkdudebtw, you can also check for a function using fn? or ifn?.#2020-11-2022:06neilyioGot it. Funny enough, when I instrument that spec you gave me, I get the error again: Var clojure.test.check.generators/simple-type-printable does not exist, clojure.test.check.generators never required#2020-11-2022:06borkdude:args (s/cat :f ifn?)#2020-11-2022:07borkdudeyeah, it seems fspec somehow needs test.check. I've never used fspec personally. It seems to be the same in CLJ:
user=> (foo (fn []))
Execution error (FileNotFoundException) at user/eval35926 (form-init3141792093562271177.clj:1).
Could not locate clojure/test/check/generators__init.class, clojure/test/check/generators.clj or clojure/test/check/generators.cljc on classpath.
#2020-11-2022:09neilyioMy issue is that I've installed test.check, I've even required it in my namespace.#2020-11-2022:10neilyioSo not sure what's happening... For now I'll take your advice and check for ifn? in my :args instead.#2020-11-2022:10borkdudewhat does your ns form look like?#2020-11-2022:29neilyioHmm it looks like this is a Figwheel-specific problem. I've made a very minimal test file:
(ns cards.core
  (:require
   [clojure.test.check :as check]
   [clojure.spec.test.alpha :as stest]
   [clojure.spec.alpha :as spec]))


(spec/fdef foo
  :args (spec/cat :f (spec/fspec
                          :args (spec/cat))))

(defn foo [f] f)

(defn bar [] nil)

(stest/instrument)

(foo bar)
#2020-11-2022:30neilyioEvaluating the final (foo bar) with clj returns the function object.#2020-11-2022:31neilyioEvaluating (foo bar) with clj -m figwheel.main returns #object[Error Error: Var clojure.test.check.properties/for-all* does not exist, clojure.test.check.properties never required]#2020-11-2022:31neilyioSlightly different error from my original one, but I'd think the problem comes from the same place?#2020-11-2022:32borkdudeYou can try to compile this without figwheel and see if that's it. And then file a bug report with figwheel#2020-11-2022:32neilyioYup, I think it's bug report time.#2020-11-2022:33neilyioThanks for talking me through this @borkdude#2020-11-2022:33borkdude:thumbsup:#2020-11-2022:44neilyiohttps://github.com/bhauman/figwheel-main/issues/272 for all those who are lost in the future.#2020-11-2022:51borkdudeHave you also tried to compile this without Figwheel, but using regular CLJS? That's not clear from this issue#2020-11-2022:55neilyioOh geez, you're right, I get the same error running with clj -m cljs.main.#2020-11-2022:56neilyioI'm just so used to starting cljs with figwheel. Thanks for following up on that.#2020-11-2022:56neilyioI'll close the Figwheel issue, what do you think I should do with this?#2020-11-2023:00borkdude#clojurescript#2020-11-2023:00borkdudeyou could link to the figwheel issue to explain your issue#2020-11-2023:00neilyioI'll do that, I'll close the figwheel issue when someone advises. Thanks again for everything.#2020-11-2023:05borkdudeI think it would be good to close the figwheel issue and comment that it's not an issue with figwheel, else you might be wasting the maintainer's time.#2020-11-2023:06borkdudeHave you tried requiring cljs.test.check?#2020-11-2023:06borkdudeinstead of clojure.test.check#2020-11-2023:07borkdudeneh, that's not it#2020-11-2023:15neilyioClosed it up, posted in #clojurescript. Will buy Bruce Hauman beer someday, will give you full credit for helping me recover from this.#2020-11-2023:15neilyioBeer coming your way too.#2020-11-2023:15borkdudeSounds good ;) Have a nice weekend! šŸ» I'm off, getting late over here in Europe#2020-11-2206:30tianshuIs this the expected behavior?
(s/def ::entry-as (s/tuple #{:as} symbol?))
(s/def ::entry (s/or :the-as ::entry-as))
(s/def ::map (s/coll-of ::entry :kind map?))

(s/conform ::map '{:as e});; => {:as [:as e]}
I was thought it should return something like {:the-as [:as e]} , because the branch in s/or is tagged as :the-as. spec version 0.2.187.
#2020-11-2207:56Alex Miller (Clojure team)By default, maps are nonconforming on keys, that logic may be modifying what you expect here. You could try adding :conform-keys true to the coll spec and see if that helps#2020-11-2210:56tianshuYes, set :conform-keys true will work. Thank you!#2020-11-2215:24mishahttps://akovantsev.github.io/corpus/clojure-slack/clojure-spec filterable/searchable log for this channel on a single static/offline page 1. double click on any word to filter, 2. cmd+f to highlight matches with browser's search, 3. click any date/time to show filtered out nearby lines, for context 4. recur data extracted from https://clojurians-log.clojureverse.org/clojure-spec pages#2020-11-2215:33mishalittle teaser:#2020-11-2215:33misha#2020-11-2216:56Alex Miller (Clojure team)@misha can you stop spamming all the channels with this and just put one in announcements or whatever?#2020-11-2220:17mishasure#2020-11-2220:30seancorfield@misha Posting the clojure-slack level link to #news-and-articles would be fine. Not to #announcements since that is for Clojure code projects/libraries. But as Alex said, posting these links individually to channels as you add indexed versions is really annoying as we already have two existing searchable archives of nearly all channels here.#2020-11-2308:28mishaThank you, Sean, if I add more - Iā€™ll announce in news channel, which I didnā€™t know about. I am sorry to hear 3 specific links are perceived as spam and annoying, and answering the same questions few times a week-is not. Especially after I actually tried finding things in other ā€œsearchable logā€, and decided that week of dev time investment is worth it instead, and is useful to others.#2020-11-2313:32Drew VerleeIs it possible to extract the argument list e.g (s/cat :x int?) from an existing spec?#2020-11-2315:04sgepigon
user=> (require '[clojure.spec.alpha :as s])
nil
user=> (s/form (:args (s/fspec :args (s/cat :x int?))))
(clojure.spec.alpha/cat :x clojure.core/int?)
;; or if you use `s/fdef`:
user=> (s/fdef foo :args (s/cat :x int?))
user/foo
user=> (s/form (:args (s/get-spec `foo)))
(clojure.spec.alpha/cat :x clojure.core/int?)
This also works for :ret and :fn as fspecs implement clojure.lang.ILookup.
#2020-11-2313:33Drew Verlee(s/arg foo) ;;=> [{:x int?}]#2020-11-2313:50borkdude@drewverlee I'm not entirely clear on what your code should do, but maybe s/form is something that's useful for you?#2020-11-2315:04sgepigon
user=> (require '[clojure.spec.alpha :as s])
nil
user=> (s/form (:args (s/fspec :args (s/cat :x int?))))
(clojure.spec.alpha/cat :x clojure.core/int?)
;; or if you use `s/fdef`:
user=> (s/fdef foo :args (s/cat :x int?))
user/foo
user=> (s/form (:args (s/get-spec `foo)))
(clojure.spec.alpha/cat :x clojure.core/int?)
This also works for :ret and :fn as fspecs implement clojure.lang.ILookup.
#2020-11-2320:07Drew Verleethanks @borkdude and @sgepigon that answers the question for me.#2020-12-0119:48Kira McLeanWhatā€™s the best to way to list the keys that a map is missing? E.g. with a problem like this for a specced map, what do you guys do to collect those key-names? Iā€™m looking at using spec to validate data in a web app so ultimately Iā€™d like to translate the missing keys into a message like ā€œ<key-name> is requiredā€ but Iā€™m not sure whatā€™s the best way to get a hold of the names of the missing keys.
{:path [],
  :pred (clojure.core/fn [%] (clojure.core/contains? % :key-name)),
  :val
  {},
  :via [,,,],
  :in []}
#2020-12-0119:56Kira McLeanI was looking through expound to try to see how they do it.. it looks like the core of it is https://github.com/bhb/expound/blob/1c0d78570be3865eab8e69c1b568c4e7acee5bd8/src/expound/printer.cljc#L187-L194, where form is a :predhttps://github.com/bhb/expound/blob/1c0d78570be3865eab8e69c1b568c4e7acee5bd8/src/expound/alpha.cljc#L317.. but I donā€™t understand how that :expound.spec/contains-key-pred works..#2020-12-0119:59Kira McLeanI donā€™t see how it returns the name of the failed key.. when I try something similar it just returns ::s/invalid, which seems like what I would expect.#2020-12-0120:31bbrinck@UPGS9BS0L Expound is just parsing the s-expression of the pred. The trick is that expound will take the pred e.g. something like (clojure.core/fn [%] (clojure.core/contains? % :expound.printer/foo)) And then it will throw away the clojure.core/fn [%] part with the (nth form 2) on line 188#2020-12-0120:31bbrinckThen it uses spec to parse the s-expression to pull out the keyword of the spec#2020-12-0120:32bbrinckHereā€™s a little script to show each step. Itā€™s just an example to show how Expound figures out each part of the :pred
(let [first-problem (first (::s/problems (s/explain-data ::some-spec {})))]
    {
     :pred (:pred first-problem)
     :part-of-pred (nth (:pred first-problem)  2)
     :match (s/conform :expound.spec/contains-key-pred (nth (:pred first-problem)  2))
     }
    )
  ;; This returns:
  #_ {:pred (clojure.core/fn [%] (clojure.core/contains? % :expound.printer/foo)),
   :part-of-pred (clojure.core/contains? % :expound.printer/foo),
   :match [:simple {:contains clojure.core/contains?, :arg %, :kw :expound.printer/foo}]}
#2020-12-0120:33bbrinckDoes that help?#2020-12-0120:45Kira McLeanah cool, yeah that does make sense. Thank you! So is that how youā€™d recommend going about getting a hold of those keys? Is there a simpler way to just get spec to tell me which req keys are missing?#2020-12-0120:49bbrinck@UPGS9BS0L Unfortunately, as of spec1, thatā€™s the way I know of to get the missing keys. Things may change in spec2, but I donā€™t know of any specific plans for this.#2020-12-0120:52bbrinckItā€™d be really nice if spec2 included both the missing key and the fully-qualified key in the case of keys included via :req-un.#2020-12-0120:53bbrinck(right now if you use :req-un, sometimes Expound will warn <can't find spec for unqualified spec identifier> because it doesnā€™t know what spec you meant by the kw :foo)#2020-12-0120:55bbrinck@UPGS9BS0L This is apparently how Phrase works as well https://cljdoc.org/d/phrase/phrase/0.3-alpha4/doc/examples#required-keys#2020-12-0121:52Kira McLeanThis is really helpful, then. Thanks for all the info!#2020-12-0217:30dogenpunkIs anyone here aware of a spec -> xml printer? That is, a library that takes explain-data and outputs an xml report?#2020-12-0217:36Alex Miller (Clojure team)I'm not aware of such a thing#2020-12-0400:07Sean HIs there a good way to have multiple generators for clojure.spec.test.alpha & clojure.spec.gen.alpha on the same spec on a namespaced keyword? In particular, in cases where we know that a spec is restricted in certain contexts, such as Datomic refs (which when sent TO datomic are either a 2-tuple, an int, or a map [component], while when received FROM datomic are an int or a map). If not, my guess would be the most sensible way to do this is to just override the specā€™s generator with a generator that generates the more restrictive form (e.g. int/map in the above example, or just int for non-component refs) and then do generation some other way (or not at all) for cases where you need the more general form? Thanks! EDIT: I think Iā€™ve figured it out. For anyone whoā€™s curious, this can be done by clojure.spec.test.alpha/instrument options#2020-12-0417:51Thomas MoermanQ: consider following snippet:
(s/valid? (s/keys :req [:nexus.annotation/id]) {:nexus.annotation/id             1
                                                :nexus.annotation/annotation-set {:invalid :map}}) => false
#2020-12-0417:51Thomas Moerman1. Why does spec take the second (invalid) key into account? 2. Is there a way to make spec ignore the second key#2020-12-0417:59Thomas MoermanApparently this is by design:#2020-12-0417:59Thomas Moerman> > In addition, the values of all namespace-qualified keys will be validated > (and possibly destructured) by any registered specs. Note: there is > no support for inline value specification, by design.#2020-12-0418:11Alex Miller (Clojure team)yes, it is by design#2020-12-0418:11Alex Miller (Clojure team)attributes have global meaning#2020-12-0716:41Aleh AtsmanHello! Is there a way to get list of keys from (s/keys...) ? And if I had few specs merged can I get back a full list of keys? If not, how do you do it? Thx#2020-12-0814:12mishahttps://akovantsev.github.io/corpus/clojure-slack/clojure-spec#t1487996119012135#2020-12-0814:25mishahttps://akovantsev.github.io/corpus/clojure-slack/clojure-spec#t1480167464011304#2020-12-0915:20Aleh Atsmanexplain data approach - maybe but clojure.spec/form - doesn't work for merged spec thanks for links#2020-12-0915:20Aleh Atsmanboth ways feels more hacky than just abstracting clojure.spec away and managing things by hands#2020-12-0915:33mishaFor merged specs probably need to walk it doing s/form and s/registry. Walking datastructures ainā€™t hacky in a Lisp :op:#2020-12-0915:35mishaThey say spec2 is gonna be easier in this regard, fwiw#2020-12-0717:37Aleh AtsmanWell, I am simple guy and solved my problem with first creating entities as plain maps, later wrapping decision on what fields to use in 2 level multimethods, only from there creating spec, and returning it together with fields šŸ™‚#2020-12-0908:26Ricardo CabralHello all, I hoe you are doing good and safe. I am a newbie in Clojure and I am trying to learn about spec. I am using spec-alpha2 and I am facing an issue trying to generate a contact as you can see in the code below. The error is Couldnā€™t satisfy such-that predicate after 100 tries. when I try to generate a sample of an email using regex. Is it possible to do like this?
(def email-regex #"^[a-zA-Z0-9._%+-]
#2020-12-0908:34lassemaattaI'm no spec expert, but my understanding is that s/and uses the first spec (in this case string?) to generate values and then checks that the value matches the rest of the conditions (in this case the regex). Because string? generates random strings, it is unlikely that it matches the very specific email regex. After generating several random strings, spec gives up as it cannot create a random string which satisfies the email regex.#2020-12-0908:39lassemaattapossible solutions: a) use a constant set of sample email addresses and use s/with-gen to pick one of the samples, b) read about test.check and the tools it provides to build custom generators and try to build an email address generator by hand, c) check out libraries like lambdaisland/regal, which provide facilities to create generators for regex#2020-12-0909:01Ricardo CabralThank you @UC506CB5W for your quick answer. I will check that#2020-12-0914:38Alex Miller (Clojure team)You may have some trouble with this as spec2 is a work in progress and with-gen in particular may have some issues. I would recommend using spec 1 for now#2020-12-0917:49Ricardo CabralThank you @U064X3EF3 so far I am not facing any big issue. I am just trying some small experiments to understand how spec2 works and also learn about the idea behind it, and so far it is working as expected#2020-12-0917:49Ricardo Cabral
(def email-regex #"^[a-zA-Z0-9._%+-]
#2020-12-1012:58konrad szydloHi, I'm trying to figure out how to implement xor for a map keys. Where I'd like to check if one of the keys is present but not both at the same time.
(s/def ::a int?)
(s/def ::b int?)
(s/def ::c string?)
Where following maps are invalid: {::c "string"} ;; missing ::a xor ::b {::a 1 ::b 2 ::c "string"} ;; both ::a and ::b present and the following maps are valid: {::a 1 ::c "string"} {::b 2 ::c "string"}
#2020-12-1014:12englishs/keys only supports regular or: https://stackoverflow.com/a/41901585 but you could combine the s/keys spec with a custom xor spec via s/and, as described in https://stackoverflow.com/a/43374087#2020-12-1014:45konrad szydloThank you. My searching on duck duck go didn't return this answer from SO. That's what I was looking for. In the docs for s/keys I saw that or is supported but not xor#2020-12-1014:12vlaaad(s/and (s/keys ...) #(= 2 (count (select-keys % [::a ::b ::c]))))?#2020-12-1622:36kennyCurious if there are any ideas around how spec2's select would allow for requiring deeply nested keys based on some dispatch function? e.g., (if the event is type "a" then a specific set of keys are required, if event type is "b" then ...)#2020-12-1623:47dgb23@kenny if I understand correctly select is there to be used in specific contexts. Not sure if that answers your question.#2020-12-1623:49kennyNo. Essentially I'm curious what the story is wrt deeply nested multi-spec and select since that is a common use case.#2020-12-1700:13Alex Miller (Clojure team)Can you explain?#2020-12-1700:33kennyPerhaps an example is best.
(s/def ::order (s/schema [:order/id
                          :order/type
                          :order/a
                          :order/b
                          :order/c]))

(s/def :user/orders (s/coll-of ::order))

(s/def ::user (s/schema [:user/orders]))

(s/select ::user [:user/orders {:user/orders [:order/id
                                       :order/type
                                       ;; if :order/type is :order.type/a then require :order/a
                                       ]}])
The required order keys depend on the order's type.
#2020-12-1700:33kennyI only require :order/a when :order/type is :order.type/a.#2020-12-1700:38dgb23So it is a separate schema for order?#2020-12-1700:39kennyWhat do you mean?#2020-12-1700:40dgb23Iā€™m curious (never used spec2) as well. My intuition is that you are trying to solve something with select when it should be a separate schema for order types. But I have to think about it/read a bit more šŸ˜„#2020-12-1700:41kennySorry, I don't understand what you mean by separate schema. Could you clarify?#2020-12-1700:42dgb23using spec/or to distinguish the order types#2020-12-1700:42dgb23and each order type has a separate spec that holds the context of order/a order/b etc#2020-12-1700:42kennyOh. Yes, that is possible here. It results in really bad error messages though (if you have a bad input, every single case of the or is outputted). multi-specs output a really nice error.#2020-12-1700:43kennyIn our case there may be hundreds of different "types". The explain-data is crazy šŸ˜µ#2020-12-1700:43dgb23i see#2020-12-1700:44dgb23have you tried to parse explain-data to kind of narrow it down? just talking off my ass here#2020-12-1700:46kennyYou can manipulate the output however you normally would šŸ™‚ The thing is, multi-specs solve this problem nicely (albeit a bit cumbersome to work with). They just don't work with select, and it seems that there is a good fit for something there.#2020-12-1700:47dgb23ah I see it now!#2020-12-1700:49kennySo I was curious if there were any thoughts on how/if something like this would be a part of spec2. For us, this use case happens everywhere.#2020-12-1700:55dgb23since select is just a spec too, then you would dispatch with a multimethod too right? so it would be something like (defmulti order-type :order/type...#2020-12-1700:55dgb23each returning a select#2020-12-1700:55dgb23that is specifically tailored to query a variant#2020-12-1700:56kennyNot sure I follow how that'd work with the above.#2020-12-1700:56dgb23me neither šŸ˜„#2020-12-1700:56kennyThe main thing is that you always declare required keys "top-level." Pretty sure what you have there wouldn't work :thinking_face:#2020-12-1700:57dgb23got me hooked Iā€™m going to play a bit with spec2 now#2020-12-1700:57dgb23what do you mean with required keys?#2020-12-1700:57dgb23schema keys are not required#2020-12-1700:59kennyNo matter the "level" (read nesting) you're at, you're always able to select which keys are required.#2020-12-1701:53dgb23
(s/def ::order (s/schema [::order-id
                          ::order-type
                          ::order-a
                          ::order-b
                          ::order-c]))

(s/def ::orders (s/coll-of ::order))

(defmulti order-type ::order-type)
(defmethod order-type ::order.type-a [_]
  (s/select ::order [::order-a]))
(defmethod order-type ::order.type-b [_]
  (s/select ::order [::order-b]))

(s/def ::order-typed (s/multi-spec order-type ::order-type))

(s/valid? ::order-typed {::order-type ::order.type-b ::order-b "foo"})

(s/def ::orders-typed (s/coll-of ::order-typed))

(s/valid? ::orders-typed [{::order-type ::order.type-b ::order-b "foo"}])
#2020-12-1701:54kennyYou've changed the problem šŸ™‚#2020-12-1701:54dgb23I get the feeling that I didnā€™t understand it in the first place#2020-12-1701:54dgb23šŸ˜„#2020-12-1701:55kennyAt the level of the "user" map, I want to select what keys are required in each map under :user/orders.#2020-12-1701:55kennyWithout renaming my :user/orders key.#2020-12-1702:00dgb23i see but those are different specs#2020-12-1702:00kennySame schema, different selected keys.#2020-12-1702:01dgb23the schema doesnā€™t know about select (just tried it)#2020-12-1702:02dgb23i have to leave it at that (itā€™s late) but was fun so far. maybe someone actually knowledgeable can find a better way#2020-12-1702:03kennyNot sure what you mean exactly šŸ™‚ This is what I'd like.
(s/select ::user [:user/orders {:user/orders [:order/id
                                              :order/type
                                              ;; if :order/type is :order.type/a then require :order/a
                                              ]}])
#2020-12-1702:06dgb23What I mean is that Iā€™m still a beginner with these things. I donā€™t really see a solution right now other than dispatching the select and conform/validate on the select spec.#2020-12-1702:07kennyOh, I understand. Yes, I was 99% certain spec2 does not cover this. I am interested in thoughts on this particular problem and how spec2 might solve it.#2020-12-1702:08dgb23i mean this sound pretty powerful (from the wiki): https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha#creating-specs-programmatically#2020-12-1702:10kennyFor sure! I'm confident something could be built to solve this (likely even in spec1). Curious if this use-case is in scope for spec2 though. It seems like a core problem.#2020-12-1714:46Alex Miller (Clojure team)Two things here - first selects do not currently have the ability to handle maps of colls of maps of ... , only nested maps. That is something we are thinking about though as it is very common. Second, there is not currently any way to indicate any sort of predicate or condition on the nested data like you're talking about but that is something we're thinking about. What you're asking for might be beyond where we end up though, not sure yet.#2020-12-1714:46Alex Miller (Clojure team)select is currently only about nested map structures (what keys are provided at each level)#2020-12-1722:24kenny> there is not currently any way to indicate any sort of predicate or condition on the nested data like you're talking about but that is something we're thinking about. This sounds exactly like what I'm after šŸ™‚ This is such a big issue for us I may need to tackle it internally for now. Would love to know what those thoughts are. Though, I assume they are not public yet?#2020-12-1722:59Alex Miller (Clojure team)sorry, not working on this right now#2020-12-1814:20Shantanu KumarDoes anybody know of a tool/library that can derive spec based validators from OpenAPI schemas? I looked at spec-tools https://github.com/metosin/spec-tools/blob/master/docs/06_openapi.md but it only appears to cover the generation (reverse) part.#2020-12-1915:53misha@kumarshantanu not sure if openapi and json-schema are same, but have a look: https://github.com/akovantsev/json-schema-to-clojure-spec#examples#2020-12-2019:27Shantanu KumarThanks @U051HUZLD, as of v3.0.3 OpenAPI has some overlap with JSON schema https://spec.openapis.org/oas/v3.0.3#properties but does not appear to be fully compatible.#2020-12-1915:55misha*Alpha quality and all that.#2020-12-2118:08Michael Stokley
(do
  (defn f [x] x)
  (s/fdef f :ret int?)
  (stest/instrument)
  (f "hello"))
i would expect this to throw, but i don't see that happening. what am i missing?
#2020-12-2118:14valeraukoare you doing this in the repl? I've had surprising behavior wrapping defn and similar in do/let. Try executing line-by-line?#2020-12-2118:14seancorfieldinstrument is for argument checking, not return value checking.#2020-12-2118:15seancorfieldinstrument checks that your function is being called correctly by other code. check is what you use to generatively test the behavior of the function itself.#2020-12-2118:16seancorfieldSee https://clojure.org/guides/spec#_instrumentation_and_testing @U7EFFJG73#2020-12-2118:37Michael Stokley@U04V70XH6, thank you!#2020-12-2118:37Michael Stokleyi misunderstood instrument#2020-12-2118:38seancorfieldI think almost everyone does when they first start using Spec.#2020-12-2119:32borkdudeThere is an unofficial spec tool which will check return values as well, called orchestra#2020-12-2119:35seancorfieldI think that library is a bit misguided since it deliberately blurs the lines that Clojure Spec draws by design. Caveat programmer.#2020-12-2119:36borkdudeSeemed worth mentioning, because a lot of people use it exactly for this reason.#2020-12-2120:34seancorfieldNo comment šŸ™‚#2020-12-2120:35borkdudeHehe, well, I don't use it either ;)#2020-12-2214:55mpenetthere are many of these libs, guardrails, orchestra, ghostwheel#2020-12-2214:55mpenetsome use instrument/check, some avoid it (intentionally)#2020-12-2214:56borkdudeI saw in the #announcements channel that guardrails now has an async instrumentation thing, so it doesn't slow down your functions but you will eventually know something is wrong - at least that's what I got from it#2020-12-2214:58mpenetyeah, it's optional. I think the approach with guardrails is by default just to log the failures#2020-12-2214:59mpenetso that's ok to do it that way#2020-12-2214:59mpenetI *reall*y wonder what spec2 will do in that regard, since apparently it's one area where there's some heavy hammocking going on. [edit] By that I don't mean Rich is heavy!#2020-12-2214:59borkdudeInteresting way to solve the slowness that instrumentation brings.#2020-12-2215:07mpenetbut yes, guardrails is the most interesting of the bunch so far imho. One thing is missing is editor support (indentation annotations), it's a bit ugly out of the box#2020-12-2215:11borkdudeyou mean like flycheck integration? that's pretty easy if you have file:line:col: ERROR message kind of output#2020-12-2215:11mpenetno, indentation, if you use something like aggressive-ident on emacs you'll get some ugly defn's with guardrails#2020-12-2215:11mpenet(same with cljfmt)#2020-12-2215:12borkdudeisn't that a general clojure tooling problem then?#2020-12-2215:12mpenetit's easy to solve, at conf level or even lib level#2020-12-2215:12mpenetyou can add annotation to the macros to give hints to editors :style/ident#2020-12-2215:12mpenetit's not standard, but I think both emacs/cursive support it at least#2020-12-2215:13borkduderight. oh, this tool forces you to use a different defn macro? hmm#2020-12-2215:13mpenetmaybe even cljfmt#2020-12-2215:13mpenetyeah#2020-12-2215:13mpenetwell no, you can also use the annotation version#2020-12-2215:13mpenetbut you also have these def defn macros#2020-12-2215:14mpenetwe use it quite a lot, and like it. Sure there are rough edges but nothing too bad#2020-12-2215:14mpenet(spec1 I mean)#2020-12-2214:52dgb23Not sure if this is a #beginners question: There is something that bothers me about (my understanding of) spec. Example: #{ā€œfooā€ ā€œbarā€} implies string?.
(s/explain-data #{ā€œfooā€ ā€œbarā€} 42)
(s/explain-data (s/and string? #{ā€œfooā€ ā€œbarā€}) 42)
What bothers/interests me here is that I have to write the second, but my first spec is a subset of string?. I know the semantics of the first might very well be heterogenous over time. But at that specific point in time the above sentence is true. It seems useful or interesting to me that a more involved and concrete spec should or could fail on and explain broader invariants if they are not satisfied. So here is my first question: How feasible would it be to infer a superset of a spec? The problem I see with this, is that the first spec can be expressed in multiple ways. For example with s/or or even worse: with reduce. Maybe the first spec function should have a function spec that checks :args to be string?, but that seems clunky. A better question might be: is this a matter of design? Are these two specs really saying something meaningfully different or is the second strictly superiour?
#2020-12-2215:09Alex Miller (Clojure team)"I have to write the second" - why?#2020-12-2216:06dgb23My intuition is that 42 should fail at string? before it fails at #{"foo" "bar"} . Or more generally: to explain why an input fails, wouldnā€™t it be more useful to say that it doesnā€™t satisfy an implied superset before I explain that it needs to satisfy an arbitrarily concrete predicate? But from your question I guess this is a matter of designing a spec.#2020-12-2216:16Alex Miller (Clojure team)it's important to shift think a type mindset to a "value set" mindset. specs are predicative descriptors of allowed values. Saying #{"foo" "bar"} means "these are the only two values allowed" vs string? which means "satisfies the string? predicate". those are semantically a lot different (I would generally hesitate to use the first one in a function signature as time is long and you'll probably come up with a 3rd value later). you might for example, spec it as string? but then validate a narrower spec inside the function.#2020-12-2216:18Alex Miller (Clojure team)it is really a matter of how you choose to design the spec for your data and what should imply to a user#2020-12-2216:48dgb23Thank you, this advice is very valuable (sic!), I have to think about it a bit moreā€¦ I might be stuck in deduction world. I have this nagging feeling that deduction, where possible, gives you ā€œfreeā€ stuff. But if I understand correctly this isnā€™t what spec is about. Itā€™s supposed to be used concretely and specifically. AKA nothing additional should be inferred from a spec.#2020-12-2216:52Alex Miller (Clojure team)yes#2020-12-2216:52Alex Miller (Clojure team)it's about values, not types#2020-12-2216:53Alex Miller (Clojure team)an interesting question though is subsumption - "does this spec subsume (include) all possible values of this other spec?"#2020-12-2216:53Alex Miller (Clojure team)and that is something that we hope to potentially provide in the future#2020-12-2216:54Alex Miller (Clojure team)b/c it lets you ask questions about spec evolution in tandem with data evolution#2020-12-2217:34dgb23So in my case string? subsumes #{ā€œfooā€ ā€œbarā€} right? But youā€™re not interested in providing that for a specific isolated spec (aka subtyping or similar) but you want be able to ask questions about different specs across a timeline or maybe across different systems. There seems to be tons of research surrounding the term ā€œsubsumptionā€ in CS, robotics, AI, Lisp, Prolog and Datalog. And also in OWL https://en.wikipedia.org/wiki/Web_Ontology_Language. Thankā€™s again šŸ™‚#2021-12-3113:49stephenmhopperHappy new yearā€™s eve, everyone! Iā€™m working on writing some form validation for a web application. In my web app, most of my requests have well-formed request bodies that are known ahead of time. However, some of the requests have fields / properties whose structure is determined by metadata entries in my applicationā€™s database. Is there a good example somewhere of programmatically generating Clojure specs for handling these at runtime? So far, Iā€™ve found that I can create a spec with (s/spec int?) (for example), but then Iā€™m not sure how to use that with (s/keys) as all of the s/keys examples Iā€™ve seen require specs to be registered in the global spec registry. Any ideas on how I can do this? Do I need spec2 for this? Also, itā€™s worth noting that changes to these metadata DB rows are infrequent enough that I could probably create the specs by reading the DB when the application first starts, but Iā€™d like to avoid doing that if possible.#2021-12-3116:28Alex Miller (Clojure team)you certainly can register the specs dynamically but it requires a bit of care with macros or eval to do so. spec 2 is a little friendlier for this use case but is not ready for real use yet#2021-12-3116:30stephenmhopperOkay, thank you, Alex. I just found https://github.com/metosin/spec-tools, which might fit my use case. TBD#2021-01-0411:20Karol WĆ³jcikHello! Is it possible to disable load of clojure.spec.alpha completely?#2021-01-0411:34hkjelsclojure.spec.alpha is not part of Clojure, so what do you mean by ā€œdisable loadā€?#2021-01-0412:40delaguardoit is not possible.#2021-01-0413:03Karol WĆ³jcik@U0B1SDL67 it seems java11 dynamically loads clojure.spec.alpha which results in error: https://clojurians.slack.com/archives/C03S1KBA2/p1609747607299900#2021-01-0413:11delaguardostrange, clojure.walk is not requiring clojure.spec.alpha#2021-01-0413:13Karol WĆ³jcikFor me the strangest thing is that even after removing clojure.walk dynamic class load requires spec.alpha which results in different error. Never seen it before.#2021-01-0413:18delaguardobut spec.alpha is require clojure.walk ā€¦#2021-01-0413:20delaguardoand clojure compiler is using clojure.spec.alpha there https://github.com/clojure/clojure/blob/05c193c031aa3a825ebb824b4135e36a79693f51/src/jvm/clojure/lang/Compiler.java#L6961#2021-01-0414:37Alex Miller (Clojure team)spec depends on clojure, and clojure depends on spec. so there is actually a cycle here conceptually.#2021-01-0414:55Karol WĆ³jcik@U064X3EF3 tried to set -Dclojure.spec.skip-macros=true without results. Here is the example: https://github.com/ekoontz/holy-lambda/blob/repro-spec/examples/hello-lambda/project.clj Wondering why Clojure compiler macroexpands on runtime#2021-01-0516:09Karol WĆ³jcikWill try to provide minimal repro during weekend.#2021-01-0414:36Alex Miller (Clojure team)See -Dclojure.spec.skip-macros=true https://clojure.org/guides/faq#skip_macros for turning off macro checking#2021-01-0523:02bennyis it possible to write specs for maps like you can in schema?
(def User
  {::id   s/Int
   ::name s/Str})
vs
(s/def ::id int?)
(s/def ::name string?)
(s/def ::user
  (s/keys :req [::id ::name]))
#2021-01-0523:08Alex Miller (Clojure team)that is how you do it in spec#2021-01-0523:13bennythereā€™s no way to do it inline like schema?#2021-01-0523:22Alex Miller (Clojure team)no, there is a philosophical difference in approach here#2021-01-0523:23Alex Miller (Clojure team)spec is trying to build a registry of spec'ed attributes. the attributes are seen as primary, the map only as container.#2021-01-0523:23Alex Miller (Clojure team)spec 2 is developing this further and will have some support for unqualified attributes with inline specs in a schema#2021-01-1222:46Alys Brooksis there any guidance on where to start adding specs to an existing project?#2021-01-1222:53Alys Brooksmy current strategy is basically to add specs to new code like I do tests. my next step will probably be trying to add specs as I happen to touch other code. however, I'm wondering about more proactive approaches.#2021-01-1223:04seancorfield@actuallyalys_slack I tend to focus on only adding specs where I think it will help me understand the code -- to avoid the temptation to treat Spec like a type system.#2021-01-1223:06seancorfieldCommon guidance is: write specs for your data structures but don't over-specify them; write specs for functions at "boundaries" (APIs of modules, for example) but don't get carried away. If you over-specify the code, it will be harder to make changes in the future because you may bake in assumptions that don't really hold as invariants over time.#2021-01-1223:14Alys Brooksgood point. i'm not as worried about over-specification because instrumentation can always be turned off or specs commented out.#2021-01-1223:16Alys Brooksbut there's still a time and mental cost to over-specified specs even so, so that's a good reminder.#2021-01-1223:19Alys Brooksi do think non-boundary specs can be useful because sometimes it's hard to know the exact shape of the data an internal function deals with or tedious to create it for the purpose of REPL experimentation. i guess that falls under "where i think it will help me understand the code"#2021-01-1223:26seancorfieldYup, I recently had to tackle a large part of our code base at work that I hadn't had to work on before and the first thing I did was to spec out some of the data structures, add some function specs, instrument the code and see what broke (and what worked) -- and then evolve the specs until everything worked again and that gave me a good understanding of what the data structures looked like and how the functions transformed them.#2021-01-1316:32mgIt appears that closed spec checking doesn't work on select in spec2. Is this something deliberate?#2021-01-1316:35Alex Miller (Clojure team)don't remember :) I think it should work? but could easily be buggy#2021-01-1316:37mgToy example:
(spec/def ::a string?)

(spec/def ::s (spec/schema [::a]))

(spec/valid? ::s {::a "hello" ::b "world"} {:closed #{::s}}) ;; => false

(spec/valid? (spec/select ::s [*]) {::a "hello" ::b "world"} {:closed #{::s}}) ;; => true
#2021-01-1316:38mgI read through the implementation of select and it looks like it doesn't do anything with :closed#2021-01-1316:42Alex Miller (Clojure team)the things you "close" are schemas#2021-01-1316:43Alex Miller (Clojure team)select has some pending rework to make it build more on top of schema#2021-01-1316:43Alex Miller (Clojure team)when that's done it should start getting that effect#2021-01-1316:44Alex Miller (Clojure team)so without thinking more, this is probably in the category of wip#2021-01-1316:47mgOk, thanks for the clarification! Sounds also like it would not be fruitful for me to submit a patch, also, so I'll wait on it#2021-01-1316:49Alex Miller (Clojure team)nah, it's a non-trivial amount of work and bound into other things
#2021-01-1815:17borkdudeIs the clojure.test.check lib an implementation detail of clojure.spec, or can we rely on the coupling to always to remain exist between the two?#2021-01-1815:59Alex Miller (Clojure team)I'd say it's more than an implementation detail, it's just somewhat hidden so that it can be dynamically loaded#2021-01-1816:00Alex Miller (Clojure team)no plans to change anything about that#2021-01-1816:00Alex Miller (Clojure team)spec generators are test.check generators (in a thunk)#2021-01-1911:31vlaaadI was thinking, is there (should there be) a way to describe something in spec as ā€œthis has to be produced from a call to that functionā€? Sort of a bit like (s/fdef some-fn :args ... :ret ::the-thing) without specifying ::the-thing anywhere else, but, like, inverted: (s/def ::the-thing (s/ret-of some-fn)) . I donā€™t think there is a good way to fit this into conform machinery, but Iā€™m just thinking about the ability to express the API contract in spec (so far expressing it in the docstring is fine šŸ™‚ )#2021-01-1911:58borkdude@vlaaad you can do this without spec maybe, just by letting the "factory" function put in some unique property which indicates it was made by that function#2021-01-1911:58borkdudeand you can validate that property using spec or using some other assertion#2021-01-1911:59borkdudeyou can also do this via .impl naming, e.g. create a defrecord in an impl namespace and have one public API function for creating those records#2021-01-1912:04vlaaadI also can use blanket (https://github.com/vlaaad/blanket) to cover the implementation details šŸ˜„ To be clear ā€” Iā€™m not looking for solutions outside of spec, I just thought the problem Iā€™m solving is about describing the API, and spec seems like a fitting tool for this use case. I also heard @richhickey is redesigning spec particularly around fn specs, hence decided to share a use case to consider šŸ™‚#2021-01-1913:55Alex Miller (Clojure team)I donā€™t understand the use case#2021-01-1913:57Alex Miller (Clojure team)Saying ā€œthis has to be produced from a call to that functionā€ seems weird#2021-01-1913:57Alex Miller (Clojure team)Data is data#2021-01-1914:05vlaaadthere is evolution over time#2021-01-1914:09vlaaadI want to express the intention ā€” if next version returns something else (or more realistically ā€” if the next version has more functions that produce something valid in a other context), it will remain valid in that other context#2021-01-1914:09Alex Miller (Clojure team)then spec the truth - what does the data look like?#2021-01-1914:09Alex Miller (Clojure team)don't couple the spec to code#2021-01-1914:09Alex Miller (Clojure team)this is the whole schema/select idea#2021-01-1914:09Alex Miller (Clojure team)schema is all possible fields that may travel together#2021-01-1914:10Alex Miller (Clojure team)select tells you what fields are selected from the schema at different points#2021-01-1914:11vlaaadbut I want to keep parts of the data implementation details#2021-01-1914:13Alex Miller (Clojure team)then don't spec those parts#2021-01-1914:14vlaaadGood point, although Iā€™m not sure it captures the intention#2021-01-1915:31emccue@vlaaad Wouldn't just speccing the return values in the namespace that effectively declares that structure be enough?#2021-01-1915:31emccueat least from an encapsulation POV?#2021-01-1917:43dgb23Would be speccing it with any? (and a docstring when we get it) a better idea in this case?#2021-01-1917:46dgb23The user can infer that ::the-thing is explicitly anything forever.#2021-01-1917:48Alex Miller (Clojure team)why bother?#2021-01-1917:49Alex Miller (Clojure team)if it truly is "implementation details", then you are just erecting scaffolding and barriers in the way of future change#2021-01-1917:52Alex Miller (Clojure team)there needs to be some balance between the agility of Clojure data and the constraints of specs - don't spec everything to death#2021-01-1920:11vlaaadSounds like a disapproving opinion. I want to express a single thing: the shape of this object is implementation detail, and despite this fact, it can be used in that context. This is not a barrier in the way of future change, and definitely not specing everything to death, this is a contract that helps user to see the relationship between parts of API.#2021-01-1920:15vlaaadI like the (s/def ::the-thing any?) btw, coupled with API fspecs with :ret ::the-thing I think it reaches the intention#2021-01-2005:35Nolanhi there! have recently run into a scenario where i would like to access specs defined in clojure.core.specs.alpha from clojurescript. those specs all seem to be in the registry in clojure as soon as you require clojure.spec.alpha, but don't seem to be included in the final registry-ref in the cljs spec namespace. i imagine this is intentional, but i'm wondering if there might still be a way to access them somehow. really appreciate any guidance on this. those are some valuable shapes!#2021-01-2314:02ikitommiwhat is the use case or rationale for s/map-of not conforming the keys by default?
(s/conform 
  (s/map-of (s/or :s string? :b boolean?) 
            (s/or :s string? :b boolean?)) 
  {"k" "v"})
; => {"1" [:s "1"]}

(s/conform 
  (s/map-of (s/or :s string? :b boolean?) 
            (s/or :s string? :b boolean?) 
            :conform-keys true) 
  {"k" "v"})
; => {[:s "k"] [:s "v"]}
#2021-01-2314:14Alex Miller (Clojure team)Usually keys are things that are not changed by conforming (keywords, strings, longs, symbols) and there is cost to conforming things, so defaults to not doing it.#2021-01-2314:46ikitommiok, thanks.#2021-01-2512:33borkdudeis there a possibility that spec1 and spec2 will be bunded in the same lib and logic that was unchanged will map to the same functions (better re-use when fixing bugs, smaller byte code and native-images)?#2021-01-2513:44Alex Miller (Clojure team)Seems highly unlikely as all functions have changed#2021-01-3017:17hadilsIs there any documentation on clojure.spec.alpha.gen (spec2)?#2021-01-3017:40seancorfield@hadilsabbagh18 Spec 2 is not ready for use yet. It's still evolving and it has a number of bugs right now.#2021-01-3017:41hadilsThanks @seancorfield#2021-01-3017:43seancorfield(and the only documentation for it is what you'll find in the repo and the auto-generated API docs: https://clojure.github.io/spec-alpha2/ )#2021-01-3018:23Alex Miller (Clojure team)https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha#2021-01-3018:24Alex Miller (Clojure team)https://github.com/clojure/spec-alpha2/wiki/Schema-and-select#2021-01-3018:27hadilsThanks @alexmiller -- I have already read those websites. I was asking about gen specifically.#2021-01-3018:30Alex Miller (Clojure team)No changes from spec 1#2021-01-3018:30Alex Miller (Clojure team)So far#2021-01-3018:57hadils@alexmiller -- my IDE indicates differently. I am requiring clojure.alpha.spec.gen -- e.g. it does not have fmap#2021-01-3018:59Alex Miller (Clojure team)Thatā€™s doesnā€™t seem right#2021-01-3019:01Alex Miller (Clojure team)I think your ide is wrong :)#2021-01-3019:02hadilsOk.#2021-01-3019:02Alex Miller (Clojure team)https://github.com/clojure/spec-alpha2/blob/9dc3344bfe6d13213bdb15ca2e7680cd8eb439e3/src/main/clojure/clojure/spec_alpha2/gen.clj#L95 #2021-01-3019:03Alex Miller (Clojure team)No changes here from spec 1#2021-01-3019:03Alex Miller (Clojure team)But it is generating those defs as wrappers so a static analyzer wouldnā€™t see them#2021-01-3019:03Alex Miller (Clojure team)If you load the ns you should see those vars though#2021-01-3019:05hadilsI have a different sha. Let me try yours and see what happens...#2021-01-3019:11hadilsI tried it and it works now. Thanks for all your help.#2021-01-3021:01Greg RynkowskiHi, Iā€™m trying to create a time generator based on given time range. I thought about using s/int-in but distribution for default generator for this spec is not very useful. E.g.
(gen/sample (s/gen (s/int-in 1 1000000000)))
=> (2 1 2 5 5 6 15 2 3 4)
(the range is from 1 to bilion while all sample results are leaning to beginning of the range) Are there any built-in generators I could use that can give me better distributed results?
#2021-01-3021:27Greg RynkowskiOh, ok, I just realised that I get better results when Iā€™m calling gen/generate. Weird. šŸ¤·#2021-01-3022:09andy.fingerhutI am not sure if what I am about to describe is what is happening in your case, but note that many spec generators try to generate "small test cases" first, then "larger/more complex test cases" only after those pass. For numbers, that sometimes means generating smaller values first, and only when those test cases pass, larger values. Default generators can be overridden if you don't like how they work.#2021-01-3022:10andy.fingerhutThe idea being if that "small" test cases fail, those are usually easier to debug the root cause and fix it, versus seeing large complex failing test cases.#2021-01-3022:32Greg RynkowskiCould be, I donā€™t know. I end up creating custom integers generator using MersenneTwister PRNG.#2021-01-3022:36Greg Rynkowski
(defn gen-int-in
  [{:keys [min max]}]
  (gen/fmap
    (fn [_] (-> (twist/next-int) (mod (- max min)) (+ min)))
    (gen/return :something)))

(defn millis->date-time
  [millis]
  (-> millis
      (Instant/ofEpochMilli)
      (.atZone (ZoneId/systemDefault))
      .toLocalDateTime))

(defn gen-local-date-time
  [[from to]]
  (gen/fmap
    millis->date-time
    (gen-int-in {:min from :max to})))
sample execution:
(gen/sample (gen-int-in {:min 1 :max 1000000000}) 15)
=>
(242825682
 150977944
 334804692
 989355517
 482024216
 627890233
 163007490
 227494133
 464164671
 747363986
 762489210
 261499033
 582274967
 50926907
 472313391)

(gen/sample (gen-local-date-time (create-future)))
=>
(#object[java.time.LocalDateTime 0x4139290 "2021-02-22T13:25:24.531"]
 #object[java.time.LocalDateTime 0x53185984 "2021-01-31T00:17:03.320"]
 #object[java.time.LocalDateTime 0xcb5632b "2021-02-23T02:22:41.585"]
 #object[java.time.LocalDateTime 0x773ccacd "2031-01-23T16:49:36.048"]
 #object[java.time.LocalDateTime 0x6fa351a8 "2021-02-09T10:45:48.003"]
 #object[java.time.LocalDateTime 0x7ee6f1a1 "2031-01-29T13:07:57.477"]
 #object[java.time.LocalDateTime 0x596d796d "2021-02-20T11:15:51.061"]
 #object[java.time.LocalDateTime 0x4896aa2f "2031-01-18T13:37:21.066"]
 #object[java.time.LocalDateTime 0x3dcd9c84 "2021-02-23T23:50:04.910"]
 #object[java.time.LocalDateTime 0x746f0965 "2021-02-21T22:55:50.011"])
(`create-future` returns pair of timestamps [now, 10-years-from-now])
#2021-02-0115:52Greg Rynkowski@U0CMVHBL2 re your answer https://clojurians.slack.com/archives/C1B1BB2Q3/p1612044579020800?thread_ts=1612040474.020100&amp;cid=C1B1BB2Q3: This is exactly the case I was experiencing two days ago. I didnā€™t understand how generators work. This talk was eye opening: https://www.youtube.com/watch?v=F4VZPxLZUdA. Creating a custom generator was an overkill, simple gen/scale was enough for what I needed.#2021-02-0115:53andy.fingerhutCool. Glad you are finding ways to get the behavior you are hoping for.#2021-02-0115:55andy.fingerhutThe "small is simpler and should be generated earlier" makes sense for some numeric inputs, e.g. if your Fibonacci function has bugs for small input values, you probably want to debug those first before figuring out why it fails when given 10,325 as input. But for your use case of date/time ranges, in most applications it seems like the values early in the range are no simpler than any others (I imagine there could be date/time use cases where just-inside and just-outside of acceptable ranges are important corner cases to ensure that you test).#2021-02-0313:59hadilsDoes spec2 work with ClojureScript?#2021-02-0314:06Alex Miller (Clojure team)no#2021-02-0314:06Alex Miller (Clojure team)but it doesn't necessarily work with Clojure either as it's still a wip :)#2021-02-0314:06Alex Miller (Clojure team)CLJS is going to wait till it's "done" to port it#2021-02-0314:09hadilsThanks#2021-02-0512:56Daniel StephensHi, is there a better way to have a completely custom spec generator?
(let [my-custom-generating-fn #(do {:something (rand-int 100) :complicated (rand-int 200)})

      g (gen/fmap #(%) (gen/return my-custom-generating-fn))]
  (gen/generate g))
all the examples I found start from some primitive like (gen/int) . Feels like what I was trying is intended to be difficult, is that because it doesn't follow some underlying random seed or something, or am I just missing the key bit
#2021-02-0513:40Alex Miller (Clojure team)Correct - generally generators should control randomness, not introduce their own sources of randomness, as that makes it impossible to either shrink or regen from the same seed#2021-02-0513:53Daniel Stephensvery useful, thanks Alex#2021-02-0514:15borkdudeA one time shout out: finding specs by keyword is now a lot easier in clojure-lsp. You can even navigate to the definition of a spec via the keyword.#2021-02-1313:08vlaaadI was playing with built-in specs and discovered there are A LOT of places to embed var metadata in defn:
(defn ^{:b 1} foo
  {:c 1}
  ([])
  {:a 1})
:a , :b and :c all end up in var meta!
#2021-02-1315:53vemv> I wonder how's that I guess it's because if there was just one place, people could accidentally place it elsewhere and get no metadata attached#2021-02-1313:09vlaaadI wonder how's that, and why :arity-n defn variation allows attr map at the end...#2021-02-1316:52Alex Miller (Clojure team)I found that when I specā€™ed it and asked Rich about it. The thought at the time was that metadata might get big (this predated some of the ^ syntax) and it would obscure the main part of the function so you could put it at the end#2021-02-1316:52Alex Miller (Clojure team)Itā€™s rarely used but I did find some places using when testing a spec without it#2021-02-1319:01vlaaadVery interesting, thanks for this explanation!#2021-02-1903:54Zak SinghI have a bit of a messy spec/generators question which Iā€™ve written up here - https://stackoverflow.com/questions/66271188/spec-for-map-with-interdependent-values-between-nested-levels - is such a thing possible?#2021-02-1904:16Alex Miller (Clojure team)Doing this kind of thing is inherently challenging. The general approach to take is: first generate a model by building up a core and extending it with gen/bind, then at the end generate the actual data structure with gen/fmap#2021-02-1904:17Alex Miller (Clojure team)As a simple example, donā€™t try to generate a square by generating random points. Instead, generate the right and left x, the top and bottom y (thatā€™s your model), then generate the points of the square using fmap at the end#2021-02-1904:20Alex Miller (Clojure team)So Iā€™m yours, maybe first generate a pool of terminals (collection of more constrained nodes, then randomly pick subsets to combine, and then maybe do some massage at the end#2021-02-1904:20Alex Miller (Clojure team)How far you go with this depends how much of the space you want to cover#2021-02-1904:22Zak SinghReformulating the problem like that makes a lot of sense - I essentially need to build up layers from the terminal case. This is really an incredibly powerful tool#2021-02-1904:26Alex Miller (Clojure team)Make sure to lean on s/gen of specs that may not match your public specs - easiest way to make new sub generators#2021-02-1904:33Alex Miller (Clojure team)Like (s/gen (s/tuple ...))#2021-02-1904:34Alex Miller (Clojure team)Or (s/gen #{:magic :values})#2021-02-1904:36Zak Singh
(def terminal-gen
  (gen/bind
    (spec/gen (spec/tuple ::terminal-name ::terminal-kind))
    (fn [[name kind]]
      (gen/hash-map
        :name (spec/gen #{name})
        :kind (spec/gen #{kind})))))
#2021-02-1904:36Zak Singhlike this?#2021-02-1905:51Zak SinghManaged to get it built! That was some fun code:
(spec/def ::kind #{"NON_NULL" "LIST" "SCALAR" "OBJECT"})
(spec/def ::name (spec/nilable string?))
(spec/def ::ofType (spec/or :terminal nil?
                            :type ::type))

(spec/def ::terminal-kind #{"SCALAR" "OBJECT"})
(spec/def ::terminal-name string?)

(spec/def ::wrapper-kind #{"NON_NULL" "LIST"})

(def terminal-gen
  (gen/bind
    (spec/gen (spec/tuple ::terminal-name ::terminal-kind))
    (fn [[name kind]]
      (gen/hash-map
        :name (spec/gen #{name})
        :kind (spec/gen #{kind})
        :ofType (gen/return nil)))))

(defn build-type
  ([max-depth] (if (= max-depth 1) terminal-gen
                                   (build-type max-depth 0 terminal-gen)))
  ([max-depth curr-depth inner-gen]
   (if (< curr-depth max-depth)
     (recur max-depth
            (inc curr-depth)
            (gen/bind inner-gen
                      (fn [inner-gen]
                        (if (= "NON_NULL" (:kind inner-gen))
                          (gen/hash-map
                            :name (gen/return nil)
                            :kind (spec/gen #{"LIST"}) ; two NON_NULLs cannot be child-parent
                            :ofType (spec/gen #{inner-gen}))
                          (gen/hash-map
                            :name (gen/return nil)
                            :kind (spec/gen ::wrapper-kind)
                            :ofType (spec/gen #{inner-gen}))))))
     inner-gen)))

(def type-gen
  (gen/bind
    (spec/gen (spec/int-in 1 5))
    build-type))
#2021-02-2115:20borkdudeA video about grasp: a tool which lets you search code using clojure.spec specs#2021-02-2218:06Alex WhittWould anyone like to jump on this discussion thread I created on the subreddit? (I'd prefer to keep it there so it doesn't disappear behind Slack's paywall) https://www.reddit.com/r/Clojure/comments/lpv8ok/spec_vs_malli/#2021-02-2221:41Alex WhittThe sentiment from the community so far has been leaning towards Malli. I'd love to get some alternative viewpoints in there.#2021-02-2221:50seancorfield@alex.joseph.whitt All I'll say is that I try hard to use "official" Cognitect stuff where it is available and we've been very happy with Spec for several years at work, although we have recently started using exoscale/coax in addition to Spec so that we can tease apart/replace our "coercing web specs" and use a more standard approach (i.e., separate coercion of strings, such as form input data, to stuff like long, bool, date... from the actual specs themselves).#2021-02-2221:54Alex Miller (Clojure team)I think your summary and the comments there are fair#2021-02-2221:57seancorfieldI have not looked at Malli in any depth but we did use Schema for a while (and abandoned it -- actually twice -- because it felt non-idiomatic with its syntax and it was only as good as your testing strategy, without generative testing: we were happy with our switch to Spec in comparison to Schema). We also tried Typed Clojure (again, twice) and abandoned that for different reasons. Spec hits a sweet spot for us -- and we use it in a lot of different ways per https://corfield.org/blog/2019/09/13/using-spec/#2021-02-2221:58Alex Miller (Clojure team)if I could wave a magic wand and have a final design and impl for spec 2 I would, but it's tackling some hard problems (beyond what we tried to tackle in spec 1). I know it feels stalled but work really does continue on it and I have hope that we will pop the stack of other work in progress back to it "soon".#2021-02-2316:06uwoI'm glad y'all are taking your time. It's worth it.#2021-02-2221:58seancorfieldYour comment about Malli embracing coercion would make me want to avoid it -- we went down that path with our own library and it was a mistake, hence our shift to Spec + coax now.#2021-02-2221:59borkdude@seancorfield Can you explain why it was a mistake?#2021-02-2222:00seancorfieldComplecting coercions with validations is just messy and it can hide errors.#2021-02-2222:00seancorfieldOur specs are simpler now, and don't need custom generators as often.#2021-02-2222:01seancorfieldThe coercions are clear and separate. We can test coercing code and validating code separately.#2021-02-2222:02seancorfieldI was one of those arguing in favor of having specs that coerced values for a long time. I was wrong about that.#2021-02-2222:02borkdudeI would love to add spec to babashka btw. But I don't feel certain about the alpha suffix and where it's going next and how long it's going to be. It is a recurring question from users. Since it's part of the current clojure people sometimes expect it to be just there. clojure.test.check is already in there, as a preparation step#2021-02-2222:04borkdudeI would also love to hear some insights from @ikitommi about the coercion philosophy in malli. Maybe it's one of those easy vs simplicity things, where a lot of people really just want the easy?#2021-02-2222:09borkdude@seancorfield Do you use clojure.spec to validate incoming web requests, in an open world way? @dominicm recently pointed out that this can be a security issue. This problem will be solved with spec2 which supports closed.#2021-02-2222:15seancorfieldI disagree that it's a problem with spec. If you have a context where you must only accept a certain subset of keys, that's what select-keys is for.#2021-02-2222:16seancorfieldWe use spec as the "source of truth" for various things and we have situations where we derive the set of keys from the spec and then either explicitly restrict the data we process to that subset or we simply trim the keys present down to that subset, depending on whether we want to flag unwanted keys or not. But that trimming/checking is independent of how we validate incoming web requests.#2021-02-2222:18borkdudeIf it's not a problem, then why is spec2 introducing closed specs again? :thinking_face:#2021-02-2222:19seancorfieldBecause people whined about it not being in Spec 1 šŸ™‚#2021-02-2222:20seancorfieldYou could certainly do closed spec checking with Spec 1 -- you just have to do it manually.#2021-02-2222:20borkdudeI think what @dominicm was hinting at was that when you allow any spec to be validated in a web request, this may trigger functions you did not expect to be executed, or something, which may lead to DoS, or whatever#2021-02-2222:21seancorfieldThat's a lot of hand-waving šŸ™‚#2021-02-2222:21borkdudeI was asking questions, I did not take a position in this. Just wondering if you had considered it#2021-02-2222:22seancorfieldI think it's a strawman argument to "blame" spec for something like that, frankly.#2021-02-2222:22borkdudeok#2021-02-2222:24seancorfieldIf he has a specific, realistic scenario where just using Spec 1 to validate an incoming web request would cause a DoS, I'd be very interested to see that.#2021-02-2222:29seancorfieldThis is what Dominic said in #yada when asked why Yada does not have integration with Spec: "Because it gives attackers access to all your spec keys. Some of which might be very slow (and not run in production in normal circumstances). This opens you up to DOS attacks, the guidance from Alex on this was to validate data before passing it to spec. On top of that, you usually want to restrict keys in the web context as you're going to pass them into the database." -- I've covered the latter part (above) and I would expect in real-world code what you put in the database isn't necessarily going to be anything like the set of keys that you get passed in a web request because the persistence model and the API model aren't likely to be a 1:1 match anyway.#2021-02-2222:51colinkahnTried following the link to the yada channel to see the conversion but perhaps it happened awhile ago. Curious what validation was suggested before passing to spec.#2021-02-2223:05seancorfieldI didn't dig far enough into the Zulip archives to see what Alex had actually said about that -- I'm pretty sure Alex didn't literally say folks should "validate data before passing it to spec" but he has talked about doing coercion on input data before passing it to spec (a separate thing).#2021-02-2223:06seancorfieldSlack has a 10,000 message limit on the free plan but most channels are mirrored to Zulip http://clojurians.zulipchat.com which has an unlimited archive and search facility (for the open source plan we're on there).#2021-02-2223:15colinkahnGot it thanks šŸ‘#2021-02-2222:30robertfwI've had cases where the data I wanted to spec had to enforce a closed nature, namely, working with an HTTP API that throw an error if you gave it data it wasn't expecting. In that case we want a spec that validates that we're not including anything extra. The same would apply for a spec describing incoming data on a system that likewise wants to error out if the user sends unexpected fields (e.g., so users get warnings if they typo a parameter name or send data they think is doing something when it is actually being ignored)#2021-02-2222:31borkdudeI guess you could ask the question: why should web APIs validate differently than functions?#2021-02-2222:31seancorfieldI don't find the first part very convincing: if you have "very slow" specs that don't run as part of production code, why have them in your production code at all? Why not have them in test code (and if they're "very slow" they're not even going to be part of your unit tests).#2021-02-2222:34seancorfieldAs for Robert's case: I covered that above -- if you do want to fail validation on additional/unknown keys, that is possible with Spec 1. It's just not directly built in. I don't see Spec 2's "closed checking" as any sort of admission that Spec 1 was wrong -- I see it much more as a convenience that lets you write a little less code in certain situations. I think most people made a big ol' mountain out of wanting closed specs when it's really just a molehill.#2021-02-2222:34seancorfield(I think this is absolutely one of those cases where people pushed for "easy" rather than "simple")#2021-02-2222:35borkdudeWell, sometimes UX matters#2021-02-2222:36ikitommi@seancorfield just to clarify: malli separates coercion from validation, for the reasons you mentioned. You first "coerce" a value, then validate and explain (and humanize the explanation) if needed.#2021-02-2222:38seancorfieldGood to know -- that wasn't clear to me from @alex.joseph.whittā€™s comment.#2021-02-2223:51Alex WhittWell, I don't know if we're any closer to a decision... so many good points brought up on both sides. I'd love to know if the Cognitect team ever considered implementing spec with a data-driven architecture like Malli's, and if so, what factors led to deciding against it. For some context, I've been an avid spec user for years, and only recently became aware of Malli. As we're starting a new project, I guess I'm experiencing some FOMO regarding some of Malli's features, but I can't escape thinking that a lot of Spec's design decisions were really on-point and forward-thinking.#2021-02-2300:04Alex Miller (Clojure team)I think Rich would claim that spec forms are data :)#2021-02-2300:05Alex Miller (Clojure team)But spec 2 already has an alternative map-based data format (which is heavily subject to change)#2021-02-2300:09Alex WhittThat thought has occurred to me as well... I think we're conditioned to think that "data" means "EDN," but quickly forget that this is a LISP.#2021-02-2309:45mpenetspec is generally very opinionated, but once you "get" the design choices it's hard to be in disagreement with it. It fits quite well with datomic, "Maximal Graph" & co approach to dealing with data, which imho is spot on (I am not a datomic or pathom user but I agree with the general ideas). Right now it's also quite bare bones but there are a few libraries that fill the important gaps (coercion, error messages, etc). Then for me at least spec2 would fix most of the important problems I have with v1 (mostly schema+select, or whatever these will be called), but even without spec1 is very usable. Most of the problems stated about spec 1 imho are very often overblown or just misunderstanding on how to use spec or its goals. But I get the frustration of malli authors who wanted spec2 now (I do too, but I rather have something carefully designed than a rushed draft) and the amount of work they poor into malli earns respect. Personally I do not agree with some of their design choices and most importantly the fact this will cause yet more fragmentation in that space. I would rather have had all that fire power put in spec libraries. I also had to deal with code bases that invested in spec-tools and broke some teeth in the process, so I prefer to be very conservative when I choose a library to fill that spot now.#2021-02-2309:50mpenetabout closed specs & data for me these are quite minor issues (if at all), I had to generate specs from another dsl on work I did recently and it's not a big deal at all. For more advanced composition I can't remember a case where i was blocked by the current api, in extreme cases you can rely on eval/macros to get to what you want but it's extremely rare in my experience at least. That said if spec2 improves that, great. For closed specs it's the same, writing a strict-keys version of s/keys is quite easy a naive version with poor error reporting is a couple of lines and a more involved one with nice error messages more work but it's under 50 loc#2021-02-2309:53mpenethaving multiple choices might be good, that's could be a sign we have a "healthy" community#2021-02-2310:47borkdudeAbout the data driven nature of malli vs spec: I think it's fair to say that malli schemas are a better fit if you want to have a programmatic way of creating/changing schemas (in spec you end up with macros for this usually) and when you want to have (de)serialization of your schemas. That's not to say that you can't do this with spec, but the UX of malli is probably better for this kind of use.#2021-02-2317:33seancorfield@mpenet Thank you for being so eloquent about this issue -- like you, we haven't found Spec 1 to be limiting and we've been using it heavily in production code ever since it appeared. And thank you for Coax which is allowing us to retire our coercing "web specs"! For a while we tracked Spec 2 at work, on a branch, and the data representation and programmatic Spec construction is pretty nice and it seems to address the concerns folks have expressed about the difficulty of building Spec 1 specs.#2021-02-2317:33seancorfield@mpenet Thank you for being so eloquent about this issue -- like you, we haven't found Spec 1 to be limiting and we've been using it heavily in production code ever since it appeared. And thank you for Coax which is allowing us to retire our coercing "web specs"! For a while we tracked Spec 2 at work, on a branch, and the data representation and programmatic Spec construction is pretty nice and it seems to address the concerns folks have expressed about the difficulty of building Spec 1 specs.#2021-02-2310:52Eamonn SullivanI've been using clojure a lot lately to transform or combine data (usually from one or more REST endpoints) from one form into another. I am using spec the same way I would have used case classes in scala or json schemas -- to validate the input and output (often exactly that, with :pre and :post conditions). I'm assuming that I'm staying away from anything controversial and that it will be easy to change these to spec2. But that may be an incorrect assumption. I'm sticking with the "official" one only because I'm so new (I really only use Clojure one or two days a week) that I'm not experienced enough to judge third-party libraries. I'm just trusting the manufacturer here.#2021-02-2310:57borkdude@eamonn.sullivan The clojure.spec.alpha lib will be around probably forever, so even if spec2 gets bundled with core, you will still be able to use the old one#2021-02-2310:58borkdudeThe main difference in spec1 and spec2 is how s/keys works, this will be replaced with s/select (this is probably an over-simplification as I haven't been following spec2 for a while - I would love to get some updates!)#2021-02-2313:11Alex Miller (Clojure team)https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha is the overview#2021-02-2313:11Alex Miller (Clojure team)s/keys actually works the same for now but will be superseded by s/schema and s/select https://github.com/clojure/spec-alpha2/wiki/Schema-and-select#2021-02-2311:32lassemaattaregarding the discussion about the strict separation of coercion & validation: are there any public non-trivial repos/projects showcasing this principle in practice (e.g. coercing and validating http requests/parameters etc)?#2021-02-2317:35seancorfieldI fear we are in another one of those situations where the non-trivial codebases using Spec heavily are all in-house/proprietary:
Clojure source 348 files 87710 total loc,
    3532 fns, 896 of which are private,
    556 vars, 29 macros, 93 atoms,
    849 specs, 33 function specs.
Clojure tests 378 files 23192 total loc,
    4 specs, 1 function specs.
That's a quick break down of stats for our codebase at work.
#2021-02-2318:29Alex WhittWe decided to go with spec, by the way. We're betting that the number and quality of libraries in spec's orbit will explode when the official release drops, seeing as it will be so much easier to interact with specs programmatically. And regardless of the ecosystem of the future, it has what we need right now. Malli is doubtless pushing the state of the art forward, but I have no doubt that many people will want to incorporate features and discoveries Metosin makes along the way into a spec-based architecture.#2021-02-2321:52vlaaad
(s/form (s/keys* :opt-un [:clojure.core.specs.alpha/exclude :clojure.core.specs.alpha/only :clojure.core.specs.alpha/rename]))
=> (clojure.spec.alpha/&
    (clojure.spec.alpha/*
     (clojure.spec.alpha/cat
      :clojure.spec.alpha/k
      clojure.core/keyword?
      :clojure.spec.alpha/v
      clojure.core/any?))
    :clojure.spec.alpha/kvs->map
    mspec__2546__auto__)
How do I introspect s/keys* spec? form says nothing about actual keys...
#2021-02-2321:55vlaaadfound it...
(-> (s/keys* :opt-un [:clojure.core.specs.alpha/exclude :clojure.core.specs.alpha/only :clojure.core.specs.alpha/rename]) :ps second s/form)
=> (clojure.spec.alpha/keys
    :opt-un
    [:clojure.core.specs.alpha/exclude :clojure.core.specs.alpha/only :clojure.core.specs.alpha/rename])
that looks very unreliable though... can I rely on regex ops data shape?
#2021-02-2321:57Alex Miller (Clojure team)what you need are specs for spec forms themselves - then you can conform the spec and get a map (you can also modify that map and s/unform it back to a spec form). some work on this in https://clojure.atlassian.net/browse/CLJ-2112 (but likely we'll never actually finish this as spec 2 will make it unnecessary)#2021-02-2322:10vlaaadWell, the form of s/keys* has no useful information at all if you look at the output#2021-02-2322:11vlaaadI.e. there are no mentions of actual expected keys#2021-02-2322:51Alex Miller (Clojure team)yeah, this has to do with the mixture of symbol / object in spec 1 which is almost entirely cleaned up in spec 2#2021-02-2410:50ikitommiabout the slow specs:
(time
  (s/valid?
    (s/keys)
    {:clojure.core.specs.alpha/ns-clauses (repeat 100000 (list :gen-class))}))
; "Elapsed time: 5069.738452 msecs"
; => true
we added auto-closing of api specs to reitit (using spec-tools) to avoid this.
#2021-02-2611:17jumarI know that you have to call stest/instrument again when you change the function definition because instrument wraps the original function and replaces the var root - when you redefine the fn, it's no longer instrumented. But I thought that just changing the fdef definition would be fine (no need to call instrument again) because the spec is hold in global registry. But any change inside fdef still requires calling instrument again - is this because the spec fn wrapper captures the fdef spec value as it was at the instrument was called?#2021-02-2611:25jumarAfter looking at spec code, I think the "problem" is that it doesn't call get-spec at the time the function is invoked but rather at the time when instrument is called and thus it only captures the spec as it was at that time - no later redefinitions are taken into account.#2021-02-2613:33Alex Miller (Clojure team)yep, exactly - this is a performance optimization#2021-02-2613:33Alex Miller (Clojure team)typical tradeoff of perf vs dynamicity :)#2021-02-2618:09seancorfield@jumar What I tend to do in situations like that is put the stest/instrument call in my test namespace as a top-level form (after ns but before any tests) so that the tests run with up-to-date instrumentation. If you want to do it in a source file, you could always have something like
(comment
  (do
    (stest/instrument `my-fn)
    (my-fn 123)) ; eval this form with both the instrument call and the my-fn call
  ,)
#2021-02-2706:14jumarFor us the most common case is working on an app in the REPL and changing functions or specs. Then you forget to call instrument and you either realize pretty late or not at all that your spec is broken or some data arenā€™t really passed in. We do have a decent test suite but not everything is covered so sometimes itā€™s only another developer who catches the issue#2021-02-2706:33seancorfieldWhen I'm writing Specs, I always have an RCF with s/exercise calls to test the specs. I also tend to write a stub -test ns when I write code and then I can run the tests in it with a hot key while I'm in the source file. The key is keeping a short feedback cycle and making sure you can run "tests" or at least "sanity check expressions" easily via hot keys while you are writing code. "forget" isn't an option: write the code so you can't forget (like the do form I suggested).#2021-02-2716:46uwo@U04V70XH6 Would love to see a youtube video about the spec dimension of your development workflow, if you ever feel like it. I remember enjoying the demonstration of your REBL workflow#2021-02-2717:44seancorfield@U09QBCNBY Noted. I'm not sure when I'll get back to making videos -- I really hate the process (and the medium): I much prefer prose and code to videos.#2021-02-2618:51borkdudeI also have a couple of helper macros here for in/unstrumentation: https://github.com/borkdude/respeced#with-instrumentation https://github.com/borkdude/respeced#with-unstrumentation They ensure the desired state in the body, but will restore the state like it was afterwards#2021-03-0215:45mpenet
(defrecord Foo []) 
(s/valid? (s/coll-of any?) (Foo.))
boom - Can't create empty: user.Foo
#2021-03-0215:46mpenetseems like a bug#2021-03-0215:46Alex Miller (Clojure team)it is, we have a ticket for it#2021-03-0215:46mpenetyou can avoid it using :into coll?#2021-03-0215:46mpenetok#2021-03-0215:47Alex Miller (Clojure team)it's one of those things that seems obvious but is way more subtle than it seems in the impl.#2021-03-0215:48Alex Miller (Clojure team)https://clojure.atlassian.net/browse/CLJ-2481#2021-03-0215:48mpenetrelated too https://clojure.atlassian.net/browse/CLJ-1975#2021-03-0215:48Alex Miller (Clojure team)yeah, it's a dupe of that#2021-03-0216:09mpenetwould setting a default for :into at coll-of level to coll? be bad (I mean in the impl itself)?#2021-03-0216:10mpenetI get the fact that every-impl is shared quite a bit so that can't be baked in at this level#2021-03-0216:11mpenetlike here https://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/alpha.clj#L592 , I guess because "performance"#2021-03-0217:39Alex Miller (Clojure team)very hard to say without a lot of thinking#2021-03-0217:39Alex Miller (Clojure team)and I rewrote half of that stuff in spec 2#2021-03-0504:08tianshuWhat is the stage spec2 working on? If the design is finished, is there anything we can help?#2021-03-0504:23Alex Miller (Clojure team)Rich is working on the design for function specs#2021-03-0616:50teodorluHow is https://clojuredocs.org/clojure.spec.alpha/fdef not good enough?#2021-03-0616:53borkdude@teodorlu I think the spec2 function spec is related to some kind of notation that alludes to defn#2021-03-0616:54borkdudesomething like the plumatic/schema defn macro (but very different, because it's not schema)#2021-03-0616:55teodorluAh, OK, so looking into "a single form that both defines a function and specifies its inputs" rather than a "normal defn and something additional"?#2021-03-0616:55borkdudeWe will only know when it comes out#2021-03-0616:56teodorluThanks!#2021-03-0617:06Alex Miller (Clojure team)correct#2021-03-0617:07Alex Miller (Clojure team)it's not just "putting the current function spec in defn", but trying to reach quite a bit beyond that into some new ideas, and it's taking a long hammock to sort through all the tradeoffs there#2021-03-0911:30AronI wish I could be part of this hammock thing, not constantly just sometimes, I feel I could contribute, especially since I have at least 3 projects on hold for years now since I've been waiting spec2#2021-03-1017:13ikitommimore discussion about the slow spec1 specs & external input: https://www.reddit.com/r/Clojure/comments/m0c64g/clojurespec_and_untrusted_input/#2021-03-1114:16mokrQuick question before I resort to rewrite my code: Is there something inherently wrong with trying to keep code DRY by extracting values directly from a spec like this?
(ns my.specs  ;; cljc-file
  (:require [clojure.spec.alpha :as s]))

(s/def :select/legal-values #{1 2 3 4})


(ns my.ui    ;; cljs-file
  (:require [clojure.spec.alpha :as s]
            [my.specs]))

(def dropdown-config [{:id      :select-legal-values
                        :tooltip "Select your value"
                        :values  (s/form :select/legal-values)},,,])
This only illustrates the part where I retrieve values from this spec. Data verification is elsewhere. What makes me ask is that Iā€™ve had this working in my Luminus+Shadow-cljs+re-frame app, but as soon as I upgrade shadow-cljs, my project wonā€™t load in the browser. From e.g. ā€œReferenceError: spec is not definedā€ I get the impression that the spec might not be ready yet. I donā€™t think it is shadow-cljs that is to blame as I encountered the same issue when upgrading yesterday as well, but after cleaning and trying a few times it suddenly started working. Maybe I got lucky with a race condition or somethingā€¦ Anyone?
#2021-03-1114:22Alex Miller (Clojure team)Yeah, this wonā€™t work in spec#2021-03-1114:23Alex Miller (Clojure team)Oh wait, Iā€™m misreading#2021-03-1114:24Alex Miller (Clojure team)Might be a bug in spec emitting the wrong namespace on the spec form? We have fixed a few things like that#2021-03-1114:26Alex Miller (Clojure team)Iā€™m keying off of ā€œspecā€ there - not sure if thatā€™s a hard coded word or a ns from data#2021-03-1115:15mokrBased on the error I donā€™t get the impression that there is a wrong namespace issue, but itā€™s not that informative really. Itā€™s often a bit hard to reduce the actual code down to a suitable snippet that captures to issue. In this case I would say the gist is: Can a spec from a required namespace be used in a normal def that calls s/form to extract the ā€œvalueā€ from that spec. Is the spec ready for value extraction at the time of def evaluation? Even though I had this working Iā€™ll just rewrite it as it seems a bit brittle. For instance IIRC I had to put the set directly inside the s/def, as s/form did not work if the spec used a var containing the set like this: (def my-set #{1 2 3} (s/def :foo/bar my-set) (def boom (s/form :foo/bar) I guess I might be abusing spec a bit, or relying on functionality that is not suitable for that purpose.#2021-03-1115:21mokrA quick rewrite where both the spec and my config refers to the same vars/defs resolved the issue. Still DRY, a few more lines of code, less ā€œcleverā€, but maybe easier to understand and less brittle. šŸ™‚#2021-03-1115:41Alex Miller (Clojure team)some of the fiddliness of this is improved in spec 2#2021-03-1117:25mokrSounds great. I was holding back on spec for a long time waiting for spec 2 šŸ˜‰ But, I understand and appreciate the attention to detail that goes into the process.#2021-03-1119:43vlaaadAnyone knows if there is a library to convert json schema to spec definitions?#2021-03-1120:02robertfw@vlaaad I've looked for this previously (and for the reverse transformation) but came up empty. luposlip/json-schema is the library I used for validation, and it seems to have sprouted schema inference since I last looked through its docs, that may not be useful in its exact form but could have some useful code within#2021-03-1815:35Eamonn SullivanHere's a question that'll probably out me as someone who doesn't understand the concept: Is is possible to spec a "closed" map, one that has some number of required and optional keys, but no other keys?#2021-03-1815:36Eamonn SullivanI'm trying to translate the JSON schema concept of
additionalProperties = false
#2021-03-1815:39Eamonn Sullivanprobably s/map-of #{...} will try that.#2021-03-1815:44Eamonn Sullivannope#2021-03-1815:44Eamonn SullivanI guess that's a spec2 thing?#2021-03-1815:45borkdudeThis will be supported in spec2, malli already supports it#2021-03-1815:46borkdudeyou could also try to do this using a predicate, (s/and ... (fn [map] check-the-keys))#2021-03-1815:46Eamonn SullivanšŸ’” Thank you!#2021-03-1815:55Alex Miller (Clojure team)the important part of our perspective in spec 2 is that we think that "closed" should be a property of the check, not a property of the spec#2021-03-1815:56borkdudegood point!#2021-03-1815:57borkdude@U064X3EF3 Are you considering doing some journal updates again in the future? I for one really enjoyed them.#2021-03-1815:58Eamonn SullivanI think I see. So, I have an API that's dumb as a bag of rocks and will spit out any blob of json that includes anything other than, say, five keys. There are many combinations of those that are allowed, but it will throw a hissy fit if a key isn't one of those five.#2021-03-1816:00Eamonn SullivanOnly one of them is required and the rest optional, which all fit nicely into (s/keys :req-un [,,,] :opt-un [,,,]) until I did a typo in one of the opt-uns.#2021-03-1816:01Eamonn Sullivanpasses the spec; fails in life.#2021-03-1816:29vemv> should be a property of theĀ check, not a property of theĀ spec (FWIW!) while one can see the value of this, it's also hard to see how this would not conflict with DRY. If one tried to DRY out n identical calls as a reusable "thing", what would be that thing be? It seems to me that inherently, the 'thing' would effectively become a spec (whether it's an actual spec, or a de-facto spec: e.g. a defn)#2021-03-1816:31Alex Miller (Clojure team)makes sense to me#2021-03-1817:00Eamonn SullivanActually, this was a bit too easy. Must have missed something?
(s/def ::environments #{:int :test :stage :live})
(s/def ::valid-top-level-keys #{:name :description :values})
(s/def ::name string?)
(s/def ::description string?)
(s/def ::values (s/map-of ::environments string?))

(s/def ::config (s/and
                 (s/keys :req-un [::name]
                         :opt-un [::values ::description])
                 (s/map-of ::valid-top-level-keys any?)))

(s/def ::release-config (s/coll-of ::config))

(comment
  (s/valid? ::release-config [{:name "something" :description "some description"}])
  (s/valid? ::release-config [{:name "something" :desciption "some description with typo"}])
  )
#2021-03-1817:00Eamonn SullivanI was already doing something similar with the :environments spec.#2021-03-1921:48pbrownHey there -- I really like this idea of closing specs at check-time in spec 2. Will there be (or is there already) a way to apply the same idea to sequential specs? Let's say I'm speccing hiccup as
(defmulti form-of first)
(defmethod form-of :default [_]
  (s/cat :tag keyword?
         :props (s/? map?)
         :children (s/* ::node)))
(s/def ::component (s/multi-spec form-of (fn [val _tag] val)))
(s/def ::node
       (s/or :component (s/and vector? ::component)
             :list (s/coll-of ::node
                              :kind (every-pred sequential?
                                                (complement vector?)))
             :atom (complement coll?)))
and define a bunch of components like
(defmethod form-of :script [_]
  (s/cat :tag #{:script}
         :props (s/? map?)
         :children (s/* string?)))
Could I later restrict ::component somehow to reject [:script ...] as part of the instruction to s/valid?? Or could my defmethods check a dynamic binding, say *illegal-tags*?
#2021-03-2315:01ognen.ivanovskiHi, I was wondering, when one has a repeating spec (e.g. non-empty string, something like a type) in spec2, what is more idiomatic: a) simply define the spec behind a keyword, e.g.
(s/def ::non-empty-string (s/and- string? (comp not str/blank?)) 

(s/def :foo/bar ::non-empty-string)
or b) make non-empty-string a spec op, e.g.
(s/defop non-empty-string [] (s/and- string? (comp not str/blank?)))

(s/def :foo/bar (non-empty-string)) 
#2021-04-0222:21Michael Stokley
(s/def :corp/employee
  (s/keys :req [:employee/first-name]))

(s/def :employee/first-name string?)

(s/def :favorite/ice-cream #{:chocolate :vanilla})

(e/expound :corp/employee {:employee/first-name "michael"
                           :favorite/ice-cream :strawberry})

;; -- Spec failed --------------------

;; {:employee/first-name ...,
;;  :favorite/ice-cream :strawberry}
;; ^^^^^^^^^^^

;; should be one of: :chocolate, :vanilla

;; -- Relevant specs -------

;; :favorite/ice-cream:
;; #{:chocolate :vanilla}
;; :corp/employee:
;; (clojure.spec.alpha/keys :req [:employee/first-name])

;; -------------------------
;; Detected 1 error
I find this pretty surprising
#2021-04-0222:21Michael Stokleygiven that the :corp/employee spec doesn't say anything about ice cream#2021-04-0222:23seancorfieldThat is by design in Spec ā€” and the reference docs on http://clojure.org talk about that. I think the rationale may do too.#2021-04-0222:24Michael Stokleythanks Sean - i'll take a look#2021-04-0222:25seancorfieldhttps://clojure.org/guides/spec#_entity_maps says ā€œWhen conformance is checked on a map, it does two things - checking that the required attributes are included, and checking that every registered key has a conforming value. Weā€™ll see later where optional attributes can be useful. Also note that ALL attributes are checked via keys, not just those listed in the :req and :opt keys. Thus a bare (s/keys) is valid and will check all attributes of a map without checking which keys are required or optional.ā€#2021-04-0222:29Michael Stokleyyeah, there it is. checks that every registered key has a conforming value. good to know#2021-04-0518:35dvingoHow do you deal with creating specs for maps whose keys require differing semantics based upon the context in which they are used? Use case, of two contexts for modeling a 'task': 1. UI input form of a domain entity. 2. DB version of a domain entity. The description must be a certain length when persisting to the DB: (s/def :task/id uuid?) (s/def :task/description (s/and string? #(> (count %) 100))) In a UI form we allow the description to be empty: (s/def :task/id uuid?) (s/def :task/description string?) (s/def ::task (s/keys ::req [:task/id :task/description])) These are obviously in conflict, but given the use case they are both valid in differing contexts. How does one design a solution using spec (alpha) for this use case?#2021-04-0518:38localshredUsually we'll just specify ns specific s/def's, since the context is different for the data. ::task-id instead of :task/id#2021-04-0518:39localshredsince the registry is global there's not many other options we could come up with that preserved the exact same namespaced key#2021-04-0518:39localshredyou can use (s/or ... and tag the different cases, then when conforming verify it's the case you're expecting#2021-04-0518:41localshred
(s/def :task/description
  (s/or
    ::db (s/and string? #(> (count %) 100))
    ::ui  string?)
#2021-04-0518:45dvingoI landed on something similar - saying the description could always be empty and then checking the DB requirements in the map spec - which knowingly goes against the design of spec in that the entity map is now aware of the semantics of its keyset#2021-04-0518:56localshredright, it gets a little muddy#2021-04-0518:42localshredthis has a different set of tradeoffs since you're now joining db and UI validation into a single spec in some shared ns... ultimately trade-offs you need to decide on#2021-04-0518:44dvingothanks for the quick reply - so in the ::task-id case your map data would not have :task/id instead it would have ns/task-id? and then at some point you transform to :task/id format before persisting?#2021-04-0518:53localshredcorrect#2021-04-0518:54localshredfor us we've kept the UI specs in ui namespaces, and the db specs correlating to the datomic keys we ultimately store them under#2021-04-0518:55localshredso the translation from front to backend keys occurs at the http layer#2021-04-0520:15dvingoOk thanks for the info, good (err bad) to hear that others have the same problems#2021-04-0520:22em@danvingo What about multi-specs? https://clojure.org/guides/spec#_multi_spec Separating your task specs into various domains is entirely okay, but if you want to keep the same structure but have different contexts, just reifying that context with a keyword tag (or however else you want to do it, multimethods are open) keeps things fairly simple and open.#2021-04-0520:25dvingoI believe the core of the problem is that I want to use the kw :task/description and have its spec be different based on some context - I don't think mutli-specs would alleviate that problem#2021-04-0520:45emIt's one of the only solutions for your exact problem statement - keeping everything else about your spec the same, but having it be treated differently depending on context. You would inject context in the map wherever you needed it, and update that context at the boundary (when submitting from frontend to DB, for example). Injecting is literally an assoc with an extra keyword, and the multi-spec just checks that context.#2021-04-0521:02dvingo
(s/def :task/type keyword?)
(s/def :task/description string?)

(defmulti task-type :task/type)

;; in this "context" I only care that :task/description is a string.
(defmethod task-type :ui/task [_]
  (s/keys :req [:task/type :task/description]))
  
;; in this "context" I want :task/description to be a non-empty string of a certain required length.
(defmethod task-type :db/task [_]
  (s/keys :req [:task/type :task/description]))
I don't see how multi-methods make this tractable. I think the answer above is the point - I'm misusing the tool fundamentally, as it is designed.
#2021-04-0521:22emYeah, on further thought you're totally right, my bad. Multi-specs are one of the only dynamic dispatch mechanisms for specs, but only really in the context of maps. My envisioned solution is too clumsy, as you'd need to change :task/description to be a map, and not a raw value. It'd work, but for every "context-variable" attribute you'd need to do a lot of extra work injecting context, as you couldn't do it at the top level. Technically, in accordance to the design and intention of spec, you'd want the information determining the specification of an attribute at the level of that attribute, so it makes sense why multi-specs work pretty much only on maps, as they can contain such extra information.#2021-04-0520:29dvingoit seems like spec's answer to this is that you should create two specs under two different names, which: 1. feels like a bad idea - naming things are hard enough already. 2. If I have existing code and want to add specs for it, this requires me to change the names of my existing codebase, also tough decision to justify#2021-04-0520:31Alex Miller (Clojure team)You are indeed at cross purposes to the principles guiding spec #2021-04-0520:32Alex Miller (Clojure team)But this is why spec requires qualified names - the qualifier exists to disambiguate context#2021-04-0520:35Alex Miller (Clojure team)https://clojure.org/about/spec is worth reading for the big picture#2021-04-0520:35dvingoQuestions: 1. What is its purpose then? 2. What is the recommended approach to solve this problem. (I can acknowledge my data design skills are subpar and I should have thought about better names up front, but one of the things I like about clojure is the iterative design it enables when discovering a domain. And well, it's too late to go back now and rewrite that name all over a large system).#2021-04-0608:50Aroni have a similar problem to yours on the frontend, when i want to write a form generator, I have to keep 3 different specs for the same entity 1. the remote state specs that the backend API expects to get when I put or post or update the resource 2. the ui state specs which contains invalid data that was inputted by the user and is fully controlled by the user, but it still needs to be validated and used 3. my local display state specs about the inputted resource that holds stuff like a list of strings to shown in a predictive input while the user is typing their stuff it's all in the same Entity! the only solution is to accept that actually no, these are separate entities and require separate specs with separate names. similarly in your case, if your context is different then you need a new spec#2021-04-0520:36dvingothanks, I've read it a few times. This is a real world problem that I don't have a good answer to at the moment and seems like other people do as well.#2021-04-0520:37Alex Miller (Clojure team)Given that you are at cross purposes with intents, there are no easy answers (well, donā€™t use spec is easy)#2021-04-0520:38Alex Miller (Clojure team)But given that qualified names have global meaning, you need to spec the union of possible global shapes#2021-04-0520:38dvingoCross purpose how? I think that's what I'm struggling to understand#2021-04-0520:39Alex Miller (Clojure team)Spec wants you to assign a spec with global meaning to qualified names#2021-04-0520:39Alex Miller (Clojure team)That is, specs are not contextual#2021-04-0520:40dvingoI see this makes more sense now#2021-04-0520:41Alex Miller (Clojure team)Unqualified names are assumed to be contextual and can be somewhat handled in s/keys with :req-un and :opt-un with contextual specs#2021-04-0520:41Alex Miller (Clojure team)(to some degree)#2021-04-0520:45dvingoperhaps the wording here should change: > These maps represent various sets, subsets, intersections and unions of the same keys, and in general ought to have the same semantic for the same key wherever it is used From "in general ought to" to "will always"#2021-04-0520:45emIt's one of the only solutions for your exact problem statement - keeping everything else about your spec the same, but having it be treated differently depending on context. You would inject context in the map wherever you needed it, and update that context at the boundary (when submitting from frontend to DB, for example). Injecting is literally an assoc with an extra keyword, and the multi-spec just checks that context.#2021-04-0520:46dvingocould you provide an example?#2021-04-0521:33dvingoI have come to see the problem with my :task/description example. task is not a useful namespace and should be updated to be globally unique. Perhaps it would be useful to add to the guide some suggestions on use and data design in Clojure - as well as some anti-patterns. I think for small apps (which large apps usually start off as) it is common to use namespaces from the domain, but which are not globally unique, such as task instead of com.myorg.myapp.task and then as the app grows you get into problems such as the above ui/db distinction but now you have that too-simple namespace all over your code and probably persisted in storage and you have a tough decision to make on how to refactor it. The guide actually includes these simple names in some places (`:animal`, :event) which I think helps encourage the notion that this is a good idea, even though it is directly against the rationale: https://clojure.org/about/spec#_global_namespaced_names_are_more_important In my specific case I am thinking, if I wish to continue using spec, the design that is aligned with spec's design would be to have: :com.myorg.myapp.task.ui/description and :com.myorg.myapp.task.db/description or similar names. Thanks for the feedback and discussion. Things are making more sense now.#2021-04-0618:34marciolSeems that this is the correct way of use. If you think about :task/description and handle it differently depending on the context you can somewhat get in trouble because the context sometimes is not devised clearly. So when you start to specify directly these keys each in their namespace, you are giving them a clear and unambiguous definition, so that you can even if needed to combine these two keys in the same map (keyset). The only extra burden is to handle the encoding of data to an external medium when all this meaningful context will be lost, but for it, I'd expect some convention established by the team in charge of such system.#2021-04-0618:50marciolAnd following the accretion paradigm, you can create a new namespace when your system grows, and eventually remove the old namespace, without any conflict or problems with contextualization, etc.#2021-04-0521:38Alex Miller (Clojure team)the guide is intended to teach spec concepts so aims for easier names, but I agree this could probably be better to align with intent. issue welcome at https://github.com/clojure/clojure-site/issues#2021-04-0521:46dvingoagreed - when starting out the simple namespaces are great - but then there's no "advanced usage" or "scaling issues" or similar for perusing when you're in the intermediate stages of spec use. Thanks - I'll add an issue#2021-04-0618:34marciolSeems that this is the correct way of use. If you think about :task/description and handle it differently depending on the context you can somewhat get in trouble because the context sometimes is not devised clearly. So when you start to specify directly these keys each in their namespace, you are giving them a clear and unambiguous definition, so that you can even if needed to combine these two keys in the same map (keyset). The only extra burden is to handle the encoding of data to an external medium when all this meaningful context will be lost, but for it, I'd expect some convention established by the team in charge of such system.#2021-04-0618:50marciolAnd following the accretion paradigm, you can create a new namespace when your system grows, and eventually remove the old namespace, without any conflict or problems with contextualization, etc.#2021-04-0608:54Aronis "specs are not contextual" the same as "if your spec depends on context, you have more than one specs"?#2021-04-0609:17borkdude@ashnur spec 2 makes selection contextual (e.g. the combination of required keys)#2021-04-0609:18borkdudebut the keys themselves are always globally described#2021-04-0609:23Aronnot allowed to use spec2 in production yet afaik#2021-04-0609:26borkdudetrue, for practical purposes it doesn't really exist yet, but this is one of the biggest issues they wanted to address I believe#2021-04-0609:32Aroni am not sure it's the same issue#2021-04-0609:33Aronthe same logic goes for selections as well, if your selection depends on the context, isn't it true you have more than one selections?#2021-04-0609:35borkdudeyes, so?#2021-04-0609:36Aroni am good šŸ™‚#2021-04-0618:57Alex Miller (Clojure team)you don't even have to remove the old one :)#2021-04-0618:59marciolexactly#2021-04-0619:02marciolIt's a paradigm shift and sometimes takes time to finally get it.#2021-04-0910:25Matti UusitaloBackground: I want to create a spec which can validate values in different branches of a document in a manner that what I get out of spec/explain-data points to the right value. For example:
{:min 0 :max 10 :values [0 1 2 3 11 2]}
I want the spec to be able to say that because 11 is not between the values given in :min and :max, it is in error. Iā€™m trying to implement the Spec protocol to add a wrapper for a spec which works like this
(defmacro value-binding [pred bind]
    `(let [wrapped# (delay (#'spec/specize ~pred))]
       (reify
         spec/Specize
         (specize* [s#] s#)
         (specize* [s# _] s#)

         spec/Spec
         (conform* [spec x#]
           (binding [~bind x#]
             (spec/conform* @wrapped# x#)))
         (unform* [spec y#]
           (binding [~bind y#]
             (spec/unform* @wrapped# y#)))
         (explain* [spec# path# via# in# x#]
           (binding [~bind x#]
             (spec/explain*
              @wrapped#
              path# via# in# x#)))
         (gen* [spec overrides# path# rmap#]
           (spec/gen* @wrapped# overrides# path# rmap#))
         (with-gen* [spec gfn#]
           (spec/with-gen* @wrapped# gfn#))
         (describe* [spec]
           (spec/describe* @wrapped#)))))
The idea is that when the validation passes this spec, the validated value is bound to the given dynamic variable. The wrapped spec can then use the value in that dynamic variable. I got this working with spec/valid? but not with spec/explain-data, which always returns an empty collection. Any ideas why? I made a test spec which prints if the dynamic variable has been bound to a value or not. Inside conform* it is bound, but inside explain* it is not.
#2021-04-0911:01Matti UusitaloSome further infoā€¦ I can make a minimal spec in unit tests, but in a larger context this breaks down#2021-04-0911:07ikitommifor that given example:
(s/explain (s/coll-of (s/int-in 0 10)) [0 1 2 3 11 2])
; 11 - failed: (int-in-range? 0 10 %) in: [4]
#2021-04-0911:18Matti UusitaloThe issue here is that values in the collection should be able to be valid or invalid depending on values in a different part of the document. The example I wrote is a minimal example that tries to convey the idea#2021-04-0911:22Matti UusitaloI would use my spec like this
(def ^:dynamic *document*)
(spec/def ::min int?)
(spec/def ::max int?)

(spec/def ::value (fn [v]
                      ; compares that v is between min and max
                   ))
(spec/def ::values (spec/coll-of ::values))

(spec/valid?
 (value-binding
  (spec/keys :req-un [::min ::max ::values])
  *document*)
{:min 0 :max 10 :values [0 1 2 3 4 5 11]})
#2021-04-0911:45ikitommiif spec supported maps with inlined entry definitions, that would be easy to do - you could create a new spec with new ::values subspec based on the whole document, but now it would require going through the global registry. I believe spec2, plumatic & malli all make this easy to do. there is also spec-tools, with a working(?) dynamic var for stuff like this, but donā€™t recommend it.#2021-04-0911:47benoitI would usually put the constraint on the map itself since it is a constraint between its elements.#2021-04-0911:48benoit
(s/and (s/keys ...)
       (fn [{:keys [min max values]}] (every? #(< min % (inc max)) values)))
#2021-04-0911:51benoitBut you want explain-data to return the value ... I see. Good luck with that šŸ™‚#2021-04-0912:00benoitCould you just redefine the spec for every document?#2021-04-0912:00benoit
(defn data-spec
  [{:keys [min max values]}]
  (s/def ::min int?)
  (s/def ::max int?)
  (s/def ::values (s/coll-of (s/int-in min max)))
  (s/def ::data (s/keys :req-un [::min ::max ::values])))

(let [data {:min 0 :max 10 :values [0 1 2 3 11 2]}]
  (data-spec data)
  (s/explain-data ::data data))
#2021-04-1103:42Matti UusitaloThe most annoying thing is when I make a simple example to see where it breaks down it works fine.#2021-04-1106:13ikitommithe data-spec is not safe as it mutates the global registry. If you have two documents with different min & max, the last one overrides the first.#2021-04-1106:13ikitommiHow did you make it work @USGKE8RS7?
#2021-04-1120:10benoitYes, you have to be careful to not use data-spec for multiple documents at once. Sometimes it is an acceptable trade-off.#2021-04-1120:35benoitI'm not seeing a way around it if you want to use s/keys and the global registry. It does not make sense to me to define a spec on the global keyword ::values that is specific to a given map. The contract that the integers in ::values must be between :min and :max is a property of the map, not the global ::values keyword. If you still want to benefit from the s/explain infrastructure, you can always write a "local spec" like this:
(defn validate-map
  [{:keys [min max values] :as data}]
  (let [s (s/coll-of (s/int-in min max))]
    (when-not (s/valid? s values)
      (throw (ex-info "Invalid map."
                      {:explain (s/explain-data s values)})))))
#2021-04-1204:45Matti UusitaloSo I finally figured out what was the problem. I had to wrap the nested spec/explain* call to a doall, because apparently explain returns a lazy sequence which canā€™t access the bound value if it is returned out of the binding block before realizing the sequence#2021-04-1204:51Matti Uusitalo@U055NJ5CC i have now
(defmacro value-binding [pred bind]
  (let [pf #?(:clj (#'spec/res pred)
              :cljs (res &env pred))]
    `(let [wrapped# (delay (#'spec/specize ~pred ~pf))]
       (reify
         spec/Specize
         (specize* [s#] s#)
         (specize* [s# _] s#)

         spec/Spec
         (conform* [spec x#]
           (binding [~bind x#]
             (spec/conform* @wrapped# x#)))
         (unform* [spec y#]
           (binding [~bind y#]
             (spec/unform* @wrapped# y#)))
         (explain* [spec# path# via# in# x#]
           (binding [~bind x#]
             (doall (spec/explain*
                     @wrapped#
                     path# via# in# x#))))
         (gen* [spec overrides# path# rmap#]
           (spec/gen* @wrapped# overrides# path# rmap#))
         (with-gen* [spec gfn#]
           (spec/with-gen* @wrapped# gfn#))
         (describe* [spec]
           (spec/describe* @wrapped#))))))
and then for example
(testing "Bound values can be referred to in specs"
    (let [test-spec
          (sut/value-binding
           (fn [v]
             (= *test-binding* v))
           *test-binding*)]
      (is (spec/valid? test-spec 123))))
at some point that breaks down without that doall because of the lazyness & bindings
#2021-04-1204:51Matti Uusitalothat cljs stuff is there because clojurescript has a different implementation of clojure.spec.alpha#2021-04-0919:37Michael Stokleygiven that all map keys are validated against the spec registry, what is the significance of the :opt argument to s/keys? documentation?#2021-04-0919:53jjttjjI think it adds the generator stuff for the optional key#2021-04-0919:53jjttjj(so it will sometimes be generated)#2021-04-0920:04Alex Miller (Clojure team)also ends up in the doc output#2021-04-1120:01Braden ShepherdsonI'm trying to spec a large, complicated map for a bunch of game data. there are several possible states the game might be in, and the fields vary depending on which state we're in. I want a spec something like (s/def ::state-pregame (s/merge ::state-core (s/keys pregame-specific-things) {::state :states/pregame}))#2021-04-1120:01Braden Shepherdsonwhere the key is that last bit, specifying a particular key and value pair for the pregame state.#2021-04-1120:08Braden Shepherdsonoh, TIL about multi-spec, which is exactly this.#2021-04-1209:15Carlowhat would be the most idiomatic way to spec a graph structure? I would want something around (`graph` is the loom library):
(s/def ::my-graph
      (s/and
       graph/directed?
       #(not= :cljs.spec.alpha/invalid (s/conform (s/coll-of ::my-node) (graph/nodes %)))))
#2021-04-1209:16Carlobut I find that predicate quite cumbersome to write#2021-04-1209:24Carlo
(s/def ::my-graph
      (s/and
       graph/directed?
       #(s/valid? (s/coll-of ::my-node) (graph/nodes %))))
is a bit better. Is there anything better still?
#2021-04-1210:01borkdude@meditans not an answer to your question, but in general: instead of = :cljs.spec.alpha/invalid use (s/invalid? ...)#2021-04-1210:02borkdudeI think you should be able to leave out s/valid? and just pass a spec#2021-04-1211:42Carlobut how would I also invoke graph/nodes ?#2021-04-1218:05emWhat does your graph data structure look like? Is it some kind of custom data type, and therefore it requires the graph/nodes accessor?#2021-04-1218:06em@meditans If it's just a map, which is preferred in most situations, you can just spec a :graph/nodes key while also getting all the benefits of spec for your other graph metadata#2021-04-1219:10Carlothe data structure is the one in the loom library. (graph/nodes %) is the invocation of the function that gives the nodes back#2021-04-1220:22colinkahnOne potential option is to use a conformer in your s/and to handle the conversion into your specable form: https://clojuredocs.org/clojure.spec.alpha/conformer
(s/and graph/directed?
       (s/conformer graph/nodes)
       (s/coll-of ::my-node))
#2021-04-1301:42Michael Stokleyare there ways to programmatically work with specs when the output of s/get-spec is an object, in some cases? are there ways folks approach this?#2021-04-1301:42Michael Stokleyi'm seeing that i can't build up expressions programmatically out of existing specs#2021-04-1301:43Alex Miller (Clojure team)this is one of the big things addressed in spec 2 (via several different options) but it can be cumbersome to do in spec 1#2021-04-1301:44Alex Miller (Clojure team)in spec 1, one approach is to use macros to emit spec forms#2021-04-1301:47Alex Miller (Clojure team)the other option is to s/form a spec object (the result of s/get-spec) to get back to a form, then manipulate it as a collection, and either s/def or eval it#2021-04-1301:49Michael Stokleywow, ok! thank you!#2021-04-1413:09vlaaadIs there a way to make fn spec that allows fn to throw exceptions? I want to express a contract "this function returns any value and can even throw"#2021-04-1413:29aviIIRC, no.#2021-04-1413:31aviThis has encouraged me to spec out some of my functions to return values that indicate success or failure. Often theyā€™re :cognitect.anomalies/anomaly values. Iā€™m pretty happy with this approach. It makes the functions more testable, I think. And by speccing out the error values you get the property testing to extend to error cases as well.#2021-04-1413:33vlaaadYeah, but I want to document that I allow user-provided callback to throw an exception. Itā€™s like a promise ā€œitā€™s okay to failā€ā€¦#2021-04-1413:33aviYeah, makes sense, reasonable. I just think thatā€™s out of scope for spec.#2021-04-1413:35aviIf I were to speculate the thinking behind that call, Iā€™d guess that itā€™s an example of the position that exceptions should be used in truly exceptional cases ā€” cases that are hard to predict and handle ā€” and can thus be thrown anywhere, at any time. (Thatā€™s a reductive summary missing much nuance but I hope potentially slightly useful.) Just speculation though, I could easily be mistaken.#2021-04-1413:36aviThis might be a little silly, but in some cases Iā€™ve written functions that catch Exceptions and then return them as values! e.g.
(s/fdef check-render-result
  :args (s/cat :result (s/or :success ::r/success-result
                             :failure ::r/failure-result)
               :path   ::fs/file-path-str)
  :ret  (s/or :success nil?
              :failure (partial instance? Exception)))
#2021-04-1414:06Alex Miller (Clojure team)function specs document non-exceptional use#2021-04-1414:06Alex Miller (Clojure team)so they don't include any way to talk about exceptions#2021-04-1414:52vlaaadunderstood#2021-04-1419:26jmromrellI am trying to spec the requests and responses to an API, but am running into issues with namespace collisions.
(ns my-app.validate.endpoint-a)

(s/def :ok-response/status #{200})
(s/def :ok-response/body ...)

(s/def ::ok-response (s/keys :req-un [:ok-response/status
                                      :ok-response/body]))

(s/def :created-response/status #{201})
(s/def :created-response/body ...)

(s/def ::created-response (s/keys :req-un [:created-response/status
                                           :created-response/body]))

(s/def ::response (s/or :ok ::ok-response
                        :created ::created-response))
The problem comes when I add validation for endpoint-b which will also have an :ok-response/body which will be different than that for endpoint-a. I am already encoding the type of response (`ok-response` vs. created-response ) in the kw namespace due to s/keys mandating that the kw name matches the key in the map being spec'd. I can continue to encode further dimensions (endpoint, method) into the namespace to avoid these collisions, but it quickly gets unwieldy:
(s/def :get-endpoint-a-ok-response/body ...)
Am I overlooking an idiomatic way of handling this problem? I'd love to see something like
(s/def ::ok-response-body ...)

(s/def ::ok-response (s/named-keys :req {:status #{200}
                                         :body   ::ok-response-body}))
#2021-04-1419:27jmromrellNote: s/named-keys above could also address support for string keys https://clojure.atlassian.net/browse/CLJ-2196#2021-04-1419:40seancorfieldSpec 2 will provide that functionality @jmromrell if Iā€™m understanding correctly what you are asking.#2021-04-1419:44jmromrellThat's great. The inflexibility of s/keys has been a pain point for me multiple times.#2021-04-1419:40seancorfieldBut it sounds like you might also want to look at multi-spec?#2021-04-1419:44jmromrellI'll look into it. Thank you!#2021-04-1505:38samI could use some guidance on using clojure.spec to validate input arguments to a function. Hereā€™s some context: ā€¢ I am debating instrumenting with fdef vs. using :pre and :post conditions. ā€¢ The function is internal to the app and the argument values are completely under the control of the programmer (i.e. there is no validation of external data going on). I was initially leaning toward instrumenting with fdef because I see this as a check for program correctness rather input validation. It should be safe to turn this check off in production with no change in behavior. But I want the instrumentation to be turned on by default during development and turned off by default in production. I can imagine ways to achieve that, but the fact that I couldnā€™t find any mention of a typical pattern for doing that (e.g. my understanding is that assert is set up to be easily enabled/disabled in a blanket manner) made me think I might be missing something.#2021-04-1505:44samAnother thing I should mention is that I have no intention of ever testing this function with generative tests. Itā€™s not a good fit for generative testing.#2021-04-1510:21vemv> I was initially leaning toward instrumenting withĀ fdefĀ because I see this as a check for program correctness rather input validation. what's this? (I can interpret it in two different ways)#2021-04-1514:37samYeah, thatā€™s fair. this = ā€œthis solution Iā€™m designingā€, not this = ā€œinstrumenting with fdefā€.#2021-04-1514:52vemvIf ease of turning on/off is the main concern, perhaps I'd go for fdef because it's the default choice, and toggling it globally, and both for inputs and outputs seems relatively easy with https://github.com/jeaye/orchestra There are reasons beyond toggling ease why some people might prefer assert to instrumentation. https://github.com/fulcrologic/guardrails or https://github.com/nedap/speced.def (disclaimer: I authored it) and probably a couple more libraries favor it. There's no fixed truth in the topic, but in absence of other concerns I'd probably 'start small'.#2021-04-1514:59sam@U45T93RA6 Thanks a ton! I need to digest this more and check out orchestra. Off the top of my head, my only concern is that I usually like to be a little wary about adding new dependencies if I donā€™t have to.#2021-04-1512:38Alex Miller (Clojure team)Are you talking about clojure.core/assert or s/assert? Seems like the latter would be good here#2021-04-1514:48samThanks! Somehow I missed s/assert , I think youā€™re right. Maybe s/assert in a :pre conditionā€¦#2021-04-1514:53Alex Miller (Clojure team)I'd put it in the mainline#2021-04-1514:54Alex Miller (Clojure team)s/assert in mainline code is designed so that it can be compiled completely out of the program via https://clojure.github.io/spec.alpha/clojure.spec.alpha-api.html#clojure.spec.alpha/*compile-asserts*#2021-04-1515:00samOkay, thank you! I really appreciate the guidance.#2021-04-1522:22Franco GasperinoHello. Where i have several defined and registered predicates, can i compose them in this state?
;; contrived example
(s/def ::string-starts-with-an-a? (s/and string? #(string/starts-with? % "a")))
(s/def ::not-an-apple? #(not= "apple" %))
(s/def ::starts-with-an-a-but-not-an-apple? (s/and ::string-starts-with-an-a? ::not-an-apple?))
#2021-04-1522:25Franco Gasperinopredicate composing after registration#2021-04-1522:26Franco Gasperinodisregard the above. my forms had not been evaluated, thus the error received was entirely my doing#2021-04-1810:52mokrI got a bit carried away experimenting with specs and I believe I might have hit a dead end unless spec can actually pick apart a string (like I could do with regex capturing groups or Instaparse). Can spec accomplish something like this contrived example?
(s/def ::title #{"mr" "mrs" "dr" ,,,})
  (s/def ::surname #{"green" "brown" "who" ,,,})
  (s/def ::title-surname (s/__ :title ::title :surname ::surname))
  (s/conform ::title-surname "mrgreen")                     ;;alt1 => [:title "mr" :surname "green"]
  (s/conform ::title-surname "mrsbrown")                    ;;alt2 => {:title "mrs" :surname "brown"}
#2021-04-1810:56borkdude@mokr The standard answer to this is: although you could, spec isn't designed for parsing strings#2021-04-1810:57mokr@borkdude Thanks, that settles it. I need to backtrack a bit.#2021-04-1821:06joelittlejohnI agree with @borkdude, spec isnā€™t the right tool for this, however I have done this kind of thing in past for fun:
(s/def ::title
  (s/alt :mr (s/cat :m #{\m} :r #{\r})
         :mrs (s/cat :m #{\m} :r #{\r} :s #{\s})
         :dr (s/cat :d #{\d} :r #{\r})))
(s/def ::surname
  (s/alt :green (s/cat :g #{\g} :r #{\r} :e #{\e} :e #{\e} :n #{\n})
         :brown (s/cat :b #{\b} :r #{\r} :o #{\o} :w #{\w} :n #{\n})
         :who (s/cat :w #{\w} :h #{\h} :o #{\o})))
(s/def ::title-surname
  (s/cat :title ::title :surname ::surname))

(s/conform ::title-surname (seq "mrwho"))
;; => {:title [:mr {:m \m, :r \r}], :surname [:who {:w \w, :h \h, :o \o}]}

(s/conform ::title-surname (seq "mrsbrown"))
;; => {:title [:mrs {:m \m, :r \r, :s \s}], :surname [:brown {:b \b, :r \r, :o \o, :w \w, :n \n}]}
#2021-04-1822:43twashingDoes cljs.spec.test.alpha/check generate bad input? I have a function spec that specifies a map for its input. gen/generate and gen/sample work fine. But calling cljs.spec.test.alpha/check fails with the error: More than one element found in structure. And it looks like the spec system is generating incorrect input. bar spec
(s/def ::check-run
  (s/keys
    :req-un
    [::action
     ::check_run
     ::installation
     ::organization
     ::repository
     ::sender]))
foo.cljs
(s/def ::payload :bar/check-run)
(s/def ::check-run-started (s/keys :req-un [::payload]))

(s/fdef check-run->cijob-created
  :args (s/cat :arg ::check-run-started))

(defn check-run->cijob-created [arg])
While the function spec only declares A, the spec system is generating B.
;; A
{:payload {:action "", :check_run {:html_url "", }}, ...}

;; B
[({:payload {:action "", :check_run {:html_url "", }}, ...}})]
workbench
(cljs.spec.test.alpha/check
  `foo/check-run->cijob-created
  {:clojure.spec.test.check/opts {:num-tests 10}})

[{:spec #object[cljs.spec.alpha.t_cljs$spec$alpha50916],
  :clojure.spec.test.check/ret
  {:shrunk
   {:total-nodes-visited 313, :depth 148, :pass? false, :result #object[Error Error: More than one element found in structure: 0], :result-data #:clojure.test.check.properties{:error #object[Error Error: More than one element found in structure: 0]}, :time-shrinking-ms 11299,
    :smallest
    [({:payload {:action "", :check_run {:html_url "", }}, ...}})]},
  :sym foo/check-run->cijob-created,
  :failure #object[Error Error: More than one element found in structure: 0]}]
#2021-04-1903:04twashingOk, figured this one out. It was failing due to my use of a specter macro + navigator. Iā€™m not sure how. But somehow this messes up test.check generators. https://github.com/redplanetlabs/specter/wiki/List-of-Macros#select-one#2021-04-1903:26twashingIā€™m assuming itā€™s some kind of strange interplay thatā€™s unworkable with Clojurescriptā€™s macro system.#2021-04-2016:09cap10morganIs it expected behavior that s/valid? needs the test.check library and needs to construct a gen for every (nested) spec it's validating against? seems counterintuitive to me, but maybe I'm missing something.#2021-04-2016:16kennyIt's only necessary when checking fspecs.#2021-04-2016:17cap10morganI'm getting "Unable to construct gen at: ..." errors when calling (s/valid? ::my-spec my-val)#2021-04-2016:18kennyI suspect ::my-spec uses s/fspec.#2021-04-2016:19cap10morganthe one it's erroring on currently looks like this: (s/def ::log-file #(instance? File %))#2021-04-2016:26Alex Miller (Clojure team)I agree with kenny, that this seems like what you'd see with s/fspec#2021-04-2016:27Alex Miller (Clojure team)and generally, s/valid? will not require test.check unless you are gen'ing#2021-04-2016:30cap10morganhmm... ok. yeah, commenting out my two fdef's eliminated it. thanks!#2021-04-2108:18danmAhoy! Is there any way to generate a spec for an anonymous function without using s/fspec? Or without s/fspec generating data to test the function with? We've got a pattern where a callback is passed into a function, and we want to validate that the callback conforms to the expected function 'shape'/arity. As I understand it we can't use s/fdef, because at the point we get the callback (as an arg to our function) it's not bound to a symbol, which s/fdef requires as the first arg. Our callback function may be side effecting/have state (store some of the callback results in an atom for later reference), so if we use s/fspec then we do get an error if the callback doesn't have the right arity, but our state also gets corrupted with all the junk data that s/fspec generated to test the function.#2021-04-2112:52Alex Miller (Clojure team)You can just spec it as ifn?#2021-04-2112:53Alex Miller (Clojure team)It wonā€™t check the ā€œshapeā€ then #2021-04-2310:29danmYeah, that's what we've ended up doing. Ta šŸ™‚#2021-04-2315:48Alex Whittin spec-alpha-2, will selects be composable? Assuming the answer is "yes," what does that look like?#2021-04-2316:22Alex Miller (Clojure team)I think the answer will be no, actually, but it depends what you mean. which part are you looking to compose?#2021-04-2316:24AronPersonally, on the frontend the concept of static views that can define a select from the global state of the UI, then combining all of these into a single one for the whole of the UI?#2021-04-2316:29Alex WhittWe have a data structure that is going to accumulate keys as it moves through our system. To model that, it would make sense to have a selection for "phase 1," then a "phase 2" that takes phase 1 and adds a few keys, and a phase 3 that builds on phase 2, and so on. It would be nice to keep that DRY. (Actually we have a data model DSL that generates our specs, so we could always do the composition at that layer, but I wanted to know what spec-2 was planning on supporting.)#2021-04-2316:32Alex WhittI guess I should've phrased my question as "are selections mergeable"#2021-04-2316:46Alex Miller (Clojure team)I think that's a very interesting and unresolved question#2021-04-2316:48Alex Miller (Clojure team)it depends somewhat on whether you see merge as an operation that produces an intersectional spec or whether it acts as parallel filters through which a value must pass (which is closer to how it's implemented)#2021-04-2316:49Alex WhittWouldn't the first option be better for generation?#2021-04-2316:49Alex Miller (Clojure team)certainly the latter seems doable#2021-04-2316:49borkdudeare the args to select dynamic, i.e. can you merge those programmatically?#2021-04-2316:49Alex Miller (Clojure team)for generation, this is not that different than s/and in that you need to generate something that "passes" both specs but it's probably more challenging here#2021-04-2316:50AronI remember having serious issues with json-schema based validation because json-schema can be async, that is remotely resolved, and circular#2021-04-2316:50Alex Miller (Clojure team)in spec 2, all of the spec can be passed as data which makes it amenable to merging the specification as data#2021-04-2316:50AronSo merging there was only possible with serious limitations and strict conventions#2021-04-2316:53Alex Miller (Clojure team)there are a couple different ways data-based specs can be created in spec 2, so you actually have several choices - custom ops with s/defop (think spec template with holes), symbolic forms as data, and specs represented as maps. all of those are implemented now (with some caveats that the map form is kind of a hack in a few spots that are likely to change)#2021-04-2317:29Alex WhittI was wondering if selection merging would be a first-class feature, but it sounds like I'll probably want to do the composition at the data model DSL layer. That will generate our specs, schemas, and selections. The implementation will be a lot cleaner with spec 2.#2021-04-2920:37twashingRunning cljs.spec.test.alpha/instrument then cljs.spec.test.alpha/check on fns w/ a namespaced keyword is producing the error: RangeError: Maximum call stack size exceeded. This seems like a bug in Clojurescript. https://clojure.atlassian.net/browse/CLJS-3023 https://clojure.atlassian.net/browse/CLJS-2995 But maybe Iā€™m doing something wrong? Iā€™ve tried on these lib pairs.
org.clojure/clojure       {:mvn/version "1.10.3"}
org.clojure/clojurescript {:mvn/version "1.10.844"}

org.clojure/clojure       {:mvn/version "1.9.0"}
org.clojure/clojurescript {:mvn/version "1.10.339"}
a.cljs
(s/fdef github-deployment->deployment
  :args (s/cat :message (s/keys :req [::github.deployment/payload])))

(defn github-deployment->deployment [{payload ::github.deployment/payload :as message}]
  )

(cljs.spec.test.alpha/instrument)
b.cljs
(cljs.spec.test.alpha/check
  `backend.topology.processor.transform.deployment/github-deployment->deployment
  {:clojure.spec.test.check/opts {:num-tests 1}})

[{:spec #object[cljs.spec.alpha.t_cljs$spec$alpha20390], :clojure.test.check/ret {:result #object[RangeError RangeError: Maximum call stack size exceeded]}, :sym backend.topology.processor.transform.deployment/github-deployment->deployment, :failure #object[RangeError RangeError: Maximum call stack size exceeded]}]
#2021-05-0109:08lassemaattaI doubt I can help you solve this, but I assume you've verified that the generator for ::github.deployment/payload works and produces values without encountering that call stack issue?#2021-05-0320:33twashingNo, thatā€™s a great suggestion. I did check that, and I was generating the spec in question. I did find the problem though. A given namespace has to handle messages, all keyed by a :payload entry. When I put a namespace on itā€¦ and place it in a file with other specs, it turned out that there was a separate :payload somewhere in the spec tree.#2021-05-0415:41NoahTheDukei'm looking to write a spec that checks a list of tuples to see if there's 0 or 1 tuple that starts with some value#2021-05-0415:42NoahTheDukei have the spec that verifies and conforms that the shape is correct:
(s/def ::handler-clause (s/cat :name (s/nonconforming
                                       (s/or :keyword keyword?
                                             :class symbol?))
                               :arglist vector?
                               :body (s/* any?)))
#2021-05-0415:42NoahTheDukeand then in the fdef i have (s/* (s/spec ::handler-clause)) for that argument#2021-05-0415:44NoahTheDukeinputs look like [:error [& args] (println args)] or [:no-error [& args] args]#2021-05-0415:44borkdude* means 0 or more, not 0 or 1#2021-05-0415:44Alex Miller (Clojure team)? is for 0 or 1#2021-05-0415:45NoahTheDukemy apologies, i mean, i'd like to have it check that there is any number of tuples that don't start with :no-error and 0 or 1 tuples that start with :no-error#2021-05-0415:49NoahTheDukeso something like
(s/fdef handler-case
  :args (s/cat :expr any?
               :bindings (s/and (s/? (s/cat :name (s/and #(= :no-error %)
                                                         (s/nonconforming
                                                           (s/or :keyword keyword?
                                                                 :class symbol?)))
                                            :arglist vector?
                                            :body (s/* any?)))
                                (s/* (s/cat :name (s/and #(not= :no-error %)
                                                         (s/nonconforming
                                                           (s/or :keyword keyword?
                                                                 :class symbol?)))
                                            :arglist vector?
                                            :body (s/* any?)))
                           )))
#2021-05-0515:49pinkfrogFor a new project, use spec or spec2? If I go with spec, will it be obsoleted sometime?#2021-05-0515:55emccuehttps://en.wikipedia.org/wiki/Osborne_effect#2021-05-0519:28dvingoI recently got into malli - https://github.com/metosin/malli can highly recommend if you're looking to invest in something - it's also quite simple to generate clojure.spec.alpha or clojure.alpha.spec forms from malli schemas if you ever do want to switch#2021-05-0521:24naomarikI've been using malli since before it had a version and it's made my life so easy#2021-05-2415:30andy.fingerhutIt seems unlikely that spec2 will obsolete spec1 in 2021.#2021-05-2415:31andy.fingerhutspec2 will not cause spec1 to change its behavior.#2021-05-0516:01Alex Miller (Clojure team)spec 2 is not ready for use yet, so spec#2021-05-0516:01Alex Miller (Clojure team)when spec 2 is ready, it will include help for migrating to it#2021-05-0517:53seancorfield@alexmiller Since this seems to be a ā€œFrequently Asked Questionā€ about Spec 2, maybe you could put a big bold notice on the repo that says ā€œEXPERIMENTAL: DO NOT USE YET!ā€ or something? šŸ™‚#2021-05-0518:35Alex Miller (Clojure team)It says "This library is a work in progress provided to gather feedback during development." and there are no releases of it so....#2021-05-0518:42emccueyeah but "v2 not ready yet" doesn't alleviate people's concerns about whether they should invest in spec 1#2021-05-0518:43Alex Miller (Clojure team)spec 1 is a thing, feel free to use it#2021-05-0518:44Alex Miller (Clojure team)the spec language between the two is substantially the same. part of releasing spec 2 will be providing a migration path from spec 1#2021-05-0618:38Aronthe word 'alpha' everywhere should give a hint#2021-05-0618:40seancorfieldTo be fair, both Spec 1 and Spec 2 have ā€œalphaā€ all over them, but Spec 1 is solid and has been in heavy production use for a lot of people for years at this point. But the lack of an artifact to depend on for Spec 2 ā€” that should be a hint šŸ™‚#2021-05-0618:40seancorfieldmutter, mutter, git deps, mutter Yes, I know šŸ˜#2021-05-0618:46borkduderemoving the .alpha suffix would break a lot of programs. is this ever going to happen? ;)#2021-05-0618:47borkdudeI could see a sudden swift of everyone moving to 1.11 if that did happen and that might be a good thing. Then we could also starting using the new kwargs stuff.#2021-05-0619:39seancorfield@borkdude My understanding is that Spec 1 will stay around as-is (with .alpha) and Spec 2 will eventually become the ā€œcoreā€ non-alpha version when it is ready.#2021-05-0619:39seancorfieldSo folks wonā€™t need to change their code, unless they want to move to Spec 2 (or start using Spec 2 alongside Spec 1 ā€” which is probably what weā€™ll do).#2021-05-0619:52Arondamn, so alpha means good after all šŸ˜„#2021-05-0620:41andy.fingerhutKinda depends on who it is calling a thing alpha, and the thing.#2021-05-0619:59seancorfieldWeā€™ve run alpha builds of Clojure itself in production since 2011 šŸ™‚#2021-05-0715:43NoahTheDukeis it possible to take a function that creates a string and use it in/as a generator? converting the function to a generator would be impractical, if not outright impossible#2021-05-0716:05Alex Miller (Clojure team)there is such a thing in test.chuck - generates string that match a regex#2021-05-0716:06Alex Miller (Clojure team)https://github.com/gfredericks/test.chuck#string-from-regex#2021-05-0716:23borkdudeRegal also has generators from regexes: https://github.com/lambdaisland/regal#use-with-specalpha#2021-05-1119:07Nolanapologies if this is documented somewhere, but i was recently downgrading from spec2 to spec1, and was surprised to find that s/keys in spec1 will validate registry keys that are present in the value, but not present in the spec:
(require '[clojure.spec.alpha :as s)

(s/def ::a int?)
(s/def ::b even?)
(s/def ::c odd?)

(s/def ::ks1
  (s/keys :req-un [::a ::b])) ; NOTE: no mention of `::c`

(s/explain ::ks1 {:a 1 :b 2 ::c 4})
; => 4 - failed: odd? in: [:user/c] at: [:user/c] spec: :user/c
#2021-05-1119:10Nolani tried using the latest version of spec1, and it seemed to still work this way. is this the intent? it seems like that runs contrary to the idea of openness that spec promotes. design-wise, it's almost certainly a bad idea to be using qualified keys that dont conform to their spec, but strictly from the spec perspective this seems counterintuitive.#2021-05-1119:11Nolani'm curious if others have run up against this or if it's a rare case in practice#2021-05-1119:19andy.fingerhutI believe this is by design. spec is for defining legal values for particular keys, no matter which maps they might appear in. I think that it is in support of openness of specs -- you can include any key in any map if you want.#2021-05-1119:22Nolani think i agreeā€”that once you specify the set of legal values for a qualified key, any instance of that key (in any map) should always have a value that conforms to the specification, no matter where it is in your program. so design-wise, i'm completely with you. where i get hung up is mainly around the presence of the :req and similar options of the s/keys form, if the s/keys op is going to validate every key that exists in the registry regardless of the keys you provide for those options#2021-05-1119:22localshredif true, what purpose does s/keys provide? Seems like it's supposed to allow you to narrow your focus of key values that you want verified#2021-05-1119:22localshredand if it's also by design, why did spec2 change it?#2021-05-1119:24Nolanthat's why i ask about the intent versus the way its currently workingā€”it may be a bug in spec2, for example#2021-05-1119:24localshredfair. definitely not saying one way is right over another, just want to understand for sure#2021-05-1119:36Alex Miller (Clojure team)this is doc'ed in the s/keys docstring and should (afaik) be the same in spec 1 / 2#2021-05-1119:37Nolanah, i see now! apologies to have missed that#2021-05-1119:37Alex Miller (Clojure team)there have been some changes around kw spec references in spec 2 and its possible something was broken there#2021-05-1119:38Nolanyep, that makes sense. i just wanted to get a bit more clarity on things given the difference i was seeing. appreciate all the help @andy.fingerhut @alexmiller šŸ™#2021-05-1121:32Nolanapologies to dredge this back up, but i'm looking at my program and realizing that i'm redundantly validating values because s/keys works this way. is there any way to avoid this? is there a recommended way to validate a subset of the keys in a map? it doesn't seem possible with spec1 without using select-keys or otherwise filtering the map first#2021-05-1121:38seancorfield@nolan If you are using qualified keys, this is documented behavior and is intentional. Qualified keys are supposed to be ā€œglobally uniqueā€ and therefore the Specs declared for them should also be globally applicable. Thatā€™s by design.#2021-05-1121:39seancorfieldIn Spec 2, select is about requiredness, theyā€™ll still be validated if they are present.#2021-05-1121:45Nolanabsolutely! i agree completely with the design ethos of global uniqueness of the specification of qualified keys. it's really a matter of difficult-to-detect redundancy in the validation. for example, if i do an expensive validation using s/keys upfront, then compute with that map, potentially adding keys to it, and later i want validate the new keysā€”`s/keys` is a deceptively dangerous choice for that use-case. now that i know this is the case, i can certainly rework things to account for the multiple validations, but it seems like it would be a common use-case to simply validate only certain keys in a map. or maybe i'm just not thinking about it the right way!#2021-05-1121:49seancorfieldThe latter, I think.#2021-05-1121:49seancorfieldIf you use unqualified keys, you only get validation of what is in s/keys ā€” because theyā€™re considered to have a ā€œlocal contextā€.#2021-05-1121:50Nolani mainly find myself running up against it when trying to avoid unnecessary work. luckily we've got all kinds of scissors and scalpels for chopping these maps up in order to validate subsets, so that's i think what i'll need to do#2021-05-1121:51Nolanthanks for all the help and direction!#2021-05-1121:51seancorfieldWell, the Spec validation is all about the key names. s/keys is more about requiredness and generation.#2021-05-1121:53Nolangot it. that's helpful. i'm going to approach this through that lens and see what comes out. thanks again @seancorfield!#2021-05-1215:07ElsoI am trying to make the best case for clojure.spec I can, and apart from instrumentation and generators, I am trying to see if there is spec-powered tooling which is worth investigating thinking of something along these lines: https://reposhub.com/python/miscellaneous/clj-kondo-inspector.html in particular, any ways to integrate spec with cursive would be of great interest, as most folks on my job use it are there things you would suggest to check out?#2021-05-1216:45dvingoI wanted to do data transformations from specs and down that path leads macro hell (see: https://github.com/dvingo/my-clj-utils/blob/master/src/main/dv/spec_util.clj#L36 with this helper I was told to just repeat the keys instead: https://clojurians-log.clojureverse.org/clojure-spec/2020-11-14/1605365585.379500) After this interaction, I have found much delight in using malli. Being based on data it makes analysis and transformation fairly trivial; additionally it has clj-kondo support already built in: https://github.com/metosin/malli#clj-kondo perhaps this clj-kondo transform and the graphviz one, plus the others built in would help make the case for adoption on your team.#2021-05-1216:41seancorfield@d.eltzner012 maybe my blog post from 2019 will help provide some useful insights for you? https://corfield.org/blog/2019/09/13/using-spec/#2021-05-1218:28vlaaadWhy the difference in spec forms of anon fns?
(s/form (s/nilable #(= 1 %)))
=> (clojure.spec.alpha/nilable
     (clojure.core/fn [%]
        (clojure.core/= 1 %)))

(s/form (s/or :maybe-one (s/nilable #(= 1 %))))
=> (clojure.spec.alpha/or
     :maybe-one
     (clojure.spec.alpha/nilable
        (fn* [p1__11188#]
         (clojure.core/= 1 p1__11188#))))
#2021-05-1218:42Alex Miller (Clojure team)no good reason#2021-05-1218:43Alex Miller (Clojure team)probably just not applying the right cleanup function on the latter#2021-05-1514:01danierouxIs there a reason that clojure.test.check.generators/let was left out of clojure.spec.gen.alpha/lazy-combinators?#2021-05-1514:45Alex Miller (Clojure team)Itā€™s a macro iirc#2021-05-1514:46Alex Miller (Clojure team)I think thereā€™s actually a ticket about this#2021-05-1514:47Alex Miller (Clojure team)As a macro, you canā€™t do the same dynaload we do with the other combinators#2021-05-1514:50Alex Miller (Clojure team)https://ask.clojure.org/index.php/4633/let-ported-from-test-check-let-to-clojure-spec-gen#2021-05-1515:37danierouxThank you#2021-05-1516:22danierouxThis is what I could come up with, it feels unwieldly:
(try
  (requiring-resolve 'clojure.test.check.generators/let)
  (catch Exception _))
(s/def ::prodquint
  (s/with-gen
    (s/and string? #(re-matches pro-dquint-regex %))
    (fn []
      (clojure.test.check.generators/let
        [pro-quint-hi (s/gen ::proquint)
         pro-quint-lo (s/gen ::proquint)]
        (str pro-quint-hi "-" pro-quint-lo)))))
#2021-05-1516:33gfredericksare you trying to produce code that still works if test.check isn't available? if so I can't see how your code accomplishes that#2021-05-1516:34gfredericksand if not, then I'd think you could just require let in the more vanilla fashion and everything would be fine#2021-05-1516:49danierouxThat is what I am trying, and failing, to accomplish. I want the spec to be there with its generator attached to it. Knowing that the generator will only be used int dev/test.#2021-05-1516:51gfredericksif you want the let call in the same spot as your spec then I think you basically end up needing eval or something equivalent#2021-05-1516:52gfrederickswell maybe not#2021-05-1516:56gfredericksthis might work
(defmacro tclet
  [& args]
  (let [e (try
            (requiring-resolve 'clojure.test.check.generators/let)
            nil
            (catch Exception e e))]
    (if e
      `(throw (Exception. ~(.getMessage e)))
      `(clojure.test.check.generators/let 
#2021-05-1517:00danierouxThank you @gfredericks!#2021-05-1517:00gfredericksdoes it work does it work?#2021-05-1517:08danierouxIt does!#2021-05-1517:09danieroux(turns out I had test.check on the prod classpath all along, now it breaks and works in expected ways)#2021-05-1816:27donavanIs there a way to do this:
(s/def ::bar #{:one :two :three})
(s/def ::foo
  (s/with-gen
    (s/keys :req [::bar])
    #(gen/fmap
      (fn [foo]
        (assoc foo ::bar :one))
      (s/gen ::foo))))
without having to create temporary specs like this due to the spec lookup loop
(s/def ::bar #{:one :two :three})
(s/def ::foo*
  (s/keys :req [::bar]))
(s/def ::foo
  (s/with-gen
    ::foo*
    #(gen/fmap
      (fn [foo]
        (assoc foo ::bar :one))
      (s/gen ::foo*))))
#2021-05-1816:33Alex Miller (Clojure team)You can gen recursive specs#2021-05-1816:33Alex Miller (Clojure team)Can you explain why you need with-gen?#2021-05-1816:35donavan::foo is actually a multi-spec and only the one value of ::bar is valid for this implementation#2021-05-1816:35donavanSo without tweaking the generator it fails to generate valid examples#2021-05-1816:36Alex Miller (Clojure team)Have you looked at the generator override map?#2021-05-1816:36donavanBy ā€œYou can gen recursive specsā€ do you mean create the s/def calls dynamically in a macro?#2021-05-1816:37Alex Miller (Clojure team)No, I mean you can have recursive specs and call gen on them#2021-05-1816:38Alex Miller (Clojure team)I think I did not get what you were trying to do#2021-05-1816:38donavanHaha, so it actually is both ultimately a multi-spec and a recursive one. The recursion ā€˜withinā€™ the generator works fine if I split the spec into the ::foo* and ::foo defs#2021-05-1816:39donavanItā€™s just the call to s/gen within the s/with-gen that loops expectedly#2021-05-1816:36donavanItā€™s being fed into another library that does the generating so I canā€™t supply the override map#2021-05-1816:40Alex Miller (Clojure team)Whatā€™s the point of the initial gen if you just override :bar?#2021-05-1816:41Alex Miller (Clojure team)Arenā€™t you always generating the same map?#2021-05-1816:43Alex Miller (Clojure team)If so, just (s/gen #{ {::bar :one} })#2021-05-1816:43Alex Miller (Clojure team)Or gen/return if that is easier to read #2021-05-1816:43donavanIā€™m not I follow exactly; the ::bar spec is used in all the multi-specs itā€™s just that some of the types itā€™s values are restricted#2021-05-1816:43Alex Miller (Clojure team)Is this simplified and there are other keys too?#2021-05-1816:44donavanOh, yes sorry this is rather simplified!#2021-05-1816:44Alex Miller (Clojure team)You left out half the problem and half the constraints#2021-05-1816:45Alex Miller (Clojure team)It may not actually be in an inifinite loop - it may just be making very big intermediate colls#2021-05-1816:45donavanThe simple case above loops forever#2021-05-1816:45donavanApologies šŸ™‚ I was just trying to ask the simplest question. I could mull it over some moreā€¦#2021-05-1816:46Alex Miller (Clojure team)If you have any coll-of specs, make sure you set :gen-max 3 on them#2021-05-1816:46Alex Miller (Clojure team)There are also some spec dynvars controlling recursion, might want to check those too#2021-05-1816:47Alex Miller (Clojure team)All that said, there are some outstanding bugs in this area and itā€™s inherently tricky#2021-05-1817:27donavanScratching a little deeper the loop Iā€™m referring to is within the implementation, as spec is currently implemented you cannot define a generator ā€˜in terms of itselfā€™ like I do above for any of the implementations such as map-spec-impl . Itā€™s deeply baked into the implementation so I suspect it wonā€™t change even though at first glance that seemed like a natural way to scratch my itch šŸ˜„ . Iā€™ll re-evaluate what Iā€™m trying to do, thanks @alexmiller#2021-05-1818:43Jeff Evansdoes it make sense to call instrument from a clojure.test fixture? https://clojuredocs.org/clojure.test/use-fixtures ex: if we have tests for various namespaces, each of which should really instrument the specs as they run#2021-05-1818:43Alex Miller (Clojure team)yes#2021-05-1818:44Alex Miller (Clojure team)and unstrument at the end :)#2021-05-1819:55borkdude@jeffrey.wayne.evans I've also got an fdef-testing library here: https://github.com/borkdude/respeced#2021-05-1820:41Jeff Evansnice, thanks. will have a look#2021-05-1821:19Jeff Evanshow about this situation? I want b to be an optional argument to my-fn, but if provided, then it should match some predicate (like map?)? this doesnā€™t quite work.
(defn my-fn
  [a & [b]]
  ;; do stuff with a and b
)

(s/fdef my-fn
        :args (s/cat :a int? :b map?)
        :ret  map?)
#2021-05-1821:19Jeff Evans
(my-fn 1)
Execution error - invalid arguments to user/my-fn at (form-init6149560983531223969.clj:1).
() - failed: Insufficient input at: [:b]
#2021-05-1821:20Alex Miller (Clojure team)you spec'ed it as a function taking 2 args (int and map)#2021-05-1821:21Alex Miller (Clojure team)you can use s/? to spec an optional part of a regex op#2021-05-1821:22Alex Miller (Clojure team)it's a little unclear to me if you meant [b] or b in the my-fn definition#2021-05-1821:23Alex Miller (Clojure team)I guess you mean what you said#2021-05-1821:23Alex Miller (Clojure team)and :args (s/cat :a int? :b (s/? map?)) should work for int and optional map#2021-05-1821:31Jeff Evansawesome. thank you, Alex!#2021-05-1902:11Joshua SuskaloAre there any libraries that have regex ops for lookahead?#2021-05-1902:12Joshua SuskaloI saw seqexp, but I'd prefer if it were as a part of spec#2021-05-1902:25Alex Miller (Clojure team)Can you give an example of where you would want that?#2021-05-1902:33Joshua SuskaloA gross overapplication of spec to a usecase it wasn't designed for: parsing a sequence of tokens output by another process#2021-05-2218:21Adam HelinsHey! I have been doing a lot of work involving generating recursive data. This is a space where actually, from what I see, neither Spec nor Malli have a working solution. Any trivial example is already too much and generation gets completely out of hand. However I haven't used Spec in a while so I might be mistaken. I'd appreciate your thoughts and comments, especially if you have encountered that kind of problems. Albeit extremely simple, with a pretty standard size of 30, generating a single instance of this example results commonly in 1000-3000 leaves (nested vectors containing that much booleans):
(s/def ::bool
  boolean?)

(s/def ::vec
  (s/coll-of ::data))

(s/def ::data
  (s/or :bool ::bool
        :vec ::vec))
There is an "exponential explosion" so that such definitions to not reflect the size linearly but rather exponentially. I tried explaining it thoroughly here and how test.check solves it with recursive-gen (the fact it is a Malli issue is irrelevant): https://github.com/metosin/malli/issues/452 So, surprisingly, it is somewhat impossible to generate truly random Clojure values where collection items can be collections as well. Unless I am of course missing something, so here I am :)
#2021-05-2415:45Joshua SuskaloPersonally I recommend writing two generation functions, one that takes a generation depth and generates a full data structure to that depth, and another that takes a generation depth and generates a "narrow" structure to the given depth. This way you can test with the first one to a low depth to see "wide" structure generation, and use a much higher generation depth for the narrow one, using a weighted rng to choose which one to generate when using a with-gen form in the spec.#2021-05-2415:45Joshua Suskalo@adam678#2021-05-2415:48Joshua SuskaloOr as you've noted, recursive-gen already provides a reasonable solution, in that case you can also use that with spec's facilities to wrap something with a custom generator.#2021-05-2416:01Joshua Suskalo
(require '[clojure.test.check.generators :as gen])

(s/def ::bool
  boolean?)

(def data-gen (gen/recursive-gen gen/vector gen/boolean))

(s/def ::vec
  (s/with-gen
    (s/coll-of ::data)
    (gen/vector data-gen)))

(s/def ::data
  (s/with-gen
    (s/or :bool ::bool
          :vec ::vec)
    data-gen))
#2021-05-2416:10Joshua SuskaloFor reference, the clojure spec generators are just a reexport of the functions in clojure.test.check.generators, which means that you can use them interchangeably.#2021-05-2416:24Alex Miller (Clojure team)well, almost interachangeably - most of the spec apis actually take a no-arg function that returns a generator instead of a generator (to allow for dynamic require/invoke)#2021-05-2515:42lassemaattaI remember wondering why e.g. with-gen accepts a "generator returning function" instead of the generator itself; can you clarify what you mean by that "dynamic require/invoke" functionality? Something to do with requiring additional namespaces within the function but only if the generator is actually invoked (e.g. in generative testing)?#2021-05-2515:44Alex Miller (Clojure team)yes, that#2021-05-2515:45Alex Miller (Clojure team)by putting it in a thunk, we can avoid dynaloading the test.check.generators ns#2021-05-2515:46lassemaattaI see, thank you#2021-05-2416:31Joshua SuskaloOh that's good to know. Thanks for the heads up on that.#2021-05-2520:05Adam HelinsThanks @suskeyhose, so as I expected it is best writing that kind of generation from scratch#2021-05-2520:07Joshua SuskaloYes, and I would say that custom generators aren't something to shy away from either. Most structures can be made with just the base ones, but email addresses or other data that has many constraints upon what is valid should have custom generators.#2021-06-0318:04Braden ShepherdsonI'm struggling with nesting s/cat calls. I've got a tuple-like thing, but it's not implemented by a real vector, rather by a deftype. so I originally wrote the spec for it with s/tuple but it failed to match. I switched to s/cat with the parts spelled out, and that works.#2021-06-0318:05Braden Shepherdsonbut then in a (s/fdef foo :args (s/cat :x ::tuple-like :y int?)) it fails, since the cats end up running together.#2021-06-0318:06Braden ShepherdsonI guess I can always write the ::tuple-like predicate as an #(instance? TheDeftype %) but that seems really clunky.#2021-06-0318:51sgepigonTry wrapping it in a s/spec to prevent them running together: (s/cat :x (s/spec ::tuple-like) :y int?). Spec 1 does this with s/spec, spec 2 will do this with https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha#nested-regex-contexts.#2021-06-0421:13Frank HenardHow do I do an ::Anything spec in Clojurescript? In Clojure I have (s/def ::Anything #(instance? Object %))#2021-06-0421:14borkdude@ballpark any?#2021-06-0421:15borkdudenote that your ::Anything spec will not tolerate nil#2021-06-0421:16Frank HenardThanks, Yeah, I'll have to do a parity match on those#2021-06-0421:17borkdudejust to be clear: the answer in both dialects is any?.#2021-06-0421:22Frank Henardright, thanks, here's what I changed it to for my cljc file: (s/def ::Anything #(s/and some? any?)) This way I can specify s/nilable in the specs that reference ::Anything. Seem like an ok solution to you?#2021-06-0421:24borkdudeit seems a little weird to me. why not just use some? and any? directly?#2021-06-0421:25Frank HenardYes, that's it! Something seemed wrong to me too, thanks again!#2021-06-0423:59Joshua Suskalos/and with any? as an item is exactly equivalent to just the rest of the items#2021-06-0716:31Franco GasperinoHaving watched the 2018 talk on monads (Maybe Not) and appreciating the case made for place oriented programming using maps and spec, the question i have - Does this lead to design of most non-pure functions accepting and returning maps (almost) exclusively?#2021-06-0716:45delaguardoWhat makes you think that?#2021-06-0717:31Franco Gasperinocomposing, avoiding imperative style, while still supporting error context#2021-06-0716:48Alex Miller (Clojure team)in Clojure 1.11, you can pass a trailing map to a function that takes kwargs, so kwarg style also accepts map invocation#2021-06-0716:49Alex Miller (Clojure team)positional args are a tradeoff - they make invocation easier by making the arg name implicit#2021-06-0716:49Alex Miller (Clojure team)but that also means they are fragile to positional changes, whereas maps / kwargs are not#2021-06-0721:19emccueIf there wasn't a perf hit for it, I could very much see myself having all args named Objective C method call style#2021-06-0721:20emccuei guess in practice thats all maps#2021-06-0915:37MatthewLispHello Does somebody happen to have a ready spec for JWT (Json web token) ?#2021-06-0918:48emccuehttps://datatracker.ietf.org/doc/html/rfc7519#2021-06-1017:52mathias_dwhi, i have a big collection of specced data that went into mongodb and lost all the namespace info. I suppose I can transform it all by recursively using the information in (s/describe) of the top-level (big) data structure. But before doing that, I thought I'd ask if something like that already exists. Anything similar? Just to complicate it a little: there are also plenty of non-namespaced and non-specced keys that got in there as well. They shouldnt be given a namespace.#2021-06-1105:55delaguardoMongodb is storing data as json. So you probably need to have some sort of convention how to encode qualified keywords to/from json#2021-06-1114:33mathias_dwThanks for you reply! Yeah, i realize that now, but the problem is that a lot of production data was stored without the namespace info before I noticed, and it's not something i can re-do. Anyway, the approach with using s/describe also doesnt work, because of things like merge etc. So guess I'll have to write custom code for the exact data structure.#2021-06-1117:41vlaaada bit surprising:
(s/valid? (s/map-of keyword? int? :kind vector?) [[:a 1]])
=> true
#2021-06-1117:45Alex Miller (Clojure team)what is that even supposed to mean?#2021-06-1117:46Alex Miller (Clojure team)seems like it's doing exactly what the docstring says to me#2021-06-1117:50vlaaadI don't know what that's supposed to mean, I'm trying to build a tool on top of spec and I have to support what it supports#2021-06-1117:53vlaaadI'll probably ignore this particular thing... I just was trying to understand what kind of data shapes various variations of s/every support.#2021-06-1117:54borkdudespec needs a spec? ;)#2021-06-1117:55vlaaadless surprising:
(s/valid? (s/every-kv keyword? int?) #{[:a 1]})
=> true
every-kv docstring states it works on associative collections...
#2021-06-1117:56borkdudewhat kind of tool are you building, or is it secret?#2021-06-1117:56vlaaad"spec forms"#2021-06-1117:57borkdudesomething cljfx related?#2021-06-1117:58vlaaadIt's a reveal plugin that builds a UI form for a given spec. Then you can fill the form to get back the data in the shape that satisfies the spec.#2021-06-1118:00vlaaadhttps://twitter.com/v1aaad/status/1390689740308074498?s=19#2021-06-1118:01vlaaad...and the form resembles the data#2021-06-1118:02borkdudeso it's way to visualize a spec form?#2021-06-1118:06vlaaadMore than that. You can bind changes in form to custom code and custom views. For example, if you have spec for vega/vega-lite data format, you can have a UI form for creating vega chart input and a vega chart itself that updates on changes to the form. This allows learning and experimenting with data-driven APIs interactively#2021-06-1118:06Alex Miller (Clojure team)every-kv is really about "seqs to map entries", but we have no way to say that in Clojure (yet)#2021-06-1118:07Alex Miller (Clojure team)Rich and I have been talking about that gap for years, some day we'll figure it out :)#2021-06-1118:08borkdudewhat's the 2-version of triple?#2021-06-1118:08Alex Miller (Clojure team)duple?#2021-06-1118:08vlaaadmap-entry?#2021-06-1118:08Alex Miller (Clojure team)pair is commonly used though :)#2021-06-1118:08borkdudeduple-blob#2021-06-1118:09borkdudeduple-flock#2021-06-1118:09borkdudesolved? ;)#2021-06-1118:09Alex Miller (Clojure team)the "seqs to" part is the hard part :)#2021-06-1118:10vlaaadbtw I think every-kv is fine, I'd expect it to accept any coll#2021-06-1118:10vlaaadbut maybe the docstring shouldn't mention associative...#2021-06-1118:11Alex Miller (Clojure team)I think that may be vestigial from earlier impls#2021-06-1416:42bortexzHi, I know spec2 is still in alpha/experimental phase and buggy. Still, I have been playing around with it to model the domain of an experimental project I have, where I want to test out the capabilities of domain modelling with schema/select. Is it possible to have a multi-spec return a schema, and do select on the multi-spec? It doesnā€™t seem to work on current git-sha, I wonder if itā€™s a bug or is it intended that this is not possible:
(spec/def :account/id string?)
(spec/def :service/id string?)
(spec/def :service-1/field string?)
(spec/def :service-2/field string?)

(defmulti account-schema :service/id)

(defmethod account-schema :service-1
  [_]
  (spec/schema [:account/id :service-1/field]))

(defmethod account-schema :service-2
  [_]
  (spec/schema [:account/id :service-2/field]))

(spec/def :account/account (spec/multi-spec account-schema :service/id))

(gen/generate (spec/gen :account/account))

(spec/valid? (spec/select :account/account [:account/id :service-1/field :service/id])
             {:service/id :service-1
              :account/id "id#1"
              :service-1/field "abcd"})
The generator works correctly, but not the validation, that fails with:
; Execution error (IllegalArgumentException) at clojure.alpha.spec.protocols/eval5667$fn$G (protocols.clj:20).
; No implementation of method: :keyspecs* of protocol: #'clojure.alpha.spec.protocols/Schema found for class: clojure.alpha.spec.impl$multi_spec_impl$reify__7055
#2021-06-1417:09Alex Miller (Clojure team)undecided stuff in this area about what can act as a schema provider#2021-06-1417:09Alex Miller (Clojure team)if you have a use case where this would be useful, would be helpful to have an https://ask.clojure.org question with request tag#2021-06-1417:12bortexzThanks Alex, I will open the question with a more detailed use case#2021-06-1521:19bortexz^ Opened the question for this https://ask.clojure.org/index.php/10696/spec2-use-case-for-multi-spec-as-schema-provider#2021-06-1716:57jcdOther than the official docs, does anyone have any recommendations for familiarizing oneself with using spec?#2021-06-1717:29Alex Miller (Clojure team)if the official docs are lacking, would be interested in how so they can be improved#2021-06-1719:47jcdI wouldnā€™t say they are lackingā€¦ but Iā€™m curious if there are talks or more introductory guides people would recommend. The official docs are good, but somewhat intimidating since their scope is comprehensive.#2021-06-1720:00borkdude@livingfossil on the official Clojure site there is a "guide" and a "reference", the guide is usually an introduction and the reference is more encompassing (hope I said that right)#2021-06-1720:09jcdaha! thanks!#2021-06-1720:22jkxyzā€œAll-encompassingā€ is the usual phrase meaning ā€œcomprehensiveā€ but the meaning was clear šŸ¤“
#2021-06-1722:13Alex Miller (Clojure team)you could just read random parts of it and then it would be less comprehensive :)#2021-06-1722:13Alex Miller (Clojure team)there are a few talks out there if that would match better to your learning style#2021-06-1722:14Alex Miller (Clojure team)https://www.youtube.com/watch?v=tEWSw8H9KJU#2021-06-1800:43JP Silvahi folks, I'm curious about the current status of https://github.com/clojure/spec-alpha2#2021-06-1800:44seancorfield@jplfdsilva Not production ready. Still being designed. Has bugs.#2021-06-1800:45seancorfieldIn particular, thereā€™s a bunch of design work going on behind the scenes around how fdef should work/be integrated with defn etc.#2021-06-1800:45JP Silva@seancorfield thanks for the update#2021-06-1800:45seancorfieldItā€™ll be a big improvement over Spec (1) when it is finalized thoughā€¦#2021-06-1800:47JP Silvaagreed. I just finished watching the Maybe Not talk (again) and was wondering when the new select spec op would be available.#2021-06-1800:47seancorfieldā€œsoonā€ šŸ™‚#2021-06-1800:48seancorfield(which, in Clojure terms, could be a while!)#2021-06-1800:49JP Silvait's an amazing idea I hope other languages would follow#2021-06-2015:10hkjelsI have a collection where I want an identifier for each entity in that collection to be unique. Can that be specified?#2021-06-2015:10hkjelsI know of distinct on coll-of, but that just enures that each entity is unique AFAICS#2021-06-2016:00Alex Miller (Clojure team)If you can write a predicate function, you can make that a spec#2021-06-2016:00Alex Miller (Clojure team)Nothing for this built in#2021-06-2108:09Jakub HolĆ½ (HolyJak)Hi! When was work on Spec 2 started? I guess late 2018? Thank you!#2021-06-2111:48Alex Miller (Clojure team)I donā€™t know, you can check the repo#2021-06-2119:37Jakub HolĆ½ (HolyJak)I checked the repo but it seems like the older commits are still Spec 1 commits, there is not lear (to me) transition between v1 and v2. I will just assume that it was around the Maybe Not talk šŸ™‚#2021-06-2113:30borkdudeMaybe Not could be seen as a milestone when Rich declared that spec1 contained some design choices that were not optimal and he wants to address in spec2. This was in November 2018.#2021-06-2113:33borkdudeStill an enjoyable talk btw#2021-06-2218:40Michael Stokleyran into a surprise with s/or just now#2021-06-2218:40Michael Stokley
;; `s/or` destructures the value that the second pred sees
(let [spec (s/and (s/or :even even? :small #(> 10 %))
                  #(= 5 %))]
  (s/explain-str spec 5))
;; => "[:small 5] - failed: (= 5 %)\n"

;; flipped order of arguments to `s/and
(let [spec (s/and #(= 5 %)
                  (s/or :even even? :small #(> 10 %)))]
  (s/explain-str spec 5))
;; => "Success!\n"
#2021-06-2218:41Michael Stokleyis there a way to avoid this?#2021-06-2218:44Michael Stokleyah, i guess it's in the docstring of s/and#2021-06-2218:44sgepigonI believe itā€™s more of s/andā€™s behavior as it ā€œflowsā€ values through (see https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha#nonflowing-sand--new) You can try https://clojuredocs.org/clojure.spec.alpha/nonconforming#2021-06-2218:55Alex Miller (Clojure team)spec 2 has a non-flowing s/and variant to cover this#2021-06-2419:12UsmanHi all, How would I go about creating random URLs that begin with https:// using spec? I'm new to clojure and spec so any support/explanations will be much appreciated. :)#2021-06-2419:49seancorfield@usman.jamil I'd probably use a string regex and gfrederick's test.chuck lib which has a generator for regexes.#2021-06-2419:49Alex Miller (Clojure team)https://github.com/gfredericks/test.chuck#string-from-regex#2021-06-2419:49Alex Miller (Clojure team)and ditto#2021-06-2419:52UsmanThanks. I will have a look at that.#2021-07-0912:19Jim NewtonI am not a spec expert, but my application has a spec interface which accepts specs and deals with them. I have some tests that are failing with randomly generated data. I'd like advise about how to prepare my tests.
(s/def ::test-spec-1 (s/* (s/alt :1  (s/cat :3 neg? :4 even?)  
                                 :2  (s/cat :5 odd? :6 pos?))))
I have a spec defined as above, ::test-spec-1, when this is applied to a sequence containing a floating point number, rather than the spec telling me that it fails to match, instead the even? function raises an exception
ERROR in (t-canonicalize-spec-type) (core.clj:1391)
Uncaught exception, not in assertion.
expected: nil
  actual: java.lang.IllegalArgumentException: Argument must be an integer: 1.0
 at clojure.core$even_QMARK_.invokeStatic (core.clj:1391)
#2021-07-0912:20Jim Newtondo I need to write my own functions int-and-odd? , int-add-pos?, etc? or is there a more idiomatic way of handling this ?#2021-07-0912:26delaguardoinstead of even? you can use (every-pred int? even?)#2021-07-0912:26delaguardosame for odd?#2021-07-0912:27Jim Newton
(s/def ::test-spec-1 (s/* (s/alt :1  (s/cat :3 (every-pred int? neg?) :4 (every-pred int? even?)
                                 :2  (s/cat :5 (every-pred int? odd?) :6 (every-pred int? pos?)))))
#2021-07-0912:27Jim Newtonis this what you mean?#2021-07-0912:28delaguardo(every-pred int? neg?) this is unnecessary because neg? is expecting number#2021-07-0912:29delaguardobut in general - yes#2021-07-0912:29Jim Newtonahhh, yes indeed that's correct#2021-07-1213:06Ian Fernandezhow can I select-keys giving a spec that has required keys?#2021-07-1213:06Ian FernandezI only want to pick the keys in a map that has in this spec#2021-07-1213:15vlaaadAre you trying to write at function that, given a spec, will return a function that, given a map will strip all keys not mentioned in spec?#2021-07-1213:16vlaaadIā€™d suggest looking at s/form to get a form of a spec that mentions those keys#2021-07-1213:35nbardiukwe are using https://cljdoc.org/d/metosin/spec-tools/0.10.5/api/spec-tools.core#select-spec to drop keys that are not in spec#2021-07-1213:44Ian Fernandez
(s/def ::my-one (s/keys :req [::a ::b]))
#2021-07-1213:45Ian Fernandezhow can I make a ::my-two that has ::my-one keys and a key ::c too?#2021-07-1214:20colinkahnSee s/merge https://clojuredocs.org/clojure.spec.alpha/merge#2021-07-1411:16mathias_dwHi, I'm sure it's been asked before, but googling didn't turn anything up: is there an easy way to associate comments to specs? My usecase is working together with some non-clojure (actually, non-programmer) people on a data definition. I'm used to spec and all the related tooling, but I can't expect the others to really understand everything from code.#2021-07-1412:12mpenetit's not supported out of the box. you need to build your own metadata api for specs right now if you need that.#2021-07-1412:13mpenetit's quite easy to do tho, basically an atom and a few functions that will work against it. We have something like that internally, we can do (-> (s/def ::foo string?) (x/vary-meta! assoc :doc "some docstring"))#2021-07-1412:13mpenets/def returns the spec key, you can take advantage of this. and from there writing x/with-doc is just a small fn#2021-07-1414:01Alex Miller (Clojure team)this is the most requested feature for spec and I expect it will be added in spec 2#2021-07-1922:04dgb23How would I go about writing a spec for a data structure that detects (disallows) cycles? Context: As a fun learning project Iā€™ve been writing a state machine for a lexer, as a map from states->inputs->[outputs/->states] (that last part is the transition). Now as is typical for these, some of the transitions are similar on different states (here because of lexemes). Instead of composing the map programmatically I thought I invent an additional behaviour and add a :with key on the transition, to say that the output is combined with the output of another transition. This way the state machine is fully described as a plain data-structure. :with merely points to The terrible thing here is that I introduced the possibility for recursion/infinite loops. So my intuition is that I can write a spec that detects cycles, but I cannot really wrap my head around it. Is this possible? My context is just an example, maybe the design is wrong, but I would be generally interested in applying spec this way and how one would go about it.#2021-07-1922:08Alex Miller (Clojure team)I think I would probably not write such a spec#2021-07-1922:09Alex Miller (Clojure team)but a spec can be any predicate function#2021-07-1922:09Alex Miller (Clojure team)so if you can write a function (no-cycles? data-structure), then that's a valid spec#2021-07-1922:10dgb23right I was getting stuck somehow in trying to make the spec both check the structure and detect the cycle at the same time#2021-07-1922:10dgb23but there is s/and ā€¦#2021-07-1922:10Alex Miller (Clojure team)(s/and ::structure no-cycles?)#2021-07-1922:10dgb23right ty šŸ˜„#2021-07-1922:11dgb23you think such a spec is a smell?#2021-07-1922:11Alex Miller (Clojure team)I just don't think you're getting any value out of using spec to write that :)#2021-07-1922:12Alex Miller (Clojure team)spec's sweet spot is describing information structures (maps of attributes) or syntactic structures (sequential regex type things). the more generic or abstract you get from that, the less value you get#2021-07-1922:15dgb23I get what youā€™re saying. But say I used spec during development time with instrumentation I would still get value from this right?#2021-07-1922:19Alex Miller (Clojure team)hard for me to say#2021-07-1922:19Alex Miller (Clojure team)keep in mind that slow specs can really slow down execution. cycle checking sounds slow. :)#2021-07-1922:20Alex Miller (Clojure team)would you be better off preventing a graph with a cycle from being created in the first place?#2021-07-1922:21dgb23I just started reading a book that thematises this. I have to think about this more deeply! ty#2021-07-2221:55vemvlong time no spec. how do I un-nilify a nilable again? spec-tools maybe?#2021-07-2222:16colinkahnWould (s/and ::nilable-spec some?) be what you want?#2021-07-2222:18vemvyes that would work! I was using some other workaround in the meantime but your snippet is more concise#2021-07-2221:56vemv(spec1)#2021-07-3021:03isakIf you are using cat, is it possible to somehow spec on all remaining elements, instead of just the next element in the sequence?#2021-07-3021:10seancorfieldWith * perhaps? (not quite sure what you're asking @isak)#2021-07-3021:12isakI'm trying to do something like this: (s/cat :a ::number :b ::number :rest ::my-spec), where ::my-spec should run on all remaining items in the sequence, not just the very next one. For example for the sequence [1 2 3 4], it should run on [3 4], not just 3.#2021-07-3021:15seancorfieldLike so?
dev=> (s/valid? (s/cat :a number? :b number? :c (s/* string?)) [1 2])
true
dev=> (s/valid? (s/cat :a number? :b number? :c (s/* string?)) [1 2 3])
false
dev=> (s/valid? (s/cat :a number? :b number? :c (s/* string?)) [1 2 "foo"])
true
dev=> (s/valid? (s/cat :a number? :b number? :c (s/* string?)) [1 2 "foo" "bar"])
true
dev=> (s/conform (s/cat :a number? :b number? :c (s/* string?)) [1 2])
{:a 1, :b 2}
dev=> (s/conform (s/cat :a number? :b number? :c (s/* string?)) [1 2 3])
:clojure.spec.alpha/invalid
dev=> (s/conform (s/cat :a number? :b number? :c (s/* string?)) [1 2 "foo"])
{:a 1, :b 2, :c ["foo"]}
dev=> (s/conform (s/cat :a number? :b number? :c (s/* string?)) [1 2 "foo" "bar"])
{:a 1, :b 2, :c ["foo" "bar"]}
#2021-07-3021:16isakYea like that, except I want to be able to run a spec on the whole collection that gets gathered#2021-07-3021:17seancorfieldI don't understand.#2021-07-3021:18seancorfieldDo you mean that you want ::my-spec to be another s/cat-based spec?#2021-07-3021:18isakWell with (s/* string?), it is saying what should be true about the individual elements, but what if I needed something to be true for the whole collection?#2021-07-3021:19seancorfieldUse s/cat there.#2021-07-3021:19isakOh hmm#2021-07-3021:21seancorfield
dev=> (s/valid? (s/cat :a number? :b number? :c (s/cat :d string? :e (s/? string?))) [1 2])
false
dev=> (s/valid? (s/cat :a number? :b number? :c (s/cat :d string? :e (s/? string?))) [1 2 3])
false
dev=> (s/valid? (s/cat :a number? :b number? :c (s/cat :d string? :e (s/? string?))) [1 2 "foo"])
true
dev=> (s/valid? (s/cat :a number? :b number? :c (s/cat :d string? :e (s/? string?))) [1 2 "foo" "bar"])
true
dev=> (s/conform (s/cat :a number? :b number? :c (s/cat :d string? :e (s/? string?))) [1 2])
:clojure.spec.alpha/invalid
dev=> (s/conform (s/cat :a number? :b number? :c (s/cat :d string? :e (s/? string?))) [1 2 3])
:clojure.spec.alpha/invalid
dev=> (s/conform (s/cat :a number? :b number? :c (s/cat :d string? :e (s/? string?))) [1 2 "foo"])
{:a 1, :b 2, :c {:d "foo"}}
dev=> (s/conform (s/cat :a number? :b number? :c (s/cat :d string? :e (s/? string?))) [1 2 "foo" "bar"])
{:a 1, :b 2, :c {:d "foo", :e "bar"}}
#2021-07-3021:22seancorfieldThe sequence regex specs "unroll" or merge into one long sequence, so you can use them to describe subsequences inside another sequence.#2021-07-3021:23seancorfieldI'm showing the Spec forms inline, but that Spec for :c (or :rest in your case) could be ::my-spec.#2021-07-3021:25isakHm, but what if I don't have a condition about the individual elements? For example, if I needed to match a number, a number, then all the remaining elements have to be in a collection that is odd?#2021-07-3021:26isakFor example, [1 2 3 4 5] would pass, but not [1 2 3 4] (contrived example, but gets at what I'm trying to do) (And I can only do a cat on the first 2 elements)#2021-07-3021:27isakI would try this but it doesn't really make sense:
(s/def ::odd-collection (s/and seqable? #(odd? (count %))))

  (s/conform
    (s/cat :a int? :b int? :rest (s/cat ::odd-collection ))
    [1 2 3 4 5])
#2021-07-3021:27seancorfield"a collection that is odd" -- what do you mean? Can you express that as a predicate?#2021-07-3021:28isakSorry I meant the count should be odd#2021-07-3021:29isakOk this works:
(s/def ::odd-collection (s/and seqable? #(odd? (count %))))

  (s/conform
    (s/cat :a int? :b int? :rest (s/& (s/* any?) ::odd-collection))
    [1 2 3 4 5 ])
#2021-07-3021:29isakNot sure if there is a better way#2021-07-3021:31seancorfieldThat was what I was about to suggest: s/& is the "and" of sequence regexes!
dev=> (s/conform (s/cat :a number? :b number? :c (s/& (s/* number?) (comp odd? count))) [1 2 3 4])
:clojure.spec.alpha/invalid
dev=> (s/conform (s/cat :a number? :b number? :c (s/& (s/* number?) (comp odd? count))) [1 2 3])
{:a 1, :b 2, :c [3]}
dev=> (s/conform (s/cat :a number? :b number? :c (s/& (s/* number?) (comp odd? count))) [1 2 3 4 5])
{:a 1, :b 2, :c [3 4 5]}
#2021-07-3021:32isakGotcha, thanks @seancorfield#2021-08-0422:46Franco GasperinoI'm looking for the preferred route on the following.. Requirement: * Read JSON messages from external streaming source. * Perform message schema validation. Search for required keys and data pairs. * Conform message and filter invalid messages. Question: * Spec definitions are centered around keywords, not strings. * Perform a recursive keywordize-keys call on original string keys? * Alternative spec definition using strings instead of keywords?#2021-08-0423:26Franco Gasperinoit appears keys -> keywords model is at least optionally suggested in a couple libraries, such as cheshire and clojure/data.json#2021-08-0423:43seancorfieldYou need to be a bit careful about just converting all JSON input to keyword-based hash maps since a malicious user could bombard your server with random JSON with long, unique strings and potentially cause performance/heap problems for you.#2021-08-0423:45seancorfield(that said, I think a lot of people do simply read the JSON as keyword-based hash maps)#2021-08-0423:46seancorfieldWe no longer have the problem of old where keywords were interned and never GC'd so it's not as dangerous as it used to be šŸ™‚#2021-08-0423:48vemvIs there a known recipe for avoiding security pitfalls when using spec at the edges of a production app? Other than "don't" :)#2021-08-0423:49seancorfieldAt work, our APIs are all behind authentication so we can "trust" the input to some degree and we do read JSON to keyword-based hash maps and then validate it with Spec. You could write a Spec that just validated the top-level keys as strings -- using a set for the valid keys, but if you're accepting values that can also be structured data that will get a bit gnarly.#2021-08-0423:50seancorfieldAs for security pitfalls with Spec being used on arbitrary data, I think you mostly need to ensure that validation can't be sent into a deep CPU hole because your specs allow arbitrary nesting and structure (again, the malicious user and the large payload issue).#2021-08-0423:54vemvsounds like a hard-to-write meta-spec :) I'm open to everything though it came to mind just now, maybe one create a simple translation layer from spec to malli which is more performant / fit for the use case. Obviously only a subset could be translated there's the precedent of https://github.com/threatgrid/flanders which is a unified DSL that can spit out Spec and Plumatic Schema alike#2021-08-0500:16dgb23Instead of making parsing more performant (which is certainly a good thing) one can treat payload size and request frequency as separate problems and simply reject stuff that goes above a certain threshold respectively. In common, simple cases itā€™s enough to have sensible heuristics for each of those. At scale one likely needs statistical models. In other words, spec is concerned with analysing stuff thatā€™s already read (parsed), so I would assume you put those kind of checks before it.#2021-08-0500:17emccue@franco.gasperino
(mu/closed-schema [:map ["thing" :string] ["other" [:map ["thing" :int]]]])
#2021-08-0500:18emccueyou can join the malli dark side#2021-08-0500:18dgb23I talked a bit with a googler who was working on systems that detect anomal request patterns on the edges a couple of years ago, was quite fascinating.#2021-08-0500:23dgb23If a stranger keeps loudly knocking on your door with a big hammer, you donā€™t ask for their credentials, just call the police.#2021-08-0500:25vemvOTOH rolling out a ML-powered infrastructure sec thingy sounds like a 10x harder problem Still one worth solving though#2021-08-0500:27dgb23I shouldnā€™t have started to think about web security before going to bedā€¦#2021-08-0519:32vlaaad
(let [s (s/cat :cat? (s/? (s/cat :int int?)))]
  (s/unform s (s/conform s [1])))
=> ((1))
should be (1)
#2021-08-0520:46colinkahnI think itā€™s this issue: https://clojure.atlassian.net/browse/CLJ-2003#2021-08-0520:59Franco Gasperino@seancorfield @vemv @denis.baudinot Thanks for the discussion.#2021-08-1016:07dev-hartmannHey folks, ist there a way to parse a JSON schema to spec at runtime?#2021-08-1219:27souenzzoIf is not this lib, is another metosin lib https://github.com/metosin/spec-tools#2021-08-1610:41dev-hartmannThx!#2021-08-1519:28vlaaads/* wrapping s/alt conform gives a vector of tuples (expected):
(s/conform (s/* (s/alt :strs (s/+ string?)
                         :ints (s/+ int?)))
             ["a" "x" 1 2])
=> [[:strs
     ["a" "x"]]
    [:ints
     [1 2]]]
s/+ wrapping s/alt conform gives a vector with tuple and a vector of tuples (unexpected):
(s/conform (s/+ (s/alt :strs (s/+ string?)
                         :ints (s/+ int?)))
             ["a" "x" 1 2])
=> [[:strs
     ["a" "x"]]
    [[:ints
      [1 2]]]]
#2021-08-1519:43dgb23so weird when you look at:
(s/conform (s/+ (s/alt :strs (s/+ string?)
                       :ints (s/+ int?)))
           ["a" "x" 1 2 "b" "c" 4 5])
#2021-08-1520:05vlaaadfixed formatting#2021-08-1521:28Alex Miller (Clojure team)I'm not seeing what's unexpected here, can you describe?#2021-08-1605:02vlaaadSecond element of second result unnecessary wrapped in a vector#2021-08-1521:06vlaaadgiven (s/def ::foo (s/* any?)) , shouldn't (s/regex? ::foo) also return true?#2021-08-1521:26seancorfieldThat operates on the (internal) spec form I believe, not on spec names.#2021-08-1521:29seancorfieldUse (s/regex? (s/get-spec ::foo)) @vlaaad#2021-08-1521:30seancorfield
dev=> (s/def ::bar string?)
:dev/bar
dev=> (s/def ::foo (s/* any?))
:dev/foo
dev=> (s/regex? (s/get-spec ::bar))
nil
dev=> (s/regex? (s/get-spec ::foo))
{:clojure.spec.alpha/op :clojure.spec.alpha/rep, :p2 #object[clojure.core$any_QMARK_ 0x18d396eb "
#2021-08-1605:01vlaaadI know how it operates, I just find it limiting#2021-08-1605:14vlaaadBecause to check if something is a regex I have to do (s/regex? (or (s/get-spec spec) spec)) instead of just (s/regex? spec)#2021-08-1606:39vlaaad
(s/form (s/keys* :req-un [::foo]))
=> (clojure.spec.alpha/&
    (clojure.spec.alpha/*
     (clojure.spec.alpha/cat
      :clojure.spec.alpha/k
      clojure.core/keyword?
      :clojure.spec.alpha/v
      clojure.core/any?))
    :clojure.spec.alpha/kvs->map
    mspec__2546__auto__)
that doesn't look right...
#2021-08-1607:01dgb23Youā€™re looking at an expanded macro.#2021-08-1607:21vlaaads/form for other specs, that are also implemented as macros, return the form used in code#2021-08-1608:41dgb23right!#2021-08-1612:45Alex Miller (Clojure team)This is a known issue thatā€™s addressed in spec 2#2021-08-1723:23richiardiandreaHi there, I have a burning question šŸ˜„ Say I have a collection of maps and I want to spec them. There a lot of maps in this collection but I want to inspect and solve errors one by one. Is there a way to say to st/instrument or s/explain to limit the number of errors outputted for that collection?#2021-08-1817:23mishayou can use s/expalain-data and trim output errors as you wish:
(-> (s/explain-data (s/coll-of even?) (range 100))
  (update ::s/problems #(take 10 %)))
(you probably will need to trim ::s/value as well) Or, you can validate each map individually, not as a collection:
(->> (range 100)
  (keep #(s/explain-data even? %))
  (take 3)) 
#2021-08-2016:29meta-metaHi, how do I write a spec to validate values in a map depending on the key? Say I have a map {:foo 42 :bar [1 2 3]} I want to validate that if :foo is defined, it's an int and if :bar is defined, it's a coll.#2021-08-2016:41Russell Mull
(s/def ::foo int?)
(s/def ::bar (s/coll-of int?))
(s/def ::mymap (s/keys :req-un [::foo ::bar]))
#2021-08-2016:52meta-metathank you! And now I see it's right there in the first part of the spec guide. I was looking at map-of and every, totally forgetting what I read the other day.#2021-08-2304:15vemvGiven an fdef , can I get its source code back? Like clojure.repl/source does with vanilla defs (my guess is that nope, but maybe someone wrote a thing?)#2021-08-2310:19reefersleepMaybe create a macro that uses a central registry to keep track of the source of each fdef, as per its fully evaluated name.#2021-08-2310:19reefersleepNot cool enough with macros to write the example, but itā€™d be something like, swap!ing into your central atom, where you keep a map of fdef-symbol -> unevaluated source, and then of course evaluating your fdef normally afterwards.#2021-08-2310:22reefersleepThis is just an off-hand idea, hope it makes sense#2021-08-2310:31vemvI think a minimalistic macro is a step in the right direction, thanks! Hadn't thought of that much In that case I wouldn't need an extra atom - I could simply let clojure.repl/source do its work e.g. (defmacro def-fdef ..) + (def-fdef the-defn the-spec) The macro would create a predictable symbol, like the-defn-fdef-source . Then I (source the-defn-fdef-source)#2021-08-2310:33reefersleephm, yeah, I guess šŸ™‚#2021-08-2310:34reefersleepLooking forward to see how you fare šŸ™‚#2021-08-2311:17Lennart BuitIsnā€™t s/form close to what you are looking for?#2021-08-2311:27vemvI'm interested in newline/comment preservation as well#2021-08-2319:05vemvNot for the faint of heart https://github.com/reducecombine/.lein/commit/ca5bccefd672191d5c703047d4e206304cac9453#2021-08-2318:10bortexzCurrently, on spec alpha 2, you can select things that are not in the schema passed. Is this expected behaviour?
(ns user
  (:require [clojure.alpha.spec :as s]))

(s/def ::schema-field1 int?)
(s/def ::schema-field2 int?)
(s/def ::not-schema-field int?)

(s/def ::schema (s/schema [::field1 ::field2]))
(s/def ::selected (s/select ::schema [::schema-field1
                                      ::schema-field2
                                      ::not-schema-field]))

(s/valid? ::selected {::schema-field1 1
                      ::schema-field2 2
                      ::not-schema-field true}) => false

(s/valid? ::selected {::schema-field1 1
                      ::schema-field2 2
                      ::not-schema-field 3}) => true
#2021-08-2318:18Alex Miller (Clojure team)schemas are not exclusive#2021-08-2318:19Alex Miller (Clojure team)so yes#2021-08-2318:24bortexzGreat, thank you! I think itā€™s great to contextually ā€˜extendā€™ schema selections to keywords not originally on the schema#2021-08-2318:27bortexzIs there any reason to have a schema in select at all? This:
(s/select ::schema [::schema-field1
                    ::schema-field2
                    ::not-schema-field]))
and this:
(s/select [::schema-field1
           ::schema-field2
           ::not-schema-field]))
seem to express the same
#2021-08-2318:30Alex Miller (Clojure team)gen#2021-08-2318:30Alex Miller (Clojure team)docs#2021-08-2318:31Alex Miller (Clojure team)closed schema checking#2021-08-2318:32bortexzI see, that makes sense, thanks#2021-08-2410:58vlaaadA bug:
(defn- retag [x v]
  (assoc (vec x) 0 v))

(defmulti command first)

(s/form (s/multi-spec command retag))
=> (clojure.spec.alpha/multi-spec current.ns/command #object[current.ns$retag 0x708d0436 
#2021-08-2411:08vlaaadMade a report on ask: https://ask.clojure.org/index.php/10969/s-form-s-multi-spec-returns-a-list-with-non-symbolic-objects#2021-08-3015:02Joshua SuskaloWhat do people do about having records conform to specs? Since it's generally good practice to use namespaced keywords with specs, but records don't support namespaced keywords on the enumerated fields, the only ways I've seen is either to make a less extensible spec by not using namespaced keywords, or to just not spec the records and prevent them from crossing interface boundaries.#2021-08-3015:06Alex Miller (Clojure team)I think those are the big ones#2021-08-3015:19Joshua SuskaloFair enough. Sometimes I wish I could just define namespaced record fields, but I can see why that'd be a challenge.#2021-08-3019:05seancorfield@suskeyhose What drives you to use records rather than hash maps?#2021-08-3019:06Joshua SuskaloPerformance. I write game engines in my spare time and things like field access time from entities can make the difference between being able to support hundreds of objects at 60 fps to being able to support thousands, which entirely changes the types of games you can make.#2021-08-3019:07Joshua SuskaloAnd in those sorts of systems, spec brings a lot of value since many of these systems have complex calling contracts that must be upheld and instrumentation and generative testing aid a lot in tracking down errors when you're able to encode those contracts via spec.#2021-08-3019:16seancorfieldAh, interesting niche. Yes, I can see performance being a good driver for that decision.#2021-08-3019:18Joshua SuskaloI'll also say that yes, game engines is very niche for Clojure development, and for someone who wants to make a very performance-sensitive game they'll have better luck with Unity's new systems, or writing something from scratch in Rust or C++, I prefer Clojure because it makes it easy to write my game code, and I don't make highly performance-sensitive games, but at an engine level I'd like to be able to make as few concessions as possible so that the overhead of being in Clojure takes away the least creative freedom.#2021-08-3019:19Joshua SuskaloAnd since I'm an engine developer by education and desire, I'm perfectly willing to take on lots of complexity in the engine if I can make writing something atop it simple and unconstrained.#2021-08-3019:52colinkahnCouldnā€™t you write a one-way conformer for records to be specā€™d? Like (def ->map (s/conformer #(into {} %))), then use something like (s/and #(instance? MyRecord %) ->map (s/keys :req-un [ā€¦.]))#2021-08-3021:19Joshua SuskaloThe problem I have with that is then I have to write code that doesn't follow the spec, but follows an implementation detail for performance. Ideally I'd still be able to do the keyword-based lookup.#2021-08-3110:20jumarAlready asked in #cider but it looks unrelated, at least for one of my projects where it started to fail with lein repl (maybe a dependency update but I believed it worked fine and don't remember any changes in dependencies).
Caused by: Syntax error macroexpanding clojure.core/defn at (clojure/spec/alpha.clj:85:1).
	at clojure.lang.Compiler.checkSpecs(Compiler.java:6972)
	at clojure.lang.Compiler.macroexpand1(Compiler.java:6988)
	at clojure.lang.Compiler.macroexpand(Compiler.java:7075)
...
	at clojure.main$loading__6721__auto____8974.invoke(main.clj:11)
	at clojure.main__init.load(Unknown Source)
	at clojure.main__init.<clinit>(Unknown Source)
	... 55 more
Caused by: java.lang.Exception: #object[clojure.spec.alpha$and_spec_impl$reify__1057 0x459f703f "
What could be a reason for this ? We use these if it matters:
[expound "0.8.9"]
                                             [orchestra "2021.01.01-1"]
#2021-08-3112:22Alex Miller (Clojure team)Sorry, not sure what that is#2021-08-3112:23Alex Miller (Clojure team)Can you make something reproducible to share?#2021-08-3112:51jumarI'll try to check if it isn't something specific to the dependencies / toolchain (which is likely)#2021-08-3119:37MatthewLispHello šŸ‘‹ I'm defining a spec using https://clojure.org/guides/spec#_collections, and i quote:
Tuple - (s/tuple double? double? double?)
Designed for fixed size with known positional "fields"
Conforms to a vector of the values
What if, i have a fixed size collection but with unknown positional "fields", and the collection is NOT homogenous ?
#2021-08-3119:48flowthingcat might work, if I understood correctly?
user=> (require '[clojure.spec.alpha :as spec])
nil
user=> (spec/def ::elements (spec/cat :int int? :string string?))
:user/elements
user=> (spec/valid? ::elements [1 "a" :b])
false
user=> (spec/valid? ::elements [1])
false
user=> (spec/valid? ::elements [1 "a"])
true
#2021-08-3119:48MatthewLispwhat about ["a" 1] ?#2021-08-3119:49MatthewLispThat's what i meant with unknown positional fields#2021-08-3119:49flowthingOh, I see.#2021-08-3119:51flowthingYeah, this is basically what Alex said, but:
user=> (spec/def ::element
         (spec/or :int int? :string string? :keyword keyword?))
:user/element
user=> (spec/def ::elements (spec/coll-of ::element :count 3))
:user/elements
user=> (spec/valid? ::elements [1 "a" :b])
true
user=> (spec/valid? ::elements [1])
false
user=> (spec/valid? ::elements [1 "a"])
false
#2021-08-3119:50Alex Miller (Clojure team)what are you expecting to validate about this collection?#2021-08-3119:50Alex Miller (Clojure team)just it's size?#2021-08-3119:51Alex Miller (Clojure team)
(s/coll-of any? :count 2)
#2021-08-3120:02MatthewLispNot just the size, an example of the collection:
[{:identity "username"
  :value     "john"}

 {:identity "age"
  :value     "30"}]
One might think this can be described with s/coll because it appears to be an homogenous collection, however, depending on the context of the application, i have different values on this collection... The way i'm defining the two entries of the collection is as follows:
;; username

(s/def :username/identity
  #{"username"})

(s/def :username/value
  "PREDICATE_HERE")

(s/def :entry/username
  (s/keys :req-un [:username/identity
                   :username/value]))

;; age

(s/def :age/identity
  #{"age"})

(s/def :age/value
  "PREDICATE_HERE")

(s/def :entry/age
  (s/keys :req-un [:age/identity
                   :age/value]))

;; now i'm trying to define the collection using :entry/username and :entry/age
#2021-08-3120:04MatthewLispi've over simplified the keywords names because i'm obfuscating the real names#2021-08-3120:04MatthewLispbut the structure is this#2021-08-3120:06Alex Miller (Clojure team)
(s/coll-of (s/or :user :entry/username :age :entry/age) :count 2)
#2021-08-3120:07Alex Miller (Clojure team)there are more options for the content, like wrapping that in s/nonconforming to avoid the s/or tag, or use s/multi-spec instead for open extension there, etc#2021-08-3120:12MatthewLispThis approach leads me with a collection having two entries of for example :entry/username as being valid I'm starting to think something using multi-spec, might work!#2021-08-3120:14MatthewLispWill look into s/nonconforming docs also#2021-08-3120:23Alex Miller (Clojure team)the last is un-doc'ed but it basically conforms, then returns the original value#2021-09-0215:4441ph4_Ph4unHi, I'm having a weird issue where I only call (st/instrument) in my code and aim to instrument only the used functions in my code base#2021-09-0215:4441ph4_Ph4unbut somehow it seems to begin to generate arguments for some of the functions in the codebase#2021-09-0215:4541ph4_Ph4unIf I'm only aiming to instrument my code, I shouldn't need clojure.test.check and its submodules, right?#2021-09-0215:4641ph4_Ph4unsomehow when I run my program it whines those need to be required.. then when I add them it begins to generate inputs and all hell breaks loose#2021-09-0215:48borkdude@theamazingekko do you by any chance use s/fspec ?#2021-09-0215:4941ph4_Ph4unand yes, I do use s/fspec#2021-09-0215:4941ph4_Ph4unoh boy do I use it!#2021-09-0215:5141ph4_Ph4unso okay.. I'm starting to get a picture here#2021-09-0215:5241ph4_Ph4unso it is possible that instrumentation may attempt to generate arguments for arguments that are functions in some cases?#2021-09-0215:5241ph4_Ph4unif theres HOFs#2021-09-0215:5541ph4_Ph4unhmm... so if this is a problem I need to maybe simplify those specs.. maybe by using ifn? instead of full-fledged f/spec#2021-09-0215:5641ph4_Ph4unare there any other workarounds.. in a way I'd like to keep that stricter form so I could then run generative tests separately later on#2021-09-0215:5741ph4_Ph4uncan I be a total pig and set clojure.spec.alpha/*fspec-iterations* to 0 šŸ˜„ šŸ˜„ šŸ˜„#2021-09-0216:00borkdudeyes, I think this is an "interesting" feature of spec#2021-09-0216:0141ph4_Ph4unyeah it seems that just restricting the iterations solved my issues for now#2021-09-0216:0141ph4_Ph4unthanks! I've been stuck with this over a day šŸ˜„#2021-09-0216:0241ph4_Ph4unbasically it was just about finding a right config.. it's very "interesting" indeed#2021-09-0317:26Joshua SuskaloYeah, fspec is mostly useful for pure functions. In general the philosophy that seems to be present in spec is anything stateful and side effecting shouldn't be significantly specced. So some of these cases where you have fspec may be best to just have ifn and then describe the exact function which must be given in detail in the docstring.#2021-09-0317:26Joshua SuskaloGenerally specs and types are no substitute for good docstrings.#2021-09-0616:05mpenet@alexmiller just curious: any progress on spec2 lately?#2021-09-0616:05Alex Miller (Clojure team)no#2021-09-0616:51CarloI'd like to model a binary tree with a map of the form {:payload 0 :left sub1 :right sub2}. Here's my spec for now:
(s/def ::binary-tree-2
  (s/with-gen
    (s/keys :req-un [::payload]
            :opt-un [::left ::right])
    #(gen/recursive-gen
      (fn [f] (gen/let [[val l r] (gen/tuple (s/gen int?) f f)]
               (merge
                {:payload val}
                (when l {:left l})
                (when r {:right r}))))
      (gen/return nil))))
But this doesn't check that ::left and ::right have the correct shape. What should I do instead?
#2021-09-0617:13CarloI added more predicates with s/and, but I still wonder if there's a better way#2021-09-0620:57Bobbi TowersI was wondering something similar a while back, and happened to find this: https://deque.blog/2017/05/20/a-clojure-spec-for-generic-binary-trees/ perhaps it might help?#2021-09-0722:27Franco GasperinoI have read that s/keys only supports keywords. I have a large external json message for which i would to perform some top-level schema validation. Rather than performing a keywordize-keys, which may descend past the desired schema and into embedded content, I was hoping to keep the message keys as strings and use validation upon them. From what i can see, this would be done similar to this:
(def last-name? #(= "last" %))
  (def first-name? #(= "first" %))
  (def age? #(= "age" %))

  (spec/def ::last-name (spec/tuple last-name? string?))
  (spec/def ::first-name (spec/tuple first-name? string?))
  (spec/def ::age (spec/tuple age? (spec/and pos-int? int?)))

  (spec/def 
   ::who
   (spec/coll-of 
    (spec/or :last ::last-name :first ::first-name :age ::age) 
    :kind map?
    :count 3
    :distinct true))
  
  (spec/explain ::who {"last" "smith" "first" "bob" "age" 10})
Is this the recommended path when using map keys which are strings, not keywords?
#2021-09-0722:37Alex Miller (Clojure team)This approach should work, you can simplify those preds using sets though, like #{ā€œfirstā€} (and at that point I would just inline them into ::first-name etc#2021-09-0722:39Alex Miller (Clojure team)You also might find it useful to wrap s/nonconforming around the s/or to simplify the conformed structure#2021-09-0722:51Franco Gasperinocan you provide a case where s/conform ::who json-map would return something undesirable, where s/nonconforming would be wanted?#2021-09-0723:08Alex Miller (Clojure team)(spec/or :last ::last-name :first ::first-name :age ::age) is going to conform to [:last ["last" "smith"]]#2021-09-0723:09Alex Miller (Clojure team)(spec/nonconforming (spec/or :last ::last-name :first ::first-name :age ::age)) will conform to ["last" "smith"]#2021-09-0723:10Alex Miller (Clojure team)if you use :into {} in your ::who spec, you can actually conform back to the original map if you want that#2021-09-0723:10Franco Gasperinogreat to know. appreciated#2021-09-0723:19Franco Gasperinodidn't need the :into. wrapping the s/or in a nonconforming was enough to return a map when calling conform#2021-09-0908:13FranklinI'm trying to run tests for a specedĀ `cljs`Ā function using karma... it seems trickier than I expected I'm getting an error that looks like this
--------------------------------------------------------------------------------
  23 | 
  24 | (defn ^:export run
  25 |   [karma]
  26 |   (-> (stest/enumerate-namespace 'quagga.utils.components.share-modal.events) stest/check)
---------^----------------------------------------------------------------------
Encountered error when macroexpanding cljs.spec.test.alpha/check.
Error in phase :compile-syntax-check
RuntimeException: No such namespace: stest
Here's how I'm running the test; not sure if its' the correct way to do it
(defn ^:export run
  [karma]
  (-> (stest/enumerate-namespace 'quagga.utils.components.share-modal.events) stest/check)
  (karma/run-tests karma
                   'quagga.utils.components.account-select.events-test
                   'quagga.components.dataview.table.events-test
                   'quagga.utils.components.share-modal.events-test))
Any clue as to how I can get it to work?
#2021-09-0910:47FranklinI've also imported/required stest
[clojure.spec.test.alpha :as stest]
#2021-09-1008:28Franklinok.. I made a lot of progress.... I now have my tests looking like this
(deftest my-gen-test (is (= [](stest/check `ranged-rand))))
#2021-09-1008:29Franklinand this fails as expected when I call run-tests from cljs.test
> (run-tests)

Testing cljs.user

FAIL in (my-gen-test2) (repl-input.cljs:1:27)
expected: (= [] (stest/check (quote cljs.user/ranged-rand)))
  actual: (not (= [] [{:spec #object[cljs.spec.alpha.t_cljs$spec$alpha38994], :clojure.spec.test.check/ret {:shrunk {:total-nodes-visited 4, :depth 2, :pass? false, :result #error {:message "Specification-based check failed", :data {:cljs.spec.alpha/problems [{:path [:fn], :pred (cljs.core/fn [%] (cljs.core/>= (:ret %) (cljs.core/-> % :args :start))), :val {:args {:start -3, :end 0}, :ret -4}, :via [], :in []}], :cljs.spec.alpha/spec #object[cljs.spec.alpha.t_cljs$spec$alpha38456], :cljs.spec.alpha/value {:args {:start -3, :end 0}, :ret -4}, :cljs.spec.test.alpha/args (-3 0), :cljs.spec.test.alpha/val {:args {:start -3, :end 0}, :ret -4}, :cljs.spec.alpha/failure :check-failed}}, :result-data {:clojure.test.check.properties/error #error {:message "Specification-based check failed", :data {:cljs.spec.alpha/problems [{:path [:fn], :pred (cljs.core/fn [%] (cljs.core/>= (:ret %) (cljs.core/-> % :args :start))), :val {:args {:start -3, :end 0}, :ret -4}, :via [], :in []}], :cljs.spec.alpha/spec #object[cljs.spec.alpha.t_cljs$spec$alpha38456], :cljs.spec.alpha/value {:args {:start -3, :end 0}, :ret -4}, :cljs.spec.test.alpha/args (-3 0), :cljs.spec.test.alpha/val {:args {:start -3, :end 0}, :ret -4}, :cljs.spec.alpha/failure :check-failed}}}, :time-shrinking-ms 2, :smallest [(-3 0)]}, :failed-after-ms 7, :num-tests 5, :seed 1631262481706, :fail [(-6 -1)], :result #error {:message "Specification-based check failed", :data {:cljs.spec.alpha/problems [{:path [:fn], :pred (cljs.core/fn [%] (cljs.core/>= (:ret %) (cljs.core/-> % :args :start))), :val {:args {:start -6, :end -1}, :ret -9}, :via [], :in []}], :cljs.spec.alpha/spec #object[cljs.spec.alpha.t_cljs$spec$alpha38456], :cljs.spec.alpha/value {:args {:start -6, :end -1}, :ret -9}, :cljs.spec.test.alpha/args (-6 -1), :cljs.spec.test.alpha/val {:args {:start -6, :end -1}, :ret -9}, :cljs.spec.alpha/failure :check-failed}}, :result-data {:clojure.test.check.properties/error #error {:message "Specification-based check failed", :data {:cljs.spec.alpha/problems [{:path [:fn], :pred (cljs.core/fn [%] (cljs.core/>= (:ret %) (cljs.core/-> % :args :start))), :val {:args {:start -6, :end -1}, :ret -9}, :via [], :in []}], :cljs.spec.alpha/spec #object[cljs.spec.alpha.t_cljs$spec$alpha38456], :cljs.spec.alpha/value {:args {:start -6, :end -1}, :ret -9}, :cljs.spec.test.alpha/args (-6 -1), :cljs.spec.test.alpha/val {:args {:start -6, :end -1}, :ret -9}, :cljs.spec.alpha/failure :check-failed}}}, :failing-size 4, :pass? false}, :sym cljs.user/ranged-rand, :failure #error {:message "Specification-based check failed", :data {:cljs.spec.alpha/problems [{:path [:fn], :pred (cljs.core/fn [%] (cljs.core/>= (:ret %) (cljs.core/-> % :args :start))), :val {:args {:start -3, :end 0}, :ret -4}, :via [], :in []}], :cljs.spec.alpha/spec #object[cljs.spec.alpha.t_cljs$spec$alpha38456], :cljs.spec.alpha/value {:args {:start -3, :end 0}, :ret -4}, :cljs.spec.test.alpha/args (-3 0), :cljs.spec.test.alpha/val {:args {:start -3, :end 0}, :ret -4}, :cljs.spec.alpha/failure :check-failed}}}]))

Ran 1 tests containing 1 assertions.
1 failur
#2021-09-1008:30FranklinI wonder if there's an existing cljs.test function that can check if stest.check fails and show me log the results#2021-09-1008:35FranklinI also think I can write a custom function... for this one using test/abbrev-result... but I wonder if there's something there already#2021-09-1013:53colinkahnhttps://github.com/borkdude/respeced has some utilities for this.#2021-09-1018:46uwoThe correct way to spec a singular value is a set correct? Silly contrived example: (s/conform (s/tuple #{:db/add} int? #{:attr} #{:value}) [:db/add 17592311384347 :attr :value])#2021-09-1018:55Alex Miller (Clojure team)yes#2021-09-1219:06Eric DahlsengDoes anyone know why spec/check seems to take forever to cleanup? In the following minimally reproducible example, Clojure takes ~60 seconds to shutdown after "Done" is printed, but exits immediately if the check call is removed:
(ns test
  (:require [clojure.spec.alpha :as spec]
            [clojure.spec.test.alpha :as spec-test]))

(spec/def ::number number?)

(spec/fdef plus
           :args (spec/or
                  :nothing (spec/cat)
                  :numbers (spec/* ::number))
           :ret (spec/or
                 :number ::number))

(defn plus [& values]
  (apply + values))

(defn -main [& args]
  (spec-test/check `plus)
  (println "Done"))
#2021-09-1219:16seancorfield@edahlseng There's probably a thread pool that it is waiting on. Call (shutdown-agents) in -main to solve that.#2021-09-1219:17seancorfieldSee the FAQ for Clojure https://clojure.org/guides/faq#agent_shutdown#2021-09-1219:22Eric Dahlseng@seancorfield, that works wonderfully! I wasn't aware of (shutdown-agents), thanks for the tip!#2021-09-1219:22seancorfieldThere's lots of good stuff in the FAQ.#2021-09-1219:26Eric DahlsengThere definitely is. I've ended up there through some Google searches, but haven't given it a direct read through otherwise. Looks like I should do that now to preempt future questions!#2021-09-1307:04Ben SlessQuestion regarding conventions and style. For some map spec ::foo having a few variants, how would you name the specs? Should keys be named :foo/key? For the variants I thought of just using or (s/def ::foo (s/or ,,,)) To name the variants I can think of ::foo.a ::foo-a :foo.type/a Recommendations and ideas very welcome#2021-09-1311:34Alex Miller (Clojure team)Thereā€™s not enough detail here for me to really answer that - how does a variant vary?#2021-09-1311:37Alex Miller (Clojure team)Generally though, I would urge you first to focus on the attributes not on the maps. Name the attributes well and fully describe the set of values that attribute can cover. If you then need multiple sets of attributes, kind of depends how they relate#2021-09-1312:02Ben SlessRight, then to be specific, consider https://clojure.github.io/tools.analyzer.jvm/spec/quickref.html#binding, which I want to spec out unambiguously#2021-09-1800:16vemvif you don't mind my asking, are you building something off t.ana? Would be curious about it (as someone who maintains two derived tools namely Eastwood and refactor-nrepl)#2021-09-1804:43Ben SlessI am exploring the possibilities. Background, clj-fast, one of my first projects, is just a bunch of heuristic inlinings and optimizations of core clojure. I can keep on working at it for every conceivable use case, but I became interested in solving the general problem. I want to implement some intermediate compiler passes - constant inlining, beta reductions, copy and constant propagation. If I get this, I can write a compiler from clojure to clojure that cuts down on the biggest performance hits in the language, iteration and dynamism#2021-09-1804:45Ben SlessThis ia clearly not for for development time, but for final artifacts it could be huge#2021-09-1312:03Alex Miller (Clojure team)this is probably a good use case for s/multi-spec#2021-09-1312:03Alex Miller (Clojure team)switching on :op#2021-09-1312:03Ben SlessBut even before op, zooming in on binding#2021-09-1312:03Ben SlessA very rough spec would be:
(s/def ::binding
  (s/keys
   :req-un
   [:binding/op :binding/form :binding/name :binding/local :binding/arg-id
    :binding/variadic? :binding/init :binding/atom :binding/children]))
#2021-09-1312:04Ben Sless(imagine I merged it with ::common)#2021-09-1312:04Ben SlessI should have split out the :opt-un, too#2021-09-1312:04Ben Slessbut the point is, there are keys which just don't occur some times#2021-09-1312:05Ben Slessfor example, :binding/arg-id occurs only when :binding/local is :arg#2021-09-1312:05Alex Miller (Clojure team)you could do a second level of s/multi-spec on :local#2021-09-1312:05Ben SlessSo I need to "subtype" bindings then or between them#2021-09-1312:06Ben Slessis there a reason to prefer multi-spec to or?#2021-09-1312:06Alex Miller (Clojure team)it's designed for open extension, and it doesn't tag the conformed value#2021-09-1312:07Ben Slessokay, and how would you name the specs you select between? would you even name them?#2021-09-1312:08Alex Miller (Clojure team)I think I would name them, and I would name them something related to the ast node type#2021-09-1312:09Alex Miller (Clojure team)in general, I find relying on s/merge and s/multi-spec to "build up" evolves better than trying to s/or#2021-09-1312:10Ben SlessI have no sense of style or habits with spec so I thought it would be better to ask instead of produce something horrendous šŸ™‚#2021-09-1312:11Ben Slesshow would you name the specs you dispatch to? ::arg-binding , ::binding.arg , :binding.type/arg?#2021-09-1312:12Ben Slesssomething else entirely?#2021-09-1312:29Alex Miller (Clojure team).'s should generally be used only in qualifiers, so I'd rule out the middle one#2021-09-1312:30Alex Miller (Clojure team)since these are ast nodes I would probably be trying to include either "ast" and/or "node" in the name of each one#2021-09-1312:31Alex Miller (Clojure team)whether you put that in the qualifier or in the name is a matter of taste#2021-09-1312:31Ben SlessIn that case, I can even start from naming it something like :node/binding then the multi case could be :node/binding.arg? :node/arg.binding?#2021-09-1312:32Ben SlessLike I said previously, I still have no sense of taste when it comes to specs, so I prefer to ask someone with experience#2021-09-1312:32Alex Miller (Clojure team)"node" is not sufficiently unique - you should treat this as a global naming space#2021-09-1312:32Alex Miller (Clojure team):clojure.tools.analyzer.node/binding#2021-09-1312:32Ben Slessack#2021-09-1312:33Alex Miller (Clojure team)this is admittedly a bit painful to use without an alias, and aliases to namespaces that don't exist don't work, so you either need to do the create-ns / alias dodge or use a real namespace#2021-09-1312:33Ben SlessIf I require the analyzer as ana I can also go with ::ana/node.binding#2021-09-1312:33Alex Miller (Clojure team)(this is going to get better in 1.11 with lightweight alias support)#2021-09-1312:34Alex Miller (Clojure team)yes, that would work#2021-09-1312:34Alex Miller (Clojure team)although .'s in names are bad#2021-09-1312:35Ben SlessI should put that on a sticky note#2021-09-1312:35Alex Miller (Clojure team)the problem is they have meaning understood in the compiler, which is: class name for symbols#2021-09-1312:37Alex Miller (Clojure team)the https://clojure.org/reference/reader#_literals spec actually says keywords "cannot contain '.' in the name part, or name classes"#2021-09-1312:35Alex Miller (Clojure team)so I'd either use node-binding or bite the bullet on moving it into the qualifier#2021-09-1312:36Ben SlessOr cheat and create-ns for my own convenience, but that seems like bad form#2021-09-1312:38Alex Miller (Clojure team)well, you'd hardly be alone :)#2021-09-1312:38Ben Slesshm :thinking_face:#2021-09-1312:38Alex Miller (Clojure team)and like I said, better solution for this is coming#2021-09-1312:38Ben SlessBut it won't be backwards compatible#2021-09-1312:39Alex Miller (Clojure team)true#2021-09-1312:39Alex Miller (Clojure team)that is important here if it's part of analyzer.jvm#2021-09-1312:39Ben SlessI'll try to be considerate in my implementation#2021-09-1312:40Ben SlessIt would be a tall order for it to be integrated into analyzer.jvm but having it as a companion library would be nice#2021-09-1312:45Ben SlessWhile we're here, I'll also use this opportunity to ask regarding namespace and artifact naming convention (and copyright) A "correct" name for the spec's ns would probably be clojure.tools.analyzer.jvm.spec, but can I distribute a library whose root namespace is clojure ? Even if the license allows it, which I'm not sure it does, it feels rude#2021-09-1312:49Alex Miller (Clojure team)I assumed you were spec'ing this to include with tools.analyzer.jvm, where it would be fine#2021-09-1312:50Ben SlessThat was my intention, yes#2021-09-1312:50Alex Miller (Clojure team)seems like it would be the greatest utility to the most people as either part of that library or as a separate contrib lib#2021-09-1312:51Alex Miller (Clojure team)currently tools.analyzer still supports pretty old clojure versions, older than spec#2021-09-1312:51Ben SlessSo it would make sense as a contrib lib#2021-09-1312:52Alex Miller (Clojure team)I think so#2021-09-1312:52Alex Miller (Clojure team)I'd have to ask Rich but I don't think he would object to that#2021-09-1312:52Alex Miller (Clojure team)we've taken the approach of appending ".specs" for stuff like this, so it would presumably be tools.analyzer.jvm.specs
#2021-09-1312:53Ben Sless:thumbsup:#2021-09-1312:54Ben SlessIf I'm doing it with the intent of it becoming a contrib library I should probably start with a tools.analyzer.specs then merge the jvm stuff on top of it#2021-09-1313:03Alex Miller (Clojure team)I mean, there isn't really anything but .jvm, so I think it would be fine to include it all in one#2021-09-1313:04Alex Miller (Clojure team)I can get a repo set up this week for you#2021-09-1322:36Franco GasperinoWhat is the recommend route for evaluating the result of a clojure.spec.gen.alpha/check result summary set with the test framework (runners, assertions, and family) of clojure.test? The test.check library appears to have defspec, but it's unclear if it's a good fit to glue to spec/fdef generative, as examples seem to favor it's own clojure.test.check.generators in the guide examples.#2021-09-1323:49Alex Miller (Clojure team)I would recommend not doing that and separating your generative tests from your unit tests#2021-09-1400:05Michael Gardnercould you elaborate on this?#2021-09-1400:53Alex Miller (Clojure team)unit tests are generally fast and you run them a lot. generative tests are usually slow and you run them less often, they also often require most setup (for things like instrumentation, mocking etc).#2021-09-2120:02johanatani have something like the following:
(deftest specd-functions-pass-check
  (stest/instrument)
  (macros/filtered-check `a-namespace/a-function (num-tests 1))
  (macros/filtered-check `b-namespace/a-function (num-tests 3))
#2021-09-2120:02johanatanwhere macros/filtered-check is the following:#2021-09-2120:03johanatan
(defmacro filtered-check [sym opts]
  `(let [fltr# (or (.get (js/URLSearchParams. js/window.location.search) "filter") "")
         check# (fn [res#]
                  (cljs.test/is (= true (-> res# :clojure.spec.test.check/ret :result))
                                (goog.string/format "spec check failure:\r\n%s"
                                                    (with-out-str (cljs.pprint/pprint res#)))))]
     (when (or (= fltr# "all")
               (clojure.string/includes? (clojure.string/join "/" [(namespace ~sym) (name ~sym)]) fltr#))
       (let [check-res# (clojure.spec.test.alpha/check ~sym ~opts)]
         (clojure.spec.test.alpha/summarize-results check-res#)
         (if (nil? check-res#)
           (cljs.test/is false "stest/check result was nil. did you pass it any valid symbols?")
           (doall (map check# check-res#)))))))
#2021-09-2120:03johanatan^^ clojurescript obviously#2021-09-2120:03johanatan(designed for figwheel-main autotesting)#2021-09-2120:04johanatanalso i don't write "unit tests" as they are subsumed by property-based tests#2021-09-2120:04johanatanthe "filtering" feature takes care of the "slow and don't run often" aspect#2021-09-1323:50Alex Miller (Clojure team)some people have written code to wrap spec check tests into clojure.test, but I don't have that at hand#2021-09-1402:51Franco Gasperinook thank you#2021-09-1402:52Franco Gasperinofollowing up on that - do you organize a generative fdef in the same namespace as the function it's associated with, or with a peer testing namespace similar to a unit test?#2021-09-2120:05johanatanpersonally i write the fdef right beside the defn#2021-09-2120:05johanatanbut that is probably a matter of personal opinion#2021-09-2120:06johanatani do have a few "integration tests" that span multiple functions / namespaces defined in the test/ area itself#2021-09-2120:06johanatane.g., "roundtrip" style tests#2021-09-2119:55johanatanis there ever a situation where the :ret value passed to a (failing) fdef :fn checker would diverge from the value you get when you run the function in question with the associated :args manually ?#2021-09-2119:55johanatan(I'm seeing this currently and it's a bit baffling)#2021-09-2119:56johanatanthese are the reported values:
:cljs.spec.alpha/value
  {:args {:maps ({:A/A ""} {:A/A ""})}, :ret {:A/A " "}},
#2021-09-2204:29colinkahnI think unless itā€™s being conformed into that different value it will be the same. You could try doing:
(s/conform (:ret (s/get-spec `your-function)) {:A/A ""}) 
as a sanity check to make sure your spec isnā€™t changing it during conforming.
#2021-09-2218:14johanatanthe :ret spec here is simply map?#2021-09-2218:15johanatanbut for completeness:#2021-09-2218:15johanatan
utils.utils=> (s/conform (:ret (s/get-spec `-merge-attribs)) {:A/A ""})
{:A/A ""}
#2021-09-2218:15johanatancould this be a bug in clojure spec ? it is in alpha after all. @U064X3EF3 ?#2021-09-2218:17Alex Miller (Clojure team)sorry, can you re-explain?#2021-09-2218:17johanatanis there ever a situation where the :ret value passed to a (failing) fdef :fn checker would diverge from the value you get when you run the function in question with the associated :args manually ?#2021-09-2218:18johanatanthese are the reported values:
:cljs.spec.alpha/value
  {:args {:maps ({:A/A ""} {:A/A ""})}, :ret {:A/A " "}},
#2021-09-2218:18johanatanthis is the manual repl invocation of the function in question:
utils.utils=> (-merge-attribs '({:A/A ""} {:A/A ""}))
{:A/A ""}
the manual repl invocation is the "correct" one
#2021-09-2218:19johanatanNotice that the one reported by spec includes a \space and the one returned from the manual invocation is merely the empty string.#2021-09-2218:20johanatan(the :ret that is)#2021-09-2218:25Alex Miller (Clojure team)it seems like you're asking me a question about your function, not spec#2021-09-2218:26Alex Miller (Clojure team)maybe there is more here than you can "see" - that "empty" string could have a Unicode non-breaking space or something in it#2021-09-2218:26Alex Miller (Clojure team)or zero-width space, whatever that weird unicode thing is#2021-09-2218:28Alex Miller (Clojure team)in general, my answer to your original question is no (unless you're using spec :stub instrumentation or something)#2021-09-2218:45johanatani think you misunderstood.#2021-09-2218:46johanatanspec says that for some input a specific return value is returned. when i run that function manually in the repl with the exact same input a different value is returned.#2021-09-2218:46johanatani.e., "" != " "#2021-09-2218:46johanatanregardless what that character is#2021-09-2218:46johanatanempty string is empty string#2021-09-2218:46johanatan" " is not empty string#2021-09-2218:47Alex Miller (Clojure team)a 1-character string with a Unicode zero-width space character looks like an empty string#2021-09-2218:47johanatanah gotcha#2021-09-2218:47johanatanis there any way to get the raw characters to be printed ?#2021-09-2218:47johanatanfrom what i've seen spec seem to report escape sequences for those#2021-09-2218:47johanatanlike \b, \f etc etc#2021-09-2218:47Alex Miller (Clojure team)not those#2021-09-2218:48Alex Miller (Clojure team)U+200B#2021-09-2218:48johanatanah, ok. thanks. any other characters like that i should know about ?#2021-09-2218:48Alex Miller (Clojure team)well, that's one I've had this problem with. there are probably others#2021-09-2218:49johanatanhehe šŸ™‚ too bad https://clojuredocs.org/clojure.core/char-escape-string doesn't exist in cljs (although seems that wouldn't cover all of these anyway)#2021-09-2218:49Alex Miller (Clojure team)you add a debug in your :fn function to print the count of the input strings or map char etc#2021-09-2218:49johanatanšŸ‘ thx#2021-09-2218:50Alex Miller (Clojure team)I'm just guessing here - this is a possible answer that matches what I see, but you should validate#2021-09-2218:50johanatanyes of course#2021-09-2218:51Alex Miller (Clojure team)these kinds of things happen with generative tests :)#2021-09-2218:55johanatanunfortunately i think it might be something else:#2021-09-2218:55johanatan
comparing v: '1 1', and: '1 1', count v: 5, count joined: 5 false
#2021-09-2218:56johanatanooh, there's two non-printable chars in there#2021-09-2218:56johanatanand they're probably in opposing positions#2021-09-2218:58johanatanfound it. silly mistake.#2021-09-2218:58johanatani added a spurious reverse on the array of values i was comparing against#2021-09-2218:59johanatanmight be worth thinking about how to better report string values (adding perhaps unicode escapes for these as well as the platform default escapes)#2021-09-2219:00johanatanthanks for your help!#2021-09-2219:07johanatanfyi... i had used simple-type-equatable as my inner-type for a recursive map gen. if i had used instead simple-type-printable-equatable, this probably wouldn't have happened#2021-09-2119:56johanatanthis is the manual repl invocation of the function in question:#2021-09-2119:56johanatan
utils.utils=> (-merge-attribs '({:A/A ""} {:A/A ""}))
{:A/A ""}
#2021-09-2119:56johanatanthe manual repl invocation is the "correct" one#2021-09-2119:58johanatan(i have deleted the target/ directory and restarted the repl so it should not be a stale code issue)#2021-09-2203:23johanatananyone?#2021-09-2506:09apbleonardWhen conforming a s/multispec (with clojure.spec.alpha) is it possible to be returned which multimethod dispatch value was used, similar to the "tags" returned when conforming an s/or?#2021-09-2507:50dgb23Not sure if I understand the question:
user=> (require '[clojure.spec.alpha :as spec])
nil
user=> (defmulti tag ::tag)
#'user/tag
user=> (defmethod tag :foo [_] (spec/keys :req [::tag]))
#object[clojure.lang.MultiFn 0x522b2631 "
#2021-09-2507:55dgb23Do you mean that after conforming the above, you somehow forget that :user/tag was used to dispatch and you want to explicitly encode that in the conformed value?#2021-09-2511:45apbleonardWell not quite. In your example I'd like the dispatch value :foo to be explicitly encoded in the conformed result.#2021-09-2512:21apbleonard...as per https://www.cognitect.com/cognicast/103: "In spec, there is valid question mark, just a validator that's Boolean. But, the more interesting function is conform, which takes the data and a spec, and gives you a de-structured, labeled version, if you will, of the data where every branching point that was detected, the result of it was labeled. If you had some arbitrary complex predicate that you needed to satisfy and it was one of threeā€“X or Y or Zā€“and it matched Y, there'll be a key there that says Y is what they supplied. Your code won't need to figure that out again because the spec did that, de-structured it, and labeled it. There's a path system in spec. It's part of the design. Everywhere there's a branch, there's a label. "#2021-09-2514:45apbleonardI guess I need an s/or. I liked the clear dispatching and the hierarchical keys offeredby multispecs though.#2021-09-2709:36arohnerAre there any libraries for turning XML XSD definitions into specs?#2021-09-2820:52johanatanis there a way to get at the unconformed value in an fdef :fn :ret value ?#2021-09-2820:52johanatani.e., the non-marked up, pristine original return value#2021-09-2820:54johanatanah, looks like: https://clojuredocs.org/clojure.spec.alpha/unform may do it#2021-10-0118:05bortexzIn spec 2, one can select from inside a collection of schemas instead of a single schema. Is there a way to make that work with collection-like maps (e.g. a sorted-map timeseries indexed on timestamp, specced with map-of) to select from the values of such map?#2021-10-0317:19Carlois clojure.spec.gen.alpha/let not available in clojurescript?#2021-10-0317:28CarloThis page https://clojurians-log.clojureverse.org/test-check/2019-02-26 and this https://github.com/clojure/test.check/blob/88eebf105fbb01db17ce12a3bff207cc395270b9/src/test/clojure/clojure/test/check/test.cljc#L18 seem to imply that I should be able to use it, but I don't see how#2021-10-0317:30CarloIf I try require
[clojure.spec.gen.alpha :as gen :include-macros true :refer-macros [let]]
it will say that there is no macro named let
#2021-10-0318:39Carlo@U04V15CAJ just because of the previous link#2021-10-0318:42colinkahnIt doesn't because let is a macro and can't be lazy loaded (I believe that's the reason anyways). You can use the non-macro version, bind which is part of clojure.spec.gen.alpha#2021-10-0318:45colinkahnJust to clarify, let is not available in the Clojure version of clojure.spec.gen.alpha either.#2021-10-0319:09Carlo@U0CLLU3QT if the problem is the lazy loading, can I somehow preload it? Yes, I was trying to use let because it seems cleaner wrt bind and fmap which is what I'm using now. So, are the tutorials outdated? Was let removed from clojure.spec.gen.alpha at some point? I thought I used it in the past šŸ˜®#2021-10-0319:10Carlomoreover, where is this code https://github.com/clojure/test.check/blob/master/src/test/clojure/clojure/test/check/test.cljc#L1054 getting the gen/let?#2021-10-0321:03Carlook, I'm so dumb, I should just have imported clojure.test.check.generators where the let macro I'm interested in is actually defined :man-facepalming:. Thanks for the help!#2021-10-0321:04colinkahnThose examples look like just test.check, which specs gen namespace lazy load behind statically defined functions. It's a design decision in spec, that all the functions in the gen namespace can be used without loading clojure.test.check.generators until needed (for example people could define specs for use in production code but only incur loading the test.check.generators namespace in their test suite where they'll use that functionality). In ClojureScript, it's the same but there is no true lazy loading, but you only need to manually require the test.check namespaces when you need them.#2021-10-0321:04colinkahn^ but yeah if you don't care about this stuff then just import test.check generators like you just posted šŸ˜Š#2021-10-0321:12Carloawesome! Now I understand why the re-export was needed and what's the problem with re-exporting the let macro. Thanks again @U0CLLU3QT#2021-10-0317:43mafcocincoI briefly used spec.alpha but didnā€™t end up doing much with it. Iā€™m interested in using spec2 for message validation in a pub/sub system. Can someone point me to the current ā€œstate of the artā€/example use cases/best practices/etc. for using spec2?#2021-10-0317:58bortexzCurrently spec 2 is a work in progres and not ready for production use, so there is not much to be found, but if youā€™re still interested to give it a try, thereā€™s some info on the githubā€™s repo wiki (https://github.com/clojure/spec-alpha2/wiki), and the talk where Rich talks about it https://www.youtube.com/watch?v=YR5WdGrpoug&amp;ab_channel=ClojureTV#2021-10-0322:28seancorfieldSpec 2 still has a lot of bugs and has gone through a lot of changes -- with a number of additional changes still planned. We have been very heavy users of Spec (1) since it first dropped -- in production code -- and for about three months we ran a branch of our code tracking Spec 2 but it just became intractable to keep tracking it and working around the bugs...#2021-10-0403:49mafcocincogot it. I will use spec 1 then. thanks.#2021-10-0501:55mafcocincoafter a bit of research, I decided to use metosin/malli. Seems to have all the features I need and I like the data-driven orientation.#2021-10-2822:19peterdeeHi! At one time there was a function spec* in spec-alpha2 that allowed programmatic definition of specs. I donā€™t see that in the current SHA version. Is it still possible to define specs programmatically?#2021-10-2822:22seancorfieldAre you looking for register @peterd? https://github.com/clojure/spec-alpha2/blob/master/src/main/clojure/clojure/alpha/spec.clj#L429#2021-10-2822:36peterdeeI am looking at some code I wrote about two years ago. (Wish I could recall!) I was using both s/spec* and s/register. I was using backquoted forms or variable values with spec*. Hmmmā€¦ It looks like the spec* calls are typically wrapped in a s/register, but s/register takes a spec, not a body (form) so my guess is that register isnā€™t going to allow programmatic operation???#2021-10-2822:40Alex Miller (Clojure team)You probably want resolve-spec or maybe create-spec depending on what you're doing#2021-10-2822:41Alex Miller (Clojure team)resolve-spec takes a form and returns a spec object#2021-10-2822:42peterdeeUgh! How did I miss that? Yes, I think resolve-spec.#2021-10-2822:42peterdeeThanks!#2021-10-2822:44Alex Miller (Clojure team)https://github.com/clojure/spec-alpha2/commit/bab9b91d2b8490e9c7bdbf3b1500636b28bb4e75#2021-10-2822:50peterdeeThanks. My program is happy again.#2021-10-2822:52peterdeeGuess I should have looked at the commit log!#2021-10-2823:06Alex Miller (Clojure team)I think https://clojure.github.io/spec-alpha2/ is relatively synced to the code too#2021-10-2823:10peterdeeSimple programmatic access is a nice feature. Iā€™m wrote a parser for MiniZinc, a popular open-source combinatorial solver language; it has its own language for defining data. (Will open source my parser soon). The specs allow you to catch users errors that the parser misses.#2021-11-0215:57Drew VerleeIs it possible to have spec generators for two different contexts? E.g in one i want my user's uuids to be a random uuid, and in another for them always to be the same value.#2021-11-0215:58Alex Miller (Clojure team)there are several functions that accept an override generator map#2021-11-0215:59Alex Miller (Clojure team)so you could swap out at point of use (during exercise, instrumentation, etc)#2021-11-0215:59Drew VerleeThanks alex, I'm reading the docs now.#2021-11-0315:19timoHi, anyone has an idea how to generate spec from json-schema?#2021-11-0315:38vlaaadOh, I wanted to make a library that does that#2021-11-0315:38vlaaadLast time I checked no solutions existed#2021-11-0320:14qqqI posted the following in #clojure , and @hiredman pointed out clojure-spec satisfies the properties
Consider the problem of String -> AbstractSyntaxTree. Solving this problem is called Parsing, and we can solve it via Regex + LL / LALR / Peg / Parsec / ... . In particular, we can view Regex / Peg as "DSLs" for String -> AbstractSyntaxTree
Now, suppose we are trying to define a DSL -- which can be viewed as a subset of s-exprs. For example, Scheme is a "s-expr DSL"; Lisp is a "s-expr DSL", edn is a "s-expr DSL", clojure is a "s-expr DSL" -- in all 4 cases, all valid data values look like s-exprs.
Question: What is the Regex / Peg equiv for doing "s-expr -> DSLs" ? I.e. what is a "pattern description language" for specifying what subset of s-exprs are valid exprs in the DSL ?
#2021-11-0320:15qqqA followup to this then is: Regex is backed by DFA / NFA theory. Other parsing techniques is backed by (limited subset of) Context Free Grammars. Is there a "theory" behind clojure-spec, or is it driven largely by battled tested useful tactics ?#2021-11-0320:17hiredmanthe theory is basically the same as parsing where parsing is sort of logically defined over any sequence of tokens of any type, but we usually think of parsing as over a sequence of character or string tokens#2021-11-0320:18hiredmanit is parsing but over lazy seqs of lazy seqs (which introduces another novel kind of recursion)#2021-11-0320:18hiredmanparsing over a complex token type#2021-11-0320:19hiredmanfrom a parsing viewpoint spec is usually just used as a recognizer "does this match the grammar or not" but it can be used as a full parser#2021-11-0402:54qqqThere are three types of input data: [a] sequence of chars (standard parsing) [b] sequence of tokens (basically parsing, but after the lexing stage) [c] TREE of tokens (what we have) I can see how [a] == [b]; but I don't see how one define a generalization that covers both [b] and [c]. I'm not saying one does not exist, just that I'm not seeing it. Can you clarify on this ?#2021-11-0402:54qqq@hiredman: ^#2021-11-0402:55hiredmana tree is a recursion of sequences, so just take the parsing over a sequence and apply it recursively#2021-11-0402:57qqqCFG is a language that defines patterns over sequences of tokens. What defines patterns over trees ?#2021-11-0402:58hiredmanif you make tokens terminals or another cfg#2021-11-0402:58hiredmanbecause the tokens are complex and also have a structure to be parsed#2021-11-0403:00qqqAs far as I can see Cfg[Token] and Cfg[Cfg[Token]] both define patterns over sequences; can you point me at a research paper / book ? I need a more detailed exposition.#2021-11-0403:01hiredmanno,I've never seen this written up#2021-11-0403:01hiredmanbut like, if you write out a grammar, say in bnf, everything is a terminal or a rule, and the terminals are the tokens#2021-11-0403:03hiredmanif instead of putting in, like, literal 'if' in the grammar as the token, you say, the terminal is actually a member of this set of sequences and describe the set as a cfg#2021-11-0403:11qqqI think the core issue here is Vec<Vec<T>> does not represent the full space of Tree<T>; so this idea of Cfg[Cfg[Token]] seems to not recognize Trees.#2021-11-0416:20respatializedCan you give an example value that satisfies Tree<T> but not Vec<Vec<T>>?#2021-11-0404:12hiredmana tree Tree<T> = Leaf<T> | Node<List<T>>#2021-11-0404:13hiredman(to make up some terrible type notation)#2021-11-0404:13hiredmanerrr#2021-11-0404:14hiredmanTree<T> = Leaf<T> | Node<List<Tree<T>>>#2021-11-0404:14hiredmanSo you walk the list in the node parsing it, where each token type is itself tree to be matched against some other parser#2021-11-0416:20respatializedCan you give an example value that satisfies Tree<T> but not Vec<Vec<T>>?#2021-11-0423:48qqq@afoltzm: All leaves in a Vec<T> have depth 1. All leaves in a Vec<Vec<T>> have depth 2.#2021-11-0423:49qqq@hiredman: Here I think is the problem Cfg[T] pattern language over Vec<T> Cfg[Cfg[T]] is a pattern language over Vec<Vec<T>>#2021-11-0423:50hiredmanthat depends on exactly what the internals of CFG are#2021-11-0423:51hiredmanVec<Vec<T>> is not a tree, so it is not what we have#2021-11-0423:51hiredmanedn/clojure data is a tree#2021-11-0423:51hiredmanso you have a recursive data type like Tree<T> = Leaf<T> | Node<List<Tree<T>>>#2021-11-0423:52hiredmanand one way to "parse" such a thing is to recursively parse everywhere the datatype does#2021-11-0423:54hiredmanso you can imagine a parsing that walks through List<String> building up a parse tree, and a fundamental operation of such a parser is doing an equality comparison of the strings in the list with the strings that are the terminals of the grammar#2021-11-0423:55hiredmannow say, what if you grammar is defined in some dsl/library that has some built in "functions" like "is-if-string?" that runs true when given the string "if"#2021-11-0423:56hiredmanso you go to your grammar and replace the if terminal with a "call" to is-if-string?, so instead of the parser doing an equality check it calls the predicate on the token#2021-11-0423:57hiredmannow change from string tokens to arbitrary data structures, and use a predicate like three-element-vector-all-0?#2021-11-0423:59hiredmannow imagine you have a higher order function 'member' that takes a set, and returns a predicate that returns true if its argument is in the set#2021-11-0423:59hiredmanwhat is a cfg? it is a description of a (possibly infinite) set that contains the strings of your language#2021-11-0500:01hiredmanso you have a cfg where the terminals are predicates, and the predicates can be defined as themselves being a cfg#2021-11-0502:46qqq@hiredman: I agree with you that a pattern language for Tree<T> exists. I also agree that this pattern language has derivation rules, where some rules are choices of the form choiceA | choiceB | choiceC | ... ; and other nodes are of some form that builds up trees. I am not sure that calling this CFG-like is correct.#2021-11-0505:38hiredmanIt is a CFG#2021-11-0505:42hiredmanRegexs describe sets, CFGs commonly do not exhaustively list all terminals instead defining sets of terminals via regexes. CFGs describe sets. Given a cfg, swap the sets described by regexes for sets described by cfgs. You've swapped sets for sets, just a different set builder notation.#2021-11-0505:45hiredman(I am well aware that CFGs are more powerful than regexes, that just means that CFGs can describe more sets than regexes can, but they are still sets)#2021-11-0505:58qqqPoint me to one text book / peer reviewed paper that shows how to use CFG to pattern math over trees.#2021-11-0506:01seancorfield@qqq I think you're being very belligerent here. If you want to find peer reviewed papers etc, you should use Google/Bing/whatever and find them yourself instead of demanding that others here do your research for you.#2021-11-0506:01seancorfieldDoes https://www.sciencedirect.com/science/article/pii/S0196885816300598 come close to what you're looking for?#2021-11-0506:02seancorfield(that was just the first vaguely relevant seemingly result from a very, very brief search that you could easily do yourself)#2021-11-0506:05qqq@seancorfield: The link you posted has nothing to do with parsing.#2021-11-0506:06qqqMy point here is: CFG, as traditionally used in CS, https://en.wikipedia.org/wiki/Context-free_grammar , is for List -> Tree. Besides @hiredman, I have never heard anyone claim that CFG can be used to parse Trees -- and that such a textbook / peer reviewed paper does not exist.#2021-11-0506:08hiredmanThe CFGs you are looking at are already being used to parse simple trees#2021-11-0506:09hiredmanIn that the terminals are sets of lists of character, not just sets of characters#2021-11-0506:10qqq@seancorfield: Would it be less "very belligerent" if I instead wrote it as: "Point me to one text book / peer reviewed paper that shows how to use CFG to pattern match over trees, and I'll be convinced your definition makes sense." ? Here, @hiredman is making a claim that standard usage of CFG does not claim. I am asking him to back up him claim with peer reviewed article, rather than his own claims.#2021-11-0506:11hiredmanA cfg is not a function with the type list -> tree, you should maybe read that Wikipedia article a little closer#2021-11-0506:12qqqGiven, an alphabet Sigma, CFGs define a subset of Sigma^* which can be mapped do an AST specified by the derivation rules.#2021-11-0506:13qqqNote that Sigma^* is over Vec<Sigma>, not a Tree.#2021-11-0506:14qqqIf you can point me to one peer reviewed theoretical CS resource that claims that CFGs recognize subsets of Tree<Sigma>, I'll concede the argument.#2021-11-0506:15qqqOtherwise, I am maintaining my claim that CFGs are only used to recognize Sigma^*, which is Vec<Sigma> in our discussion so far.#2021-11-0506:15hiredmanThey don't need to recognize Tree<Sigma>, if you accept that tokens in you "language" can be any type, not just strings, then you make you token type Tree#2021-11-0506:16hiredmanSo sigma is a set whose elements are trees#2021-11-0506:17qqqSigma, in typical CS literature is either {0, 1}, or alphanumeric. Is there anyone else in the CS Theory community that uses the definition you use ?#2021-11-0506:18hiredmanSure, because parsers typically parse text, but just because it's typical that doesn't mean that is all there is#2021-11-0506:19qqq@seancorfield in DMs asked me to drop this conversation, so I'm leaving it here as is.#2021-11-0506:19hiredmanA cfg is a logical/mathematic object, where "string" is an abstract notion that can be anything#2021-11-0506:27qqqI'm making one more statement because I think I can resolve our disagreement: Traditional CFGs have type signature: Vec<Sigma> -> bool I am looking for a pattern language to specify functions of type signature: Tree<Sigma> -> bool @hiredman if you "add trees to Sigma" as you suggest, CFGs still have type signature Vec<Sigma> -> bool the only difference is that Sigma is now defined as Sigma = Letter | Tree ... Now, in this case, I will concede that the CFG is "recognizing lists of trees", but this CFG is not recognizing individual trees. I.e., the type signature is Vec<Tree> -> bool; whereas I am after Tree -> bool.#2021-11-0512:44Alex Miller (Clojure team)Does this have anything to do with spec? Can we move further conversation to #off-topic or something?#2021-11-0515:37jerger_at_ddaHi, is there a way to generate a good documentation from speced functions? I know the discussion about spec meta from last year but did'nt find any update ... anything new ?#2021-11-0515:42Alex Miller (Clojure team)the doc generator we use for core and contrib (autodoc) does make docs for specs#2021-11-0515:44Alex Miller (Clojure team)but autodoc is frustratingly hard to use in practice#2021-11-0515:44jerger_at_ddado you know what authodoc version clojure uses?#2021-11-0515:44jerger_at_ddaI can find a tag 1.1.2 but that's 5 years old ...#2021-11-0515:44Alex Miller (Clojure team)yeah :)#2021-11-0515:46Alex Miller (Clojure team)you can see an example with the spec doc at https://clojure.github.io/clojure/clojure.core-api.html#clojure.core/defn#2021-11-0515:46jerger_at_ddašŸ‘#2021-11-0515:46jerger_at_ddasaw this already šŸ™‚#2021-11-0515:47jerger_at_ddaIf clojure doc is generated with 1.1.2, I will give the autodoc-lein an update to use the 1.1.2 version also ...#2021-11-0515:47Alex Miller (Clojure team)autodoc is conceptually well structured (split into separate programs to "collect" data and export docs, but super difficult to use well in practice#2021-11-0515:48jerger_at_ddayea .. a read about#2021-11-0515:49jerger_at_ddado have a link for the setup clojure uses itself ?#2021-11-0515:49Alex Miller (Clojure team)https://github.com/clojure/clojure-api-doc#2021-11-0515:50Alex Miller (Clojure team)but Clojure's use of autodoc is particularly weird#2021-11-0515:50Alex Miller (Clojure team)it's using 1.1.2 though#2021-11-0515:55jerger_at_ddacool - thanx. If we find our way through I will give feedback ... maybe we did some steps in right direction valuable for others :-)#2021-11-0515:56Alex Miller (Clojure team)I would love to take the good stuff in autodoc and turn it into something actually usable (Clojure's specific usage of it for core and contrib is additional layer of bespoke weirdness over the top)#2021-11-0515:56Alex Miller (Clojure team)but it's hard for that to ever get enough priority#2021-11-0515:57jerger_at_ddaWe will spend one day on this ... let's see wether there is going sth on šŸ™‚#2021-11-0515:42Eugenwhat spec (or related) libraries are there for testing database services are there ? I heard about such a library on a podcast but don't remember which one. Context: I would like to test a Java ERP solution and services will depend on data. I am curious if generative testing can help with this.#2021-11-0518:04Colin P. HillWhat do you want the library to do?#2021-11-0707:12Eugenhi, I would like to do db integration tests and insert a bunch of related records based on the relationship betwe them. I want to test user roles - and that requires user-role record but also depends on user records and maybe account or permission records. These should be generated as well .#2021-11-0707:12Eugenkind of along those lines#2021-11-0707:12EugenI remember a library designed for this purpose - but don't remember which#2021-11-0813:15Colin P. HillIt sounds like you might actually be describing a library made at my company: https://github.com/reifyhealth/specmonstah Obviously a bit of conflict of interest in me highlighting it, but if it helps, I haven't personally worked on it and haven't yet even learned how to use it šŸ˜…#2021-11-0822:24Eugenyep, that is the one#2021-11-0822:24Eugenthanks#2021-11-0517:04qqq@alexmiller: Valid point, future (if any) debates on whether CFGs can recognize trees will be moved elsewhere. If you don't mind, I do have a question for you. 'spec' is basically a pattern language which, among other things, can be viewed as a shorthand for describing functions with type signature 'Tree -> bool' . (Given a spec, some s-exp trees are accepted; others are rejected.) Question: in the design of spec, is there some formal theoretical foundation spec is built upon (say, the tree equiv of regex / cfg / ll parsing / lr parsing / ...), or has it largely been a collection of useful techniques (i.e. we should be able to use it to validate pattern A, pattern B, pattern C, ... ).#2021-11-0517:53Alex Miller (Clojure team)Spec is built on two logical foundations - sets (maps as sets of attributes) and regular expressions (not regex as string matching but the more general idea). The latter impl is based on Matt Mightā€™s work with regular expression derivatives, but that's really an implementation detail. This is not about trees or parsing, it's about whether a value is in the predicative set described by the spec. As it is inherently predicative, and predicates can be anything, this is I think much broader than pattern matching.#2021-11-0517:54Alex Miller (Clojure team)From day 1, the ideas of being predicate based, and automatically generative were considered to be required.#2021-11-0518:02Alex Miller (Clojure team)specs work on values, there is no s-exp tree#2021-11-0518:02Alex Miller (Clojure team)there's no explicit notion of trees at all#2021-11-0518:04Alex Miller (Clojure team)specs are descriptors/validators of sets of accepted values. some specs are composite and describe composite values. conceptually, I guess you could say those are trees, but that is more implicit than explicit.#2021-11-0518:05Alex Miller (Clojure team)if specs are "is this value in the set?", generators are "give me example values in that set", so these are intrinsically related#2021-11-0519:36Colin P. HillGiven a map and a spec that describes it, is there a way to yield a map which only contains those keys which were specified? I don't want to write my spec so that it rejects unspecified keys, but I also don't want to write what is effectively another description of the map shape in the select-keys that I'll need before handing the map down to the database.#2021-11-0519:42Alex Miller (Clojure team)no, I know people have made things around spec#2021-11-0519:42Alex Miller (Clojure team)in spec 2 we have added a validation flag to validate in a "closed" mode#2021-11-0519:43Alex Miller (Clojure team)we very much believe this should not be a property of the spec (but it may be a useful property of an act of checking)#2021-11-0519:45Colin P. HillBig agree on that. I'm working with a custom extension to spec which adds closed map validations. It's been useful for making sure we don't hand things off in places where extra fields might be damaging, but there are other places where we want to describe the same shape while allowing it to be extensible. Currently trying to rewrite a use of it to follow some other strategy.#2021-11-0519:40qqqAre there any guarantees on the running time of spec? I.e. given a s-exp with at most N nodes and at most D depth, if there some big-Oh expression on the time it takes the spec validator to decide whether the s-exp belongs in the spec ?#2021-11-0519:41Colin P. HillA spec can be any arbitrary predicate, so not in the general case#2021-11-0519:41Alex Miller (Clojure team)given that predicates are arbitrary functions, no#2021-11-0601:41qqqSince spec allows arbitrary predicates, which can use arbitrary time / memory, how do most reason about the time/space usage of spec functions / algorithms ? Do they (a) not worry about it, since in most cases, it's fine or (b) have sufficient knowledge of spec internals to know at what time what predicate is called on what, etc ...#2021-11-0602:08seancorfield@qqq The same way you reason about any other functions in Clojure.#2021-11-0603:49qqq@seancorfield: Are you saying this because spec internals are so simple one should be able to reason about it's runtime like one reasons about map / filter / vec-ops ... ; or are you trying to end the conversation before the discussion on spec internals can start ?#2021-11-0603:58seancorfieldYou said "arbitrary predicates, which can use arbitrary time / memory" -- can you reason about those predicates?#2021-11-0604:00seancorfieldBecause if you start from the premise that the complexity of the predicates can be arbitrary, then you have to see that the complexity of Specs that are built on predicates can also be arbitrary, right?#2021-11-0604:03seancorfieldHow would you reason about any library that can take in arbitrary functions (predicates) and calls them on a data structure you pass in?#2021-11-0604:17seancorfieldTwo people have already said there are no guarantees on the run time of Spec -- and I agree -- and there are also no guarantees on the space usage (for the same reason: it's built on arbitrary predicates). But Spec isn't "magic", it's open source so anyone can inspect the source code and reason about it just as much as they can with any other open source library.#2021-11-0604:18seancorfieldI think, in general, people don't worry much about the internals of any library like that -- unless they see unpredictable or unacceptable time/space usage and then they profile their code and identify the root cause.#2021-11-0604:19seancorfieldBut with any of these complex libraries (rules engines and databases/query-based systems also come to mind), you just have to be careful about the complexity of the "queries" you run -- and there are nearly always pathological cases that catch us out. That's just kind of the way things are in software.#2021-11-0604:21seancorfieldAt work, we've used Spec very heavily ever since it was released. We use it for a wide variety of things (I've blogged about our usage of Spec at http://corfield.org if you're interested) -- and we do use it in production code for data validation. Most of the data structures we validate are pretty flat and, in general, relatively small -- so the Specs around them are relatively straightforward. And, in that context, we have seen no performance problems.#2021-11-0604:23seancorfieldBut there are (obviously) going to be data structures that are sufficiently complex that a full Spec is also going to be very complex and checking might be "slow". There might be values that have sufficiently complex semantics that a fully-fleshed out Spec might be "slow" -- but that's true whether you use Spec or whether you write the validation logic by hand (with the caveat that a generic system like Spec is always likely to have more overhead than hand-crafted code).#2021-11-0604:24seancorfieldDoes that help answer your questions @qqq?#2021-11-0604:51qqq@seancorfield That is a reasonable interpretation of my question, but not the way I was intending. Consider for a moment, a strict, non-lazy map. Let's call it smap. Let f be some arbitrary function. What is the running time of (smap f lst) Now, you could make the argument that because f can take arbitrary memory/time, we can't say anything about (smap f lst). On the other hand, we can say, well it's running time is (reduce + (map (fn [x] (running_time 'f x)) lst)). One can argue about whether this is useful, but knowing the internals of how smap works definitely allows us to create some expression (in terms of f, lst) that expresses the running time of (smap f lst). Now, if we look at https://clojure.org/guides/spec -- unless I am reading it incorrectly, it teaches the reader a mental model of "what value spec will return." What it does not seem to do however, is cover the internals of how spec computes the answer. I.e. it talks about "what", not "how".#2021-11-0604:52qqqSo now, what does this have to do with my earlier question of whether predicates can take arbitrary memory / time. Because spec allows this flexibility, it seems that the answer to "what is the running time of this spec query" depends on the predicates we pass it -- and this likely means that we need to figure out (1) how often will our predicate be called, (2) what data will it be called on -- and this requires a quite detailed knowledge of spec internals.#2021-11-0605:36seancorfieldBut this is true of any library that you pass both functions and data structures into and have it call those functions on those data structures. There's nothing novel about Spec in that regard.#2021-11-0604:55qqqLet's consider this example of Regex vs Spec. In normal regex, we can't specify arbitrary predicates. So it goes something like this: regex -> nfa -> dfa -> run DFA on input string. So we can say: well, the running time for a regex of size S on input of length N will be O(N & 2^S). [The DFA can be exponential in size of the NFA]. This is easy to assert without knowing too much about the internals of how the regex is matched. On the other hand, in the case of spec, because we can pass arbitrary predicates; it seems to reason about spec's runtime / space, we have to figure out how often / when / what input the user specified predicates are called upon -- this is something which AFAIK can only be understood via understanding spec internals, i.e. HOW spec recognizes, rather than WHAT spec recognizes. Does this sound reasonable? @seancorfield#2021-11-0605:36seancorfieldI already answered this in the thread that you either ignored or chose not to respond to.#2021-11-0605:00qqqOne practical question here is: do you ever do "untrusted user input => parse as edn => run spec on it"; do you know whether there exists some malicious input of size N which can cause spec to take 2^N (or even N^6) time ?#2021-11-0605:39seancorfieldJust parsing "untrusted user input" as EDN (or JSON) can be subject to malicious input attacks. That's why there are often recommendations to not do this -- but, practically speaking, people do this all the time. Adding Spec to the mix adds more processing and increases that risk.#2021-11-0605:04EddieSome of this is touched on in a few of the docstrings. https://clojuredocs.org/clojure.spec.alpha/every and https://clojuredocs.org/clojure.spec.alpha/coll-of come to mind. Both are intended to declare a collection where all the elements match a spec/predicate, but s/every will sample the collection and s/coll-of will exhaustively check each element.#2021-11-0605:11Alex Miller (Clojure team)I think most specs are pretty closely tied to the size of collections (s/coll-of will check all N elements of the coll, s/map-of will check all N entries of a map etc). if you're using regex specs, I would look to the regex derivatives papers for complexity bounds (https://matt.might.net/papers/might2011derivatives.pdf and related there but it's exponential in N). From a practical pov though, regexes are mostly used to describe syntax for things like macros, and in general most macro calls have small N (you don't invoke a macro with 100s of args or whatever).#2021-11-0605:12Alex Miller (Clojure team)"do you know whether there exists some malicious input of size N which can cause spec to take 2^N (or even N^6) time" - no, you don't know this, and it is possible.#2021-11-0605:36seancorfieldI already answered this in the thread that you either ignored or chose not to respond to.#2021-11-0605:39seancorfieldJust parsing "untrusted user input" as EDN (or JSON) can be subject to malicious input attacks. That's why there are often recommendations to not do this -- but, practically speaking, people do this all the time. Adding Spec to the mix adds more processing and increases that risk.#2021-11-0606:29qqqDoes spec do memoization / caching / dynamic programming? If not, can we generate 2^N time on O(N) size input as follows: (s/def :foo (s/or :a something_that_uses_foo :c something_that_uses_foo)) then running it on s-expr of (1 (1 (1 (1 (1 (1 ... )))))) This is similar to the cfg rule A => B | C , where B / C both use A. If we use plain backtracking, this can easily go exponential. In the case of strings, I believe packrat / peg gets around this issue by memoization -- using O(N * S) memory where N = length of input, S = # of states in grammar. How does Spec handle this issue? Clearly it has to backtrack at s/or . The question is whether it memoizes results or if it just re-evaluates everything.#2021-11-0606:30qqq@seancorfield: I have a custom "untrusted string" -> s-exp parser that takes O(N) space, O(N) time and O(1) stack space. So the "untrusted user input => edn" step is not a problem for me. However, the "edn => spec" step currently scares me. (Among other things, the example just raised above).#2021-11-0607:11ikitommi@qqq here's an old post about using spec with untrusted inputs: https://quanttype.net/posts/2021-03-06-clojure-spec-and-untrusted-input.html#2021-11-0607:46qqq@ikitommi: Is this how the attack works? You pull you some dependency Foo that has defined some slow spec ::foo-slow-spec. You are unaware of these slow specs (and don't care because you're not explicitly using them). An attacker constructs a HTTP request that you run spec on. They add a field "::foo-slow-spec", which triggers the spec from Foo dependency. And the defense to this attack is to ensure that the incoming data only has keys explicitly mentioned in the spec?#2021-11-0608:07ikitommiyes. you could also have a slow spec in your codebase, as normal input, that is slow.#2021-11-1121:01dominicmIs there a version of flowing s/and for spec1 or a workaround that produces the same effect? (a version that doesn't pass conformed values to subsequent predicates, as described in https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha#nonflowing-sand--new).#2021-11-1122:29colinkahn(s/and (s/nonconforming <your-spec>) ā€¦)#2021-11-1122:30colinkahnYouā€™ll need to wrap each spec that you want to avoid conforming for in s/nonconforming, more specifically#2021-11-1122:52dominicm@U0CLLU3QT you are wonderful, thank you kindly. You've got me out of a very tricky hole!#2021-11-2204:12Andrew ByalaHi, everyone. I have a beginner question about Spec's namespacing as it pertains to maps. I'd like to create two definitions within the namespace, such that the following passes:
(s/conform ::account     {:account-id "abc123",   :amount 50})
(s/conform ::transaction {:from-account "abc123", :to-account "def456", :amount 25})
Note that both definitions reference the same non-namespaced key of amount, but I'd like to define the amount of an account to be an int between -100 and 100, and the amount of a transaction to be any positive int. Furthermore, I'd like to use the same spec for the account-id on an account, as well as both the from-account and to-account keys in a transaction. I can't see any way within a map to say that the value of a particular key corresponds to a specification by name. I'm trying to find a way to do something like this, but can't see how:
; This is not working code... I'm trying to find a way to get here.
(s/def ::transaction-amount pos-int?)
(s/def ::balance-amount #(<= -100 % 100))
(s/def ::account-id string?)

; Both :from-account and :to-account should be mapped to the spec for ::account-id
(s/def ::transaction (s/keys :req-un [[:from-account :as ::account-id]
                                      [:to-account :as ::account-id]
                                      [:amount :as ::transaction-amount]]))
; The :amount here is a ::balance-amount, but the :amount for a transaction is a ::transaction-amount
(s/def ::account (s/keys :req-un [::account-id
                                  [:amount :as ::balance-amount]]))
#2021-11-2204:32seancorfield
(s/def :account/amount  ...)
(s/def :transaction/amount ...)
Then you can use those qualified names in the :req-un part and it will match :account in both cases, with different semantics.
#2021-11-2204:36seancorfieldAlthough a lot of examples use :: that's just shorthand for a qualified name that "happens to match" the current namespace -- but qualified keywords should really really reflect their domain semantics -- they don't need to match code namespaces.#2021-11-2204:36seancorfieldDoes that help @abyala?#2021-11-2204:39seancorfieldWith the account IDs, you can define a common ::account-id and then just define the other names as aliases:
(s/def ::account-id ..)
(s/def ::from-account ::account-id)
(s/def ::to-account ::account-id)
And, again, as above you can use qualified names that match the domain semantics rather than :: for the code namespace.
#2021-11-2204:42Andrew ByalaNifty, @seancorfield. This seems to work for everything I'm looking for:
; This is reused for both transactions and accounts.
(s/def ::account-id string?)

(s/def :transaction/from-account ::account-id)
(s/def :transaction/to-account ::account-id)
(s/def :transaction/amount pos-int?)
(s/def ::transaction (s/keys :req-un [:transaction/from-account :transaction/to-account :transaction/amount]))

(s/def :account/account-id ::account-id)
(s/def :account/amount #(<= -100 % 100))
(s/def ::account (s/keys :req-un [:account/account-id :account/amount]))
#2021-11-2204:42Andrew ByalaIs that what you were suggesting?#2021-11-2204:43seancorfieldThat looks great, yes!#2021-11-2204:43seancorfieldAnd now you can see how the qualified keys provide separate semantic "namespaces" rather than code namespaces.#2021-11-2204:44Andrew ByalaYep, I see that. The piece I was missing was how :req-un would strip that semantic namespace back off again, letting the actual key be whatever I wanted, Super helpful!#2021-11-2204:51Andrew ByalaSeparate question -- do you find you tend to create a separate namespace just to hold specs for other namespaces, especially if they're shared? Or do you tend to put specs within the namespaces containing other logic?#2021-11-2204:52seancorfieldSpecs for data structures tend to live in their own namespace, no code. Specs for functions live in the same namespace as the function. Roughly.#2021-11-2204:53seancorfieldI wrote about our use of Spec here https://corfield.org/blog/2019/09/13/using-spec/ based on what we do at World Singles Networks.#2021-11-2209:42vemvHas it been hammocked (officially or not) to give the notion of coverage to specs? e.g. for an (s/or), verify at runtime, integrated with a given test suite, that all branches of the or have been exercised. (I know that generative testing is a thing! I want it the other way around here: start by code and reach the spec, not vice versa)#2021-11-2209:43vemvOne big use case, is that for any defn that returns s/or :result ,,, :error ,,,I should rest assured that the test suite has coverage for the green and red paths of the spec alike#2021-11-2213:29Alex Miller (Clojure team)Haven't talked about it#2021-11-2213:52Colin P. HillIs there a writeup anywhere about the reason that instrument doesn't validate :ret and :fn? I was wondering if this was considered and decided against, or there were some non-obvious difficulties. There's a library out there that does this, but I've got this nagging suspicion that it was left out of the first-party spec library for a reason.#2021-11-2214:00Alex Miller (Clojure team)https://clojure.org/guides/faq#instrument_ret#2021-11-2214:01Colin P. HillAh my bad, should have looked around for a FAQ, of course it's on there. Thank you!#2021-11-2422:11danieroux
(s/def ::percentage (s/double-in :min 0 :max 100 :NaN? false :infinite? false))
  (s/explain ::percentage (float 50))
#2021-11-2422:12danierouxGot tripped over by this just now. The data that came in was java.lang.Float. There is no s/float-in as far as I can see?#2021-11-2503:46Drew Verleedouble-in says it specs a 64-bit floating point number https://clojuredocs.org/clojure.spec.alpha/double-in what did s/expain return?#2021-11-2503:47Alex Miller (Clojure team)Clojure doesn't really support floats in general (other than for interop). you can of course make your own spec for floats, or cast into a double etc.#2021-11-2508:21danierouxThank you, I will avoid using floats in Clojure then.#2021-11-2503:49Drew Verleemy brain just tripped over the fact a 64 floating point number isn't a float. err, or i guess it is, but not the 64 bit variety. as an aside, are the other numbers just not buoyant?#2021-11-2920:00Oliver GeorgeHas anyone backported schema/select ideas from spec 2? I'd quite like them please.#2021-11-2920:10Alex Miller (Clojure team)I'm not aware of anyone that's done this, but as a warning the impl in spec 2 is pretty buggy and still a wip#2021-11-2920:25kenny(ish) ran out of time. Also buggy. https://github.com/ComputeSoftware/spec1-select#2021-11-2923:08Oliver GeorgeThank you both.#2021-11-3007:38Jakub HolĆ½ (HolyJak)Hello! How is it possible that a nillable spec (v1) fails with NPE? This works just fine and as expected:
366   (s/def ::org-ds (s/nilable #(satisfies? % jdbc.p/Connectable)))
367   (s/conform ::org-ds nil) ;=> nil
but when I check specs using s/instrument s s/fdef then it fails in a test with NPE:
Caused by: java.lang.NullPointerException: null
 at clojure.core$instance_QMARK___5404.invokeStatic (core.clj:144)
    clojure.core$find_protocol_impl.invokeStatic (core_deftype.clj:536)
    clojure.core$satisfies_QMARK_.invokeStatic (core_deftype.clj:569)
    clojure.core$satisfies_QMARK_.invoke (core_deftype.clj:569)
    jakub.specs$fn__224908.invokeStatic (specs.clj:367)
    jakub.specs/fn (specs.clj:366)
    clojure.spec.alpha$spec_impl$reify__2059.conform_STAR_ (alpha.clj:923)
    clojure.spec.alpha$nilable_impl$reify__2556.conform_STAR_ (alpha.clj:1839)
    clojure.spec.alpha$conform.invokeStatic (alpha.clj:164)
    ...
    clojure.spec.alpha$deriv$fn__2425.invoke (alpha.clj:1537)
...
    clojure.spec.alpha$regex_spec_impl$reify__2509.conform_STAR_ (alpha.clj:1703)
...
    clojure.spec.test.alpha$spec_checking_fn$conform_BANG___3024.invoke (alpha.clj:121)
Why??? šŸ™
#2021-11-3007:42flowthingHmm, doesn't satisfies? take the protocol as the first argument?#2021-11-3010:47Jakub HolĆ½ (HolyJak)Thank you! You were right, incorrect arg order t osatisfies? seems to have been the problem#2021-11-3022:07johanatanin general, how flexible should specs be on the inputs that they accept / won't crash on. for example, in clojure.spec.alpha the kvs->map spec will crash if you pass it any non-collection:
=> (s/explain-data ::s/kvs->map [0])
nil
=> (s/explain-data ::s/kvs->map 0)
#object[Error Error: 0 is not ISeqable]
#2021-11-3022:07johanatanusing something such as: (s/and coll? ...) would avoid this problem#2021-11-3022:07johanatan(for the definition of kvs->map)#2021-11-3023:25Alex Miller (Clojure team)Well that function is essentially an internal spec to support s/keys* so it's only used in a coll context#2021-12-0100:43johanatanyea, i had a few of those too but since these are registered "globally" they're fair game to my (spec-based) fuzzer. so, i'm making the ones in my project "defensive" against this (although it should never happen from production code)#2021-12-0100:52johanatani.e., this is where the inputs are coming from (with the one exceptional case i've found [which crashes] excluded):
(s/cat :spec (s/with-gen qualified-keyword? #(gen/elements
                                                 (clojure.set/difference
                                                  (set (keys (s/registry)))
                                                   #{::s/kvs->map})))
        :val any?)
#2021-12-0108:06wontheone1Hello! I made a function to return specs depending on arguments.
(defn- search-schema-spec-for-of-and-defaultValue [of defaultValue]
  (s/keys :req-un [:build-api.search-schema/key
                   of
                   :build-api.search-schema/valueType
                   :build-api.search-schema/cardinality]
          :opt-un [:build-api.search-schema/of
                   defaultValue
                   :build-api.search-schema/doc]))
Itā€™s called like the following,
(search-schema-spec-for-of-and-defaultValue
 :build-api.search-schema.listing/scope :build-api.search-schema.enum.one/defaultValue)
But I get the following error
Unexpected error (AssertionError) macroexpanding s/keys at (src/sharetribe/build_api/routes.clj:136:3).
Assert failed: all keys must be namespace-qualified keywords
(every? (fn* [p1__1917#] (c/and (keyword? p1__1917#) (namespace p1__1917#))) (concat req-keys req-un-specs opt opt-un))
I just wanna factor out common parts and make different specs based on of and defaultValue as they are the differentiator, any advice on how to achieve?
#2021-12-0109:23vlaaaduse macros or eval instead of defn ā€” s/keys is a macro#2021-12-0110:35wontheone1Thanks! @U47G49KHQ I defined common spec separately and combined with s/and and this seems to work good enough for me!
(def build-api-search-schema-common-spec
  (s/keys :req-un [:build-api.search-schema/key
                   :build-api.search-schema/valueType
                   :build-api.search-schema/cardinality]
          :opt-un [:build-api.search-schema/of
                   :build-api.search-schema/doc]))

(s/and build-api-search-schema-common-spec (s/keys :req-un [:build-api.search-schema.listing/scope]
                                                     :opt-un [:build-api.search-schema.enum.one/defaultValue]))

...etc...
#2021-12-0116:55colinkahnIf youā€™re combining map specs you might want to look at s/merge as well#2021-12-0117:19wontheone1Thanks#2021-12-0800:50Drew Verlee> For this we have exercise, which returns pairs of generated and conformed values for a spec What if we just want the generated values?#2021-12-0800:51Drew Verleei can fish them out, but i'm guessing there is a convention rather then threading with calls to first.#2021-12-0802:00Alex Miller (Clojure team)you can call gen/sample if you just want generated values https://clojure.github.io/spec.alpha/clojure.spec.gen.alpha-api.html#clojure.spec.gen.alpha/sample#2021-12-0802:01Alex Miller (Clojure team)sorry, https://clojure.github.io/test.check/clojure.test.check.generators.html#var-sample is the docstring for that#2021-12-0803:49Drew Verleethanks alex. ill read that shortly!#2021-12-0804:38Drew Verleethat likely does what i need. it would be nice if i had away to easily jump from one doc to another. that fns args are are just & args and the doc string is that its a lazy version of another fn. not much to go off! i guess i should just jump to the code?#2021-12-0804:41Drew Verleewell, im guessing that would be very educational
(lazy-combinators hash-map list map not-empty set vector vector-distinct fmap elements
  bind choose one-of such-that tuple sample return
  large-integer* double* frequency shuffle)
#2021-12-0804:41Drew Verleebut not very direct šŸ™‚#2021-12-0816:53borkdudeI'm working on better clojure.spec support for babashka. I'm testing expound with instrumentation. How do people generally get to see expound's output when running with instrumented functions? What do you have to configure, besides the explain printer? It seems you have to catch the ExceptionInfo and push the ex-data through s/explain-out yourself?#2021-12-0816:54borkdudeBesides that question: what are some libraries that leverage clojure.spec that I should test.#2021-12-0817:11delaguardoreitit with spec based coercions#2021-12-0817:12borkdudereitit doesn't work because it isn't pure Clojure, it has custom Java classes#2021-12-1222:58vemvhttps://github.com/nedap/speced.def perhaps, we've wanted to use it for bb scripting sometimes (@UHJH8MG6S)#2021-12-2221:53lucian303spec tools: https://cljdoc.org/d/metosin/spec-tools/0.10.5/doc/readme#2021-12-0823:03abrooksWe have a library that provides and conforms against specs for incoming complex values. Within the values can be maps and s/keys will, as the doc string says: > In addition, the values of all namespace-qualified keys will be validated (and possibly destructured) by any registered specs. If the caller puts their own namespaced keys in the map (which is fine) and they have defined specs for these, those specs will be validated (and conformed) when we validate or conform the larger structure. This creates problems in providing good error messages since, when we conform, we don't know if the caller has violated our spec or their own spec. There's no way to conform only our parts of the spec first to validate the shape of their input without also conforming and failing on their values. This leads to a very confusing error message that we can't help to resolve.#2021-12-0823:04abrooksIs there a sane work around for this? I'd really like an s/keys option or similar that doesn't validate keys beyond those expressed.#2021-12-0823:26Alex Miller (Clojure team)a) you could narrow the data before conforming b) hopefully you can differentiate by key namespace either for narrowing or for error gen?#2021-12-1714:30abrooks(a) doesn't work since we actually want the data preserved in the conformed result. I'll look at (b) but would love to mention that having local specs (similar to local Clojure hierarchies) is really what I think I want as a user. You can have the global spec (just like global-hierarchy) but also provide a local subset or curated spec for a particular context.#2021-12-1714:35Alex Miller (Clojure team)yeah, that's not in line with the spec rationale https://clojure.org/about/spec#_global_namespaced_names_are_more_important#2021-12-1718:38abrooksOh, I love namespaced names and the general global usage is fine but, as Stu and Rich have noted, a la carte features are valuable for application domain use cases not just programming domain use cases. I'd love to have spec for application domain use cases which is exactly what hierarchies allows for. Global for programming domain, a la carte for application domain. Often the specs overlap so you want to be able to re-use them between the two.#2021-12-1718:38abrooksmetosin/malli supports this, FWIW.#2021-12-1020:07Zach Mitchell, PhDI'm trying to represent a piece of data of the form [:some-keyword int int] where the two ints should not be equal. Is there a way to encode this relationship between the two ints in a spec? Should the spec just encode the shape of the data and leave validating this relationship to something else?#2021-12-1020:22Alex Miller (Clojure team)specs can be any predicate so the answer to all such questions is generally yes :)#2021-12-1020:23Alex Miller (Clojure team)one option is to combine a first structural spec with a second value checking one with s/and#2021-12-1020:23Alex Miller (Clojure team)s/and flows conformed values so that can provide structure for the second one#2021-12-1020:24Alex Miller (Clojure team)(s/and (s/cat :k keyword? :a int? :b int?) #(not= (:a %) (:b %))) something like that#2021-12-1020:40Zach Mitchell, PhDExcellent! Thank you#2021-12-1411:04Ben SlessWith multi specs, unlike with or I can't get back the matching case when calling conform. What should I do in case I want to take actions based on the spec the input matches?#2021-12-1412:06jkxyzYou can use whatever dispatch function you use for the multi-spec to also branch in other parts of the code. The obvious case is a :type key or something similar#2021-12-1415:33Ben SlessI was hoping I wouldn't need to retread this path because I considered it an implementation detail#2021-12-1416:39jkxyzYou could use conformer to tag each branch#2021-12-1417:01jkxyzIf it's a closed set of branches you can use or with a predicate that matches on the "type" -- which I suppose is probably the reason that multi-specs don't conform with a tag. If you're trying to parse a structure against an open set of branches you could end up with any value as the tag, so you'd need another multimethod to dispatch to code which handles the conformed values. If you don't expect consumers of the namespace to extend the spec I'd say just use or#2021-12-1417:05jkxyzIf it really is just an implementation detail then I'd say use conformer to conform to some useful value#2021-12-1518:08seancorfieldI haven't tried to repro this (and therefore haven't commented on it) but wondered if someone here could take a look? https://clojureverse.org/t/does-clojure-spec-supports-records/8428#2021-12-1518:46Alex Miller (Clojure team)Records will generally only work with :req-un / :opt-un - this is covered in the spec guide https://clojure.org/guides/spec#2021-12-1518:51Alex Miller (Clojure team)oh, this is https://clojure.atlassian.net/browse/CLJ-1975#2021-12-1613:04danieroux
(require '[clojure.spec.alpha :as s])
  
  (s/def ::not-in-map-spec pos-int?)
  (s/def ::in-map-spec string?) 
  (s/def ::the-map 
    (s/keys :req [::in-map-spec]))
  
  (def m 
    {::in-map-spec     "yes"
     ::not-in-map-spec -13})
  
  (s/valid? ::the-map m) ; => false
  (s/explain ::the-map m) ; => -13 - failed: pos-int? in: [:user/not-in-map-spec]
#2021-12-1613:05danierouxThis surprises me, that a key thatā€™s not in my spec, makes the spec invalid?#2021-12-1613:08Alex Miller (Clojure team)This is covered in the docstring for s/keys - all keys are checked#2021-12-1613:48danierouxHmm. Thatā€™s a conundrum in my usage, which I assume questions my usage: I have a big-map. I want to grab a part of it and save it away. Then use the rest to flow along. I canā€™t grab a part of it, because the rest is invalid. Do I explicitly select-keys to save it away?#2021-12-1810:58ikitommiYou could also use add-on libraries like spec-tools: https://cljdoc.org/d/metosin/spec-tools/0.10.5/doc/spec-driven-transformations#transforming-map-specs#2021-12-1614:05Alex Miller (Clojure team)Yes, you can do that#2021-12-1619:55pavlosmelissinos
(s/def :demo/m1 (s/keys :req-un [::a ::b ::c ::d ::e]))

(s/def :demo/m2 (s/keys :req-un [::a ::b ::c]))
Does spec provide a way to go from m1 to m2 programmatically? (I suppose I'm looking for the inverse of s/merge)
#2021-12-1620:10Alex Miller (Clojure team)spec is not a transformation engine, it's a validation/verification library#2021-12-1620:10Alex Miller (Clojure team)or are you asking about spec transformation#2021-12-1620:12Alex Miller (Clojure team)assuming so, not really, it really encourages building up by composition, not by taking away#2021-12-1620:18pavlosmelissinosI meant spec transformation, yes. I'm writing generative tests based on some s/keys specs and I'll have to dissoc some of the keys after the generation anyway, so I wondered if I could skip that step. Definitely not a deal-breaker and the justification behind the design decision is sound, as always. Thanks. šŸ™‚#2021-12-2020:31wilkerluciohello, is there a place that explains how to properly implement the s/Spec protocol? I'm not sure about some of the methods, also, is there a place that explains the :path, :via and :in parts of the spec problems? (I see in the spec docs it talks about but doesn't mention exact those keys)#2021-12-2020:43Alex Miller (Clojure team)No, we don't really consider that public api#2021-12-2020:44Alex Miller (Clojure team)But if you have a question I'll try to answer it :)#2021-12-2021:52wilkerluciomostly about the :path, :via and :in, what is expect on each and what makes it different from one another?#2021-12-2021:58Alex Miller (Clojure team)there is some info at https://clojure.org/guides/spec#_explain#2021-12-2022:03Alex Miller (Clojure team)although those are words in the error message that map to the keys: ā€¢ :path - vector of keyword tag "path" segments (usually from :or, :cat, :alt, etc) ā€¢ :via - vector of spec names from the root ā€¢ :in - vector of data values from root to failing value (will be omitted if it's the root value)#2021-12-2022:08Alex Miller (Clojure team)there's an implicit tree being recursively walked here. the nodes are specs (name = :via element). Each edge has a name (:path element) and the value to validate (:in element)#2021-12-2022:08Alex Miller (Clojure team)in some cases, some of those are synthetic, created by the spec#2021-12-2022:38wilkerluciothe examples I'm seeing are mostly related to s/keys, where I see things like this:
(s/def ::a string?)

(s/explain-data (s/keys :req [::a]) {::a 1})

{:clojure.spec.alpha/problems ({:path [:com.piposaude.model.bulk-insert-test/a],
                                :pred clojure.core/string?,
                                :val 1,
                                :via [:com.piposaude.model.bulk-insert-test/a],
                                :in [:com.piposaude.model.bulk-insert-test/a]}),
 :clojure.spec.alpha/spec #object[clojure.spec.alpha$map_spec_impl$reify__1998
                                  0xabd19ae
                                  "
#2021-12-2022:38wilkerlucioin this case all :path, :via and :in are the same value#2021-12-2022:38wilkerluciobut from what I got to you, is like: :in - path from root :path - local path :via - specs in the process is that a fair simplification of them?#2021-12-2022:39wilkerlucioand do you have examples that can clear up the distinction?#2021-12-2117:37Alex Miller (Clojure team)they are all "paths", but :in is data, :path is tags, and :via is specs#2021-12-2111:46timoHow do I activate specs in the repl? Running my tests with kaocha fail with spec-validation but in the repl these do not show up. :thinking_face:#2021-12-2111:54pavlosmelissinostake a look at instrumentation: https://clojure.org/guides/spec#_instrumentation_and_testing#2021-12-2112:05timothanks :thumbsup: didn't connect instrument to what I need#2021-12-2312:57thumbnailHi! We noticed clojure.spec sometimes generating #inst "NaN-NaN-NaNTNaN:NaN:NaN.NaN-00:00" (which seems invalid). Upon investigation we noticed the [impl dumps a large-integer into js/Date.](https://github.com/clojure/clojurescript/blob/a4673b880756531ac5690f7b4721ad76c0810327/src/main/cljs/cljs/spec/gen/alpha.cljs#L113-L114) which can yield the above result. Is this to be expected?#2021-12-2313:53Alex Miller (Clojure team)I'd say that's a bug, can you file a question at https://ask.clojure.org ?#2021-12-2313:53Alex Miller (Clojure team)It should really be filed under the test.check library, which is where the code is#2022-12-2613:09misha@alexmiller greetings! https://github.com/clojure/core.specs.alpha/commit/938c0a9725a284095baa2387dff1a29c3f1e26ac I wonder, why not just do?
(s/or ::local-name       ::local-name
        ::seq-binding-form ::seq-binding-form
        ::map-binding-form ::map-binding-form
Had to port some code leveraging core.specs from 1.10 to 1.9 due to these. I get the "alpha" status and all. But the question above is a general one. I'd assume dispatch keywords are as part of a spec as the spec name itself, since it is not wild to assume there would be a bunch of code dependent on conformed values, eventually. Why not use the same qualified keys where available (sans e.g. inline specs)? Is there any specific reason or unobvious consequence? Thanks!
#2022-12-2613:30Alex Miller (Clojure team)We wanted to make path names better as they show up in error messages#2022-12-2613:32mishagood point!#2022-12-2613:32Alex Miller (Clojure team)Generally we don't use qualified names for paths as they are contextual in the parent spec #2022-12-2613:37mishaI can see how it often applies to regex specs. For most or specs I found :: way more suitable. Sans the ugly unreadable path, ofc.#2022-12-2613:45Alex Miller (Clojure team)The path is the whole point #2022-12-2622:46bortexzI would like to spec a coll-of maps that is sorted on a specific keyword, is there a simple way to do this?#2022-12-2701:27colinkahnSomething like (s/and (s/coll-of map?) #(apply <= (map :some-kw %))) perhaps (tested using CLJS so not sure if Java can compare using <= on strings).#2022-12-2709:14bortexzI was trying to avoid the custom predicate, but I guess thereā€™s no other way, thanks!#2022-01-0111:44Ho0manHi everyone, an extremely weird thing is happening using spec.alpha ... Please check this simple snippet. Why does the spec/explain-data at the end try to validate the sample record against the dfntn-spec when I'm not even including it in the spec/keys passed to it ??! Am I doing something wrong or is this a bug ? : ā€¢ Clojure Spec Version : [org.clojure/spec.alpha "0.2.194"]
(ns hermes.lib.system.alaki
  (:require [clojure.spec.alpha :as spec]
            [clojure.string :as clj-str]
            [com.stuartsierra.component :as component]
            [hermes.lib.system.utils :as utils]
            [taoensso.timbre :as timbre]))

;;------------------------------------------------------------------;;
;;------------------------------------------------------------------;;

(defn multimethod-dispatch-fn
  [x]
  (let [system? (-> x :component/system? true?)]
    (case system?
      true  (-> :systemized-component)
      false (-> x :component/type))))

(defmulti dfntn-spec multimethod-dispatch-fn)
(defmulti create multimethod-dispatch-fn)

(spec/def :component/name simple-keyword?)

(spec/def :component/config
  (spec/multi-spec dfntn-spec
                   multimethod-dispatch-fn))

(spec/def :component/deps
  (spec/map-of simple-keyword? simple-keyword?))
(comment

  (def sample {:component/type    [:Schema :v-0-0-1]
               :component/system? false
               :component/name    :schema-1
               :component/config  {:serde  :nippy
                                   :topics [{:topic-name "~~"
                                             :partitions 1}]}
               :component/deps    {}})

  (spec/explain-data
    (spec/keys :req [:component/name
                     :component/deps
                     ])
    sample)

  )

;;------------------------------------------------------------------;;
;;------------------------------------------------------------------;;
#2022-01-0112:36Ho0manHi everyone, here is another instance that I just don't get what's going on :
(ns hermes.lib.system.alaki
  (:require [clojure.spec.alpha :as spec]
            [clojure.string :as clj-str]
            [com.stuartsierra.component :as component]
            [hermes.lib.system.utils :as utils]
            [taoensso.timbre :as timbre]))

;;------------------------------------------------------------------;;

(defn multimethod-dispatch-fn
  [x]
  (let [system? (-> x :component/system? true?)]
    (case system?
      true  (-> :systemized-component)
      false (-> x :component/type))))

(defmulti config-spec multimethod-dispatch-fn)
(defmulti create      multimethod-dispatch-fn)

;;------------------------------------------------------------------;;
;;------------------------------------------------------------------;;

(spec/def ::component-config
  (spec/multi-spec config-spec
                   multimethod-dispatch-fn))

;;------------------------------------------------------------------;;
;;------------------------------------------------------------------;;

(def sample
  {:component/type [:Schema :v-0-0-1]
   :serde          :nippy
   :topics         [{:topic-name "~~"
                     :partitions 1}]})

(defmethod config-spec [:Schema :v-0-0-1] [& _] any?)

(spec/valid? (config-spec sample) sample)
(spec/explain-data (config-spec sample) sample)

(spec/valid? ::component-config sample)
(spec/explain-data ::component-config sample)

;;------------------------------------------------------------------;;
;;------------------------------------------------------------------;;
While the last spec/valid? returns true the subsequent spec/explain-data returns :
#:clojure.spec.alpha{:problems [{:path [[:Schema :v-0-0-1]], :pred hermes.lib.system.alaki/config-spec, :val {:component/type [:Schema :v-0-0-1], :serde :nippy, :topics [{:topic-name "~~", :partitions 1}]}, :via [:hermes.lib.system.alaki/component-config], :in []}], :spec :hermes.lib.system.alaki/component-config, :value {:component/type [:Schema :v-0-0-1], :serde :nippy, :topics [{:topic-name "~~", :partitions 1}]}}
ā€¢ Spec Version : [org.clojure/spec.alpha "0.3.214"]
#2022-01-0113:52lassemaattaI think that the behavior you noticed in your first example is normal for s/keys, see the docstring: > In addition, the values of all namespace-qualified keys will be validated (and possibly destructured) by any registered specs.#2022-01-0310:54Ho0manYear you're right. Thanks a lot @U0178V2SLAY.#2022-01-0112:36Ho0manHi everyone, here is another instance that I just don't get what's going on :
(ns hermes.lib.system.alaki
  (:require [clojure.spec.alpha :as spec]
            [clojure.string :as clj-str]
            [com.stuartsierra.component :as component]
            [hermes.lib.system.utils :as utils]
            [taoensso.timbre :as timbre]))

;;------------------------------------------------------------------;;

(defn multimethod-dispatch-fn
  [x]
  (let [system? (-> x :component/system? true?)]
    (case system?
      true  (-> :systemized-component)
      false (-> x :component/type))))

(defmulti config-spec multimethod-dispatch-fn)
(defmulti create      multimethod-dispatch-fn)

;;------------------------------------------------------------------;;
;;------------------------------------------------------------------;;

(spec/def ::component-config
  (spec/multi-spec config-spec
                   multimethod-dispatch-fn))

;;------------------------------------------------------------------;;
;;------------------------------------------------------------------;;

(def sample
  {:component/type [:Schema :v-0-0-1]
   :serde          :nippy
   :topics         [{:topic-name "~~"
                     :partitions 1}]})

(defmethod config-spec [:Schema :v-0-0-1] [& _] any?)

(spec/valid? (config-spec sample) sample)
(spec/explain-data (config-spec sample) sample)

(spec/valid? ::component-config sample)
(spec/explain-data ::component-config sample)

;;------------------------------------------------------------------;;
;;------------------------------------------------------------------;;
While the last spec/valid? returns true the subsequent spec/explain-data returns :
#:clojure.spec.alpha{:problems [{:path [[:Schema :v-0-0-1]], :pred hermes.lib.system.alaki/config-spec, :val {:component/type [:Schema :v-0-0-1], :serde :nippy, :topics [{:topic-name "~~", :partitions 1}]}, :via [:hermes.lib.system.alaki/component-config], :in []}], :spec :hermes.lib.system.alaki/component-config, :value {:component/type [:Schema :v-0-0-1], :serde :nippy, :topics [{:topic-name "~~", :partitions 1}]}}
ā€¢ Spec Version : [org.clojure/spec.alpha "0.3.214"]
#2022-01-1020:15slipsetWe were discussing at work today, and this cropped up:
ardoq.core> (spec/def :foo/foo string?)
;; => :foo/foo
ardoq.core> (spec/def :foo/bar int?)
;; => :foo/bar
ardoq.core> (spec/def ::test (spec/keys :req [:foo/foo]))
;; => :ardoq.core/test
ardoq.core> (spec/valid? ::test {:foo/foo "foo" :foo/bar 1})
;; => true
ardoq.core> (spec/valid? ::test {:foo/foo "foo" :foo/bar "1"})
;; => false
In the last example, :foo/bar is not mentioned in ::test but seems to still be checked. What am I not seeing?
#2022-01-1020:19mpenets/keys will cause all namespace qualified keys to be checked#2022-01-1020:23slipsetRight, itā€™s there in the docstring: > In addition, the values of all namespace-qualified keys will be validated > (and possibly destructured) by any registered specs. TIL.#2022-01-1020:26slipsetWhich still is a bit strange since then:
ardoq.core> (s/def :foo/foo string?)
;; => :foo/foo
ardoq.core> (s/def :foo/bar int?)
;; => :foo/bar
ardoq.core> (s/def ::test (s/keys :req []))
;; => :ardoq.core/test
ardoq.core> (s/valid? ::test {:foo/foo "foo" :foo/bar "1"})
;; => false
ardoq.core> 
Which makes you wonder why even bother with :req ?
#2022-01-1020:28mpenetYou still need to be able to express what key set is required#2022-01-1020:28mpenetI guess for opt it's debatable. It's mostly for gen there#2022-01-1020:29slipsetTrue#2022-01-1020:30mpenetAlso if you have a ns qualified key that points to a value that does not conform to the spec with the same key, no matter where, it's likely a smell. Integrant pushes toward this for instance, I am not a fan of that part of the lib.#2022-01-1020:32Alex Miller (Clojure team)opt is used for gen#2022-01-1114:08mpenetthat might be of interest here#2022-01-1310:52Jakub HolĆ½ (HolyJak)Is there some place I can check for the status of spec2 and whether there is any tentative timeline? šŸ™#2022-01-1312:57Alex Miller (Clojure team)No tentative timeline#2022-01-1314:28vlaaadhow about status?#2022-01-1314:44Alex Miller (Clojure team)not sure what you're looking for#2022-01-1314:44Alex Miller (Clojure team)it exists in development state#2022-01-1315:05vlaaadMaybe Not was given in November 2018. Itā€™s been 3 years since the announcement of spec 2. In no way Iā€™m trying to rush Clojure development ā€” I see a lot of stuff done during this time, particularly around tools deps, but Iā€™m nonetheless curious if spec alpha 2 is delayed due to other areas having higher priority during these 3 years, or was there maybe some unexpected turn of events that resulted in spec design going into unforeseen direction? I remember reading something about function specsā€¦ Curious how itā€™s going there.#2022-01-1315:15Alex Miller (Clojure team)still thinking about how to express the relationships we want to express at the function spec level. the provide/consumes stuff in Maybe Not is part of it, and mostly realized in schema/select, but there are other aspects too like (for example) wanting to be able to (easily) say that the elements in the output of filter are a subset of the elements in the input. and how to integrate that with the params in function definition in a clean way. I think we'll be spending more time on spec in '22.#2022-01-1314:10Ronny LiHi everyone, in https://m.youtube.com/watch?v=YR5WdGrpoug from 2018, Rich talks about defining a function that only requires a subset of a previously defined schema using s/select. Is this available now? I'm asking because I looked through the spec docs but didn't see anything. I might've just missed it though.#2022-01-1314:38mpenetNope. There is a lib that attempted to bring that to spec1 but it's an experiment I think.#2022-01-1314:46Alex Miller (Clojure team)spec2 is a wip and has schema/select https://github.com/clojure/spec-alpha2 (docs: https://github.com/clojure/spec-alpha2/wiki/Schema-and-select - differs a bit from the talk)#2022-01-1817:53MatElGranHi all, cross-posting here from #beginners hoping to have some feedback to my (super clojure beginner) problem, hope itā€™s ok. https://clojurians.slack.com/archives/C053AK3F9/p1642508701489300#2022-01-1910:27jerger_at_ddaWhile sitting in a activity-pub workshop ... does anybody know a clojure based activity-pub implementation?#2022-01-1915:30Ben SlessMaybe this would be better as a general #clojure or #find-my-lib q?#2022-01-1916:37jerger_at_ddathe reason to ask here was that I would expect spec to be the foundation. But I will try your suggestions šŸ™‚#2022-01-2903:08Drew Verlee@U3LRXRAT0 Are we talking about this? https://github.com/BasixKOR/awesome-activitypub ? I can't see how spec will help with implementing a Protocal, spec helps validate clojure data.... Skimming this https://www.w3.org/TR/activitypub/ makes me think that all this protocal is just like a recommendation on how label your data so other actors can try to read it? Which other actors? Why would i want to send them messages?#2022-01-2909:38jerger_at_ddaYes, you found the intended activity pub :-) Activity pub's data-structures are described by linked data = LD (a global way to describe data structures // labels but also composition & validity). Spec is deep inspired by LD, so their concepts maps perfectly together. That's the reason why I expect spec will play a relevant part in a clojure adaption of activity pub. The value add of LD is to describe data in case of integration between systems. You can describe data in many other ways ... but I know no other way integrating more or better - so LDs value is that your not forced to invent an other (imperfect) way to express your integrated data structures. ActivityPub adds a way to consume and publish data in a asynchronous and only temporarily connected world. So your systems avail. is not bound strictly to the availability of the systems integrated by yours. That's not trivial to implement ...#2022-01-3123:51hiredmanI looked at implementing the ld spec described algorithms in clojure at one point. The description of the algorithms were extremely imperative when I looked at them, constantly mutating maps, and pretty much assumed you were writing javascript#2022-01-3123:52hiredmanhttps://json-ld.org/spec/ED/json-ld-api/20121225/#context-processing kind of thing#2022-02-0110:01pavlosmelissinosOk so here's a scenario I'm not sure how to approach: I've changed some specs and introduced a bug. Now my generative tests don't work because "Couldn't satisfy such-that predicate after X tries." I'm pretty sure I've identified the spec that contains the bug but I can't tell exactly why it's wrong. My first idea was to generate a config (even if it doesn't satisfy all the predicates) and then use explain-data on it and navigate :clojure.spec/problems. However, :clojure.spec/gen and :clojure.test.check.generators/sample refuse to even show me the generated data if it's invalid. Is there an alternative that would work in this case?#2022-02-0110:05lassemaattahave you tried generating values directly from that "possibly broken" spec instead of the whole config?#2022-02-0110:08lassemaattaI usually debug such problems by starting from the simplest case which generates successfully and then gradually add more restrictions until I find the spec/predicate, which fails to pass the data#2022-02-0110:26pavlosmelissinosSure but I was wondering if spec could assist with that. The culprit was a custom predicate that's part of the property tests. It's used to ensure that the generated data satisfies certain properties (e.g. make sure that a key only appears once in a vector of maps). However, in the process it introduced a regression (it used concat on a vector, which turned it into a list). After some digging I figured it out and fixed it; I was just wondering if there was a less manual way to do it, with spec, to save me some time in the future. As in: 1. generate the data 2. run explain-data 3. get back a map with s/problems 4. see something like ":v should be a vector, not a list"#2022-02-0114:21Alex Miller (Clojure team)There has been some work in test.check to provide better reporting and eventually I would like to help surface some of that through spec for cases like this#2022-02-0114:45pavlosmelissinosThat's good to know, thanks šŸ™‚ To be honest I'm a bit surprised there's no way to get the generated data if it doesn't satisfy all the predicates but I'm sure there's a reason it's like that. I mean I understand it's an all or nothing situation but on the other hand having the incomplete state would be useful for debugging.#2022-02-0114:52Alex Miller (Clojure team)there's good reason to verify generated values match the spec, but there is a general problem with not getting example failures if no matching generated value can be found#2022-02-0110:10karol.adamiecwith spec, is it good idiomatic usage/taste to say that for example a reduce over collection should satisfy a predicate? in real terms, lets imagine we have a spec of a Security. And we build a holding. So each security, that is included in holding has a weight. And naturally all weights must sum up to a 100. Is this too convoluted and going too far with spec? Goal is to use spec not only to validate inputs, but also generate artificial holdings, that make sense.#2022-02-0110:19vlaaadYou can have arbitrary predicates composed with s/and , itā€™s very idiomatic IMO#2022-02-0110:20vlaaade.g. your holding spec can be something like (s/and (s/coll-of ::security) holdings-add-up-to-100?) where holdings-add-up-to-100? is your predicate#2022-02-0110:20karol.adamiecnice. good to know... now off to write a generator šŸ˜‚#2022-02-0110:20vlaaadyeah, that šŸ˜„#2022-02-0110:20lassemaattathat's the fun part šŸ™‚#2022-02-0110:21vlaaadbut I would say itā€™s all idiomatic and good taste#2022-02-0110:21vlaaadespecially the generator#2022-02-0110:22karol.adamiecyeah, sounded like a good idea, but it is easy to get overexcited and go in directions one was not meant to travel in...#2022-02-0110:25vlaaadthat depends on where your specs are going to be used. if itā€™s for testing the code that relies on ā€œholdings add up to 100ā€ invariant, then it makes sense to invest in generator. If that code is not of super importance, you can have your generator to be a set of examples as a starting pointā€¦#2022-02-0110:26karol.adamiecyeah, ideally we would use this extensively in unit tests. First, to catch edge cases that real data does not expose easily. and second to get rid of huge and obnoxius fixtures. šŸ™‚#2022-02-0110:27karol.adamiecbut a good point, generator writing is optional.#2022-02-0712:17Benjamin
(defn foo [] 'fo)
(s/fdef foo :ret int?)
(stest/instrument `foo)
(foo)
I don't get how :ret in fdef works. Why doesn't it throw when I do this?
#2022-02-0712:19Lennart Buitret specs are not instrumented by stest/instrument#2022-02-0712:19Benjaminah#2022-02-0712:22Lennart Buityou can use stest/check to check the entire function spec, as per https://clojure.org/guides/spec#_testing#2022-02-0914:42BenjaminThere is no way in keys to say "use this spec" like:
(s/def ::foo (s/keys :req [(::bar string?)]) )
something like this right? What is the most concise alternative?
#2022-02-0914:52Alex Miller (Clojure team)you mean to specify an ad hoc spec for a key? no#2022-02-0914:53Alex Miller (Clojure team)
(s/def ::bar string?)
(s/def ::foo (s/keys :req [::bar]))
#2022-02-0914:53Alex Miller (Clojure team)spec 2 has some facility for specifying ad hoc specs for unqualified keys in a schema#2022-02-0914:55Benjamin
(s/def ::foo
  (fn [m]
    (and
     (s/valid? ::foo-default-keys (dissoc m ::special))
     (s/valid? ::foo-special-key (select-keys m  [::special])))))
if I do this my design is wrong I feel like
#2022-02-0915:00Alex Miller (Clojure team)why is ::special so special?#2022-02-0915:01Alex Miller (Clojure team)I find things that are hard to spec often point to problems in the code/data design#2022-02-0917:29AronWouldn't this statement be true in general? What I mean is, when something is problematic it's not enough to know it's problem in "code/ data design" because it's not clear how to unproblem it šŸ™‚#2022-02-1022:07Braden ShepherdsonI'm running into this with some fdef specs in my tests. apparently there's some other problem that spec is endeavouring to explain? https://clojure.atlassian.net/browse/CLJ-2481#icft=CLJ-2481#2022-02-1022:12Alex Miller (Clojure team)you can see in the comments this was closed as a dupe of https://clojure.atlassian.net/browse/CLJ-1975 which is still open#2022-02-1022:13Alex Miller (Clojure team)You can work around it by adding a :kind predicate to the s/coll-of#2022-02-1120:59QuestHow are you shimming Clojure.spec instrumentation in dev &amp; test, particularly with deps.edn? I was using lein :injections for this but find no equivalent mechanism in the deps.edn world. A test fixture solves test part, but I cannot find any version of this solution that doesn't require a (use-fixture ...) at the top of every test namespace. Does someone have an alternative solution? (Recommend an improved test runner?) Shimming in :dev seems to have two unclean approaches. 1) Make the editor inject on cider-connect [which varies for each editor] 2) Make the app itself support spec instrumentation as part of its official config-driven initialization -- & wait until point of invocation to (require) spec test.deps (as these aren't available in prod build)#2022-02-1121:02QuestClosest solution for dev I've seen is https://underwhelm.net/blog/2019/12/19/tools-deps-injections/ , which works but is fragile#2022-02-1121:35QuestFrom reading around, seems the solution of #2 is the "best approach," but it's a heavier pitch to my new team than I'd like https://github.com/jeaye/orchestra/pull/52#2022-02-1215:07vemvas much as I'm a Lein fan / advanced user I really dislike :injections - they don't map 1:1 to an actual Clojure future so often people end up devising non-clean solutions that other folks are certainly solving in an idiomatic way.#2022-02-1215:10vemvuse-fixtures is fine and has the advantage that it will work with basically every test runner. You can always roll a tiny linter ensuring that each ns has this declaration. Otherwise using a bespoke test runner seems acceptable, there are a few e.g. eftest , circleci.test , kaocha , Polylith's.#2022-02-1417:36QuestHmm, I raised the use-fixtures PR but it changes 70 files. & you can't mix :use & :each fixtures in test. Making the linter check for the fixture is a good idea. If the team balks I'll mention some of the other test runners; it might be a hard sell, but would be cleaner in the end. Thanks for the thoughts & advice!#2022-02-1615:40Colin P. HillIs there any way to spec a map that can mostly be described with map-of, but has a few special keys with their own types? E.g.
{::special-1 {:I-should-be :a-map}
 ::special-2 :I-should-be-a-keyword
 "standard-key-1" "standard-val-1"
 "standard-key-2" "standard-val-2"
 ; ...
 "standard-key-n" "standard-val-n"}
#2022-02-1615:42Colin P. Hill(This is a structure already in use that I'm speccing for documentation purposes. I wouldn't generally design something that functions both as a record and as a map of arbitrary well-typed values.)#2022-02-1615:43kennyDoes s/merge a s/map-of and s/keys help?#2022-02-1615:45Colin P. HillThat was my first thought, but I'm not seeing a way to write the map-of part. If I did (map-of string? string?), the special keys and values would fail validation.#2022-02-1615:46Alex Miller (Clojure team)yes, you can do this by spec'ing it as a collection of kv pairs and then describe the pairs. I have a blog post about the destructuring spec at https://cognitect.com/blog/2017/1/3/spec-destructuring that has this characteristic#2022-02-1615:48Alex Miller (Clojure team)the key is you want (s/every (s/or ...kv-tuple-shapes...) :into {})#2022-02-1615:56Colin P. HillHmm, I've been defaulting to using coll-of rather than every, but I'm seeing that it blows up when I try to do that with :into {}#2022-02-1616:07Alex Miller (Clojure team)blows up how?#2022-02-1616:34Colin P. Hill
(s/conform (s/coll-of (s/cat :k string? :v string?) :into {}) {"foo" "bar" "baz" "bork"})

Execution error (Error) at (<cljs repl>:1).
nth not supported on this type cljs.core/PersistentArrayMap
=> :repl/exception!
Could be a quirk of the cljs implementation
#2022-02-1616:39Alex Miller (Clojure team)no, that's why the example above uses s/every instead#2022-02-1616:39Colin P. HillOkay, figured that might be the case, it's just surprising to me#2022-02-1616:39Alex Miller (Clojure team)coll-of assumes the coll is nth-able (which maps aren't). we may actually have a ticket to consider that, don't remember#2022-02-1716:29SimonHello Very Senior Clojure Spec question here: Iā€™m new to Clojure-spec, but have been using Clojure for about a year now. During my thesis I implemented the compiler for Secure Guarded Commands by Flemming Nielson (my supervisor). https://link.springer.com/chapter/10.1007/978-3-030-41103-9_7 The language syntax allows the compiler to analyse the explicit, implicit, bypassing, correlation, sanitised, and indirect information flows. Such that to make sure at compile time that no information is leaked. Would it be valuable to add similar information flow analysis to Clojure Spec?#2022-02-1716:32Alex Miller (Clojure team)what problem would this solve?#2022-02-1716:35Alex Miller (Clojure team)I guess I'm wondering if you're looking to extend spec or use the information in specs to do this analysis independently?#2022-02-1717:14SimonAt a minimum something like this:#2022-02-1717:22SimonI think i have some people doing something similar. https://dl.acm.org/doi/pdf/10.1145/3468264.3473127#2022-02-1717:28Alex Miller (Clojure team)that image makes me think you are wanting something even more fundamental, like a change to the language#2022-02-1717:29Simonnot neccessarily, I think we could also use a syntax that is closer to that of spec.#2022-02-1717:30Simonitā€™s sort of an extension of a type checker.#2022-02-1717:31Simonand if clojure.spec checks types, then I think it would be possible to do?#2022-02-1717:32Alex Miller (Clojure team)well, spec is emphatically not a type checker :)#2022-02-1717:32Alex Miller (Clojure team)it's a predicate value system applied at runtime#2022-02-1717:34Alex Miller (Clojure team)some people have explored using it for static checks (https://github.com/arohner/spectrum and some work with https://github.com/clojure/core.typed)#2022-02-1717:35Alex Miller (Clojure team)but that's certainly not the conception or use of it in core#2022-02-1805:09Drew VerleeAlso, philosophically speaking, correctness isn't a property of types, but of ... <waves hands>...#2022-02-1813:50Colin P. HillWhat's the difference between a type system and a predicate value system? If a type is a set of values, and a set can be defined by a membership predicate, then it seems like these are the same thing implemented in different ways#2022-02-1918:05Ben SlessThe core predicates are, but can you represent in a type system a sorted collection? Represent constraints such as "the index keys set must be equal to the set of values associated with this key for all records"? Predicates are way looser, you'll have to work hard to formalize an entire type system around them#2022-02-1918:07Ben SlessIf Phil Wadler and his friends are right, type systems are logic systems. Find a logic system that is predicative and you've found the appropriate type system#2022-02-2114:05Colin P. HillI can represent in a type system a sorted collection ā€“ if the type system is implemented with predicates! That is to say, the distinction you're drawing seems to beg the question: if predicative systems count as type systems, then the answer to your question is trivially "yes".#2022-02-2114:06Colin P. HillStatic type checkers are always developing new capabilities. Dependent and refinement types are an area of active research, and Rust's linear types were theoretical until Rust implemented them. Before these implementations, it still would have been correct to describe those ideas abstractly and call them type systems.#2022-02-2114:07Colin P. HillSimilarly, coherent models which lack a total formalization, like runtime predicate checks, might also reasonably be called type systems, just as a statically typed language built without a theoretical model in mind already has a type system waiting to be described.#2022-02-2114:28Ben SlessI'm asking a different question - what is the logic system which corresponds to predicative types. There is some calculus associated with it. The realization type systems correspond to a logical calculus system (such as lambda calculus) takes it far from type checkers. There is a formal theory there waiting to be discovered#2022-02-2114:47Colin P. HillI don't really know what logic system would correspond to it, but I'm not sure why we would need an answer to that question. We've had plenty of type systems whose formal theories were only developed after the implementations, haven't we?#2022-02-2114:51Ben SlessI don't think so. These developments have sometimes been independent and later converged under the same name, see HM type system, one logician one computer scientists. The field of logic, being less constrained, usually develops these first, but it is not a strict requirement#2022-02-2114:52Ben SlessMaybe predicate types will be the first#2022-02-2114:56Ben SlessMaybe you can reduce them to another system#2022-02-2115:22Colin P. HillThe existence of this paper would suggest to me that Java's type system was first built, and only later analyzed formally: https://www.researchgate.net/publication/220299005_A_Formal_Type_System_for_Java#2022-02-2115:23Colin P. HillA skim suggests to me that, as of that paper's publication, the formal analysis work wasn't even yet complete (but I might be wrong about that)#2022-02-2115:31Ben SlessI'm drawing a distinction between a specific type system implementation, a theoretical type system, and a logic system. Could be that formal analysis was applied to Java's type system after it was written, but it must have corresponded to an existing logical system (or a yet undiscovered one) https://www.youtube.com/watch?v=IOiZatlZtGU#2022-02-2119:38bronsa> Rust's linear types were theoretical until Rust implemented them FYI there's an impl of linear types in this 1991 paper https://www.cs.utexas.edu/users/hunt/research/hash-cons/hash-cons-papers/BakerLinearLisp.pdf#2022-02-2206:03Ben SlessI think linear types relate to substructural logic https://en.m.wikipedia.org/wiki/Substructural_logic#2022-02-2217:00Colin P. Hill> FYI there's an impl of linear types in this 1991 paper I stand corrected! But I think my point is still valid, and I hope it's still clear#2022-02-2217:02Colin P. Hill> Could be that formal analysis was applied to Java's type system after it was written, but itĀ must haveĀ corresponded to an existing logical system (or a yet undiscovered one) That last parenthetical note makes all the difference, imo. It means that one does not need to point to a known equivalent logical system to be able to say that something is a type system.#2022-02-2217:26Ben SlessThe idea underlying this is that there isn't a sound type system that doesn't have a corresponding logical system. You either find the one which already maps on to it, or discover a new one šŸ™‚#2022-02-2219:49Colin P. HillWell hold on, adding sound as a qualifier changes everything šŸ˜† Plenty of real-life type systems have been unsound, and have in some cases even been useful despite that.#2022-02-2219:51Colin P. HillWhich is why I don't think the question of logic systems bears upon my original question. Something can fail to correspond to any known logical system, or even be found to be unsound when you try to formalize it, while still being a type system.#2022-02-1813:50Colin P. HillHeck, some of the core predicates are type checks, just at runtime#2022-02-1820:28respatializedThis conversation seems like it's going to open Pandora's box: http://lambda-the-ultimate.org/node/5604#2022-02-1820:29respatializedhttp://lambda-the-ultimate.org/node/412#2022-02-2318:43jjttjjLet's say you have to integrate with several providers by providing them an endpoint the receives json from them and must return json. The json structures each provider gives you and expect back are pretty different even though the essence of the operation is always sort of the same, though each platform has its quirks. I attempt to smooth over some of the differences, and build out my functionality on a common internally use data structure. In these situations, do you bother spec'ing the structures given/expected by each provider? Or the internal common structure only? Or both? Originally I thought I could get away with focusing on only my internal structure, but I'm finding the platforms are different enough in complicated ways that I really want specs for each one. My usage of spec here is to aid in writing tests (not necessarily generative ones)#2022-02-2320:13Colin P. HillSince they don't turn domain changes into compilation errors like static types sometimes do, afaict the worst case scenario is that you decide later that keeping them up to date isn't worth the effort and just delete them. No harm in just writing them now if they're useful now, imo.#2022-02-2320:16Colin P. HillPersonally I'm a big fan of specs as a way of documenting data structures, even if I never use them for anything else. The fact that I can instrument them for tests and REPL sessions is just gravy.#2022-02-2403:38jjttjjMakes sense, thanks!#2022-02-2407:28Ben SlessBoth. This lets you have a contract with them on one hand,a and verify your transformation to the common model on the other#2022-02-2819:24zendevil.ethSo here we have :req and :opt to be used in s/keys. That doesnā€™t stop other keys tht are not listed in opt or req from being accepted right? How do you limit the keys that can be accepted to only the req and opt keys?#2022-02-2819:24Braden Shepherdsonthat's deliberately not supported by spec. use select-keys or similar.#2022-02-2819:30zendevil.ethwhy this design decision?#2022-02-2819:39Braden Shepherdsonthe focus was on enforcing minimum needs.#2022-02-2819:40Braden Shepherdsonspec 2 does allow for this, because there are legitimate use cases for it. and the implicit checking of bonus keys with their own specs can be a DoS attack vector, if those other keys have slow specs.#2022-02-2819:40Braden Shepherdsonworse, since it's extra data getting validated, you can send perfectly valid requests that take tens of seconds or more to validate, but don't stand out in monitoring or logs as errors or unusually large...#2022-02-2819:41zendevil.ethwhatā€™s the solution to the dos attack when comprehensively validating the input#2022-02-2819:42Braden Shepherdsonhttps://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha#closed-spec-checking here's the spec 2 docs.#2022-02-2819:42zendevil.ethso we should use spec2 right?#2022-02-2819:43Braden Shepherdsonwell, perhaps not. it's still alpha. just mentioning it.#2022-02-2819:43zendevil.ethisnā€™t spec alpha too?#2022-02-2819:43Braden ShepherdsonI'd say the most straightforward thing is to just select-keys first, then validate with spec 1.#2022-02-2819:43Braden Shepherdsonsure, but it's been battle tested in a way the spec 2 hasn't.#2022-02-2819:46Alex Miller (Clojure team)spec 2 has not been released, and you shouldn't use it for real stuff#2022-02-2819:55jjttjjStating the obvious, but you can always use a custom predicate to limit the spec in addition to s/keys#2022-02-2822:09zendevil.ethLetā€™s say Iā€™m trying to conform a map to a spec. I want each key of the map to have values of a certain type. How can I write a spec for that?#2022-02-2822:10zendevil.ethI have this:
(s/def ::create-sessions-params (s/and (s/keys :opt-un [::students ::start_date ::start_time ::timezone ::instructor])))
#2022-02-2822:10zendevil.ethbut then for each key, I want to impose a type to the keyā€™s value too#2022-02-2822:10zendevil.ethhow will I do that?#2022-02-2822:12ghadi(s/def ::instructor string?) etc.#2022-02-2822:12ghadi(the s/and in your example doesn't do anything - remove it)#2022-02-2822:13ghadiI can read your question in several ways, might want to elaborate#2022-02-2822:21zendevil.eth@ghadi what would be the best way to restrict the keys to just the keys in the opt?#2022-02-2822:21ghadi?#2022-02-2822:21ghadirestrict what?#2022-02-2822:22zendevil.ethlike it shouldnā€™t conform if there are keys other than the ones in the :opt-un list#2022-02-2822:22zendevil.ethopt-un or req-un or opt or req#2022-02-2822:23ghadisame answer as in the thread above - transform lightly, then check#2022-02-2822:24ghadiselect-keys or similar#2022-02-2822:31zendevil.ethcan I use one of these libraries to achieve the effect rather than select keys: https://github.com/metosin/spec-tools https://github.com/wilkerlucio/spec-coerce#2022-02-2823:21zendevil.ethSo would this be good?:
(defn keys-conform? [m k]
 (every? (set k) (keys m)))
(def create-sessions-keys [::students ::start_date ::start_time ::timezone ::instructor])
(s/def ::create-sessions-params (s/and (s/keys :opt-un create-sessions-keys) #(keys-conform? % create-sessions-keys)))
#2022-02-2823:21zendevil.ethkeys-conform makes sure that the keys in the map are only the ones that are in the keys vector#2022-02-2823:28seancorfields/keys is a macro and cannot be passed a var like that, as I recall.#2022-02-2823:28Alex Miller (Clojure team)unfortunately, that won't work in s/keys#2022-02-2823:44zendevil.eththat should definitely be a feature in clojure spec#2022-02-2823:45zendevil.ethhow come we canā€™t use variables in it#2022-02-2823:47Alex Miller (Clojure team)spec forms are macros and at compile time, we wouldn't know your runtime key set. other choices have been made for spec 2. It is possible to retrieve the keys from the spec using specs on the spec forms themselves but it's pretty ugly to do so - most people end up just walking the form as data to do this#2022-02-2823:28Alex Miller (Clojure team)so you'd have to repeat the vector of keys there#2022-02-2823:29Alex Miller (Clojure team)btw, there is a faq entry about this here: https://clojure.org/guides/faq#exclusive_keys#2022-02-2823:30seancorfieldWhat we do is get the spec form (via the Spec API) and lift the keys out of it to use in other contexts -- we treat the Spec itself as the "system of record".#2022-02-2823:34zendevil.eth@seancorfield can you please elaborate?#2022-02-2823:51hiredman
user=> (s/def ::f (s/keys :req [:a/x]))
:user/f
user=> (s/form ::f)
(clojure.spec.alpha/keys :req [:a/x])
user=>
#2022-02-2823:51hiredmanyou use s/form to inspect specs#2022-03-0111:35zendevil.ethcan someone explain to me why Iā€™m seeing this error?
"res is " {:response {:status 400, :body "{\"spec\":\"(spec-tools.core/spec {:spec (clojure.spec.alpha/keys :req-un [:spec$34944/timezone :spec$34944/name]), :type :map, :leaf? false})\",\"problems\":[{\"path\":[],\"pred\":\"clojure.core/map?\",\"val\":null,\"via\":[],\"in\":[]}],\"type\":\"reitit.coercion/request-coercion\",\"coercion\":\"spec\",\"value\":null,\"in\":[\"request\",\"body-params\"]}", :headers {"Content-Type" "application/json; charset=utf-8", "X-XSS-Protection" "1; mode=block", "X-Frame-Options" "SAMEORIGIN", "X-Content-Type-Options" "nosniff"}}, :request {:protocol "HTTP/1.1", :remote-addr "127.0.0.1", :headers {"host" "localhost", "content-type" "application/x-www-form-urlencoded", "content-length" "39"}, :server-port 80, :content-length 39, :content-type "application/x-www-form-urlencoded", :uri "/api/schools", :server-name "localhost", :body #object[java.io.BufferedInputStream 0x79432e4b "java.io.BufferedInputStr
Iā€™m making the request like so:
(require '[peridot.core :refer [content-type request session])

(-> session
      (request "/api/schools"
               :params {:name name
                        :timezone (or timezone "America/New_York")}
                                    
               :request-method :post))
this is my reitit handler:
["/schools" {:swagger {:tags ["schools"]}}
   ["" {:post {:description "create a school"
               :parameters {:body {:timezone string?
                                   :name string?}}
               :handler create-school-handler}}]
...]
FYI I have these three middleware ([reitit.ring.coercion :as rrc]):
rrc/coerce-exceptions-middleware
rrc/coerce-request-middleware
rrc/coerce-response-middleware
#2022-03-0111:37zendevil.ethI donā€™t see the error when I remove the middleware#2022-03-0119:46seancorfield@ps Based on \"pred\":\"clojure.core/map?\",\"val\":null -- it's saying that it got null via JSON but expected a (non-nil) hash map, if I'm reading it correctly. Maybe ask in #reitit?#2022-03-0120:12zendevil.eth@seancorfield why would it get null via json? When I remove the coerce middleware, the params are right there in the handler#2022-03-0120:17seancorfieldNo idea. That's why I suggested asking in #reitit (which I do not use).#2022-03-0120:18seancorfield(since it seems to be related to the reitit middleware, more than Spec)#2022-03-0917:40colinkahnAre there any libraries to use with spec for preparing data to be run against a spec? For example, we want to evolve a spec with a required field to keep contracts strong, but knowing this wonā€™t be compatible with old pre-existing stored data in the system. Weā€™re considering declaring (via some library) a default for when that new key is missing, preparing the data w/ those defaults, so it will validate against the spec. I know this is possible via something like spec-tools coercers, but since coupling coercing w/ specs seems like itā€™s a known anti-pattern I assume using it for defaults in this way would be as well. This is kind of a round-about question about having strong specs for some places (think consumers vs producers), which I feel like is more of a solved thing in spec2 (use the select api to make certain things required in specific places), but without writing custom predicates I see no clear way to this (deeply) with the current version of spec.#2022-03-0917:58Colin P. HillI think the "right" solution per spec's design philosophy (at least at the time the first version was written ā€“ maybe the thinking has moved since then) is that, if the new version of your spec will reject values that were valid under the old version of your spec, then you should simply create a new spec and leave the old one behind for compatibility, possibly with the new key added as an optional value.#2022-03-0920:40colinkahn@U029J729MUP thanks, having a compatibility spec makes sense as we update the stricter versions.#2022-03-1108:32ikitommiare there or could there be plans to add support for inline function specs into clojure.core? Microsoft has suggested an https://devblogs.microsoft.com/typescript/a-proposal-for-type-syntax-in-javascript/, based on TS. I would really like to see something similar for clojure, could be spec or something pluggable that could be implemented with malli too. now there is the Plumatic syntax (schema+malli), ghostwheel (spec), guardrails (spec), aave (malli) and many others.#2022-03-1108:34ikitommijust for docs, I think the community would be eager to built actual tooling on top of that (ides, linters, instrumentation etc)#2022-03-1113:12Alex Miller (Clojure team)Possibly, although those specs may be a different form than what we have now#2022-03-1609:19ikitommiI was hoping for something more concrete, but thanks anyway šŸ™‚#2022-03-1521:30Mark WardleHello all. Using clojure.spec a lot now, and also using for generative testing. Working really well, but in one domain I need to generate 'unique' [obv only in that test run] integer identifiers. Unfortunately, they also need to match a specific pattern and have a Verhoeff check digit. My spec and a custom generator is: (defn gen-identifier "A generator of identifiers of the specified type. Parameters: - t : one of :info.snomed/Concept :info.snomed.Description or :info.snomed/Relationship." [t] (gen/fmap #(let [partition (rand-nth (seq (partitions-for-type t)))] (Long/parseLong (verhoeff/append (str % partition)))) (s/gen (s/int-in 100000 Long/MAX_VALUE)))) https://github.com/wardle/hermes/blob/46cfee5b8005ffe8e26d86dcbbb239ba7b8ef01a/src/com/eldrix/hermes/rf2spec.clj#L18 I can do this for some by simply filtering out duplicates if I'm generating a single batch, but there are other times when that is not possible. I guess I could fallback to an incrementing counter and forego using a generator, or map through generated entities and override with an autoincrementing integer generator from an atom, but is there something in spec I'm missing? I've seen the list-distinct-by but that appears to offer guarantees only within the list obviously. I was thinking about something that starts with something based on time but then increments to satisfy the other generative predicates. Then it starts to feel a bit tricky! Anyone else faced this issue, or have any suggestions? I am missing something obvious? All suggestions welcomed. Thank you#2022-03-1522:20colinkahnWhen using test.check Iā€™ve generated these using an atom as a counter in the past. Then itā€™s a matter of using gen/fmap where itā€™s generator is all the different things that should be related, and the transform function assigns the ids to be consistent. The generator looks like this:
(def counter (atom 0))

(def gen-unique
  (gen/no-shrink (gen/fmap (fn [_] (str (swap! counter inc))) (gen/return nil))))
Interested though if other people have different ways to do this. I donā€™t know if itā€™s a downside, but itā€™s extra coordination at the gen/fmap to ā€œreconcileā€ the parts (but Iā€™ve found this necessary at some level for almost all generators iā€™ve written).
#2022-03-1522:46Mark WardleOh that looks really good thank you. I shall give that a try. Hadn't used gen/no-shrink either before! Thanks again. #2022-03-1715:46jeffparkerBoth the sound and video quality of Rich's Spec talk at LispNYC in Nov '16 are pretty challenging for the viewer. Here's a https://vimeo.com/689129571 version to reduce the friction. šŸ€#2022-03-1715:57Alex Miller (Clojure team)Any chance you'd be willing to offer that up to the clojuretv youtube channel?#2022-03-1716:12jeffparkerSure, we'll be adding it to the https://www.youtube.com/channel/UCv33UlfX5S4PKxozGwUY_pA/videos youtube channel as well. What's the best way to get it to clojuretv?#2022-03-1717:18richhickeyVery cool - thanks!#2022-03-1717:18richhickeyVery cool - thanks!#2022-03-2804:31Oliver GeorgeHello is there a way I can get better errors for something like this (below). Essentially, I'm looking to get-in a deeply nested data structure to run a predicate test on something. Wondered if someone has written some kind of spec-in which I can use.
(s/explain
  (s/and (s/cat :s map?)
         (s/and (s/conformer #(get-in % [:s :db :route :route-params :uuid])) string?))
  [{:db {:route {:route-params {:uuid 1}}}}])

1 - failed: string?  
Ideally, the explain-data should include a path which would indicate where in the nested data I should look.
#2022-03-2806:07colinkahnI think to control exactly the message you want you'd need to define a deftype implementing the spec protocols (Specize and Spec). There isn't any documentation really for how to do that since those protocols are not meant for public use (afaik). If you're ok with the message being more verbose, predicate specs are kind of self describing. For example:
(s/def ::s (s/and (s/cat :s map?)
                  #(string? (get-in % [:s :db :route :route-params :uuid]))))

(s/explain ::s [{:db {:route {:route-params {:uuid 1}}}}])
Will give you:
{:s {:db {:route {:route-params {:uuid 1}}}}} - failed: (string? (get-in % [:s :db :route :route-params :uuid])) spec: :cljs.user/s
#2022-03-2804:34Oliver George(this sort of thing gives me an incremental strategy to moving away from s/assert by mapping them over to s/fdef tests)#2022-03-2810:21onetomIs there a way to automatically rename the unqualified keys to their fully qualified equivalent, eg. during conformance? For example:
(s/<something-like-conform>   (s/keys :req-un [:ns1/a :ns2/b])   {:a 1 :b 2})
  ;; => {:ns1/a 1, :ns2/b 2}
The idea would be to maximize the amount of code, which uses fully-qualified keys.
#2022-04-1908:28onetomI wonder if my question was unclear, or why hasn't it received any interaction?#2022-04-2704:47AronI would guess that the reason is simply people not having experience with this problem. Guessing again, I imagine that because the mapping from unqualified to qualified is always dependent on the context (could be that more than one place has unqualified that map to the same qualified, or could be that there are lots of qualified keywords but only a few places where there are unqualified versions of the, these two scenarios are indistinguishable just when looking at the keyword), and so such mappings are probably always written by hand, no one has tried to automate it. I would be very interested in the reality though, after all this guessing šŸ˜…#2022-04-2708:14Linus EricssonI did experience the problem. However, in my case, I found that I probably tried to overuse the conforms functionality in spec. However, there has been utility functions for update-keys https://github.com/clojure/clojure/blob/master/changes.md#33-update-keys-and-update-vals which could be used for qualifying keys. Also clojure.set/rename-keys could work`.`#2022-04-2704:47AronI would guess that the reason is simply people not having experience with this problem. Guessing again, I imagine that because the mapping from unqualified to qualified is always dependent on the context (could be that more than one place has unqualified that map to the same qualified, or could be that there are lots of qualified keywords but only a few places where there are unqualified versions of the, these two scenarios are indistinguishable just when looking at the keyword), and so such mappings are probably always written by hand, no one has tried to automate it. I would be very interested in the reality though, after all this guessing šŸ˜…#2022-03-3018:26bortexzSometimes I find the best place to put fn specs as metadata on de fn, like tests
(defn a 
 {:spec (s/fspec ...)}
 []
 ...)
is this something being considered for spec2? technically you can already do this, is there tooling around using specs on metadata?
#2022-03-3018:33Alex Miller (Clojure team)there are a lot of tradeoffs (var meta is evaluated in particular) and yes there are some things being considered here#2022-03-3022:34Oliver George@alexmiller have you peeked at the survey responses? Does it help justify prioritisation/investment in v2 spec?#2022-03-3022:46Alex Miller (Clojure team)I have, although I didn't really need the survey to tell me people are interested in spec :)#2022-03-3113:04Colin P. HillI imagine checking the survey to confirm that people want spec2 is rather like checking a thermometer to confirm that lava is, indeed, hot#2022-04-0703:27didibusI find I often have this scenario:
{:type :foo
 :bar 1
 :baz 2}
Where I want my spec to say that the keys are :type, :bar and :baz, but also that the value of :type must be :foo. What's the most straightforward way to write that spec? Bonus point where the generator for it properly constructs a map where :type is always :foo.
#2022-04-0704:08Alex Miller (Clojure team)have you looked at multi-spec?#2022-04-0704:10Alex Miller (Clojure team)seems like it would be a great match here#2022-04-0705:31didibusMulti-spec is actually where I have this problem, because the generator doesn't generate the correct :type#2022-04-0705:34didibusLike if we take the spec guide example: https://clojure.org/guides/spec#_multi_spec Assume that instead of: (s/def :event/type keyword?) we have (s/def :event/type #{:event/search :event/error}) If you try to generate an :event/event, they won't be valid to any of the type of events most of the time, because the generator doesn't know which of the :event/type to pick for the different multi-specs#2022-04-0707:45flowthingI'd like to write a spec for a clojure.data.xml data structure and use said spec for generation as well. For example, say I have something like this:
{:tag :foo
 :content [{:tag :bar
            :content ["..."]}]}
I'm not sure what the best way to do that would be, though. That is, how do I write (spec/def ::foo ,,,) that specifies that :tag needs to be :foo and :content must conform to the ::bar spec, etc. Is some combination of spec/keys and spec/and the way to go?
#2022-04-0708:47lassemaattaIt's been a while since I've used spec so this might be horribly wrong, but it might also solve both your problems šŸ˜ƒ šŸ§µ#2022-04-0708:47lassemaatta#2022-04-0708:48lassemaatta#2022-04-0708:51flowthingThank you! I'll give that a go. I tried flailing about with multi-spec a bit, but I didn't think of the merge into a base element spec, that's clever. :thumbsup:#2022-04-0708:51lassemaattaI tested this for at least two seconds, so caveat emptor šŸ™‚#2022-04-0708:51flowthingSure thing, no worries. šŸ™‚#2022-04-0709:19flowthingActually, it's probably easier to write a spec for the Hiccup syntax using tuples and/or regexp ops.#2022-04-0709:21flowthing
(spec/def ::messageId
  (spec/tuple #{:messageId} string?))

(spec/def ::from
  (spec/tuple #{:from} string?))

(spec/def ::soap/Header
  (spec/tuple #{::soap/Header}
    ::messageId
    ::from
    ;; etc
    ))

(spec/def ::soap/Envelope
  (spec/tuple #{::soap/Envelope}
    ::soap/Header))

(->
    (spec/gen ::soap/Envelope)
    (gen/generate)
    (xml/sexp-as-element)
    (xml/emit-str)
    (pretty-print-xml-string)
    (println))

;;=>
<a:Envelope xmlns:a="">
  <a:Header>
    <messageId>I67NC7</messageId>
    <from>733R0T1MGsY9cnx943</from>
  </a:Header>
</a:Envelope>
#2022-04-0715:00lassemaatta@U0K064KQV the example I gave above might offer one way to use generators with multi-specs. but as I said, I'm no expert in this stuff so there may be better ways to do it
#2022-04-1211:52Mark WardleRealise this will be mostly a matter of opinion, but where do you place your function specifications? I tried them in a dedicated specs namespace, but I'm preferring them just above the function definition. Are there any disadvantages to keeping the function spec and the function together in the same namespace?#2022-04-1212:12Colin P. HillI strongly prefer colocating them with the functions. Tends to make the namespacing more meaningful and communicative, and (as a fringe benefit) means you're jumping around less when you change things.#2022-04-1212:31Mark WardleThanks! That's helpful. #2022-04-1213:41Colin P. HillThe only disadvantage that comes to mind is that it might lead to one namespace owning a spec which is later revealed to most properly belong elsewhere, perhaps in a more general-purpose namespace ā€“ but this is just a specific case of the general unsolved problem of changing knowledge and needs. It can also be worked around by moving the spec to the "right" place, leaving behind an s/def in the old place that just aliases the spec under the old name, and documenting the latter as deprecated.#2022-04-1213:42Mark WardleThanks Colin. Makes sense!#2022-04-1215:01seancorfieldAnother "vote" for having function specs directly above the function they describe. We use separate .specs namespaces for data specs. I talk a bit about that in https://corfield.org/blog/2019/09/13/using-spec/#2022-04-1215:03Mark WardleThanks Sean. That's a helpful blog post thank you. All makes sense!#2022-04-1212:13Colin P. HillIf there are some specs used by a number of functions, I'll sometimes just put those at the top of the file rather than immediately before some particular function.#2022-04-1908:25onetomat https://youtu.be/dtGzfYvBn3w?t=6346 there is a slide, saying: > ā€¢ DO NOT use check, instrument or (turn on) assert in production > ā€¢ they are for making sure your program works - use in advance > ā€¢ but spec can play a runtime role > ā€¢ validating inputs from external systems or users The args conformance and exception throwing feature of instrument seems like a very useful guard to have even for production code, especially, when it's checking relationships between the input arguments. It would help us pinpoint code-paths in our code, which is not covered by tests or specs. Q1: what does "validating inputs from external systems or users" refer to? using function :pre [...] conditions? what spec functions should i use, if not s/assert? it seems like that the unconditional s/assert* was made to be used in :pre` conditions... Q2: is there a way to do something like instrument, but only for checking :args, so i wouldn't need to repeat such logic as function :pre conditions?#2022-04-1909:04Linus EricssonQ1: I would suggest using a construct like
(when-not (s/valid ::spec data) 
  (throw (ex-info "error in..." {:error-data error-data}))
because asserts (and pre-posts i think) can be disabled by various options to the JVM. Be explicit with your throws after validation. Q2: I would not recommend using instrument or pre when validating input from users or other systems. Rather make this validation a separate step in the processing of data. The way to handle incorrect data can differ between various systems and sources of data.
#2022-04-1911:54onetomi didn't know u can turn off :pre & :post conditions with *assert*, thanks!#2022-04-2005:42Arondo I have to throw if something is not valid?#2022-04-2006:12SantiagoYou can also use something like Failjure instead of throwing, but I think the general way is to make use of JVM exceptions#2022-04-2014:30Linus EricssonRe throw or not - it depends on how your application handles errors in inputdata. Usually this is very application specific. Usually it's not good to use exceptions for flow control.#2022-04-2101:59AronThere is no reason for validation or for tests to throw, not in cases that I expect.#2022-04-2003:20onetomi wanted to generate some datomic lookup-refs with spec, but s/cat always returns a seq, not a vector. is there some built-in / idiomatic / recommended way to spec vectors? i saw there are :kind and :into options for s/coll-of, but i couldn't combine it with s/cat; i always got Couldn't satisfy such-that predicate after 100 tries. error. i found a recommendation here: https://gist.github.com/mhuebert/8fdeedae57bf797778054dcf8f33ab8b but my gut feeling tells me, that there might be a simpler way.#2022-04-2003:26Alex Miller (Clojure team)unfortunately, there is not a simpler way - it is very challenging (in spec 1) to spec vectors and get all of conform, unform, gen etc to work. we've fixed this in spec 2 but I don't have a good answer in spec 1.#2022-04-2013:27KelvinWould s/tuple work?#2022-04-2013:46Alex Miller (Clojure team)for some limited cases, yes (it's fixed size and can only be positionally described, not with the regex ops)#2022-04-2020:50onetomI just started looking into spec2. Found this page, which specifically mentions how to solve my issue with non-flowing s/and-: https://github.com/clojure/spec-alpha2/wiki/Differences-from-spec.alpha (s/def ::vcat (s/and- (s/cat :i int?) vector?)) Thanks @U064X3EF3 for the great docs!#2022-04-2021:03Alex Miller (Clojure team)not mentioned there, but that's actually reified for you in s/catv (vector-only) and s/cats (sequence-only) in spec 2#2022-04-2021:04Alex Miller (Clojure team)I don't think that ever made it back to those docs, but they're there#2022-04-2021:05Alex Miller (Clojure team)and before you ask, no I would not recommend using spec 2 at this time, it has a number of bugs :)#2022-04-2022:51Cam SaulIs there a way to pass along/access/use the overrides from a top-level call to s/gen inside a custom generator defined with s/with-gen or (s/spec ... :gen ...)? Example. Supposed I want to generate some sort of query with the keys :database and :table. I'd like to have a custom generator for :table so its values are based on the value of :database... however if I use an overriden generator for :database That generator is invisible from inside my custom generator.
(s/def ::database string?)

(s/def ::table
  (s/with-gen
    string?
    (fn []
      (gen/bind
       (s/gen ::database)
       (fn [db]
         (gen/fmap (fn [table]
                     (str db \. table))
                   gen/string))))))

(s/def ::query
  (s/keys :req-un [::database ::table]))

(gen/sample (s/gen ::query {::database (fn [] (gen/return "my_database"))}))
;; =>
[{:database "my_database", :table "."}
 {:database "my_database", :table "7.$"}
 {:database "my_database", :table "T2."}
 {:database "my_database", :table "xv.HĀØ"}
 {:database "my_database", :table "aB.ƍn"}
 {:database "my_database", :table "Q3MCh.ƇƔ"}
 {:database "my_database", :table "F68.Ā·%:"}
 {:database "my_database", :table ".>Ƭ"}
 {:database "my_database", :table "jBk.ƇP ĀµrfL"}
 {:database "my_database", :table "d.Ā„Ā¶Ā«Ć¢\b"}]
Is there some way to do this? I know I could do the whole thing with test.check.generators/let or bind but I'd be missing out on all the niceness from spec maps since AFAIK there's nothing like s/keys in test.check
#2022-04-2023:20Cam SaulI guess I could get it working by redefining the entire spec with s/def instead of trying to pass an override for the generator, or maybe just with-redefs -ing clojure.spec.alpha/registry-ref itself... both feel icky but since this just for generative testing it's probably not the end of the world.#2022-04-2023:22Cam SaulAnother option might be to replace all my specs with functions that return specs based on previous state, but for map specs I guess I'd have to use Spec Alpha 2 so I could programmatically create them with schema or the like. Maybe that's a bit better. But creating a bunch of ad-hoc specs programatically actually seems like more work than if I forgot about spec altogether and just used test.check.generators directly#2022-04-2115:03colinkahnI don't think there's any way provided in spec to make your custom generator see the override (you could get creative and bind a dynamic var, but that's outside of specs functionality and probably a bit unexpected). test.check does have hash-map, which will generate a map of keys to values like (gen/hash-map :database db-gen :table table-gen). I would probably use fmap though here and do something like:
(gen/fmap (fn [[db-str table-str]] {:table (str db-str \. table-str) : database db-str})
           (gen/tuple (gen/return "my_database") gen/string))
Most things can be solved using fmap and then reconciling the generated parts in the fmap fn somehow. This gives better shrinking as well than using bind/let.
#2022-04-2121:20Cam SaulI tried binding dynamic vars at every possible place but couldn't figure out how to get it to work consistently. Thanks, I think I'll just stick to using test.check stuff without spec for now. I managed to put together a generator that works sort of like s/keys with :opt or :opt-un , not sure how well it's going to handle the shrinking. I'll have to wrap my head around that a bit more#2022-04-2614:55Dave RussellHello! Is there a good way to find all currently instrumented vars? It looks like (st/unstrument) will return anything that has been unstrumented, so that's a way to tell (sort-of), but for our use-case we'd like to: ā€¢ Preserve current instrumentation state ā€¢ Instrument all vars ā€¢ Do something ā€¢ Restore previous instrumentation state Which allows us to enable instrumentation when, e.g., running tests but keeping any instrumentation state in the REPL#2022-04-2615:08Colin P. HillLooks like no, unless you want to access a private var and risk breakage later. The instrumented vars are kept in a global atom. https://github.com/clojure/spec.alpha/blob/f23ea614b3cb658cff0044a027cacdd76831edcf/src/main/clojure/clojure/spec/test/alpha.clj#L150#2022-04-2615:09Alex Miller (Clojure team)isn't there a function for this in stest?#2022-04-2615:09Colin P. HillDoesn't appear to be. There's one that does the opposite though, turning off instrumentation within a dynamic scope.#2022-04-2615:10Alex Miller (Clojure team)There's instrumentable-syms and checkable-syms but I guess not instrumented-syms#2022-04-2615:11Dave RussellYep šŸ™‚ instrumented-syms would have been perfect#2022-04-2615:11Dave RussellRight now we basically have:
(let [currently-instrumented-vars (st/unstrument)]
   (set/difference (set (st/instrument)) (set currently-instrumented-vars)))
As a way to figure out what syms we should restore
#2022-04-2615:12Colin P. HillA with-instrument-enabled macro parallel to the with-instrument-disabled would have been pretty slick, to let this be done in a call scope rather than with global state changes.#2022-04-2615:30Alex Miller (Clojure team)feel free to file requests at https://ask.clojure.org ...#2022-04-2615:39Dave RussellWill do -- thanks!#2022-04-2615:48Dave Russellhttps://ask.clojure.org/index.php/11816/support-checking-instrumented-syms-clojure-spec-test-alpha#2022-05-0512:30pavlosmelissinosI have some kind of user configuration in the form of edn files. They're quite complex and rigorously spec'd, so I want to help the users identify potential problems (e.g. typos, settings that are no longer used etc), to help them manage this complexity. Assuming that it's not feasible at the moment to simplify the maps and given that spec doesn't support closed maps, what's the best way to do the above? There's a bunch of multi-specs in the specs that seem to break libraries like spell-spec and spec-tools. Example: given (s/def ::m (s/keys :req [:foo] :opt [::bar])) I want to get some warning for (def m {:foo 1 :baar 1}) Reasoning: In my use case it's more likely that baar is a typo, than a correct key that is used somewhere else. I have no way of knowing without asking the user, so how can I do that? I understand that closed maps are an antipattern in Clojure but the use case is real unfortunately. Happy to consider alternatives in any case! Thanks!#2022-05-0512:34Alex Miller (Clojure team)https://clojure.org/guides/faq#exclusive_keys#2022-05-0512:38pavlosmelissinosOooh, didn't think of that, thanks! šŸ™‚#2022-05-0604:13West
(s/def ::params (s/or :string string?
                      :map map?))

(defn is-null? [val]
  (or (empty? val)
      (= "null" val)
      (nil? val)))

(s/def ::non-null-params
  (s/and ::params
         #(not (is-null? %))))

(gen/sample (s/gen ::non-null-params))
Why does ::non-null-params still give me empty maps and empty strings?
#2022-05-0604:23seancorfieldBecause s/or produces a pair. Either [:string s] or [:map m]#2022-05-0604:23seancorfieldis-null? could have [[_ val]] as its arglist and it will work.#2022-05-0604:24seancorfieldOr you could test (is-null? (second %))#2022-05-0604:25WestAh ok. Then clearly I didn't want to use s/or I wanted to choose one or the other.#2022-05-0604:25seancorfield(technically it's a MapEntry so (val %) is "more correct" than (second %)#2022-05-0604:25seancorfields/or is how you do it, but it returns a tagged pair to tell you which branch matched.#2022-05-0604:30West
(s/def ::non-null-params
  (s/and ::params
         #(not (is-null? (val %)))))
Thanks Sean. It's working just as expected!
#2022-05-0604:31seancorfieldI think Alex has said that Spec 2 will have a non-conforming or that doesn't do that, or at least some easy way to avoid it...#2022-05-0604:33West"Will have", as in "when it's released", or "is being talked about". Can I test it now?#2022-05-0604:35seancorfieldIt's only available via git deps and it's kind of buggy.#2022-05-0604:36WestAlright, maybe an adventure for another day.#2022-05-0604:36seancorfieldWhat you can do with Spec 1 is used the undocumented nonconforming function, which is currently documented for Spec 2 (but might go away -- it will still be in Spec 1 tho'):
(s/def ::params (s/nonconforming (s/or :string string? :map map?)))
#2022-05-0604:37seancorfieldSpec 1 isn't going away. Spec 2 will most likely morph into the actual "core" Spec at some point and folks will either migrate from Spec 1 to "Spec" or use both side-by-side.#2022-05-0604:39seancorfieldFor a while at work, we maintained a branch of our code base migrated to use Spec 2, and help Alex test stuff, but it was such a moving target and several places were buggy and/or incomplete so after several months we abandoned trying to keep our branch updated and decided to wait for Spec 2 to mature.#2022-05-0604:40seancorfieldSpec 2 has a lot of improvements over Spec 1 but it is also quite different in several places.#2022-05-0604:41WestEither way, I'm excited to see how it turns out.#2022-05-0817:39Benjaminhttps://developers.google.com/discovery/v1/reference/apis wouldn't it be cool to generate spec from this?#2022-05-0817:55kennyYou definitely could. Cognitectā€™s aws-api does just that from the AWS api structured docs (smithy files). I'm not aware of anyone doing it for GCP but no reason you couldn't. Since the AWS api -> spec part of aws-api isn't open source and I needed part of that code, I wrote a version available here https://github.com/kennyjwilli/smithy-spec. Could be a starting point for GCP. #2022-05-0817:56BenjaminI'll think about it#2022-05-1610:11pithyless@U02CV2P4J6S I remember seeming something like this on Slack - perhaps it was this? https://github.com/ComputeSoftware/gcp-api#2022-05-1610:12pithylessOh, I just realized this is by @U083D6HK9 facepalm#2022-05-1610:14pithyless<meme of Homer Simpson slowly backing away>#2022-05-0923:18nodenameWhen I pull the problems from an exception that I receive from spec when running a test, sometimes I see :pred nil? and sometimes I see
:pred #object[clojure.core$nil_QMARK_
                0x5c45bb7f
                "
This is making it hard to write the test properly. Any thoughts? Or is there a better way to check that a certain test results in a certain set of problems?
#2022-05-0923:46Alex Miller (Clojure team)can you share any more info?#2022-05-0923:47Alex Miller (Clojure team)the intention is to return nil? there#2022-05-1000:29nodenameOh I think I see my problem, itā€™s not in the return value but in my is predicate. The returned info did say nil?. It works if I say
(is (= problems
...
:pred 'nil?
...))
with a quote before the nil?in my own data structure.
#2022-05-1107:00Mark WardleDear all, is the spec registry cleared between tests? I was just bitten by a surprising 'spec not found' exception during instrumentation in a project that uses one of my other projects as a library. My tests run successfully from the command-line, but when run in a brand-new, just loaded REPL, they fail because a specification was not found. It's an easy fix, as I haven't included a require, but surprised my tests didn't catch it. Finding it quite easy to accidentally omit the necessary requires as one flits back and forth between namespaces so the specs get loaded and then everything works. It's only in the context of trying to use as a library that I got an exception.#2022-05-1107:08seancorfieldThis seems no different to working with multimethods or protocols where you need to require the namespaces for the various implementations ahead of use?#2022-05-1107:55Mark WardleI guess so. But in that situation, my tests would fail. Here they worked, perhaps because the specs were loaded from one namespace, persist into a test on another namespace. At least, I presume that is what is happening.#2022-05-1108:30seancorfieldIf you're running multiple tests in the same process, state will persist, just as it would with multimethods or protocols.#2022-05-1109:45Mark WardleThanks. I suppose trying to reset the global registry prior to running a test namespace might work, but sounds more complicated. I guess it an issue because I have specifications not linked to a clojure namespace reflecting my domain - so one doesn't notice easily if you don't have a require in a specific namespace.#2022-05-1109:47Mark WardleIs there an argument in favour of a test runner resetting global state before running a namespace?That would, I think, fix the issue.#2022-05-1109:53Mark WardleI'm finding it difficult that running all tests fails to throw an exception, but specifying a single namespace does throw an exception: I understand why this happens, but it is confusing, at least to me!#2022-05-1116:05seancorfieldI don't think that a test runner could realistically "reset (all) global state)" -- partly because the test runner itself maintains global state about what tests it ran and what failures and successes (and some of that state is baked into clojure.test directly) unless you arrange to run the tests for each namespace in a separate process, which will add a huge amount of overhead to running your test suite.#2022-05-1116:07seancorfieldAt work, we run tests for each artifact's project in separate processes (via the Polylith test runner). Well, isolated classloader instances. We also have our build.clj perform each test run in a separate process (a separate JVM process) so we can test modules in isolation in our monorepo.#2022-05-1116:09seancorfieldThe Spec registry contains all the clojure.core.specs to, BTW.#2022-05-1118:10Mark WardleThanks Sean. I didn't know that. So trying to reset the internal atom might not be such a good idea. Just shows the issues with global state :)#2022-05-1911:30Ho0manHi everyone, I have a fully specd and complex data structure x-pack where I want to override the ::iid generator for all its sub-entities so they can all share the same value (they all use ::i-commons/iid ), However as the supersession rule in spec/gen explains I am not able to override ::i-commons . Any suggestions for a workaround? Or am I getting something very wrong?
(spec/def ::___x-pack___
  (spec/keys
    :req-un [::i-commons/iid
             ,,,]))

(spec/def ::x-pack
  (spec/spec ::___x-pack___
             :gen (fn []
                    (->> (spec/gen ::___x-pack___)
                         (gen/fmap map->X-Pack)))))
,,,,

(gen/generate (spec/gen ::x-pack {::i-commons/iid #(gen/return "ID1")}) 
Thanks
#2022-05-2809:31BenjaminHi, was there not a way to add a spec to the docstring of a function?>#2022-05-2811:42jumarI'm not sure about that but some tools like Cider show it when you try to show a docstring for the function.#2022-05-3010:02teodorluYou could change the docs yourself by changing the var metadata: https://clojuredocs.org/clojure.core/alter-meta%21#2022-05-3112:50folconI've been reminding myself how to use spec and one thing I can't remember is how to get better errors from this sort of thing?
(defn load-order [order-book {:keys [side id] :as order}]
  {:pre [(s/valid? :order/order order)]}
  (assoc-in order-book [side id] order))
Which does give me this:
Execution error (AssertionError) at fruit-economy.sim.market/load-order (market.clj:23).
Assert failed: (is (s/valid? :order/order order))
But considering it's a spec, I was expecting a little more? I've hacked it a bit by wrapping it in an is, which gives me:
FAIL in () (market.clj:2)
expected: (s/valid? :order/order order)
  actual: (not
           (s/valid?
            :order/order
            {:price 1,
             :size 425.0,
             :side :sell,
             :id 10229,
             :good-kw :inventory}))
But I'm sure there's a better way? EDIT: @jcf suggested using fdef to specify args and instrument, so I've swapped to doing that and calling instrument on debug as so in my core ns:
(when (debug?)
  (let [instrument (requiring-resolve 'clojure.spec.test.alpha/instrument)]
    (instrument 'fruit-economy.sim.market/load-order)))
To be honest this is pretty great, but perhaps there's something I'm missing? So I thought I'd pop the question here! REF: https://github.com/Folcon/fruit-economy/commit/a92b4b5329eabd7347240804f04224b6440ebdb2
#2022-05-3117:28jcfYou rang? šŸ™ˆ#2022-05-3117:28folconOh not ringing šŸ˜ƒ, attributing šŸ˜‰#2022-05-3117:29jcfThanks for the shout out, @U0JUM502E! šŸ™‡#2022-06-0507:59Bart KleijngeldI have a map of which I'd like to specify two optional keys, but they must not exist both. I'm trying to express this in spec but I'm having a hard time. You can't use and and or in the :opt part of the s/keys spec either, so that makes it particularly hard. Can someone help me out?#2022-06-0512:41Bart KleijngeldI've come up with a solution, but would love to hear from more experienced users.
(defn only-one-key-of [kws]
  (fn [m]
    (not
     (set/subset? kws
                  (into #{} (keys m))))))

(s/def ::some-map
  (s/and
   (s/keys :req [:a]
           :opt [:b :c :d])
   (only-one-key-of [:b :c])))

(comment
  (s/valid? ::some-map {:a 1 :b 2 :c 3})  ;; correctly gives `false`
)
#2022-06-0513:14Alex Miller (Clojure team)I think that's the right approach#2022-06-0513:17Bart KleijngeldNice. Thanks for your feedback#2022-06-0813:26roltsmall warning: only-one-key-of might be a bit misleading, it won't give the expected result if there are more than 2 keywords#2022-06-0813:27Bart KleijngeldYes, it was not very well-defined, it also didn't for just one parameter. I've redone the implementation to be more robust now. Thanks for pointing it out though šŸ™‚#2022-06-1418:10Drew VerleeI don't understand what spec is trying to tell me here. Isn't {:keys ... valid clojure?
-- Spec failed --------------------

  (... ... [{:keys [http-client max-retries backoff retry?], :as client} ...] ...)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

should satisfy

  vector?

-- Relevant specs -------

:clojure.core.specs.alpha/arg-list:
.... (more spec info...)
#2022-06-1418:14Victor Maia AldecĆ“aI don't understand either, but you can try to use this lib to get better spec messages: https://github.com/bhb/expound#2022-06-1418:15Drew Verleethat is expound formatted šŸ™‚#2022-06-1418:20Alex Miller (Clojure team)Do you have the message without expound?#2022-06-1418:32Christian JohansenThrowing in a guess: maybe you forgot to put optional arguments in a vector? (defn blabla [a b & {:keys [c d]}] ,,,)#2022-06-1420:36Drew Verleei could go back and re-create the errors it if it helps. The minimal change that made the errors go away was to
(defn foo [{:keys [x] :or {::x "x"}] x) 
to
(defn foo [{:keys [x] :or {x "x"}] x) 
note the :or {::x to to just x. that is, the value of the hashmap of the :or arg has to be a symbol not a key. This same error seemed to generate several different expound messages (at least thats how it seems to me).
#2022-06-1420:51Alex Miller (Clojure team)the keys of :or are always the symbols being bound, which must be unqualified, so neither :x or ::x are valid in that spot#2022-06-2416:18joost-diepenmaatHope Iā€™m not spamming people, but I would love to get some feedback on a tool we made to help with creating spec generators: https://git.sr.ht/~jomco/proof-specs proof-specs can be run as a leiningen alias or from the repl and will exercise all data specs (keyword specs) in a set of namespaces and report on any errors. Weā€™re using it ourselves as part of our test suite to ensure we always write generating specs and to make it easier to find out which part of complex specs are broken. Thanks for any feedback :-)#2022-06-2416:24joost-diepenmaatMain reason we wrote the thing is that sometimes spec generators just error out with very little context making it hard to figure out which specs even work. proof-specs will at very least give you a list of every spec that wonā€™t generate which helps a lot in figuring out whatā€™s going on.#2022-06-2422:09raymcdermottThis might be a silly question but is there a s-exp spec that can be applied recursively?#2022-06-2422:20Alex Miller (Clojure team)No, and I think you can quickly come up with many reasons why#2022-06-2423:18raymcdermottSure. Like I said, was a silly question. I just wanted to make sure I hadn't missed anything before moving on other options.#2022-06-2709:36reefersleepBut I guess youā€™re free to make a function that does recursive stuff and use it in a spec? šŸ™‚#2022-06-2709:39raymcdermottyes, also specs can be applied recursively even if it's silly šŸ‘ŗ#2022-06-2711:38borkdude@U04V5V0V4 What are you looking for? Something like tree-seq + spec? This is what grasp basically does: https://github.com/borkdude/grasp#2022-06-2712:10raymcdermottok - thanks, I remember talking about it with you but I've never looked at the code šŸ‘€#2022-06-2712:13raymcdermottto be more straightforward: I'm looking to conform all of Clojure. That's what I'm really looking to do.#2022-06-2718:59raymcdermott@U04V15CAJ one small thing I noticed in your reify spec is that you use (s/cat :reify #{'reify} ... why do you use a set rather than just the symbol 'reify? Cos that's what I do and it works but maybe set gets you something else that I'm missing?#2022-06-2719:01borkdudeBecause of how spec works?
user=> (require '[clojure.spec.alpha :as s])
nil
user=> (s/valid? 'foo 'foo)
Execution error at user/eval154 (REPL:1).
Unable to resolve spec: foo
user=> (s/valid? (s/cat :foo 'foo) ['foo])
Execution error at user/eval156 (REPL:1).
Unable to resolve spec: foo
user=> (s/valid? #{'foo} 'foo)
true
user=> (s/valid? (s/cat :foo #{'foo}) ['foo])
true
#2022-06-2719:03borkdudeI.e. I want to match the literal symbol 'reify#2022-06-2719:20raymcdermottyes ... actually it's neater than #(= 'reify %)which is the ugly thing I have šŸ™‡:skin-tone-3:#2022-06-2719:51reefersleepI use sets for comparison all the time. it's great!#2022-06-2719:52raymcdermottyeah, seems a ton more elegant and more transparent than a function#2022-06-2720:14Alex Miller (Clojure team)oh but it is a function ;)#2022-06-2720:25raymcdermottho ho#2022-06-2807:38reefersleepSometimes it can make sense to make it a var as well. Like
(def valid-foo? #{:x :y :z})
(valid-foo? :x) ;=> :x
#2022-06-2422:11raymcdermottEg (s/cat symbol? s/+ any?) but recurring #2022-06-2717:01souenzzoYou can give it a name, then recur to this name:
(s/def ::foo
  (s/cat :k keyword?
    :v map?
    :others (s/? ::foo)))
=> :user/foo
(s/valid? ::foo
  [:a {} :b {} :c {}])
=> true
(s/valid? ::foo
  [:a {} :b {} :c])
=> false
(s/valid? ::foo
  [:a {} :b {} :c []])
=> false
#2022-06-2718:55raymcdermottha, yes - actually that's what we ended up doing! šŸ™‚#2022-06-2718:56raymcdermottthank you @U2J4FRT2T#2022-06-2422:12raymcdermott[on the phone so syntax ain't great]#2022-06-2821:26Braden Shepherdson(I'm not sure if there's a better place for specmonstah questions; please direct me elsewhere if there is.) I'm using the (superb) specmonstah library to generate a big chunk of test data. One can use :constraints {:some_field #{:uniq}} to ensure a particular field is unique. however, sometimes the field is only unique within some more limited namespace. is there any way to write a custom constraint? I want a field that's unique among all those that share the same parent (another field).#2022-07-0407:21Martynas MHey. Is there a good way to spec an atom that has a list of functions? šŸ˜„ Is there at least a way to spec an atom?#2022-07-0407:23vlaaadAny predicate can be used as a spec#2022-07-0407:26Martynas MAh, I see. Thanks.#2022-07-0407:28vlaaade.g.
(s/def ::fns (s/coll-of ifn?))
(s/def ::my-atom 
  (s/and (s/conformer 
           #(if (instance? clojure.lang.Atom %) 
              @% 
              ::s/invalid))
         ::fns))

(s/valid? ::my-atom (atom [dec identity inc])) ; => true
#2022-07-0407:29vlaaadmaybe this use of conformer is not idiomatic. tbh Iā€™m not sure whatā€™s an idiomatic use of conformer#2022-07-0407:31vlaaadI think this is not idiomatic, because spec is a ā€œlibrary [that] specifies the structure of dataā€ (as per https://clojure.org/guides/spec), and atoms are state, not data#2022-07-0407:32vlaaadso maybe you could consider not speccing atoms? šŸ™‚#2022-07-0407:33Martynas MI think I'll not do it.#2022-07-0513:26Colin P. HillAtoms are stateful containers of data. It often makes a lot of sense to spec atoms, especially if they're a highly structured central data store for your application ā€“ similar to having a formal schema for a database. The atom function even has an option for including a validator, which could be built on spec (though be mindful of performance and consider making it no-op in prod). https://clojuredocs.org/clojure.core/atom#example-567add80e4b01f598e267e8f#2022-07-0513:28Colin P. HillA naive (in the sense that it doesn't check function signatures) spec for a list of functions might be (s/* ifn?)#2022-07-0414:18Eric DvorsakIn the core.specs, isn't ::map-special-bindings always going to be a subset of ::map-bindings? how would ::map-special-bindings spec ever be used given that it's only part of :map-binding-form spec which is defined as (s/def ::map-binding-form (s/merge ::map-bindings ::map-special-binding)) https://github.com/clojure/core.specs.alpha/blob/master/src/main/clojure/clojure/core/specs/alpha.clj#L29-44#2022-07-0414:22kwladykahttps://clojure.wladyka.eu/posts/form-validation/#trick-about-mirroring-s-def-and-via here is use case showing the difference (s/def ::created ::instant) vs (s/def ::created (s/and ::instant))#2022-07-0414:23kwladykaI donā€™t know the exact code which you are asking, but I know the difference between this too#2022-07-0414:26Eric DvorsakI'm not sure I understand your answer, do you mean the intention might just be to produce better error messages?#2022-07-0414:27kwladyka:via [:foo/created], vs :via [:foo/created :foo/instant],#2022-07-0414:28kwladykaI mean without s/and you miss the information about the full path, because it cut corners to make path simpler#2022-07-0414:28kwladykas/and keep full path and this is ONLY ONE WAY to get this information#2022-07-0414:33Eric Dvorsakok, although there's no and involved in the code I linked to (clojure.core.spec.alpha), it's a merge of a spec map-bindings with another map-special-binding after double checking it seems to me like the only point of map-special-binding is to keep the map-binding-form spec open to new keywords#2022-07-0414:36kwladykaI donā€™t know this part of the code, so I canā€™t really give you deeper answers#2022-07-0514:58LycheeseI have a map which needs to have at least one of three possible keys. Sometimes one of those keys is present but with an empty map. I want to not count that as fulfilling the requirement in the or in :test/map but also not invalidate the entire map if one of the other two keys is present and valid. Any way of doing that without first dissoc'ing those keys with empty values?
(s/def :test/b ;; c and d look similar
  (s/and not-empty
         (s/map-of number? (s/multi-spec mm :test/type))))

(s/def :test/map
  (s/keys :req [:test/a
                (or :test/b
                    :test/c
                    :test/d)]
          :opt [:test/e
                :test/f]))
#2022-07-0515:01Alex Miller (Clojure team)with spec, your only option would be a conformer that manipulates the map ahead of the spec. or you can s/and an arbitrary predicate that does anything you want.#2022-07-0808:57LycheeseHow would I go about making a certain key necessary when that key can exist at different depths in a map (and it only needs to exist at least once)? Do I need to normalize the map and put it at one level or is there another way?#2022-07-0810:36reefersleepUse clojure.walk to see if exists at least once?#2022-07-0811:48LycheeseHow would I go about incorporating that into a spec validity check?#2022-07-0811:56reefersleep
(s/def ::guarantee-key-in-map (write-your-fn-here))

 (s/def ::my-map
   (s/and ::guarantee-key-in-map
          (s/keys  :req [:other-key
... and so on
#2022-07-0812:10reefersleepsomething like that?#2022-07-0812:11reefersleep(s/valid? ::my-map some-map)#2022-07-0812:11reefersleepI donā€™t write that many specs, so donā€™t copy verbatim šŸ˜„ it probably doesnā€™t compile#2022-07-0812:13reefersleepOr simpler, without the map thing:
(s/def ::guarantee-key-in-map (write-your-fn-here))

(s/valid? ::guarantee-key-in-map some-map)
#2022-07-0812:18reefersleepThe key takeaway is that your spec is something separate from e.g. s/keys. I donā€™t think you can use s/keys to say arbitrary things about certain values in your map. But you can do so with a separate spec that you can s/and#2022-07-0812:19reefersleepAnybody, feel free to take the wheel and show @U02FM0NNZAB how Iā€™m wrong šŸ™‚#2022-07-0812:59LycheeseI think I only now got what Alex Miller was alluding to in the post above. Feel kinda dumb now. Thank you^^ I think I got my test case working the way I want it to. Now I just need to remember where I wanted to use this šŸ˜…#2022-07-1719:59apbleonardIf I had a map with :billing-address and :shipping-address keys whose associated values were expected to be valid against the same ::address spec, could I express that in spec/keys directly - i.e. without creating intermediary ::billing-address and ::shipping-address specs? Also not sure if (spec/def ::billing-address ::address) or something like it, is an option?#2022-07-1720:42Alex Miller (Clojure team)you would need to do the latter, which is an option#2022-07-1720:43Alex Miller (Clojure team)it's often convenient to have both structural specs (like address) and named attributes that refer to those#2022-07-1722:20apbleonardOk fab - thanks @alexmiller Probably seen that in code many times but not really thought about it much. Trying to build a API data dictionary at work that promotes specifying attributes over HTTP request/responses bodies (to avoid the repetition that brings.) Looks like I will need to list these named specs in the dictionary as well as the attributes themselves.#2022-07-2808:13Ben SlessThis is a bit more relevant to generators, but how can I use the generated result from one generator as the baseline for another? I want to be able yo generate sequences representing events in time or request and response, and for all of them to share an ID, for example#2022-07-2808:26lassemaattaperhaps https://clojure.github.io/test.check/clojure.test.check.generators.html#var-bind? or the let variant, as hinted in the docstring#2022-07-2808:28lassemaattaor perhaps fmap#2022-07-2808:35lassemaattahttps://www.youtube.com/watch?v=F4VZPxLZUdA presentation by Gary Fredericks gives nice examples on how to compose generators#2022-07-2808:38Ben SlessJust guessing because I haven't looked yet, but bind sounds exactly right, thanks! My intuition is from what it means for implementing logic engines, where bind is akin to conjunction#2022-07-2808:42Ben SlessSo let's just run this through, I have a generator for maps, and I want some values from g1 to continue through g2, most brutal way would be merging these values on top of what comes out of g2, but there surely is a better way#2022-07-2808:52lassemaattaI'm no expert on the matter, but I remember once doing something like (->> (gen/tuple random-id-gen seq-of-requests-gen) (fmap (fn [[id reqs]] (map #(assoc % :id id) reqs)))) when I just wanted to "give me some random requests with the same random id" etc.#2022-07-2814:57pppaultuple into fmap is also a solution i used for a map generator#2022-07-2814:59pppauli made a recursive map generator based on example data so my gen looks like (fmap (tuple (map (elements elements)))) and basically you can replace any elements with another fmap (self similar structure)#2022-07-2814:59pppauli saw bind when looking for my solution, but i didn't see how it was going to be better than fmap#2022-07-2815:04pppaulmerging values from previous generators is ok, those other gens are able to shrink, fmap is a bit inefficient, but maybe you could make a new generator that merging for your problem. so long as your fmap isn't doing a lot of work, i don't think it'll be a problem#2022-07-2909:53Ben SlessIs there a way to recover the keys from a map generator before it runs?#2022-07-2914:38pppauli don't know what that means. recover something before it runs doesn't even sound like it makes sense logically#2022-07-2914:38Ben SlessIf I have a map generator, can I know what keys it will generate before running it?#2022-07-2914:38pppaulyes#2022-07-2914:39Ben SlessCool, how?#2022-07-2914:39pppaul(gen/map (gen/elements ...list-of-keys) (gen/elements ...list-of-vals))#2022-07-2914:40Ben SlessI mean going the other way, I already have the map generator, I want to take it apart#2022-07-2914:40pppaulthere is also (gen/hash-map)#2022-07-2914:41pppauli don't get it. are you saying that you are using someone else's gen, that has fixed keys, and you can't see what keys those are from their code?#2022-07-2914:43lassemaattaI was browsing through the generator code today and I got the impression that generator instances are opaque so you can't for example access the source spec. or at least that's my understanding of it#2022-07-2914:45pppaulyou could make a data structure that creates gens, then put that on as meta-data. but you would be creating a wrapper around the whole gen library at that point#2022-07-2915:43Ben SlessIt was a matter of efficiency. If I want to use bind to inject values from one generator to the next, there's no reason to generate those values in the second generator as well, right?#2022-07-2917:21lassemaattathis is just a guess (and perhaps not a good idea), but if you really want to do that kind of optimization (e.g. the value is really expensive to generate and you know the qualified kw you want to replace) then perhaps you could leverage the overrides arg of clojure.spec.alpha/gen :thinking_face: for example something not completely unlike this: (bind random-id-gen (fn [id] (c.s.a/gen ::coll-of-maps {::id (fn [] (return id))}))), where you don't want to execute the default generator for ::id for each map#2022-07-2918:34Ben SlessIt's not critical, maybe I'm overly concerned about stepping into pathological situations#2022-08-0108:16Mark WardleI simply have functions that return generators that wrap another generator and replace the values in the generated map (using fmap) with the explicitly defined ones. Eg see https://github.com/wardle/hermes/blob/main/src/com/eldrix/hermes/rf2.clj I have lots of cases where I need to be careful to use the same identifier in many generated structures. #2022-08-0420:19Stel AbregoHowdy y'all, I'm wondering if it's possible to "bubble up" a spec error from s/every. Here's what I mean:
(s/def :nuzzle/title string?)
(s/def :nuzzle/rss? boolean?)
(s/def :nuzzle/page-entry (s/keys :req [:nuzzle/title] :opt [:nuzzle/rss?]))
(s/def :nuzzle/user-config
  (s/and
   (s/keys :req [:nuzzle/base-url])
   (s/every (fn [[key _]] (or (keyword? key) (vector? key))))
   (s/every (fn [[key val]] (or (not (vector? key))
                                (s/valid? :nuzzle/page-entry val))))))
(s/explain
 :nuzzle/user-config
 {:nuzzle/base-url ""
  [:blog-posts :test-post] {:nuzzle/title 234 :nuzzle/rss? true}})
For every key-value pair in the config map where the key is a vector, I want to validate the associated value against a spec (in this case :nuzzle/page-entry). I'm using s/every because vector keys can be arbitrary but their values must conform to :nuzzle/page-entry. The problem is that when a schema error occurs for a :nuzzle/page-entry (for example :nuzzle/title is not a string), I get an s/explain output like this:
[[:blog-posts :test-post] #:nuzzle{:title 234, :rss? true}] - failed: (fn [[key val]] (or (not (vector? key)) (valid? :nuzzle/page-entry val))) in: [1] spec: :nuzzle/user-config
Is there any way to refactor this to get a better error message about the specific problematic key-value pair (`:nuzzle/title 234`)?
#2022-08-0420:29Alex Miller (Clojure team)I don't know if this will help but you may find the techniques used in https://cognitect.com/blog/2017/1/3/spec-destructuring to be helpful here#2022-08-0420:30Alex Miller (Clojure team)that is, treating the map as a coll of an s/or of k/v tuple types (rather than as a map)#2022-08-0420:32Stel Abrego@U064X3EF3 I will read up on this, thank you!#2022-08-0517:53Dan BoitnottIs there a function to test whether a value is valid and conformed to a spec? For example, given (s/valid? ::myspec val) => true , is there an equivalent such that (s/<something>? ::myspec (s/conform ::myspec val)) => true , or alternatively a way to generate a spec for the conformed value such that (s/valid? (s/<something> ::myspec) (s/conform ::myspec val)) => true? The problem I'm trying to solve is that I have a family of functions which expect to receive conformed values but, because user inputs are involved, need to confirm the inputs are conformed.#2022-08-0517:54Alex Miller (Clojure team)if you have conformed values, then s/unform should give you the unconformed value#2022-08-0517:54Alex Miller (Clojure team)so (s/valid? ::myspec (s/unform ::myspec conformed-value)) ?#2022-08-0517:56Dan BoitnottCool! Lemme give that a shot. Thanks!#2022-08-0518:00Dan BoitnottWorks, thanks! So, is this an idiomatic use of spec? Or am I running into this because I'm misusing conform?#2022-08-0518:06Alex Miller (Clojure team)this is an api provided for un-conforming values, so seems idiomatic to me#2022-08-0522:38Dan BoitnottIn case it's at all interesting, here's what I came up with:
(defn conformed-or-throw
  "If x is a valid and conformed returns x. If x is valid and un-conformed,
  returns (spec/conform spec x). Otherwise throws an ex-info with msg."
  [spec msg x]
  (if (s/valid? spec x) (s/conform spec x)
      (let [unformed (try (s/unform spec x)
                          (catch Exception _ (throw-with-spec spec (str msg " (unable to unform)") x)))]
        (if (s/valid? spec unformed) x
            (throw-with-spec spec (str msg " (unformed)") unformed)))))
#2022-08-0522:39Dan BoitnottWhere throw-with-spec is:
(defn throw-with-spec [spec msg x]
  (throw (ex-info (str msg "\n" (exp/expound-str spec x)
                       "-------------------------\n"
                       (with-out-str (pprint x))
                       "-------------------------")
                  {:spec spec :x x})))
#2022-08-1717:36Stel AbregoIs dynamically redefining specs good practice? I want to check if certain values in a config map match values from another location in the config map so I'm using this function:
(s/def :nuzzle/author keyword?)
;; Dynamically redefine :nuzzle/author based on config
(defn redefine-author-spec [config]
  (s/def :nuzzle/author (s/and keyword? (-> config :nuzzle/author-registry keys set))))
It totally works for my use case but I'm wondering if this is something that is typically not recommended.
#2022-08-1717:52phronmophobicI couldn't find anything directly related, but it does seem to go against the spirit of an open system with global semantics: https://clojure.org/about/spec#_unify_specification_in_its_various_contexts > Specs for data structures, attribute values and functions should all be the same and live in a globally-namespaced directory. https://clojure.org/about/spec#_informational_vs_implementational > Invariably, people will try to use a specification system to detail implementation decisions, but they do so to their detriment. The best and most useful specs (and interfaces) are related to purely information aspects. Only information specs work over wires and across systems. We will always prioritize, and where there is a conflict, prefer, the information approach.#2022-08-1718:13Alex Miller (Clojure team)I would not recommend it due to the redefinition, but dynamically creating a spec is not inherently bad and can be useful. I think my question then is whether this is really a spec or something weaker.#2022-08-1718:13Alex Miller (Clojure team)you also don't necessarily have to put the spec in the registry#2022-08-1718:14Alex Miller (Clojure team)that is, you could just use s/valid? on a spec you create and pass around#2022-08-1718:14Alex Miller (Clojure team)(won't work when validating s/keys with namespaced keys)#2022-08-1722:24Stel Abrego@U064X3EF3 That makes sense. I really just want the amazing bhb/expound error messages so I'm trying to shove stuff into specs that maybe shouldn't be there. I think I could use a local spec instead. Thank you!#2022-08-1912:32Yehonathan SharvitIā€™ve heard Rich saying a couple of times that map specs should be open. Does a similar advice apply for APIs of REST endpoints? Should we allow extra keys (not mentioned in the payload schema) in request payloads? If yes, should we drop extra keys or should we pass them down the stream? If no, why?#2022-08-1913:03ikitommiIMO definitely don't pass anything extra by default from untrusted sources, it's a security risk (e.g. allowing a :credit-checked true to be set to fresh orders). In Java, as it's mostly closed/classes and the JSON-libs have either "strip extras" or "fail on extras". With clojure & open maps, you need an extra tool to do that automatically, given a spec/schema. I recall reitit-ring defaults to "strip extras" with all of spec, schema & malli (supports also "failing on extras" & "explicitly open" for all three)#2022-08-1913:08ikitommiI would use "explicitly open" only from internal (trusted) sources, and when the use case benefits from it.#2022-08-1914:55Yehonathan SharvitWhat do you think about rejecting request with extra keys?#2022-08-1915:23jumarThat's a bad idea I believe- with distributed systems you need flexibility and openness in what you accept. That gives you more freedom in how you do deployments, etc#2022-08-2005:13didibusI think Rich has slightly changed his opinion on that matter, since Spec 2 allows closed map specs. I've had conversations about it with the core team, especially around security use cases, and to validate maps from the producer side. And they seem to recognize those use cases. But there's a time and place for both.#2022-08-2005:16didibusPersonally, on an API, I would drop them. If someone sent a well formed payload that just has more stuff on it, why reject the request?#2022-08-2020:58Yehonathan SharvitIā€™ve heard people say that if you tolerate an extra field, you might have a forward compatibility issue in case you make this extra field part of the schema.#2022-08-2023:17didibusAt the end of the day, you got to decide what scenario you'd rather make annoying to your clients. Do you annoy them by erroring on a request that has everything it needs, but also has more stuff? Do you annoy them in the extremely rare scenario that you didn't error when they were giving you extra stuff, for years, letting them make calls without issues, and one day you might decide to make a field of the same exact name and break their superfluous data calls?#2022-08-2023:20didibusI prefer the former personally, because in practice I find there are more scenarios where that comes in useful. Say there's a field you no longer need? Or say you're doing a deployment, and the client and server are upgrading to support a new field at the same time, if the client deploys first and calls with additional fields on server that still don't support that behavior, etc. But honestly, I find this whole situation kind of unimportant, all these are pretty rare. Most clients tend to call with exactly the fields that the API asked for most of the time.#2022-08-2023:21didibusWell, actually I take that back. The upgrade situation is pretty common. Let me talk about it a bit more.#2022-08-2023:23didibusYou want to add a mandatory new field to the API... How would you do that? The new code requires the new field, it's not optional, you want to make it mandatory. But the current API doesn't take that field, and clients are calling that API without the field. How do you time the deployment of new client code which now always include this new mandatory field, and the server code that mandates it?#2022-08-2023:25didibusIf you deploy the server code first, your validation will error at all current clients which have not upgraded to pass the new field. And even if you didn't validate, well the API wouldn't work, cause your new logic needs that field, it's mandatory.#2022-08-2023:26didibusBut if you have all clients upgrade first, they'll see errors, because the current server code doesn't allow additional fields.#2022-08-2023:26didibusIt's a catch 22#2022-08-2023:27didibusIf your current code though simply ignored additional fields, you're out of the pickle. Tell all clients that a new server code will be deployed on some date, and require a mandatory additional field. Clients can start passing the data even if the current code ignores it. And when they all do, you can deploy the new server code.#2022-08-2105:30Yehonathan SharvitVery insightful scenario @U0K064KQV!#2022-08-2105:49Yehonathan SharvitIā€™ve summarized your insights in a https://twitter.com/viebel/status/1561228623331397632?s=20&amp;t=xJGKWSvqC6QDn42N8rgNvA @U0K064KQV.#2022-08-2111:44pithylessThere's one more common case: silently stripping extra params and returning success may hide incorrect usage of an API (which may or may not be obvious and discovered till much later). Consider if we have an http endpoint where I added an optional filter query-param: /search?status=pending If I'm incorrectly using the filter param (e.g. calling it with status not current_status) it would strip the invalid param and still return results (just not the ones I expected). Now you're relying on tests, etc. which may or may not fail; but also may fail much later and it may take longer to understand what went wrong. I find it useful to define public APIs with strip-extras behavior (for all the reasons @U0K064KQV mentioned above), but to define them as strict and fail-fast when in development mode (for better developer experience and faster identification of contract inconsistencies)#2022-08-2112:25Yehonathan SharvitYeah. There is also an intent concern#2022-08-2121:18didibusI wonder if APIs should return warnings. Succeed, but warn :thinking_face: I also wonder if clients should be allowed to choose the behavior they prefer, can the api take "mode=strict" for example.#2022-08-2121:50Yehonathan SharvitInteresting.#2022-08-2909:27dergutemoritzSorry for the late reply but I just came across this. I would like to add one more perspective: The scenario @U0K064KQV gave is a breaking change (i.e. requiring more from the client). In this case, I would rather provide a new API which requires the new param (possibly just with a v2 suffix. Maybe remove the old version after a transitional period if it's desirable to not keep it around indefinitely. Basically do the same thing as Rich recommends for library APIs here: https://youtu.be/oyLBGkS5ICk?t=1944 In practice, you end up with a similar flow as in the solution suggested by @U0K064KQV but arguably it's a bit more transparent what's going on, e.g. it's very easy to determine how many clients are using the new API already. Also, you're still protected from breaking clients who accidentally were sending this newly required param before.#2022-08-2916:31didibusYes, it's a breaking change where you will no longer handle the prior case. Upgrading to V2, generally implies V1 is still supported. If I do that, I too prefer versioning over making the new field optional. But sometimes, especially for microservices, you will not continue to support the old way, and need a path to migration that doesn't require downtime or cause issues.#2022-08-3109:00dergutemoritzBut you have a transitional period of supporting both in your case, too (i.e. when some clients already send the to-be-required new param and others don't yet), it's just less explicit. I guess if you're sitting in the same organization as all consumers, it's not much of an issue as you can just tightly coordinate the upcoming breaking change. But if you're offering a service that's consumed by third parties outside of your control, I'd definitely prefer the explicit approach.#2022-08-3109:01dergutemoritzP.S.: Just re-watched the rest of the spec-ulation talk and found that it even has a section about (web) services, too, discussing that exact situation: https://youtu.be/oyLBGkS5ICk?t=3958#2022-08-3110:12Yehonathan SharvitWhat does Rich say about that? I cannot listen to the talk right now#2022-08-3110:29dergutemoritzBasically the same thing: Don't break an existing operation foo, make a foo2 instead.#2022-08-3110:29dergutemoritz(break by requiring more)#2022-08-3114:54Yehonathan SharvitIn case the new argument is mandatory, Rich advises us to give a new name to the API. But what about the case where the new argument is optional and of a different type that the one sent by the client?#2022-08-3115:56dergutemoritzAh right, he doesn't discuss the forward compatibility issue. I for one prefer to not allow extra keys on that level for that reason, especially when we're talking plain JSON, i.e. non-namespaced keys. Also, it helps guard against typos for optional params (i.e. clients intending to provide an optional param but because it's misspelt, it's silently ignored). Of course, the latter could also be achieved by other means e.g. have a "warning" backchannel as suggested by @U0K064KQV above.#2022-08-3115:56didibus> But you have a transitional period of supporting both in your case, too (i.e. when some clients already send the to-be-required new param and others don't yet), it's just less explicit. I guess if you're sitting in the same organization as all consumers, it's not much of an issue as you can just tightly coordinate the upcoming breaking change. But if you're offering a service that's consumed by third parties outside of your control, I'd definitely prefer the explicit approach. > Ya, we agree. I'm talking in micro-service architecture, multiple services are part of a coherent system, maybe owned by one or more internal teams. You don't need to support multiple versions of each microservices internally, that's just too much to commit. You can force the consumers to take an update project for the new behavior. Once they did it they can go to prod, there's no transitory period where we support both, clients now pass extra fields, but the current server ignores them. Then you do deployment of the server and it now starts using those additional fields. With paying 3rd party customers to a service, you probably can't force them all to migrate early, and probably need to support both behaviors for a while.#2022-08-3115:57dergutemoritzIndeed - context matters šŸ˜„#2022-08-3115:58didibusI still don't think that erroring on additional fields has advantages in either of those cases though, at least can't think of any here.#2022-09-0103:08Yehonathan Sharvit@U0K064KQV @U06GVE6NR Did you know that the HTTP Prefer Header could be set to either ā€œhandling=strictā€ or ā€œhandling=lenientā€? https://twitter.com/viebel/status/1565174064821551105?s=20&amp;t=ijcp8FtsCUVa34sCvoCOAA#2022-09-0103:28didibusInteresting that was considered. I can see this having value, client can choose. They can even in devo be strict to test they didn't make typos and all, and in prod be lenient if they want.#2022-08-2016:57walterlHi. šŸ‘‹ I'm spec'ing a Ring What is the As I understand it, I would have to check the values manually and separately (unlike with Nm. I see one can't use string keys with s/keys.#2022-08-2109:20Alejandroafter reading a book called Domain Modeling Made Functional I wonder what would be a good way to attach type information to pieces of data. I'm not talking about a type system, but about reasoning of how data flows through components. I'd like have specs that claim a function only accepts data that went through some kind of processing. E.g., an email is parsed and validated on the edge of the system, but not confirmed yet by the user, and I'd like to make sure it's never used inappropriately. How would you do this? Using keywords (with inheritance, probably) or with records?#2022-08-2111:37Alex Miller (Clojure team)Seems like a perfect use of metadata#2022-08-2120:34AlejandroHm. That's interesting, I'll try it, thanks#2022-08-2121:32didibus> I'd like have specs that claim a function only accepts data that went through some kind of processing. E.g., an email is parsed and validated on the edge of the system, but not confirmed yet by the user, and I'd like to make sure it's never used inappropriately This tends to be pretty trivial in Clojure and with Spec. You can use metadata, but I'd say in general, you don't even have too, since you'll probably be working with maps only for your model anyways, you can just add more data to them. For example, on your User entity (which should be a plain Clojure map), just add a :type key where the values are the various state of a User through your system. So maybe you have a {:type :verified-user} and a {:type :registered-user}. Now in each function, where you expect a :verified-user, well just have a spec for that which will also assert that :type is :verified-user. This will probably be pretty close to the book you read, where I'm assuming they use custom ADT types to model this. Alternatively, you can enhance the user with more info, and validate the info, such as adding an :email-verified true on the User only after its been verified, then have a spec on the function that asserts that :email-verified exists and is true on the user.#2022-08-2121:35didibusYou could make this metadata, but it's harder to use spec to validate meta, not super hard, but you kind of need two specs, one for the data and one for the meta. The advantage of meta is if you don't want them to participate in equality. Like if you still want two users to be equal even if one is verified and the other is not.#2022-08-2122:21didibus
(require '[com.myapp :as-alias app])
(require '[com.myapp.user :as-alias user])

(s/def ::user/type #{:registered-user :verified-user})
(s/def ::user/username string?)
(s/def ::user/email string?)
(s/def ::app/user
  (s/keys :req-un [::user/type ::user/username ::user/email]))
(s/def ::app/verified-user
  (s/and ::app/user #(= :verified-user (:type %))))
(s/def :app/registered-user
  (s/and ::app/user #(= :registered-user (:type %))))
So now on functions that expect a :verified-user you can do:
(s/fdef somefn
  :args (s/cat :verified-user ::app/verified-user)
  :ret ::app/verified-user)
And on the function that verifies a registered-user you can do:
(s/fdef verify-user
  :args (s/cat :registered-user ::app/registered-user)
  :ret ::app/verified-user)
#2022-08-2122:30didibusAnd if different types of users also had different set of keys, you can multi-spec the ::app/user spec.#2022-08-2114:11popeyeWhen to use clojure spec ?#2022-08-2114:26danierouxhttps://clojure.org/about/spec#2022-08-2218:02lilactownis there a reason that there's no concrete Spec type/class? I have a desire to extend a library I'm writing to dispatch on whether a value is a reified spec value, and I can't easily do that without taking a dependency directly on clojure.spec.alpha#2022-08-2218:03lilactowni.e. the only way I can tell whether I've been passed a spec is by using s/spec?#2022-08-2218:04Alex Miller (Clojure team)It's not a type#2022-08-2218:13Alex Miller (Clojure team)You could dynamically load and resolve that function #2022-08-2218:22lilactownthat doesn't work in CLJS#2022-08-2218:23lilactownin various places in clojure.spec.alpha an object is created via (reify Spec ,,,) and returned, which is what I want to extend the behavior of#2022-08-2218:27lilactownI don't care about the name, just wishing there was a concrete type I could extend#2022-08-2218:49Alex Miller (Clojure team)sets are also specs, and functions. there is not "one type" to check for#2022-08-2218:53Alex Miller (Clojure team)the behavior is defined via the protocol. things like with-gen are a good example of extending by wrapping. but maybe there is some better way to do what you need, not sure#2022-08-2320:29vlaaadDoes something like that make sense?
(defmulti dispatch identity)
(defmethod dispatch :default [v]
  (:spec v))

(s/valid? 
  (s/multi-spec dispatch :spec)
  {:spec (s/keys :req-un [::a ::b]) :a 1 :b 2})
=> true
The real code in :default branch would be more complex, but the idea is that I have a complex rule to infer spec from the value, and I don't want to encode it in a single predicate that executes the rule and calls s/valid? inside because it will lose the error reporting/explanation. I.e. I don't want to do this:
(s/valid?
  (s/spec #(s/valid? (:spec %) %))
  {:spec (s/keys :req-un [::a ::b]) :a 1})
=> false
because (s/explain-data ...) will just say that (s/valid? (:spec %) %) is false and not that the :b is missing
#2022-08-2320:31vlaaadExplanation comparison:
(s/explain
  (s/spec #(s/valid? (:spec %) %))
  {:spec (s/keys :req-un [::a ::b]) :a 1})
=> {:spec ... :a 1} - failed: (valid? (:spec %) %)

(s/explain
  (s/multi-spec dispatch :spec)
  {:spec (s/keys :req-un [::a ::b]) :a 1})
=> {:spec ... :a 1} - failed: (contains? % :b)
#2022-08-2418:52vlaaadcheck this out:
(defmacro def-dynamic-spec [name & fn-tail]
  (let [multi-name (symbol (str name "-multi-fn"))]
    `(do
       (defmulti ~multi-name (constantly nil))
       (defmethod ~multi-name nil 
a macro that creates a spec that dynamically infers a value spec using a function of the value, while preserving error reporting/explanation data
#2022-08-2508:37vlaaadAny opinions?#2022-08-2508:37vlaaadAny opinions?#2022-08-2409:47Ishaan Guptahow far is the idea that spec2 schemas are just sets of keys gonna be taken? is there going to be a full set of set operations for schemas like there's a full set of regex operations for a sequence?#2022-08-2411:59Alex Miller (Clojure team)It may be take even less far than it is :) not sure yet#2022-08-2409:48Ishaan Guptais there any utility in a schema intersection?#2022-09-0702:58walterlFrom the https://clojure.org/guides/spec: > Note that the :ret and :fn specs are not checked with instrumentation as validating the implementation should occur at testing time. How can one enable :fn and :ret validation during tests, without using stest/check?#2022-09-0705:08seancorfieldPerhaps https://stackoverflow.com/questions/40697841/howto-include-clojure-specd-functions-in-a-test-suite ?#2022-09-0713:20KelvinI use the Orchestra library to instrument :ret (alongside the default :args): https://github.com/jeaye/orchestra#2022-09-0920:30Cam SaulWhat's the easiest way to do something like a "soft cut" e.g. core logic conda to lock a regex spec into validating an optional argument with a specific spec if it meets some condition, and failing otherwise? Suppose I have a spec like this:
(s/def ::my-map
  (s/keys :opt-un [...]))

(s/def ::my-spec
  (s/cat :my-map (s/? ::my-map) :options (s/* (s/cat :k keyword? :v any?))))
If I pass in a map that fails the ::my-map spec I get an error like
identity - failed: Extra input in: [0] spec: ::my-spec
because it then tries to use that argument as an option. How can I force it to validate maps with the ::my-map spec? I've tried stuff like (s/& map? ::my-map) but that doesn't work either
#2022-09-0921:28Alex Miller (Clojure team)just checking - you have an optional followed by kv pairs, and with what input do you get this error? something that's neither a map or keyword as first arg?#2022-09-0921:30Alex Miller (Clojure team)I'm not sure if there's an easy alternative for you - seems like you could possibly treat this as two specs based on even/odd arg count. it's an unusual usage but you could probably build an s/multi that switched based on that and used different specs#2022-09-0921:33Cam SaulI ended up doing something like
(s/& 
 (s/cat :my-map (s/? map?) :options (s/* (s/cat :k keyword? :v any?)))
 (s/keys :opt-un [:my-map])
which works okay I guess but is a little icky I think
#2022-09-0921:36Cam SaulHere's a more detailed example including something with neither a map nor keyword first arg. I just get an extra input error
(s/def ::k
  integer?)

(s/def ::my-map
  (s/keys :req-un [::k]))

(s/def ::my-spec
  (s/cat :my-map (s/? ::my-map) :options (s/* (s/cat :k keyword? :v any?))))

(s/explain-str ::my-spec [{:k 100}])
(s/explain-str ::my-spec [{:k 100} :x :y])
;; => "Success!\n"

(s/explain-str ::my-spec [100])
;; => "(100) - failed: Extra input in: [0] spec: ::my-spec\n"

(s/explain-str ::my-spec [{}])
;; => "({}) - failed: Extra input in: [0] spec: ::my-spec\n"
#2022-09-0921:38Cam Sauldefmulti is actually one real-world example of where you have a usage like this. The metadata attribute map is optional and after that it supports dispatch-fn as a positional argument and then a few optional key-value arguments like :default and :hierarchy if I recall correctly.#2022-09-0921:42Cam SaulThis came up for me because I was working on a def macro that did something similar (supports and optional attribute map and keyword options) and I tried to tweak the spec to validate one of the keys in the attribute map specifically and it made things fail with confusing errors. In the example above I want the last failure to tell me it's failing because :k is not an integer?#2022-09-0921:46Cam SaulThis is the working version I came up with:
(s/def ::my-spec
  (s/& (s/cat :my-map (s/? map?) :options (s/* (s/cat :k keyword? :v any?)))
       (s/keys :opt-un [::my-map])))

(s/explain-str ::my-spec [{}])
=> "{} - failed: (contains? % :k) in: [:my-map] at: [:my-map] spec: ::my-map\n"
I was wondering if maybe there was some easier way to do this, e.g. some way to define my so-called "soft cut" inline rather than having to wrap the whole thing in s/&
#2022-09-0921:30thiruIf I were to start using Spec today is it recommended to use Spec 1 or Spec 2? Also, my codebase is Clojure 1.8 and can't upgrade. The only issue I see with this is that ident? is missing but that's easy enough to add. Is there any strong reasons to avoid using Spec with Clojure versions before 1.9?#2022-09-0921:31Alex Miller (Clojure team)spec 2 has not been released and is not under active dev at this moment#2022-09-0921:31Alex Miller (Clojure team)I think someone did have a backport of spec at one point (we are not maintaining anything official though)#2022-09-0921:32Alex Miller (Clojure team)https://github.com/tonsky/clojure-future-spec#2022-09-0921:33thiruExcellent, thank you for the super fast response Alex! šŸ™‚#2022-09-1214:18Kelvinhttps://clojurians.slack.com/archives/C1B1BB2Q3/p1662759068905259?thread_ts=1662759019.232099&amp;cid=C1B1BB2Q3 Interesting - I wasnā€™t aware of this, does that mean spec2 is delayed indefinitely? (Posting outside of thread since I donā€™t want to derail)#2022-09-1214:34Alex Miller (Clojure team)It just means that at this moment we are not actively working on it#2022-09-1814:04maleghastHello All, was just wondering if there is already a tool out there for deriving spec from a data structure..? Basically I want to consume JSON and render it as EDN / Clojure and then reverse engineer a spec from that data structure.#2022-09-1814:24lispycloudsi knew about https://github.com/stathissideris/spec-provider havent tried it in recent times#2022-09-1814:24maleghastThanks @U7ERLH6JX I will go take a look...#2022-09-2001:59Drew VerleeYeah, malli can do that too. but in general, i would think three times before going that route.#2022-09-2018:30phronmophobicI'm trying to use spec to build a generator for valid destructure bindings. I'm starting with the specs in clojure.core.specs.alpha.
(s/def ::seq-binding-form
  (s/and vector?
         (s/cat :forms (s/* ::binding-form)
                :rest-forms (s/? (s/cat :ampersand #{'&} :form ::binding-form))
                :as-form (s/? (s/cat :as #{:as} :as-sym ::local-name)))))
Generating ::seq-binding-form values struggles because it's just generating random vectors that hopefully satisfy the s/cat. This seems to get fixed with (gen/fmap vec ...):
(s/def ::seq-binding-form
  (s/with-gen
    (s/and vector?
           (s/cat :forms (s/* ::binding-form)
                  :rest-forms (s/? (s/cat :ampersand #{'&} :form ::binding-form))
                  :as-form (s/? (s/cat :as #{:as} :as-sym ::local-name))))
    #(gen/fmap
      vec
      (s/gen
       (s/cat :forms (s/* ::binding-form)
              :rest-forms (s/? (s/cat :ampersand #{'&} :form ::binding-form))
              :as-form (s/? (s/cat :as #{:as} :as-sym ::local-name)))))))
Unfortunately, the version using gen/fmap now usually throws a stackoverflow exception. However, I noticed that if I ignore the vector requirement for ::seq-binding-form, then the stack overflow exception goes away.
(s/def ::seq-binding-form
  (s/cat :forms (s/* ::binding-form)
         :rest-forms (s/? (s/cat :ampersand #{'&} :form ::binding-form))
         :as-form (s/? (s/cat :as #{:as} :as-sym ::local-name))))
Is there a way to prevent and stackoverflow exceptions and have it generate a vector?
#2022-09-2018:37phronmophobicIt looks like spec2 might have better support for this. I don't have any real reason to stick with spec1, so I'll see if I can get this working with spec2.#2022-09-2018:37Ben SlessWhat if instead of fmap you use and conforming?#2022-09-2018:38phronmophobicI don't know about conforming, but I'll take a look#2022-09-2018:38Ben SlessMaybe it's called conform, lemme check#2022-09-2018:39phronmophobicI was checking out spec2 based off this previous thread, https://clojurians.slack.com/archives/C1B1BB2Q3/p1650425212914829?thread_ts=1650424802.839689&amp;cid=C1B1BB2Q3#2022-09-2018:39Ben Slesshttps://clojuredocs.org/clojure.spec.alpha/conformer#2022-09-2018:40phronmophobicgiving it a try!
#2022-09-2018:51phronmophobicI don't think that solves this use case. It still doesn't generate ::seq-binding-form as a vector. It does convert it if I conform the value, but I think it's worth trying spec2#2022-09-2018:53Ben SlessGen doesn't emit conformed values?#2022-09-2018:59phronmophobicIt's totally possible I'm doing it wrong. My attempt was:
(s/def ::seq-binding-form
  (s/and
   (s/cat :forms (s/* ::binding-form)
          :rest-forms (s/? (s/cat :ampersand #{'&} :form ::binding-form))
          :as-form (s/? (s/cat :as #{:as} :as-sym ::local-name)))
   (s/conformer vec)))
#2022-09-2019:01Alex Miller (Clojure team)the specs need to be updated to have this work with spec 2#2022-09-2019:03Alex Miller (Clojure team)to use catv instead of cat etc#2022-09-2019:38phronmophobicIt works!#2022-09-2117:04phronmophobicJust to follow up, test.check is magic. For the right problems, it finds tricky use cases that I would never think of.
> (let [[ & [ & [& [& [& [& hi]]]]]] [42]]
    hi)
(42)
> (let [[& {:as m}] [:a 42]]
    m)
{:a 42}
#2022-09-2117:09Alex Miller (Clojure team)generation isn't magic, but I'm pretty sure shrinking is#2022-09-2619:15kennyCurious - why does the Spec form for s/double-in return a s/and with a series of predicates attached for each option passed (e.g., :min, :max, etc), and something like s/every does not include the https://github.com/clojure/spec.alpha/blob/13bf36628eb02904155d0bf0d140f591783c51af/src/main/clojure/clojure/spec/alpha.clj#L558-L567?
(s/form (s/double-in :min 2))
=> (clojure.spec.alpha/and clojure.core/double? (clojure.core/fn [%] (clojure.core/<= 2 %)))
(s/form (s/every string? :min-count 2))
=> (clojure.spec.alpha/every clojure.core/string? :min-count 2)  
#2022-09-2619:18Alex Miller (Clojure team)This is a limitation of how double-in is written as a composite but without custom form support. This has actually been addressed in spec 2 (not sure if this particular one has been fixed there but it could be)#2022-09-2619:21kennyOk. So the desired output for s/double-in's (and its ilk) form would be similar to that of s/every -- returning the kv options passed to the form instead of the composite?#2022-09-2619:33Alex Miller (Clojure team)yep, I think the s/form should reproduce the original not expose how it's implemented#2022-09-2619:34Alex Miller (Clojure team)
spec-alpha2 % clj
Clojure 1.10.1
user=> (require '[clojure.alpha.spec :as s])
nil
user=> (s/form (s/double-in :min 2))
(clojure.alpha.spec/double-in :min 2)
#2022-09-2619:34kennyMakes sense. Thanks for the response. #2022-09-2810:49joost-diepenmaatGood morning. I might be mis-remembering something, but isnā€™t there some analogue to (s/keys :req-un .. :opt-un ā€¦) that works for strings instead of keywords?#2022-09-2810:50joost-diepenmaatI thought that (s/strs ā€¦) would exist but noā€¦#2022-09-2811:55Alex Miller (Clojure team)no#2022-09-2816:25joost-diepenmaatFair enough. Thanks. #2022-10-0115:27NiclasDoes anybody know of a lib that allows me to inline fn specs in the fnā€™s metadata? Looking for something like:
(defn myfn
  {:spec (...)}
  ...)
ā€¦ which would be equivalent to:
(defn myfn ...)

(s/fdef myfn ...)
#2022-10-0115:55teodorluI don't know of a library that does this specifically, but I think it should be doable with a macro. Perhaps something like this:
;; idea: create a macro defn+spec

(defn+spec myfn
  {:spec (,,,)}
  []
  ,,,)

;; so that it expands to

(do
  (defn myfn
    {:spec (,,,)}
    []
    ,,,)

  (s/fdef myfn (:spec (meta #'myfn))))
#2022-10-0201:30colinkahnThere's https://github.com/danielcompton/defn-spec which has links to some other libs too in the README#2022-10-0211:42FiVoIs there a standard way to reuse specs for conformed values? Do I need write a spec for the "normal" as well as conformed version of the data? As an example:
(s/def ::tuple (s/and vector?
                      (s/cat :first identity :second identity)))

(def tuple (s/conform ::tuple [1 2]))

(defn foo [t] t)

(s/fdef foo :args (s/cat :t ::tuple))

(foo tuple) ; => error
I would like to reuse ::tuple for my funciton spec.
#2022-10-0214:05Alex Miller (Clojure team)Whatā€™s the error? (and why not use s/tuple in that spec?)#2022-10-0214:42FiVoThe ::tuple spec was just an example and maybe it was not clear what I am asking. I am mainly wondering if there is a way to avoid the ::tuple-conformed spec in the following
(s/def ::tuple (s/and vector?
                        (s/cat :first int? :second int?)))

  (def conformed-tuple (s/conform ::tuple [1 2]))
  conformed-tuple
  ;; => {:first 1, :second 2}

  (s/valid? ::tuple conformed-tuple)
  ;; => false

  (s/def ::first int?)
  (s/def ::second int?)
  (s/def ::tuple-conformed (s/keys :req-un [::first ::second]))
  (s/valid? ::tuple-conformed conformed-tuple)
  ;; => true
#2022-10-0214:44FiVoYou actually kind of answered my question here https://stackoverflow.com/questions/41686718/specs-for-conformed-specs-asts#2022-10-0214:55FiVoSo I think the question becomes how do people usually generate both specs for the above case?#2022-10-0219:30Brian BeckmanSOLVED! Hello, speccers. Iā€™m really stuck and would be very grateful for advice! Iā€™m writing specs to generate test cases for the back end of a compiler (LPython [sic]; http://lpython.org). The compiler back-end accepts a kind of S-expression syntax (nice!?). TL;DR: I want to generate S-exprs like (const 42 (I 4)) ; :head const, :val 42, :type (I 4) I can easily get A. [const 42 (I 4)] ; error: need round brackets outside B. (const 42 I 4) ; error: need round brackets inside C. (const 42 ((I 4))) ; error: too many brackets inside but not what I want without un-nestable post processing. DETAILS: (I 4) could be other types like (F 16) and (C 32) in actual code, so the :type spec (it seems), must be a s/cat to get the internal round brackets:
(s/def ::ttype  (s/cat :head #{'I}, :bytes #{4}))
(->> ::ttype s/gen gen/generate) ; ~~~> (I 4)
I run into trouble right away trying to spec ::iconst to generate (const 42 (I 4)) A. If I spec ::iconst as a tuple, I get a well-formed expression, but with square brackets outside!:
(s/def ::iconst (s/tuple #{'const} integer? ::ttype))
(->> ::iconst s/gen gen/generate) ; ~~~> [const 42 (I 4)]
B. If I spec ::iconst as a s/cat, the ::ttype field gets spliced instead of appended:
(s/def ::iconst (s/cat :head #{'const} :value integer?
            :type ::ttype))
(->> ::iconst s/gen gen/generate) ; ~~~> (const 42 I 4)
B. If I try a nested s/cat for the ::type field, itā€™s no help:
(s/def ::iconst (s/cat :head #{'const} :value integer?
            :type (s/cat :singleton ::ttype)))
 (->> ::iconst s/gen gen/generate) ; ~~~> (const 42 I 4)
C. If I try s/coll-of :into () :count 1, I get TOO MANY brackets!
(s/def ::iconst (s/cat :head #{'const} :value integer?
            :type (s/coll-of ::ttype :into () :count 1)))
(->> ::iconst s/gen gen/generate) ; ~~~> (const 42 ((I 4)))
The only solution Iā€™ve been able to see is a post-processing step, but this is EXTERNAL to the spec and doesnā€™t scale to my realistic recursively nested examples:
(s/def ::iconst (s/tuple #{'const} integer? ::ttype))
(->> ::iconst
   s/gen
   gen/generate
   (apply list)) ; ~~~> (const 42 (I 4))
Z. Even s/with-gen takes me back to the future:
(s/def ::iconst-with-gen
 (s/with-gen ::iconst
  (fn [] (tgen/let [iconst ::iconst]
      (apply list iconst)))))
(->> ::iconst-with-gen s/gen gen/generate) ; ~~~> fails after 100 tries
Iā€™m really stuck and would be most grateful for ideas!
#2022-10-0219:51phronmophobicI think wrapping the ::type spec with s/spec will do what you want:
(s/def ::ttype (s/spec (s/cat :head #{'I}, :bytes #{4})))
(s/def ::iconst (s/cat :head #{'const} :value integer?
                       :type ::ttype))
(->> ::iconst s/gen gen/generate) ;; (const -241544 (I 4))
#2022-10-0219:54Brian BeckmanSOLVED! thanks!#2022-10-0219:57Brian BeckmanIs there a general rule-of-thumb I can add to my ā€œintuition-bankā€ about this? I would have thought the s/spec wrapper to be redundant!#2022-10-0220:15Brian BeckmanMaybe I should just think of it as making a ā€œnestableā€ from a ā€œspliceableā€ spec in s/cat#2022-10-0221:11phronmophobicyea, it's not totally obvious. it is mentioned in the docs, https://clojure.github.io/spec.alpha/clojure.spec.alpha-api.html#clojure.spec.alpha/spec >
Can also be passed the result of one of the regex ops -
> cat, alt, *, +, ?, in which case it will return a regex-conforming
> spec, useful when nesting an independent regex.
https://clojure.org/about/spec#_defining_specs > spec should only be needed when you want to override a generator or to specify that a nested regex starts anew, vs being included in the same pattern.
#2022-10-0222:53Brian BeckmanYouā€™re very generous! The doc is not emphatic about it, but it is clear. I found this SO that helped me quite a bit, too https://stackoverflow.com/questions/37583708/clojure-spec-alt-vs-or-for-sequence-spec#2022-10-0510:32popeyeI have output response where , response fields are different in different icons, Now I wanted to apply the validation for each input fields, Since this is dynamic in nature, Are we able to create a dynamic spec for validation ?#2022-10-0515:55colinkahnHave you looked at multi-spec?#2022-10-0610:31pavlosmelissinosI have a (s/def ::items (s/coll-of ::item :min-count 1)). How do I create a new spec from it that has some extra constraints, e.g. (s/def ::items-2 (s/coll-of ::item :min-count 1 :gen-max 6)) without repeating the common ones?#2022-10-0612:05Alex Miller (Clojure team)I donā€™t think you can easily do that#2022-10-0612:12pavlosmelissinosAlright, thanks šŸ™‚ It would have been nice to have but I can work around it, so it's not a big deal.#2022-10-0620:27Cora (she/her)šŸ‘‹:skin-tone-2:#2022-10-0620:29Cora (she/her)is there a way to spec maps but without defining entities for the keys?#2022-10-0620:29Alex Miller (Clojure team)map? is a valid spec :)#2022-10-0620:29Cora (she/her)I mean a spec for a combination of specific keys with specific values#2022-10-0620:30Alex Miller (Clojure team)what kind of keys / what kind of values ?#2022-10-0620:30Alex Miller (Clojure team)you can s/and any custom predicate you might have#2022-10-0620:31Cora (she/her)say I want a spec for a map of {:key1 "a string" :key2 :a-keyword} without s/def :key1 and s/def :key2#2022-10-0620:32Cora (she/her)I can write arbitrary functions for these but it would be nice if there were a way to go just {:key1 string? :key2 keyword?}#2022-10-0620:33Alex Miller (Clojure team)
user=> (s/def ::m (s/keys :req-un [:foo/key1 :foo/key2]))
:user/m
user=> (s/valid? ::m {:key1 "a string" :key2 :a-keyword})
true
#2022-10-0620:33Cora (she/her)that doesn't validate the values, though?#2022-10-0620:34Alex Miller (Clojure team)but no, you can't get the value check without also def'ing the key specs#2022-10-0620:34Cora (she/her)gotcha#2022-10-0620:34Cora (she/her)ok#2022-10-0620:34Cora (she/her)for reference, with malli you can do:
[:map
   [:id string?]
   [:tags [:set keyword?]]
   [:address
    [:map
     [:street string?]
     [:city string?]
     [:zip int?]
     [:lonlat [:tuple double? double?]]]]]
#2022-10-2715:32souenzzobtw malli is more structural https://en.wikipedia.org/wiki/Structural_type_system spec is more nominal https://en.wikipedia.org/wiki/Nominal_type_system spec will force you to give a unique name to each piece of data that you have. it is nice to have both available in clojure not a better vs worst. each one has it own advantages and penalties#2022-10-2716:17Cora (she/her)for sure, I just greatly prefer structural and was working on a project that only had spec#2022-10-2717:28souenzzoI just greatly prefer named and I'm working on a project that only had malli let's switch chairs šŸ˜‚#2022-10-0620:35Cora (she/her)I'm on a project using spec, though, and was hoping for something similar. sounds like I need to define the key specs#2022-10-0620:35Cora (she/her)thanks, Alex#2022-10-0710:33reefersleepI mean, you could write your own spec-outputting function that works like you describe šŸ™‚#2022-10-0710:50reefersleepThis is not what you want, the failure message would not be very helpful. But just a back-of-the-napkin sketch to inspire you šŸ™‚
(defn map-spec [validation-map]
  (s/def (fn [input]
           (and (clojure.set/subset? (set (keys input))
                                     (set (keys validation-map)))
                (->> input
                     (filter (fn [[k v]]
                               (contains? (set (keys validation-map))
                                          k)))
                     (map (fn [[k v]]
                            (let [validation-fn (get validation-map k)]
                              (validation-fn v))))
                     (every? identity))))))
#2022-10-1008:57reefersleepActually, it might be a bit difficult to get the usually good feedback messages from the spec validating functions unless you venture into macro territory, so itā€™s probably a bit harder than the initial impression I gave. Still, might be an interesting exercise. But probably, itā€™s a good idea to follow the conventions laid out by spec, as @U064X3EF3 says. Wrapping spec generation in macros can also deter tool usage (such as ā€œgo to spec definition of keywordā€ in Cursive - I specifically lived through that šŸ™‚ )#2022-10-1012:53popeyeI have a input for password , how to do a spec validation for password ?
:password {:type :encrypted}
`
#2022-10-1120:31Brian BeckmanDear friends: Seeking explanation. I noticed that certain with-gen specs describe as unknown. To wit:
(s/def ::foo (s/with-gen (fn [x] (integer? x))
               (fn [] (gen/return 42))))
(s/valid? ::foo 1234)
;; => true
(gen/generate (s/gen ::foo))
;; => 42
(s/describe ::foo)
;; => :clojure.spec.alpha/unknown
But here is one that works:
(s/def ::bar (s/with-gen integer?
               (fn [] (gen/return 43))))
(s/valid? ::bar 2345)
;; => true
(gen/generate (s/gen ::bar))
;; => 43
(s/describe ::bar)
;; => integer?
What gives, please & thanks?
#2022-10-1120:56Alex Miller (Clojure team)(fn [x] (integer? x)) is an anonymous function, which is just an opaque object so there is no way in Clojure to work backward from that to a form#2022-10-1120:57Alex Miller (Clojure team)with things like integer? we are working some magic to see that's a function tied to a var and demunging a class name to recover it#2022-10-1120:58Alex Miller (Clojure team)in spec 2, we are able to get rid of all of that and better capture forms by having well separated symbolic and object layers, so eventually this will be improved#2022-10-1121:09Brian BeckmanThat makes sense, Alex. ty. I did notice that sometimes (specifically when there is no with-gen) describe can report an anonymous function:
(s/def ::baz (fn [x] (integer? x)))
(s/describe ::baz)
;; => (fn [x] (integer? x))
#2022-10-1121:16Alex Miller (Clojure team)in that case, s/def is a macro and can capture the form#2022-10-1121:17Alex Miller (Clojure team)with-gen is a function and the args are evaluated before invocation#2022-10-1121:27Alex Miller (Clojure team)there is another option too that is a macro and can specify a custom generator:
user=> (s/def ::foo (s/spec (fn [x] (integer? x)) :gen (fn [] (gen/return 42))))
:user/foo
user=> (s/describe ::foo)
(fn [x] (integer? x))
#2022-10-1121:28Alex Miller (Clojure team)(although this is a different known problem in that it omits the custom gen)#2022-10-1122:02Brian Beckmangreat stuff. tyvm#2022-10-1203:00Brian BeckmanFeature Request: codox-friendly docstrings for specs šŸ™‚ Iā€™m up to a few hundred specs in my project now and would love to catalogue them automatically in the same way as I do defs and defns.#2022-10-1204:15Alex Miller (Clojure team)This is the most requested feature in the issue tracker#2022-10-1204:15Alex Miller (Clojure team)In scope for spec 2, but havenā€™t worked on it yet#2022-10-1212:38Brian BeckmanIn practice, I put docstrings in functions that validate or generate from the specs, so the lack is not a blocker.#2022-10-1419:39Brian BeckmanOBSERVATION: I noticed a handy feature: specs created via s/with-gen can automatically filter out nil. Is it generally true that such specs never produce nil when the spec-part of s/with-gen forbids nil? I guess, stated that way, itā€™s obvious that it should be generally true. This is a nice feature for me because my project generators create trees with level-crossing semantical constraints that can go wrong in combinatorial ways. Any time anything goes wrong (e.g., 0 to a negative power somewhere in the tree), I just barf out nil as if in a maybe monad (itā€™s about 2% of the time, so itā€™s fine). Itā€™s nice that I get automatic filtering of the generated values at the spec level. Actually, itā€™s brilliant! It let me strip out the maybe monad from my code (simplifying it greatly) and just rely on nil punning and some-> and some->> operations. Consider the following:
(s/def ::nil-producing-spec
  (s/or :nil nil? :int integer?))

(def nil-producing-generator
  (s/gen ::nil-producing-spec))

(gen/sample nil-producing-generator)
;; => (nil nil -1 -2 nil 6 3 nil nil nil)

(s/def ::nil-rejecting-spec
  (s/with-gen
    integer?
    (fn [] nil-producing-generator)))

(gen/sample (s/gen ::nil-rejecting-spec))
;; => (0 -4 -2 -5 0 -64 0 -4 6 0)
#2022-10-1502:20lassemaattaI think s/gen checks the generated values and verifies they pass the spec (https://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/alpha.clj#L287). You could swap integer? with something like pos-int? and see that it now filters out more than just nils.#2022-10-1818:54ghadispec derived generators automatically filter spec derived values by the validity of the spec#2022-10-1818:55ghadithis means that a great technique is to attach a sloppy generator to a specific spec#2022-10-1818:56ghadie.g. give a generator that generates all dates to a spec/predicate for holidays#2022-10-1820:11kpavIs there a way to generate only required or optional keys?#2022-10-1820:19kpave,g, Given this spec:
(s/def ::required-key string?)
  (s/def ::optional-key string?)
  (s/def ::foo (s/keys :req [::required-key]
                       :opt [::optional-key]))
I'd like a way to force generation of:
{:required-key "foo"}
And also:
{:required-key "foo"
   :optional-key "bar"}
#2022-10-1820:29Alex Miller (Clojure team)the latter should be what you get by default - required and sometimes optional#2022-10-1820:29Alex Miller (Clojure team)you could filter with fmap to get the former#2022-10-1820:30kpavThis is assuming I am given an unknown spec. Basically, I'd like to know which keys are required, and which keys are optional#2022-10-1820:31Alex Miller (Clojure team)there are some 3rd party libraries that have functions to parse that from the spec form#2022-10-1820:32kpavah ok thanks! I'll take a look into those#2022-10-3003:20jmmkis it a bad idea to import/rename a spec. Or is there a better way to accomplish this I have a few different something/client.clj namespaces and I need to pass instances of the clients to a handler constructor. since the unqualified name client conflicts, I create new names for them e.g.
(ns myapp.api.client 
  (:require [clojure.spec.alpha :as s]))

(s/def ::client (partial satisfies? ApiClient))
(ns myapp.other.client 
  (:require [clojure.spec.alpha :as s]))

(s/def ::client (partial satisfies? OtherClient))
(ns myapp.handler 
  (:require [clojure.spec.alpha :as s] 
            [myapp.api.client :as ac] 
            [myapp.other.client :as oc]))

(s/def ::api-client ::ac/client)
(s/def ::other-client ::oc/client)

;; handler constructor accepts config with :api-client and :other-client
(s/def ::config (s/keys :req-un [::api-client ::other-client]))
#2022-10-3003:34seancorfieldOn the face of it, this seems reasonable to me... although that last line is missing :: yes?#2022-10-3003:35seancorfield(and now your ::config structure has to have unqualified keys called :api-client and :other-client -- is that what you intended?)#2022-10-3004:06jmmk> although that last line is missing :: yes? Yes, those keys should have :: > (and now your ::config structure has to have unqualified keys called :api-client and :other-client -- is that what you intended?) It is what I intended, but I am not sure if my reasoning is sound. For a bit more context, Iā€™m using integrant, so these client args will be passed from a ā€œsystem mapā€ I guess it boils down to wanting to use shorter/easier keys (while still disambiguating the two separate clients) to refer to these, both within the system config and within the handler.#2022-10-3011:37Christian JohansenYou are effectively throwing away the namespace that originally disambiguated them and then giving them new names. Why not just use namespaced keys instead? Less indirection, easier to understand what things are.#2022-10-3013:12jmmkIt feels a bit weird to me that map keys are tied to their values/semantics, but maybe that's the "spec way" and I just need some more time getting used to it? it also feels verbose in the config and verbose to destructure because they both share the name client and at some point I have to change the name anyway
;;;; rename

;; config
:myapp/handler
{:other-client #ig/ref :myapp.api/client
 :api-client #ig/ref :myapp.other/client}

;; construct
(s/def ::api-client :ac/client)
(s/def ::other-client :oc/client)

(defmethod ig/init-key :myapp/handler
  [_ {:keys [api-client other-client]}]
  (handler api-client other-client))
vs
;;;; use namespaced keywords throughout

;; config
:myapp/handler
{:myapp.api/client #ig/ref :myapp.api.client
 :myapp.other/client #ig/ref :myapp.other/client}

;; construct
(defmethod ig/init-key :myapp/handler
  [_ config]
  (let [{api-client ::ac/client
         other-client ::oc/client} config]
    (handler api-client other-client)))
#2022-10-3016:55seancorfieldI wouldn't bother destructuring them for that call -- I'd use (handler (::ac/client config) (::oc/client config))#2022-10-3017:48jmmkWould it be fair to say then that if I am using a spec and want to pass instances (or things that conform to that spec) in a map that it makes the most sense to use the namespaced keyword for that, regardless of the extra verbosity?#2022-10-3018:36jmmkAnd a followup to that: If preferring the full keyword is generally a good idea, what then is an appropriate use case for req-un ? Is it for things you donā€™t want/need to specify as precisely? beyond the above potential misuse of it, I am using it for things like port, host config#2022-10-3019:33seancorfield@U08FV2128 Namespaced keywords are idiomatic and I think in the case you're describing -- passing two different configurations into another function -- it makes sense for the original hash map to use namespaced keywords. If you have keys that "belong to" or are "associated with" some specific concept and/or subsystem within your app, then namespaced keywords mean you can talk about those things everywhere within your app without any conflicts, and also see where the data comes from or is going to.#2022-10-3019:39seancorfieldUnqualified keywords are fine when there is no possibility of a conflict and/or there is no inherent "domain" associated with the key. I'd say, for a web app that starts up a server on a given port, listening to a specific host, then it's fine to use :port and :host there. If you mean port/host for specific services you access from within your app, it's probably better to qualify them unless there's a strong reason not to. For example, interacting with a database via next.jdbc is going to require unqualified keywords for :port, :host, etc so it might make sense to have :db/config as a key in your overall config whose value is a hash map with :port, :host, etc. Keywords don't need to match namespaces. At work we use qualifiers like wsbilling and wsmessaging to identify data belonging to those subsystems as high-level keys (but we also have a mix of full namespace qualified keys and unqualified keys across the apps for various contexts).#2022-10-3019:44jmmk> Keywords don't need to match namespaces. At work we use qualifiers like wsbilling and wsmessaging to identify data belonging to those subsystems as high-level keys (but we also have a mix of full namespace qualified keys and unqualified keys across the apps for various contexts). How do you use those non-namespaced keywords with specs?#2022-10-3019:54seancorfield(s/def :wsbilling/id int?) and then (s/def :wsbilling/transaction (s/keys :req [:wsbilling/id ..])) -- is that what you're asking?#2022-10-3020:37Christian JohansenTo use non-namespaced keys with spec you use :req-un and :opt-un#2022-10-3022:05jmmk> Keywords donā€™t need to match namespaces It didnā€™t click until I saw your example that specs can use arbitrary namespaces @U04V70XH6 @U9MKYDN4Q thanks for all the advice. I am getting back into a clojure project I wrote many years ago as a pile of spaghetti and trying to impose some order on it and use spec instead of my home-grown data validation/coercion#2022-11-0421:13jmmkI have read various comments/advice on where spec fits into the workflows of producing/consuming data, but Iā€™m still not exactly clear on when and where ā€œparsingā€ / ā€œcoercionā€ should happen vs ā€œvalidationā€ / ā€œconformingā€, and specā€™s ability to parse or ā€œdestructureā€ things (like a binary format for example) seems to blur the lines a little bit. Iā€™m also confused where tools like https://github.com/exoscale/coax come in Example: I request/receive some data payload and convert it into an EDN structure more or less matching the shape of the raw payload (e.g. json deserialization) I only care about a subset of the fields in the payload, and I need to: ā€¢ ensure the right fields are present ā€¢ pull them out of the source data, parse them, and put them into my applicationā€™s representation Which parts is spec appropriate for? Would it make sense to use multiple specs to check both the raw data and my application representation? Specifically regarding parsing, should a spec pull apart ā€¢ string value containing delimited values like v1/v2/v3 ā€¢ date string mm/dd/yyyy or should those use normal string/date handling functions to parse them, and leave the spec as a simple regex to ensure the format looks correct (I suspect the answer is ā€œit dependsā€, but Iā€™m not sure where to draw the line)#2022-11-0501:21jmmkI think I have misunderstood the sequence specs and thought I could apply them to strings. However, the v1/v2/v3 really is a sequence but in string format, which again brings up the question of when the parsing should happen.#2022-11-0502:09jmmkIt feels like the spec validation is not very useful before parsing because parsing implicitly includes validation Iā€™m leaning towards: ā€¢ parse input data into internal structure, parsing/converting fields as needed ā€¢ write spec against the internal structure. validate transformed data before passing it among internal functions I still donā€™t see the use case for the coercion libs#2022-11-0502:39jmmkAfter reading https://groups.google.com/g/clojure/c/Tdb3ksDeVnU/m/ryc28SteAgAJ and I consider how the fully qualified keywords work in spec, itā€™s a bit clearer, but I still donā€™t have a fully-formed idea of how to move between two data types with the help of spec/coax#2022-11-0515:50mpenetCoax is mostly used to convert from json to edn for us, either from http requests or inside db fields. In our case we cannot serialize as edn (or transit/fressian&co) because other languages have to be able to use this data as well. It does no validation tho, it will coerce if it can/has-to otherwise it leaves values as is.#2022-11-0515:50mpenetIt never calls valid? or conform (there is a helper fn that combines both)#2022-11-0515:51mpenetIirc it uses almost nothing from spec internally, and thatā€™s intentional. Itā€™s strictly a transformation pass infered from a spec form/coax-registry#2022-11-0515:52mpenetUsually if you want/need validation that comes after#2022-11-0515:59mpenetA common (imho) mistake I often see in the wild is trying to tie your specs with coercion, using s/and and conformers. That turns things into franken-specs quickly, makes conforming all kind of broken when you need it and basically close the doors to have multiple coercion strategies for a spec depending on usage context#2022-11-0516:00mpenetThe way spec is designed, doing coercion has to be a separate pass imo#2022-11-0516:02mpenetSome other validation engines/libs have different strategies. But that has to be thought beforehand, allowing to compose operations when values are ā€œvisitedā€.#2022-11-0516:07mpenetPersonally I quite like the spec+coax combo, I might be biased (I wrote coax), but we use it heavily at work and itā€™s been quite frictionless so far#2022-11-0516:50jmmk@U050SC7SV so a coax pass will run and transform some fields to match the desired spec before the data is ever used?#2022-11-0516:52mpenetYes#2022-11-0516:53mpenetItā€™s a necessary evil in our case#2022-11-0516:54mpenetIf you can use transit/edn etc itā€™s better#2022-11-0517:13jmmkIā€™m dealing with multiple third-party APIs which makes this more interesting because the data format and structure is not easily usable without large transformations I currently do mostly ā€œmanualā€ parsing, but Im trying to simplify it and add spec into the equation where appropriate#2022-11-0519:54jmmk@U050SC7SV do you use ā€œreverseā€ coercions to go the opposite direction, i.e. get data back into in the format necessary to interact with the other system I guess itā€™s really just another spec with a different qualifier e.g. :application/field might go from string to int and :other-system/field might go from int to string#2022-11-0519:55mpenetYes in one particular case. Itā€™s one of the good things with coax. Coercion rules are first class#2022-11-0519:56mpenetYou can have as many as you want per spec. Itā€™s an argument to coerce calls basically#2022-11-0519:57mpenetSo same spec. Different coercers#2022-11-0612:09walterl@U08FV2128 I have the same question, so rather than providing an outright answer, I'll report on what I'm currently trying. I validate user input from the HTTP API with spec A, coerce it to my app-internal representation manually (will have a look at coax), and validate the app-internal data in the rest of my app with spec B. So, e.g., JSON input field :quantity is validated with (s/def :order.input/quantity (s/and string? #(re-matches #"^\d+$" %))), and coerced into field :order/quantity, validated with (s/def :order/quantity nat-int?). This models the two "sides" of the HTTP API interface with separate specs: User ā†[spec A]ā†’ API ā†[spec B]ā†’ Rest of the system I have no idea if this is The Right Way, but it works well, while being fairly verbose, even a little tedious.#2022-11-0612:30jmmk@UJY23QLS1 that sounds similar to what I have come up with so far.#2022-11-0611:35eighttrigramsHey, I've got a data structure (which I cannot change) like
[{:identifier "abc" :other-key-1 "abc"}
 {:identifier "SELF" :content {}}
 {:identifier "def" :other-key-2 []}]
I would like to do the following: 1. Determine if there is exactly one item in the vector of maps which has the :identifier "SELF". 2. For that item, determine if there exists the key :content 3. Make sure its value is a map (whose properties I also would like to spec) Can anyone think of a good way of doing this in spec or give me pointers of how to think about this?
#2022-11-0612:34walterl
(require '[clojure.spec.alpha :as s])

(def d [{:identifier "abc" :other-key-1 "abc"}
        {:identifier "SELF" :content {}}
        {:identifier "def" :other-key-2 []}])

(defn- self-id? [{:keys [identifier]}] (= "SELF" identifier))

(s/def :self/identifier #{"SELF"})
(s/def :self/content (s/keys)) ; TBD
(s/def ::self (s/keys :req-un [:self/identifier :self/content]))
(s/def ::not-self (s/or :!m (complement map?) :!self (complement self-id?)))

(s/explain-data ::self (first (filter self-id? d)))

(s/def ::d (s/and (s/coll-of map?)
                  (s/cat :before (s/* ::not-self)
                         :self ::self
                         :after (s/* ::not-self))))

(s/explain-data ::d d)
(s/conform ::d d)
; {:before [[:!self {:identifier "abc", :other-key-1 "abc"}]],
;  :self {:identifier "SELF", :content {}},
;  :after [[:!self {:identifier "def", :other-key-2 []}]]}
#2022-11-0614:42eighttrigramsThis is fantastic! Thank you so much, @UJY23QLS1#2022-11-0614:44walterl> "...pointers of how to think about this" Thing of it like regex: you want to match 0 or more non-self maps, followed by a self map, followed by 0 or more non-self maps. (An s/not spec would've been handy here.)#2022-11-0614:47eighttrigramsYeah, like a regex. It totally makes sense. The docs even stated this, but just didn't come up with this in the moment.#2022-11-0614:47eighttrigrams> An s/not spec would've been handy here. :thinking_face: Interesting. I see, for the :not-self ...#2022-11-0614:51walterlYeah, I would've liked to do (s/not ::self) rather than having to define a negation of ::self manually, because now they could drift out of sync.#2022-11-0614:54eighttrigramsYes. I have an intuition that it should generally be possible to make such a thing. One would need to account for the different, uhm, don't know the name, the different things like s/keys , s/cat and so on. The negation of the s/keys was an s/or with complement. Not in the mood to solve that puzzle right now, but certainly an interesting challenge šŸ˜€#2022-11-0912:34prncWould anyone be able to share a way / ā€˜workaroundā€™ for of adding docs to specs? Has anyone gone beyond a map keyword -> docstring? ā€” which is what Iā€™m about to do, so asking here for some lessons learnt :) (Iā€™m assuming this must have been done / attempted a number of times (since it is the most requested feature for spec2 IIUC but I have not run into any specific examples, caveats or advice))#2022-11-0912:48mpenetessentially: bake your own metadata registry#2022-11-0912:48mpenetthere's nothing built-in#2022-11-0914:25Alex Miller (Clojure team)this is one of the most highly voted requests we have and it's still on our list for spec 2#2022-11-1019:11Ishaan Guptawhat's up with this spec-alpha2 behaviour?#2022-11-1019:27Alex Miller (Clojure team)seems like it's whether it evals to a var or not so probably a bug#2022-11-1019:27Alex Miller (Clojure team)if you can drop it on http://ask.clojure.org I'll make a ticket and take a look at some point#2022-11-1113:43jmmkIs there a better way to accomplish this? I find in each place I am using conform, I want it to fail (throw) if it is invalid
(defn conform! [spec v]
  (let [conformed-value (s/conform spec v)]
    (if (s/invalid? conformed-value)
      (throw
        (ex-info
          "value did not conform to spec"
          (s/explain-data spec v)))
      conformed-value)))
#2022-11-1114:00Alex Miller (Clojure team)Looks good to me#2022-11-2408:59ChrisIs anyone aware of any projects that convert JSON-schema to specs? An internet search only turns up projects based on Java libs, but it would be preferable to have a spec created. (Most preferable would be to create the schema from a spec, but probably a much harder problem given the relative expressiveness!)#2022-11-2409:35tatutI have some old code (not up to current json schema) https://github.com/tatut/json-schema you can use that as a predicate#2022-11-2409:35tatutbut it doesnā€™t create clojure specs EDIT: nevermind, you wanted schema from specs#2022-11-2410:19ChrisI wanted either šŸ™‚ This is neat, thanks for sharing. Might be a good base for a ->spec version if I canā€™t find an existing implementation. #2022-11-2417:03jmayaalvthere is also spec-tools https://github.com/metosin/spec-tools/blob/master/docs/04_json_schema.md#2022-11-2417:28Chris Exactly what I wanted, thank you!#2022-11-2916:51Ben LiebermanWhat is the most idiomatic way to spec "if this key exists in a map, that key should not exist?" I'm finding this a bit more confusing than I initially expected. A short example:
;; map with either short or long name but not both
(def a-map {:short-name "foo"}) ; should be valid
(def a-bad-map {:short-name "foo" :long-name "foobar"}) ; should be invalid
I've tried various combinations of s/and , not and other core fn's without luck.
#2022-11-2916:59Ben Liebermanmy latest try, for example, blows up the stack:
(s/def ::short-route-name (s/and string? #(s/invalid? (s/conform ::long-route-name %))))
(s/def ::long-route-name (s/and string? #(s/invalid? (s/conform ::short-route-name %))))
I also tried using or but that fails too.
#2022-11-2917:05pavlosmelissinosspec favors open maps by design; it doesn't really support statements like "this key shouldn't exist"#2022-11-2917:06jkxyzYou can use a predicate like #(not (and (:foo %) (:bar %))) #2022-11-2917:07Ben LiebermanI saw that Malli has both an enum spec and an option to close maps but I want to learn spec before I give Malli a try, even though both those features would be useful to me here#2022-11-2917:11pavlosmelissinosThere are also third party libraries, like https://github.com/bhauman/spell-spec that "close" specs if you really want that functionality.#2022-11-2917:12pavlosmelissinosI wouldn't try to model "negative" specs from within spec, it's not designed for that.#2022-11-2917:13Ben LiebermanWell I guess I feel slightly less silly then for finding this pretty non-obvious#2022-11-2917:14pavlosmelissinosAn interesting approach is that followed by cognitect's aws-api; instead of saying "this is what a valid response doesn't look like", it specs anomalies: https://github.com/cognitect-labs/anomalies#2022-11-2917:18Ben LiebermanI'll have to check that out, thanks @UEQPKG7HQ#2022-12-2113:53vemvhttps://clojure.org/guides/spec#_entity_maps says: > Also note that ALL attributes are checked via keys, not just those listed in the :req and :opt keys. I'm curious, does this behavior also exist in spec2? (I never invested time in spec2 at all as it's still wip)#2022-12-2113:55vemvThe broader question that prompted me to ask this is: is such behavior a universally good idea? I have no opinion from my side. I don't recall it ever bothering me. Perhaps there's the performance/security consideration of checking unforeseen keys? I wouldn't mind about that either given that I can remove extraneous keys at the edges (with some or other approach - there are a few)#2022-12-2113:58Alex Miller (Clojure team)yes, it exists, although s/keys is really deprecated in favor of schema/select in spec 2#2022-12-2113:59Alex Miller (Clojure team)but there is also the notion of closed spec checking which limits things somewhat (and plumbing to add other such options during checking)#2022-12-2114:02vemv> yes, it exists, although s/keys is really deprecated in favor of schema/select in spec 2 So it also exists when using select in a vanilla fashion, right?#2022-12-2115:20Alex Miller (Clojure team)I think so? can't remember#2022-12-2316:07valerauko
(s/coll-of ::thing
           :kind vector?
           :min-count 1
           :distinct true)
why does this cause a boxed math warning? (unchecked inc)
#2022-12-2413:05Ben SlessListening to Alex's answers on spec2 on the Clojure Stream on modeling relationships between functions' inputs and outputs, have you considered using a relational language? E.g. datalog?#2023-01-1015:45Ben LiebermanIn the (presumably unlikely) case where test/check does not find any failures in a function I've spec'ed, what is the expected output? e.g., I have a function foo and I check it like so
(->> (test/check 'foo) test/summarize-results)
In my case I'm just getting back a map of {:total 0} which seems suspect to me given the docs for these functions.
#2023-01-1015:48Alex Miller (Clojure team)maybe `foo instead of 'foo ?#2023-01-1015:49Alex Miller (Clojure team)(that is, a resolved symbol name)#2023-01-1015:50Alex Miller (Clojure team)I suspect it's not finding foo and thus not testing anything, see the guide too https://clojure.org/guides/spec#_testing#2023-01-1015:50Ben Liebermanoh, right. I forgot about that syntax and thought the ` was just an italicized '#2023-01-1015:51Ben LiebermanThanks!#2023-01-2019:18Colin P. HillDoes anyone have any usage examples where s/and's chained conforming behavior was useful? I find I almost always have to work around it with a lambda or s/nonconforming because the functions and specs I'm composing the s/and from don't expect conformed input, and that makes me wonder if I'm missing something about the intended usage patterns.#2023-01-2019:34colinkahnConceptually the same, clojure.spec.alpha itself uses that for keys* using &- https://github.com/clojure/spec.alpha/blob/master/src/main/clojure/clojure/spec/alpha.clj#L1810#2023-01-2020:00Alex Miller (Clojure team)I find it is often useful when chaining predicates. the main thing that makes it not useful most often are when you put something in it that gives you a conformed value that differs from the original (particularly s/or). I'd be curious what the most common cases are for you when it's an issue. in spec 2, there is a non-flowing s/and- (name subject to change)#2023-01-2021:03Colin P. HillI'll dig around in my code in a minute to see if I can find a succinct and reasonable example ā€“ my current project isn't so much a useful product as it is a playground to get a feel for what can be done with specs beyond simple validation, so most of the stuff in my immediate reach is kinda ramshackle and/or at the edges of what is intended. But yeah, s/or is the main culprit. Non-transforming conformations aren't an issue ā€“ if every spec in the chain simply returned its argument, s/and would be indistinguishable from its non-flowing variant.#2023-01-2021:07Colin P. HillBut also, non-transforming conformations are less interesting. Conformation is one of the main things I'm trying to get a feel for right now. I found the Jira tickets where you discouraged one of the spec-tools maintainers from viewing specs as a coercion engine, so atm I'm trying to explore what kinds of transformations are supported, and how they can be used.#2023-01-2021:12Alex Miller (Clojure team)I think itā€™s likely that we will add a nonconforming or option - thatā€™s the most common need. Mostly wondering if thereā€™s anything else#2023-01-2021:16colinkahn> Conformation is one of the main things I'm trying to get a feel for right now. From various talks I've interpreted conformation being suggested in two places: ā€¢ To make use of existing specs (like the keys* example conforms the tuples into a map for the existing keys spec). ā€¢ In macros, where you're parsing at compile time for syntax reasons.#2023-01-2021:17colinkahn(I think those were probably talks by @U064X3EF3 so I'm sure he could say if those are accurate or not šŸ˜„)#2023-01-2021:20Colin P. Hill> Mostly wondering if thereā€™s anything else Probably not. Maybe alt, being the regex equivalent of the same. The other transformers don't really make things more deeply nested than they were before, so they're not usually an inconvenience. The case that prompted the question is a little unusual ā€“ I have a multi-spec in an and, so while there's a contract on what that multi-spec is supposed to represent, I can't necessarily be sure how it will conform (and since conformation output can be non-obvious I don't necessarily want to include that in the contract and count on it being obeyed). Easy solution in nonconforming, but it got me thinking that I may not rightly understand why and is designed the way it is and might not be using the tool in the best way.#2023-01-2021:26Alex Miller (Clojure team)the purpose of conforming is to tell you why and how a value matched a spec#2023-01-2021:26Colin P. Hill> In macros, where you're parsing at compile time for syntax reasons. Parsing is kinda what I've been fixing on for my current understanding (not just macros, could also be for data-oriented APIs). It seemed like standard conformations all aimed to describe the implicit structure of their input while generally leaving the scalars at the leaves of the tree alone ā€“ exactly what a parser does with a stream of tokens, just here with potentially more initial structure than just a stream.#2023-01-2021:27Alex Miller (Clojure team)the "why" part is why s/or and s/alt tell you which path was taken, why s/cat separates it's concatenated parts, etc#2023-01-2021:28Alex Miller (Clojure team)s/and's flowing is extremely useful when chaining predicates (where typically the value is not not modified, just passed through) ala (s/and int? pos? even?)#2023-01-2021:31Colin P. HillHow does that differ from how the non-flowing variant will behave? I thought conforming with a predicate function was essentially an identity operation, so here even? receives the same value that pos? did#2023-01-2021:32Alex Miller (Clojure team)https://insideclojure.org/2019/12/06/journal/#2023-01-2021:33Alex Miller (Clojure team)the original value (not the conformed value) is passed through s/and-#2023-01-2021:35Alex Miller (Clojure team)for specs that are pred fns, doesn't affect conform (but does affect gen)#2023-01-2021:35Colin P. HillOh interesting. I'm not immediately seeing how it affects gen, could you say more on that?#2023-01-2021:36Alex Miller (Clojure team)it's explained in that blog#2023-01-2021:36Colin P. HillAh sorry, skimmed poorly#2023-01-2021:36Alex Miller (Clojure team)well s/and- is - it only gens from the first predicate. s/and gens, then filters#2023-01-2021:37Alex Miller (Clojure team)I expect you will often need a custom generator with s/and-#2023-01-2021:41Colin P. HillDoes this mean that and- will be an exception to the general rule that a spec's generator retries generation until it gets something that conforms to the (whole) spec? Or does and do some more proactive filtering ahead of that step with its 2nd through last terms?#2023-01-2021:51Alex Miller (Clojure team)I don't think it's an exception - they will all try 100 times and all spec generators check s/valid? on the generated values#2023-01-2021:51Alex Miller (Clojure team)s/and-, by not using anything but the original pred, is less likely to generate valid values though#2023-01-2021:53Colin P. HillGot it, that makes sense. I wasn't aware that the original flavor built its underlying pre-verification generator out of more than one of its terms#2023-01-2021:54Colin P. HillI'll have to read the implementation at some point, because it's not immediately obvious to me how that could even be done :thinking_face:#2023-01-2021:54Alex Miller (Clojure team)generate values from the first pred. only keep ones that satisfy 2nd pred. only keep those that satisfy 3rd pred, etc.#2023-01-2021:55Colin P. HillJust a chain of such-thats so the generation tooling handles filtering before the spec does its final check?#2023-01-2021:55Alex Miller (Clojure team)yes, so it helps to think about the ordering of preds in s/and#2023-01-2021:56Alex Miller (Clojure team)this is covered in https://clojure.org/guides/spec#_using_sand_generators#2023-01-2021:56Colin P. Hillack, my poor skimming embarrasses me again#2023-01-2021:56Alex Miller (Clojure team)I forgive you, there is a lot to read :)#2023-01-2511:06BenjaminHi, I would like to extract a spec from a function spec (`s/fdef`). I realized since the spec is speccing an arglist this might be a bit challenging. Right now I wanted to extract the 2nd arg spec always. Context: https://github.com/WorksHub/leona/issues/4#2023-01-2511:10Benjamin
(defn droid-resolver [ctx query value])

(s/def ::int int?)
(s/def ::string string?)
(s/def ::any any?)

(s/fdef droid-resolver
  :args (s/cat :a ::int :b ::string :c ::any)
  :ret ::int)

(let [[_ _ _ _ b-spec]
      (s/form (:args (s/get-spec #'droid-resolver)))]
  b-spec)
:leona.core-test/string
#2023-01-2511:11Benjaminthis would not fly would it? It would fail if someone defines a spec differently than cat ?#2023-01-2513:13Alex Miller (Clojure team)Yes, it would fail if that spec changed#2023-01-2513:14Alex Miller (Clojure team)Thereā€™s not really a way to get the spec of a specific arg like that#2023-01-2513:19BenjaminOk thanks!#2023-02-0117:59Ben Liebermanquestion about resolving specs: is there a way to require a spec out of the global registry as if it were a regular var in a ns? I have discovered that I can't do something like (require [foospec.bar :refer [baz]]) presumably because baz is not interned in the foospec.bar ns but is instead in the registry? Is that correct?#2023-02-0118:13Alex Miller (Clojure team)correct, specs are not vars#2023-02-0118:14Alex Miller (Clojure team)you could put a spec in a var with (def my-spec (s/get-spec ::foo)) if you wanted to for some reason#2023-02-0118:15Ben Liebermanno, not necessary, I'm fine with it being in the registry. So if require the whole foospec.bar ns say as fspec and then resolve ::fspec/baz its just looking in the registry for that fully qualified name?#2023-02-0118:16Ben Liebermanthat's how I have been doing it at any rate#2023-02-0118:16Alex Miller (Clojure team)not sure I understand "require the whole foospec.bar ns say as fspec " but fspecs are also just specs in the registry (but their keys are fully-qualified symbols, not keywords)#2023-02-0118:18Alex Miller (Clojure team)s/fdef just creates a spec with s/fspec, then adds it to the registry (which is just a map) with the fq symbol as the key#2023-02-0118:18Ben Liebermansorry, that was a poor choice of alias on my part#2023-02-0118:18Ben LiebermanI am not talking about function specs, just any spec in general#2023-02-0118:19Alex Miller (Clojure team)then yes, that is how it works#2023-02-0120:16seancorfield@U03QBKTVA0N Don't forget there's :as-alias now in a require so you can create an alias for resolving Spec names without having to actually load a real namespace.#2023-02-0120:18seancorfield
(ns using.specs
  (:require [i.do-not.exist :as-alias idne]))

::idne/bar ;=> i.do-not.exist/bar
and no i/do_not/exist.clj file is needed for this.
#2023-02-0123:35Ben Liebermanoh thanks @U04V70XH6 I did not know about as-alias. That's helpful.#2023-02-0709:13djtangohello I seem to remember that there was a spec for Clojure (ie someone wrote spec to describe the clojure syntax). Did I imagine this? I'm struggling to google for it#2023-02-0709:15dgb23you might be thinking of this: https://github.com/clojure/core.specs.alpha#2023-02-0709:16djtangothank you - this is the one šŸ™#2023-02-1518:58stopaHey team, noob question. Consider the following object:
(s/def ::type #{:foo :bar}) 
(s/def ::obj (s/keys :req [::type ::a ::b] :opt [::c]))
How would I express something like: If ::type is :foo, ::c is required, but otherwise it's not?
#2023-02-1519:00Alex Miller (Clojure team)you can s/and a function that tests any property you like#2023-02-1519:00Alex Miller (Clojure team)but you might also want to look at s/multi-spec https://clojure.org/guides/spec#_multi_spec#2023-02-1519:01stopaNice! Thanks @U064X3EF3!#2023-02-1603:18stopaHey @U064X3EF3 one more question:
(s/def ::type #{:biz :baz})
(s/def ::foo string?)
(s/def ::bar string?)
(s/def ::kux string?)
(s/def ::common (s/keys :req [::type ::foo ::bar]))
(s/def ::biz-obj ::common)
(s/def ::baz-obj
  (s/merge ::common
           (s/keys :req [::kux])))

(defmulti type-mm ::type)
(defmethod type-mm :biz [_] ::biz-obj)
(defmethod type-mm :baz [_] ::baz-obj)

(s/def ::obj (s/multi-spec type-mm ::type))

(gen/generate (s/gen ::obj)))
Here I make it so if the type is :baz, then ::kux is required. But one thing I did want to do: Sometimes I want to generate the different instances of the objects -- i.e ::biz-obj and ::baz-obj But the way I wrote it, I would get an invariant. If I wrote:
(gen/generate (s/gen ::biz-obj)))
I would sometimes get values with ::type :baz This is because I don't constraint the type in this layer. I guess I have two questions: Is this the right approach? And if so, how would I go about properly constructing ::biz-obj and ::baz-obj?
#2023-02-1617:29Alex Miller (Clojure team)well you could constrain the type in those layers#2023-02-1617:29Alex Miller (Clojure team)or make a generator that inserted the expected type#2023-02-1617:29Alex Miller (Clojure team)using gen/fmap#2023-02-1815:11stopaOne question for my learning: How would I constrain the type in those layers?
(s/def ::biz-obj (s/keys :req [::type ::foo ::bar]))

(s/def ::baz-obj
  (s/merge ::biz-obj
           (s/keys :req [::kux])))
Since these are both s/keys, and they both must take ::type as one of the validations, how would I say "for this map, ::type is just this part"? Or did you have something else in mind?
#2023-02-2117:16stopaA noob naming question, that I guess doesn't really have an answer, but I thought I'd try anyways. Consider:
(s/def ::triple (s/cat :e string? :a string? :v string?))
Now I may call this vec a triple:
["e", "a", "v"]
But what should I call the (s/conform ::triple) version of it?
{:e "e", :a "a", :v "v"}
I am thinking parsed-triple or conformed-triple, but I wonder if there's a current idiom. If I have some functions that work on one type vs another, I worry about wonky names.
#2023-02-2117:33delaguardonamed-triple#2023-02-2117:33delaguardoor just record#2023-02-2117:33nbardiukhttps://clojure.org/guides/spec#_collections mentions Conforms to map with named keys based on the cat tags . I would add more naming options like tagged-triple or named-triple . Python has named tuple which looks similar#2023-02-2117:34stopaOo nice idea! Thanks team#2023-02-2117:36Ben SlessNamed tuple#2023-02-2208:42serioganamed tuple may be confusing as far as element order is not garanteed#2023-02-2209:15delaguardoWhy should order matter after conforming with the spec?#2023-02-2209:19dgb23If you're already using tripple as a name, then name it tagged or named tripple as well instead of tuple IMO#2023-02-2209:19dgb23tripple is more precise than tuple#2023-02-2209:20seriogaThe word tuple means ā€œfinite ordered listā€. https://en.wikipedia.org/wiki/Tuple It can be vague that named predicate drops the ā€œorderedā€ characteristic. :-)#2023-02-2209:21dgb23From the same article: > Many programming languages offer an alternative to tuples, known as https://en.wikipedia.org/wiki/Record_(computer_science), featuring unordered elements accessed by label.#2023-02-2209:23seriogabtw python's named tuples are ordered and support access by index.#2023-02-2209:23dgb23record tripple or just record seem to be good names šŸ™‚#2023-02-2209:25delaguardoIt is even matching with clojure.core/record šŸ™‚#2023-02-2414:30pavlosmelissinosGiven an incomplete map and its spec, can I get spec to give me back the list of missing keys (required and optional)? With explain-data I can get the required keys but what about optional ones? Context: I'm playing with the idea of (dynamic) spec-driven config presets, i.e. there's a process that creates named incomplete maps and then another process/user selects a preset and fills-in the gaps.#2023-02-2818:12stopaHey team! I am making a graphql-like query language, that looks something like this:
{"users" {"$" {"where" {"bookshelves.books.title" "The Count of Monte Cristo"}}
          "bookshelves" {}}
 "books" {}}
Here, I have forms, which have an option-map , and further children. What I would want here is a combination of keys (to say what $ should be), and map-of, to describe that it can have children. I wasn't sure how to do this, so the best approach I could think of, was to massage the data into a kind of vector, like:
[["users", {"where" {"bookshelves.books.title" "The Count of Monte Cristo"}}, ["bookshelves", {}, []]], 
 ["books", {}, []]]
So then I could type this vec, like so:
(s/def ::where (s/map-of string? ::triple-model/value))

(s/def ::option-map (s/keys :opt-un [::where]))

(s/def ::form (s/cat
               :k string?
               :option-map ::option-map
               :children (s/coll-of ::inner-form)))

(s/def ::forms (s/coll-of ::form))
Is this the idiomatic way to do things with spec, or would you do it differently?
#2023-03-0819:31phronmophobic> I wasn't sure how to do this, so the best approach I could think of, was to massage the data into a kind of vector, like: I'm not totally sure I understand what you're trying to do, but massaging data before validation sounds like an anti-pattern.#2023-03-0819:49phronmophobicmaybe something like:
(s/def ::options (s/keys))
(s/def ::form
  (s/coll-of
   (s/or
    :options (s/tuple #{"$"}
                      ::options)
    :sub-form (s/tuple string? ::form))
   :into {}))
#2023-04-0715:01stopa(realized I never responded -- I think this could work! Thank you @U7RJTCH6J)#2023-03-1718:22Joerg SchmueckerHi, Is there a project that shows best practices for spec? Not a sample but some lib that is actually used. Thanks#2023-03-1719:11seancorfieldSpec can be used in a lot of different ways so I wouldn't expect any single project to be a good exemplar of it... Is there some specific aspect of Spec that you're seeking guidance on?#2023-03-1720:38Joerg SchmueckerJust basics, how do I spec parameters on a function best. What are typical project setups, i.e. are the specs in the code or the tests. Do you instrument specs in prodcution or not (or when)? Just all the things that make the idea workable.#2023-03-1720:39Joerg SchmueckerI think any well ā€œspec-dā€ code base woudl be a good starting point. I just donā€™t know how to google for one.#2023-03-1720:54phronmophobicThose are all good questions. https://grep.app is pretty good for finding code in the wild, https://grep.app/search?q=s/def.#2023-03-1720:55Joerg SchmueckerThank you,. I will give that a try.#2023-03-1720:55seancorfieldhttps://corfield.org/blog/2019/09/13/using-spec/ talks about the various ways we use Spec at work. Happy to answer any follow-up Qs.#2023-03-1720:57Joerg SchmueckerSean, I did read your blog. Great content. For me it isnā€™t obvious how to get from the concept to the practical implementation.#2023-03-1720:58seancorfieldI suspect there's a lot more use of Spec in large application codebases and those are nearly all closed source.#2023-03-1720:59Joerg SchmueckerI would love to see the Nubank code base šŸ™‚#2023-03-1721:00Joerg SchmueckerI work with a lot of FSI customer and I am trying to figure out if we can simplify some of those problems.#2023-03-1722:49dgb23> Do you instrument specs in prodcution or not (or when)? I personally wouldn't use instrumentation outside of development. If you use spec for API/request validation, or generally stuff that comes from the outside of your program, then you want to be somewhat rigorous with it and nail it down to sufficient detail. Otherwise I would say just use it when it helps, incrementally. For example it can be useful as a thinking tool, to design the shape of your data. Or you can use conform to parse things into a richer structure. But it's really just that, a tool. I played around with it a lot and looked at how people use it in OSS projects. It's very nice and composable, and you can do a lot with it. I think playing around with it is a good way to get familiar with what you can do. Then forget about it, just write code and it will introduce itself naturally.#2023-03-1722:55dgb23Here's are more involved specs (tools.deps): https://github.com/clojure/tools.deps/blob/master/src/main/clojure/clojure/tools/deps/specs.clj Specs can also be small and informal, basically just a set of names https://github.com/cognitect-labs/anomalies/blob/master/src/cognitect/anomalies.cljc#2023-03-1722:58seancorfieldAnother example from a library: https://github.com/seancorfield/next-jdbc/blob/develop/src/next/jdbc/specs.clj -- all about instrumenting library functions during development#2023-03-1723:16Joerg SchmueckerFantastic just what I needed. Thank you so much.#2023-03-1723:26dgb23> Generative scenario testing This is a very interesting section. I've been thinking about something similar in the back of my head for a while. I named them workflows, but I like "scenario" very much as a term here! > Deriving code from specs I've been struggling with something similar for quite a while and was never quite happy with my results. Have you been using form and conform to parse the specs themselves?#2023-03-1723:30seancorfieldI'm not in front of my work codebase -- I'm taking every Friday off this month -- but I'm pretty sure we use form and walk the result to find the s/keys definition and "parse" that to derive everything we need. If you remind me on Monday, I can provide more detail.#2023-03-1723:38dgb23Ah right, its a specific subset of possible specs that make sense in that context#2023-03-3014:03practicalli-johnhttps://practical.li/clojure/clojure-spec/ covers how I use clojure.spec along with practical examples and coding videos I've also recently been using spec with Reitit to validate API requests and responses (which I will be writing up this month)#2023-03-3105:46Hugh PowellHi folks, has anyone managed to generate Swagger 2 output for specs that places definitions in the :definition top level key, rather than in-lining them in the :parameters and :response keys inside :paths. It looks like https://github.com/metosin/ring-swagger#more-complete-example, but https://github.com/metosin/spec-tools/blob/master/docs/05_swagger.md#full-example https://github.com/metosin/spec-tools/blob/master/src/spec_tools/swagger/spec.cljc#L81.#2023-04-1216:17ikitommiInterested in PR to spec-tools to add support?#2023-04-1309:02Hugh PowellHere's my https://github.com/metosin/spec-tools/pull/274. Let me know what you think.#2023-04-2320:50karol.adamiecHi, I get the mysterious "Additional data must be non-nil." Below is a sample. What do i do wrong?
(defn mean
  [numbers]
  (when-let [num-count (seq numbers)]
    (/ (reduce + 0 num-count) (count num-count))))

(defn within-range?
  [avg coll]
  (if (empty? coll)
    true
    (let [min-val (apply min coll)
          max-val (apply max coll)]
      (<= min-val avg max-val))))

(defn avg-unchanged-by-order?
  [avg coll]
  (let [shuffled-coll (shuffle coll)
        shuffled-avg (mean shuffled-coll)]
    (= avg shuffled-avg)))

(s/fdef mean
  :args (s/cat :numbers (s/nilable (s/coll-of (s/double-in :NaN? false :infinite? false))))
  :ret  (s/nilable number?)
  :fn (fn [{:keys [args ret]}]
              (if-let [coll (:numbers args)]
                (and (within-range? ret coll)
                     (avg-unchanged-by-order? ret coll))
                true)))

(stest/check `mean)
#2023-04-2321:04phronmophobicThe docs suggest using stest/abbrev-result to get more info about what happened when it failed, https://clojure.org/guides/spec#_testing. Have you tried looking at that info?#2023-04-2321:13karol.adamiecif i do as per docs (stest/abbrev-result (first (stest/check mean`) i do not see any more relevant information. There is the sample case that "failed", but it is correct input and if i give it to mean or its validators/helpers everything seems fine :/#2023-04-2321:14phronmophobicWhat is the failing sample?#2023-04-2321:23karol.adamiecit is a bit random. sometimes it says spec failed and collection is [ 1.0000004768371582 1.000000238418579 2.147483648E9 ] and calculated mean is 7.158278833333336E8 other times there is nothing inside, other than the message in exception "additional data must not be null".#2023-04-2321:24karol.adamiecstacks point at explainInfo --- Contents: 0. clojure.spec.test.alpha$explain_check 1. invokeStatic 2. "alpha.clj" 3. 390#2023-04-2321:26karol.adamiechttps://clojurians-log.clojureverse.org/clojure-spec/2019-08-02#2023-04-2321:27karol.adamiecfound some old @U064X3EF3 explanations about similiar thing... but unsure if that still applies...#2023-04-2321:30phronmophobicone thing that is suspicious is that mean is called in the validation function. I would try to narrow down where the problem might be. Examples: ā€¢ run stest/check with just the :args specification ā€¢ run with just the :ret ā€¢ run with just :fn ā€¢ run with just :fn and only check within-range? ā€¢ etc #2023-04-2321:39karol.adamiecyeah, that mean call maybe sth... i changed the avg-unchanged-by-order? to be a dummy that always returns true and now 1000 tests pass...#2023-04-2321:40karol.adamiecthanks šŸ™‚#2023-04-2321:41phronmophobicI think thereā€™s a way to write those sorts of checks, but Iā€™m not sure how to do it off the top of my head.#2023-04-2321:43phronmophobicTheyā€™re definitely supported by test.check #2023-04-2321:43karol.adamieci would think calling function under test is ok... it is not like recursion...#2023-04-2321:46karol.adamiecyeah, all pass if i return true, but still do all the calcs and logic. so just calling mean seems to not be the issue#2023-04-2323:01phronmophobicAnother thing that just popped into my head is that your test expects the order of the numbers not to make a difference in the calculated mean, but that's not true for all floating point numbers!#2023-04-2323:02phronmophobichttps://stackoverflow.com/questions/21373865/different-results-when-adding-same-doubles-in-different-order#2023-04-2323:02phronmophobicnot sure why that didn't occur to me earlier :doh:#2023-04-2405:14karol.adamiecYou saved my bacon sir! Swapped the spec to be ints, not doubles, and behold it is rock solid now. That mysterious error pops up when the spec is non-deterministic, which Alex has said years ago more less. So a better error message would be nice šŸ™‚#2023-04-2406:03karol.adamiecSo a new question arises, what now? What would be best way to move forward? 1. drop the invariant (yikes...) 2. replace the generator in spec to gen ints, even though we say doubles... 3. use a tolerance delta in the invariant <= my favourite i think... but adds so much noise ;/#2023-04-2422:53phronmophobicIt depends on your goals. If you just want a function that calculates the mean of a sequence of numbers correctly, then I would use a library. However, there might be other goals (like learning spec) where you might do something different.#2023-04-2422:58phronmophobicThere are lots of fiddly trade offs when working with floats. Most libs that work with floats use a number of tricks like comparing within some tolerance for tests. Another trick is to pre sort the numbers before summing to improve accuracy and consistency. Iā€™m not an expert here which is why I would just use a library.
#2023-04-2506:21karol.adamiecyeah, the goal is to use spec to learn. i am not implementing new statistical package... yet :D#2023-04-2506:24karol.adamieci went with tolerance delta. The biggest takeaway is: if your spec is non-deterministic, you will get the mysterious exception mentioned above. Easy trap to fall into and burn, but once burned unlikely to happen again. And seeing that error immediately points in right direction. Typical newbie problem...#2023-04-2507:45phronmophobicI actually haven't used s/fdef . For these types of things, I usually go straight to https://clojure.org/guides/test_check_beginner. The nice thing is that you can create separate tests for each property (eg. within-range, avg-unchanged-by-order?) and run them independently for however long it takes to feel comfortable that your implementation works. For avg-unchanged-order?, doing one shuffle only tests a small part of the test space. I would probably write it to use https://github.com/clojure/math.combinatorics to test a broader part of the input space.#2023-04-2507:50phronmophobicFor test.check, there's a way to get the RNG seed to make tests consistent and reproducible. There's probably also a way to do that with s/fdef so that avg-unchanged-order? is consistent/reproducible and avoids the problem you were running into.#2023-04-2507:56karol.adamiecThanks for ideas. Will think on that. One point intrigued me especially. You say: _I actually haven't used `s/fdef` . For these types of things, I usually go straight to <https://clojure.org/guides/test_check_beginner|test.check>. The nice thing is that you can create separate tests for each property (eg. within-range, avg-unchanged-by-order?) and run them independently for however long it takes to feel comfortable that your implementation works._ One could add `s/fdef` to helpers as well too. It seems to mee then, that desired test "pyramid" would be: 1. s/fdef for most important and public facing parts 2. test.check for these as well, 3. exaple based classic unit tests, to present few cases to readers. It gets hard to get a glance of "shapes" involved without examples.#2023-04-2507:59karol.adamiecwill try to roll with this and see how my perceptions change šŸ˜„#2023-04-2508:00karol.adamieci do feel there is this a bit awkward dichotomy with spec/gen and test.check/gen i.e. some things are wrapped, but sooner or later (rather sooner) one HAS TO land in test.check land and learn it good.#2023-04-2508:01karol.adamiecOTOH, using test.check for double generator, where there is a nice s/double-in in spec feels wrong... so i end up doing things twice... and have extra pass => can i do that using spec namespace only...#2023-04-2508:03phronmophobicI still use stuff like s/def and s/double-in. You can get the generator to use with test.check via s/gen.#2023-04-2508:06karol.adamiecin official docs they do mention that one should use test.check as last order of biz. as that will force that dependency on prod build. So it seems like the spirit of what i should do is try to avoid test.check in my prod namespaces, and reach for that only in testing namespaces. Although i have to say, i do not really follow the argument there, as spec depends on it anyway... so who cares whether i include that explicit or it is sucked in in deps of deps...#2023-04-2508:08phronmophobicspec dynamically loads it so you can load spec without test.check#2023-04-2508:11karol.adamiecyes it does. so i do not fully understand this: There are three ways to build up custom generators - in decreasing order of preference: 1. Let spec create a generator based on a predicate/spec 2. Create your own generator from the tools in clojure.spec.gen.alpha 3. Use test.check or other test.check compatible libraries (like https://github.com/gfredericks/test.chuck) The last option requires a runtime dependency on test.check so the first two options are strongly preferred over using test.check directly.#2023-04-2508:12karol.adamiectest.check bytecode ends up in my prod slim and shiny build anyway... What is the perspective here that i miss? Maybe some more intricate interactions when designing libs based on libs...#2023-04-2508:13karol.adamiecOr just a rule of thumb to make code as future proof as possible as this will be the direction?#2023-04-2508:13phronmophobicI don't think test.check should show up unless you explicitly include it.#2023-04-2508:14phronmophobicmaybe some other dependency is pulling it in?#2023-04-2508:17karol.adamiechmm... i might have misunderstood the wording then. i assumed that spec is using test.check namespace internally#2023-04-2508:18karol.adamieci have not inspected the build image btw, just to be clear#2023-04-2508:19karol.adamiecbut what you say makes more sense for sure šŸ™‚#2023-04-2508:19karol.adamiecand clears my confusion, so this is 99% correct i think#2023-04-2508:19phronmophobicIt does use test.check , but only for functionality starting at generators and later, https://clojure.org/guides/spec#_generators#2023-04-2508:20phronmophobicwhich you won't usually need at runtime#2023-04-2508:20karol.adamiecspec generators rely on the Clojure property testing library https://github.com/clojure/test.check. However, this dependency is dynamically loaded#2023-04-2508:20karol.adamiecyes, to that was the part that set my mind on it#2023-04-2508:21karol.adamiecso, i hack away happily using spec namespace, but ... it will load in test.check at some point even in prod. even without me saying it has to#2023-04-2508:21karol.adamiecso we go back to the 1.2.3. above. why would that matter?#2023-04-2508:22karol.adamiecanyway. i can live with not knowing for now. Thank You for help šŸ™‚#2023-04-2508:23phronmophobicIf your prod code doesn't use generators or things that need them, then you don't need test.check as a prod dependency and it won't be included#2023-04-2508:24phronmophobichttps://github.com/clojure/spec.alpha/blob/ad06cdc7407c11990c7e93206133fb14eb62cacf/src/main/clojure/clojure/spec/gen/alpha.clj#L55#2023-04-2508:25phronmophobicThat's what all the delay and dynaload stuff is doing.#2023-04-2508:27phronmophobicand if test.check isn't included and you try to use generators, you'll get an exception, https://github.com/clojure/spec.alpha/blob/ad06cdc7407c11990c7e93206133fb14eb62cacf/src/main/clojure/clojure/spec/gen/alpha.clj#L40#2023-04-2508:37karol.adamiecnice. šŸ‘#2023-05-2619:28djtangoI'm using metosin data specs and have a datastructure like this:
[{:type "foo" :message "foo-msg"}]
In my code I have various maps of type->msg
(def errors-a
  {:foo "foo-msg-a"
   :bar "bar-msg-a"})

(def errors-b
  {:foo "foo-msg-b"
   :bar "bar-msg-b"})

(defn keyset [m]
  (->> m keys (map name) (into #{})))

(defn ->error-spec [errors]
  [{:type (s/spec (keyset errors))
    :message string?}])
s/spec expects a macro and so s/explain-data outputs the symbol (e.g. keyset) instead of the set inlined. I've tried writing a macro to inline the set produced but have been getting errors to do with the mixing of macros and trying to evaluate values at compile time. Anyone able to help me write a macro so (s/spec (keyset errors)) inlines to (s/spec #{"foo" "bar"}) as this makes the explain-data most understandable
#2023-04-2516:08karol.adamiecI have spent some time searching for information about clojure.spec.alpha.test/check and how to wrap it up so it is used in a test run. So how can I wrap around an above (stest/check mysymbol)` in a (deftest)?#2023-04-2516:24lassemaatta(disclaimer: it's been a while since I've used spec) isn't the return value of stest/check just data? Can't you just verify it?#2023-04-2516:25karol.adamiecyes i can. i can wrap it up and dress in an itbody, and request the return data result key to be a :success. or whatever. It just seems like making a bit of a piruette...#2023-04-2516:26karol.adamiecso i lost my confidence and start questioning my reasons...#2023-04-2516:29lassemaattaThere's https://clojure.org/guides/test_check_beginner#_defspec in test.check for when you have generators providing the values and the actual function under test. but I don't think that's applicable to fspec's (or if there's some similar defsomething macro) for them.#2023-04-2516:30karol.adamiecyes. defspec seems like "half way there"#2023-04-2516:31karol.adamiecIt very well mignt be that fspec is not really a 'testing' area but more for instrumentation and repl.#2023-04-2516:37lassemaattaI'm not sure I'd agree. My understanding is that instrumentation is there to verify that other code invokes your function correctly (ie. it checks the :args part of fspec), but verifying that your function actually does what it's fspec promises (the :ret and :fn parts ) is the responsibility of s/check (= testing)#2023-04-2516:54karol.adamiec
(t/deftest mean-invariant-properties
  (t/testing "Should pass fspec exercise"
    (t/is (nil? (:check-failed (stest/summarize-results (stest/check `mean)))))))
#2023-04-2516:54karol.adamiecso this works. but it does not look 'good'#2023-04-2516:55karol.adamiecusually stuff that looks wrong, or is too much work ends up being me trying to force round pegs in square holes. hence me asking šŸ˜‚#2023-04-2517:06lassemaattaLooks ok to me :man-shrugging: I guess you could try minimizing the code if you really want to; remove the t/testing part and perhaps create some helper fn, which does the e.g. the (nil? (:check-failed (stest/summarize-results part to make it shorter.#2023-04-2517:08karol.adamiecyeah, but that is not even "correct". it also might throw and there is another keyword for checking that... It is not a big deal wrapping it in small macro and attaching it by invoke and pass of a namespace... I am just surprised and worried that is not there.#2023-04-2517:10karol.adamiecmore worried. šŸ™‚ but thanks for sharing your views on that. šŸ™‡#2023-04-2517:10karol.adamiecflying solo often has me questioning my understanding, especially for new areas šŸ™‚#2023-04-2517:21lassemaattaa very similar discussion from a few years ago: https://groups.google.com/g/clojure/c/0UjSFT926vg šŸ™‚#2023-04-2517:24lassemaattabut yeah, it seems there's no out-of-the-box integration with fdef and clojure.test. Though it seems some test libraries (kaocha) bridge the gap and try to make it easier. But in any case I wouldn't feel very worried about it. My guess is that stest/check was designed to have a strict scope and return the data as data and it's up to you to decide what to do with that data.#2023-04-2516:19karol.adamiecor the reason i can not find that is that is not supposed to be done?#2023-04-2516:19karol.adamiecOne can argue that property tests as invaraibly a bit random are not a great fit for pipelines...?#2023-04-2520:30Ben Liebermanare pre/post maps in a function my best bet for spec'ing input/output of parsing datetimes from strings? for instance I have this relatively simple conversion (-> "2023-10-27T09:00:00.000Z" Instant/parse (OffsetDateTime/ofInstant (ZoneId/systemDefault))) but unfortunately the API I'm consuming does not include the Z at the end. Is it better to have two specs, one a regex pattern for input and one a call to #(instance? OffsetDateTime %) as output?#2023-05-1118:56Mark WardleHi. Could you use an s/or spec and conform, and then do what is needed based on the result of that conform? You could provide different predicates using different regexps (or even use instance? so your fn can take in lots of different things) and then conform, and then use the result to operate on the result. You'd parse one way for one type, or another for another type. Or am I misunderstanding your question? It sounds as if this isn't necessarily a spec question but a polymorphism question in which you want to act on a variety of different possible inputs?#2023-05-1119:12Ben LiebermanI think the input will always be uniform unless the controller of this API is changing it, which is out of my hands. I think using conform could be the right path forward, but I'd have to go back and look at this code. Haven't had a chance in a while. Thanks for the suggestion.#2023-05-1119:16Mark WardleI didnā€™t spot that your question was from April sorry! #2023-05-1119:16Ben LiebermanNo problem! I appreciate the reply šŸ™:skin-tone-3:#2023-05-0513:52wcohenare there any best practices or existing examples for trying to deploy cljs.spec.test.alpha/check with a jsdom environment available (without jumping to shadow yet)#2023-05-1015:02Henrique PrangeI have created specifications for a file entity, with separate definitions for documents and images. To differentiate between the two, I used s/or to define the :file/content-type according to the media type. However, Iā€™d like to ensure that the :file/content-type matches the media type of the file being checked, depending on whether it is an image or a document. Specifically, I would like to ensure that image files are of content-type "image/jpeg" or "image/png" only, and that document files are of content-type "application/pdf" or "image/jpeg" only. Below is the specification I have created:
(s/def :file/content-type (s/or :doc #{"application/pdf" "image/jpeg"}
                                :img #{"image/jpeg" "image/png"}))
(s/def :file/height int?
(s/def :file/width int?
  
(s/def :file/doc (s/keys :req [:file/content-type]))
(s/def :file/img (s/keys :req [:file/content-type :file/height :file/width]))
Can someone please advise on how I can ensure that image files and document files have the appropriate content type?
#2023-05-1118:17Mark WardleHi Henrique... (Are you using Clojure with WO now?) You can use s/and for this kind of stuff with any predicate(s) you want. If you want, instead of using two different specs, you can combine them into a single spec and then have a multi-spec on the type: (s/def :file/content-type #{"application/pdf" "image/jpeg" "image/png"}) (s/def :file/height int?) (s/def :file/width int?) (s/def :file/type #{:img :doc}) (defmulti file-spec :file/type) (defmethod file-spec :img [_] (s/and (s/keys :req [:file/type :file/content-type :file/height :file/width]) #(#{"image/jpeg" "image/png"} (:file/content-type %)))) (defmethod file-spec :doc [_] (s/and (s/keys :req [:file/type :file/content-type]) #(#{"application/pdf" "image/jpeg"} (:file/content-type %)))) (s/def ::file (s/multi-spec file-spec :file/type)) Here I'm assuming the file data comes with :file/type and file:content-type. Alternatively, you can define two specs as you did and use whichever one you want. When I started, I thought I'd specify everything with clojure spec, but there is a long tail with diminishing returns, so I'll often not specify some things, knowing I can supplement with runtime checks and use spec a la carte.#2023-05-1118:22Mark WardleThis is what it would look like with two separate specs meaning that your code has to make the choice as to which type to check against: (s/def :file/content-type #{"application/pdf" "image/jpeg" "image/png"}) (s/def :file/height int?) (s/def :file/width int?) (s/def :file/doc (s/and (s/keys :req [:file/content-type]) #(#{"application/pdf" "image/jpeg"} (:file/content-type %)))) (s/def :file/img (s/and (s/keys :req [:file/content-type :file/height :file/width]) #(#{"image/jpeg" "image/png"} (:file/content-type %)))) (s/conform :file/doc {:file/content-type "application/pdf"}) (s/conform :file/img {:file/content-type "application/pdf"}) (s/conform :file/doc {:file/content-type "image/jpeg"}) (s/conform :file/img {:file/content-type "image/jpeg" :file/height 100 :file/width 100})#2023-05-1213:16Henrique PrangeHey Mark! Great to hear from you! Iā€™m no longer working with WebObjects, and Iā€™m now using Clojure and Datomic on my current project. Thanks for your help with the spec question. Iā€™m using spec mostly to validate that my Datomic entities are complete and consistent, and itā€™s been working really well so far. I didnā€™t know that I could use defmulti with spec, which is really helpful! Ultimately, I ended up following your second suggestion and making separate specifications for images and documents using s/and. Thanks again for your advice!#2023-05-1214:04Mark WardleNo problem at all. I still have a legacy WO application in production that I'm replacing bit by bit over time with new Clojure based components. If I squint, I see the older dynamism of the objective C WO and the key value coding / key paths form WO in Clojure and its general approach, although I sometimes miss the batteries-included approach on occasions, I definitely feel I'm not fighting the framework as much here.#2023-05-1702:11john2xI'm trying to understand if this is an okay usage for spec, and if it's possible. I have user spec like so:
(s/def ::email string?)
(s/def ::id string?)
(s/def ::user (s/keys :req [::email ::id]))
I'm going to store this user in DynamoDB, but to do that I need to transform from DynamoDB's shape to my ::user spec. So far I have record->user and user->record helper functions to transform from one to the other. It works, but I need to manually type in the mapping if I add new attributes. Can I do this with spec? e.g. create a spec for the DynamoDB record based on ::user, and have the transformation functions auto-created as well? Or is there another approach I should look into?
#2023-05-1716:39phronmophobic> have the transformation functions auto-created as well Are the transformation doing something more than removing the namespace of the keyword?#2023-05-1716:45phronmophobic> Can I do this with spec? e.g. create a spec for the DynamoDB record based on ::user, and have the transformation functions auto-created as well? Spec is agnostic to where the data comes from or is going to. It's possible to write this sort of functionality in clojure, but I might worry about artificially creating an https://en.wikipedia.org/wiki/Object%E2%80%93relational_impedance_mismatch. It's hard to recommend anything specific without knowing more about the use case. ā€¢ How do you plan to use Dynamo? ā€¢ Are interacting with existing data or do you have more control over the format used to store data? ā€¢ Are you connecting to Dynamo via https://github.com/Taoensso/faraday or some other option?#2023-05-1723:45john2x> Are the transformation doing something more than removing the namespace of the keyword? Basically this
(defn user->record
  "Transform from ::models.user/user to DynamoDB record data that can be written"
  [user]
  {:id {:S (::models.user/id user)}
   :email {:S (::models.user/email user)}})

(defn record->user
  "Transform raw DynamoDB record data to ::models.user/user spec"
  [data]
  {::models.user/email (get-in data [:email :S])
   ::models.user/id (get-in data [:id :S])})
I guess this is my OOP background leaking. But I would prefer if I'm using and passing around ::user maps everywhere, and only use the DynamoDB {:id {:S "foo"}} shape on the edges.
#2023-05-1723:47john2x> ā€¢ How do you plan to use Dynamo? > ā€¢ Are interacting with existing data or do you have more control over the format used to store data? > ā€¢ Are you connecting to Dynamo via https://github.com/Taoensso/faraday or some other option? Nothing fancy. Just intending to use it as a document store. I do have full control over the data. And I'm using com.cognitect.aws/api#2023-05-1723:53phronmophobicWhat is the :S key?#2023-05-1723:54john2xIt's just dynamodb's way of saying that the value is a string#2023-05-1723:54phronmophobicIt seems like the translation to/from could be automatic rather than manual. Is there a reason not to just iterate through all the keys in the provided map and strip/append the keyword namespace info?#2023-05-1723:59john2xyeah I think that could work. It might get tricky if I use more complex specs/shapes (e.g. nested). Fortunately I don't see this work doing anything too fancy with the schema for now#2023-05-1800:01phronmophobicFor inspiration, you can check out how https://github.com/seancorfield/next-jdbc interfaces with databases. Another higher level API that might provide some ideas is datalevin, which also provides a clojure interface over a key/value store, https://github.com/juji-io/datalevin#use-as-a-key-value-store.#2023-05-1914:13EdI've definitely written something that converts back and forth between DDB's representation and clojure's literal data. A quick google found this that you could use for inspiration: https://github.com/doo/clj-dynamodb/blob/master/src/clj_dynamodb/convert/to_dynamodb.clj (nothing to do with me - all my impl's have been closed source)#2023-05-1914:19EdBut I think it's generally a better idea to explicitly convert your data structures from DDB land to Clojure data and back with function calls, rather than rely on something like spec to coerce the data from one structure into another. I think that reduces the utility of the spec.#2023-05-1922:36john2xYeah, I'm starting with just manually writing the conversion for now. Thanks for all the suggestions#2023-05-2412:32OknoLombardahow is this possible? valid? returns false, meaning v doesn't conform to spec, but explain prints "Success!" as though it does#2023-05-2412:51Alex Miller (Clojure team)in general, cases of this are bugs#2023-05-2412:51Alex Miller (Clojure team)if you have one, please put it on https://ask.clojure.org#2023-05-2414:43OknoLombardaposted minimal example there https://ask.clojure.org/index.php/12957/inconsistent-results-from-valid-explain-when-trying-values#2023-05-2414:50Alex Miller (Clojure team)Can you add the spec? (preferably not tied to metosin stuff). Maybe you can (s/form ::sp) to see it?#2023-05-2415:04OknoLombardawill this do?
case> (s/form (ds/spec ::sp cards))
(spec-tools.core/spec
 {:spec (clojure.spec.alpha/keys :req-un [:case$sp/cards]),
  :type :map,
  :leaf? false,
  :name :case/sp})
case> (s/form :case$sp/cards)
(spec-tools.core/spec
 {:spec
  (clojure.spec.alpha/coll-of
   (spec-tools.core/spec
    {:spec clojure.core/int?,
     :type :long,
     :leaf? true,
     :name :case$sp/cards})
   :into
   []),
  :type :vector,
  :leaf? false,
  :spec-tools.core/synthetic? true})
#2023-05-2415:24OknoLombardaI was told it might be issue of spec-tools itself, so I posted it on their github repo as well https://github.com/metosin/spec-tools/issues/275#2023-06-0112:17gregHi, I noticed this behaviour that even if the keyword is not explicitly included in s/key, it is implicitly included:
(s/def :person/name string?)
(s/def :person/surname string?)
(s/def :person/email string?)
(s/def ::profile
  (s/keys :req [:person/name
                :person/surname]))

(comment
 (s/explain ::profile {:person/name "Tom"
                       :person/surname "Wilkins"
                       :person/age 29})
 ; => Success!
 
 (s/explain ::profile {:person/name "Tom"
                       :person/surname "Wilkins"
                       :person/email nil})
 ; => nil - failed: string? in: [:person/email] at: [:person/email] spec: :person/email 
In the above example :person/email is not included in ::profile but the spec is complaining about it. Can it be disabled?
#2023-06-0113:09Alex Miller (Clojure team)This is by design (see the docstring of s/keys)#2023-06-0113:20gregThank you @U064X3EF3, I read it. Can I compose a spec to validate a map in some other way so these namespaced spaces are not implicitly included?#2023-06-0113:22gregThe problem I'm facing is that I can't control the data I receive, and in the codebase I work with, there is lots of specs namespaced by category, here and there, often duplicated (another issue with overriding entries in registory). I'm trying to tackle these issues one by one, instead of making big changes in one go. One of these namespace specs conflicts with keys used in maps I try to validate.#2023-06-0113:56Alex Miller (Clojure team)In short, no, with s/keys. You can instead select-keys the data to just the keys you want to check before validating#2023-06-0114:22gregYes, unfortunately this strategy would not work with instrumenting functions šŸ˜ž#2023-06-0301:22gregOk, I managed to find a workaround to this problem. What I did was writing a wrapper to wrap Malli schema under Spec protocol. That map validation is performed by Malli despite the instrumentation is executed via Spec api. This way I managed to avoid s/keys behaviour. The other issue I had in the same repo was that these subspecs (`:person/name`, :person/surname , etc) were defined multiple times in different parts of the project resulting in overriding global registry. Thanks to that wrapper I also managed to avoid that problem.#2023-06-0518:34joshchoIs there a way to browse specs nicely in Emacs? Maybe a minibuffer that shows all the specs defined, or a way to quickly jump to specs.#2023-06-0607:58lassemaattahttps://docs.cider.mx/cider/usage/misc_features.html#browsing-the-clojure-spec-registry#2023-06-0717:48KelvinIs there a good way to dynamically create new s/or specs at runtime? As s/or is a macro, it feels almost impossible to do that, hence Iā€™d have to go down to or-spec-impl which is very much discouraged:
(defn json-schema-or-spec
  "Create a `s/or` spec from a coll-valued JSON Schema `type`"
  [schema-types]
  (let [pairs (map (fn [schema-type]
                     (case schema-type
                       "null"    [:null ::null]
                       "boolean" [:boolean ::boolean]
                       "integer" [:integer ::integer]
                       "number"  [:number (type->spec "number")]
                       "string"  [:string (type->spec "string")]
                       "array"   [:array (type->spec "array")]
                       "object"  [:object (type->spec "object")]))
                   schema-types)
        keys  (mapv first pairs)
        preds (mapv second pairs)]
    (s/or-spec-impl keys preds preds nil))
(or alternatively end up creating a case statement with 128 different cases)
#2023-06-0717:54Alex Miller (Clojure team)you can write that case with a macro around s/or :)#2023-06-0717:55Alex Miller (Clojure team)but maybe a different kind of spec is better, like s/multi-spec?#2023-06-0717:56KelvinWriting a macro around s/or was my first attempt:
(defn- schema-type->pair
  [type->spec schema-type]
  (case schema-type
    "null"    `[:null ::null]
    "boolean" `[:boolean ::boolean]
    "integer" `[:integer ::integer]
    "number"  `[:number (~type->spec "number")]
    "string"  `[:string (~type->spec "string")]
    "array"   `[:array (~type->spec "array")]
    "object"  `[:object (~type->spec "object")]))

(defmacro coll-schema-spec
  [type->spec schema-types]
  (let [schema-type->pair (partial schema-type->pair type->spec)]
    `(s/or ~@(mapcat schema-type->pair schema-types))))
#2023-06-0717:57KelvinUnfortunately it did not go very well: I ended up encountering a lot of
Don't know how to create ISeq from: clojure.lang.Symbol
errors because I was passing in schema-types as a variable
#2023-06-0718:00Alex Miller (Clojure team)well I think you're on the right track there, but you don't want ~@ - this is one of those cases that might be easier as literal construction with list cons etc#2023-06-0718:01Alex Miller (Clojure team)~ turns off quoting and turns eval back on, but you don't want to eval#2023-06-0718:02Alex Miller (Clojure team)also, schema-type->pair is just a map#2023-06-0718:13KelvinStill getting the same error; I think the fundamental issue is that schema-types is a coll whose values are only known as runtime, so trying to fit it inside any compile-time macro like s/or will only result in failure.#2023-06-0718:13Kelvin(And itā€™s these times when I wished we used malli in our projects)#2023-06-0718:19Alex Miller (Clojure team)can you just register these schemas at runtime then?#2023-06-0718:19Alex Miller (Clojure team)think about it as dynamically making static specs, not statically making dynamic specs#2023-06-0718:20KelvinSo you actually get that already with the ::null, ::boolean, and ::integer specs#2023-06-0718:21KelvinThe issue comes with those other specs whose implementations depend on the JSON Schema user input (the type->spec function elides a lot of complexity)#2023-06-0718:22KelvinFor example with array and object youā€™re basically creating s/coll-of and s/keys specs from user input - not really conducive for registering static specs#2023-06-0718:29Alex Miller (Clojure team)not conducive at compile time, but totally fine at runtime#2023-06-0718:37Ben SlessYou can dispatch on the count of schema-types to unrolled calls to s/or#2023-06-0718:38KelvinI tried doing that, but that didnā€™t work either#2023-06-0718:38KelvinSince s/or requires keyword literals as keys, not variables representing those keywords#2023-06-0718:39Ben SlessOh right#2023-06-0718:39Ben SlessWell you can use eval#2023-06-0718:40KelvinI used eval to pass in my schema-types variable to the macro, but that gave a ā€œcanā€™t eval localsā€ error#2023-06-0718:45KelvinAnyways, going back to @U064X3EF3ā€™s point, Iā€™m not sure what he meant by ā€œregister these schemas at runtimeā€ since when I think of registering specs, I automatically think of s/def#2023-06-0718:45KelvinI guess you can also generate the spec keywords at runtime, but given the potentially infinite schema values that does not seem to be a good idea#2023-06-0718:50Alex Miller (Clojure team)ultimately, the spec registry is a map in your Clojure runtime. you can put things in it anytime, not just when you load a namespace that calls s/def#2023-06-0718:50Alex Miller (Clojure team)(all of this is more readily available in the spec 2 api, both making specs and registering them without s/def)#2023-06-0718:51KelvinAnd spec 2 is supposed to have more support for dynamic, data-driven specs like the varardic s/or Iā€™m trying to make, right?#2023-06-0719:02KelvinAnyways I think Iā€™m going to go with or-spec-impl as my near-term solution#2023-06-0719:03KelvinIt works like a charm just like any other spec, and thereā€™s not really any downside other than a strongly worded docstring#2023-06-0720:44Alex Miller (Clojure team)spec 2 has a data form you can use to make a spec, and a non-macro way to register a spec#2023-06-1113:14Sturmis it possible to use spec to validate maps where keys are strings? eg. to check {"name" "Joe" "age" 66} to make sure "name" and "age" are present and that they match string? and int? respectively. I might be missing something obvious, but I can only see examples with keyword-based maps.#2023-06-1113:18Alex Miller (Clojure team)You can use map-of but not s/keys#2023-06-1113:20Sturmthanks @U064X3EF3, I'll have a play with map-of#2023-06-2221:01Chris LesterI have a strange error (because I can't find any references to this) on having a spec from another namespace that is only required and used in a (comment ...) block causing a compile failure. Is that a special case similar to unknown reader tags in the comment block that will cause it to fail since it attempts to parse that anyway?#2023-06-2221:06seancorfieldYou mean this reader error:
> clj
Clojure 1.11.1
user=> (comment
    (require '[foo.bar :as quux])
    ::quux/wibble
Syntax error reading source at (REPL:4:0).
Invalid token: ::quux/wibble
user=>
#2023-06-2221:07seancorfieldThat's because the contents of the comment must be syntactically valid Clojure tokens, and because the require is runtime and not evaluated, the auto-resolving keyword prefix ::quux cannot be expanded.#2023-06-2221:10seancorfieldYou could either add the require to your ns (even tho' it wouldn't be used elsewhere -- you could use :as-alias instead of :as if you don't want it to be actually loaded assuming you're on a recent enough version of Clojure) or you could use the fully-qualified Spec name instead of using the alias.#2023-06-2221:25Chris LesterThanks Sean, makes sense, I added a partner dev who cleaned up that namespace since it wasnā€™t used (coming from a typed language and the non use of that ns bugged him).#2023-07-0401:46Joerg SchmueckerIs it possible that spec and test.check do have some incompatibilities? I am getting cannot cast Generator to fn exceptions when using specs in the tests.#2023-07-0401:54hiredmanSpec generators are test.check generators wrapped in a 0 art function#2023-07-0401:54hiredmanArg#2023-07-0408:12Joerg SchmueckerSo how do I use a spec generator in test.check? Unwrap them or use the ā€œsurfaceā€ provided by ā€œspec.alphaā€.#2023-07-0408:13Joerg SchmueckerMore importantly, how would i find out about htat? Without having to bother a human.#2023-07-0520:14MegaMattWondering why this code produces the prn statements that it does.
(let [shape {:path1 true :path2 {:nested true}}
        path-exists? (fn [p o]
                       (let [p (map keyword (str/split p #"\."))
                             missing? (= :missing (get-in o p :missing))]
                         (prn o)
                         #_(not missing?)
                         true))]
    (s/def ::something (s/and (s/or :path1-exists (partial path-exists? "path1"))
                              (s/or :path2-exists (partial path-exists? "path2.nested"))
                              (s/or :path3-exists (partial path-exists? "path2.nested2"))))
    (s/valid? ::something {:path1 :here :path2 {:nested :here :nested2 :blah}}))
Res:
{:path1 :here, :path2 {:nested :here, :nested2 :blah}}
[:path1-exists {:path1 :here, :path2 {:nested :here, :nested2 :blah}}]
[:path2-exists [:path1-exists {:path1 :here, :path2 {:nested :here, :nested2 :blah}}]]
Iā€™ll post the why iā€™m doing this in a thread in case others can suggest a better approach.
#2023-07-0520:16MegaMattIā€™m making a macro which takes a list of dot delimited strings and makes a spec which ensures the shape of the data eg. (defshape ["a.b.c", "a.d"]) would ensure you have something like
{:a {:b {:c "test"}} :d true}
#2023-07-0520:17MegaMattand sure, just s/and would work but the errors are not helpful to the developer thus i wanted s/or so that i can have a keyword that indicates the problem.#2023-07-0520:25seancorfields/and is conforming, passing the conformed value into the second and subsequent specs.#2023-07-0520:25seancorfieldI think if you wrap each s/or in s/nonconforming that will produce what you want.#2023-07-0520:26MegaMattthank you.#2023-07-0520:46MegaMattUpdate:point_up:: s/nonconforming solved my issue.#2023-07-1207:39Joerg SchmueckerHow would I spec an higher order function such as map. The function parameter passed in should validate against
(s/def ::f (s/fspec :args (s/cat any?) :ret any?))
but if I try to validate (valid? ::f (fn [i] (* i 2))) then this fails because it will generate non numerical values according to the above spec.
#2023-07-1213:01colinkahnYou can replace your fspec with fn? or ifn?#2023-07-1213:48Joerg Schmueckeryes, I can but then I donā€™t check that the signature/shape matches. The real signature is something like this:
(s/fspec :args (s/cat :state any?) :res (s/cat :state any :result any?))
I always want to get a vector with 2 or more elements back.
#2023-07-1222:49Joerg SchmueckerI know, itā€™s much more than a type system! This is the first thing I did find that would be easier with a static type system. Thatā€™s why I mentioned it.#2023-07-2823:11johanatanI would put a runtime guard in the body of the function. #2023-07-2823:11johanatanReturn nil when wrong input #2023-07-2823:12johanatanPretty sure this is how I handled this in the past #2023-07-2605:18Joerg SchmueckerSorry, another question that I couldnā€™t find a good answer for via G. How do I spec a nested structure? Here is the code snippet that I tried.
(s/def ::my-spec (s/cat :a-b (s/cat :s string? :i int?) :s2 string?))

(s/explain-data ::my-spec [["test" 1] "world"])
;; => #:clojure.spec.alpha{:problems [{:path [:a-b :s], :pred clojure.core/string?, :val ["test" 1], :via [:ct-test/my-spec :ct-test/my-spec], :in [0]}], :spec :ct-test/my-spec, :value [["test" 1] "world"]}
I guess I could use a tuple in the spec and that would resolve it but thatā€™s more limiting that s/cat. Different question? Is there any good way to put these answers into Google?
#2023-07-2605:55phronmophobicI think you can get the nesting you want by wrapping the inner s/cat with s/spec. https://clojurians.slack.com/archives/C1B1BB2Q3/p1664745079479859?thread_ts=1664739041.707149&amp;cid=C1B1BB2Q3#2023-07-2605:57phronmophobic
(require '[clojure.spec.alpha :as s])

(s/def ::my-spec (s/cat :a-b (s/spec (s/cat :s string? :i int?)) :s2 string?))

(s/valid? ::my-spec [["test" 1] "world"]) ;; true
#2023-07-2707:22Joerg Schmueckergotcha thanks!#2023-08-1506:10Lidor CohenHello everyone šŸ‘‹ I made this function to generate maps with a given depth and width:
(s/def ::any (s/or :s string? :i int?))

(defn map-gen [x y]
  (->> (repeatedly x  #(-> keyword?
                           s/gen
                           g/generate))
       (map (fn [k] [k (if (pos? y)
                         (map-gen x (dec y))
                         (-> ::any
                             s/gen
                             g/generate))]))
       (into {})))
but for some reason it is very slow, what am I misusing here?
#2023-08-1608:14pithylessYour recursion is generating an exponentially increasing large map, so the time is also going to go up. For example: (map-gen 10 y) where y is 1 to 5 goes up in time:
y=1 "Elapsed time: 14.155375 msecs"
y=2 "Elapsed time: 50.327042 msecs"
y=3 "Elapsed time: 748.101916 msecs"
y=4 "Elapsed time: 2843.250333 msecs"
y=5 "Elapsed time: 28140.05775 msecs"
#2023-08-1608:15pithylessSo, it would be helpful if you specify how big are the maps you're generating, what kind of time you're seeing, and what kind of time you'd expect (since "slow" and "fast" are relative).#2023-09-0214:20kokonutHi, I just realized that spec's instrumentation doesn't really check :ret and :fn field. I also read some discussions that say we should be able to use test/check. But is there any way to enable those fields on repl in development phase (not unit testing)?#2023-09-0214:53nikolavojicichttps://github.com/jeaye/orchestra#2023-09-0215:42gnlAlso https://github.com/fulcrologic/guardrails#2023-09-0219:41kokonutThanks.#2023-09-0619:06mgSo it seems like maybe spec2 is dead in the water? Haven't seen any announcements related to this in years and the repo hasn't had a major change since 2021...#2023-09-0619:12seancorfieldNot dead. Stalled. Last I heard, Rich was still thinking about how to integrate specs into function definitions in a way he liked. Alex has said a couple of times that spec2 will likely become core spec (non-alpha) at some point.#2023-09-0620:55gnlGiven his recent announcement and departure from Nubank, might there be reason to cautiously hope...? šŸ‘€#2023-09-0620:58gnlP.S. Perhaps you could nudge him to take a look at the Guardrails/Ghostwheel syntax as a particularly succinct alternative ā€“ having that in core would be pretty fantastic. Not that I consider this development very likely, but you don't ask, you don't get. šŸ™‚#2023-09-0619:15mgAh, thanks for the update. Maybe we should chip in for a nicer hammock#2023-09-0710:41ikitommiRe: function definitions. I wish Clojure would get a spec how to define inline types in the core + optional tooling for enforcement (spec, schema, malli). In js-land: https://tc39.es/proposal-type-annotations/#2023-09-0912:30Jacksson Enrique Mosquera Rivas#2023-09-0916:18seancorfieldIt looked like you got answers to this question in #C053AK3F9 - are you still stuck?#2023-09-2109:03vemv(One of those "just curious" questions šŸ™‚) Is there an intrinsic reason why spec (1) is slower than Malli? It's relatively common to hear that it's not performant enough to be used in something like a web handler. However it doesn't sound too crazy to implement some sort of drop-in replacement for a few key functions like conform. One would still use the regular spec/def for definitions - the custom checker could simply read the original registry.#2023-09-2110:43nikolavojicicNever had a significantly slow spec such that it matters for web app... For data generation yes, it can be slow and can be fixed by tweaking generators. Is there an example?#2023-09-2111:38vemv'That matters' is the tricky part... for many webapps a 1ms delay doesn't appear to matter, but if you are trying to serve thousands of requests per second, it becomes the bottleneck#2023-09-2115:12seancorfieldWe use conform heavily in our main REST API at work and it's "fast enough". I guess if you have an app where Spec's performance matters and Malli satisfies while Spec does not, then use Malli? We've never seen Spec as a bottleneck. JDBC stuff, yes. Elastic Search stuff, yes. Spec, never.#2023-09-2115:21vemvI once participated in a project where we used, roughly speaking, spec for code (fdefs and the like), and malli for data. Wasn't bad, I might even enjoy exercising all those skills on a good day, but it doesn't seem ideal. I'm a spec1 fan and would rather use it everywhere... but its performance is theoretically inferior. Again, you can say the same about e.g. Rails or Python web servers. In reality CPU is rarely the bottleneck, as you hint... but some of us enjoy squeezing the last drop of performance. Knowing it can do fast coercion, validation, etc can help in the occasional team-wide decision making processes.#2023-09-2115:24vemvReflecting a bit, a later project used Malli in a more advanced way. I had the impression that you could go as far as building "your own spec1" but in Malli terms. Would seem a clearer path than hacking on spec1, as it is a stalled project.#2023-09-2322:33jasonjcknw.r.t conform vs decode/encode in malli a lot more optimization work is done at 'compile time' (in the sense of constructing a chain of lambda functions to arrive at a a parser) e.g. in malli , it's possible your decoder/conform compiles to an identity function#2023-09-2322:34jasonjckn#2023-09-2322:36jasonjcknThis is partly due to different designs, because the construction of the decoder/parser takes into account the structure of the input, if you know you're only trying to create morphisms between JSON <-> EDN , you can optimize more#2023-09-2322:38jasonjcknPersonally, i'm a huge fan of malli, i think its closer to rich hickey's definition of simplicity than spec version 1, since specs/schema are just data, ... this could change with spec version 2, but i'm a convert for now.#2023-10-1716:44Ho0manHi everyone, I wanna spec sth like this :
[[:__      :TXC    :BTC    :ALT    :PAXG   ]
 [:TT1     0M      0M      0M       0M     ]
 [:TT2     0M      0M      0M       0M     ]
 [:TT3     0M      0M      0M       0M     ]
 [:TT4     0M      0M      0M       0M     ]
 [:Others  0M      0M      0M       0M     ]
 ]
but the following spec won't work :
(spec/def ::X-Header-Row     (spec/cat :dont-care #{:__} :exposure-id-s (spec/+ ::Exposure-ID)))
(spec/def ::X-Row            (spec/cat :g-id ::Group-ID :weights (spec/+ decimal?)))
(spec/def ::X                (spec/cat :header ::X-Header-Row  :rows (spec/+ ::X-Row)))
What is it that I'm not getting about how spec/cat works and how can I spec that table ? Thanks a lot in advance
#2023-10-1716:55Alex Miller (Clojure team)won't work .... because?#2023-10-1717:39Alex Miller (Clojure team)regex ops combine to spec one collection, so you need to do something to separate the outer collection and the inner collection "levels"#2023-10-1717:40Alex Miller (Clojure team)probably the shortest path to that is to wrap s/spec around the ::X-Row cat#2023-10-1811:51Ho0manThanks a lot, Alex#2023-11-0803:16cflemingI have a case where conform returns :clojure.spec.alpha/invalid, but explain returns ā€œSuccess!ā€œ. Am I missing something?
(def test-data (rest '(defn F "s||C_)C^?" {} ([[& {:keys [], :strs [a2p*1.+lNz3+9.-fc? !2_!T2*8_.aw7.H14*7 B_-T*E5+3lKy-gE Ca++-F*_G58!7+? VM.3-E**S*R+09eW f-F?J-D_?r*xdjb.D]} :as kQ.zi] & yeok5+8+3_O_.mXJZ] "MyN-Rmu[Y" nil :Wd--IU9M*! ":*ULe,h" :p-:?6+__S- J91*) "EcO")))
=> #'cursive.extensions.specs/test-data
(s/conform :clojure.core.specs.alpha/defn-args test-data)
=> :clojure.spec.alpha/invalid
(s/explain :clojure.core.specs.alpha/defn-args test-data)
Success!
#2023-11-0803:22hiredmanhttps://ask.clojure.org/index.php/13390/surprising-behaviour similar recent ask#2023-11-0803:23cflemingYeah that looks similar.#2023-11-0803:26cflemingIā€™ve also found that generating code samples using the core specs doesnā€™t work well - it does work sometimes but not others, and Iā€™m not sure why:
(gen/sample (s/gen ::core/bindings))
Error printing return value (ExceptionInfo) at clojure.test.check.generators/fn (generators.cljc:435).
Couldn't satisfy such-that predicate after 100 tries.
#2023-11-0803:49cflemingIā€™ve commented on the ask with some cases. Looks to me like the cases should be valid, so perhaps the bug is in conform.#2023-11-0803:58hiredmanThe way s/and composes for generators is unlikely to get you a working generator for more complex specs, basically it takes the generator first spec, and then uses the rest of the specs in the and as a predicate to filter out generated values that don't match the entire s/and#2023-11-0803:59hiredmanIt is limited to 100 tries#2023-11-0803:59cflemingHmm, I see. Thanks for the explanation.#2023-11-0804:00hiredmanI think there might be something you can bind to increase such-that's limit, but in general if you hit it you usually need to write a custom generator instead of relying on s/and's composed one#2023-11-0804:01cflemingIs there a good way to figure out which spec is problematic? And can I override generators for the built-in specs?#2023-11-0804:01cflemingThe stacktrace didnā€™t seem especially useful.#2023-11-0804:03hiredmanspec has a way to define specs with custom generators, which is helpful for your own specs, for existing specs I sort of recall some mechanism for overriding but I am not sure#2023-11-0804:03hiredmanas far as figuring out which spec, I don't know#2023-11-0804:03cflemingOk, thanks. Iā€™m currently generating these forms with my own code, and then just checking the result against the spec.#2023-11-1600:03cflemingIā€™m trying to spec a map which I thought was homogeneous (i.e. key type matching a spec -> value type matching a spec), so I used map-of. Now it turns out that these maps can occasionally have other entries where the key is a constant keyword and then the value is something different to the other value type. I canā€™t figure out the best way to spec this - any advice?#2023-11-1601:03Alex Miller (Clojure team)it's a bit cumbersome but you can spec the map as a collection of tuple (entry) types#2023-11-1601:04Alex Miller (Clojure team)https://www.cognitect.com/blog/2017/1/3/spec-destructuring is a complicated example of this#2023-11-1601:06cflemingI see, so Iā€™d define two s/tuple types, and then a coll-of :kind map? using s/or over the two types?#2023-11-1601:06Alex Miller (Clojure team)
(s/def ::map-bindings
  (s/every (s/or :mb ::map-binding
                 :nsk ::ns-keys
                 :msb (s/tuple #{:as :or :keys :syms :strs} any?)) :into {}))

(s/def ::map-special-binding
  (s/keys :opt-un [::as ::or ::keys ::syms ::strs]))

(s/def ::map-binding-form (s/merge ::map-bindings ::map-special-binding))
is kind of the crux of that - an s/merge of special keys via s/every with an s/keys
#2023-11-1601:12cflemingOk, thanks for the pointer, I think Iā€™ll need some time to digest that but I can figure it out from there.#2023-11-1601:14Alex Miller (Clojure team)that example is probably more complicated than what you need but that's the main idea#2023-11-1601:15cflemingCan I s/merge an s/map-of with an s/keys?#2023-11-1601:15Alex Miller (Clojure team)not successfully unless the map-of uses keyword keys#2023-11-1601:15cflemingIt doesnā€™t, unfortunately.#2023-11-1601:16Alex Miller (Clojure team)it's merging the requirements of the two specs so any data has to satisfy both#2023-11-1601:17Alex Miller (Clojure team)note that the tuple side (s/every of s/tuple) has an :msb branch that is effectively matching all of the known kws with any?#2023-11-1601:17Alex Miller (Clojure team)and then the s/keys side is s/opt so it's just ignoring anything else#2023-11-1601:23cflemingSo when I do an s/merge, every entry has to match both sides of the merge? Iā€™m not sure I understood that.#2023-11-1601:30Alex Miller (Clojure team)yes#2023-11-1601:30Alex Miller (Clojure team)you are merging the requirements of the spec#2023-11-1601:32Alex Miller (Clojure team)so, it's kind of like s/and, but sub specs are effectively checked in parallel whereas s/and flows data through and checks sub specs serially#2023-11-1601:33Alex Miller (Clojure team)this also has implications for s/conform but you may not care about that#2023-11-1601:33cflemingHereā€™s my little test case:
(s/def ::normal-key string?)
  (s/def ::normal-val string?)
  (s/def ::constant (s/map-of ::normal-key ::normal-val))
  (s/def ::foo (s/coll-of string? :kind vector?))
  (s/def ::special-case (s/keys :req-un [::foo]))

  (s/conform ::constant {"foo" "bar"})
  (s/conform ::special-case {:foo ["test"]})

  (s/def ::mixed (s/every (s/or :constant (s/tuple ::normal-key ::normal-val)
                                :outlier (s/tuple #{:foo} ::foo))))

  (s/def ::end-result (s/merge ::mixed ::special-case))
  
  (s/conform ::end-result {"foo" "bar"})
  (s/conform ::end-result {:foo ["test"]}))
#2023-11-1601:36cflemingSo I started with a map of string to string using ::constant. Then I realised I need ::special-case too. However my ::end-result canā€™t conform {"foo" "bar"}#2023-11-1601:37cfleming(s/explain ::end-result {ā€œfooā€ ā€œbarā€}) {ā€œfooā€ ā€œbarā€} - failed: (contains? % :foo) spec: :cursive.extensions.specs/special-case#2023-11-1601:40Alex Miller (Clojure team)req-un has to be opt-un there#2023-11-1601:40Alex Miller (Clojure team)as string/string entries won't match that#2023-11-1601:41cflemingAh, of course - that works, thanks!